Frameworks and tools for statistical big data in the humanities

在过去40年左右的时间里，大多数发达国家的数据档案馆都收集了大量的统计数据，包括由学术项目计算机化的历史数据。然而，这些几乎都是DATASETS的集合，基本上都是以表的形式组织起来的，数据归档目录只将每个数据集作为一个整体来描述，而行和列的详细结构通常只在原始创建者的高度可变的文档中描述。这使得通过下载整个英国数据档案的内容来分析整个英国的历史是不可能的；档案搜索界面是为想要下载数据集进行详细统计分析的社会科学家设计的，而不是为人文主义者或普通公众设计的，他们想要回答诸如“1841年巴斯有多少家庭佣人？”等具体问题。即使你找到了你想要的数字，你如何确切地引用它？两个大型统计数据库项目采用了不同的方法。首先，英国历史（GBH）地理信息系统，通过英国愿景网站访问，拥有14,099,469个数据值，涵盖了过去200年来英国人口、经济、社会和政治历史的大多数方面，每一个都在单独的一行，只有一个表，每个都有日期、地理覆盖区域和主题的信息。区域和主题通过与正式本体（层次列表）的联系进行记录，前者与报告单元的计算机化边界相连，因此数据既可以绘制为时间序列，也可以绘制为地图。其次，美国国家科学基金会资助了由匹兹堡大学领导的历史信息与分析合作项目（CHIA），该项目将从2013年1月至少持续到2015年12月，通过创建一个在线基础设施，使世界各地的贡献者能够将他们的数据插入到以GBH GIS为模型的信息架构中，来收集自1600年以来全世界的历史统计数据。这种使用专家的“众包”是创建真正大型历史数据结构的唯一途径。拟议的研究将直接资助GBH地理信息系统的设计师在中国文物影响评估中发挥更大的作用，并招募英国和欧洲的捐助者。然而，这里的主要目的不是使GBH地理信息系统或文物影响评估系统更大，而是使人文主义者和公众更容易找到和使用这些内容。这是一个相对较小的项目，有三个主要产出。首先，一个更好的统计词库或主题本体——如果您愿意，也可以是数字的杜威十进分类法——为贡献者和用户提供一种共享语言。这将以社会科学家开发的现有词典为基础，但增加了一些更广泛的标题，以帮助非专业人士入门，并增加了更多的变体术语，使用户能够找到“工作”和“职业”。其次，用户友好的界面不是用于查找“数据集”，而是用于查找特定的数字：通过点击地图指定所覆盖的区域，通过图形时间轴指定时间段，并通过在“领域本体”中逐步选择较窄的术语来确定主题。在可能的情况下，它将允许用户从简单的总数深入到更详细的数据。第三，为计算机而不是人提供单独的接口，提供单独或作为长流的数据值，每个值都包含有关日期、区域和主题的完整信息。一个目的是为每个数据值提供自己的持久网址（“统一资源标识符”），使其能够被绝对精确和自信地引用。然而，这种“数据馈送”可以被搜索引擎，如谷歌或必应，用来索引我们的内容，并提供新的访问形式；或者它可以连接到大型分析引擎，使其他研究人员能够真正分析英国或整个世界的历史。”

Over the last forty or so years, data archives in most advanced countries have assembled very large collections of statistics, including historical data computerised by academic projects. However, these are almost all collections of DATASETS, mostly organised essentially as tables, and data archive catalogues describe each dataset only as a whole, while the detailed structure of the rows and columns is usually described only in highly variable documentation from the original creators. This makes it quite impossible to analyse the history of the UK as a whole by downloading the contents of the UK Data Archive as a whole; and archive search interfaces are designed for social scientists wanting to download datasets for detailed statistical analysis, not for humanists -- or the general public -- wanting answers to specific questions such as "how many domestic servants were there in Bath in 1841?". Even once you find the number you want, how exactly do you cite it?Two big statistical database projects take a different approach. Firstly, the Great Britain Historical (GBH) GIS, accessed via the Vision of Britain web site, holds 14,099,469 DATA VALUES, covering most aspects of Britain's demographic, economic, social and political history over the last 200 years, each on a separate row of just one table, and each with information on date, geographical area covered and subject matter. Area and subject are recorded through linkage to formal ontologies -- hierarchical lists -- the former linking to computerised boundaries for the reporting units, so data can be both graphed as time series and mapped. Secondly, the US National Science Foundation have funded the Collaborative for Historical Information and Analysis (CHIA), led by the University of Pittsburgh and running from January 2013 until at least December 2015, to assemble historical statistics for the whole world since 1600 by creating an online infrastructure enabling contributors worldwide to insert their data into an information architecture modelled on the GBH GIS. Such "crowd-sourcing" using specialists is the only way really big historical data structures can be created.The proposed research would directly fund the architect of the GBH GIS to play a larger role within CHIA, and recruit British and European contributors. However, the main aim here is not to make either the GBH GIS or the CHIA system even bigger, but to make it easier for humanists and the general public to find and work with the contents. This is a relatively small project with three main outputs. Firstly, a better statistical thesaurus or topic ontology -- if you like, a Dewey Decimal Classification for numbers -- providing a shared language for contributors and users. This will be based on an existing thesaurus developed by social scientists but with some broader headings added to help non-specialists get started, and many more variant terms added, enabling users to find out about "jobs" as well as "occupations". Secondly, a user-friendly interface for finding not "datasets" but particular numbers: specifying the area covered by clicking on a map, the period via a graphical timeline, and the topic by selecting progressively narrower terms within the "domain ontology". Where possible, it will let users drill down from simple totals to more detailed data. Thirdly, a separate interface for computers, not people, providing data values either individually or as a long stream, each with complete information about the date, area and topic. One purpose is simply to provide each data value with its own enduring web address (a "Uniform Resource Identifier"), enabling it to be cited with absolute precision and confidence. However, this "data feed" could be used by search engines, like Google or Bing, to index our content and provide new forms of access; or it could be linked to large scale analytic engines, enabling other researchers to actually analyse the history of Britain or the world as a whole".