The Applied Semiotics of Visual Modelling

在数字时代，每个社会成员都需要处理、审问和理解复杂的数据。此外，我们中越来越多的人参与或依赖于信息必须绝对准确的专业领域，例如软件的设计和创建。图形形式的交流被广泛认为是交流某些类型信息的有效方式，无论是通过简单的数据可视化，如报纸文章附带的条形图，还是通过指定计算机系统的复杂图表。因此，了解这些图形形式是如何工作的有很多固有的好处，以及是什么使它们在某些任务中或多或少地有效(相对于书面文字)。符号学是将这些图形符号分析为如此多的符号排列的视角。符号学领域始于查尔斯·皮尔斯的工作，他对符号的类型、它们的组成部分、组成部分之间的关系等进行了深入的分析。尽管在同一问题上有几个初步的方法和许多不同的观点，但还没有人将符号学方法深入应用到图解语言的分析中，以预测它们的认知潜力；这就是本项目着手做的。我们将从皮尔士的角度分析几个广泛使用的数据可视化，包括那些被交互使用的数据可视化。也就是说，我们将使用皮尔斯对手势类型的分类以及它们与不同类型信息的相关性来预测可视化对特定任务的潜在有效性和适用性。我们将把这些预测与认知科学和人机界面的发现联系起来，从不同的角度研究这种可视化。这项评价将为现有技术的使用提供指导方针，并为数据评估的设计制定通用标准。我们项目的下一步是使用上面解释的参照系来分析概念图的逻辑。概念图是为知识表征而设计的，已被发现在某些任务中比符号替代更有效。图解逻辑不仅仅是数据可视化；它是一种精确定义的图形符号(例如，凡纳图或概念图)，以及一组用于以逻辑上有效的方式转换和可能组合图的规则。尽管它们不是灵丹妙药，但当用图表来代替“传统的”符号选择时，就会产生某些好处：在某些情况下，图表有效地揭示了比其构建所需的更多的信息。这是图解交流能力的一个关键来源，在图解交流中，信息是“通过结构”揭示的，如果信息是用符号语言呈现的，那么需要通过某种推理链来推断这些信息。这种发生的方式是皮尔士符号学的核心部分，也是符号学以外的学科，如人机交互和认知科学的极大兴趣。我们的贡献之一将是我们试图协调和找到这些不同的描述之间的一致或不一致之处，这些不同的描述是图表和数据可视化的“自然”或“直观”的力量。通过这种方式，我们的目标是使人们能够更好地利用现有的符号，并为未来设计更有效的符号。

In the digital age, every member of society needs to process, interrogate and understandcomplex data. In addition, more and more of us take part in or depend on specialised fields in whichinformation has to be absolutely precise, such as the design and creation ofsoftware. Graphical forms of communication are widely recognised as an effective way ofcommunicating certain types of information, whether through a simple data visualisation such asa bar chart accompanying a newspaper article, or through a complex diagram which specifies acomputer system. Thus, there are many benefits inherent to understanding how these graphicalforms work, and what makes them more or less effective (relative to, say, the written word) forcertain tasks.Semiotics is the perspective that analyses these graphical notations as so many arrangements ofsigns. The field of semiotics began with the work of Charles Peirce, who provided a richlythorough analysis of the types of signs, their components, the relations of the components toone another and so on. Despite several initial approaches and numerous differing perspectiveson the same problem, no one has yet applied the semiotic method in depth to the analysis ofdiagrammatic languages to make predictions of their cognitive potentials; that is what thisproject sets out to do.We will begin by analysing several widely used data visualisations, including those which areused interactively, from a Peircean perspective. That is, we will use Peirce's classificationof types of sign and their correlation with different types of information to make predictionsabout the potential effectiveness and suitability of the visualisations for particulartasks. We will correlate these predictions with findings from cognitive science and HCI, wheresuch visualisations are studied from different points of view. This evaluation will result inguidelines for the use of existing techniques and generalised criteria for the design of datavisualisations. The next step in our project is to use the frame of reference explained above to analyse thelogic of concept diagrams. Concept diagrams were designed for knowledge representation and havebeen found to be more effective than symbolic alternatives for some tasks. A diagrammatic logicis more than data visualisation; it is a precisely defined graphical notation (e.g. Venndiagrams or concept diagrams), together with a set of rules for transforming and possiblycombining diagrams in logically valid ways. Although they are not a panacea, when diagrammaticlogics are used in place of "traditional" symbolic alternatives, certain advantages can arise:in some circumstances a diagram effectively reveals more information than was required for itsconstruction. This is a key source of the power of diagrammatic communication, wherebyinformation is revealed "by construction" that would need to be inferred as a result of somechain of reasoning if the information were presented in a symbolic language. The means by whichthis takes place are a central part of Peircean semiotics, and are also of great interest todisciplines outside semiotics, such as human-computer interaction and cognitive science. One ofour contributions will be our attempt to reconcile and find agreements or inconsistencies amongthese different accounts of the "natural" or "intuitive" power of diagrams and datavisualisation. In this way, we aim to enable people to make better use of existing notationsand design more effective notations for the future.