Recognizing cognitively demanding situations in design and measuring them for the semi-automated analysis of empirical studies in method development: AutoCodIng

为了能够开发出合适的设计方法，我们使用了实证研究来识别产品开发中的困难。对于这些研究的评价，使用了数据收集方法，这些方法在数据准备过程中通常需要高度的解释。一方面，由于所使用的方法，这种类型的数据收集强烈地干预了设计人员的工作流程，另一方面，只提供了关于复杂评估后出现的困难的信息。此外，由于评估的复杂性，调查的区间非常有限。这一研究项目的目的是调查困难的可持续测量的指标，并将其转化为算法。该算法旨在自动化功能分析中的设计困难，从而为大量参与者客观地识别它们。在进行功能分析的设计任务加工过程中，受试者的眼动注视数据作为数据基础。使用凝视数据而不是定性数据获取(例如观察方案、记录)旨在增加评估过程中的客观性。对于半自动记录，需要确定凝视数据中的困难指标。基于眼动跟踪数据，建立了三种不同的评价方法，并对它们进行了比较。为此目的(1.)个人眼球跟踪指标，(2.)基于机器学习和(3)的眼睛跟踪度量的组合。开发了特定于上下文的度量，并在算法中进行了操作。将开发的算法的准确性与参与者回顾指出的困难进行了比较。利用所开发的算法，应该可以对大量参与者的功能分析过程进行客观检查。因此，可以确定方法支持需求，并可以在方法研究的框架内提出或开发个别培训内容。

In order to be able to develop suitable design methods, empirical studies are used to identify difficulties in product development. For the evaluation of these studies, data collection methods are used which usually require a high degree of interpretation in the data preparation. On the one hand, this type of data collection intervenes strongly in the workflows of designers due to the methods used and, on the other hand, only provides information on difficulties that arise after a complex evaluation. Besides, the investigated intervals are strongly limited due to the complexity of the evaluation. The aim of this research project is to investigate continuously measurable indicators for difficulties and to convert them into an algorithm. The algorithm is intended to automate design difficulties in functional analysis and thus objectively identify them for large numbers of participants. Eye tracking gaze data of participants during the processing of design tasks for functional analysis serve as a data basis. The use of gaze data instead of qualitative data acquisition (e.g. observation protocols, transcripts) is intended to increase the objectivity during evaluation. For the semi-automated recording, it is necessary to determine indicators for difficulties in the gaze data. On the basis of the eye-tracking data, three different evaluation approaches are built up and compared with each other. For this purpose (1.) individual eye tracking metrics, (2.) a combination of eye tracking metrics based on machine learning and (3.) context-specific metrics are developed and operationalized in algorithms. The accuracy of the developed algorithms is compared with the difficulties retrospectively indicated by the participants. With the developed algorithms it should be possible to objectively examine the procedure in functional analysis for large numbers of participants. Thus, methodological support needs can be identified and individual training content can be proposed or developed within the framework of method research.