Recognizing cognitively demanding situations in design and measuring them for the semi-automated analysis of empirical studies in method development: AutoCodIng

为了能够开发出合适的设计方法，我们采用实证研究来识别产品开发中的困难。为了评价这些研究，使用的数据收集方法通常需要在数据准备中进行高度的解释。一方面，由于使用的方法，这种类型的数据收集强烈地干预了设计人员的工作流程，另一方面，仅提供在复杂评估后出现的困难的信息。此外，由于评估的复杂性，研究区间受到很大的限制。本研究项目的目的是研究连续可测量的困难指标，并将其转化为算法。该算法旨在自动化功能分析中的设计困难，从而客观地识别大量参与者。参与者在功能分析设计任务处理过程中的眼动追踪注视数据作为数据基础。使用注视数据代替定性数据采集（如观察协议、记录）旨在增加评估过程中的客观性。对于半自动记录，需要确定注视数据困难的指标。在眼动追踪数据的基础上，建立了三种不同的评价方法，并进行了比较。为此目的（1.）单个眼动追踪指标，（2.）基于机器学习的眼动追踪指标组合，以及（3.）在算法中开发和操作特定于上下文的指标。开发算法的准确性与参与者回顾性指出的困难进行了比较。利用开发的算法，应该可以客观地检查大量参与者的功能分析过程。因此，可以确定方法支持需求，并可以在方法研究的框架内提出或开发个人培训内容。

In order to be able to develop suitable design methods, empirical studies are used to identify difficulties in product development. For the evaluation of these studies, data collection methods are used which usually require a high degree of interpretation in the data preparation. On the one hand, this type of data collection intervenes strongly in the workflows of designers due to the methods used and, on the other hand, only provides information on difficulties that arise after a complex evaluation. Besides, the investigated intervals are strongly limited due to the complexity of the evaluation. The aim of this research project is to investigate continuously measurable indicators for difficulties and to convert them into an algorithm. The algorithm is intended to automate design difficulties in functional analysis and thus objectively identify them for large numbers of participants. Eye tracking gaze data of participants during the processing of design tasks for functional analysis serve as a data basis. The use of gaze data instead of qualitative data acquisition (e.g. observation protocols, transcripts) is intended to increase the objectivity during evaluation. For the semi-automated recording, it is necessary to determine indicators for difficulties in the gaze data. On the basis of the eye-tracking data, three different evaluation approaches are built up and compared with each other. For this purpose (1.) individual eye tracking metrics, (2.) a combination of eye tracking metrics based on machine learning and (3.) context-specific metrics are developed and operationalized in algorithms. The accuracy of the developed algorithms is compared with the difficulties retrospectively indicated by the participants. With the developed algorithms it should be possible to objectively examine the procedure in functional analysis for large numbers of participants. Thus, methodological support needs can be identified and individual training content can be proposed or developed within the framework of method research.