Collaborative Proposal: HSD-DHB-MOD The Grammars of Human Behavior

执行主任将对作为观察员行动感知和理解以及行为者行动规划和执行基础的内部表述和相关过程进行实验研究和计算建模。为了便于仔细的实验和正式的理论，PI将主要通过视觉系统来处理行为表征问题，询问我们如何理解使用我们的视觉的其他人的行为？也就是说，我们如何执行从描述简单动作的图像序列到允许动作识别、模仿等的相应内部表示的映射？PI将进一步探索更高层次的认知表征和机制，用于对他人的动作和行为进行分类、推理和判断。该方法基于一种新的心理表征和过程的形式理论，有助于动作理解和规划，PI认为该理论提供了一种紧凑但强大且可扩展的计算方法来分析和综合复杂动作(和动作序列)，其基础是一组非常小的原子姿势元素(关键帧或锚)以及它们组合的相应概率和语法规则。这种用于动作表示的概率“姿势语法”方法类似于用于语音识别的最新技术(例如，隐马尔可夫模型)，但用关键的姿势轮廓代替音素；这种增强的转换语法还很好地反映了机器人中用于健壮的拟人运动的复杂的新控制理论技术。动作表征系统不是一成不变的，而是占据了与不同行为“空间”相对应的层级信息结构的光谱：机电空间，用于运动规划和生产；认知空间，涉及动作识别、分析和评估的表征；视觉运动空间，编码和组织人类动作引起的视觉运动；以及语言运动空间，由概念/符号动作编码组成。这里不包括后一个空间，PI的理论、计算和实验努力试图澄清和正式描述这些空间中的表示的性质，以及至关重要的是，跨空间的表示的映射。值得注意的是，他们探索了一种被称为视觉-运动表征的候选动作表征，在促进对观察到的动作的理解方面，它可以概括并与用于产生运动的实际运动表征产生共鸣。此外，他们提出了一种从离散的动作元素或锚中获得这种表示的有前途的方法。广泛的影响：该项目将在心理学(例如，给定退化的生物运动的稳健的社会判断)、运动学(例如，运动学或病理学/康复中的运动轮廓的分析/建模/训练)、机器人学(例如，拟人机器人的控制)、人类和计算机视觉(例如，数字视频中的自动动作识别)以及与解释和产生人类/类人动作有关的其他领域的研究和应用方面取得重大进展。

The PIs will undertake an experimental study and computational modeling of the internal representations and associated processes that underlie action perception and understanding by observers, and action planning and execution by actors. To facilitate both careful experimentation and formal theory, the PIs will approach the behavior representation problem primarily through the visual system, asking how do we understand the actions of others using our vision? That is, how do we perform mappings from image sequences depicting simple actions to the corresponding internal representations that allow action recognition, imitation, etc? The PIs will further explore higher-level cognitive representations and mechanisms used to categorize, reason about, and judge the movements and actions of others. The approach is based on a novel formal theory of the mental representations and processes subserving action understanding and planning, which the PIs believe provides a compact but powerful and extensible computational approach to the analysis and synthesis of complex actions (and action sequences) based on a very small set of atomic postural elements ("key frames" or "anchors") and the corresponding probabilistic, grammatical rules for their combination. This probabilistic "pose grammar" approach to action representation is similar to state of the art techniques used for speech recognition (e.g., hidden Markov models), but with key postural silhouettes taking the place of phonemes; such augmented transition grammars also nicely reflect sophisticated new control-theoretic techniques in robotics for robust anthropomorphic movement. The action representational system is not monolithic, but rather occupies a spectrum of informational structures at hierarchical levels corresponding to different behavior "spaces": mechatronic space, used in movement planning and production; cognitive space, involving representations for action recognition, analysis, and evaluation; visual motion space, which encodes and organizes visual motion caused by human action; and linguistic motion space, comprised of conceptual/symbolic action encoding. Excluding here the latter space, the PIs' theoretic, computational, and experimental efforts seek to clarify and formally describe both the nature of the representations in these spaces and, crucially, the mapping of representations across spaces. Notably, they explore a candidate action representation, referred to as a visuo-motor representation, which, in facilitating the understanding of observed actions, may recapitulate and resonate with the actual motor representations used to generate movement. Moreover, they present a promising approach for obtaining this representation from discrete action elements or anchors.Broader Impacts: This project will lead to significant advancements in both research and applications in psychology (e.g., robust social judgments given degraded biological motion), kinesiology (e.g., analysis/modeling/training of movement profiles, as in athletics or pathology/rehabilitation), robotics (e.g., control of anthropomorphic robots), human and computer vision (e.g., automated action recognition in digital video), and other fields concerned with the interpretation and production of human/humanoid action.