Collaborative Proposal: HSD-DHB-MOD The Grammars of Human Behavior

pi将对内部表征和相关过程进行实验研究和计算建模，这些过程是观察者行动感知和理解的基础，也是参与者行动计划和执行的基础。为了方便仔细的实验和正式的理论，pi将主要通过视觉系统来处理行为表征问题，询问我们如何使用我们的视觉来理解他人的行为？也就是说，我们如何执行从描绘简单动作的图像序列到允许动作识别、模仿等的相应内部表示的映射？pi将进一步探索用于分类、推理和判断他人的动作和行为的更高层次的认知表征和机制。该方法基于一种新的形式理论，即心理表征和过程有助于动作理解和计划，pi认为它提供了一种紧凑但功能强大且可扩展的计算方法，用于基于非常小的原子姿势元素（“关键帧”或“锚点”）及其相应的概率和语法规则组合的复杂动作（和动作序列）的分析和合成。这种概率“姿势语法”的动作表示方法类似于用于语音识别的最新技术（例如，隐马尔可夫模型），但关键的姿势轮廓取代了音素；这种增强的转换语法也很好地反映了机器人技术中用于鲁棒拟人化运动的复杂的新控制理论技术。行动表征系统不是单一的，而是在不同的行为“空间”对应的层次上占据了一系列信息结构：用于运动计划和生产的机电空间；认知空间，包括动作识别、分析和评价的表征；视觉运动空间，对人的动作引起的视觉运动进行编码和组织；语言运动空间由概念/符号动作编码构成。排除后一个空间，pi的理论、计算和实验努力试图澄清和正式描述这些空间中表征的本质，以及至关重要的跨空间表征的映射。值得注意的是，他们探索了一种候选动作表征，被称为视觉运动表征，它可以促进对观察到的动作的理解，可以概括和共鸣用于产生运动的实际运动表征。此外，他们提出了一种从离散动作元素或锚点获得这种表示的有希望的方法。更广泛的影响:该项目将导致心理学研究和应用方面的重大进步（例如，基于退化的生物运动的强大社会判断），运动机能学（例如，运动轮廓的分析/建模/训练，如在田径或病理学/康复中），机器人技术（例如，拟人化机器人的控制），人类和计算机视觉（例如，数字视频中的自动动作识别），以及其他与人类/类人行为的解释和生产有关的领域。

The PIs will undertake an experimental study and computational modeling of the internal representations and associated processes that underlie action perception and understanding by observers, and action planning and execution by actors. To facilitate both careful experimentation and formal theory, the PIs will approach the behavior representation problem primarily through the visual system, asking how do we understand the actions of others using our vision? That is, how do we perform mappings from image sequences depicting simple actions to the corresponding internal representations that allow action recognition, imitation, etc? The PIs will further explore higher-level cognitive representations and mechanisms used to categorize, reason about, and judge the movements and actions of others. The approach is based on a novel formal theory of the mental representations and processes subserving action understanding and planning, which the PIs believe provides a compact but powerful and extensible computational approach to the analysis and synthesis of complex actions (and action sequences) based on a very small set of atomic postural elements ("key frames" or "anchors") and the corresponding probabilistic, grammatical rules for their combination. This probabilistic "pose grammar" approach to action representation is similar to state of the art techniques used for speech recognition (e.g., hidden Markov models), but with key postural silhouettes taking the place of phonemes; such augmented transition grammars also nicely reflect sophisticated new control-theoretic techniques in robotics for robust anthropomorphic movement. The action representational system is not monolithic, but rather occupies a spectrum of informational structures at hierarchical levels corresponding to different behavior "spaces": mechatronic space, used in movement planning and production; cognitive space, involving representations for action recognition, analysis, and evaluation; visual motion space, which encodes and organizes visual motion caused by human action; and linguistic motion space, comprised of conceptual/symbolic action encoding. Excluding here the latter space, the PIs' theoretic, computational, and experimental efforts seek to clarify and formally describe both the nature of the representations in these spaces and, crucially, the mapping of representations across spaces. Notably, they explore a candidate action representation, referred to as a visuo-motor representation, which, in facilitating the understanding of observed actions, may recapitulate and resonate with the actual motor representations used to generate movement. Moreover, they present a promising approach for obtaining this representation from discrete action elements or anchors.Broader Impacts: This project will lead to significant advancements in both research and applications in psychology (e.g., robust social judgments given degraded biological motion), kinesiology (e.g., analysis/modeling/training of movement profiles, as in athletics or pathology/rehabilitation), robotics (e.g., control of anthropomorphic robots), human and computer vision (e.g., automated action recognition in digital video), and other fields concerned with the interpretation and production of human/humanoid action.