CRII: CHS: Enabling Behavior Sensing via the Cloud and its Application to Public Speaking

公众演讲通常是人们最害怕的任务之一；其中一个后果是，即使在反复练习演讲后，许多人也会发现站在观众面前演讲太仓促了。人们往往渴望提高他们的公共演讲技能，但缺乏资源和社会污名可能会阻碍他们获得他们所寻求的个性化培训的能力。PI在这个项目中的目标是在他之前工作的基础上建立一个研究计划，以开发一个无处不在的(基于云的)自动化社交感知框架，该框架可以识别和解释人类的非语言数据(包括面部表情、语气、肢体语言等)，然后在他们想要的地方和时间向用户提供建设性的反馈。对人类所有的非语言行为进行建模仍然是一项具有挑战性的工作。使用我们面部的43块肌肉，我们可以产生10,000种独特的面部表情组合；声调、肢体语言和生理学元素等模式增加了复杂性。虽然计算机现在可以识别微笑和皱眉等基本表情，但对个人意图的自动解释仍然是一个活跃的探索领域(例如，客户微笑并不一定表明S满意)。这项研究代表着朝着开发算法和实现一个实用的框架迈出了一步，该框架可以捕获和解释非语言数据，同时在公开演讲的背景下提供有意义的反馈。项目成果最终将改变适应和学习社交技能的方式，这将对有社交困难的人(例如，阿斯伯格综合症患者)产生广泛影响。人类的非语言行为可能是微妙的，经常是令人困惑的，甚至可能看起来相互矛盾。虽然计算机算法在客观和一致地感知人类细微的行为方面比人类更可靠，但人类的智能目前在解释语境行为方面要优越得多。这项研究采用了一种将计算机算法与人类智能相结合的方法，以实现对非语言行为的自动感知和几乎实时的解释。PI的方法是开发一个健壮且可扩展的基于Web的感知框架，通过利用云基础设施自动捕获和分析S的个人行为，而不需要来自最终用户的任何主要计算资源。这项工作将包括三个阶段：开发一个支持云的感知平台，用于自动识别非语言行为；开发算法，使用所谓的群体智慧将行为数据与人类判断相结合，以生成有意义的见解、解释和社交建议；以及运行以用户为中心的迭代研究，以验证该框架对普通公众和从业者的有效性。这项工作还将导致在设计计算机界面方面做出核心贡献。由于行为建模方法通常假设大量的自然主义数据，最好是在野外收集的；因此值得注意的是，PI的感知框架有可能收集最大的自然主义非语言数据集之一。

Public speaking is a task that people often rank as their top fear; one consequence is that even after repeatedly practicing a presentation many find they end up speaking too hastily when standing before the audience. People often desire to improve their public speaking skills, but lack of resources and social stigma may impede their ability to obtain the personalized training they seek. The PI's objective in this project is to build on his prior work and establish a research program to develop a ubiquitously available (Cloud based) automated social sensing framework that can recognize and interpret human nonverbal data (including facial expressions, tone of voice, body language, etc.), and then present constructive feedback to its users where they want and when they want. Modeling of the full range of human nonverbal behavior remains a challenging endeavor. Using the 43 muscles in our face, we can produce 10,000 unique combinations of facial expressions; modalities such as vocal tone, body language, and elements of physiology add to the complexity. While computers can now recognize basic expressions such as smiling and frowning, the automated interpretation of an individual's intent remains an active area of exploration (e.g., a smiling customer does not necessarily indicate that s/he is satisfied). This research represents a step towards developing algorithms and implementing a practical framework that can capture and interpret nonverbal data while providing meaningful feedback in the context of public speaking. Project outcomes ultimately will transform the way social skills are adapted and learned, which will have a broad impact on people with social difficulties (e.g., those with Asperger's syndrome). Human nonverbal behaviors can be subtle, are often confusing, and may even appear contradictory. While computer algorithms are more reliable than people at sensing subtle human behavior objectively and consistently, human intelligence is currently far superior at interpreting contextual behavior. This research adopts an approach that couples computer algorithms with human intelligence towards automated sensing and interpretation of nonverbal behavior in nearly real time. The PI's approach is to develop a robust and scalable Web-based sensing framework that will automatically capture and analyze an individual?s behavior by exploiting the Cloud infrastructure, without requiring any major computational resources from the end-user. The work will include three phases: development of a Cloud-enabled sensing platform for automated recognition of nonverbal behavior; development of algorithms for combining the behavioral data with human judgment using the so-called wisdom of the crowd to generate meaningful insights, interpretations, and social recommendations; and running user centric iterative studies to validate the framework for the general public as well as practitioners. The work will also lead to core contributions in designing computer interfaces. And since behavioral modeling methods typically assume a large amount of naturalistic data, preferably collected in the wild; it is therefore noteworthy that the PI's sensing framework has the potential to collect one of the largest naturalistic nonverbal datasets.