Interpreting Human Behaviour in Video using FSA's and Object Context

使用 FSA 和对象上下文解释视频中的人类行为

• 批准号: 0534837
• 负责人: David Forsyth
• 金额: --
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2006
• 资助国家: 美国
• 起止时间: 2006-03-01 至 2010-02-28
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0534837&HistoricalAwards=false
• 关键词: Interpreting; Human; Behaviour; Video; using

PI: David ForsythTitle: Interpreting Human Behavior in Video using FSA's and Object ContextUnderstanding what people are doing in a video is one of the great unsolved problems of computer vision. A fair solution opens tremendous application possibilities. The proposed work will use existing tools from the speech and object recognition community --- in particular, finite state automata or FSA's --- to obtain an understanding of activities that depend on detailed information about the body.The particular focus is everyday activity. In this case, a fixed vocabulary either doesn't exist, or isn't appropriate. For example, one does not know words for behaviors that appear familiar. One way to deal with this is to work with a notation (for example, laban notation); but such notations typically work in terms that are difficult to map to visual observables (for example, the weight of a motion). The alternatives are either to develop a vocabulary, or to develop expressive tools for authoring models.This project will explore the third approach of building tools for authoring models of behavior quickly and expressively using finite-state methods. Research will explore a class of models that are easy to author from existing, or easily available, data. The interpretation of what someone is doing is affected by the objectsnearby --- a person standing near a bus stop is doing something different from a person standing near an office door. The models studied make it practical to investigate this phenomenon of object context, using recent advances from the object recognition literature.Evaluating models for everyday behaviors is hard, because there is no prospect of obtaining a large collection of marked up video (among other things, there isn't a vocabulary in which to mark it up). This project will use proxies --- statistics that are hard to measure from video without accurate inferences of behavior, but easy to measure in other ways --- to evaluate behavior representations. These will make it possible to tell whether, for example, a model of buying a beverage represents the concept accurately.Intellectual merits: This project will produce very large finite state models of behavior using the same hierarchical authoring methods used in speech. There will be a particular emphasis on behaviors which require one to understand the kinematic configuration of the body, a topic that has been very difficult to study to date, with an intention of identifying basic building blocks of a vocabulary of everyday behavior. The results should include datasets of public behavior that can be disseminated, without encountering privacy concerns. New insights into the structure of human motion and behavior should emerge from (a) observations of people in public; (b) the process of authoring models; and (c) methods for identifying and modelling compositional structure in motion.Broader impact: This project should make substantial progress on one of the key open and applicable problems in computer vision. Methods that can search video for particular behaviors and compute statistics of behaviors have a wide range of applications, including human-computer interfaces built around computers that can watch the body; an improved understanding of what people do in public which will result in better architectural planning; more efficient management of surveillance data, allowing searches for dangerous behaviors while preserving privacy. Education and access: This project will contribute to the graduate training of several students, and work described will contribute to a planned text on computing with human motion.URL: http://luthuli.cs.uiuc.edu/~daf/action.html

PI：David Forsythtitle：使用FSA和对象上下文解释视频中人们在视频中所做的事情是计算机视觉的最大尚未解决的问题之一。 一个公平的解决方案开启了巨大的应用程序可能性。 拟议的工作将使用语音和对象识别社区中的现有工具 - 尤其是有限状态自动机或FSA - 以了解依赖有关身体详细信息的活动的理解。特别的重点是日常活动。 在这种情况下，固定的词汇要么不存在，要么不合适。 例如，人们不知道看起来熟悉的行为的单词。处理此问题的一种方法是使用符号（例如，Laban符号）；但是，这些符号通常用难以映射到视觉可观察物的术语（例如，运动的重量）。替代方案要么是开发词汇量，要么开发用于创作模型的表达工具。该项目将探讨使用有限态方法快速而表达的行为模型来构建工具的第三种方法。研究将探索一类模型，这些模型易于从现有或易于获得的数据中撰写。 对某人正在做的事情的解释受到对象的影响 - 一个站在公共汽车站附近的人正在做一些与站在办公室门附近的人不同的事情。 研究的模型使研究对象环境的这种现象是实用的，使用对象识别文献中的最新进展。对日常行为的评估模型很难，因为没有一个可以获得大量标记的视频集合的前景（除其他事项外，没有一个可以标记它的词汇）。 该项目将使用代理---统计数据，这些统计信息很难从视频中测量而无需准确推断行为，但易于以其他方式衡量 - 以评估行为表示。 这些将使例如购买饮料的模型是否准确地代表概念。 将特别强调行为，这些行为需要一个人了解身体的运动学配置，这一主题很难研究，目的是确定日常行为词汇的基本构件。 结果应包括可以传播的公共行为数据集，而不会遇到隐私问题。对人类行动和行为结构的新见解应从（a）公开的人们观察中出现； （b）创作模型的过程； （c）识别运动中识别和建模组成结构的方法。BOADER的影响：该项目应在计算机视觉中的关键开放和适用问题之一上取得重大进展。 可以搜索视频的特定行为和行为统计的方法具有广泛的应用，包括围绕可以观察身体的计算机建立的人类计算机接口；对人们在公共场所所做的事情有了深入的了解，这将导致更好的建筑计划；更有效地管理监视数据，可以在保留隐私的同时搜索危险行为。 教育与访问：该项目将有助于几名学生的研究生培训，所描述的工作将有助于使用人类Motion.url的计算文本：http：//luthuli.cs.uiuc.edu/~daf/~daf/action.html