SBIR Phase I: Using Deep Learning and Action Recognition to Automatically Digitize Human Actions at Scale: Putting Workers at the Center of the Next Industrial Revolution

该项目更广泛的影响/商业潜力是使用高频率收集的视觉信息自动分析人体运动。在制造业中，？占美国GDP的11%[经济分析局]对这组新的大规模时间和运动数据的收集和解释使得能够在对装配过程的理解和人类生产力的优化方面实现显著的改进。此外，制造商可以使用该系统来标记过程错误或偏差？理想的是在真实的时间里？允许在偏差进一步向下传播价值链之前进行缓解。正是这两项功能使制造商能够避免成本高昂的返工和召回，提高工人的准确性，发现优化流程和创收的新机会，并标记工人安全问题。因此，工人和管理层围绕效率和竞争力的共同目标达成一致？减少对自动化的恐惧，同时大幅提高生产力，并可能保护就业机会。 这个小型企业创新研究第一阶段项目将提高原型深度学习后端的鲁棒性，用于从视频数据流中自动识别动作。虽然物体识别现在很常见，但动作检测呢？例如，在一个实施例中，推断行为者和物体的行为和意图尚未解决或商业化。事实上，它仍然是一个活跃的研究领域。解决这个问题需要克服工业环境中的技术障碍，包括改变演员、照明条件和摄像机视角;手动标记大量视频数据;将大量数据传输到云端和从云端传输;以高置信度准确推断;开发直观的人机界面，未来可能包括非传统渠道，如AR/VR，文本到语音，触觉反馈等。

The broader impact/commercial potential of this project is to enable the automatic analysis of human motion using visual information gathered at high frequency. Within the manufacturing context?which represents 11% of US GDP [Bureau of Economic Analysis]?the collection and interpretation of this new set of large-scale time-and-motion data enables dramatic improvements in the understanding of assembly processes and optimization of human productivity. In addition, manufacturers can use this system to flag process errors or deviations?ideally in real time?allowing for mitigation before the deviations propagate further down the value chain. Just these two capabilities enable manufacturers to avoid costly rework and recalls, improve worker accuracy, discover new opportunities to optimize processes and generate revenue generation, and flag worker safety issues. As a result, workers and management become aligned around shared goals of efficiency and competitiveness?reducing fears of automation while dramatically improving productivity and possibly protecting jobs. This Small Business Innovation Research Phase I project will improve the robustness of a prototype deep learning back-end for automatic action recognition from a stream of video data. While object recognition is now commonplace, action detection?e.g., inferring the behavior and intentions of actors and objects over time?has not yet been solved or commercialized. In fact, it is still an active research area. Solving this problem requires overcoming technical hurdles in industrial settings that include changing actors, lighting conditions and camera perspectives; manual labelling of large volumes of video data; transmitting large volumes of data to and from the cloud; accurately inferencing with high levels of confidence; and developing intuitive human/system interfaces that may, in the future, include unconventional channels such as AR/VR, text-to-speech, haptic feedback, amongst others.