MediaPipe: A Framework for Perceiving and Processing Reality
MediaPipe: A Framework for Perceiving and Processing Reality
复制标题
MediaPipe：感知和处理现实的框架
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Matthias Grundmann
中科院分区：
文献类型：
--
作者：
C. Lugaresi;Jiuqiang Tang;Hadon Nash;C. McClanahan;Esha Uboweja;Michael Hays;Fan Zhang;Chuo;M. Yong;Juhyun Lee;W. Chang;Wei Hua;Manfred Georg;Matthias Grundmann
构建一个处理感知输入的应用程序不仅仅是运行一个机器学习模型。开发者必须利用各种设备的功能；平衡资源使用和结果质量；并行运行多个操作，并使用流水线；并确保时间序列数据正确同步。MediaPipe框架解决了这些挑战。开发人员可以使用MediaPipe轻松快速地将现有和新的感知组件组合到原型中，并将其推进到完善的跨平台应用程序中。开发人员可以配置使用MediaPipe构建的应用程序来有效地管理资源（CPU和GPU），以实现低延迟性能，处理时间序列数据（如音频和视频帧）的同步，并测量性能和资源消耗。我们展示了这些功能使开发人员能够专注于算法或模型开发，并使用Me-diaPipe作为迭代改进其应用程序的环境，其结果可在不同的设备和平台上重现。MediaPipe将在https://github.com/google/mediapipe上开源。
Building an application that processes perceptual inputs involves more than running an ML model. Devel-opers have to harness the capabilities of a wide range of devices; balance resource usage and quality of results; run multiple operations in parallel and with pipelining; and ensure that time-series data is properly synchronized. The MediaPipe framework addresses these challenges. A developer can use MediaPipe to easily and rapidly combine existing and new perception components into prototypes and advance them to polished cross-platform applications. The developer can configure an application built with MediaPipe to manage resources efficiently (both CPU and GPU) for low latency performance, to handle synchronization of time-series data such as audio and video frames and to measure performance and resource consumption. We show that these features enable a developer to focus on the algorithm or model development, and use Me-diaPipe as an environment for iteratively improving their application, with results reproducible across different devices and platforms. MediaPipe will be open-sourced at https://github.com/google/mediapipe .