SBIR Phase II: Efficient Custom Machine Learning for Embedded Intelligence in the Internet of Things

这个小企业创新研究（SBIR）第二阶段项目的更广泛的影响/商业潜力将导致通过水平平台技术部署机器学习（ML）解决方案的性能，功率和成本的显着改善，从而实现许多垂直应用。这种改进将加速智能系统的部署，并通过本地化智能提高可伸缩性。我们的技术自动化硬件设计，实现和部署到现场可编程门阵列（FPGA）平台。我们最初的垂直目标是：安全和监控、预测性维护和医疗保健，在物联网设备的增长市场中占数千亿美元的份额，并通过提高部署和运营效率以及降低社会成本，产生更大的经济影响。通过在关键工业设备上部署预测性维护技术，这些关键技术的性能、功耗和可扩展性的改进将改善公共安全，提高家庭医疗保健服务的智能化，并提高制造和能源系统的效率。这些技术的广泛部署将带来大量的能源节约和相应的碳排放减少，减少负面事件造成的经济损失，提高可扩展性和对预测或主动负面事件的响应时间，并由于低成本、低功耗和物理上小的传感器系统而降低部署和操作成本。提议的项目侧重于为ML应用程序设计高性能，节能的平台，以及相关的设计工具和库。神经网络被大量用于许多机器学习问题，但优化有效部署目前需要大量的试验和错误的选择设计空间。我们的深度神经网络（DNN）优化框架应用位宽优化、权值共享和自动修剪，将计算和权值存储需求减少10倍以上，同时分析结果质量影响并使用微调再训练来最小化或消除精度下降。然后，我们的高级综合（HLS）工具将优化的网络转换为硬件，同时应用流水线、功能单元并行、资源共享和特定于平台的优化。这些工具一起自动化和加速了分析、优化和实施ML硬件部署的过程，减少了硬件设计的时间和所需的专业知识。我们的部署平台是模块化的、可组合的平台，适用于小型、低成本的音频/视频信号处理、特征提取和分类、系统控制（如泛倾斜变焦相机）以及决策或云服务通信部署。我们将扩展第一阶段项目的竞争优势，为安全/监视、预测性维护和医疗保健垂直领域的解决方案提供功能，并将平台、工具和IP库紧密集成。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase II project will result in a significant improvement in the performance, power, and cost of deploying machine learning (ML) solutions through horizontal platform technologies that enable many vertical applications. This improvement will accelerate deployment of intelligent systems and improve scalability through localized intelligence. Our technology automates hardware design, implementation, and deployment to Field-Programmable Gate Array (FPGA) platforms. Our initial target verticals: Security and Surveillance, Predictive Maintenance, and Healthcare represent hundreds of billions USD in growth markets for IoT devices and substantially more economic impact through improved efficiency in deployment and operations and reduced societal costs. Improved performance, power consumption and scalability of these key technologies will lead to improved public safety, improved intelligence in home healthcare services, and more efficient manufacturing and energy systems through deployment of Predictive Maintenance technologies on key industrial equipment. Wide deployment of these technologies will lead to substantial energy savings and a corresponding reduction in carbon emissions, reduced economic loss due to negative events, improved scalability and response time to predicted or active negative events, and lower cost in deployment and operations due to low cost, low power, and physically small sensor systems.The proposed project focuses on design of high performance, energy-efficient platforms for ML applications, and associated design tools and libraries. Neural networks are heavily used for many machine learning problems but optimizing for efficient deployment currently requires extensive trial-and-error for the large design space of options. Our deep neural network (DNN) optimization framework applies bit-width optimizations, weight sharing and pruning automatically to reduce computation and weight storage demands by more than 10X, while analyzing quality of results impact and using fine-tuned retraining to minimize or eliminate accuracy degradation. Our high level synthesis (HLS) tool then translates optimized networks to hardware while applying pipelining, functional unit parallelism, resource sharing, and platform-specific optimizations. Together these tools automate and accelerate the process of analyzing, optimizing and implementing ML for hardware deployment, reducing time and required expertise for hardware design. Our deployment platforms are modular, composable platforms for small, low-cost deployments of audio/video signal processing, feature extraction and classification, systems control (e.g. pan-tilt-zoom cameras), and communications to decision-making or cloud services. We will extend competitive advantages from our Phase I project with features for solutions in the security/surveillance, predictive maintenance, and healthcare verticals, and tight integration of platforms, tools and IP libraries.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.