SBIR Phase II: Efficient Custom Machine Learning for Embedded Intelligence in the Internet of Things

小型企业创新研究(SBIR)第二阶段项目的更广泛影响/商业潜力将导致通过支持许多垂直应用的水平平台技术部署机器学习(ML)解决方案的性能、能力和成本显著提高。这一改进将加快智能系统的部署，并通过本地化智能提高可扩展性。我们的技术将硬件设计、实施和部署自动化到现场可编程门阵列(FPGA)平台。我们最初的目标垂直市场：安全和监控、预测性维护和医疗保健代表了物联网设备增长市场中数千亿美元的价值，并通过提高部署和运营效率以及降低社会成本产生了显著更大的经济影响。这些关键技术的性能、功耗和可扩展性的提高将导致公共安全的改善，家庭医疗服务的智能化，以及通过在关键工业设备上部署预测性维护技术来提高制造和能源系统的效率。这些技术的广泛应用将带来大量的能源节约和相应的碳排放减少，减少负面事件造成的经济损失，改善对预测或活动负面事件的可扩展性和响应时间，并由于低成本、低功耗和物理上的小传感器系统而降低部署和操作成本。拟议的项目重点是为ML应用程序设计高性能、节能的平台，以及相关的设计工具和库。神经网络被广泛用于许多机器学习问题，但优化有效的部署目前需要对选项的巨大设计空间进行广泛的反复试验。我们的深度神经网络(DNN)优化框架应用位宽优化、权重共享和自动剪枝，将计算和权重存储需求减少到原来的1/10以上，同时分析结果质量影响，并使用微调再训练将精度降级降至最低或消除。然后，我们的高级综合(HLS)工具将优化的网络转换为硬件，同时应用流水线、功能单元并行性、资源共享和特定于平台的优化。这些工具共同自动化并加速了分析、优化和实施ML以进行硬件部署的过程，从而减少了硬件设计所需的时间和专业知识。我们的部署平台是模块化的可组合平台，用于音频/视频信号处理、特征提取和分类、系统控制(例如平移-倾斜-变焦摄像头)以及决策或云服务通信的小型、低成本部署。我们将通过安全/监控、预测性维护和垂直医疗保健领域的解决方案以及平台、工具和知识产权库的紧密集成来扩展我们第一阶段项目的竞争优势。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase II project will result in a significant improvement in the performance, power, and cost of deploying machine learning (ML) solutions through horizontal platform technologies that enable many vertical applications. This improvement will accelerate deployment of intelligent systems and improve scalability through localized intelligence. Our technology automates hardware design, implementation, and deployment to Field-Programmable Gate Array (FPGA) platforms. Our initial target verticals: Security and Surveillance, Predictive Maintenance, and Healthcare represent hundreds of billions USD in growth markets for IoT devices and substantially more economic impact through improved efficiency in deployment and operations and reduced societal costs. Improved performance, power consumption and scalability of these key technologies will lead to improved public safety, improved intelligence in home healthcare services, and more efficient manufacturing and energy systems through deployment of Predictive Maintenance technologies on key industrial equipment. Wide deployment of these technologies will lead to substantial energy savings and a corresponding reduction in carbon emissions, reduced economic loss due to negative events, improved scalability and response time to predicted or active negative events, and lower cost in deployment and operations due to low cost, low power, and physically small sensor systems.The proposed project focuses on design of high performance, energy-efficient platforms for ML applications, and associated design tools and libraries. Neural networks are heavily used for many machine learning problems but optimizing for efficient deployment currently requires extensive trial-and-error for the large design space of options. Our deep neural network (DNN) optimization framework applies bit-width optimizations, weight sharing and pruning automatically to reduce computation and weight storage demands by more than 10X, while analyzing quality of results impact and using fine-tuned retraining to minimize or eliminate accuracy degradation. Our high level synthesis (HLS) tool then translates optimized networks to hardware while applying pipelining, functional unit parallelism, resource sharing, and platform-specific optimizations. Together these tools automate and accelerate the process of analyzing, optimizing and implementing ML for hardware deployment, reducing time and required expertise for hardware design. Our deployment platforms are modular, composable platforms for small, low-cost deployments of audio/video signal processing, feature extraction and classification, systems control (e.g. pan-tilt-zoom cameras), and communications to decision-making or cloud services. We will extend competitive advantages from our Phase I project with features for solutions in the security/surveillance, predictive maintenance, and healthcare verticals, and tight integration of platforms, tools and IP libraries.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.