Hardware/Software Co-Design for Machine Learning Accelerators

点击翻译按钮获取中文摘要

Artificial intelligence (AI) dominates our daily life, from automatic language translation to face detection. This success is due to the widespread adoption of artificial neural networks. These networks are inspired by the brain and solve tasks by letting the machine learn automatically, without requiring to be programmed beforehand. This machine learning approach has been known for over 30 years; however, up until now, there were simply not enough computational power available to handle real size problems. The widespread adoption of high performance parallel processors, such as Graphics Processing Units (GPUs), has enabled the development of complex neural networks able to solve real size problems. This has triggered the deep learning revolution, which is set to revolutionize virtually every field, from robotics to business. However, these new networks require devices capable of delivering even greater computational power, and more importantly, with high energy efficiency. Most of these workloads runs on the cloud or on Internet's edge (e.g. mobile devices) where energy consumption is the prime concern. As a result, a new race among companies has started to design the most energy efficient accelerators specialized for machine learning. The development of new hardware is a very time consuming, multi-years, process. This involves a lot of manual design and testing requiring large teams of designers. Furthermore, hardware companies tend to take a conservative approach and design their new hardware for well known, well established machine learning techniques. This hinders the adoption of new machine learning techniques since hardware is always designed for obsolete techniques. The long term goal of this proposal is to develop a set of common principles for fully automating the design of accelerators, with the short term five years goal focusing on neural networks in particular. This involves both automating hardware design as well as software interface design, known in the community as co-design. When a new accelerator emerges, it can simply be dropped in as a replacement for the older one and will just work out of the box. This will also lead to a rapid turnaround time from design to hardware, which ultimately means more accelerator diversity, each specialized for a particular type of problems. This will enable the adoption of new revolutionary machine learning applications that might otherwise never see daylight. Canada is considered by many as the leader for machine learning and its researchers have pioneered many advances in the field. Given the importance that specialized hardware plays in enabling machine learning applications, this research program will strengthen this position on the system side by training the next generation of system-focused machine learning expert. We intend to release all our software as open source, to encourage uptake of our techniques among researchers and industry players.