PathFS: A File System for the Hierarchical Cloud

点击翻译按钮获取中文摘要

The advent of the Internet of Things (IoT) has led to a large amount of generated data, new requirements forreal-time processing, and the proliferation of distributed heterogeneous computing resources. In emergingscenarios, such as Smart Cities and Industry 4.0, files, videos, and pictures are produced by sensors on theedges of the Internet. Transporting this data over Internet links to cloud data centres for processing and storage places several bandwidth requirements on the network and is not compatible with applications that require low latency. Edge computing has emerged as an alternative approach that leverages small regional data centres, from a few computers to a few racks, at the Internet edges to carry out processing tasks and store data. This proximity to the location of data generation enables new systems that provide low latency and reduce network load.This proposal is a collaboration between the University of Toronto (UofT) and Konica Minolta Business Solutions (Canada) Ltd. . It aims to create PathFS, a new distributed file system (DFS) designed to be deployed over a hierarchy of data centres that span from the edge of the network for the wide-area cloud. PathFS is designed to support novel applications that can benefit from edge computing by providing low latency access to data and enabling local execution of data-intensive workloads. PathFS will simplify the development of edge applications by making it possible for distributed components to communicate with each other using a familiar file system interface. For example, a camera monitoring a factory's production line could store images of products to PathFS. A nearbyboard computer could then read the images to detect defects, using a light inference ML model. In turn, a morepowerful computer could perform factory-wide analytics by using PathFS to collect data from multipleproduction lines. Finally, a process running on a traditional wide-area cloud data centre could use PathFS toperform analytics using data collected from multiple factories.