PathFS: A File System for the Hierarchical Cloud

物联网(IoT)的出现导致了海量数据的产生，对实时处理提出了新的要求，分布式异构计算资源激增。在智能城市和工业4.0等新兴场景中，文件、视频和图片是由互联网边缘的传感器产生的。通过互联网链路将这些数据传输到云数据中心进行处理和存储对网络提出了多个带宽要求，并且与需要低延迟的应用程序不兼容。边缘计算已经成为一种替代方法，它利用位于互联网边缘的小型区域数据中心，从几台计算机到几个机架，执行处理任务和存储数据。这一靠近数据生成位置的位置使新系统能够提供低延迟并降低网络负载。该计划是多伦多大学(UofT)和柯尼卡美能达商业解决方案(加拿大)有限公司的合作项目。它的目标是创建PathFS，这是一种新的分布式文件系统(DFS)，旨在部署在从网络边缘到广域云的数据中心层次结构中。通过提供对数据的低延迟访问并支持数据密集型工作负载的本地执行，PathFS旨在支持可从边缘计算中受益的新型应用程序。通过使分布式组件能够使用熟悉的文件系统接口相互通信，PathFS将简化边缘应用程序的开发。例如，监控工厂生产线的摄像头可以将产品的图像存储到PathFS中。然后，附近的计算机可以读取图像，使用光推理ML模型来检测缺陷。反过来，更强大的计算机可以通过使用路径文件系统从多条生产线收集数据来执行工厂范围的分析。最后，在传统的广域云数据中心上运行的进程可以使用从多个工厂收集的数据的路径文件系统进行分析。

The advent of the Internet of Things (IoT) has led to a large amount of generated data, new requirements forreal-time processing, and the proliferation of distributed heterogeneous computing resources. In emergingscenarios, such as Smart Cities and Industry 4.0, files, videos, and pictures are produced by sensors on theedges of the Internet. Transporting this data over Internet links to cloud data centres for processing and storage places several bandwidth requirements on the network and is not compatible with applications that require low latency. Edge computing has emerged as an alternative approach that leverages small regional data centres, from a few computers to a few racks, at the Internet edges to carry out processing tasks and store data. This proximity to the location of data generation enables new systems that provide low latency and reduce network load.This proposal is a collaboration between the University of Toronto (UofT) and Konica Minolta Business Solutions (Canada) Ltd. . It aims to create PathFS, a new distributed file system (DFS) designed to be deployed over a hierarchy of data centres that span from the edge of the network for the wide-area cloud. PathFS is designed to support novel applications that can benefit from edge computing by providing low latency access to data and enabling local execution of data-intensive workloads. PathFS will simplify the development of edge applications by making it possible for distributed components to communicate with each other using a familiar file system interface. For example, a camera monitoring a factory's production line could store images of products to PathFS. A nearbyboard computer could then read the images to detect defects, using a light inference ML model. In turn, a morepowerful computer could perform factory-wide analytics by using PathFS to collect data from multipleproduction lines. Finally, a process running on a traditional wide-area cloud data centre could use PathFS toperform analytics using data collected from multiple factories.