Elements: An ML Ecosystem of Filament Detection: Classification, Localization, and Segmentation

自从物体检测算法优于人类以来，十年已经过去了。在此期间，计算机视觉领域取得的前所未有的成就使许多人相信目标检测是一个已经解决的问题。然而，当涉及到科学图像，如显微镜，望远镜，航空，卫星和医学图像时，通用的目标检测算法远非完美。基于物体纹理特征的像素级精确分割和物理属性识别仍然是许多跨学科研究领域的突出挑战。太空气象就是这样一个领域。与地球极端事件类似，极端空间天气事件可能对人类产生巨大的经济和附带影响。对太阳灯丝的连续和自动监测在实现可靠的空间气象预报/预测系统方面发挥着不可或缺的作用，从而使社会的许多基础设施方面（如电网和全球定位系统）做好了急需的技术准备。我们的机器学习生态系统为专家的指尖带来了对细丝动态行为的自动、准确和可靠的分析。这个生态系统的主要贡献是两个注释细丝的数据产品，以及四个对这些细丝进行注释（定位，识别和分割）的软件产品。这个模块化生态系统可以在未来轻松扩展，超越奖项的生命周期，因为社区预计将实现更快，更高效的模块，并取代现有的模块。在整个项目的开发过程中，我们与国家太阳观测站（NSO）的仪器/数据专家进行了磋商，以适当利用我们从全球振荡网络集团的六个地面观测站整合的观测图像和元数据，它们一起提供了一个完整的磁盘和连续的（24/7）该项目的主要重点是定位和分割称为暗条的特定太阳事件，并确定其磁场手性。也就是说，在这个项目中研究的新概念，如检测算法，增强引擎和分割损失函数，这是敏感的对象的粒度，保持不可知的事件/感兴趣的对象的类型。此外，已发布的注释细丝数据集可以为计算机视觉社区提供服务，作为目标是高精度分割对象的算法的测试平台。我们的机器学习生态系统由两个数据产品和四个软件产品组成。最大的手动注释细丝数据集合和不断增长的自动注释细丝集合是两个主要的数据产品。主要的软件产品是（1）一个增强引擎，为用户提供几乎无限的半真实细丝实例，（2）用于细丝定位，分割和分类的深度神经网络算法，（3）高精度分割损失函数（对观察到的细丝的粒度敏感），指导分割任务，以及（4）一个可部署的探测模块，它可以真实的执行定位、分割和分类任务。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Since object-detection algorithms outperformed humans, one decade has passed. During this period, the unprecedented achievements of the Computer Vision domain made many believe that object detection is a solved problem. However, when it comes to scientific imagery such as microscopic, telescopic, aerial, satellite, and medical images, the general-purpose object-detection algorithms are far from perfect. A pixel-precise segmentation of objects and identification of their physical properties based on their texture features are still outstanding challenges in many interdisciplinary areas of research. Space Weather is one such area. Extreme space-weather events, similar to extreme terrestrial events, can have drastic economic and collateral impacts on mankind. Continuous and automatic monitoring of solar filaments plays an integral role in achieving reliable space-weather forecast/prediction systems, which consequently results in the technical preparedness much needed in many infrastructural aspects of the society, such as the power grid and the GPS systems. Our Machine Learning Ecosystem brings automatic, accurate, and reliable analyses of filaments’ dynamic behavior to the experts’ fingertips. The main contributions of this ecosystem are two data products of annotated filaments, and four software products which carry out the annotation (localization, identification, and segmentation) of these filaments. This modular ecosystem can be easily expanded in the future, beyond the lifetime of the award, as faster and more efficient modules are expected to be implemented by the community and replace the existing ones. Throughout the development of this project, we consult with the instrument/data experts from the National Solar Observatory (NSO) for proper utilization of the observation images and metadata we integrate from the six ground-based observatories of the Global Oscillation Network Group, that together provide a full-disk and continuous (24/7) coverage of the Sun.The primary focus of this project is on the localization and segmentation of a specific solar event, called a filament, and the identification of its magnetic field chirality. That said, the novel concepts investigated in this project, such as the detection algorithm, the augmentation engine, and the segmentation loss function which is sensitive to granularities of objects, remain agnostic to the type of the event/object of interest. Moreover, the released datasets of annotated filaments can serve the Computer Vision community as a testbed for algorithms that aim at high-precision segmentation of objects. Our Machine Learning Ecosystem consists of two data products and four software products. The largest collection of manually annotated filaments data, and a continuously-growing collection of automatically annotated filaments are the two main data products. The main software products are (1) an augmentation engine that provides users with practically unlimited semi-real filament instances, (2) a deep neural network algorithm for localization, segmentation, and classification of filaments, (3) a high-precision segmentation loss function (sensitive to granularities of the observed filaments) that guides the segmentation task, and (4) a deployable detection module which carries out the localization, segmentation, and classification tasks in real time.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.