Algorithms and Tools for Big Data Analysis and Automated Real Time Optimal or Near Optimal Decision Making for Industrial Systems

数据科学和数据工程可以说是本世纪最重要的研究领域之一。这些领域基于科学和工程的基本分支，即信息技术、传感器技术、统计学、运筹学、最优化、人工智能、数据挖掘和机器学习。在以人为中心的应用中，其中一些技术现在正被研究人员推荐用于以机器为中心的应用，在这些应用中，数据由机器制造，决策也由机器基于机器对机器(M2M)的学习做出。目前，一个重要的研究问题是如何利用可用的大数据集，因为根据定义，它们包含大量高速获取的各种形式的数据。传统的数据处理和分析技术已经不能满足需要。该提案的目标是开发专门设计的算法和工具，用于分析和提取来自工业系统的大数据中的知识。该方法基于一种称为数据逻辑分析(LAD)的方法，这是一种基于布尔逻辑推理的数据挖掘和机器学习方法。它以模式的形式提取知识，这些模式区分和描述数据集，并识别一些感兴趣的现象。在并行计算框架中将考虑用于在监督和非监督学习中提取模式的不同LAD‘S算法；即枚举技术、混合整数线性规划和元启发式算法，主要是遗传算法和蚁群。将使用的两个并行框架是Hadoop MapReduce和Spark；这两个框架都可以在开源环境中使用，因此它们可以向公众提供。我们打算向科学界展示在开源环境中放大的算法。因此，每个感兴趣的人都可以使用它们，改进它们，并对它们进行补充。这项研究的影响是学习、发现、理解尚未完全理解的物理复杂现象的可能性，并在决策中利用这些知识。根据这些算法将用于的具体应用，这些知识可以提高安全性和安全性，节省能源，保护环境，并提高消耗自然资源的效率。它还将导致智能系统，能够在正确的时间做出正确的决定。最终，这将导致自我维持和可持续的系统。

Data science and data engineering have arguably become some of the most important research fields in this century. These fields are based on fundamental branches of science and engineering, namely, information technology, sensor technology, statistics, operations research, optimization, artificial intelligence, data mining and machine learning.Along with human centric applications, some of these techniques are now being recommended by researchers in machine centric applications in which data is manufactured by machines and decisions are also made by machines based on Machine to Machine (M2M)' learning.Presently, an important research question is how to exploit the available Big Data sets since, by definition, they consist of large volumes of data, acquired at high velocity, and in a variety of forms. Traditional data-processing and analysis techniques become inadequate.The objective of this proposal is to develop algorithms and tools that are designed specifically to analyze and to extract knowledge from Big Data that are obtained from industrial systems. The extracted knowledge should lead to an understanding of how various components of a complex system influence each other and interact with their environment, and how an accurate prediction of the degradation can be obtained in a parallel computing framework.The proposed methodology is based on an approach called Logical Analysis of Data (LAD), which is a data mining, machine learning approach that is based on Boolean logical reasoning. It extracts knowledge in the form of patterns that distinguish and characterize sets of data, and that identify some phenomena of interest. Different LAD' s algorithms that are used to extract patterns in supervised and unsupervised learning will be considered in parallel computing frameworks; namely, enumeration techniques, mixed integer linear programming, and metaheuristics algorithms, mainly genetic algorithms, and ant colonies. The two parallel frameworks that will be used are Hadoop MapReduce and Spark; both are available in an open source environment, thus they are available to the public.We intend to present to the scientific community scaled up algorithms in an open source environment. As such, every interested individual can use them, improve upon them and add to them. The impact of this research is the possibility of learning, finding, understanding physical complex phenomena that are not fully understood yet, and the exploitation of this knowledge in decision making. Depending on the specific applications in which these algorithms will be used, this knowledge can lead to an increase in safety and security, energy savings, protection of the environment, and increased efficiency in consuming natural resources. It will also lead to intelligent systems that can make the right decision at the right moment. Eventually, this will lead to self-sustaining and sustainable systems.