Towards Predicting Socio-economic Systems by Mining Social Media Data

社交媒体数据被定义为在社交上下文中或在社交交互期间生成的任何数据。社交媒体用户每天都会产生大量的数据。例如，每三天就有10亿条推文被转发。这些数据是了解社会趋势、情绪、观点和意图的宝贵来源。拟议研究的目标是研究如何建立模型来挖掘社交媒体数据，以了解主要的社会趋势和模式，并设计有效的工具来预测复杂的社会经济系统。与其他数据挖掘问题类似，许多任务都可以在社会数据挖掘的保护伞下实现，包括分类、聚类、推荐和预测。具体的例子包括基于社交媒体合作者和用户对产品、话题或内容的情绪对社交媒体合作者和用户进行分类、社交网络中的社交角色预测、从社交网络中提取交易网络、对社交网络中的关系类型进行分类(例如个人与职业)、社交链接预测和推荐、新闻话题的趋势预测、预测社区中的下一个趋势话题、以及基于从社交媒体采样的集体情绪和情绪来预测股市表现。挖掘社交媒体数据有其自身的挑战。社交数据非常嘈杂。这意味着，在社交媒体数据中，信噪比非常低。一个著名的例子是由名人主导的推特。此外，大多数推文都是关于用户的日常、多余和不重要的活动。社交数据通常是暂时的，也就是我们所说的大数据。其他挑战包括隐私问题和社交媒体的可信度。在这项研究中，将解决其中的一些问题。挖掘社交媒体数据包括三个主要组成部分：数据收集和预处理、分析和呈现。在数据收集中，使用社交网站提供的API收集社交数据。还应用了一些预处理任务，例如文本处理(只要我们在处理内容)、噪声去除和匿名化以保护用户隐私。分析组件包括广泛的机器学习和数据挖掘任务，如分类、聚类、推荐和预测。一个众所周知的例子是在给定当前社交网络拓扑的情况下预测未来的社交链接(谁将在未来连接到谁)。它解决了众所周知的链接预测问题。第三个组件使用可视化技术显示分析结果。在这项研究中，主要重点是为主要针对社会经济问题的分析部分开发方法，如预测股市表现、社会政治危机和公共卫生风险。对于其他两个组件，我们将使用可用的工具。挖掘社交媒体数据具有广泛的应用，从营销到社会科学和健康科学，再到政治。让我们渐进地假设Twitter是一个非常大的社交传感器。虽然噪音非常大，但通过适当的噪音过滤，我们能够理解非常重要的社会趋势，如客户意图、政治观点和社会不当行为模式。建议研究的其他潜在应用包括：欺骗检测、用于个性化的用户特征分析、股市趋势预测、情绪分析、个人对个人推荐、社区检测、影响力和声誉分析、社会角色预测(谁在做什么)、社会链接分类、检测网络滥用以及垃圾用户检测。

Social media data is defined as any data generated either in a social context or during a social interaction. Large amount of data is generated everyday by social media users. For example, one billion tweets are twitted every three days. This data is a valuable source to understand social trends, sentiments, opinions, and intentions. The objective in the proposed research is to study how to build models for mining social media data to understand major social trends and patterns and design effective tools to predict complex socio-economic systems. Similar to other data mining problems, many tasks can be implemented under social data mining umbrella including classification, clustering, recommendation, and prediction. Specific examples include classifying social media collaborators and users based on their sentiments toward a product, topic, or content, social role prediction in social networks, extracting transaction network from social network, classifying the type of relationship in social networks (for example personal vs. professional), social link prediction and recommendation, trend prediction for news topics, predicting next trending topic in a community, and predicting stock market performance based on collective mood and sentiment sampled from social media. Mining social media data has own challenges. Social data is very noisy. It means, in social media data, the signal to noise ratio is very low. A well-known example is Twitter which is dominated by celebrities. Also the majority of tweets are about daily, redundant, and non-important activity of the users. Social data is usually temporal and also known as what we call it big data. Other challenges are privacy issues and credibility of social media. In this research, some of these issues will be addressed. Mining social media data comprises three main components: data collection and pre-processing, analytics, and presentation. In data collection, social data is collected using APIs provided by social sites. Some pre-processing tasks are also applied such as text processing (as long as we are dealing with content), noise removal, and anonymization to protect user privacy. Analytics component includes a wide range of machine learning and data mining tasks such as classification, clustering, recommendation, and prediction. One well-known example is to predict future social links (who will become connected to whom in future) given current social network topology. It addresses the well-known problem of link prediction. The third component presents the analytics result using visualization techniques. In this research, the main focus is to develop methods for the analytics component mainly for socio-economic problems such as predicting stock market performance, social-political crisis, and public health risks. For the two other components, we will employ available tools. Mining social media data has a wide range of applications from marketing to social and health sciences to politics. Let's asymptotically assume Twitter as a very large social sensor. Although it is very noisy, by appropriate noise filtering, we are able to understand very important social trends such as customer intentions, political opinions, and patterns of social miss-conduct. Other potential applications of the proposed research include: deception detection, user profiling for personalization, stock market trend prediction, sentiment analysis, person to person recommendation, community detection, and influence and reputation analysis, social role prediction (who is doing what), social link classification, detecting network abuse, and spam-user detection.