Towards Predicting Socio-economic Systems by Mining Social Media Data

点击翻译按钮获取中文摘要

Social media data is defined as any data generated either in a social context or during a social interaction. Large amount of data is generated everyday by social media users. For example, one billion tweets are twitted every three days. This data is a valuable source to understand social trends, sentiments, opinions, and intentions. The objective in the proposed research is to study how to build models for mining social media data to understand major social trends and patterns and design effective tools to predict complex socio-economic systems. Similar to other data mining problems, many tasks can be implemented under social data mining umbrella including classification, clustering, recommendation, and prediction. Specific examples include classifying social media collaborators and users based on their sentiments toward a product, topic, or content, social role prediction in social networks, extracting transaction network from social network, classifying the type of relationship in social networks (for example personal vs. professional), social link prediction and recommendation, trend prediction for news topics, predicting next trending topic in a community, and predicting stock market performance based on collective mood and sentiment sampled from social media. Mining social media data has own challenges. Social data is very noisy. It means, in social media data, the signal to noise ratio is very low. A well-known example is Twitter which is dominated by celebrities. Also the majority of tweets are about daily, redundant, and non-important activity of the users. Social data is usually temporal and also known as what we call it big data. Other challenges are privacy issues and credibility of social media. In this research, some of these issues will be addressed. Mining social media data comprises three main components: data collection and pre-processing, analytics, and presentation. In data collection, social data is collected using APIs provided by social sites. Some pre-processing tasks are also applied such as text processing (as long as we are dealing with content), noise removal, and anonymization to protect user privacy. Analytics component includes a wide range of machine learning and data mining tasks such as classification, clustering, recommendation, and prediction. One well-known example is to predict future social links (who will become connected to whom in future) given current social network topology. It addresses the well-known problem of link prediction. The third component presents the analytics result using visualization techniques. In this research, the main focus is to develop methods for the analytics component mainly for socio-economic problems such as predicting stock market performance, social-political crisis, and public health risks. For the two other components, we will employ available tools. Mining social media data has a wide range of applications from marketing to social and health sciences to politics. Let's asymptotically assume Twitter as a very large social sensor. Although it is very noisy, by appropriate noise filtering, we are able to understand very important social trends such as customer intentions, political opinions, and patterns of social miss-conduct. Other potential applications of the proposed research include: deception detection, user profiling for personalization, stock market trend prediction, sentiment analysis, person to person recommendation, community detection, and influence and reputation analysis, social role prediction (who is doing what), social link classification, detecting network abuse, and spam-user detection.