Data analytics for click stream logs

点击翻译按钮获取中文摘要

Pelmorex Media Inc. is a multi-media company specializing in weather and traveler-related content and technology. Flagship brands include The Weather Network and its French counterpart MétéoMédia, Canada's most popular weather information Web sites and applications, as well as the most frequently watched specialty television networks and top commercial weather services. Pelmorex television networks are in more than 11 million Canadian homes and in 99% of the Canadian cable and satellite subscribers, making it among the most widely distributed television services in Canada. This project focuses on the online component of Pelmorex offerings. Specifically, its objective is to provide new insights into customers online behaviours and business value for the company. This objective is to be accomplished by taking advantage of daily logs of customers transactions and other information available to Pelmorex such as weather and travel data. The services that Pelmorex provides online are free for consumers; the revenue is generated solely through advertising. Although the online services have been popular, over 3.5 billion page views per year, the company is seeking opportunities for further improvement by taking advantage of the daily logs. The analysis of such logs will 1) improve content effectiveness by better understanding customers' behaviour, 2) improve the quality of the provided products by identifying and prioritizing enhancements, and 3) increase sale effectiveness through targeted advertising. To achieve such objectives, the daily logs will be analyzed to generate insights about users, discover patterns and similarities among them, identify patterns of their behaviour, and segment users using Pelmorex-specific segmentation in a way suitable for targeted advertising. As visitors' traffic on the Pelmorex properties is intense, the size of daily logs of their transactions is proportionally massive; this Big Data poses challenges for data processing and analysis. Moreover, traditional machine learning approaches were designed for smaller data sets and require adaptation for Big Data. In recent years there has been extensive interest in Big Data which resulted in the development of a large number of new technologies such as Hadoop, MapReduce, NoSQL, Pig, Hive, Apache Mahout, and Spark. Big Data together with the diversity of new technologies will present major challenges for this project that requires investigation.