Data analytics for click stream logs

Pelmore x Media Inc.是一家专注于天气和旅行者相关内容的多媒体公司 技术旗舰品牌包括The Weather Network和法国同行Météomédia，加拿大最受欢迎的天气信息网站和应用程序，以及收视率最高的专业电视网络和顶级商业天气服务。Pelmore x电视网络覆盖了1100多万个加拿大家庭以及99%的加拿大有线电视和卫星电视用户，使其成为加拿大分布最广泛的电视服务之一。 这个项目的重点是Pelmore产品的在线组件。具体地说，它的目标是为公司提供对客户在线行为和商业价值的新见解。这一目标是通过利用客户交易的每日日志和Pelmore可获得的其他信息，如天气和旅行数据来实现的。Pelmore在线提供的服务对消费者是免费的；收入完全来自广告。尽管在线服务一直很受欢迎，每年的页面浏览量超过35亿次，但该公司正在通过利用每日日志来寻找进一步改进的机会。对这些日志的分析将1)通过更好地了解客户的行为来提高内容的有效性，2)通过识别和区分优先级来提高所提供产品的质量 增强，以及3)通过有针对性的广告提高销售效率。为了实现这些目标，将分析每日日志，以产生对用户的洞察，发现他们之间的模式和相似之处，确定他们的行为模式，并以适合定向广告的方式使用Pelmore特定的细分用户。 由于游客在Pelmore物业上的流量很大，他们交易的每日日志大小成比例地庞大；这些大数据给数据处理和分析带来了挑战。此外，传统的机器学习方法是为较小的数据集设计的，需要适应大数据。近年来，人们对大数据产生了广泛的兴趣，导致了大量新技术的发展，如Hadoop、MapReduce、NoSQL、Pig、Have、ApacheMahout和Spark。大数据和新技术的多样性将给这个需要调查的项目带来重大挑战。

Pelmorex Media Inc. is a multi-media company specializing in weather and traveler-related content and technology. Flagship brands include The Weather Network and its French counterpart MétéoMédia, Canada's most popular weather information Web sites and applications, as well as the most frequently watched specialty television networks and top commercial weather services. Pelmorex television networks are in more than 11 million Canadian homes and in 99% of the Canadian cable and satellite subscribers, making it among the most widely distributed television services in Canada. This project focuses on the online component of Pelmorex offerings. Specifically, its objective is to provide new insights into customers online behaviours and business value for the company. This objective is to be accomplished by taking advantage of daily logs of customers transactions and other information available to Pelmorex such as weather and travel data. The services that Pelmorex provides online are free for consumers; the revenue is generated solely through advertising. Although the online services have been popular, over 3.5 billion page views per year, the company is seeking opportunities for further improvement by taking advantage of the daily logs. The analysis of such logs will 1) improve content effectiveness by better understanding customers' behaviour, 2) improve the quality of the provided products by identifying and prioritizing enhancements, and 3) increase sale effectiveness through targeted advertising. To achieve such objectives, the daily logs will be analyzed to generate insights about users, discover patterns and similarities among them, identify patterns of their behaviour, and segment users using Pelmorex-specific segmentation in a way suitable for targeted advertising. As visitors' traffic on the Pelmorex properties is intense, the size of daily logs of their transactions is proportionally massive; this Big Data poses challenges for data processing and analysis. Moreover, traditional machine learning approaches were designed for smaller data sets and require adaptation for Big Data. In recent years there has been extensive interest in Big Data which resulted in the development of a large number of new technologies such as Hadoop, MapReduce, NoSQL, Pig, Hive, Apache Mahout, and Spark. Big Data together with the diversity of new technologies will present major challenges for this project that requires investigation.