Data analytics for click stream logs

Pelmorex Media Inc.是一家多媒体公司，专门从事天气和旅行相关的内容， 技术.旗舰品牌包括The Weather Network及其法国同行MétéoMédia，加拿大最受欢迎的天气信息网站和应用程序，以及最常观看的专业电视网络和顶级商业天气服务。Pelmorex电视网络在加拿大超过1100万家庭和99%的加拿大有线和卫星用户中，使其成为加拿大分布最广的电视服务之一。 该项目侧重于Pelmorex产品的在线组件。具体而言，其目标是为公司提供有关客户在线行为和商业价值的新见解。这一目标将通过利用每日客户交易日志和Pelmorex可获得的其他信息（如天气和旅行数据）来实现。Pelmorex在线提供的服务对消费者是免费的;收入完全通过广告产生。虽然在线服务一直很受欢迎，每年超过35亿的页面浏览量，该公司正在寻求机会，通过利用每日日志进一步改进。对这些日志的分析将1）通过更好地了解客户的行为来提高内容的有效性，2）通过识别和优先考虑 增强，以及3）通过有针对性的广告提高销售效率。为了实现这些目标，将分析日常日志，以生成有关用户的见解，发现用户之间的模式和相似性，识别他们的行为模式，并以适合定向广告的方式使用Pelmorex特定的细分来细分用户。 由于Pelmorex物业的访客流量非常大，其交易的每日日志的大小也相应地非常大;这种大数据对数据处理和分析提出了挑战。此外，传统的机器学习方法是为较小的数据集设计的，需要适应大数据。近年来，人们对大数据产生了广泛的兴趣，这导致了大量新技术的开发，如Hadoop，MapReduce，NoSQL，Pig，Hive，Apache Mahout和Spark。大数据和新技术的多样性将为这个需要调查的项目带来重大挑战。

Pelmorex Media Inc. is a multi-media company specializing in weather and traveler-related content and technology. Flagship brands include The Weather Network and its French counterpart MétéoMédia, Canada's most popular weather information Web sites and applications, as well as the most frequently watched specialty television networks and top commercial weather services. Pelmorex television networks are in more than 11 million Canadian homes and in 99% of the Canadian cable and satellite subscribers, making it among the most widely distributed television services in Canada. This project focuses on the online component of Pelmorex offerings. Specifically, its objective is to provide new insights into customers online behaviours and business value for the company. This objective is to be accomplished by taking advantage of daily logs of customers transactions and other information available to Pelmorex such as weather and travel data. The services that Pelmorex provides online are free for consumers; the revenue is generated solely through advertising. Although the online services have been popular, over 3.5 billion page views per year, the company is seeking opportunities for further improvement by taking advantage of the daily logs. The analysis of such logs will 1) improve content effectiveness by better understanding customers' behaviour, 2) improve the quality of the provided products by identifying and prioritizing enhancements, and 3) increase sale effectiveness through targeted advertising. To achieve such objectives, the daily logs will be analyzed to generate insights about users, discover patterns and similarities among them, identify patterns of their behaviour, and segment users using Pelmorex-specific segmentation in a way suitable for targeted advertising. As visitors' traffic on the Pelmorex properties is intense, the size of daily logs of their transactions is proportionally massive; this Big Data poses challenges for data processing and analysis. Moreover, traditional machine learning approaches were designed for smaller data sets and require adaptation for Big Data. In recent years there has been extensive interest in Big Data which resulted in the development of a large number of new technologies such as Hadoop, MapReduce, NoSQL, Pig, Hive, Apache Mahout, and Spark. Big Data together with the diversity of new technologies will present major challenges for this project that requires investigation.