Mining and parsing information from e-commerce websites

电子商务网站信息的挖掘和解析

• 批准号: 461800-2013
• 负责人: Yang, Zijiang, Cynthia
• 金额: $ 1.82万
• 依托单位: York University
• 依托单位国家: 加拿大
• 项目类别: Engage Grants Program
• 财政年份: 2013
• 资助国家: 加拿大
• 起止时间: 2013-01-01 至 2014-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=542780
• 关键词: Mining; parsing; information; commerce; websites

There are many e-commerce websites on the market, but none of them use the same format. Traditionally, it has been very difficult to get the information from each of the popular e-commerce vendors and parse that information to a usable format. It remains a challenge to analyze the price change by time in the same ecommerce website or compare the price of the same products in different e-commerce websites. In addition, for many e-commerce websites key information is protected using image, which does not have any impact from user views. However, web crawler cannot parse image. This project is proposing to develop an algorithm which takes in a URL of a product page from any e-commerce website and is able to parse common information and recognize the information protected by image. The algorithm should analyse the pages and pick out the information and strip all the unnecessary information. The innovation of the proposed research lies in two perspectives. Firstly, this project will propose the optimal feature extraction and classification methods for image recognition in order to improve the image recognition accuracy and computational efficiency. The proposed method will have greatly enhanced recognition power and accuracy and exhibit a great modeling capability in handling image data. Secondly, aimed at Procurify's specific situation (image data), a practical solution will be provided which includes page analysis, information parsing algorithm, image data extraction and data storage. Time efficiency is often considered.

市场上有很多电子商务网站，但没有一个使用相同的格式。传统上，从每个流行的电子商务供应商获取信息并将该信息解析为可用的格式非常困难。分析同一电子商务网站中随时间的价格变化或比较相同产品在不同电子商务网站中的价格仍然是一个挑战。另外，很多电商网站的关键信息都是通过图片来保护的，不会对用户浏览造成任何影响。然而，网络爬虫无法解析图像。该项目提议开发一种算法，可以从任何电子商务网站获取产品页面的 URL，并能够解析常见信息并识别受图像保护的信息。该算法应该分析页面并挑选出信息并去除所有不必要的信息。本研究的创新之处在于两个方面。首先，该项目将提出图像识别的最佳特征提取和分类方法，以提高图像识别的准确性和计算效率。所提出的方法将大大增强识别能力和准确性，并在处理图像数据时表现出强大的建模能力。其次，针对Procurify的具体情况（图像数据），提供实用的解决方案，包括页面分析、信息解析算法、图像数据提取和数据存储。时间效率常常被考虑。