Mining and parsing information from e-commerce websites

市场上有许多电子商务网站，但没有一个使用相同的格式。传统上，从每个流行的电子商务供应商获得信息并将该信息解析为可用的格式是非常困难的。分析同一电子商务网站中随时间的价格变化或比较不同电子商务网站中相同产品的价格仍然是一个挑战。此外，对于许多电子商务网站来说，关键信息是使用图像保护的，这不会对用户视图产生任何影响。然而，网络爬虫不能解析图像。该项目建议开发一种算法，该算法从任何电子商务网站获取产品页面的URL，并且能够解析常见信息并识别受图像保护的信息。算法应该分析页面，挑出信息，去掉所有不必要的信息。本研究的创新之处在于两个方面。首先，本计画将提出最佳的影像辨识特徴撷取与分类方法，以提升影像辨识的准确度与运算效率。所提出的方法将大大提高识别能力和准确性，并表现出很大的建模能力，在处理图像数据。其次，针对Procurify的具体情况（图像数据），给出了一个实用的解决方案，包括页面分析、信息解析算法、图像数据提取和数据存储。时间效率是经常考虑的问题。

There are many e-commerce websites on the market, but none of them use the same format. Traditionally, it has been very difficult to get the information from each of the popular e-commerce vendors and parse that information to a usable format. It remains a challenge to analyze the price change by time in the same ecommerce website or compare the price of the same products in different e-commerce websites. In addition, for many e-commerce websites key information is protected using image, which does not have any impact from user views. However, web crawler cannot parse image. This project is proposing to develop an algorithm which takes in a URL of a product page from any e-commerce website and is able to parse common information and recognize the information protected by image. The algorithm should analyse the pages and pick out the information and strip all the unnecessary information. The innovation of the proposed research lies in two perspectives. Firstly, this project will propose the optimal feature extraction and classification methods for image recognition in order to improve the image recognition accuracy and computational efficiency. The proposed method will have greatly enhanced recognition power and accuracy and exhibit a great modeling capability in handling image data. Secondly, aimed at Procurify's specific situation (image data), a practical solution will be provided which includes page analysis, information parsing algorithm, image data extraction and data storage. Time efficiency is often considered.