Mining and parsing information from e-commerce websites

点击翻译按钮获取中文摘要

There are many e-commerce websites on the market, but none of them use the same format. Traditionally, it has been very difficult to get the information from each of the popular e-commerce vendors and parse that information to a usable format. It remains a challenge to analyze the price change by time in the same ecommerce website or compare the price of the same products in different e-commerce websites. In addition, for many e-commerce websites key information is protected using image, which does not have any impact from user views. However, web crawler cannot parse image. This project is proposing to develop an algorithm which takes in a URL of a product page from any e-commerce website and is able to parse common information and recognize the information protected by image. The algorithm should analyse the pages and pick out the information and strip all the unnecessary information. The innovation of the proposed research lies in two perspectives. Firstly, this project will propose the optimal feature extraction and classification methods for image recognition in order to improve the image recognition accuracy and computational efficiency. The proposed method will have greatly enhanced recognition power and accuracy and exhibit a great modeling capability in handling image data. Secondly, aimed at Procurify's specific situation (image data), a practical solution will be provided which includes page analysis, information parsing algorithm, image data extraction and data storage. Time efficiency is often considered.