SGER: Re-purposing Web Content through End-User Programming

今天有大量的Web内容可用，但它们并不总是以支持最终用户需求的形式出现。例如，很容易找到波特兰的酒店列表(假设用户指定了预期的州，例如俄勒冈州与缅因州、德克萨斯州或康涅狄格州等)，但要根据到波特兰会议中心的距离对它们进行排序就不那么容易了。实现这一目标所需的所有数据和服务已经存在，但它们的形式不适合这项任务。一个快速增长的开发人员社区正在通过创建以新方式结合现有Web内容和服务的“mashup”来解决这个问题。然而，创建混搭需要高水平的编程专业知识。该提案概述了Marmite的开发，该工具将使日常终端用户能够轻松地从网页中提取内容，以数据流的方式处理它，将其与其他数据源集成，并将其定向到各种有用的接收器，例如数据库、地图服务和可编译的源代码，从而进一步定制。该探索性项目侧重于三个高风险问题：(1)使选择要爬行的内容变得容易，(2)开发混合的数据流/电子表格用户界面(UI)，显示提取了哪些内容以及内容是如何转换的，以及(3)开发用于处理数据流中的异常的技术。这项研究的成功将导致一个工具，它将允许普通网络用户创建mashup，潜在地刺激许多新类型服务的创建。预计Marmite将适用于许多基于Web的场景，并会引起初创公司和现有Mashup开发人员的兴趣。该项目计划每年吸引25名本科生和研究生参加由国际和平研究所教授的课程，并有可能通过卡内基梅隆大学人机交互研究所和CyLab的企业合作伙伴进行技术转让。项目网址(http://www.cs.cmu.edu/~jasonh/projects/marmite/)将用于传播更多信息和成果。

There is a tremendous amount of web content available today, but it is not always in a form that supports end-users' needs. For example, it is easy to find a list of hotels in Portland (provided a user specifies the intended state, e.g., Oregon vs. Maine or Texas or Connecticut, etc.), but not so easy to sort them by distance to the Portland convention center. All of the data and services needed to accomplish this goal already exist, but they are not in a form amenable to this task.A rapidly growing community of developers is addressing this problem by creating "mashups" that combine existing web content and services in new ways. However, creating a mashup takes a high level of programming expertise. This proposal outlines the development of Marmite, a tool that will let everyday end-users create mashups by making it easy to extract content from web pages, process it in a data-flow manner, integrate it with other data sources, and direct it to a variety of useful sinks, such as databases, map services, and compilable source code that can be further customized.This exploratory project focuses on three high-risk issues: (1) making it easy to select what content to crawl, (2) developing a hybrid dataflow / spreadsheet user interface (UI) that shows what content has been extracted and how that content is transformed, and (3) developing techniques for handling exceptions in the dataflow. Success in this research will result in a tool that will let average web users create mashups, potentially stimulating the creation of many new kinds of services. It is expected that Marmite will be applicable across many web-based scenarios and to be of interest to startups and existing developers of mashups. This project plans to enlist the annual participation of 25 undergrad and graduate students through courses taught by the PI, and has the potential for technology transfer through corporate partners at Carnegie Mellon University's Human Computer Interaction Institute and CyLab. The project Web site (http://www.cs.cmu.edu/~jasonh/projects/marmite/) will be used for disseminating additional information and results.