SBIR Phase I: Rething Recommendations

这个小企业创新研究（SBIR）第一阶段项目解决了学习个人选择行为的预测模型的问题，使用任何单个个体行为的稀疏信息。该项目的智力价值是通过将选择行为建模为替代品排列的分布，并使这种观点在规模上可实现，来开发这个问题的新的简约观点。在这种范例中，数据单元是两个备选方案之间的一个实体。这类数据可以在从产品评论到交易数据的各种环境中获得。但从模型的角度来看，精确的计算甚至表示都是困难的。该项目将侧重于开发近似的解决方案，在高维统计的最新进展的精神，利用稀疏近似的潜力，这样的模型。鉴于大量的数据可用于建立这样的模型，这将是重要的算法开发是服从并行化的方式让人想起地图/减少计算范式.开发的算法将适合这种范式，关键算法步骤分解为单个个体收集的数据。总之，这个项目将开发一个大规模并行的方法，使用来自各种来源的非结构化数据来建模个人选择行为。这个项目更广泛的影响/商业潜力在于实现从“搜索”到“发现”的新兴的、普遍的过渡。从电子商务到线下零售，再到在需求方平台上为广告商匹配展示，都可以看到这种转变。在这个过渡中的关键绊脚石是似乎需要为给定的上下文构建属性丰富的模型，而不是黑箱方法。本项目所采取的方法属于后一种。作为一个具体的例子，商品销售的任务要求一个离线零售商决定正确的产品分类，从牙膏到服装;这里的方法将以完全数据驱动的方式为这样的决策提供动力。在另一个方向上，基于模型的服务广告可以在网络上的各种产品和主题中捕捉冲浪者的偏好，可以使用这里的方法进行大规模和无规则的启用。这里的方法所能达到的粒度级别是"参数化"属性驱动方法所不能达到的。总之，这个项目开发的工具有潜力“发现”PageRank算法对搜索的作用。

This Small Business Innovation Research (SBIR) Phase I project addresses theproblem of learning predictive models of individual choice behavior using sparseinformation on the behavior of any single individual. The intellectual merit of theproject is developing a novel parsimonious view of this problem by modelingchoice behavior as a distribution over permutations of alternatives, and makingthis view implementable at scale. A unit of data in this paradigm is a singlecomparison between two alternatives. Data of this sort can be derived in a varietyof contexts ranging from product reviews to transaction data. While being aparsimonious modeling viewpoint, exact computation, or even representing suchmodels is intractable. The project will focus on developing approximate solutionsthat, in the spirit of recent advances in high-dimensional statistics, exploit thepotential of sparse approximations to such models. Given the vast quantities ofdata available to build such models it will be important for the algorithmsdeveloped to be amenable to parallelization in a manner reminiscent of theMap/Reduce computational paradigm. The algorithms developed will fit thisparadigm with key algorithmic steps decomposing across data collected for asingle individual. In summary, this project will develop a massively parallelizableapproach to modeling individual choice behavior using unstructured data from avariety of sources.The broader impact/commercial potential of this project rests in enabling theemerging, all pervasive transition from 'search' to 'discovery'. This transition canbe witnessed in sectors ranging from e-commerce to offline retail to matchingimpressions to advertisers on demand side platforms. The key stumbling block inthis transition is the seeming requirement to build attribute rich models for a givencontext as opposed to a black box approach. The approach taken in this projectis of the latter variety. As a concrete example, the task of merchandising requiresan offline retailer to decide on the right assortment of products to carry insegments ranging from tooth paste to clothing; the approach here will power suchdecision making in an entirely data driven fashion. In a different direction, servingads based on models that capture a surfer's preferences across the various silosof products and topics on the web can be enabled at scale and incrediblegranularity using the approach here. The level of granularity made possible bythe approach here cannot be achieved with 'parametric' attribute drivenapproaches. In summary, the tools developed in this project have the potential todo for `discovery' what the PageRank algorithm did for search.