SBIR Phase I: Rething Recommendations

这个小型企业创新研究(SBIR)第一阶段项目解决了使用关于任何单个个体的行为的稀疏信息来学习个体选择行为的预测模型的问题。该项目的智力价值在于，通过将选择行为建模为对备选方案的排列的分布，并使该观点可在规模上实现，从而为这个问题开发了一种新的简约观。这一范例中的数据单位是两个备选方案之间的单一比较。这类数据可以在从产品评论到交易数据的各种上下文中派生出来。虽然是孤立的建模观点，但精确的计算，甚至表示这样的模型是困难的。该项目将侧重于开发近似解决方案，本着高维统计最新进展的精神，开发对此类模型进行稀疏近似的潜力。考虑到可用于建立这种模型的大量数据，重要的是所开发的算法能够以一种让人想起Map/Reduce计算范例的方式进行并行化。开发的算法将符合这一范例，关键的算法步骤将分解为针对单个个人收集的数据。总而言之，这个项目将开发一种大规模并行的方法，使用来自各种来源的非结构化数据来建模个人选择行为。该项目的更广泛的影响/商业潜力在于使新兴的、无处不在的从“搜索”到“发现”的转变成为可能。这种转变可以在从电子商务到线下零售再到匹配需求侧平台上的广告商印象的各个行业都可以看到。这种转变中的关键绊脚石似乎是需要为给定的上下文构建丰富的属性模型，而不是使用黑盒方法。在这个项目中采取的方法属于后一种。作为一个具体的例子，商品销售的任务要求线下零售商决定合适的产品类别来销售从牙膏到服装的各个细分市场；这里的方法将以完全数据驱动的方式支持这样的决策。在不同的方向上，基于模型的服务广告可以捕捉冲浪者在网络上各种产品和主题的偏好，使用这里的方法可以实现大规模和令人难以置信的粒度。这里的方法所实现的粒度级别不能用“参数”属性驱动的方法来实现。总而言之，在这个项目中开发的工具有可能像PageRank算法在搜索中所做的那样来进行“发现”。

This Small Business Innovation Research (SBIR) Phase I project addresses theproblem of learning predictive models of individual choice behavior using sparseinformation on the behavior of any single individual. The intellectual merit of theproject is developing a novel parsimonious view of this problem by modelingchoice behavior as a distribution over permutations of alternatives, and makingthis view implementable at scale. A unit of data in this paradigm is a singlecomparison between two alternatives. Data of this sort can be derived in a varietyof contexts ranging from product reviews to transaction data. While being aparsimonious modeling viewpoint, exact computation, or even representing suchmodels is intractable. The project will focus on developing approximate solutionsthat, in the spirit of recent advances in high-dimensional statistics, exploit thepotential of sparse approximations to such models. Given the vast quantities ofdata available to build such models it will be important for the algorithmsdeveloped to be amenable to parallelization in a manner reminiscent of theMap/Reduce computational paradigm. The algorithms developed will fit thisparadigm with key algorithmic steps decomposing across data collected for asingle individual. In summary, this project will develop a massively parallelizableapproach to modeling individual choice behavior using unstructured data from avariety of sources.The broader impact/commercial potential of this project rests in enabling theemerging, all pervasive transition from 'search' to 'discovery'. This transition canbe witnessed in sectors ranging from e-commerce to offline retail to matchingimpressions to advertisers on demand side platforms. The key stumbling block inthis transition is the seeming requirement to build attribute rich models for a givencontext as opposed to a black box approach. The approach taken in this projectis of the latter variety. As a concrete example, the task of merchandising requiresan offline retailer to decide on the right assortment of products to carry insegments ranging from tooth paste to clothing; the approach here will power suchdecision making in an entirely data driven fashion. In a different direction, servingads based on models that capture a surfer's preferences across the various silosof products and topics on the web can be enabled at scale and incrediblegranularity using the approach here. The level of granularity made possible bythe approach here cannot be achieved with 'parametric' attribute drivenapproaches. In summary, the tools developed in this project have the potential todo for `discovery' what the PageRank algorithm did for search.