Using deep learning and street-level imagery to predict social and political attributes

点击翻译按钮获取中文摘要

In the last five years, one of the most interesting applications of machine learning has been in the area of affective computing. This domain of applied machine learning has used computer vision techniques to predict how humans feel about images they see. When applied to street- level and satellite imagery, affective computing gives us a method to try to understand how the built environment makes us feel or act. As Owen notes in his statement, these visual cues have been used extensively to build "feel-of-a-street" models that predict social and economic outcomes for a given place based on how the place looks from the street. Owen discusses a few of these in his literature review, such as Gebru et al. (2017)'s predictions of the 2008 presidential election based on the types of street-parked cars. In geography, the classification of Multidimensional Open Data of Urban Morphology (MODUM) produced by Alexiou et al. (2016) aimed for a general-purpose classification based on data about urban structure and morphology. Owen's dissertation aims to do this through the look and feel of a place, constructed from street and satellite-level imagery.This is not just a parlor trick: the ability to predict political information using street-level imagery allows us to really get a feel for how physical context matters for social processes, as well as opening the door into "now-casting" important social and political traits. In this sense, street-level computer vision models have truly expanded the reach of social science. However, further work is needed to determine whether these predictions are reliable. This is the work Owen proposes.