Anticipating Human Motion and Activities (P3)

该项目的目标是预测不同粒度的人类行为，可以是一个人的未来活动，也可以是详细的人类运动。与其单独预测两个粒度，不如联合预测它们。首先，如果人类的意图是未知的，则只能在非常短的几秒钟的时间范围内预测详细的人类运动。我们假设，通过预测活动，从而预测人的意图，也可以预测更长时间范围内的详细人体运动。其次，详细的人体运动提供了更好的可视化和预测活动的解释。该模型不仅预测活动，还预测将执行活动的人体运动。我们将特别关注不确定性的建模。事实上，未来是模糊的，未来可能会发生多种情况。例如，给定一个视频片段，显示一个未知的人从橱柜里拿了一个杯子，我们不能确定这个人是会泡茶还是咖啡。因此，我们的目标是预测多种活动序列和人类运动，反映未来可能出现的各种情况。然而，如果有更多的信息，未来的不确定性可以减少。例如，如果我们知道这个人喜欢茶而不是咖啡，那么这个人更有可能泡茶。即使这个人以前没有被观察过，国家、日期、时间或当前物体等外部因素也会提供额外的背景信息，从而降低预测的不确定性。虽然咖啡在德国的早餐中非常受欢迎，但在英国，茶是首选，如果我们知道没有咖啡，甚至可以排除煮咖啡。为了减少预测的不确定性，因此，我们的目标是模拟外部因素和条件的模型，用于预测这些外部因素的人体运动和活动。

The goal of the project is to anticipate human behavior at different granularities, which can be the future activities of a person, but also detailed human motion. Instead of forecasting both granularities independently, it is beneficial to anticipate them jointly. First, detailed human motion can only be forecast for a short time horizon of very few seconds if the intention of the human is unknown. We hypothesize that by anticipating activities, and thus the intention of the person, it will be possible to forecast detailed human motion also for longer time horizons. Second, detailed human motion provides a better visualization and interpretation of the forecast activities. Instead of just having a forecast activity, the model also forecasts the human motion that will be performed to execute the activity. We will in particular focus on modeling the uncertainty. In fact, the future is ambiguous and multiple scenarios can happen in the future. For example, given a video snippet that shows an unknown person taking a cup from the cupboard, we cannot be sure if the person will make tea or coffee. Our goal is therefore to forecast multiple sequences of activities and human motion that reflect the diversity of possible future scenarios. The uncertainty of the future, however, can be reduced if additional information is available. For instance, if we know that the person prefers tea over coffee it is more likely that the person makes tea. Even if the person has not been observed before, external factors like the country, date, time, or the present objects provide additional background information that reduces the uncertainty in the prediction. While coffee is very popular for breakfast in Germany, tea is preferred in the UK and if we know that no coffee is available, making coffee can even be excluded. In order to reduce the forecast uncertainty, we therefore aim to model external factors and condition the models for anticipating human motion and activities on these external factors.