The Atlas Benchmark: an Automated Evaluation Framework for Human Motion PredictionShow others and affiliations
2022 (English)In: 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), IEEE , 2022, p. 636-643Conference paper, Published paper (Refereed)
Abstract [en]
Human motion trajectory prediction, an essential task for autonomous systems in many domains, has been on the rise in recent years. With a multitude of new methods proposed by different communities, the lack of standardized benchmarks and objective comparisons is increasingly becoming a major limitation to assess progress and guide further research. Existing benchmarks are limited in their scope and flexibility to conduct relevant experiments and to account for contextual cues of agents and environments. In this paper we present Atlas, a benchmark to systematically evaluate human motion trajectory prediction algorithms in a unified framework. Atlas offers data preprocessing functions, hyperparameter optimization, comes with popular datasets and has the flexibility to setup and conduct underexplored yet relevant experiments to analyze a method's accuracy and robustness. In an example application of Atlas, we compare five popular model- and learning-based predictors and find that, when properly applied, early physics-based approaches are still remarkably competitive. Such results confirm the necessity of benchmarks like Atlas.
Place, publisher, year, edition, pages
IEEE , 2022. p. 636-643
Series
IEEE RO-MAN proceedings, ISSN 1944-9445, E-ISSN 1944-9437
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
URN: urn:nbn:se:oru:diva-102683DOI: 10.1109/RO-MAN53752.2022.9900656ISI: 000885903300093Scopus ID: 2-s2.0-85138517567ISBN: 9781728188591 (electronic)ISBN: 9781665406802 (print)OAI: oai:DiVA.org:oru-102683DiVA, id: diva2:1718542
Conference
31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) - Social, Asocial, and Antisocial Robots, Napoli, Italy, August 29 - September 2, 2022
Funder
EU, Horizon 2020, 1010172742022-12-132022-12-132022-12-13Bibliographically approved