Skip to Main content Skip to Navigation
Journal articles

Predicting Intentions of Pedestrians from 2D Skeletal Pose Sequences with a Representation-Focused Multi-Branch Deep Learning Network

Abstract : Understanding the behaviors and intentions of humans is still one of the main challenges for vehicle autonomy. More specifically, inferring the intentions and actions of vulnerable actors, namely pedestrians, in complex situations such as urban traffic scenes remains a difficult task and a blocking point towards more automated vehicles. Answering the question "Is the pedestrian going to cross?" is a good starting point in order to advance in the quest to the fifth level of autonomous driving. In this paper, we address the problem of real-time discrete intention prediction of pedestrians in urban traffic environments by linking the dynamics of a pedestrian's skeleton to an intention. Hence, we propose SPI-Net (Skeleton-based Pedestrian Intention network): a representation-focused multi-branch network combining features from 2D pedestrian body poses for the prediction of pedestrians' discrete intentions. Experimental results show that SPI-Net achieved 94.4% accuracy in pedestrian crossing prediction on the JAAD data set while being efficient for real-time scenarios since SPI-Net can reach around one inference every 0.25 ms on one GPU (i.e., RTX 2080ti), or every 0.67 ms on one CPU (i.e., Intel Core i7 8700K).
Complete list of metadata

https://hal-mines-paristech.archives-ouvertes.fr/hal-03059635
Contributor : Fabien Moutarde <>
Submitted on : Saturday, December 12, 2020 - 9:22:51 PM
Last modification on : Monday, July 26, 2021 - 3:34:03 PM
Long-term archiving on: : Saturday, March 13, 2021 - 7:35:44 PM

File

algorithms-13-00331-v2.pdf
Publisher files allowed on an open archive

Identifiers

Citation

Joseph Gesnouin, Steve Pechberti, Guillaume Bresson, Bogdan Stanciulescu, Fabien Moutarde. Predicting Intentions of Pedestrians from 2D Skeletal Pose Sequences with a Representation-Focused Multi-Branch Deep Learning Network. Algorithms, MDPI, 2020, 13 (12), pp.331. ⟨10.3390/a13120331⟩. ⟨hal-03059635⟩

Share

Metrics

Record views

246

Files downloads

161