Socially Aware Reinforcement Learning-based Crowd Robot Navigation

This is Hafiq’s MSc project. We explore the use of Reinforcement Learning (RL), especially the Deep Reinforcement Learning (DRL) in robot navigation in crowded environment. The ultimate goal is to improve the local and global planning of the navigation system through RL targeting the performance in crowded or complex environment where existing local and global planning techniques are not efficient. In this work, we incorporated the sense of danger (collision risk) into the observation space of the agent during crowd navigation, and used waypoints to increase the density of rewards.

People

• Hafiq Anas
• Ong Wee Hong
• Owais Ahmed Malik

Data/Codes

• Deep Reinforcement Learning-Based Mapless Crowd Navigation with Perceived Risk of the Moving Crowd for Mobile Robots: https://github.com/ailabspace/drl-based-mapless-crowd-navigation-with-perceived-risk
• Comparison of DQN with Q-Learning and SARSA for Robot Local Navigation: https://github.com/ailabspace/comparison-of-dqn-with-q-learning-and-sarsa-for-robot-local-navigation

Publications

• Hafiq Anas, Wee Hong Ong, Owais Ahmed Malik, “Deep Reinforcement Learning-Based Mapless Crowd Navigation with Perceived Risk of the Moving Crowd for Mobile Robots”, arXiv preprint (2023) https://doi.org/10.48550/arXiv.2304.03593
• Hafiq Anas, Wee Hong Ong, Owais Ahmed Malik, “Deep Reinforcement Learning-Based Mapless Crowd Navigation with Perceived Risk of the Moving Crowd for Mobile Robots” presented at 2nd Workshop on Social Robot Navigation: Advances and Evaluation held in conjunction with IEEE International Conference on Intelligent Robots and Systems 2023 (IROS 2023), 5 October 2023 (pdf)
• Hafiq Anas, Wee Hong Ong, Owais Ahmed Malik, “Comparison of Deep Q-Learning, Q-Learning and SARSA Reinforced Learning for Robot Local Navigation”, the 9th International Conference on Robot Intelligence Technology and Applications (RITA 2021), 16-17 December 2021. https://doi.org/10.1007/978-3-030-97672-9_40 (pdf)

Media

Video demo for the updated version of our paper: Deep Reinforcement Learning-Based Mapless Crowd Navigation with Perceived Risk of the Moving Crowd for Mobile Robots. In this work we define the risk perception as k most dangerous obstacles. We have also improved the learning performance by using waypoints to increase the reward density.

Video demo for paper: Deep Reinforcement Learning-Based Mapless Crowd Navigation with Perceived Risk of the Moving Crowd for Mobile Robots. This is an initial result with risk perception from the (one) dangerous obstacle.

Video demo for paper: Comparison of Deep Q-Learning, Q-Learning and SARSA Reinforced Learning for Robot Local Navigation