Bridging-the-Gap

Regularized Reinforcement Learning for Improved Classical Motion Planning with Safety Modules

Bridging_the_Gap.mp4

Requirements

• ROS 1
• Python version at least 3.6
• Python packages: pytorch, rospy

Running on a real robot

• Download real_robot folder to the robot computer
• Use move_base package for generating global plan and goal
• Run cdrl.py

Training on simulation

• Create your simulation environment of choice. In our case we used a realistic hospital world taken from: https://github.com/aws-robotics/aws-robomaker-hospital-world. We integrated moving people using: https://github.com/srl-freiburg/pedsim_ros. The steps for combining them can be found in PTDRL Repository - Simulation
• Download your robot. In our case we used Turtlebot with waffle-pi.
• Clone folder.
• Train RL: train.py. save_model=True arg to save neural networks.
• Train Supervisor: train_supervisor.py.

Simulation, robot and training algorithm

• Bridging the Gap was written in the OpenAI ROS fashion. This means that training, simulation, and robot environments are separated.
• task_env.py provides all the context we want the robot to learn for the RL task, in this case, navigating fast and safely. It contains the main functions: step reset and _init_env_variables. It includes the move_base related functions: _set_init_pose, _send_goal, and tune_parameters.
• robot_en.py contains all the functions associated to the specific robot that you want to train. It also provides the connection with move_base. It contains the main callback functions: _scan_callback, _odom_callback, and _costmap_callback.
• params.yaml includes:

1. Which local planner is used
2. The set of parameters of the local planner
3. A list of rooms the robot goes through inside the environment

Hospital world

The simulator of the hospital was taken from: https://github.com/aws-robotics/aws-robomaker-hospital-world. which is amazon's representation of a hospital. It is very rich in the sense of quantity and quality of objects simulated, and it represents realistically a hospital.

Notes

The models have to be downloaded manually from: https://app.ignitionrobotics.org/fuel/models into local models folder.

Pedestrain simulator

The pedestrian simulation is acquired using pedsim_ros package. The package is based on: https://arxiv.org/pdf/cond-mat/9805244.pdf social force model. This package allows to choose the quantity and type of pedestrians, and motion patterns. The package was taken from: https://github.com/srl-freiburg/pedsim_ros. Download and install it.

Notes

The hospital world of amazon has to be moved to pedsim_gazebo_plugin/worlds folder. Notice that the line: plugin name="ActorPosesPlugin" filename="libActorPosesPlugin.so" has to be added at the end of the file to allow pedestrian movement.

Notice that the pedestrian simulator has to account for obstacles in the world. This should be described in .xml found in pedsim_simulator/secnarios.