Back to news

Safe Road-Crossing by Autonomous Wheelchairs by information fusion

Giu 28, 2024

For our work, we have focused on a road-crossing scenario for the swarm system composed by an autonomous wheelchair and a drone, using multiple sensors whose information is fused to estimate the level of danger and to support the decision making process for crossing. 

We have proposed as a proof-of-concept a simplified scenario using wheeled robots equipped with two cameras and a set of distance sensors as mock-up models of the actual system. We then perform off-line danger evaluation using an analytical danger function based on kinematics considerations.

A dataset was recorded and further information can be found at https://huggingface.co/datasets/carlogrigioni/safe-road-crossing-aw-dataset.  The experiment set up is shown in the following picture:

esempio attraversamento

In red is the trajectory of the robot playing the role of the obstacle and in blue the crossing path.
The video shows the results of the sensor fusion and the risk evaluation for road-crossing. On the right, the real kinematic of the obstacle is described by the graphs of relative distance, speed and acceleration between the obstacle and the stationary wheeled robots playing the role of the wheelchair and the drone. Top right and top center are the videos from the two cameras showing the incoming obstacle. This one is detected through the machine learning model YOLO v8.

Finally, the central graph show in pink the output of the range sensors, in blue the actual danger value for crossing and in orange the danger value obtained by the fusion the output of the two cameras and of the range sensors.

A traffic light indicates a safe (green) or dangerous (red) situation for crossing.

All results and further information are described in the paper “Safe Road-Crossing by Autonomous Wheelchairs: a Novel Dataset and its Experimental Evaluation” accepted at the workshop “19th International Workshop on Dependable Smart Embedded Cyber-Physical Systems and Systems-of-Systems” (DECSoS2024) of the “43rd International Conference on Computer Safety, Reliability and Security” (SafeComp) DOI: 10.48550/arXiv.2403.08984.