In an effort to enhance the computational efficiency and resource consumption of the DR-SPAAM (Distance Robust Spatial-Attention and Auto-regressive Model) for person detection in 2D range data, knowledge distillation techniques were employed. We have worked with the DROWv2 Dataset as employed on the original DR-SPAAM network. The core objective was to transfer the detection capabilities of the original, more complex DR-SPAAM model—acting as the “teacher”—to a smaller, more lightweight “student” model. This process involved training the student network not only on the ground truth labels but also on the rich, soft-target probability distributions and intermediate feature representations generated by the teacher model.
By leveraging the distilled knowledge, the aim was to create three compact and faster versions of DR-SPAAM, significantly reducing its inference time and memory footprint, making it even more suitable for real-time person detection on resource-constrained robotic platforms and embedded systems, while striving to maintain a comparable level of accuracy to the original, larger architecture. The results were satisfactory and the differences in speed were quite substantial while the student networks’ performance are not compromised.
For the first student, named generically “Student”, the actual results were slightly better compared with the original teacher. The “Student” network outputs an average precision of 0.723 whereas the teacher outputs an average precision of 0.722, which outperforms the teacher.
The other two students are named the “Tiny-Student”, where the results are very close to the ones of the teacher network (AP=0.712), and “Micro-Student”, which is very efficient, but the performance takes a hit. The improvement in speed is quite significant, where the three students have a performance of 111.2 FPS for the “Student”, 150.7 FPS for the “Tiny-Student” and a performance of 163 FPS for the “Micro-Student”.
The original DR-SPAAM has a speed of 87.3 FPS, as tested on an NVIDIA A2000 GPU with 4GB memory. We also conducted additional tests and the GFlops and GMacs were also calculated.
| Model | Parameters | GFlops | GMacs |
| Teacher | 1977667 | 102.81 | 51.09 |
| Student | 741155 | 59.53 | 29.53 |
| Tiny-Student | 125011 | 9.1 | 4.47 |
| Micro-Student | 31723 | 3.16 | 1.54 |
Table 1. Metrics for evaluating computational efficiency metrics.
| Model | F1 | AP | ERR | FPS |
| Teacher (DR-SPAAM Architecture) | 0.697 | 0.722 | 0.697 | 87.3 |
| Student (DR-SPAAM Architecture) | 0.699 | 0.723 | 0.688 | 111.2 |
| Tiny-Student (DR-SPAAM Architecture) | 0.690 | 0.712 | 0.682 | 150.7 |
| Micro-Student (DR-SPAAM Architecture) | 0.655 | 0.690 | 0.6392 | 163 |
Table 2. Metrics for evaluating the performance of the detector.