TADP: Task-Aware Deformable Prediction for Single-Stage 3D Object Detection

Summary

Abstract Summary

Most single-stage 3D object detectors complete different tasks with the same extracted features. Nevertheless, it is impossible to project features into a common space that is adaptive for all the tasks. We present a novel task-aware deformable prediction (TADP) method for single-stage 3D object detection to solve this problem. Firstly, a triple feature refinement aggregation module is designed to extract three-level features adaptively. Additionally, we design the multi-scale feature aggregation block to fuse multi-scale features in a scale-aware manner. Finally, the prediction of each task is deformed with the designed plug-and-play task-aware deformation head. It can percept the emphasis and interaction of each task. We also designed three different deformation modules. The experimental results demonstrate that the proposed deformation head shows good results on other detection methods. The experimental results on the KITTI dataset demonstrate that the car mAP is 80.91%, surpassing many state-of-the-art methods on the KITTI benchmark.

Key Points

Most single-stage 3D object detectors complete different tasks with the same extracted features
Nevertheless, it is impossible to project features into a common space that is adaptive for all the tasks.
We present a novel task-aware deformable prediction (TADP) method for single-stage 3D object detection to solve this problem.
Firstly, a triple feature refinement aggregation module is designed to extract three-level features adaptively.
Additionally, we design the multi-scale feature aggregation block to fuse multi-scale features in a scale-aware manner.

Method

We design a single-stage object detection method called task-aware deformable prediction for single-stage 3D object detection (TADP). Fig. 2 shows our designed method, which consists of three parts: (i)sparse blocks (SP Blocks); (ii)triple features refine and fusion, which contains tripe feature refinement aggregation (TFRA) and multi-scale feature aggregation (MSFA); (iii)task-aware deformable head (TADH).

As shown in Fig. 2 , the Sparse Blocks (SP Blocks) encode the point clouds. We convert point clouds to voxel format like SECOND.

Method — TADP: Task-Aware Deformable Prediction for Single-Stage 3D Object Detection — Fig. 2: The figure shows the pipeline of our method TADP. (a) downsample and encode the point clouds. (b) show TFRA structure refine the three-scales features. (c) show the MSFA structure, scale maps Features and fused with SFA. (d) show the TADH, predict the deformation map, and deform the task prediction.

Experiment

The experiments use the KITTI dataset. We mainly detect car class. The KITTI dataset divides the model evaluation difficulty into Easy, Moderate, and Hard levels. The comparison experiment uses the test set and submits it on the KITTI benchmark. Ablation experiments are evaluated using the validation set because of restricted access to the test set. Fig. 6 shows the visualization of the detection results.

According to the official KITTI evaluation metric, the 3D and BEV detection results are evaluated with mean average precision (mAP).

Experiment — TADP: Task-Aware Deformable Prediction for Single-Stage 3D Object Detection — Fig. 3: The pipeline of TADH. The first branch of the features directly generates traditional task predictions through predict line. The second branch through P-Stack generates DMap. The upper features provide height attention. Finally, the deformation to the prediction result is generated through the Deform-line.

Sources and Demonstrations

Figures are reproduced from Wang et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.