【泡泡点云时空】DETR3D:一种基于DETR的3维物体检测算法

要点

    facebook提出的DETR目标检测算法在研究领域中火热，本篇文章是清华大学与MTI合作的发表在CORL2021的对DETR扩展的算法，实现了3D的物体检测。但是本算法并不是基于激光点云或者伪激光(pseudo-LiDAR)的，而是直接基于多相机的照片来检测物体的3D包围盒。同时算法继承了DETR算法的优点，无需做繁杂的后处理如非极大值抑制(NMS)，以及使用了多头注意力来提升算法效果等，该算法曾经（2021年10月）在nuScenes数据集上达到了第一的成绩。

主要创新点

算法流程

算法效果

    在nuScenes数据集上，在不使用NMS的情况下，大部分指标都比FCOS3D要好，但是mATE要大于FCOS3D，作者觉得可能是FCOS3D直接预测了BoundingBox的深度所以mATE会更好一些。

总结

    对于想要在BEV下基于环视图像做3D物体检测的同学有参考意义，本篇文章延续了DETER算法的优点，即不需要做NMS后处理，简化了过程与计算量，同时避免了三维重建这种大量消耗计算的事情，作者直接对BoundingBox的三维信息做预测，通过相机的内外参数转换到图像空间，做法相对轻量，是一个比较好的启示。

参考资料

    We introduce a framework for multi-camera 3D object detection. Incontrast to existing works, which estimate 3D bounding boxes directly frommonocular images or use depth prediction networks to generate input for 3D ob-ject detection from 2D information, our method manipulates predictions directlyin 3D space. Our architecture extracts 2D features from multiple camera imagesand then uses a sparse set of 3D object queries to index into these 2D features,linking 3D positions to multi-view images using camera transformation matrices.Finally, our model makes a bounding box prediction per object query, using aset-to-set loss to measure the discrepancy between the ground-truth and the pre-diction. This top-down approach outperforms its bottom-up counterpart in whichobject bounding box prediction follows per-pixel depth estimation, since it doesnot suffer from the compounding error introduced by a depth prediction model.Moreover, our method does not require post-processing such as non-maximumsuppression, dramatically improving inference speed. We achieve state-of-the-artperformance on the nuScenes autonomous driving benchmark.