Autonomous Driving is the task of navigating a vehicle with reduced driver interaction. This requires accurate perception of the surroundings, which includes three dimensional object detection. In this area, deep learning methods show great results, and they usually use either images, point clouds or a fusion of both. These methods are often evaluated with full sensor availability. However, in a safety critical system, unexpected scenarios such as sensor failure must be accounted for. Therefore, the objective of this thesis is to develop a deep learning architecture that is robust against sensor failure.
To accomplish this, we have designed an architecture that is heavily inspired by leading LIDAR object detection models. In order to be robust when the LIDAR is unavailable, we perform a learned projection from the 2D image to an artificial 3D point cloud. This is possible due to the fact that deep learning depth estimation models have reached a high performance. The two point clouds are merged into a common representation, which allows the model to perform the detections jointly and thus work if either sensor fails. The final contribution is a novel training procedure with varying sensor availability.
The results show that the model is indeed robust against sensor failure. However, it is not quite on par with the respective state-of-the-art models that specialize on camera, LIDAR or fusion. We are confident that the results can improved considering the limited amount of tuning and multiple ideas that are left as future work. Finally, we show that sensor failure robustness can help the model generalize in a broader sense.
PJ, lecture hall, Fysikgården 2B, Fysik Origo