Data collection in different challenging weather conditionss

#1

Hello,

Table 1 of the paper “nuScenes: A multimodal dataset for autonomous driving” tells that the part of the data collection was done in the rain as well. This is an interesting scenario in many ways. I have a few questions in that regard.

  • I am assuming that the sensors are mounted on the rooftop of the vehicle, Without some form of protective gear, the quality of data could be compromised. How are these sensors protected from rain and snowfall?
  • Did you find any issues like distortion, interference, signal outage in the sensory information due to the weather conditions or due to the use of protective gear?

Awaiting ur reply,

Regards,
Raghava Modhugu

#2

Hi, we intentionally did not make use of any mechanisms to protect or clean the sensors. That means lidar/radar/camera may have raindrops on the, that can impact the sensor footage even 10+ minutes after the rain stopped. Also note that while raindrops can be removed, some impacts, such as the limited range of lidar in heavy rain, cannot be resolved.
Regarding “distortion, interference, signal outage” I did not notice any, but I also did not do a thorough statistical analysis of the signal w.r.t. weather.
Surprisingly the performance drop of object detectors on our rain data is not very high. I believe this is 1) because the rain is very low and 2) because of very different data/class distributions during rain - you won’t see many pedestrians out etc.
appendix