Radar data labels

#1

Hi,

I have got both radar data and annotation in the same scene.
How do I EASILY know which object each radar point belongs to?

For the example below, we can see a truck was annotated in the front camera view, and it had 13 radar points from the reading of its sample annotation. I can access the radar pcd file to read all the radar points. How do I know which 13 radar points this truck have?
As it already have the number of the radar points in the truck’s sample annotation, I think the exact 13 radar points should also be accessed easily.

Or I have to find the radar point belongs to that truck if the global position of the radar point falls into the bounding box of this truck?

And according to the detailed info in sample_annotation as below:

"num_radar_pts":           <int> -- Number of radar points in this box. Points are counted during the radar sweep identified with this sample. This number is summed across all radar sensors without any invalid point filtering.

I suppose there must be a internal function can read all these radar points.

And if possible, please email me any detailed documents about radar sensor and its outputs. It seems that there is a dataset describing the radar return, but it’s not available on this site.

Thanks,
Feng

Screenshot%20from%202019-05-23%2017-22-12
Screenshot from 2019-05-23 17-22-12.png940×598 401 KB

Screenshot%20from%202019-05-23%2017-21-46
Screenshot from 2019-05-23 17-21-46.png1129×994 59.2 KB

Screenshot%20from%202019-05-23%2017-22-30
Screenshot from 2019-05-23 17-22-30.png1503×608 108 KB

#3

Hi @fjin4,

we don’t provide the information which radar point falls inside which object.
Here is some of the code used to count the number of radar/lidar returns that fall into a box: https://pastebin.com/uWFC2skG
You can adjust it to return which points belong to which object.

Please contact holger@nutonomy.com for more information on the radar sensors.

1 Like
#4
#5
#6

Thanks for the code! This was helpful to identify the points pertaining to pc returns from “RadarPointCloud.from_file()” file. However, I want to obtain this information for multiple-sweeps. I see that annotations are present only at the key frame. So for 7 radar sweeps, we should have one keyframe with annotations.
So, if I use the same code with “RadarPointCloud.from_file_multisweep()”, with nsweeps as 1, I am not able to find points corresponding to annotated object.
However, nusc.render_sample_data() still returns radar points from inside the box on the annotated objects.

It will be great if you could help in identifying radar data labels for multiple sweeps of radar data.

Thank you,
Madhumitha Sakthi

#7

Take a look at this function: https://github.com/nutonomy/nuscenes-devkit/blob/master/python-sdk/nuscenes/nuscenes.py#L274
It gets the boxes for keyframes and if you select the sample_data_token of a non-keyframe it will interpolate between the annotations. Now you could compare the interpolated boxes to the current radar returns. That would be much more accurate than just testing if any of the past 7 sweeps has a radar return inside the box.

#8

Can you explain how I get the sample_data_token of non-keyframes? I thought keyframes are annotated frames and those are the ones having sample_tokens. Currently I am getting the sample_token of the first sample in the scene:
sample_token = my_scene[‘first_sample_token’]

and than run a while loop until no next sample is available:
my_sample= nusc.get(‘sample’, sample_token)
sample_token = my_sample[‘next’]

How would I be able to get the samples in between those keyframes to get radar data and the interpolated object boxes as described in your post above?

#9

I answered the question myself. If anyone is also having trouble with this, look at this post: Non-key_frame radar sample_data labeling

You can get the radar data for the first sample token and then just get the next radar token from the “next” field (this is then not a keyframe, keyframes are approx. every 7th step has radar has 13Hz).

#10

Glad you were able to solve it!