Radar points of object annotation

#1

Hi,
I run the code as follows to see the number of radar points of all objects annotations in one sample. But I find the number of radar points of all objects are really rare. I get the result below:

nusc = NuScenes(version=‘v1.0-mini’, dataroot=‘datapath’, verbose=True)
my_scene = nusc.scene[2]

first_sample_token = my_scene[‘first_sample_token’]
nusc.render_sample(first_sample_token)
plt.show()

my_sample = nusc.get(‘sample’, first_sample_token)
print(‘how many annotations does this sample have:’)
print(len(my_sample[‘anns’]))

my_anno_token = my_sample[‘anns’][22]
my_anno_metadata = nusc.get(‘sample_annotation’,my_anno_token)
print(‘The 22th annotation of this sample:’)
print(my_anno_metadata)

nusc.render_annotation(my_anno_token)
plt.title(‘The 22th object annotation in this sample’)
plt.show()

print(‘List out the number of radar points and lidar points of all objects annotations:’)
for i in my_sample[‘anns’]:
amd = nusc.get(‘sample_annotation’,i)
print(amd[‘num_radar_pts’], amd[‘num_lidar_pts’])

how many annotations does this sample have:
40
The 22th annotation of this sample:
{‘token’: ‘0e70ceb9294b47ee8039a5c2ba98f677’, ‘sample_token’: ‘8687ba92abd3406aa797115b874ebeba’, ‘instance_token’: ‘254ca291f1ee4223a0b1e392d4885fd2’, ‘visibility_token’: ‘4’, ‘attribute_tokens’: [‘c3246a1e22a14fcb878aa61e69ae3329’], ‘translation’: [1319.41, 1031.387, 1.821], ‘size’: [2.339, 6.418, 3.964], ‘rotation’: [0.3343250529872262, 0.0, 0.0, 0.9424578287356354], ‘prev’: ‘’, ‘next’: ‘6d88e84846bf4a53a37020db07122f52’, ‘num_lidar_pts’: 1389, ‘num_radar_pts’: 1, ‘category_name’: ‘vehicle.truck’}
List out the number of radar points and lidar points of all objects annotations:
0 5
0 7
0 26
1 0
0 7
1 3
0 21
0 25
0 24
0 1
0 5
0 50
4 2
1 8
5 473
0 2
2 0
0 1
2 0
9 335
0 7
0 24
1 1389
1 1
1 13
2 20
1 0
0 31
0 5
0 5
1 0
3 0
1 7
0 4
0 3
1 0
1 1
0 10
0 35
0 2445

The 22th annotation object is a big truck behind the ego vehicle, but the radar point is zero. And the radar point of the 39th annotation object which is a car just on the left side of the ego vehicle is also zero. But the lidar points for these two objects are quite normal. Is there anything wrong in the code or it’s true that the radar didn’t detect any of them? As my result shows, all the other objects also have very rare radar points.

myplot
myplot.jpg1800×900 320 KB

myplot
myplot.png1800×900 301 KB

myplot11
myplot11.jpg1600×2400 924 KB

#2

Hi. These 3 fields specify which radar returns are filtered: https://github.com/nutonomy/nuscenes-devkit/blob/master/python-sdk/nuscenes/utils/data_classes.py#L277
By default we only take highly confident returns, which may be why you see so few.
This line shows you how to do that:

pc = RadarPointCloud.from_file(os.path.join(nusc.dataroot, pointsensor['filename']), invalid_states=range(18), dynprop_states=range(8), ambig_states=range(5))

You can also globally change the setttings:

RadarPointCloud.invalid_states = range(18)
#3

Thanks a lot for your reply! I went to the code of the definition of RadarPointCloud.from_file() and saw the three filters. It shows that " To keep all radar returns, set each state filter to range(18)." Does it mean if I don’t want to do any filtering, I should set invalid_states=range(18), dynprop_states=range(18), ambig_states=range(18) or I should use “invalid_states=range(18), dynprop_states=range(8), ambig_states=range(5)” as you suggested? Thanks!

#4

You can see all the options in the comment:

    invalid_state: state of Cluster validity state.
    (Invalid states)
    0x01	invalid due to low RCS
    0x02	invalid due to near-field artefact
    0x03	invalid far range cluster because not confirmed in near range
    0x05	reserved
    0x06	invalid cluster due to high mirror probability
    0x07	Invalid cluster because outside sensor field of view
    0x0d	reserved
    0x0e	invalid cluster because it is a harmonics
    (Valid states)
    0x00	valid
    0x04	valid cluster with low RCS
    0x08	valid cluster with azimuth correction due to elevation
    0x09	valid cluster with high child probability
    0x0a	valid cluster with high probability of being a 50 deg artefact
    0x0b	valid cluster but no local maximum
    0x0c	valid cluster with high artefact probability
    0x0f	valid cluster with above 95m in near range
    0x10	valid cluster with high multi-target probability
    0x11	valid cluster with suspicious angle
    dynProp: Dynamic property of cluster to indicate if is moving or not.
    0: moving
    1: stationary
    2: oncoming
    3: stationary candidate
    4: unknown
    5: crossing stationary
    6: crossing moving
    7: stopped
    ambig_state: State of Doppler (radial velocity) ambiguity solution.
    0: invalid
    1: ambiguous
    2: staggered ramp
    3: unambiguous
    4: stationary candidates
    pdh0: False alarm probability of cluster (i.e. probability of being an artefact caused by multipath or similar).
    0: invalid
    1: <25%
    2: 50%
    3: 75%
    4: 90%
    5: 99%
    6: 99.9%
    7: <=100%

Since the code does not check whether a filter actually makes sense, the two alternatives you suggested should do exactly the same (accept everything).

#5

Thanks for your quick reply, that’s very clear. I can see more points through modification of these 3 parameters now.