Hello colleagues,
I’ve just started to use the NuScenes dataset for my Master thesis, however I’m strugling to retrieve the Point cloud data / vector using the function below:
Please, can you support me how could I visualize the velocity data for example.
Thank you in advance!
Hi @Leandro_Eduardo_Mart, the nuScenes devkit provides a few rendering methods you can use to visualize the radar data.
For example, if you want to render the radar data in the bird’s eye view (BEV), you can do:
nusc.render_sample_data(my_sample['data']['RADAR_FRONT'], nsweeps=5, underlay_map=True)
Alternatively, if you want to visualize the radar data in an image, you can do:
nusc.render_pointcloud_in_image(my_sample['token'], pointsensor_channel='RADAR_FRONT')
Do check out the nuScenes tutorial for more details: https://www.nuscenes.org/nuscenes?tutorial=nuscenes
Hello @Whye_Kit_Fong thanks a lot for your fast repply 
Currently rendering is not my problem, I am able to see it in my tests
However since I’m working with sensor fusion I’d like to retrieve the radar data for each point.
Yesterday, I was able to retrieve somo data for the aformentioned render. Using the following code:
pc = RadarPointCloud.from_file(r’D:\Meus_documentos\Mestrado\YoloV5\yolov5\Nuscene\v1.0-mini\samples\RADAR_FRONT\n015-2018-11-21-19-38-26+0800__RADAR_FRONT__1542800376930984.pcd’)
pc
print(pc)
data = pc.points.astype(dtype=np.float32)
data[8] #vx_comp
data [8] result
array([-3.58403772e-02, -2.03126688e-02, -5.99706881e-02, -1.92602068e-01,
-1.29787654e-01, -1.08094811e-01, -1.60761163e-01, -1.68081820e-01,
-1.53200716e-01, 6.95560500e-03, 1.62855797e-02, -7.63690412e-01,
-8.70206058e-02, -2.51390096e-02, -5.29952794e-02, 1.07711062e-01,
9.47757959e-02, 7.22977892e-02, 6.29804730e-02, -7.71299228e-02,
-1.39897794e-01, -8.00487709e+00, -6.68413788e-02, -1.76215827e-01,
-8.01043606e+00, -2.18048826e-01, -1.75610170e-01, -2.05206964e-02,
-1.11219704e-01, -8.67524967e-02, 1.66111551e-02, -6.11628354e-01,
6.36762381e+00, -2.64386870e-02, -1.25792101e-01, -7.53235668e-02,
-1.89500839e-01, 1.23721160e-01, -1.45851478e-01, 1.53933197e-01,
-1.75075065e-02, -1.93948984e-01, 1.19011986e+00, -2.14846954e-02,
-1.95859700e-01, -2.44484609e-03, -7.49484301e-02, -4.73330617e-02,
-2.15848405e-02, -8.20530701e+00, -1.26934737e-01, -4.04719748e-02,
-9.72509459e-02, 1.86158910e-01, -2.65236162e-02, -2.19446599e-01,
-1.51756510e-01, -5.30254096e-02, 6.37596846e-01, -8.37158039e-02,
-5.10084853e-02, -1.06322713e-01, -9.48221460e-02, -1.42176643e-01,
-1.38736412e-01, -1.05555907e-01, -5.14228716e-02], dtype=float32)
But, I cannot associate the velocity with the specific point and more than that how can I related this to the rendered image…
Thank you in advance!
Apologies for the late reply @Leandro_Eduardo_Mart
You seem to be on the right track 
here’s a code snippet that you could take inspiration from as well:
import os.path as osp
from nuscenes.nuscenes import NuScenes
from nuscenes.utils.data_classes import RadarPointCloud
# Load the nuScenes dataset (mini-split, in this case).
nusc = NuScenes(version='v1.0-mini', dataroot='/data/sets/nuscenes', verbose=False)
sample_record = nusc.sample[100]
pointsensor_token = sample_record['data']['RADAR_FRONT']
pointsensor = nusc.get('sample_data', pointsensor_token)
pcl_path = osp.join(nusc.dataroot, pointsensor['filename'])
pc = RadarPointCloud.from_file(pcl_path)
pc = pc.points.T
print(pc.shape) # shape: (N, 18), where N is the number of radar points
The 18 dimensions correspond to:
x y z dyn_prop id rcs vx vy vx_comp vy_comp is_quality_valid ambig_state x_rms y_rms invalid_state pdh0 vx_rms vy_rms
For more details, do check out: https://github.com/nutonomy/nuscenes-devkit/blob/05d05b3c994fb3c17b6643016d9f622a001c7275/python-sdk/nuscenes/utils/data_classes.py#L315
So if you want to associate, say, vx_comp to each specific point, you would just need to read off the corresponding first three entries (i.e. x, y and z)
Thanks a lot @Whye_Kit_Fong
I’ll try the code later this week and Let you know if it worked!
Have a nice week.
Greetings from Brazil
Leandro
Hi @desmond13 I’m sorry for my late response, I’m sorry I did not get your question?
Did you ask my method for fusing those infomartion?
Hi @Leandro_Eduardo_Mart, no problem.
Exactly, I would like to understand if you are fusing somehow the information between the RADAR and the camera. If yes, how are you doing it? What are the methods you are using to do it?
Hi @desmond13 I’m sorry for my late repply I have had some personal problems and my research was a little delayed due to it…
Nevertheless, I’m still figuring out how to perform the fusion, for sure I’ll will use the pixel level information in ploting the radar points into the front camera image but more than that I’ll also have a parallel process of the radar data itself to be fused in a late state of my network, however I’m not sure exactly how I’ll do it thus I’m reading several papers in order to understand better how other researches have done it in similar works.
Have a nice 2022!
Greetings from Brazil
Leandro
Hello @Whye_Kit_Fong, @Leandro_Eduardo_Mart
I used the previous code to visualize pcd radar files,
However, I do not understand why my array shape is (18,61) while we have 18 dimensions and the width = 125 ? Could you please clarify this point ?
Thank you
@Hou_Arto if you end up with an array of shape (61, 18) using my code snippet above, then that implies that the radar point cloud contains 61 points.
I’m not sure where your width = 125 comes from though, since my code snippet does not contain any such variable.
WIDTH 125 is simply part of the header:
Hi @Whye_Kit_Fong i have a question regarding the from_file_multisweep function of the RadarPointCloud class. The returned pointcloud (pc) has the size (18,n), but i can’t figure out which field corresponds to which measurement. According to this post the height information of each radar point is set to 0. But the only array containing zeros is pc[14], which doesn’t match with the example of the header fields you posted right before me. Because according to the example of the header fields the z coordinate is stored in pc[2].
So my question is: Is there a difference between the from_file function and the from_file_multisweep function regarding the returned pointcloud format? And if so how is the header format?
Thanks in advance!!
