Adding RGB Info to each Point of Point Cloud

#1

I want to add RGB info from camera, to each point of Lidar Point Cloud, and have the point cloud in the 3D space i.e. Lidar co-ordinate frame. How can i do this?

Would projecting the Point Cloud from Lidar co-ordinate frame to Camera co-ordinate frame, adding RGB pixel info to each point of point cloud, then doing an inverse projection back to 3D space i.e. Lidar co-ordinate frame be the right approach?

Thanks.

#2

Note that there is already a function to map a pointcloud to an image: https://github.com/nutonomy/nuscenes-devkit/blob/master/python-sdk/nuscenes/nuscenes.py#L632

If you modify that a bit you can retrieve the RGB values at each point. If you just save the index of the point, there is no need to project anything back. But note that some points won’t be inside any image (e.g. high or low points).

1 Like
#3

Understood. Only the points within the image will have RGB info. Thank you.:smiley:

#5

can you share the code? for extracting the RGB information from the Image, back-projecting it to LiDAR space, to get a Colored PointCloud.