How to use nusc and nuim at the same time

#1

Hi, thank you for sharing and maintaining an amazing dataset.

I am trying to use utilize a segmentation mask for the detection task.
To do that, I want to get the nuim segmentation mask from nusc ‘sample_data’.
(For example,
nusc_sample = nusc.get(‘sample’, nusc_sample_token)
nusc_cam_token = nusc_sample[‘data’][‘CAM_FRONT’]
nusc_sample_data = nusc.get(‘sample_data’, nusc_cam_token)

semantic_mask, instance_mask = nuim.get_segmentation(nusc_cam_token)
)
As I understood from the tutorial and schema, nusc and nuim share sample_data tokens.
However, when I try to get nuim data from nusc token, nusc token is not listed in nuim ‘sample_data’ dictionary key.

Am I doing something wrong or nusc and nuim do not share sample_data token?
Thank you.

#2

Hi. You are right. nuScenes and nuImages are completely disjoint. We merely followed a similar database schema and used a similar taxonomy and instructions. In fact, a 2d image dataset based on nuScenes would not have been very diverse, with only 1000 video clips. nuImages contains 100,000 video clips, specifically mined for rare classes, hard examples and lots of objects.

#3

Hi, Thanks for the reply. Just a quick follow-up question:
Can we get semantic masks from the nuscenes camera images? I know we can extract bounding boxes on the images but is it possible to get semantic masks similar to kitti?

  • Best Regards,
    Pramit
#4

@Pramit_Dutta yes. you can get both the semantic and instance masks for each image by doing:

semantic_mask, instance_mask = nuim.get_segmentation(key_camera_token)

Do check out the nuImages tutorial for more details: https://www.nuscenes.org/nuimages#tutorials