How to generate video with predicted 3D boxes?

shashwat14 · March 2, 2020, 1:22am

Hi,

I’d like to generate a video of the 20s scene with the predicted 3D boxes projected on to the 2D image (let’s say cam_front). Is there an existing functionality that allows me to do that? If not, what’s the suggested way to go about this?

Thanks!

holger-motional · March 2, 2020, 2:20am

Please work through the tutorial:
https://www.nuscenes.org/tutorial
The function is nusc.render_scene_channel().

github.com

nutonomy/nuscenes-devkit/blob/master/python-sdk/nuscenes/nuscenes.py#L1133


            key = cv2.waitKey()


        if key == 27:  # if ESC is pressed, exit.
            cv2.destroyAllWindows()
            break


    cv2.destroyAllWindows()
    if out_path is not None:
        out.release()


def render_scene_channel(self,
                         scene_token: str,
                         channel: str = 'CAM_FRONT',
                         freq: float = 10,
                         imsize: Tuple[float, float] = (640, 360),
                         out_path: str = None) -> None:
    """
    Renders a full scene for a particular camera channel.
    :param scene_token: Unique identifier of scene to render.
    :param channel: Channel to render.
    :param freq: Display frequency (Hz).

shashwat14 · March 2, 2020, 2:41am

Thanks for the response! I have seen this. I believe one need to write their own routine by following this for rendering the prediction.

holger-motional · March 2, 2020, 2:55am

Nope. Just run that function. Note that it’s not a “prediction”, but rather the ground-truth annotations.