Errors when submitting valid data to lidarseg challenging

Hi:

I try to submit my predictions to lidarseg challenging (https://eval.ai/web/challenges/challenge-page/720/overview)

I valid my submission on my own machine using provided script validate_submission.py (https://github.com/nutonomy/nuscenes-devkit/blob/master/python-sdk/nuscenes/eval/lidarseg/validate_submission.py) but when I submit it, there is an error reported on server:

Traceback (most recent call last):
File "/code/scripts/workers/submission_worker.py", line 452, in run_submission
submission_metadata=submission_serializer.data,
File "/tmp/tmpgi_kk_3z/compute/challenge_data/challenge_720/main.py", line 118, in evaluate
validate_submission(nusc_, submission_dir, eval_set='test', verbose=False)
File "/usr/local/lib/python3.6/site-packages/nuscenes/eval/lidarseg/validate_submission.py", line 40, in validate_submission
'Error: The folder containing the submission.json ({}) does not exist.'.format(results_meta_folder)
AssertionError: Error: The folder containing the submission.json (/tmp/tmpvjlm_49o/submission/test) does not exist.

I think this is because the validate_submission.py only can handle the folder, not the zip file which is required when submitting to the server.

Could you fix this problem?

Thanks!

Hi,
Is your submission organized in your zip as per https://github.com/nutonomy/nuscenes-devkit/blob/master/python-sdk/nuscenes/eval/lidarseg/README.md#results-format ?:

└── results_folder
    ├── lidarseg
    │   └── test        
    └── test
        └── submission.json

Depending on how you zipped your submission, it might have ended up as such:

└── results_folder
    └── results_folder
        ├── lidarseg
        │   └── test      
        └── test
            └── submission.json

I would suggest zipping your submission via:

cd <SUBMISSION_FOLDER>
ls  # You should see your `test` and `lidarseg` folders here
zip -r ../<SUBMISSION_NAME>.zip *  # This places your zip archive one level above
cd .. 
ls  # You should see your zip archive here; you can submit this one

Let me know if it works for you!

Yes~ it works. Thanks :slight_smile:

1 Like