Errors when submitting valid data to lidarseg challenging

#1

Hi:

I try to submit my predictions to lidarseg challenging (https://eval.ai/web/challenges/challenge-page/720/overview)

I valid my submission on my own machine using provided script validate_submission.py (https://github.com/nutonomy/nuscenes-devkit/blob/master/python-sdk/nuscenes/eval/lidarseg/validate_submission.py) but when I submit it, there is an error reported on server:

Traceback (most recent call last):
File "/code/scripts/workers/submission_worker.py", line 452, in run_submission
submission_metadata=submission_serializer.data,
File "/tmp/tmpgi_kk_3z/compute/challenge_data/challenge_720/main.py", line 118, in evaluate
validate_submission(nusc_, submission_dir, eval_set='test', verbose=False)
File "/usr/local/lib/python3.6/site-packages/nuscenes/eval/lidarseg/validate_submission.py", line 40, in validate_submission
'Error: The folder containing the submission.json ({}) does not exist.'.format(results_meta_folder)
AssertionError: Error: The folder containing the submission.json (/tmp/tmpvjlm_49o/submission/test) does not exist.

I think this is because the validate_submission.py only can handle the folder, not the zip file which is required when submitting to the server.

Could you fix this problem?

Thanks!

#2

Hi,
Is your submission organized in your zip as per https://github.com/nutonomy/nuscenes-devkit/blob/master/python-sdk/nuscenes/eval/lidarseg/README.md#results-format ?:

└── results_folder
    ├── lidarseg
    │   └── test        
    └── test
        └── submission.json

Depending on how you zipped your submission, it might have ended up as such:

└── results_folder
    └── results_folder
        ├── lidarseg
        │   └── test      
        └── test
            └── submission.json

I would suggest zipping your submission via:

cd <SUBMISSION_FOLDER>
ls  # You should see your `test` and `lidarseg` folders here
zip -r ../<SUBMISSION_NAME>.zip *  # This places your zip archive one level above
cd .. 
ls  # You should see your zip archive here; you can submit this one

Let me know if it works for you!

#3

Yes~ it works. Thanks :slight_smile:

1 Like