How long does it take to approve the submission

#1

I recently submitted for singlecoil challenge.
It shows a red cross mark against the submission.
I wanted to know is it wrong submission or it is in approval stage.
How long does it take to see the submission on leaderboard ?

Thank You
Pranjal Sahu
psahu

#2

Hi Pranjal,

we found an issue in the fastMRI test set. We are transitioning to a new test dataset, which should be available tomorrow. Until then, I’m afraid, submissions to the online leaderboard will not work. Hold tight, more information about the issue will be released tomorrow.

Best,
Jure

#3

Hello
I faced the same issue after using the version 2 test data. The submission was not approved. Can I get the cause of the error so that I don’t repeat it.
Thank You.
Username: psahu

#4

Hi @pranjal_sahu,

The reason your submission wasn’t approved is because the evaluation script was unable to unzip the uploaded directory. How are you zipping it?

When I run “unzip psahu_submission.zip” I get:

Archive:  psahu_submission.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of psahu_submission.zip or
        psahu_submission.zip.zip, and cannot find psahu_submission.zip.ZIP, period.

To answer your original question. It takes 5minutes for the submission to be evaluated.

#5

It turns out the problem is with the format of the dropbox URL, the way its zipped is fine. You need to append ?dl=1 instead of ?dl=0. This causes a direct download rather than a web preview of the content.

See https://help.dropbox.com/files-folders/share/force-download

#6

Great then I will re-submit it and check again.

#7

I just did a submission with dl=1 and it is not approved this time also. Maybe some another problem this time. Please help !

Thank You
Pranjal

#8

The last URL you submitted doesn’t link to any public content. Are you sure you made the folder public?

Also I noticed that your last submission had a different name to your original one. You original submission’s file was named reconstructions_test8.zip and the last one you submitted was reconstructions_test.zip. Double check you’re linking the correct URL.

#9

Hello Tullie

This is embarrassing but I still haven’t been able to make a single successful yet.
I have checked thoroughly dl=1 and the link does points to the zip file.

Also, I have a suggestion, the limit of one submission should be on successful submissions and not the failed ones, that way one does not have to wait for a day to resubmit.

Also, are the entries such as URL of paper and GitHub repo needed at this stage?
Please put a star mark beside the options which are compulsory for submission.

Thank You
Pranjal Sahu
psahu

#10

Hi @pranjal_sahu,

We’ve opened it up to multiple failed submissions and only one accepted submission. We added optional labels to the submission form and made the URL submission parser more robust. Can you please try submitting again? Let us know if it doesn’t work.

Thanks,
Tullie

1 Like
#11

Hi,
I have the same issue as Pranjal Sahu.
I’ve submitted for singlecoil challenge and It wasn’t approved for the first time and I changed URL but now, there is nothing in my submission.
Can you please tell me the reason ?
Thank you,
Chi-Hieu

#12

Please check these-

  1. Is the url direct downloadable i.e. if you paste the url on the browser and hit enter does the file download directly starts ?
  2. The folder name has to be ‘reconstruction’ as mentioned at the submission instructions page. I was putting reconstructions_test and it was not taking it.

After these steps it worked for me.

Thanks
Pranjal

#13

Thanks Pranjal,

How long does it take for submission ? I submitted yesterday and then the submission page has been always processing

Thank you very much and I hope you have a good weekend.
Best regards,
CH

#14

Hi @chpham,

It should only take 5mins for the submission to be approved.

I took a look at all the URLs that you submitted and all but one of the links weren’t loading for me - can you make sure they’re public?

The results for the one link that did work were formatted incorrectly. The reconstructions should be a zipped folder containing all test volumes in the correct h5 format.

Thanks,
Tullie

#15

Hi @tullie,
Thank you for your answer.
I checked the download link and changed another one for sure. In fact, I tried to download with my terminal command in linux via "wget 'my-submitted-link’ " and it worked. So, the link is downloadable.
I saved each reconstruction (with the size of width x320 x320) under the name “reconstruction” in a h5 file, which has the same name as the test file. But the submission page is still freezing after submission. Does it need thus your approve to be submitted ?
Best regards,
CH

#16

Hi @chpham,

The submission should evaluate automatically.

There was an issue with both of your latest links but the fault is on the robustness of the url parsing code rather than you. Notably:

  1. The Amazon link had an unknown content-type in the request header
  2. The Google link is getting stuck on a preview “Google Drive can’t scan this file for viruses.” page

Re 1 - we’ve changed the code to handle unknown content-type’s as zip files. If you go to the leaderboard you’ll see that your Amazon linked results are now approved.

Re 2 - still looking for a solution here. Apparently web hosting support was deprecated in 2016 (https://gsuiteupdates.googleblog.com/2015/08/deprecating-web-hosting-support-in.html)

Thanks for persevering as we make the submission form more robust.

Tullie

1 Like
#17

I have fixed the downloadable link problem in google drive. But still the submission is not approved. Suggestions would be helpful.