Submission Rules

How to make a submission

Pose estimated for images from the lightbox and sunlamp sets must be submitted in a single .csv file. In the file, each line must correspond to an image, containing the following information:

• a string indicating one of two categories: "lightbox" or "sunlamp"
• the filename of the image (e.g. img000014.jpg, img000021.jpg, ... )
• 4 floats describing the orientation quaternion
• 3 floats standing for the relative position vector.

\(\begin{array}{cccccccccc} \hline \mbox{line in file} & category & filename &[q_1 & q_2 & q_3 & q_4]& [r_1 & r_2 & r_3]\\ \hline [0]& string & string &float,& float,& float,& float,& float,& float,& float & \\ [1]& string & string & float, & float,& float,& float,& float,& float,& float & \\ ... \\ \hline \end{array}\)

The lines describing the poses have to be a 1:1-match with the dataset. That is, submissions are only valid if they include a line for each member of the dataset and not any lines beside it.

In our starter-kit we provide some functionality to generate submission-files according to this format, so we suggest you to make use of this.

Submission processing

As soon as a file is submitted, two steps will happen in our servers: a) preliminary validation and b) scoring step. During the validation step we try to filter any invalid submissions (e.g. invalid fields in the submission files, unknown filenames, etc.), and make sure that the submission contains pose estimates for all sunlamp and light box images.

If a problem arises at this stage an error code and a description of the problem will appear to help the user debug the problem. This will not decrease the submission quota for the team. Otherwise, if the preliminary validation was passed, the submitted file will be stored in our servers and appear in the submission list.

Then scoring is run on the submission. If scoring is successful, the result will appear on the leaderboards, otherwise it will appear in the private submission list as an invalid submission.