The ranking of the submissions is based on the accuracy of pose estimates on the synthetic test set. This score is indicated on the leaderboard in the Best Score column.
We also compute score for submissions on real images of the Tango satellite mockup, to evaluate the transferability of the pose estimation model/algorithm. Even though we indicate this score on the leaderboard (the Real Image Score column), the submissions are ranked based by their score on the synthetic images only.
During the competition, submissions are evaluated on a subset of all test images, in order to avoid 'overfitting' on the test set. At the end of the competition we re-evaluate all submissions on the complete test set, and rank the teams according to this final score.
Pose score calculation
The pose score for one image is calculated in two steps: first we compute separate orientation and translation scores, then we combine these two.
The position score for image i is simply the 2-norm of the position error (difference of estimated and ground truth position vectors), normalized with the ground truth distance of the satellite: