Your submissions will be ranked in two leaderboards. Predictions for images from the lightbox category will be evaluated in the lightbox leaderboard and images from the sunlamp category will be evaluated in the sunlamp leaderboard. The total score is indicated on each leaderboard in the Best Score column. There is no ranking for performance on synthetic images, as the task is to bridge the domain gap between synthetic and real images.
In additon to the score, a separate column will describe the orientation score and position score separately (see pose score calculation for details).
During the competition, submissions are evaluated on a subset of all test images. We do this to encourage the development of genuine pose estimation methods and diminish the impact of exploiting the aggregated feedback of the leaderboard, i.e. overfitting to increase one's rank. At the end of the submission period, all submission will be re-evaluated on the complete test set and each team will be ranked by this final score.
Pose score calculation
The pose score is calculated in two steps: first, we compute the orientation and positions score, then we combine these two.
The position error for image i is the difference between the estimated and the ground truth position vectors, normalized by the ground truth distance of the satellite.