Mapping, i.e., the generation of spatial representations of an environment by mobile systems like robots, autonomous vehicles, or drones, is an essential part of machine intelligence. There are many scientific aspect involved in it including substantial research areas like Simultaneous Localization and Mapping (SLAM). To be able to assess the quality of a map, to compare different methods, to do a performance evaluation of one or multiple systems, or to implement a loss function for machine learning in data-driven AI, it is hence of interest to have map quality metrics that allow to capture and quantify how “good” or “bad” a map is. This is not as trivial as it sounds as there are quite some higher level aspects in the notion of quality of a map, which go well beyond the pure metric accuracy. For example, a representation that may not be metrically accurate, but that preserves the topology of an environment can be still very useful.
Publications
(If you can not get access to the publication via the DOI link, click on [Preprint PDF] to get a preprint copy via ResearchGate)
[1] I. Varsadan, A. Birk, M. Pfingsthorn, S. Schwertfeger, and K. Pathak, “The Jacobs map analysis toolkit,” in Workshop on Experimental Methodology and Benchmarking in Robotics Research, Robotics Science and Systems (RSS), 2008. [Preprint PDF]
[2] I. Varsadan, A. Birk, and M. Pfingsthorn, “Determining Map Quality through an Image Similarity Metric,” in RoboCup 2008: Robot WorldCup XII, Lecture Notes in Artificial Intelligence (LNAI), Springer, 2009, pp. 355-365. https://doi.org/10.1007/978-3-642-02921-9_31 [Preprint PDF]
[3] A. Birk, “A Quantitative Assessment of Structural Errors in Grid Maps,” Autonomous Robots, vol. 28, pp. 187-196, 2010. https://doi.org/10.1007/s10514-009-9159-2 [Preprint PDF]
[4] A. Birk, “Using Recursive Spectral Registrations to Determine Brokenness as Measure of Structural Map Errors,” in IEEE International Conference on Robotics and Automation (ICRA), 2010. https://doi.org/10.1109/ROBOT.2010.5509322 [Preprint PDF]
[5] S. Schwertfeger, A. Jacoff, J. Pellenz, and A. Birk, “Using a Fiducial Map Metric for Assessing Map Quality in the context of RoboCup Rescue,” in IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), Kyoto, 2011, pp. 1-6. https://doi.org/10.1109/SSRR.2011.6106762 [Preprint PDF]
[6] S. Schwertfeger and A. Birk, “A Short Overview of Recent Advances in Map Evaluation,” in IEEE International Symposium on Safety, Security, Rescue Robotics (SSRR), College Station, Texas, 2012. https://doi.org/10.1109/SSRR.2012.6523906 [Preprint PDF]
[7] S. Schwertfeger and A. Birk, “Map evaluation using matched topology graphs,” Autonomous Robots, vol. 40, pp. 761-787, 2015. https://doi.org/10.1007/s10514-015-9493-5 [Preprint PDF]
[8] S. Schwertfeger and A. Birk, “Using Fiducials in 3D Map Evaluation,” in IEEE International Symposium on Safety, Security, Rescue Robotics (SSRR), 2015. https://doi.org/10.1109/SSRR.2015.7442997 [Preprint PDF]