Start Submission Become a Reviewer

Reading: Robotic Template Library

Download

A- A+
Alt. Display

Software Metapapers

Robotic Template Library

Authors:

Ales Jelinek ,

Brno University of Technology, CZ
X close

Adam Ligocki,

Brno University of Technology, CZ
X close

Ludek Zalud

Brno University of Technology, CZ
X close

Abstract

Robotic Template Library (RTL) is a set of tools for dealing with geometry and point cloud processing, especially in robotic applications. The software package covers basic objects such as vectors, line segments, quaternions, rigid transformations, etc., however, its main contribution lies in the more advanced modules: The segmentation module for batch or stream clustering of point clouds, the fast vectorization module for approximation of continuous point clouds by geometric objects of higher grade and the LaTeX export module enabling automated generation of high-quality visual outputs. It is a header-only library written in C++17, uses the Eigen library as a linear algebra back-end, and is designed with high computational performance in mind. RTL can be used in all robotic tasks such as motion planning, map building, object recognition and many others, but the point cloud processing utilities are general enough to be employed in any field touching object reconstruction and computer vision applications as well.

How to Cite: Jelinek, A., Ligocki, A. and Zalud, L., 2021. Robotic Template Library. Journal of Open Research Software, 9(1), p.25. DOI: http://doi.org/10.5334/jors.353
33
Views
6
Downloads
  Published on 20 Sep 2021
 Accepted on 05 Aug 2021            Submitted on 07 Oct 2020

(1) Overview

Introduction

Geometry is ubiquitous in robotics. We need it to describe robot’s pose, plan its trajectory, model the environment it is operating in and all of these tasks require a good way of handling rigid transformations, shape representations, data approximations and object registration. While rigid motions are well explored, with rich theory [1] and coverage in most robotic libraries [2, 3, 4], the shape representation and approximation is mostly treated either in the perspective of computer graphics [5], regression analysis [6] or probabilistic robotics [7]. There is nothing bad with either of these approaches, because robotics needs all of them, but there is mostly no simple transition between these views of the same thing [8] and this was the primary motivation of our research in the field geometric data processing. Computational efficiency was also in the centre of attention, because direct implementation of mathematical formulas often does not lead to an optimal algorithm.

Robotic Template Library (RTL) development started together with our research of the fast vectorization algorithms and is therefore a repository of code we used to illustrate our results in research papers. It is a known fact, that reproducibility of research in robotics is an issue [9, 10] and we would like to help the initiative to fix it. These algorithms are also what make RTL truly unique among other similar projects. RTL provides an implementation of the fast total least squares (FTLS) vectorization [11] of the ordered point clouds – an optimized algorithm for approximation of ordered data, which provides computational performance similar to the point-eliminating methods [12], while preserving all the benefits of the TLS regression. An augmentation of the previous approach [13] for optimization of global error is present in RTL as well.

Robotics is a rapidly evolving domain aggregating a large range of scientific and technical disciplines in the pursuit of autonomous – maybe even intelligent – machines. And indeed, this implies there are already many software libraries for general robotics as well as point cloud processing, linear algebra, computer vision and dozens of other tasks. Why yet another library? The reason stems from the purpose of current state of the art libraries in the field. There are large, well established projects overlapping with RTL, e.g the Robot Operating System [2], the Point-Cloud Library [3] and the Mobile Robot Programming Toolkit [4], but these are more focused on mature methods and algorithms for practical problem solving and tend to be too “high-level” for development of new methods we are interested in. For this reason, we have adopted the mathematical library Eigen [14] as a base for our research and built on top of it a standalone project, which is easy to adapt and experiment with. As such, RTL classifies as a small, geometry focused library for evaluation of our research and simple integration into other projects.

Implementation and architecture

The content of the library is sorted into several modules according to the functionality they provide. Let us first review the content provided and discuss the technical details later. The modules of the Robotic Template Library are as follows:

  • Core: The module provides base functionality such as constant definitions independent from STL implementation, algebraic primitives such as vectors, matrices and quaternions, as well as representations of the most important geometric objects such as line segments, polygons, bounding boxes, view frustums etc. Especially the linear algebra part heavily builds on the Eigen library and is often only a thin wrapper tweaking object interface with only a few additional functionality. The geometry related objects are implemented as templates for Euclidean space of arbitrary dimension. STL compliant type traits relevant to RTL are present here as well.
  • Input/Output: This module provides an interface to other formats for storage and presentation of RTL objects. STL compliant command line output is present for logging and the most basic user interaction. The second output possibility are the LaTeX export classes. One of them is dedicated to 2D vector drawings, graphs etc., the other contains a basic renderer of 3D scenes, next one covers basic tables and the last one allows to aggregate multiple figures into a single document and automate compilation of the LaTeX code. Resulting vector graphics can be used to provide high quality visualization of the computations possibly leading to direct generation of reports from experiments such as in case of examples in Figure 1. All figures and tables in this paper were generated using this module.
  • Segmentation: Algorithms for point cloud processing into continuous clusters of points are covered in this part of RTL. This procedure is an important step for outlier removal and a guarantee of continuity (according to given parameters) enabling the fast vectorization algorithms described further. There are two variants of the segmentation algorithm: one is suited for circular point clouds from static scans of rotating LiDARs, while the other is designed for processing of a continual stream of input points. Both segmentation algorithms are based on the same criterion, which makes their results easy to combine. An example of 2D and 3D segmentation is given in Figure 2.
  • Vectorization: This module contains algorithms for fitting of geometrical primitives to continuous clusters of point. Traditional point-eliminating approaches are covered (Reumann-Witkam [20] and Douglas-Peucker [19] algorithm), but the main focus is on total least squares (TLS) fitting. There is an implementation of the fast total least squares (FTLS) vectorization [11] and the augmentation [13] for optimization of global error. These fast algorithms are used for extraction of approximation lines in 2D and 3D and for planes in 3D (see Figure 3). As the vectorization procedure is separable into several steps, the library is designed to allow modifications of the processing pipeline, which results in many optional features such as approximating polyline construction, or global error optimization. The unit test of these algorithms also includes an extensible benchmark for comparison with other similar methods (e.g. Figure 4).
  • Transformation: Geometrical transformations are definitely a core feature of the RTL, however they aggregate so many related objects, that they were moved into a separate module. This module covers the most important basic operations: translation, rotation and rigid transformation of applicable objects from the Core module. All transformation work in general in N-dimensional space and can be composed together.
    There is also a mechanism to organize the transformations, because in applications, we usually need to represent a hierarchy of transformations between different coordinate systems. For more general case RTL provides a tree structure, where nodes correspond to poses and edges to actual transformations between them. For simple point-to-point transitions there is a chain structure, which is also returned, when pose-to-pose transformation is queried from the tree. To allow multiple types of transformations to appear in these structures, there is a generic type allowing to store any transformation desired and a complex treatment of uncertain return types of its application on other objects.
  • Test: Testing module provides tools for the library examination, however templates for automated testing of instantiation of other templates and random number generation might be handy in RTL applications as well. There is also a module storing additional information on the data types used throughout RTL such as maximal allowed error in tests or human readable description of the type for logging and error reporting.
Figure 1 

Automatic graphics output from experiments with RTL (a) Visibility diagrams [15]. (b) Correspondences [15]. (c) Pose uncertainty [16].

Figure 2 

Segmentation of 2D and 3D point clouds. Different colors distinguish separate continuous clusters, black is reserved for outliers and helper drawings (a) Empty room scan with noise [15]. (b) Helix point cloud with step changes in radius.

Figure 3 

Line segment and polygon extraction as used in [17]. (a) Extracted line segments. (b) Extracted polygons.

Figure 4 

Speed benchmark of vectorization algorithms – semicircular point cloud [15]. (FTLS [11] – green, INC [18] – dark green, DP [19] – black, RW [20] – gray).

The downloadable content of the RTL package also contains two sections with no additional functionality, but still important for the library:

  • Unit tests: Any larger project needs a validation mechanism of some kind and unit testing was chosen as an optimal solution for the RTL. The tests contain code to examine vast majority of the classes and methods available in RTL, in case of the vectorization module, there are basic benchmarks as well. The most of the tests are implemented using the GTest suit and the test module providing an automated way of testing of the library, but the tests of more complex algorithms in the segmentation, vectorization and LaTeX export modules provide graphical output in a .pdf file for visual examination. From the user’s point of view, the tests are useful for examination whether changes to the library did not broke the rest of the code and can serve as a companion to the examples, since they cover RTL more completely.
  • Examples: Basic usage examples may help to start with RTL faster. In a set of several short and simple snippets, a densely commented code shows how to harness core features of the RTL. Only some core features and the transformations are covered at the time of publishing of this paper, but considerable expansion of this section is expected in future releases. The main read-me file also contains instructions on building and running of this supportive code.

The Robotic template library is designed to be highly efficient in computational tasks. This is achieved using the Eigen [14] library as a back-end since it provides a very optimized code for general algebra and much more. Usage of templates and static polymorphism saves processing power on run-time and allows stricter type checking during compilation. Default constructors perform no initialization and functions usually do not check validity of their arguments, moving the responsibility on user’s side. Such behaviour is thoroughly documented and allows to omit redundant checks, when the data are guaranteed to be valid by other means. Of course, higher-level code comes with lower emphasis on performance subtleties. Transformation management structures are a good example of a balanced approach to ease of use and performance, because highly optimized transformations are expected to be applied millions of times per second, while modifications of the tree are much less frequent and thus allow some more computational costs. Separate case are the LaTeX output classes, where only the most significant optimizations were carried out, since the bottle necks of this output are outside the RTL.

As any other larger project, RTL also maintains coherent structure over its modules, unified code style and naming convention. This is especially useful in combination with the template nature of the library and allows to create a very general code through meta-programming techniques. To aid programmers with such applications, we provide type traits compatible with those from C++ STL. Table 1 summarizes available traits and also shows a portion of RTL in a new perspective of common features and mutual relationships.

Table 1 

Type traits of the Robotic template library, when applied on selected template objects. Type properties are examined in the left part of the table, while the applicability of geometrical transformations is summarized to the right. The traits are named in a positive manner, so if e.g. an object Obj has a metric defined, rtl::has_metric<Obj>::value is , otherwise it would be .

Quality control

RTL is distributed with a set of tests, which are used to ensure its proper functionality and can be used as a control mechanism, if the user makes some changes within the library. The tests are separated to focus only on a small portion of the code resembling the principles of unit testing. Some more advanced features of the library expect the more basic parts to be functional, e.g. it is recommended to run the test on vector algebra before the point cloud processing one. According to CLion integrated development environment, the code coverage of the tests is greater than 95 percent.

A feedback on proper function of user’s code appears in two ways. In case of unit tests, no output means no errors, otherwise a human readable description of what happened is provided. In case of general programming, compile time assertions are verbose as well and run-time behavior should simply correspond to the specification in the documentation available on-line.

(2) Availability

Operating system

Any operating system capable of compiling and running C++17 code. (Ubuntu 20.04 is tested)

Programming language

C++17 is mandatory for use of the library. (tested with CMake 3.15 and GCC 10.0)

Additional system requirements

There are no special requirements. RTL is provided as a header-only library.

Dependencies

The Eigen library [14] (v3.3.9 is tested) and the Standard Template Library (STL) of the C++ language are mandatory. To compile the library tests, GTest is needed as well, however the library itself works without it (v1.10.0 is tested).

List of contibutors

  • Ales Jelinek – development, architecture, testing and benchmarking
  • Adam Ligocki – development, testing
  • Ludek Zalud – supervisor, technical feedback

Software location

Code repository and archive

Name: GitHub

Persistent identifier: https://github.com/Robotics-BUT/Robotic-Template-Library

Licence: MIT

Publisher: Brno University of Technology

Version published: commit: 5161de846cef2fb253c364c9f14ab80b48153c46

Date published: 05.10.2020

Language

English

(3) Reuse potential

RTL has several modules for different purposes, which all meet in robotic applications. It can be used to build more advanced algorithms such as motion planing, scene interpretation, data fusion, feature extraction, localization, mapping and others, because it contains the base tools for dealing with geometry in space and for processing of point clouds. It is worthy both for development (due to generation of visualizations using the LaTeX export module) as well as for production (because the code is highly optimised).

However there is no restriction on using only a portion of the library for solving a specific task from a different field. Geometry is extensively used in computer graphics, physical modelling or even game development. The point cloud processing can find its usage in 3D scanning applications, signal processing and computer vision. LaTeX export is useful in all applications, where vector graphics needs to be generated directly from a program. Especially if the Eigen library is used for computation, RTL can be employed as a graphics output back-end.

Since RTL contains implementation of separately published algorithms, it can also be used in comparative studies, surveys and benchmarks for independent comparison of our work with results of other researchers.

RTL is published with all source codes available under the MIT licence, which makes it freely modifiable to any degree desired. The library is written with modularity and extensibility in mind so adding new features is possible and even encouraged. The code is hosted on GitHub, where all the feedback, bug reports and issue tracking should take place. We do not provide any official support for users of RTL, but we are open to discussion on features and grateful for bug reports, since quality and reliability of the library is our important goal.

Acknowledgements

A large portion of work on the Robotic Template Library was carried out on Brno University of Technology, Faculty of Electrical Engineering and Communication, Department of Control and Automation as a part of the PhD. training of the authors. We are grateful for its supportive environment which makes free development of new ideas possible.

Funding statement

The completion of this paper was made possible by the grant No. FEKT-S-20-6205 – “Research in Automation, Cybernetics and Artificial Intelligence within Industry 4.0” financially supported by the Internal science fund of Brno University of Technology.

Competing interests

The authors have no competing interests to declare.

References

  1. Blanco JL. A tutorial on se(3) transformation parameterizations and onmanifold optimization; 09 2010. 

  2. Stanford Artificial Intelligence Laboratory, et al. Robotic operating system. [Online]. Available: https://www.ros.org. 

  3. Rusu RB, Cousins S. 3D is here: Point Cloud Library (PCL). In: IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, May 9–13, 2011. DOI: https://doi.org/10.1109/ICRA.2011.5980567 

  4. Claraco, J-LB. Mobile robot programming toolkit. [Online]. Available: https://www.mrpt.org. 

  5. Bellocchio F, Borghese NA, Ferrari S, Piuri V. Reconstruction. 3D Surface Reconstruction. New York, NY: Springer New York; 2013. [Online]. Available: http://link.springer.com/10.1007/978-1-4614-5632-2. DOI: https://doi.org/10.1007/978-1-4614-5632-2 

  6. Deming EW. Statistical Adjustment of Data, dover book ed. Mineola, New York: Dover Publications; 2011. 

  7. Thrun S, Burgard W, Fox D. Probabilistic Robotics. The MIT Press; 2005. 

  8. Cadena C, Carlone L, Carrillo H, Latif Y, Scaramuzza D, Neira J, Reid I, Leonard JJ. Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age. IEEE Transactions on Robotics. dec 2016; 32(6): 1309–1332. [Online]. Available: http://ieeexplore.ieee.org/document/7747236/. DOI: https://doi.org/10.1109/TRO.2016.2624754 

  9. Bonsignorio F. A New Kind of Article for Reproducible Research in Intelligent Robotics [From the Field]. IEEE Robotics & Automation Magazine. Sep 2017; 24(3): 178–182. [Online]. Available: http://ieeexplore.ieee.org/document/8036322/. DOI: https://doi.org/10.1109/MRA.2017.2722918 

  10. Robotics and the art of science. Nature Machine Intelligence. Jun 2019; 1(6): 259–259. [Online]. Available: http://www.nature.com/articles/s42256-019-0066-8. DOI: https://doi.org/10.1038/s42256-019-0066-8 

  11. Jelinek A, Zalud L, Jilek T. Fast total least squares vectorization. Journal of Real-Time Image Processing. apr 2019; 16(2): 459–475. [Online]. Available: http://link.springer.com/10.1007/s11554-016-0562-6. DOI: https://doi.org/10.1007/s11554-016-0562-6 

  12. Shi W, Cheung C. Performance Evaluation of Line Simpli_cation Algorithms for Vector Generalization. The Cartographic Journal. Mar 2006; 43(1): 27–44. DOI: https://doi.org/10.1179/000870406X93490 

  13. Jelinek A, Zalud L. Augmented Postprocessing of the FTLS Vectorization Algorithm – Approaching to the Globally Optimal Vectorization of the Sorted Point Clouds. In: Proceedings of the 13th International Conference on Informatics in Control, Automation and Robotics. SCITEPRESS – Science and and Technology Publications. 2016; 216–223. [Online]. Available: http://www.scitepress.org/DigitalLibrary/Link.aspx?doi=10.5220/0005962902160223. DOI: https://doi.org/10.5220/0005962902160223 

  14. Guennebaud G, Jacob B, et al. Eigen v3. http://eigen.tuxfamily.org, 2010. 

  15. Jelinek A. Vectorized point clouds for mobile robotics. Ph.D. dissertation, Brno University of Technology. Faculty of Electrical Engineering and Communication; 2017. 

  16. Jelinek A, Ligocki A. On registration of vector maps with known correspondences. Advances in Electrical and Electronic Engineering. 09 2019; 17: 330–342. DOI: https://doi.org/10.15598/aeee.v17i3.3451 

  17. Jelinek A, Ligocki A. Lines to planes registration. In: Proceedings of the 16th International Conference on Informatics in Control, Automation and Robotics – Volume 1: ICINCO, INSTICC. SciTePress. >2019; 625–632. DOI: https://doi.org/10.5220/0007834906250632 

  18. Nguyen V, Gächter S, Martinelli A, Tomatis N, Siegwart R. A comparison of line extraction algorithms using 2D range data for indoor mobile robotics. Autonomous Robots. Jun 2007; 23(2): 97–111. DOI: https://doi.org/10.1007/s10514-007-9034-y 

  19. Douglas DH, Peucker TK. Algorithms for the Reduction of the Number of Points Required to Represent a Digitized Line or its Caricature. Cartographica: The International Journal for Geographic Information and Geovisualization. Oct 1973; 10(2): 112–122. DOI: https://doi.org/10.3138/FM57-6770-U75U-7727 

  20. Reumann K, Witkam APM. Optimizing Curve Segmentation in Computer Graphics. In: Proceedings of International Computing Symposium. Amsterdam: North-Holland Publishing Company. 1974; 467–472. 

comments powered by Disqus