Hello Robot

A simple application in C# built with Machina could look like Code Sample 1:

In this example, the sequence of actions is as follows:

• Define a new instance of a Robot object, giving it a name, and defining its brand.
• Connect to a physical robot at given IP and Port values.
• Display the "Hello Robot!" message on the device’s display.
• Set the internal linear speed value to 100 mm/s.
• Move to Cartesian XYZ coordinates (400, 300, 500) in the robot’s reference frame.
• Rotate –90 degrees around the unitary positive Y vector (0, 1, 0).
• Move 250 mm in positive Z.
• Wait for 2000 milliseconds.
• Set the rotational values of the robot’s six axes to (0, 0, 0, 0, 90, 0) degrees respectively.
• Halt program execution by requesting user input, giving time to the robot to complete these actions.
• Disconnect from the controller before leaving the program.

The result of the execution of this program on an ABB IRB 1200 robot is illustrated in Figure 1.

Syntax

Machina is designed to program machines that perform motion in three-dimensional space. It features a high-level API of instructions to describe kinematic transformations, composed of simple English verbs that denote calls to action. Its interaction model is highly influenced by the cognitive principles behind the Logo programming language [36] and its popular application to turtle graphics. Some of its syntactical flavor and state-based settings are influenced by the Processing programming language [37].

The Robot class is the main public interface with the library. It features methods such as Move, Rotate and Transform to request kinematic transformations, Speed and Precision to request changes in motion properties, or Connect and ControlMode as management methods. A program can be created by the simple concatenation of a sequence of actions, and running it will result in either the buffered execution of those actions on the connected device, or the offline compilation of that same program into the device’s native language.

Where applicable, most of the API methods have two syntactical flavors: the function name with the To suffix, and the plain version. This denotes the difference between absolute and relative action requests respectively. Relative actions build on top of the current state of the robot, while absolute actions set those values independently of former states. For example, a MoveTo(400, 300, 500) call will set the position of the robot’s Tool Center Point (TCP) to XYZ (400, 300, 500) mm, regardless of its previous position. If followed by a relative Move(0, 0, 250) call, the resulting absolute position of the robot would be (400, 300, 750). This applies to TCP location, orientation, and robot axes values, but also to internal robot motion properties such as speed, acceleration, precision, analog out values, etc.

A breakdown of the most relevant methods in the Robot class can be found in Tables 1, 2, 3.

Action Model

The internal programming and control model of Machina is based on the atomic form of Action objects. Actions are the basic units of interaction with the machine, and follow the principles of the Enactive Robotics model for concurrent machine control [48]. An Action is defined as a change in some of the properties that define the state of a robot. For example, this can mean to perform some kind of spatial motion (Move, Rotate), change the execution properties of forthcoming actions (Speed, MotionMode), hold execution for a certain amount of time (Wait), change the IO values (WriteDigital, WriteAnalog), etc.

Tables 1 and 2 denote a loose correlation between Robot methods and Action types. This is not coincidental, as most API calls are basically thin wrappers that create an Action object internally and issue a request to the core Control class to execute that action Figure 2. This pattern presents a clear mental model to the user, who can easily relate the statement arm.Move(0, 0, 250) to the meaning “ask the robot to move up 250 mm,” and doesn’t need to care about the internal representation of such actions. It also helps maintain the modularity of the library and simplifies the extension of its functionality.

Actions are platform-agnostic. They are objects which store values for the state properties they are meant to modify, but do not need information on their target machine in order to exist–even if they are created through the Robot class. This means that programs are created internally as lists of Action objects, but that these actions have no real meaning until they are applied to a particular device. Depending on its nature, such a device may or may not be able to accommodate those changes; it is the responsibility of the developer who extends the library for a new machine type to decide which actions have effects on it and what their effects are. This means that calling the bot.ExtrusionRateTo(100) method on a non 3D printer is a valid operation, but will have no effect on the machine when executed online or compiled offline, beyond a warning message. This allows to maintain the cross-compatibility of the same API between different types of robots. However, it is important to note that the philosophy of the library is to provide a high-level programming language as constant in behavior as possible across different machine types. While developers are welcome to contribute to the project with extensions for new machine types, only those who maintain the library’s consistency will be accepted to the main project.

Machina can be extended with new actions by inheriting from the main Action class, and adding corresponding methods to the Robot class. Child actions must implement the ToString() method with a human-readable representation of the effect of the action; this helps the user understand the abstractions behind them. Child actions must also implement the ToInstruction() method to return a string mirroring the necessary API call to create such Action. This is extremely useful as a form of serialization, and provides a way to marshal actions when transferring programs between different implementations of the Machina framework, either through different programming languages or communication protocols. Code Samples 2 and 3 show examples of this behavior for the action in our Hello Robot program (Code Sample 1).

Control Modes

Machina can be used for offline programming. In this scenario, the library works by buffering all actions into a program, which can then be compiled into the device’s native language, manually loaded and ran on it. This is very similar to the typical way Arduino boards are programmed [17].

Code Sample 4 shows the Hello Robot example rewritten to work in offline mode. In this case, program contains a string representation of the Machina actions post-processed into ABB’s native RAPID language (Code Sample 5). The same program, compiled in Universal Robots’ URScript language is shown on Code Sample 6.

Note how, by default, Machina inserts comments throughout the program with human-readable descriptions of the code. This helps novice users understand the correlation between native instructions and the original Machina actions, facilitating debugging and serving as an entry point to native robot programming. This behavior can be customized through the overloads in Compile().

When working in online mode, Machina is intended to be used in control applications running on a host on the same network as the controlled device. In this scenario, the device’s controller must be running an instance of a custom Machina driver module written in the device’s language. Depending on the device, the driver may automatically be deployed by the application, or should be manually set up by the user. The driver module allows the Machina application to exchange information with the device via whichever communication protocol the device accepts. Following the Arduino analogy, this would be akin to interfacing with the board via the Firmata protocol [38].

Figure 3 describes a high-level representation of a sample communication scheme between a Machina host application and the driver. In this example:

• Communication is established between host and driver through the device’s available communication protocol.
• A handshake operation is performed to verify version compatibility and to update the host with initial machine state.
• Host streams action requests to the driver.
• Driver sends acknowledgement messages on action completion.
• Driver streams motion update messages when available.

In the example, communication with ABB devices works by running a driver module that creates a TCP server on the device, accepting incoming socket connections. On successful connection, the client receives string state messages in the format >21 X Y Z QW QX QY QZ; and >22 J1 J2 J3 J4 J5 J6; with the values for the current Cartesian and joint poses respectively. A request to transform linearly to a Cartesian pose can be sent as @ID 1 X Y Z QW QX QY QZ;, and upon completion of this request, the driver will send an acknowledgment message @ID 1; including the id value of the completed action. If capable of, the driver may also send state messages periodically, to keep the client updated on the motion state of the device.

However, Machina doesn’t impose a particular specification on how drivers for new robots should be implemented. Due to the lack of universality in firmware applications, and the variability of technologies available on back-end interfaces, different devices may require different communication protocols, connection schemes, and/or message formatting. Developers willing to extend Machina for new machines are welcome to choose how to develop their own driver modules, and which technology stack to implement in order to interface with them. The only requirement is to maintain the updating logic of the robot cursors that represent the asynchronous state of the system (see State Model section).

State Model

When using software applications to control hardware devices, there is a tremendous mismatch between the execution time frames of both parties: a large program may execute in a handful of milliseconds on the digital side, but may require several minutes, if not hours, to complete on its analog counterpart. For this reason, there is an inherent challenge in coordinating the elements of a system whose parts work at such different time scales.

Machina approaches this problem by implementing a system of layered machine state representations named cursors. A Cursor is a virtual representation of the state of the robot, defined as the values of all possible properties a particular device may exhibit at a certain stage of the robot’s execution timeline. These may include position, orientation, IO values, temperature, etc., properties which are particular to the device’s capabilities. A simple extension of the Cursor object for a six-axis robotic arm could implement properties such as Position vector, an Orientation quaternion or an array of angular values for the Axes, whereas a cursor representing a conventional 3D printer may implement Position and Temperature, but not require Orientation.

Since Actions are platform-agnostic, their effect depends on the state of the device at the time of execution. An Action is said to be applied to a Cursor when the cursor representing that state changes its properties based on the nature of the Action. An example of this would be the Position of the cursor changing from (400, 300, 500) to (400, 300, 750) after a Move(0, 0, 250) action has been applied to it. This model is particularly helpful when controlling machines that do not natively accept motion instructions in relative form, since the absolute values for low-level instructions are always available application-side through the Cursor representation of the robot. It is also useful when switching between actions defined in Cartesian and joint coordinates, as the state can be maintained in parallel on both spaces, with translations between them updated through forward and inverse kinematic equations.

The asynchronous differences between the host application run time and the robot operation timeline are tracked through a set of cursors representing the different stages of program execution. To better understand this architecture, it will be useful to break down the different stages in the execution cycle of an Action Figure 4:

• An Action is issued when a request to execute that action has been invoked. This happens typically on most Robot API methods such as MoveTo(400, 300, 500). Issued actions are immediately applied to the IssueCursor, which therefore maintains a representation of the future state of the robot after program completion, and is the base state on top of which new issued actions are applied.
• Upon issue, Machina queues actions into a first in, first out buffer, an manages the queue according to control mode and robot execution. An Action is released when it leaves the buffer, and a request to execute that action is sent to the controller. In offline mode, all actions are released at once upon Compile(). In online mode, Machina stages the release of actions to the controller in discrete blocks based on the amount of pending actions on it. Depending on the characteristics of the device, its microcontroller may not have enough resources to simultaneously handle communication, instruction parsing and smooth execution of large programs [39]. Blocks of blockSize actions are automatically released to the controller whenever less than pendingCount actions are pending to be executed, with blockSize and pendingCount being customizable. This prevents the device from overflowing with data transfers and memory requirements, and allows for the possibility of on-the-fly modifying or cancelling long programs that have been completely issued but only partially released to the controller. It also facilitates programming styles that are more reactive to the robot execution state. Released actions are immediately applied to the ReleaseCursor, which therefore maintains a representation of the state of the robot after execution of all the actions that have been sent to it.
• Once on the device controller, an Action is executed whenever the changes it represents have been fulfilled by the driver module, and hence the state of the real device reflects those changes. Or, in simpler terms, whenever the robot has completed running that Action. On successful execution, the host will receive an acknowledgment message from the driver with information about the action that was just executed. Executed actions are immediately applied to the ExecutionCursor, which therefore maintains a representation of the state of the real device right after the last completed action. It is noteworthy that in some instances, and specially with motion actions, program execution on the robot may move on beyond the current instruction even before the robot has fully reached its target position. This is usually the case when the controller tries to smooth the motion trajectory between forthcoming targets. In this cases, the action is considered executed as well.
• Optionally, some devices have the capacity to run multiple threads and send periodic updates on the state of the robot during execution. In this case, such states can be applied to the MotionCursor, which therefore maintains the closest representation to real-time tracking of the state of execution, including intermediate states between actions.

Figure 5 is an example diagram of the different cursor representations of the execution timeline in our initial Hello Robot program. Assuming that the initial position of the robot is (400, 200, 500), and that Machina is set to stream to the controller in blocks of six actions, this would be the state of the cursors after three seconds of execution:

• The host application executes in a few milliseconds, and is currently halted waiting for user input. Because all actions in the program have been issued, the IssueCursor is currently at the last action AxesTo(0,0,0,0,90,0).
• The Machina buffer releases actions to the robot controller in blocks of six. The state of the ReleaseCursor is updated up until Wait(2000).
• As the robot moves at a linear speed of 100 mm/s, three seconds into the program run time the robot has yet to complete execution of Move(0,0,250). Therefore, the ExecutionCursor is still on the previous action, Rotate(0,1,0,–90).
• If possible, the device updates Machina with the state of motion at small time intervals. Three seconds into the program, the MotionCursor would be approximately at position (400, 300, 600).

Feedback

Machina tries to foster programming styles that are reactive to, rather than prescriptive about, the robot execution state, with interactivity, system input and on-the-fly decision making being at the forefront of real-time robotic applications.

The library exposes a collection of Events linked to changes in the Cursors. These notify the subscribers of the nature of the changes and other useful information. ActionIssued, ActionReleased and ActionExecuted are raised throughout the different stages of the action execution cycle (see State Model), while MotionUpdate is raised anytime real-time information on the state of the robot is received from the controller.

Code Sample 7 shows a small program that starts by issuing motion in a horizontal square 50 × 50 mm loop. Whenever the robot has finished executing an action, ActionExecuted is raised. If no actions are pending to be executed, then a new loop is issued, hence creating an infinite loop that can only be interrupted by user input.

Other Features

Machina is designed as an introductory platform to real-time robotics for non-experts; in a way, it is like the 21st century Logo turtle of industrial robots. As such, many of the design decisions made during its development were oriented towards fostering simplicity, intuitiveness, safety and getting applications working right away with minimal setup.

The complete .NET library, as well as other related projects, is fully open source [47]. The aim of this is to open up the field of robotics to curious individuals and power users, either for educational, participation or customization purposes. Furthermore, special consideration is given to thoroughly commenting the source code, to facilitate usage and extensibility of the library.

Additionally, the library has no significant dependencies. It features a full set of custom geometry and robot-related data types, and only references members from the .NET framework. Currently, the only exception is a dependency on the Robot Communication Runtime by ABB Robotics [40], used to interface with ABB controllers and automatically run driver modules in the controller. This dependency is scheduled for deprecation in the nearby future.

Machina.NET is the core library of the Robot Ex Machina project [49], an ecosystem of libraries and applications designed around the same core Enactive Robotics principles [48], and designed to provide access to real-time robot programming and control from a variety of different platforms. The name is a made-up pseudo Latin expression meaning robot from the machine, and it evokes this project’s spirit of infusing agency and responsiveness into otherwise passive machines.

Future Work

The Machina project is currently under active development, and significant efforts are being made to improve its functionality across wider range of devices, and make it safer and more robust to use. In particular, current development is focusing mainly on the following aspects:

• KUKA robots: the library currently supports on/offline control of ABB and Universal Robots, and can generate offline code for KUKA robots. However, due to the author’s lack of access to the latter for testing, online control of KUKA robots is still not available as a feature. Developers willing to extend this functionality to the project are welcome to contact the author or submit pull requests to the project.
• Library of robot models: the library is purposeful generic, with compilers and low-level communication working at the brand level, without the need to incorporate model-specific data. This allows the library to seamlessly connect and control any robot model of a particular vendor. However, this lack of model specifications such as dimensions, joint limitations and mesh geometry makes it impossible to introduce more advanced functionality such as forward/inverse kinematics, out-of-reach computations, singularities, collision detection or visual simulations. While the process of documenting commercial robot models is rather tedious, it could open up important avenues for increased robustness and safety. The future of the library may involve a hybrid model where basic functionality is available for a “generic” robot from a particular vendor, while the above-mentioned advance features might be available if the particular model is available in the robots’ library.
• Forward/inverse kinematics solvers: for the reasons explained above, the library currently incorporates no FK/IK solvers, and relies on the robot controller to choose the best configuration for the arm at run time. On top of the limitations explained above, this lack prevents switching between relative and absolute instructions when changing from joint to cartesian space actions, and vice versa. Further work should go into providing suitable FK/IK solvers for the robot models available, in order to overcome these limitations.
• Enhanced safety: as a consequence of the former, the library does not have the capacity to prevent the user from issuing actions that would result in out-of-reach locations, traverse singularities, or collide with known objects. This could potentially pose a safety threat to humans, especially given the novice nature of the users the library is trying to target. The incorporation of robot model libraries and FK/IK would enable safety computations, and the possibility of adding options like StrictMode to ensure additional safety measures and robot halting upon known errors.

Safety

It is very important to note that industrial robots can pose a serious safety threat to humans working around them. Robotic actuators are very powerful machines but, for the most part, extremely unaware of their environment; they may not be aware when hitting an object, and continue execution uninterrupted with fatal consequences. Therefore, special attention must be given to the safety of the humans working or interacting with robots, especially if the audience of this project is users who are new to robotics.

In particular, individuals working with real-time robot control should pay special attention to the following measures:

• Be adequately trained to use that particular device.
• Be in good physical and mental condition.
• Operate the robot under the utmost safety measures.
• Follow the facility’s and facility staff’s safety protocols.
• Make sure the robot has the appropriate guarding in place, including, but not reduced to, e-stops, physical barriers, light curtains, etc.

These guidelines are offered as recommendations, but ultimately, users should follow the protocols in place by their robot/shop managers.

Machina.NET is under active development, and while it will yield warning and error messages on ill-defined actions, due to the current limitations it may not prevent users from executing them. The software is provided “as is,” without warranty of any kind, and the author/s should not be liable for any claim or damage arising from its use.

Archive and Code Repository

Name: Machina.NET

Persistent identifier: https://doi.org/10.5281/zenodo.2579370

URL: https://github.com/RobotExMachina/Machina.NET

Licence: MIT

Publisher: Zenodo

Version published: 0.8.9

Date published: 27 February 2019