A real-time creative coding environment for KUKA industrial robot arms. For painting, performance, and other kinds of art.

Similar projects worth following
Crash is a real-time motion control system which encompasses motion planning, inverse kinematics, and online trajectory generation. A semi-real-time, networked, stream-based, graph-structured, live-programmable planning system incorporates sensor input into motion planning and tool control. It communicates with KUKA robots using the Realtime Sensor Interface (RSI). The system is being developed for my machine-mediated painting experiments.

After intensive training human craftspeople can produce artifacts of unmatched quality, while adapting to variations in materials, design constraints, and available tools. Modern manufacturing technologies perform well when the goal is to create very many of an object, but break down when every object must be designed and fabricated to fit unique constraints.

For example, in architecture no two buildings are exactly the same down to the last detail and so the design of each one needs to be adapted to the space where it will be realized and the requirements of the particular project. Even when final plans are drawn they are often adapted on the fly during construction because some critical constraint was missed which only became clear with the physical work underway.

Sometimes the environment for the design is not known in advance, like in the case of building on other planets or in disaster areas. Other times a design calls for materials that have varying material properties, like wood, and some adaptation of the design is needed to make use of the material available. Right now our automation can only be used under carefully controlled circumstances and work must fall back to humans in every other case.

Ideally machines would relieve us architects, engineers, and designers from some of the burden of adapting to the dynamic world our creations must survive in. The only way to achieve that is to build higher order tools which can learn and improvise on their own.

  • Real-time Control with ROS

    Owen Trueblood03/31/2019 at 04:13 3 comments

    Real-time feedback is an important ingredient in tool embodiment. That is, when a tool reacts fluidly to human intention and the result of its action is available immediately and continuously it can feel like an extension of the body of the user. Like riding a bike, writing with a pen, or making music on a piano. After learning little attention needs to be paid to moving the right way and so the conscious mind can occupy the abstract space of the task the tool provides leverage over.

    Industrial robots are not usually built with real-time feedback in mind, because most often they repeat the same basic motions over and over again in a static environment. And real-time control can be extremely dangerous, because a bug in the software regulating the motion can lead to unexpected motion of the robot. The only control usually exposed to a human user is the ability to start or stop the activity of the machine, or change its speed. The first significant technical hurdle of this project lies in figuring out how to overcome the base limits of the controller to enable real-time control over every joint in the robot.

    Remote Control with KUKAVARPROXY

    The KRL programming language environment on the KUKA Robot Controller v2 (KRC2) used in this project provides programming primitives for controlling and triggering action based on basic industrial I/O, but its a pain to set up such I/O and the control frequency is very poor (roughly a few Hz). It's meant mainly for switching on equipment like welders or reading inputs like proximity switches, not low latency communication.

    Ideally the robot could be controlled over Ethernet from a regular PC. A common tool for doing that with older KUKA industrial robots is a third-party tool called KUKAVARPROXY. This tool runs under Windows 95 on the KRC and sets up a server that allows global variables to be read and written over a network connection via a simple protocol. A script written in KUKA Robot Language (KRL) running on the robot can read and set global variables to communicate with external client PCs.

    There is some support for this mode of control in commercial software, such as the excellent RobotDK. By running a specified KRL script on the robot RoboDK is able to connect and read the state of the robot's joints and execute basic motion commands. RoboDK has a Python API that allows mediated control of the robot. In the video above a Processing sketch is used to generate simple motion commands that are sent to a Python server (code here) which calls the RoboDK API to communicate with KUKAVARPROXY on the robot and consequently the running KRL script. For setting up jobs in RoboDK this sort of functionality is sufficient, but it does not provide an avenue to the real-time control that I'm after. There is a critical flaw with this mode of communication which has to do with how the robot does motion planning during KRL execution.

    To be able to plan motion properly KRL scripts are executed from two places at once. There is the normal "program counter" which is tied to the currently running command. But there is lookahead functionality as well, which executes commands into the future to fill the motion planner's buffer. Executing motion that depends on variables which can be changed at any time throws a wrench into this lookahead mechanism. Disabling lookahead means that the robot will come to a stop after every single motion command, making it impossible to build up fluid motion from the available motion primitives in KRL like lines and arcs.

    I spent a while investigating KUKAVARPROXY and KRL before ruling them out. One of the products of that work was a rudimentary transpiler from Clojure to KRL. You can find an example of that tool used to implement a KUKAVARPROXY remote control script on GitHub here.

    Real-time Control with RSI

    If KRL is fundamentally limited when it comes to real-time control because of the lookahead feature, what are other options? What's...

    Read more »

View project log

Enjoy this project?



Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates