close-circle
Close
0%
0%

Robot Overlord

3D simulation and control software for robots

Dan Royer
Following
Liked Like project
Similar projects worth following
3.9k views
12 comments
60 followers
likes
View Gallery
You're a dev building a robot and you want a 3d simulation and/or some control software for your machine. A well known, easy to use interface. You also have finite time and want to enjoy life, so you need something simple to code and painless for your customers. Robot Overlord is for you.

Robot Overlord (RO) gamifies industrial robot programming. It makes controlling them an intuitive experience driven by immediate visual feedback by displaying a 3D model of each robot and allowing a user to manipulate them in real time. It will enable telerobotics and drive-by-wire. It will enable simulating robots for better project planning and shorter development times. It wants to work with VR.

Think of it as ROS for Windows & Mac users. The focus is on ease of use; familiar, consistent user interface; and abstracting away device specific things like config files or text commands.

My first robot was a crab robot. All I knew back then was that hardware mistakes are expensive and software mistakes are cheap. So I put a 3D model of my robot into an empty video game and taught it to walk. When that worked, I copied the walk code into my actual robot (based on the 3d model) and it worked. Since then I've added all my favorite robots to one app to keep development effort to a minimum.

At present each robot has a context-sensitive menu to control it. If the robot is connected to the simulation, they move together. In some cases pushing the real robot affects the simulation. Work has begun on undo/redo, which will later evolve into record & playback.

There is a lot of opportunity to improve the software ease of use, add achievements, steam distribution, a point system, etc. I'd like people to plug in a robot, give it a command, and let players on the web game their way to an optimal solution. I'd like a VR interface for tele-robotics. I want my robots on the first experimental SpaceX trips to the moon, and then I want them to build our moon base by remote control. Join me!

  • 1 × Eclipse
  • 1 × Java
  • 1 × OpenGL
  • 1 × Fusion360
  • 1 × Maven

View all 10 components

  • Thoughts on building robot programs

    Dan Royer • 04/10/2017 at 01:24 • 1 comment

    Programming robots visually is proving to be a big challenge. I thought I laid all the ground work but now I'm not so certain.

    Each derivation of Robot has a derivation RobotMotionState (a snapshot of the robot at a moment in time). By comparing RobotMotionStates I hoped to test for robot/robot collisions.

    I've also got a system to command a robot to move, which changes the RobotMotionState, checks that it's internally valid (robot can reach that far) and externally valid (doesn't hit anything in the world).

    I've been stuck for months trying to build a system of RobotPrograms. Something like the Adobe Premiere or Flash timeline, where blocks of time have one command each. I couldn't wrap my brain around it enough to come up with generalize-able cases that could be written once and work for all robots.

    In a dream last night I flipped things around a bit. What if a timeline has keyframes and each keyframe is a RobotMotionState?

    • Move the timeline to a new position, move the robot to the new state, keyframe is automatically created.
    • Move to an existing keyframe, adjust the robot, done.
    • Delete an existing keyframe.
    • Drag a keyframe along the timeline.
    • Clone a keyframe.

    Ideally there should be a way to interpolate between two RobotMotionStates. That way as the read head is moved along the timeline the robot(s) move between their states. There must also be a way to calculate the difference between two RobotMotionStates and send *just that change* to the robot as a command.

    Oy. I'm not sure this is any easier.

  • A puzzle in Java

    Dan Royer • 03/29/2017 at 18:08 • 0 comments

    Robot Overlord Java app contains lots and lots of classes, some of which are robots and their gui.

    I want robot developers to have an easy time adding their robots to RO. A simple interface, minimal distraction, and examples to work from are Good Things. I'm told that RO can use a Service Provider Interface (SPI) to load a jar it's never seen before, a jar that contains an implementation of the interface. I would then

    • make a RobotInterface,
    • make every robot I've already got use that interface
    • move each robot to a separate jar and load said jars through SPI
    • make a separate github project for each robot
    • advertise these plugins via tutorials so that you can fork a repo, adjust to taste, and publish your new thing.

    What I'm discovering is that SPI is tricky tricky.

    • I can't find any online examples where someone has done this successfully.
    • I have not yet got RO to load my first robot's jar file, tho I'm trying. Is the jar packaged wrong? Maybe it doesn't say "yes, I have that interface!" in the right way.
    • Is RO not even seeing the jar? I'm told SPI looks for any jar on the classpath. I printed out the classpath, then put the robot jar in one of those classpath folders and ran the app again. Nothing.

    There are several possible points of failure, none of which can be clearly eliminated as possibilities. Worse, I'm not sure how these plugins would be debugged. Running RO would not give a lot of insight into the plugins' inner workings. Would I still be able to tweak code in real time? That is a must.

    So I ask you, dear reader: am I way off track? What do?

    I should note here that I do not want to have to run RO from the command line with a custom classpath. While I'm able to do it, I doubt that the people who buy robots and use them will even know how to open a command line. Imagine a grade school teacher trying to set up for their students, or your aged mother who's used to OSX. It ain't happening. You don't want that tech support phone call and neither do I.

  • Making it easier to develop for Robot Overlord

    Dan Royer • 03/28/2017 at 18:12 • 0 comments

    To make RO easier for developers, I have published the API at http://marginallyclever.github.io/Robot-Overlord-App/

    Next, the robots currently supported by RO will be moved to separate projects with their own github repositories.

    A crucial feature here is keeping it easy for the end user, whom I assume (in a worst case) know nothing about computers. They shouldn't have to modify the classpath or open a shell. I'd like it to be as easy as Arduino's board support installer - pick from a list of plugins online, download on demand, and go.

    Next, one or more tutorials will be made showing how to fork a repo, modify the robot type to your needs, and then publish your new plugin such that RO can find and install your plugin. Much easier than having to wade through the entire RO project and make a pull request.

    After that I run out of ideas. Comment with your suggestions, please.

    Here's the latest robot I've added to RO: A new stewart platform.

    https://www.instagram.com/p/BSEX_tMApi0/

  • uArm support grows

    Dan Royer • 09/09/2016 at 17:58 • 0 comments

    uFactory sent us the STL files for their uArm robot yesterday, and I was able to integrate the correct look and feel in the afternoon.

    https://www.instagram.com/p/BKHZxXlDOp8

    Also improved the lighting and made sure WASD+mouse can fly if the cursor is over the picture part of the app. Much easier to fly around the world.

    I already have firmware that runs the uArm connected to RO. I have a uArm on my desk waiting for a new shoulder servo and then I will be making a video of the two working together.

    Maybe now developers will start to get on board. BCN3D MOVEO? 7Bot Duo? Send me a robot so I can integrate your machine. Please tweet them and ask if their machine supports #robotOverlord for me, OK?

    The 2500th Makelangelo robot was turned on in the last couple of days in Muscat, Oman. I missed it because it came 10 days earlier than anticipated. My day is so full I had to hire an assistant and HIS day is full.

  • VR robot programming

    Dan Royer • 08/30/2016 at 16:00 • 0 comments

    Seems someone beat me to it. Oh well! We'll do it anyways.

  • Robot Scratch

    Dan Royer • 08/12/2016 at 21:22 • 0 comments

    Since the last update

    I've been fleshing out the undo/redo system.

    I made every button and menu item into an Action, which encapsulates both the visible thing that is activated by the user and the behavior once it is activated. This way I can move Actions around the GUI to improve the layout with ease. Each Action issues a Command pattern, which forms the Undo/Redo stack. Actions also pass along their label name. So changing a position shows as "undo change position" instead of "undo change vector3f". It obviously easier to understand.

    Most Actions are variations of the same thing, so I made an ActionSelectFile, ActionSelectString, ActionSelectNumber, and ActionSelectVector3f. All robots are being updated to the new standard so that they look and behave uniformly. So... that's nice.

    What's next: Robot Scratch

    All that work was important and useful and Good... but also escapism. I've been dreading the next step for a while because it requires major skills and I'm not sure I'm up to the task. I get to the point: To program each robot in a human-friendly way I can see no better option than Scratch. Programming - as a discipline - sucks. I should know! I've been a programmer for 25 years. I'm 10 years past the average coder's half-life.

    Anything to make coding more intuitive and less error-prone is a good thing. The easiest thing to fix in programming is to remove syntax errors, which Scratch does with such grace... it's poetic.

    Critique my method

    My theory is to make every robot run a RobotProgram which has a Events, Control, Sensing, Operators, and More blocks like Scratch. Move a robot to point X (which is an Entity in the world), wait for signal Y, send signal Z, Loop, Branch.... it a lot of work.Is there a faster way to get what I want?

    Asked on Scratch forums, too.

    https://scratch.mit.edu/discuss/topic/213147/?page=1#post-2147190

    So I've run this up the flag pole. Now to give it a day or three to see if anyone salutes while I work on my other prototypes.

  • Added uArm support (AKA how to add your 3 axis arm to Robot Overlord)

    Dan Royer • 08/08/2016 at 14:31 • 0 comments

    Arm3 on the left, uArm on the right.

    In the last week I've seen three people trying to solve Inverse Kinematics in reddit.com/r/robotics for paletizing robots like the uFactory uArm. Since this model of robot arm already exists in Robot Overlord, I thought it might be fun to add support for the uArm and show how the code changes. Now you'll be able to customize dimensions for your new designs.

    In the java code There is a package of classes called com.marginallyclever.robotOverlord.arm3, which contains everything to create a basic three axis paletizing robot arm like the ABB IRB 460 / uArm / LiteArm i2. Most notably is the Arm3Dimensions class, which holds all the unchanging numbers about the machine's size and starting position.

    Adjacent to that package I created com.marginallyclever.robotOverlord.arm3.uArm, which has a UArm class and a UArmDimensions class. UArmDimensions were set thanks to reddit user thingythangabang and this link he sent in.

    Lastly in src/main/resources/META-INF/services/com.marginallyclever.robotOverlord.Entity I added

    com.marginallyclever.robotOverlord.arm3.uArm.UArm

    So that RO knows uArm is an Entity that can be loaded when a user clicks the "Add..." button.

    RO is young, so expect big improvements in the near future. Once I have STL files of each moving section of the uArm I'll be able to replace the crude models above and it will look suh-weeeeeeet.... It will mean UArm will have to override drawIK() to display the correct Model files. If you made it this far, it shouldn't be a challenge.


    Also mad props to Freenode IRC #java users surial, dreamreal, and yawkat for helping me this morning with a "why java do dat?" question. Minimum viable product!


    I just received my first donation for developing RO, for which I'm super pumped. Thanks, Elie!

    Let me know what you think in the comments below, and please share with others.

  • Undo/Redo Adding and Removing Entities

    Dan Royer • 08/04/2016 at 21:39 • 0 comments

    Undo/Redo already covers movements inside individual robots. I just expanded it to cover adding and removing entities from the world. You can now add a robot, make it move, undo those moves, make some new moves, delete the whole robot, change your mind, and the undeleted object will have the same state that it had when it was deleted. When an entity is deleted it does not actually leave RAM, it is detached from the world and becomes a kind of dark matter.

    I've also expanded it to cover changes to Model source file name. Load an STL, change the STL, undo to change back.

    I'm working on expanding it to cover changes to Entity positions in the world, including the camera.

    Edit the next day: no, not the camera. Just every Entity.

    Look for these changes in the next official release.

  • Robot Overlord release 5 (v.1.3.0)

    Dan Royer • 08/04/2016 at 17:57 • 0 comments

    https://www.marginallyclever.com/product/robot-overlord-software/

    • Added "About this robot" dialog to each robot.
    • Added double-click on currently selected robot changes selection to camera
    • Added model STL loading
    • Improved Add... and Remove...
    • Fixed "Check for update" broken by changing github project name.

  • Added "About this robot"

    Dan Royer • 08/04/2016 at 07:36 • 0 comments

    Double click a robot.

    Scroll down the context sensitive menu to "About this robot". Click the button.

    Enjoy an informative popup with a link to more information.

View all 13 project logs

Enjoy this project?

Share

Discussions

Dan Royer wrote 08/05/2016 at 20:29 • point

@TheotherMike The goal is to easily generate motion profiles for each type of robot through a common interface.  One stand alone app for each robot type is stupid.  Gamified systems integration is where it's at.

  Are you sure? yes | no

TheotherMike wrote 08/06/2016 at 10:03 • point

That was more or less my question... I think it would be very helpful to make good documentation and tutorials how to work with RO and also about the "common interface" and what "gamified" means in your case.

  Are you sure? yes | no

QuasarAlpha wrote 08/04/2016 at 00:56 • point

This is an interesting project you have going on. Once you get RO working with VR do you have any other development plans for it? Things like allowing objects to be added for the robot(s) to interact with? 

  Are you sure? yes | no

QuasarAlpha wrote 08/04/2016 at 20:07 • point

Oh very cool! Thanks!

  Are you sure? yes | no

TheotherMike wrote 08/03/2016 at 20:56 • point

Hi Dan,

no, no, as I wrote! I don´t have mixed up anything ! THAT is the point ! ;-)  But it seems it´s not...

Think of a real robot arm equipped with a simple controller and a "cartesian" firmware, such as grbl or so. Its calibration is just linear and lets say g1 X10 equals to a movement of 10°.

Now, in ROverlord, the arm geometry is implemented which can be seen on the screen. Now using ROverlord, open a standard cartesian gcode and the multiple paths are shown in ROverlord (compare with a gcode simulator). Using pan/tilt/shift etc. move the gcode-paths (the whole pathway package) as you like to place them accordingly with respect to the robot model in order to adjust distances etc..

Now connect the arms end effector or the tool tip with the pathway and define its angle/distance/offset to the gcode pathway package. Now, let the tool tip and with it the whole arm move along the gcode path and read out the arms angle movements.

The readout then contains the angular position of each joint and its velocities which can be converted to a new gcode set which can be fed to the cartesian controller.

With this, many, many diferent robot arms could be controlled without tackling the IK on the controller. Instead one could use the ROverlord to visually see, what it will do, fine tune the arms positions as there are several solutions how to drive along the gcode paths....would be a visual Robot-CAM-Program...

  Are you sure? yes | no

Dan Royer wrote 08/03/2016 at 22:33 • point

I don't know why you want to send angle values.  GRBL is sending cartesian XYZ translations and trusting the firmware to move in straight lines.  If I send angle values only I cant' trust the firmware to move in straight lines.  Perhaps I am mistaken.  From here It sounds like your fundamental assumptions need work, the issue you want to fix isn't actually an issue to begin with.  taking gcode from one machine and running it on another is theoretically possible, provided the gcode does not have machine-unique codes that are essential to successful simulation.

  Are you sure? yes | no

TheotherMike wrote 08/04/2016 at 07:15 • point

It was just a thought... Many guys out there want to have a Robot arm (me too ;-)), and main problem is not the mechanical part but the Controlling part. There are some high priced CAM-Systems for Robot programming, out of reach for 99% of us. Everytime one Needs to adopt the special arm geometry, wether in IK or in the post processor.

As far as I now understand, ROverlord is a movement Simulator which Shows how it will move according to the code one has created somehow.

My thought was, use ROverlord to visually create the pathway and Export the code to control every Robot with a simple Controlling System because all movements have been converted to the angular movements of the arm. Provided the angle readout is executed often, one can again trust the Firmware because it just follows the angles in the same way it follows the XYZ translations....

3D CAM Systems for CNC-mills are more or less available and I think it would be beneficial to use an arm similarly.

I searched the WWW for examples I have in mind...but only found one: "RoboDK"

But maybe I´m completely wrong, so please apologize... Perhaps it´s better to ask the other way round, how would you do the following:

You want to laser engrave an Image (lets say you can use a vector Format, not Pixel by pixel) onto a wooden board using a self built Robot arm with 5 axis. Assuming the Laser control is working and the only Thing you want to do is path planning?

Now a friend Comes, but with a 4 axis arm, wanting to engrave the same Image...

  Are you sure? yes | no

Dan Royer wrote 08/04/2016 at 17:46 • point

Each robot developer creates their own IK and builds the RO model of their
robot.  The Robot Overlord developer (me) deals with creating path
solutions (generating tool paths) which are then sent to the robot. 
what happens inside the robot is the robot developer's business.  If you
want to send angle values instead of gcode, you can do that.

  Are you sure? yes | no

TheotherMike wrote 08/05/2016 at 08:14 • point

So, you can load an Image or dxf-file to RO and create the paths for engraving it using your own Robot model ??

  Are you sure? yes | no

TheotherMike wrote 08/03/2016 at 07:18 • point

Dear Dan,

awesome Project and I read about it several times before!!

A question that comes immediately into my mind....is there an Option to use the R.-Overlord to read/open a common gcode (generated for an cartesian mill or printer), connect the tip of the milling bit/end effector/Extruder nozzle of the Robot model with the gcode, read out all the angles and velocities and export a modified gcode which includes the Special geometries/kinematics of the Robot? With this, one would "not have to deal" with inverse kinematics. Would be somehow like the old cnctoolkit.

So, one could use it as a multifunctional post processing and visualisation tool for different robot geometries. At least for robots with low number of degrees of freedom...

Kind regards

Mike

  Are you sure? yes | no

Dan Royer wrote 08/03/2016 at 17:35 • point

Hey thanks, @TheotherMike.

inverse kinematics should be handled in the robot firmware.  Gcode shouldn't have anything to do with IK.  It sounds like you've mixed two separate things.

If two robot will obey the same gcode, their IK/FK doesn't matter.

  Are you sure? yes | no

Similar Projects

eye
112
users
5
skull
3
IronBot is a 3-in-1 DIY robot-learning kit, which can be assembled robotic arm, biped robot or humanoid robot by kids age 8 and up.
Project Owner Contributor

IronBot-A Smartphone-Operated Robot!

tab

eye
479
users
5
skull
6
This is a friendly DIY Robot I made for a class at my university that uses Matlab over a closed connection to process images and sound
Project Owner Contributor

[Insert Robot name here] the Matlab Robot

Augusto

eye
382
users
6
skull
4
In this project we spray Pesticide/Fertilizers by Robot Instead of manual, it save the lives of farmers and increase the work speed.
Project Owner Contributor

Mobile Controlled Pesticide Spraying Robot

sharanago

Does this project spark your interest?

Become a member to follow this project and never miss any updates

close

Yes, delete it Cancel

close

Report project as inappropriate

You are about to report the project "Robot Overlord", please tell us the reason.

close

Send message

Your application has been submitted.

close

Remove Contributor

Are you sure you want to remove yourself as a contributor for this project?

Project owner will be notified upon removal.