Close

Simulation Progress in OpenAI/Mujoco

A project log for OpenK9

Open source, low-cost robotic quadraped for reinforcement learning

ben-nortierBen Nortier 03/08/2018 at 14:001 Comment

This is really an AI project, not a hardware project.

What I'm trying to do, is to create an economical physical robot by utilising simulation as far as possible. Building a hardware prototype, then working on the control system, then iterating on the hardware, etc. etc. is a very laborious, expensive process.

The good people at OpenAI (and others), are using simulated environments to train and develop robots orders of magnitude more cheaply (as far as labour is concerned). Using a simulated environment allows you to iterate very rapidly, and conduct experiments in faster than real-time. Here's an example of an OpenAI robot being developed: https://www.theverge.com/2017/5/16/15648158/openai-elon-musk-robotics-ai-one-shot-imitation-learning

I want to build a robot dog, but my budget is very close to zero (relatively speaking, compared to a company like Boston Dynamics), and I'm interested in using machine learning to achieve this goal.

i’m using OpenAI Gym as a starting point, together with Tensorflow and Mujoco. Mujoco is a physical simulation environment, designed for simulating robotics.

Essentially, I need to create a simulated environment that is close enough to the real world physical properties of the mechanics, actuators and sensors of the robot so the controls mechanisms can be transplanted to a real robot.

Lucky for me, there is a good starting point in OpenAI Gym — half a cheetah. I found the best listed solution in the OpenAI gym, written by a certain generous Mr. Pat Coady:

https://learningai.io/projects/2017/07/28/ai-gym-workout.html

Using his code and OpenAI Half Cheetah I trained this:

Then improved the mechanical parameters and the reward:

Added more legs:

Created a body similar to the Netflix Black Mirror Metalhead bot:

Improved the mechanical and reward parameters:

More improvements:

And now I have a reasonable model. Bridging the gap between the simulated model and an actual mechanical one is going to be tricky. It would require

  1. Accurately modelling the mechanical parts, including tolerances.
  2. Accurately modelling the joints.
  3. Accurately modelling the actuators.

It's going to be quite a challenge.

If you're interested in the code, all the code is on Github: https://github.com/bjnortier/max

Sorry for the lack of details, please ask questions if you have any :)

Discussions

Paul Crouch wrote 03/09/2018 at 13:36 point

Interesting. You're starting at the simulation then going forward to the physical, where I'm doing the opposite. Not sure I'll ever get to simulation/modelling, but watching with interest. Keep it up!

  Are you sure? yes | no