Open Source Hardware with Machine Learning

Open source hardware environment for machine learning so that these algorithms can eventually be applied to more complex real life systems.

Similar projects worth following
The goal of this project is to provide an open source platform for people to learn about applying Artificial intelligence (AI) to a mechanical system. The system is a classic inverted pendulum that was created using low cost 3D printer motion mechanisms, open source electronics, and controlled using Tensorflow. The pendulum will use an arduino controller to provide the motion control and feedback to the laptop. The open source hardware will have an easy to use interface to communicate to the AI algorithm running on a laptop. This project will bridge the gap between software and hardware so that the the focus of the project can be implementing the machine learning control algorithm on hardware rather than building the physical control system environment. Building one of these systems on your own is not easy, it requires an extensive knowledge base in mechanical and electrical systems, as well as, embedded programming.

We started our machine learning algorithm on some Custom OpenAI Environments to minimize algorithm development time and also to shorten the time spent training on hardware.  The training is broken into several phases.  Each phase aims to improve the AI in some way, just as a person learning a sport would do practice drills.  The first phase focuses on flipping the pole up, and balancing the pole once it is up, separately.  The second phase tries to combine the two actions into a single event.  The third phase is the hardest, it flips the pole up, balances it and then deals with random force injections into the system, simulating wind, or a person instantaneously adding force to the cart/pole.

  • 1 × NEMA 23 Stepper Motor 23HS22-2804S
  • 1 × Digital Encoder 600PPR LPD3806-600BM-G5-24C
  • 3 × Solid Wheel Kit SKU 480
  • 11 × GT2-2M Timing Belt SKU 470-By-the-Foot
  • 1 × V-Slot Gantry Plate (20mm) SKU 2055

View all 13 components

  • First Movement

    Tom4 days ago 2 comments

    Finally got some of the bugs worked out of the electronics and software. It was exciting to get things up and running, so we put together some simple concepts. On this one we set up "P" proportional controller with a negative gain such that it would cause the system to go unstable. The reason for this was to see if we had enough speed and torque to flip the pendulum upright. 

  • ​Tensorflow-gpu

    Alex04/13/2018 at 15:56 0 comments

    Last night I installed all of the cuda and cudnn drivers (Not as difficult as I thought it was going to be) for my graphics card in an attempt to speed up the training time.  It did not work and I think the reason why is because each predict session is interrupted by some cpu instructions that interpret the output of the of the predict thus causing a bottle neck in the data flow.  My new plan is to see if by using some tensorflow libraries to do the same thing if I can eliminate/minimize these instructions so that it runs faster.  I also plan on looking into running multiple OpenAI sessions simultaneously to help speed up the prototyping process.

  • Disturbance Rejection

    Alex04/08/2018 at 22:26 2 comments

    After adding in the the disturbance rejection environment I trained the Tensorflow model some more to see where it would end up.  After about another 4 hours of training it was performing spectacularly.  It also learned to give up immediate reward for future reward as you can see it will allow the pole to swing down in favor of not going off of the screen which would end the episode immediately.

  • Chips Flying!!

    Tom04/08/2018 at 22:12 0 comments

    I was able to get the bracket designed to mount the encoder to the bearing carriage as shown below. 

    We use a X1 Mill that was converted to CNC running EMC2. 

    Here is an action shot of the chips flying. 

    Here is the final product. CNC machining is just way to much fun. 


    Tom04/08/2018 at 18:06 0 comments

    The hardware arrived!! We got it put together late last night and hopefully will be able to get some motion tonight. We need to create some custom parts to hold everything together. The system uses a piece of extruded aluminum as the rail. The cart currently has four bearings and rides really smooth on the rail. The whole system is belt driven using GT2 belt. The thing that I really like about this design is that the motor will sit stationary during operation which will reduce the inertia of the cart. I am concerned that the little NEMA 17 stepper motor will not have sufficient speed and torque for the job. We have two backup options, a NEMA 23 stepper motor or a 24V servo system. Each will require some more custom parts and potentially some software rework. We will start with the NEMA 17 and see where it gets us. 

  • Disturbance Rejection

    Alex04/08/2018 at 15:43 0 comments

    This morning I created a new environment to add to the practice drills for the AI.  The disturbance rejection environment adds a random force to the cart, on average, every 600 turns.  At the same time a random force is also added to the pole, on average, every 400 turns.  This additional practice drill is supposed to help the AI learn how to deal with outside forces, such as a finger instantaneously adding a force to the pole or cart.  You can check out the new environment at my Github

  • Preliminary Training Progress

    Alex04/07/2018 at 22:26 0 comments

    This is the progress after 4.5 hours of training.

  • Inverted Pendulum Environment

    Alex04/07/2018 at 21:07 0 comments

    Today I built the inverted pendulum environment with OpenAI's framework.  You can download it at my github

View all 8 project logs

Enjoy this project?



jaroslaw.ceborski wrote 2 days ago point

 I like your ideal a lot, for outside the box thinking! Do you think it may be applicable to self balancing robot? Instead of using PID controller?

  Are you sure? yes | no

Alex wrote 2 days ago point

Yes, of course.  Our whole project is based around opening the doors for bigger and better projects, such as, a self balancing robot.

  Are you sure? yes | no

Alex wrote 2 days ago point

There's even an OpenAI environment already created for it!

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates