The central conceit behind the mimic robotic arm is the wireless control of the robotic arm using muscle sensors (EMGs). Before getting initial any of the details, let's show it working.

Now that you've seen it working, the question is how?

Despite looking somewhat complicated - looking at you breadboard - the guts of the project are actually quite simple. Let's break it down into the mechanics, electronics, and software components.

Mechanics:

The basic construction of the arm is similar to any of a thousand CAD models you can find on the internet for a toy robotic arm. We happened to use this model, but the choice is totally optional.

From there it was a simple matter of 3D printing and assembly to get a basic platform to work with. One thing to note is the addition of a base plate. That's really key to stop it from tipping over.

With the easy mechanics out of the way we can move on to the electronics:

Electronics:

On the surface, the electronics behind the EMGs look very complicated. I mean just look at this monstrosity of a breadboard!

But really this is just once circuit duplicated multiple times for each of the motors. In reality, the actual mechanism is rather simple.

There are only two stages. First, an op-amp based amplification circuit and a level shifter scales the signal into the 0-3.3V range of the ESP32 chip.

Second, a series of high pass filters get rid of the low frequency noise. As you can see from this oscilloscope reading, there is a ton of low frequency noise when a muscle adjacent to the one you are measuring gets flexed. By getting rid of this noise you prevent neighboring probes from interfering with one another.

Once everything has been properly filtered and amplified it can be sent directly to the analog digital converter pins of the ESP32 board.

Of course, you don't have to use a bread board either. Eventually we replaced it with a nice looking custom PCB.

One other thing worth mentioning here is that there is also simpler setup for joystick control.  They don't need filtering/amplifying so the leads can hook directly to the analog to digital pins on a second ESP32. This allows for control of the arm without the hassle of putting on a bunch of probes. It was very useful for testing.

Regardless of whether it is the joysticks or EMGs, once the signal is on the ADC pins it is time for software to take over.

Software:

The software aspect of the project is where things really start to get complicated. Broadly speaking it can be split into four different stages. 

+ Signal Processin

The first stage is signal processing. In this stage the client EMG on the sensor interprets the incoming data and prepares an instruction for the servos based on that interpretation. To understand how, it is first necessary to grasp the concept of velocity control. In a velocity control scheme the servos are not explicitly told to go to any particular angle. Instead they are told to increment, decrement, or stay still for each clock cycle. Over multiple clock cycles if the instruction is increment forward, for instance, the arm appears to have an outwardly smooth forward motion. Velocity control is at the heart of the signal processing step. For each motor on the arm, the client reads the incoming data and decides if it is above some threshold. If it is, then the client knows that the corresponding motor needs to increment. If the data is below the threshold that corresponds to a rest state. To move backwards there are extra sensors on the arm that measure the extension of the arm. When those exceed the threshold the client know the motor needs to reverse.

Once the client has decided what each motor needs to do, it packages it all up in the JSON format. The signal is now officially processed.

+ Message Sending

This is where the second phase comes in: message sending. In the message sending phase the sensor based clients send their JSON formatted strings to the server based on the arm. This is accomplished using the standard wifi.h libraries. The sensor ESP32s are configured as clients and given the IP address for the server based on the arm.

One thing of interest to note here is the hardware timer. A hardware timer is used to gatekeep the speed at which these messages are being sent. This is important for two reasons. First, it stops the messages from coming in too fast and overwhelming the server. Second, the speed of the timer indirectly sets the speed of the arm. The arm's angle gets updated one degree per message, so sending more messages per second means more degrees per second.

+ Client Selection

With JSON messages coming in from both the EMG and joystick clients, the server needs a way of telling who to listen to. This is accomplished using a web server. The web server creates a web page with buttons on it that allow for selection between modes. It looks like this:

When you hit a button on this webpage a communication is sent back to the server that sets a flag declaring whether the EMG should be used or the joysticks. When a messages comes in and starts to get decoded by the string handler that flag is checked against an ID embedded in the incoming message JSON. If the ID doesn't match the flag then the message originated from the wrong client and gets ignored.

+ Servo Update

The last stage of the software involves actually updating the servos. In this step the properly screened incoming JSON is decoded using a string handler and its instructions are stored in various dummy variables. Next, a series of logical statements interpret the dummy variables and decide whether to increment, decrement, or hold steady each motor. Lastly the duty cycle (a way of describing the pulse width modulation signal sent to the motors) for each motor is modified accordingly.

This entire process then repeats itself 40 times a second resulting in a clean motion.

Well anyway that's the project! Thanks for sticking around this long. If you'd like to see the code or anything else about the project, it's all included in a zip file in the files section on this page.