Getting the numbers

The images are captured with the notebook's camera with OpenCV and then processed to get more clear images for the numbers, then with Tesseract I get the numbers in text format.

Choosing where to slide

Using PyBrain I create a neural network (multilayer perceptron) and balance its weights with a genetic algorithm using PyEvolve.

The input of the network is a matrix with the log2 of each number divided the maximum log2 number, and the maximum number of the output is the direction where we should slide the stylus.

A console-terminal version of 2048 was made for training.

 Github AI Repo link

Sliding the stylus pen

To slide the stylus I used a parallel SCARA I made long ago, it uses an arduino to drive the servos.

Here is an image to understand the math behind the code.

Github Parallel SCARA Repo link