Close

First findings and how "the real-time mode" works

A project log for Custom control app for Mind Designer Robot

This project is about reverse-engineering the BLE communications protocol and getting a custom app working

adriajunyent-ferreadria.junyent-ferre 12/27/2019 at 13:270 Comments

By following the procedure I described in my previous update, I was able to obtain a log file with all the data from the BLE communications that took place between my phone and the Mind Designer Robot. One issue with this procedure, compared to the procedures I had used in the past with wifi communications, is that it wouldn't allow me to check the log in real time. This meant that in order for me to link the messages in the log with the actions of the robot, I had to make it follow a know pattern with known timing. I started with the "real-time control mode", where you can control the car making it move forward, rotate or move backward depending on which arrow you press. I pressed "forward, forward, forward, right, backward, left, forward" and left periods of about 5 seconds between each set of commands. I also wrote down the time when I started.

The log in Wireshark showed many things. I filtered the entries making Wireshark show the messages that had something to do with the MAC address of the robot, this left only the ATT messages being displayed. Btw, an identical procedure with better pictures is show in this article from Medium.

Looking at the messages, it was fairly easy to get an idea of what was going on: the first lot of messages between the app and the toy were "service discovery requests" (please forgive me for using the wrong terminology here) from the app asking the toy to provide the list of BLE services it offers. This same list can be obtained from any BLE device using a very handy Android app called nRF Connect, which helps double-checking if things make sense.

 This is the list of services as seen from nRF Connect:

Some services are generic and I presume pretty much any BLE device will feature them, whereas some others listed as "unknown" are likely used to implement very application-specific functions.

By looking at the I/O graph in Wireshark, which plots the frequency of messages being exchanged over different periods of time, it was fairly straightforward to detect the period that corresponds to my key-presses in the application:

As you can see, the ~5 second gaps are pretty obvious and by looking at the messages that were exchanged in these periods it was possible to identify the purpose of some of them:

I marked the messages that matched the beginning of the ~5 second periods and focused my efforts on reading what was being sent from the phone.

Apparently, the phone was accessing a service with UUID 0000fff0-0000-1000-8000-00805f9b34fb and writing to a characteristic within that service with UUID 0000fff2-0000-1000-8000-00805f9b34fb. Values were written every ~100ms with alternating values: the value 0x31303030 was very common and it would follow other values (e.g. 0x30303031). 

nRF Connect allows you to connect to BLE services and read or write to the characteristics by had. This helps testing if messages make sense. Alternatively, if a sequence of messages with "precise" timing was important I could have used a BLE module with Arduino, for example this one from Adafruit with the nrf8001 is very easy to use.

I tried writing a couple of values to the aforementioned UUID and I figured the following:

0x31303030 makes the robot stop

0x30303031 makes the robot start moving forward

0x30303032 makes the robot start moving backwards

0x30303033 makes the robot start turning to the right

0x30303034 makes the robot start turning to the left

Apparently, the 100ms gap between issuing 0x30303031 and 0x31303030 that I had observed were the shortest time the Android app would allow the user to command the robot to move. If one of the "start moving" commands above was sent, the robot would move indefinitely until the stop command was received.

I noticed a couple other things: 

- the robot would hang the BLE connection if nothing was sent to it for a certain period of time (about a minute), when this happened, the robot would play an audio sample saying "I'm disconnecting".

- when using the official Android app the robot will say "I'm ready to play" or something along these lines when a playing mode was chosen. First I thought this was a sign that the robot had to be told that one wanted to operate it in "real time control mode" or any other mode. However, it was possible to send the motion commands I describe above as soon as the BLE connection starts. Therefore, I'm inclined to think that the Android app can send messages to the robot to command it to play audio samples and it does so without any implication on the operation logic of the robot. 

The real-time control mode is very different from the other modes of operation of the robot, where it follows motion commands similar to those in Logo programming (ie "move forward 1 cm, rotate to the right by 90 degrees", etc). I haven't opened the case of the robot but given how precise the motion is, I'm inclined to thing that it uses stepper motors or even DC motors with simple encoders (given the noise it makes when it moves). Therefore, it wouldn't make sense for the Android app to rely on the timing between the motion commands to make the robot move. Therefore, I suppose that it is possible to send Logo-type commands to control the robot that are different from the "indefinite" motion commands I describe above. My next task will be to find how these commands are sent.

The stuff I described in this post took me a couple of hours to figure out. I was a bit disappointed to see that the commands where identical to the actions associated to the key pressed because I was hoping I would be able to implement a fine joystick control to get the robot to move around complex paths in real time. Still, I'm eager to find out more about the other commands the robot is ready to take from the application, who knows if there's a way to give commands of displacement of the individual wheels, which could be used to implement a custom path planning.

Stay tuned for more updates.

Discussions