Close
0%
0%

Augmented Mekamon Robot

Take control of the Mekamon robot using Bluetooth and give it autonomous behaviour using a Pi and custom camera head.

Similar projects worth following
Reach Robotics’ MekaMon is a gaming robot with no environment sensors but cool four-legged motion, normally controlled with a phone app. This project demonstrates how it can be enhanced and made into a fully autonomous robot.
I used a packet sniffer to work out how to issue the right commands over Bluetooth to take control of the Mekamon. I put a Raspberry Pi 3 on it and built a gimballed camera head with a standard Pi camera and an ultrasound rangefinder on it (which I didn’t end up using).
I produced a demo that makes the head follow a coloured ball and walks the robot around to keep the ball in range. This used Python and the OpenCV, Adafruit BluefruitLE, and PiCamera libraries.
I revised the design to use a Pi CM3 for its dual camera inputs, upgraded the head to a custom stereo camera, and made 3D-printed parts to replace the messy aluminium and perspex parts. I have made some progress towards doing visual SLAM + navigation using ROS and RTAB-Map.

This was a university project for my Robotics degree at UWE Bristol, UK. My aim was to see if Mekamon could be adapted cheaply to be used in education. I’ve just finished the degree but I’m interested in continuing this project or seeing other people pick it up as there’s a lot of ideas I didn’t have time to complete or even try.

The project as it is does not damage, or change, anything on the robot; it’s a stack of parts that simply zip-ties on top of it.

I have uploaded example code and the .STL files for the printed parts.

I have a ton of notes on the problems I encountered and can add details if anyone needs them.

As the design changed over time there’s effectively 2 main versions of this project with different levels of complexity:

Simpler & cheaper:

  • Raspberry Pi 3
  • Single standard Pi camera
  • Python + OpenCV + Bluefruit BLE library
  • Great for doing ball-following demo

More complex:

  • Pi CM3 Lite + official breakout board (which is ~£100 alone)
  • Dual pi-compatible cameras of better quality than the standard one.
  • ROS install, allows much more sophisticated stuff.

Clearly, it’s not a mature design but an experimental setup that you would adapt to suit. If you just need one camera you might be ok with a Pi Zero, which could allow you to mount the computer on the gimbal head, making it much smaller and avoiding the awkward FFC ribbon cables. (I will try this if I get monocular visual odometry working.) Or you might not need the stabilising gimbal at all, or use different sensors, different computer and so on.

The main thing with this project is it shows the Mekamon can be hacked! 

Update: Someone I know is releasing an alternative to the official CM3 board, it's way smaller and very cool. Check it out here: http://stereopi.com/

py - 17.36 kB - 06/22/2018 at 11:42

Download

py - 2.78 kB - 06/22/2018 at 11:42

Download

py - 6.06 kB - 06/22/2018 at 11:42

Download

py - 8.75 kB - 06/22/2018 at 11:42

Download

launch - 731.00 bytes - 06/22/2018 at 11:42

Download

View all 15 files

  • 1 × Raspberry Pi CM3 Lite Or other Pi
  • 1 × Raspberry CM3 I/O breakout board if using CM3
  • 1 × Zip ties! inc. a big one to strap the bottom plate to the Mekamon
  • 1 × MicroSD card - Sandisk Ultra, 32GB, class 10 only because 8GB wasn't enough for installing OpenCV
  • 1 × CPU heatsink - scavenged part from an old graphics card cooler kit

View all 19 components

  • 1
    Gimbal

    The gimbal is an implementation of the open source project STorM32bgc http://www.olliw.eu/2013/storm32bgc/ using a generic v1.32 board from Gearbest, and £10 gimbal BLDC motors from Hobbywing.

    Here is the reference for the STorM32 Remote Control commands for the v1.32

    board (there is a newer ref for newer boards):

    http://www.olliw.eu/storm32bgc-v1-wiki/Serial_Communication

    and pin reference for the same:

    http://www.olliw.eu/storm32bgc-v1-wiki/Pins_and_Connectors

  • 2
    BLE control of Mekamon

    (see mmdriver_node.py in the files)

    Reach Robotics haven’t yet released an API or anything to allow custom control of the Mekamon. I couldn’t make sense of the BLE services and characteristics. I took it apart and saw it uses a Nordic nRF51822. (and its brain is a PIC32 (and it has a connector called “Shhh! secret connector!”)). I thought that maybe it uses the standard Nordic UART service, and one shows up on a Android BLE info app.

    I watched the packets coming from the Mekamon phone game app using Wireshark and an Adafruit nRF BLE packet sniffer. It took me about a week to work out how to talk to it. It is using the UART service, not BLE characteristics. When it connects there’s a lot of mysterious stuff that goes by but then it settles down and sends the same command repeatedly until you use the onscreen joysticks in the app, when they change accordingly – these are clearly movement commands; 3 signed bytes for fwd-back, strafe, turn. If I replicated all this exactly I could take control. I worked out by elimination that a lot of the startup stuff is unnecessary except for two.

    The frame structure is a header byte, a command byte, optional payload bytes, a checksum byte and a zero terminator byte. The meaning of the first byte was unknown and there was a weird behaviour where bytes that should be zero get replaced with 1-5, depending on which of the 3 axes were active. I laboriously made a table of what causes what, before someone told me about Constant Overhead Byte Stuffing!

    So for example, when in joystick mode but idle, it sends hex:

    02 06 01 01 01 0C 00

    which is COBS header, command #6 for motion, zero on the 3 axes, checksum, terminator.

    Adafruit Python BluefruitLE source & install guide:

    https://github.com/adafruit/Adafruit_Python_BluefruitLE

  • 3
    Blob-following demo

    The file for the non-ROS blob demo is “non_ROS_blobtracker.py” - Requires OpenCV and Adafruit_Python_BluefruitLE

    This demo is quite simple and far from perfect.

    BluefruitLE with BLueZ and DBus can be unreliable. It’s not clear to me where the fault lies. Sometimes it just needs:

    sudo hciconfig hci0 up

    or a reboot.

    ROS version:

    This consists of these nodes running on the Pi:

    • Camera publisher - custom Python node
    • Image republishers - standard library node - compresses images for optional use by other machines over the network.
    • Blob tracker - custom Python node - finds objects and makes motion decisions
    • Robot controller - custom Python node - converts ROS motion commands to Meka-mon motion commands and sends them over BLE.

    First, create ROS packages “mmstereocam” and “mmnavigate” and copy the provided files to make this ROS workspace structure:

    On the Pi via SSH, run:

    roslaunch mmstereocam blobtracker.launch

    Then use rqt_image_view  from your PC  and select /stereo_camera/image_blob/compressed, and you should see it highlight your coloured object while the gimbal follows it. You will need to do some tuning for your gimbal setup, camera and target object.

    Power up the Mekamon and run:

    rosrun mmnavigate mmdriver_node.py

    This should present a list of nearby BLE UART devices. Select the Mekamon. It should connect and display a stream of commands being sent, and the robot will now walk in response to the ball.

    rqt_graph looks like:

View all 4 instructions

Enjoy this project?

Share

Discussions

Meteorinca wrote 01/30/2019 at 23:55 point

Congrats at getting a job there! 

https://mekamon.com/blog/we-liked-this-guys-mekamon-mod-so-much-job

Question: Is there an easier way now with v2 to control it via bluetooth & raspberry pi?

  Are you sure? yes | no

Wes Freeman wrote 02/01/2019 at 13:56 point

Thanks! Though I've moved on now :( But no, V2 is the same in that regard. I worked on alternative interfaces but I don't know when or if that stuff will be released. I'm under NDA so I can't give details sorry.

  Are you sure? yes | no

AnykeyNL wrote 11/12/2018 at 12:33 point

Hi,

Instead of bluetooth sniffing, did you checkout the Mekamon Swift playground files. They come with the source swift files, so you can see all the bluetooth code they send themselves :-)

https://mekacademy.mekamon.com/swift-playground/mekacademy/MekAcademy.playgroundbook.zip

enum PacketType: UInt8 {
    case AddOnGet = 1
    case AddOnSerialGet = 36
    case Animation = 2
    case Transform = 6
    case GameState = 7
    case BatterySetup = 10
    case FirmwareVersion = 11
    case GaitSetAll = 13
    case ConnectionEstablished = 16
    case SetHeadColour = 46
    case JointAnglesSetup = 60
}

  Are you sure? yes | no

Wes Freeman wrote 11/13/2018 at 10:14 point

Hi, I wish I'd known about that at the time!

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates