A Self-Driving Car using a Raspberry Pi Zero | Details

Overview

This project describes how to build a micro Self Driving Car (uSDC), a smaller version of Ogma Corp's 1/10th scale R/C based self driving car. Both versions use a controller to drive the SDCs and a front-facing camera, to provide input into an online Machine Learning predictive algorithm that learns how to autonomously drive around a track. Details of both version can be seen on GitHub here.

The Machine Learning algorithm uses an online learning technique known as Sparse Predictive Hierarchies (SPH). The predictive hierarchies take as input streaming data, in this case a camera image and steering information, and predicts (infers) what the next steering information it expects to be shown. The predicted steering information can also be fed back into the predictive hierarchy to enable autonomous driving behaviour.

A Unity based simulation OgmaDrive was used to prototype the SDC, using C# scripts and Ogma Corp's EOgmaNeo library (that contains the SPH implementation).

Control of the uSDC

There are two Python3 scripts included in the GitHub repository to test the uSDC:

motorTest.py - A script to test the Explorer pHAT, Steam controller, and drive the uSDC around.
main.py - The main script containing using the EOgmaNeo library. Allowing for learning, inference, and self-driving on the RPi ZeroW.

Steam controller daemon

Both of the included python scripts require the Steam controller daemon to be running. After the daemon has been started, it emulates the Steam controller as a Xbox controller. PyGame can then connect to and obtain joystick and button information. The motorTest.py and main.py scripts can then make use of the controller to drive the uSDC around.

Starting the daemon requires the following bash command:

sudo python3 ~/steamcontroller/scripts/sc-xbox.py start

and the following command to stop the daemon:

sudo python3 ~/steamcontroller/scripts/sc-xbox.py stop

The motorTest python script

To test the Steam controller and Explorer pHAT motor driver, the following commands and script can be used:

cd ~/EOgmaDrive/Configuration3
sudo python3 ~/steamcontroller/scripts/sc-xbox.py start
sudo python3 motorTest.py
sudo python3 ~/steamcontroller/scripts/sc-xbox.py stop

Similar controls as used in the main.py script are used here:

The joystick controls left or right steering,
Trigger buttons apply forward/backward drive,
The `A` button cleanly exits the script.

The `trimming` global variable allows for trimming of the forward/backward motion, so that applying motion using the triggers ensures that the uSDC travels in a straight line.

The main python script

The `main.py` Python3 script is used for training (manual driving) and inference (self driving). All processing occurs on the Raspberry Pi ZeroW board.

It can be started using the following bash commands:

cd ~/EOgmaDrive/Configuration3
sudo python3 ~/steamcontroller/scripts/sc-xbox.py start
sudo python3 main.py
sudo python3 ~/steamcontroller/scripts/sc-xbox.py stop

The uSDC can then be controlled using the Steam controller with:

The analogue joystick controls left or right steering.
Trigger buttons apply forward/backward drive.
The `A` button toggles between training and prediction modes.
The `B` button saves the current camera image.
The `X` button saves the current state of the hierarchy.
The `Y` button exits the script.

If you train and save out the current state of the hierarchy, that saved state can be reloaded by starting the script with a `load` parameter. For example:

sudo python3 main.py load

Note: Saving the hierarchy to the a file on the SD card can take a minute or so to perform. Console text will announce when saving starts, and also when saving has completed.

If the uSDC doesn't travel in a straight line using one of the trigger buttons, a `trimming` global variable can be modified to compensate for any drift. For example this could be set to `trimming = 0.2`

An `RGB` image is captured from the camera module at each time step. This is converted into a grey scale image. And an EOgmaNeo pre-encoder (`ImageEncoder`) is used to transform the dense camera image into a sparse chunked representation, that alongside the current motor values, is sent into the EOgmaNeo predictive hierarchy.

When the `A` button is initially pressed the `training` global variable is toggled. And instead of using the Steam controller input to steer the uSDC and send values into the EOgmaNeo predictive hierarchy (training mode), the hierarchy's prediction of the next motor steering values are used (inference mode). Be aware that during inference mode a constant forward drive is applied to both motors. Pressing the `A` button again can toggle back to training mode.

Testing the uSDC

Online learning only occurs within the `main.py` script and the EOgmaNeo predictive hierarchy when forward drive is active. To navigate around a track, a number of laps are required before the predictive hierarchy can confidently make steering predictions.

With learning only enabled during forward motion, it's possible to reverse out of steering mistakes made when driving around a track.

The more laps of a track that occur during training mode, the more confident the predictive hierarchy will be when switching to inference mode (that is, steering only use hierarchy output predictions and current camera input).

As seen in our YouTube video of the uSDC, we use an [InfiniTrax](http://infinitrax.com/product.php) modular track to perform more elaborate training and inference testing.

The simplest way to check that everything is working is to perform the following:

Apply constant left/right steering.
Apply forward drive so that the uSDC spins on the spot.
After 30-60 seconds, release the joystick and trigger button.
Press the `A` button, and the uSDC should spin on the spot.
Press the `A` button again to toggle back to training mode.

Try the above but steering in the opposite direction. And test out saving and reloading of the hierarchy to see whether it remembers how to spin on the spot.

Project Details