Since the last update, I've continued making bits of sporadic process.

I've:

• Used the FIT SDK code generator to get some C code that can write .fit files
• Written a network service to handle WiFi connectivity
• Written a logging service to write GPS data to a .fit file
• Put it all together and generated some .fit GPS tracks.

So I'll break those down a bit more here.

FIT SDK

The FIT (Flexible & Interoperable data Transfer, apparently) file format is a very flexible, openly documented data format used by a bunch of different GPS/activity tracker products, with the main driving force seeming to be Garmin (perhaps via their acquisition of Dynastream?).

There's an SDK available for download, which includes the various specifications and a tool for generating C/C++/C# and Java code to encode and decode fit files: https://www.thisisant.com/developer/resources/downloads/

As far as flexible data interchange formats go, FIT is fairly sane and well thought out.

The whole file format is based on "messages", where a message is just a set of fields. Each field has a type, which can either be a base type like "uint8" or "string", or a more specific enumerated type such as "WorkoutStep" which uses one of the base-types as its underlying representation. All of these messages and types are defined in a big spreadsheet in the SDK.

A FIT file consists of a header stating what kind of file it is ("Settings", "Device", "Activity" etc), and a series of messages with the actual content.

The file protocol allows for only encoding the fields which you actually care about, so even though the "Lap" message which stores data about a lap has 114 fields defined - from start time and position to "avg_saturated_hemoglobin_percent" - you only actually have to encode the fields you care about, and all decoders will gracefully handle the difference.

While I'm usually pretty cynical about proprietary file formats, I really do think FIT's a pretty elegant and effective design, and it's worth having a read through the documentation in the SDK.

Code Generator

FIT has the concept of a "profile" which is the subset of messages and fields that a particular device supports, and the code generator can generate suitable code for that profile. So, in order to generate just the code and definitions that I need for my particular project, I had to write (modify) a "profile.csv" file, which is the input to the code generator. In the .csv file you state which messages and fields you want, and then the Windows-only code generator will spit out C/C++/C#/Java code for it for you.

I wrote a very thin wrapper over the top of the generated code, which gives me just enough to create the FIT files I need. You can see the result here: https://github.com/usedbytes/tbeam-fit

Network Service

The next piece is the network service, which interfaces with the ESP32 WiFi and HTTP request libraries.

It's got two main responsibilities:

• Monitor the various network events from the ESP32 network stack, and report the relevant ones to other services which register for notifications, such as "network connected"
• Provide an interface to perform HTTP GET/POST requests

I'm not entirely happy with the request interface, but it does provide a pretty flexible way to perform network transactions, which should work fine for this project and is probably applicable to other projects, too.

The main interface is this structure:

struct network_txn {
    struct service *sender;

    esp_http_client_config_t cfg;

    // Content-Type header
    const char *content_type;

    // Callbacks for performing send and receive
    network_txn_send_cb send_cb;
    network_txn_receive_cb receive_cb;

    // Filled in by transaction handler
    esp_err_t err;
    int result;
};

 The caller fills in the 'cfg' structure with things like the URL, and provides two optional callbacks: 'send_cb' and 'receive_cb', then it sends a pointer to this request structure to the network service.

The network service will open a HTTP connection, and then call 'send_cb', which should send whatever data it wants to the server, which it can do in chunks. By doing this via a callback, I can have lots of flexibility for where the data comes from. For instance there might be a request were we just send a static string, in which case 'send_cb' would something like this:

esp_err_t string_send_cb(struct network_txn *base, network_send_chunk_fn send_chunk, esp_http_client_handle_t client)
{
    esp_err_t err = send_chunk(client, "Send my string", strlen("Send my string"));
    if (err != ESP_OK) {
        return err;
    }

    // We send an empty chunk to say that we're done
    err = send_chunk(client, NULL, 0);
    if (err != ESP_OK) {
        return err;
    }

    return ESP_OK;
}

Or, we might want to send a file, in which case the callback could read sections of the file into a local buffer, and then send it over the network in chunks without ever having the load the whole thing into memory.

esp_err_t network_file_post_send(struct network_txn *base, network_send_chunk_fn send_chunk,
                   esp_http_client_handle_t client)
{
    struct network_file_post_txn *txn = (struct network_file_post_txn *)base;
    char buf[512];
    esp_err_t err;
    int ret;

    while ((ret = fread(buf, 1, sizeof(buf), txn->fp)) > 0) {
        esp_err_t err = send_chunk(client, buf, ret);
        if (err != ESP_OK) {
            break;
        }
    }
    if (!feof(txn->fp) || ferror(txn->fp)) {
        err = ESP_FAIL;
        goto done;
    }

    err = send_chunk(client, NULL, 0);
    if (err != ESP_OK) {
        goto done;
    }

done:
    fclose(txn->fp);
    return err;
}

This would also be important if I implement heatshrink compression, as I would decompress on the fly before sending to the network.

Logging Service

The last new service is the logging service, which is currently pretty hacked together, and just creates a .fit file when it's started, and writes GPS data to the file until it's stopped.

Pulling it all together

So with all those pieces in place, I've made it so that as soon as the main service detects that network is available (via the network service), it attempts to upload all FIT files which are in the flash filesystem. If the upload is successful, it deletes the local copy.

As soon as the GPS gets a lock, the logging service is started, which starts recording GPS data. When the device is powered off, the file is finalised and stored to flash, ready for upload next time the network gets connected.

With a dead simple golang HTTP server, I can receive the files and save them to my computer:

package main

import (
        "fmt"
        "io/ioutil"
        "net/http"
        "os"
        "path/filepath"
)

func run() error {

        fmt.Println("Listening...")

        http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
                err := r.ParseForm()
                if err != nil {
                        fmt.Println("failed to parse form")
                        http.Error(w, fmt.Sprint(err), http.StatusInternalServerError)
                        return
                }

                v := r.Form.Get("filename")
                if v != "" {
                        b, err := ioutil.ReadAll(r.Body)
                        if err != nil {
                                fmt.Println("failed to read body")
                                http.Error(w, fmt.Sprint(err), http.StatusInternalServerError)
                        }

                        fname := filepath.Clean(filepath.Join(".", v))

                        fmt.Println("Writing to", fname)

                        f, err := os.Create(fname)
                        if err != nil {
                                fmt.Println(err)
                                http.Error(w, fmt.Sprint(err), http.StatusInternalServerError)
                                return
                        }
                        defer f.Close()

                        _, err = f.Write(b)
                        if err != nil {
                                fmt.Println(err)
                                http.Error(w, fmt.Sprint(err), http.StatusInternalServerError)
                                return
                        }

                        fmt.Println("Done.", fname)
                }
        })

        return http.ListenAndServe(":8080", nil)
}

func main() {
        err := run()
        if err != nil {
                fmt.Println(err)
                os.Exit(1)
        }
}

And my favourite part is that the generated FIT files are consumed just fine by the Garmin Connect website :-D

Here's a little walk around a local green:

Complete with a couple of stops to avoid cows:

It's been a little while since I posted an update here, but I have been doing some work in the meantime, slowed down a bit by illness, decorating, and the new 8 GB Raspberry Pi 4 getting released.

Let's start with that:

ESP32 development on Pi 4

<tangent>

I've been keen for quite some time on using Arm powered machines for as much as I can. Back in 2103 I bought the first Samsung Chromebook which used a dual-core Cortex-A15 Samsung Exynos chip, and I used that as my main laptop until I eventually bought an Intel Inside Thinkpad at the end of 2016. My main machine at my day job is an Arm powered HP Envy x2 with a Snapdragon 835, but at home I've been using an i5 2500k desktop for the past 10 years. I've been itching to try and replace at least some of that machine's time with something Arm based, but the devices just haven't existed. They're all too underpowered (Cortex A53 cores), too low on RAM (less than 8 GB), or too pricey (the $4000 Ampere eMAG desktop). I came very close to picking up a Honeycomb LX2K, but before I actually did, the 8 GB Pi 4 was announced (somewhat out of nowhere) - so that was a no-brainer: A usable amount of horsepower (4 x A72), with an excellent software ecosystem, for less than £100. I bought one on release day.

So, since then I've been leaving the i5 desktop turned off and using the Pi 4 almost exclusively. I've installed tuptime to see how much time I use each machine for, and I expect I'll do some more blogging about my experiences over on my blog at https://blog.usedbytes.com.

Anyway, how is any of that relevant? Well, this ESP32 project is my primary project at the moment, so I need to be able  to work on it on the Pi 4.

</tangent>

To set-up the ESP-IDF development on a Pi 4 I did have to do some messing around. I'm running the new 64-bit Raspberry Pi OS image, which means it's a native arm64 system. ESP-IDF doesn't have any support for arm64, and therefore some hoops need jumping through.

Building Toolchains

Espressif do provide builds of their tools for 32-bit Arm, which we can use. They also provide source code for all of them. I started out by building the Xtensa toolchain using Espressif's crosstool-ng fork, and their decent instructions: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/linux-setup-scratch.html

It took about 2 hours on the Pi 4, building on an SD card, which to be honest isn't too bad (my desktop probably would have taken half that. I'm sure a fast modern high-core-count x86-64 machine like AMD's Threadripper could do it in minutes).

I found that this was only the first step. Upon trying to run the ESP-IDF install.sh script it started complaining about the other tools I was missing. Frustratingly, the first one was xtensa-esp32s2-elf-gcc - You need a whole separate toolchain for the new "S2" version of the ESP32, even if you aren't planning to target it!

Anyway, I set up crosstool-ng to build that toolchain too, and left it another two hours.

Installing 32-bit Arm prebuilts

After that, you need the binutils for the low-power coprocessor, as well as an openocd build - and I didn't feel much like building those from scratch so I started trying to use Espressif's 32-bit Arm prebuilt packages.

There are some threads on the Espressif forums about modifying the "tools.json" file to add an entry for arm64 which refers to the "linux-armel" binaries, but I just went the manual route of downloading the packages and putting them on my $PATH.

Next issue is when you try to run any of the arm32 components, you see the unhelpful error:

$ openocd
-bash: /home/pi/sources/esp/openocd-esp32/bin/openocd: No such file or directory

I've done enough working with Arm boards to know that this usually indicates a problem with the dynamic linker/loader. The reason the message is so unhelpful is because the program really can't do anything much until it opens the loader - and that's failing.

We can use the "file" utility to get a look at what's up:

$ file `which openocd`
/home/pi/sources/esp/openocd-esp32/bin/openocd: ELF 32-bit LSB pie executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=f229b58b5b2556ebc80d4e826b8fe1c1e8e229ae, stripped

Here's the relevant bit: 

interpreter /lib/ld-linux.so.3

And:

$ ls /lib/ld-linux.so.3
ls: cannot access '/lib/ld-linux.so.3': No such file or directory

Fine. So the loader doesn't exist. This is because the application was built for a single-architecture (not multiarch) 32-bit Arm system, where the loader is simply called /lib/ld-linux.so.3. However, this Pi 4 is a 64-bit multiarch system, with two supported architectures: AArch64 (64-bit arm) and "armhf" (which is basically 32-bit Arm). So we have these two loaders:

$ ls -1 /lib/ld-linux*
/lib/ld-linux-aarch64.so.1
/lib/ld-linux-armhf.so.3

The armhf loader should work OK (I'm not 100% sure if there can be problems related to hard versus soft float?), so we can just set up a symbolic link to point the loader the application is expecting to the armhf loader and we're in business:

$ sudo ln -s /lib/ld-linux-armhf.so.3 /lib/ld-linux.so.3
$ openocd -v
Open On-Chip Debugger  v0.10.0-esp32-20200526 (2020-05-26-09:28)
Licensed under GNU GPL v2
For bug reports, read
        http://openocd.org/doc/doxygen/bugs.html

Next, we need to persuade the ESP-IDF scripts to use our local tool installations instead of trying to download them itself. The relevant script is "export.sh" which is the thing you have to run to get the ESP tool(s) added to the environment. It calls idf_tools.py, which handles getting and loading tools, and it has a "--prefer-system" argument to make it just use already-installed tools instead of installing its own.

I didn't find a way to pass this to the export.sh script, so I just patched it:

$ git diff
diff --git a/export.sh b/export.sh
index 9be1b0f5d..db4d492ad 100644
--- a/export.sh
+++ b/export.sh
@@ -76,7 +76,7 @@ function idf_export_main() {
     # Call idf_tools.py to export tool paths
     export IDF_TOOLS_EXPORT_CMD=${IDF_PATH}/export.sh
     export IDF_TOOLS_INSTALL_CMD=${IDF_PATH}/install.sh
-    idf_exports=$(${IDF_PATH}/tools/idf_tools.py export) || return 1
+    idf_exports=$(${IDF_PATH}/tools/idf_tools.py export --prefer-system) || return 1
     eval "${idf_exports}"
 
     echo "Checking if Python packages are up to date..."

 The last part is making sure we have the appropriate Python virtual environment set up. This would normally be done by the "install.sh" script, but we can't run that due to the unsupported architecture. Thankfully, idf_tools.py does all the heavy lifting, and you can ask it to only set up the Python environment. Oh, but you need to make Python 3 your default interpreter first (Why oh why do the Pi foundation still ship Python 2.7 by default in 2020?) And, you'll also need g++ for one of the python dependencies:

$ sudo apt-get install python3 python3-pip python3-setuptools
$ sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 10
$ sudo apt install g++
$ tools/idf_tools.py install-python-env

 And now, finally we can successfully export and run idf.py. It warns that a few tool versions aren't exactly what it prefers, but that should be harmless:

$ . ./export.sh
Setting IDF_PATH to '/home/pi/sources/esp/esp-idf'
Adding ESP-IDF tools to PATH...
WARNING: using an unsupported version of tool cmake found in PATH: 3.13.4
WARNING: using an unsupported version of tool openocd-esp32 found in PATH: v0.10.0-esp32-20200526
WARNING: using an unsupported version of tool ninja found in PATH: 1.8.2
Checking if Python packages are up to date...
Python requirements from /home/pi/sources/esp/esp-idf/requirements.txt are satisfied.
Added the following directories to PATH:
  /home/pi/sources/esp/esp-idf/components/esptool_py/esptool
  /home/pi/sources/esp/esp-idf/components/espcoredump
  /home/pi/sources/esp/esp-idf/components/partition_table
  /home/pi/sources/esp/esp-idf/components/app_update
  /home/pi/.espressif/python_env/idf4.2_py3.7_env/bin
  /home/pi/sources/esp/esp-idf/tools
Done! You can now compile ESP-IDF projects.
Go to the project directory and run:

  idf.py build

Back to business: Adding "Services"

I was unsure for a while how to proceed with the software part of the project from an architecture point of view (Not sure that's a good thing, as my day job is as a software architect, but heigh ho).

My microcontroller projects are generally simpler, bare-metal interrupt-driven affairs, but the ESP32 is a much more complex system with an RTOS and multiple cores, with a pretty complete C library with filesystems, sockets and all sorts. More akin to a full Linux system than a microcontroller. In this kind of environment, it seems like multiple tasks are the way to go, so I've written a simple message-based service architecture, where I have multiple tasks or "services", which can talk to each other by sending messages across queues.

The interface to the service manager looks like so:

struct service *service_register(const char *name, void (*fn)(void *self), UBaseType_t priority, uint32_t stack_depth);
struct service *service_lookup(const char *name);

int service_send_message_from_isr(struct service *service, const struct service_message *smsg,
                                  BaseType_t *xHigherPriorityTaskWoken);
int service_send_message(struct service *service, const struct service_message *smsg,
                         TickType_t timeout);
int service_receive_message(struct service *service, struct service_message *smsg,
                            TickType_t timeout);

int service_stop(struct service *service);
int service_start(struct service *service);
int service_pause(struct service *service);
int service_resume(struct service *service);
void service_sync(const struct service *service);

// For use by service "fn" routines only
void service_ack(struct service *service);

And a basic service implementation would look like:

static void simple_service_fn(void *param)
{
        struct service *service = (struct service *)param;

        while (1) {
                struct service_message smsg;
                if (service_receive_message(service, &smsg, portMAX_DELAY)) {
                        continue;
                }

                switch (smsg.cmd) {
                case SERVICE_CMD_STOP:
                        // Do "stop" stuff
                        break;
                case SERVICE_CMD_PAUSE:
                        // Do "stop" stuff
                        break;
                case SERVICE_CMD_START:
                        // Do "start" stuff
                        break;
                case SERVICE_CMD_RESUME:
                        // Do "start" stuff
                        break;
                case SIMPLE_SERVICE_CUSTOM_CMD:
                        // Do whatever custom cmd is meant to
                        break;
                default:
                        // Unknown command
                        break;
                }

                // Acknowledge the command
                service_ack(service);
        }
}

The lifecycle is similar to what Android uses. Start and Stop are intended to be infrequent and fully initialise/de-initialise whatever the service is managing. Pause and Resume should be lighter weight - just temporarily stopping activities to be resumed later.

I've implemented four services:

• Main service 
  • Basically the main task loop
  • Implemented as a service to allow it to receive messages from the other services
• GPS service
  • Manages the GPS module
  • Start - powers up the module and configures message/rate parameters
  • Stop - powers down the module
  • Pause - Not implemented, but will put the module to sleep without powering it off
  • It can publish "GPS locked" and "GPS position" events to other services
• Accel Service
  • Samples the accelerometer on a timer
  • Calculates a variance value per-second (will form the basis of a movement detector)
  • The accelerometer powers up and down instantly, so no distinction between Start/Stop and Pause/Resume
• PMIC Service
  • Handles the AXP192 power management IC
  • Allows other services to request voltage rails
  • Monitors the IRQ/power button
  • Will add support for battery monitoring etc.

I expect to be adding at least a few more services:

• Logging service
  • Handle writing the track data to files
• Network service
  • Keep an eye out for network connections and upload files
• LoRaWAN Service
  • TTN mapper and GPS tracking

Next steps

Now I need to tidy up the actual logging. I think I have decided to use the "FIT" file format used by Garmin devices. This is a format with a detailed open specification: https://www.thisisant.com/resources/fit-sdk/, but not such wide support in tools. I spent some effort adding file-writer support to a golang library: https://github.com/tormoder/fit, and to be honest I quite like it as a format.

So, next step will be to add support for creating FIT files on the device, containing GPS and accelerometer data.

I'm a bit worried about the file size. I've only got around 1-2 MB of flash available, which I estimate to give an hour or two of logging time with high-rate accelerometer data. I want the full accelerometer data to begin with, so I can use that to work out good road-quality metrics - perhaps I can get the data-rate down once I have done some analysis.

I'm also looking at compression - perhaps heatshrink: https://github.com/atomicobject/heatshrink, which should knock around 30% off the file sizes.

 As always, the code is on GitHub: https://github.com/usedbytes/tbeam

With the main components at least powered on, I need to start adding more control and purpose.

GPS Config

First, is being able to configure the GPS module. Instead of the default setting, I want to tell it to:

• Use one of its low-power modes
• Send only the data I need

u-blox have their own command/communication protocol for their GPS modules, called "UBX" - it's all detailed in the M8 Receiver description - Including protocol Specification document.

UBX is nice and simple. Two sync characters ('µb'), two bytes of packet type information, a length field, a payload, and a checksum. So first order of business is to write or find some code which I can use to send and receive UBX messages.

There are some libraries out there:

• Sparkfun have one for Arduino - but I'm not using the Arduino environment: https://github.com/sparkfun/SparkFun_Ublox_Arduino_Library
• Another Arduino library: https://github.com/loginov-rocks/UbxGps
• A full-featured implementation designed to run on Linux: https://github.com/KumarRobotics/ublox
• A generated implementation with its own domain specific language: https://github.com/arobenko/cc.ublox.commsds
• ...and so many more.

I really don't need much functionality, so I decided to just write my own. I only plan to support a couple of messages, and the core of my implementation is around 200 lines of C - in this case I think the small targeted implementation makes sense instead of trying to integrate one of the larger more featureful libraries.

I'm still in prototyping mode, so there's no sensible encapsulation or interface definition - but the code is here:

https://github.com/usedbytes/tbeam/blob/b835efd1482f628b7bd0bb4345208d666a2bbf4f/main/gps.c

The crux of it is the receive_ubx function, which can be called with data received from the serial port. It searches for valid UBX messages, and returns them. It's stateful so you don't need to worry about messages getting split by the serial driver, you just call it each time you have data and it will return messages as it finds them. The intended usage is something like so:

#define BUF_SIZE 256
uint8_t serial_buf[BUF_SIZE];

while (1) {
        len = serial_receive(serial_buf, BUF_SIZE);

        uint8_t *cursor = serial_buf;
        while (len > 0) {
                // On success, receive_ubx() will update 'p' to point just
                // after the end of any message it finds.
                uint8_t *p = cursor;
                struct ubx_message *msg = receive_ubx(&p, len);
                if (msg != NULL) {
                        // Do something with msg
                        free(msg);
                } else if (p == cursor) {
                        // Something went wrong - bail
                        break;
                }
                len = serial_buf + BUF_SIZE - p;
                cursor = p;
        }
}

... but I've realised in writing this log, that:

1. I'm using it wrong in my code (I don't handle multiple messages in one serial receive)
2. The interface is horrible for using it right - you need to maintain two "cursor" pointers

So I'll probably refine that a bit.

Anyway, my transmitter and receiver code is functional, so I've used it to send a:

• UBX-CFG-PRT to set the serial port to only send UBX messages, not NMEA
• UBX-CFG-MSG to set it to send UBX-NAV-PVT messages (position updates) once per second

And then use the receiver to receive UBX-NAV-PVT messages and decode them into (Timestamp + Longitude + Latitude).

SPIFFS for data storage

Next up I need somewhere to log the GPS data. I don't have an SD card on my board (I kinda forgot about that - some of the TTGO boards do, but not this one), so the only non-volatile storage I have available is the main flash of the ESP32.

ESP-IDF has a spiffs implementation, so I've set up a 1 MB partition to write the data to. Right now I'm writing 16 bytes every second, which will fill up the megabyte in ~18 hours. That's OK - but perhaps a little on the short side. Also once I start logging more data (acceleration, battery etc.) it would reduce, so I'll probably implement some kind of simple compression scheme (store difference from previous sample or something).

Uploading the data

Having the data on the ESP32 is no use if I can't get it off.

I hack-hack-hacked the WiFi station and simple HTTP request examples to attempt to connect to my WiFi at boot, then attempt to upload any files in the spiffs partition via HTTP PUT requests. If that succeeds, it deletes the file from flash (though I'm not correctly receiving HTTP responses, so that's probably broken).

To handle the PUT requests, I found this simple gist which builds on top of the Python 2 SimpleHTTPServer to handle any PUT request. This is obviously a terrible idea in production, but fine for testing.

With all those things in-place, I can now record GPS data, and upload it back to my PC.

Processing the data

The last part is converting my raw binary records to something more useful like a GPX file. golang is my second language of choice, so I whipped up a simple go program which reads in one of the uploaded binary files and writes the data out to a GPX file. Code on Github: https://github.com/usedbytes/tbeam/blob/5eeb99a9c0dc5f6b80e3913240c0e602d334ae66/go/process.go

With that, I can read in one of my logs, save it out as a GPX, and then view it in Viking. Here's a little excerpt from my test ride yesterday:

This feels like good progress.

Next I really need to start tidying up the code so that I can more easily extend the functionality to add in the accelerometer data, some battery data, and a TTN mapper task.

Next up, I've designed a case to keep everything secure and together. There are a few T-Beam cases on Thingiverse already, but they all seem to include the OLED which I don't have, so I jumped into OpenSCAD to design my own.

I made the base thin over the two LEDs so that they can shine through (at least somewhat), and also made the wall thin next to the buttons, with little protrusions which are meant to push onto the switch. I made the clearance just a little too large, so it's quite tricky to push the buttons, but it will do for now.

I've pushed the CAD files (and code updates) to my repo on GitHub: https://github.com/usedbytes/tbeam. It's all very messy at the moment while I'm prototyping, but I'll be neatening it up as I move towards the final implementation.

I also soldered in my accelerometer. It uses analogue outputs, which is good for my use-case because I want to monitor it during deep-sleep, and that seems easier to do on analogue pins than having to handle i2c traffic from the ESP32 low-power coprocessor code.

I found the ESP32 documentation around the ADC routing a bit confusing, but did eventually identify three pins which I can access on the T-Beam, and that can be used for ADC1 from the low-power coprocessor.

The T-Beam's RTC + ADC1 GPIOs are pretty well filled up with the board's functionality. I found three pins I could use. Only two of them are broken out to pads on the headers, the third (IO37) is available as a pad on one of the not-populated 0402 components. Annoyingly, IO37 is also wired to the GPS module time pulse output, so I had to remove R41 to disconnect that. I'm not planning to use the time pulse, so this shouldn't really be a problem - but if I do want it I will just wire it to a different pin.

The not-populated pad is R67 in the schematic, and the pad closest to the WiFi antenna is IO37. VN and VP are for the internal hall effect sensor, but I don't plan to use that so I'll use those.

I've wired the accelerometer VCC pin to IO25. It's meant to have a very low active current (400 uA, not that I believe it) - so a single GPIO should easily be able to source enough current, and having it on a GPIO makes it easy to turn on and off. It's probably more efficient than spinning up the unused AXP192 DCDC1 output, too.

Which a tiny bit more code, I can read out the voltage for each axis:

// Power up the accelerometer VCC GPIO
io_conf.intr_type = GPIO_INTR_DISABLE;
io_conf.pin_bit_mask = (1ULL << GPIO_NUM_25);
io_conf.mode = GPIO_MODE_OUTPUT;
io_conf.pull_up_en = GPIO_PULLUP_DISABLE;
io_conf.pull_down_en = GPIO_PULLDOWN_DISABLE;
gpio_config(&io_conf);

gpio_set_drive_capability(GPIO_NUM_25, GPIO_DRIVE_CAP_1);
gpio_set_level(GPIO_NUM_25, 1);

// Set up the ADC channels
adc1_config_width(ADC_WIDTH_BIT_12);
adc1_config_channel_atten(ADC1_CHANNEL_0, ADC_ATTEN_DB_11);
adc1_config_channel_atten(ADC1_CHANNEL_1, ADC_ATTEN_DB_11);
adc1_config_channel_atten(ADC1_CHANNEL_3, ADC_ATTEN_DB_11);


while(1) {
        int val0 = adc1_get_raw(ADC1_CHANNEL_0);
        int val1 = adc1_get_raw(ADC1_CHANNEL_1);
        int val2 = adc1_get_raw(ADC1_CHANNEL_3);
        printf("0: %5d 1: %5d 2: %5d\n", val0, val1, val2);

        vTaskDelay(300 / portTICK_PERIOD_MS);
}

0:  1535 1:  1935 2:  1915                                                                                                                  
0:  1488 1:  1948 2:  1883                                                                                                                  
0:  1515 1:  1808 2:  1789                                                                                                                  
0:  1459 1:  1801 2:  1774                                                                                                                  
0:  1570 1:  1804 2:  1808                                                                                                                  
0:  1509 1:  1846 2:  1936                                                                                                                  
0:  1508 1:  1833 2:  1917                                                                                                                  
0:  1507 1:  1857 2:  1870                                                                                                                  
0:  1453 1:  1879 2:  1889                                                                                                                  
0:  1520 1:  1840 2:  1793                                                                                                                  
0:  1577 1:  1870 2:  1744                                                                                                                  
0:  1457 1:  1861 2:  1835                                                                                                                  
0:  1574 1:  1794 2:  1745                                                                                                                  
0:  1522 1:  1818 2:  1774                                                                                                                  
0:  1487 1:  1831 2:  1847                                                                                                                  
0:  1474 1:  1814 2:  1838

Next up on my list was to get the buttons on the board working, and set it up to run on battery power.

The T-Beam board has three buttons:

• RST - Connected to ESP32 CHIP_PU
  • Will just reset the ESP32 without affecting anything else
• USER - Connected to ESP32 IO38, with external pull-up
  • When pressed, pulls the IO low
• PWR - Connected to the AXP192 PWRON pin
  • This isn't connected to the ESP32, but we can configure the AXP192 to send an interrupt to the ESP32 when it's pressed
  • The AXP192 IRQ line is wired to IO35

Setting up the ESP32 for the USER button was simple, and I could basically use the code from the example: We just need to set-up IO38 as an input and enable a falling-edge interrupt:

static xQueueHandle gpio_evt_queue = NULL;

static void IRAM_ATTR gpio_isr_handler(void* arg)
{
    uint32_t gpio_num = (uint32_t)arg;
    xQueueSendFromISR(gpio_evt_queue, &gpio_num, NULL);
}

...

void app_main(void)
{

    io_conf.intr_type = GPIO_PIN_INTR_NEGEDGE;
    io_conf.pin_bit_mask = (1ULL << GPIO_NUM_38);
    io_conf.mode = GPIO_MODE_INPUT;
    io_conf.pull_up_en = 0;
    io_conf.pull_down_en = 0;
    gpio_config(&io_conf);

    gpio_install_isr_service(0);
    gpio_isr_handler_add(GPIO_NUM_38, gpio_isr_handler, (void*)GPIO_NUM_38);

    int cnt = 0;
    uint32_t io_num;
    while(1) {
        if(xQueueReceive(gpio_evt_queue, &io_num, 1000 / portTICK_RATE_MS)) {
            printf("Button pressed. GPIO[%d] intr, val: %d\n", io_num, gpio_get_level(io_num));
        }
        printf("cnt: %d\n", cnt++);
    }
}

Getting the interrupt from the AXP192 for the power button took a little more work. The AXP192 sets the IRQ line high when an enabled interrupt event happens, and it stays high until all of the pending interrupt events have been cleared my writing to the status registers over i2c.

So, I needed to implement some new functionality in the axp192 library to let me set up the interrupt masks and process and clear the events.

I've pushed these to my branch: https://github.com/usedbytes/axp192/blob/dev/axp192.h#L190

axp192_err_t axp192_read_irq_mask(const axp192_t *axp, uint8_t mask[5]);
axp192_err_t axp192_write_irq_mask(const axp192_t *axp, uint8_t mask[5]);
axp192_err_t axp192_read_irq_status(const axp192_t *axp, const uint8_t mask[5], uint8_t status[5], bool clear);

 With those in-place, I could enable the "short press" button interrupt in the AXP192, and process it in the ESP32.

I've set it up so that when the power button is pressed, the ESP32 writes to the AXP192 to turn everything off:

static void power_off()
{
    // Flash LED
    axp192_write_reg(&axp, AXP192_SHUTDOWN_BATTERY_CHGLED_CONTROL, 0x6a);

    // Save whatever state needs saving...

    // Power off all rails.
    axp192_set_rail_state(&axp, AXP192_RAIL_DCDC1, false);
    axp192_set_rail_state(&axp, AXP192_RAIL_DCDC2, false);
    axp192_set_rail_state(&axp, AXP192_RAIL_LDO2, false);
    axp192_set_rail_state(&axp, AXP192_RAIL_LDO3, false);

    vTaskDelay(1000 / portTICK_PERIOD_MS);

    // Turn off.
    axp192_write_reg(&axp, AXP192_SHUTDOWN_BATTERY_CHGLED_CONTROL, 0x80);

    for ( ;; ) {
        // This function does not return
    }
} 

When it's off, a press of the power button powers the system up, which is hard-wired in the AXP192 "Mode A".

The battery charging functionality comes up in a sane default state, 4.2 V end voltage and ~700 mA charge current, so I left that alone.

So now everything is set to run off an 18650 cell, and I can charge it from the USB port.

LoRaWAN - The Things Network

With GPS, the buttons and the battery working, next was the LoRa radio.

I searched around a bit and found this project on Github: https://github.com/manuelbl/ttn-esp32

With a little bit of fiddling I could get it to build in my project - but I had to use the dev branch to support my version of ESP-IDF.

Here's the pin configuration I used:

#define TTN_SPI_HOST      HSPI_HOST
#define TTN_SPI_DMA_CHAN  2
#define TTN_PIN_SPI_SCLK  GPIO_NUM_5
#define TTN_PIN_SPI_MOSI  GPIO_NUM_27
#define TTN_PIN_SPI_MISO  GPIO_NUM_19
#define TTN_PIN_NSS       GPIO_NUM_18
#define TTN_PIN_RXTX      TTN_NOT_CONNECTED
#define TTN_PIN_RST       TTN_NOT_CONNECTED
#define TTN_PIN_DIO0      GPIO_NUM_26
#define TTN_PIN_DIO1      GPIO_NUM_33

 I registered on The Things Network (TTN) - which is a LoRaWAN network using community-run gateways. After creating a new application and loading the uinque IDs onto my device, following the great Getting Started guide for ttn-esp32, I flashed the board, attached the antenna, and powered it up...

And then nothing really happened. The LoRaWAN coverage at my house must not be very good. After a bit of waving the board around, I did see one activation message pop up on the TTN console, but I couldn't get the device activated and sending data.

Thankfully, the TTN map shows a gateway not terribly far from my house, so I popped the board in my backpack and cycled a few miles up the road to see if I could get a better signal.

When I was nearby the gateway, I powered the board up and checked the console on my phone:

Success! (What's the payload?)

Honestly I'm a little surprised that the range isn't better, but there's probably a few contributing factors - it's very flat around here, which isn't good for radio propagation, and I'm sure my antenna isn't the best.

So LoRa works. Reading the TTN best practice guide my code is definitely non-compliant. The ttn-esp32 code only supports doing a brand-new activation (join) every time, which is exactly what the TTN guide says you shouldn't do.

The underlying LMIC library, which handles all of the LoRaWAN stack, does have functionality to pull out the session keys needed, but that's not exposed via ttn-esp32. There's some discussion about it in the issues:

• https://github.com/manuelbl/ttn-esp32/issues/5
• https://github.com/manuelbl/ttn-esp32/issues/15

There's also some example there of how to do it - so I'll be giving that a go in the future.

However, with the lack of LoRaWAN coverage at my house, I think I'll buy my own gateway before I spend any more time with the LoRa code.

Just the "simple" task of turning on the T-Beam is perhaps a little more complicated than what I'm used to from a microcontroller board. It includes a fully blown Power Management Integrated Circuit (PMIC) chip, which has 3 LDO linear regulators and 3 DC-DC buck converters, controllable over i2c.

The chip is an X-Power AXP192, which unfortunately I can't find a full English datasheet for, however I have found enough resources to get by. This is the same PMIC used in the m5stack modules, so there is a fair amount around.

The things I've been using are:

• A "brief" English datasheet, which lacks any detail: http://www.x-powers.com/en.php/Info/down1/id/29
• A full Chinese datasheet, which is pretty readable with the help of Google Translate: https://github.com/m5stack/M5-Schematic/blob/master/Core/AXP192%20Datasheet%20v1.13_cn.pdf
• A GitHub repo with some documentation which seems to be cherry-picked translations of the datasheet: https://github.com/ContextQuickie/TTGO-T-Beam/wiki/AXP192-Information
• An Arduino library: https://github.com/m5stack/M5StickC/blob/master/src/AXP192.cpp
• A "hardware agnostic" library, in C rather than Arduino: https://github.com/tuupola/axp192

Alongside the PMIC itself, we need to know how its voltage rails are wired in to the components on the board. Annoyingly, the first hit on Google seems to go to the wrong GitHub repo. This one appears to have the correct schematic and example code: https://github.com/Xinyuan-LilyGO/LilyGO-T-Beam

The board seems to have been designed with a few different options in mind, with various components not populated. It's not entirely clear from the schematic what is to be expected from the board, so I spent some time checking out the various circuits.

On my board, things are wired like so:

In addition to the AXP192, there's a discrete LDO 3.3 V regulator, which just powers the on-board CP2104 USB-to-Serial converter from the USB bus. As far as I can tell, this isn't connected at all when running on battery power, so it shouldn't leech any unnecessary power.

Initial code

I grabbed tuupola's library and the ESP32 IDF and got started. I ran into a few issues with the way tuupola's library uses build system #defines to set register defaults, and it didn't give me the level of control I was hoping for, so I've forked it and will see about merging my changes back when I'm happy with it: https://github.com/usedbytes/axp192

I did run into one slightly embarrassing issue by accidentally disabling DCDC3 as the first thing my code did when the ESP32 boots - that immediately powers of the ESP32, and made it a little tricky to re-flash good code. I had to spam the reset button and hope that the flashing tool managed to get in just before the bootloader jumped to my buggy code. I think there are pins I could use to force the chip to stay in the bootloader if I got really stuck, but I managed to get some good code back on after a couple of attempts.

With my modifications to the library, I could quickly start controlling all the different power rails, and got the GPS powered up and outputting data - so we're off to a good start.

    // Power off everything we don't need
    axp192_set_rail_state(&axp, AXP192_RAIL_DCDC1, false);
    axp192_set_rail_state(&axp, AXP192_RAIL_DCDC2, false);
    axp192_set_rail_state(&axp, AXP192_RAIL_LDO2, false);

    // Set the GPS voltage and power it up
    axp192_set_rail_state(&axp, AXP192_RAIL_LDO3, false);
    axp192_set_rail_millivolts(&axp, AXP192_RAIL_LDO3, 3300);
    axp192_set_rail_state(&axp, AXP192_RAIL_LDO3, true);

    // Set up a UART for the GPS
    uart_config_t uart_config = {
        .baud_rate = 9600,
        .data_bits = UART_DATA_8_BITS,
        .parity    = UART_PARITY_DISABLE,
        .stop_bits = UART_STOP_BITS_1,
        .flow_ctrl = UART_HW_FLOWCTRL_DISABLE,
        .source_clk = UART_SCLK_APB,
    };
    uart_driver_install(UART_NUM_1, BUF_SIZE * 2, 0, 0, NULL, 0);
    uart_param_config(UART_NUM_1, &uart_config);
    uart_set_pin(UART_NUM_1, GPIO_NUM_12, GPIO_NUM_34,
                 UART_PIN_NO_CHANGE, UART_PIN_NO_CHANGE);

    uint8_t *data = (uint8_t *) malloc(BUF_SIZE);

    for ( ; ; ) {
        // Read data from the GPS
        int len = uart_read_bytes(UART_NUM_1, data, BUF_SIZE, 20 / portTICK_RATE_MS);
        // Send the data to the serial console
        fwrite(data, 1, len, stdout);
        fflush(stdout);
        vTaskDelay(500 / portTICK_PERIOD_MS);
    }

After a few seconds of warm-up, I was rewarded with a nice GPS fix! (location redacted...):

$GNGSA,A,3,03,04,11,17,22,,,,,,,,3.40,2.04,2.72*1B
$GNGSA,A,3,71,85,,,,,,,,,,,3.40,2.04,2.72*11
$GPGSV,2,1,07,03,81,244,37,04,26,179,37,11,27,145,34,17,47,293,32*74
$GPGSV,2,2,07,22,70,076,31,36,24,142,32,49,30,174,38*48
$GLGSV,1,1,04,70,56,046,,71,67,210,37,72,16,218,23,85,39,147,29*61
$GNGLL,XXXX.XXXXX,N,XXXXX.XXXXX,E,212332.00,A,A*7A

Next up I want to try the battery charging and monitoring functionality. 

This project started out from the intersection of a couple of threads:

Towards the end of 2019, I went to a talk about LoRaWAN by @rwhb2: https://rwhb.me/talks/lorawan-ttn-talk.pdf, and (relatedly) at around the same time some LoRa gateways started getting set up around Cambridge where I live. This sounded like a pretty cool technology which I was keen to have a play with.

Then a little later I heard from Digitspace, who were interested in sponsoring a project with products from their store. They have a pretty good selection of kit in a variety of categories - one of which is an array of different LoRa boards.

I found the T-Beam board, which has LoRa and GPS, making it a good candidate for a self-contained LoRaWAN mapper, which isn't very original but seems like a reasonable first LoRa project. I've also been interested for a while in trying to track road quality while cycling, and this board looks like a reasonable fit for that too - an 18650 battery gives a good amount of capacity, the GPS means it can track the position without needing an external source of location data, and an accelerometer can be easily added.

Digitspace were kind enough to send me a free voucher for the two main parts of this project:

• TTGO T-Beam ESP32 WiFi/BT GPS and LoRa module
• GY-61 ADXL335 Accelerometer

In return, I agreed to write up the project here. It looks like makers are free to submit their own requests for sponsorship via their website.

The Digitspace store seems pretty good. They have plenty of selection, and from what I can tell it's almost all maker-friendly open-source gear. At the moment, the product pages don't have a whole lot of information on them, but because the designs are open you can find it via your favourite search engine.