Project Logs

Collapse

• Brief video demo of RemuterMCS

  WJCarpenter • 4 days ago • 0 comments

  The user interface is done enough, and I finally got around to making a short demo video of the gadget in action.

  It's just the basics of operating it, so I don't go into detail about all the configuration options and so on.

• Is this radio on?

  WJCarpenter • 03/29/2021 at 02:18 • 0 comments

  I've pretty much finished fooling around with the firmware on the M5StickC. It's been great fun clicking the buttons and seeing the AHK pop-ups on my computer screen, but I now want to work on making AHK actually do the things it says in the pop-ups.

  I got a M5StickC Plus, which has a larger display, and adapted both of the screen renderings to the larger size. The code now runs fine on either kind of device, but you do have to say which it is in a RemoterMCSconfig.h before building and uploading the firmware. They use different TFT video controller chips. I don't think it's technically possible to have one firmware that serves both devices. If I ever figure out a way to do that, I can eliminate the configuration step.

  Here is a photo comparing the two devices, M5StickC Plus on the left, M5StickC on the right. (I don't know if they are intentionally different shades of orange or if it's just variation from different manufacturing runs.) For both displays, the dashed vertical bar on the right side is the battery indicator. The stripe on the left side is Received Signal Strength Indicator (RSSI) of the connected computer. The RSSI varies quite a bit, but it does give a coarse indication of strong or weak signal. (I don't understand why, but it seems a Bluetooth server, which is what these are, must ask the Bluetooth client for its RSSI value. Since both ends are receiving signals, I expected to be able to ask my radio what RSSI it had seen. But, apparently not.)

  One thing I have to come back to and hope to improve is battery life. There's a lot I don't know about Bluetooth LE, even though I know a lot more today than I did when I started this project. I want to figure out how to put the ESP32 into light or deep sleep most of the time to save power. I know how to do that, but it kills my Bluetooth conversation with the computer. This could be something that needs a code change in the ESP32-BLE-Keyboard library, or it may be something I can overcome with other BLE or ESP32 calls in conjunction with that library. I have to study that some more.
  

  My simple battery longevity testing was disappointing, but fairly conclusive. Regardless of how much delay() I put into each iteration of the Arduino loop(), the lifetime of the M5StickC battery is between 40 and 50 minutes. It's not too surprising that the amount of delay() doesn't matter too much since the processor continues to run at full speed. On the ESP32, delay() just makes an RTOS call to suspend the foreground task for that amount of time. The clock keeps ticking.

  I repeated the longevity timing with a modified program that didn't do anything with the Bluetooth radio. It wasn't even initialized. I got between 110 and 120 minutes that way. More than double, less than triple. It gives me some encouragement that if I can crack the light/deep sleep puzzle so I can continue BLE when I wake up, I should be able to get a lot of battery mileage. When starting from scratch, it can take from 1-10 seconds to establish the Bluetooth connection, which is pretty unacceptable for this use case. 

  I just need to figure out how the Bluetooth remote control folks do it. I'm sure their devices sleep most of the time. I paged through the BLE documentation intended to facilitate the low power part, but there's a lot of jargon and a lot of layers of APIs, so it's taking me a while to match things up. I haven't yet come across a good example, although there are plenty of examples that don't worry about power. 

  EDIT: I figured out how the remote control people do it. They use more special purpose Bluetooth chips that are optimized for it. The ESP32 will probably never be able to be that efficient, but it should be able to do a lot better than it does for me right now.

• Battery percentage, note 3

  WJCarpenter • 03/26/2021 at 04:01 • 0 comments

  I suppose it's due to the relatively small battery, but things seem to go pretty unevenly as it charges and discharges. I am still using a strictly linear approximation for how much battery remains, but I'm a little skeptical that it's correct or any more than grossly meaningful. I draw a bar along one of the long edges of the display to show the remaining battery.

  I verified by checking registers in the AXP192 that the automatic shutdown voltage is 3.0v, as documented for the M5StickC. FWIW, the L1 and L2 warning levels are 3.45v and 3.40v, respectively, but they are not used for anything by default.

  With my previously described technique of recording the high and low voltages observed, the lowest I ever saw was a little above 3.4v. I suspected it was the granularity of the 10 minute intervals that was givng me that puzzling value, so I changed the timing to observe and record the low voltage every 40 seconds, and that showed me a low voltage of 3.0778v. That pretty much confirms that the AXP192 is behaving as expected and shutting things down at 3.0v.

• Bright breath

  WJCarpenter • 03/26/2021 at 03:53 • 0 comments

  For the M5StickC, the brightness of the TFT display is not controlled the way you might expect, with a PWM operation on a GPIO pin from the ESP32 processor. Instead, the backlight is wired to an output pin of the AXP192 power controller. The voltage supplied by that pin can be programmed to provide 1.8v to 3.3v in 0.1v increments.

  M5's Arduino IDE library for the M5StickC provides an AXP192 method called ScreenBreath(). I don't know why it's called that, but what is does is set a value to control the output voltage from that pin on the AXP192. That saves you a small amount of trouble in doing it yourself over the I2C interface. Interestingly, even though the AXP192 would accept values 0 - 15, the M5 API limits the upper range to 12. (I suspect that's based on how much voltage the TFT backlight LED would tolerate. The M5StickC product page says "2.8v max200ma". The AXP192 default output voltage is 2.8v. Maybe not worth sacrificing a device to find out what happens when you overdrive the backlight LED. :-) )

  I did some experiments, and the display is completely dark with any value of 7 or below (at least for my eyes with the M5StickC that I'm testing).

  So, realistically, the usable values are 8, 9, 10, 11, 12. 

  I provide a configuration item for setting the display brightness in my sketch. I don't limit it at either end, but the above limits are still there.

• Battery percentage, note 2

  WJCarpenter • 03/22/2021 at 02:29 • 0 comments

  I know a little bit about the battery parameters in the M5StickC now. The power controller is the AXP192. The manufacturer makes a comprehensive AXP192 data sheet available. Unfortunately for me, most of it is in Chinese. With the help of Google translate and code published by M5, I was able to figure out quite a bit. (There is an English data sheet available, but it's only 4 pages, compared to more than 50 pages for the Chinese data sheet.)

  • The AXP192 targets charging to a battery voltage of 4.20v, plus or minus 5%.
  • M5 has configured the AXP192 to automatically shut things down at 3.00v.
  • M5 provides an API called GetBatteryStatus(), but it returns an 8-bit value that is a collection of flags from an AXP192 register. The datasheet told me what each bit was.

  (M5's AXP192 class includes helpful methods for reading and writing AXP192 register values. Alas, they are all private methods, so I can't use them directly to get things that M5 didn't expose. It's not insurmountable, but it's inconvenient.)

  I found out experimentally that if I plug in the device when it's a little below 4.20v but within 5%, the AXP192 doesn't start charging it. And, since the AXP192 switches things to run completely on the USB current, the battery won't discharge and things will stay that way. OTOH, when the battery is being charged, it only got up to about 4.18v. That's within that 5% range (in fact, only about a half percent difference), but I expected it to be more on target. I don't know if that's due to AXP192 logic or if my specific battery won't take more charge.

  I recorded the voltage as the battery discharged, as described in my previous note. The lowest recorded voltage was 3.53v. That's quite a bit higher than the 3.00v shutdown voltage I expected. I'm going to run a few more charge/discharge cycles to see if that low number changes. When the numbers settle down, I will probably still use a linear calculation for the percentage. That's probably close enough for this kind of experience. What you really want to know is when the device is going to quit due to a low battery.

  The application logic I am using to record the voltage range at this point:

  • For any new lower voltage, record it regardless of charging or plugged-in status.
  • If the device is plugged, not charging, and the difference from the measured voltage is more than 5% different from 4.2v, record it as a new high water mark. (I'm not really sure what happens if the AXP192 can't get the battery up to the target voltage range.)

• Icons for everyone

  WJCarpenter • 03/20/2021 at 02:07 • 0 comments

  I picked out the UI icons for microphone, camera, and speakers. There are zillions of icons to look through at https://thenounproject.com/, all of which are better than I could make myself, even for this fairly simple shapes. I fiddled with different choices quite a bit by viewing them on the M5StickC display. To make it easy to try things, I wrote a simple sketch IconsAtPlay that is a bit too long to show here. It displays the icons in multiple ways, in both 80x80 and 40x40 sizes.

  To make it simple to get the sizes I needed in uniform square backgrounds, I started with SVGs and converted them to XBMs with everybody's favorite graphics manipulation tool, ImageMagick.

  I was originally going to overlay a "?" for the unknown status and a circle-slash for disabled status, but I didn't like the look of it. Instead, I just draw a diagonal line a few pixels wide for disabled status. I also draw a 1-pixel box around the icon when that target has focus on the user interface. The target with focus uses the 80x80 icon size, and the other targets use the 40x40 size.

  Here is a photo of the device (mocked up with the IconsAtPlay sketch) showing microphone muted, camera enabled, and speakers in an unknown state. The microphone target has focus.

  Below are the original forms of the icon graphics. All of the icons are licensed under Creative Commons 3.0. They're displayed here in black on a transparent background. I provide the colors when I render them on the device. 

  Microphone: Microphone by Chunk Icons from the Noun Project
  https://thenounproject.com/icon/829069/

  Camera:     webcam by Kawalan Icon from the Noun Project
  

  https://thenounproject.com/icon/2131644/

  Speaker:    Speaker by Alex Arseneau from the Noun Project
  

  https://thenounproject.com/icon/872876/

• Splash!

  WJCarpenter • 03/18/2021 at 15:57 • 0 comments

  I've designed a splash screen, which is the first thing you see when you power the device up. It's also the screen which tells you whether or not it's connected to Bluetooth, which is nice to know.

  (Instead of this sloppy picture, I had intended to read the pixel values from the display and show it here in screenshot fashion. I never got that to work, despite an encouraging comment in the library code. Reading pixels always came back as 0 values no matter what I tried.)

  I originally started working with the "built in" fonts from the M5StickC Arduino library because I didn't feel like messing around with figuring out how the fonts worked in the Adafruit GFX library, and I also wasn't sure it was worth the flash space. It turns out that the M5StickC "M5Display" already includes all that stuff behind the scenes, including all the standard Adafruit converted "FreeFonts". The space is already being consumed unless you go to some trouble to chop those fonts out. So, I switched over to using them and experimented until I got sizes and styles that I liked.

  What's the splash screen displaying?

  The middle line is the hardware Bluetooth address of the device, which is commonly called the MAC. When the device advertises itself for pairing, the part in blue is included in the device name. That's in case there are multiple devices floating around and you need to be able to tell them apart.

  The "connecting ..." part of "BT connecting ..." repeatedly types itself as an animation while connecting is taking place. Once connected, the entire line changes to a blue "BT connected". The display rests there for 1-2 seconds and then switches to the normal view showing the icons for the microphone, camera, and speakers.

  You might not notice that this display has a bias for people who hold the device in the right hand with the "M5" button away from them. There is a configuration option to rotate the text 180 degrees for those who prefer it that way. There is no performance difference either way.

  After I took the picture, I changed the middle line to use a sans serif font, which made it more readable. That's probably it for the visual design of the splash screen, except for overlaying the battery and signal strength indicators that will also be on the main screen. I might also move things around to make them more centered, but to be honest I didn't notice it being off-center so much until I saw this still photo. I'll make a video of the splash screen in action later. I'm pretty pleased with how it turned out.

• Battery percentage, note 1

  WJCarpenter • 03/17/2021 at 03:27 • 0 comments

  I plan to give some kind of visual indication of the amount of battery power remaining, but the problem is determining some kind of quantification for that. M5 does not provide a direct API to answer the question "how much battery left?", but they do provide APIs for the AXP192 power controller to get all sorts of interesting information.

  I naively thought there would be some kind of common knowledge way of taking LiPo battery voltage and translating that directly into what I want. And, it's possible that there is, but I didn't stumble across it. I did find quite a bit of discussion about it. Here is a link to one discussion of it. This person provided a handy table of voltages and percentages, but I'm kind of skeptical, because

  • In the first place, my fully charged battery voltage was a bit lower than their 100% figure, 
  • and I have no idea what my low-voltage is when the device shuts itself down. 
  • I'm also pretty sure the discharge rate isn't linear, but I don't know what it is instead.
  • As my device ages, the battery capacity will degrade.
  • And, finally, there could be variation so that your battery is different from my battery (or two of my batteries might be different from each other).

  Here is my tentative plan. I haven't implemented it yet, so it's subject to change.

  • I'll use the "Preferences" API to store some battery-related values in flash. 
  • When the device is fully charges (as detected from the power in and out values), I'll record the battery voltage if it's different by more than a few percent from the last "high" value I stored, whether higher or lower. I don't expect any higher "high" values, except for a little jitter, but I expect lower "high" values as the battery ages. 
  • Periodically, maybe every 10 minutes or so, I'll store the current battery voltage if it's lower by more than a few percent from the last "low" value I stored. 
  • I'll run the device on battery power until it shuts off, fully charge it, and I'll repeat that a few times.
  • Using those "high" and "low" values, I'll ignore science and assume a linear discharge between them and convert that to a percentage.

  Note to self: consult the wisdom of crowds about how LiPo voltages change as they degrade over time.

• Turbines to speed!

  WJCarpenter • 03/14/2021 at 18:24 • 0 comments

  I wanted to find out how long the M5StickC can run on its battery under various conditions. That would give me some idea of the usability of RemotMCS as a ... remote. How to measure?

  The simplest thing I thought of was just having it "Serial.println()" something every once in a while. The Arduino IDE serial console would timestamp them so I could know when the last one came in. The problem with that is that the M5StickC charges its battery and runs off USB power when it's plugged in. I can predict in advance that the answer would be something like infinity.

  What about doing the equivalent over Bluetooth? Paired to a PC, I could just leave an editor window open and have the M5StickC "type" some time information periodically. RemuterMCS doesn't know time of day, but it does know the amount of time since the ESP32 startup. This scheme would probably work, but it has a couple of drawbacks. My primary concern is that I don't know how much the Bluetooth interaction would affect power consumption, so I wasn't sure I could get a baseline of "while doing nothing". Second, I know from past experience that I have a good shot at goofing up the recording now and then. If the battery life is 4-5 hours, then it's an overnight wasted on a bungled test. Finally, I have a preference for something that is simpler for someone else to replicate so they could perform the measurements on their own specific M5StickC devices. In fact, I have a few of these and might want to measure them all to see if there is variation in available power.

  The solution that I adopted was to periodically store timing information on the M5StickC itself. On start-up, I read it back out and report it. I report it with "Serial.println()" and, in the early stages of the code, on the device display. I don't know if I'll show it on the device display when the UI is complete.

  With true Arduinos (without SD cards or other storage devices), that's done using a mechanism called PROGMEM. On an ESP32, PROGMEM is a no-op. Instead, the ESP32 Arduino libraries provide a mechanism called "Preferences", which is a straightforward key-value store. It operates on some part of the device's flash memory. I didn't chase down the details of the flash partition scheme and where the preferences values live; I just tried it and it worked as advertised.

  Now, of course, I don't know how much power it takes for the ESP32 to write those values to flash via the Preferences API. I'm just hoping it's not much. I write the time since startup every 10 minutes. A 10 minute (rounded down) granularity for battery life should be fine unless I get into some really detailed fine-tuning, which I don't expect to do. It also means I have 10 minutes of false starts and shenanigans before the value gets overwritten after a device restart.

  The mechanism is enabled with a single define. I won't quote all of the code here, but this is the gist (snapshot in time; check the git repo in case it changes, which it surely will).

  #define LONGEVITY 1
  
  #if LONGEVITY
  #include <Preferences.h>
  #endif
  
  #if LONGEVITY
  const char *key_naptime = "naptime";
  const char *key_longevity = "longevity";
  unsigned long lastSaveMillis = 0;
  unsigned long saveEveryMillis = 600000;
  unsigned long storedLongevity = 0;
  unsigned long storedNaptime = 0;
  Preferences preferences;
  #endif
  
  void setup() {
  // ...
  #if LONGEVITY
    preferences.begin("RemuterMCS");
    storedLongevity = preferences.getULong64(key_longevity);
    storedNaptime = preferences.getULong64(key_naptime);
    preferences.end();
    Serial.printf("Stored longevity = %ld, naptime micros = %ld\n", storedLongevity, storedNaptime);
  #endif
  // ...
  
  void loop() {
  // ...
  #if LONGEVITY
    unsigned int now = millis();
    if ((now - lastSaveMillis) > saveEveryMillis) {
      lastSaveMillis = now;
      preferences.begin("RemuterMCS");
      preferences.putULong64(key_longevity, now);
      preferences.putULong64(key_naptime, napDurationMillis);
      preferences.end();
    }
  #endif
  // ...
  }
  

  The "naptime" referenced in the code is because...

  Read more »

• Clicking the buttons

  WJCarpenter • 03/06/2021 at 20:22 • 0 comments

  Since the RemuterMCS device user interface is a series of dolphin-like clicks and pops, I wanted to outsource the button handling as much as reasonable. I didn't feel like writing fiddly timing code myself. There are many button-handling libraries for the Arduino IDE ecosystem. I spent a little time looking at a few of those, and I have settled on using AceButton by Brian Parks.

  The main reasons that I like that particular library:

  1. It provides the button features I want (single click, double click, multiple buttons) and some others that I might find a use for later.
  2. The quality of the documentation and sample code is great. Also, the author's explanation of why and how they wrote the library is consistent with how I think about things.
  3. It uses an event callback scheme for button activity. Novices may find that a bit confounding, but it's second nature to me and will greatly simplify the code I write using the library.
  4. The library API looks clean and sensible.
  5. The most recent versions of it are included in the Arduino IDE library manager, which is convenient.

  I did some quick experiments with this library, tweaking the sample code, to make sure it works as advertised. I didn't doubt that it would, once I had read through the extensive documentation. Here is a sketch, KeysAndClicks, that reacts to single and double clicks by printing the targeted reaction on the serial console (which it expects to be configured for 115,200 speed).

  /**
   * Reacts to single and double clicks, but the reaction just prints
   * what should happen.
   */
  
  #include <AceButton.h>
  #include <M5StickC.h>
  
  using namespace ace_button;
  
  const int LED_PIN = M5_LED;
  const int LED_OFF = HIGH;
  const int LED_ON  = LOW;
  
  // The pin number attached to the button.
  #define ACTION_BUTTON_PIN M5_BUTTON_HOME
  #define MODE_BUTTON_PIN   M5_BUTTON_RST
  
  AceButton actionButton(ACTION_BUTTON_PIN, HIGH, 1);
  AceButton modeButton(MODE_BUTTON_PIN, HIGH, 2);
  AceButton *buttons[] = {&actionButton, &modeButton};
  const uint8_t NUMBER_OF_BUTTONS = sizeof(buttons) / sizeof(buttons[0]);
  
  void setup() {
    Serial.begin(115200);
    Serial.println("Starting");
  
    pinMode(LED_PIN, OUTPUT);
    turnLedOff();
  
    pinMode(ACTION_BUTTON_PIN, INPUT_PULLUP);
    pinMode(MODE_BUTTON_PIN,   INPUT_PULLUP);
  
    // using method 3, ClickVersusDoubleClickUsingBoth, to distinguish clicks
    ButtonConfig* actionButtonConfig = actionButton.getButtonConfig();
    actionButtonConfig->setEventHandler(handleButtonEvent);
    actionButtonConfig->setFeature(ButtonConfig::kFeatureDoubleClick);
    actionButtonConfig->setFeature(ButtonConfig::kFeatureSuppressClickBeforeDoubleClick);
    actionButtonConfig->setFeature(ButtonConfig::kFeatureSuppressAfterClick);
    actionButtonConfig->setFeature(ButtonConfig::kFeatureSuppressAfterDoubleClick);
  
    ButtonConfig* modeButtonConfig = modeButton.getButtonConfig();
    modeButtonConfig->setEventHandler(handleButtonEvent);
    modeButtonConfig->setFeature(ButtonConfig::kFeatureDoubleClick);
    modeButtonConfig->setFeature(ButtonConfig::kFeatureSuppressClickBeforeDoubleClick);
    modeButtonConfig->setFeature(ButtonConfig::kFeatureSuppressAfterClick);
    modeButtonConfig->setFeature(ButtonConfig::kFeatureSuppressAfterDoubleClick);
  }
  
  typedef struct targetDevice {
    int id;
    const char *name;
  } targetDevice_t;
  
  targetDevice_t microphone = {0, "microphone"};
  targetDevice_t camera     = {1, "camera"};
  targetDevice_t speaker    = {2, "speaker"};
  targetDevice_t *targetDevices[] = {µphone, &camera, &speaker};
  const uint8_t NUMBER_OF_TARGET_DEVICES = sizeof(targetDevices) / sizeof(targetDevices[0]);
  int activeTargetDeviceNumber = 0;
  
  #define ACTION_CLICK_NONE   0
  #define ACTION_CLICK_SINGLE 1
  #define ACTION_CLICK_DOUBLE 2
  int actionClickType = ACTION_CLICK_NONE;
  
  void loop() {
    manageLedState();
  
    actionButton.check();
    if (actionClickType != ACTION_CLICK_NONE) {
      Serial.print("action click "); Serial.println(actionClickType == 1 ? "SINGLE" : "DOUBLE");
      actionClickType = ACTION_CLICK_NONE;
    }
  
    int previousActiveTargetDeviceNumber...

  Read more »

View all 16 project logs