PREFACE TO POSTING SOME NEW PROJECTS

OK, it's been a long time since I updated this project log.   Once again, THIS "rPi bare metal vGuitar" project is the overall project that encompasses all the other projects that I have on hackaday.  Even if it's not immediately apparent to the casual reader how they all relate, the other projects, including the 3D printer, synth box, experiments with accelerometers, and even the ws2812B switch array, are all related to my efforts to produce an rPi based, bare metal, vGuitar rig.

In my last log, from Nov 2019, I said I needed to create a "proof of concept" Looper based on the rPi/circle bare metal code that I have labored on.   Now, 10 months later, I have very nearly done that to a level that is demonstrable, and hope, in the next few weeks, to not only post that to Hackaday, but to also record a song (make a youtube "performance" video), along with a "making of" video, that will show some of the various pieces of this super-project "put together" in a working floor rig.

But before I do that, I will need to post a couple of (at least one) other sub-project(s) that have taken place since my last Log entry, and the reason for this entry is to provide some context for those projects, and to "tell the story" behind them.

---

A WORD ON THE DEVELOPMENT ENVIRONMENT

I think the purport of this project may get lost sometimes.  The point is that others can, if they wish, build these projects.   But to do that, you need to at least briefly understand the development environment.

The Circle and additional code that I have developed for the Raspberry Pi is a bare metal development environment loosely based on the Arduino development environment.  I do all of my development on a Windows based machine.   Windows 10 at this time.  You don't have to, and this code should work on Linux, for example, but I use Windows.

I assume that the reader can install the Arduino devenv, and add the publicly available Teensy support.   With that, and the gnu arm-none-eabi compiler, and perhaps a particular version of gnu make, basically you should be able to build my example Circle projects, and worst case, stick the resulting kernel7.img onto a bootable rPi SD card and boot up the app.

Everyone has their own preferences and development environments.  

I really don't like the whole world of hyper-integrated IDE's that we have grown into, like the MS Visual Development Environment or the Android IDE.   They are slow, and klunky, and force unimaginably complex architectures and directory structures on the projects and development thought processes.  I don't want to run Linux on my "main" personal machine, which is also (always going to be) my main development machine.   I didn't even want to run Linux on the project platform lol ... the whole point of the "bare metal" approach is to get away from "do everything for everybody" operating systems, and the complexity involved with them, and get back to something where it is somewhat reasonable for a single person to understand the entire deployment environment, as well as to be able to build it in seconds, versus hours.  I don't want to do linux builds, or friggin learn how to write a general purpose driver for the whole world.  I just want the sound card to work in my dedicated audio application, lol, and I wanna know HOW it is working.

Don't even get me started on Apple.   You can't even develop and test an IOS app without years worth of systems growth, subscribing to services, buying a MacBook, submitting your app for approval, and somehow getting it ONTO a frigging iPad.   If it was easier, perhaps this whole project would not be taking place.   But Apple is the worst of em when it comes to proprietary black box dependencies and complexities.  And most "mobile" apps these days absolutely suck in terms of learnability.   Nothing like having to press an odd button to see what it does in order to learn how a program works.  Heck, I've downloaded Apps and can't even figure out what they're supposed to be trying to accomplish.  No words.  No instructions.  No consistency.  Big round yellow buttons.  What is that?

So my code is just straight C++.  Open Source and compiled by GNU.   Stuck on a SD card, which runs in an rPi.

I'm also an old-school Perl hacker.     I still find it an amazing language in terms of pulling together disparate pieces of junk and making something that runs.   So this is one place I will probably diverge from most readers, but I want to describe a few other things in my development environment that are related to my 30+ year Perl history.

First of all, I just use a pretty simple Text Editor as the basic core of my personal IDE .... I use an old copy of KOMODO, from Activestate, and the associated Perl distribution that is/was available from them at some point.  

...

Then I stick a Teensy between the Windows machine and the rPi.
...

I wrote a "putty-like" Windows console application in Perl, that deals with com ports, works with the Arduino IDE (to disconnect the comm ports while building, supports Ansi color codes, and contains protocols to upload, through the teensy, built kernel images (among other things) to the rPi.    After editing a bunch of stuff, a single keystroke in Komodo kicks off the "make", and a javascript script that I wrote for Komodo tells the console application that a file needs to be sent to the rPi.   The console application can tell the teensy to reboot the rPi, and when it reboots, it comes up in my "bootloader" for 3 seconds (before loading whatever kernel is on the SD card).  The console application sees this, and uploads the new kernel to the bootloader, which then tells the rPi to do a "soft boot" (just start running the new kernel that was uploaded directly in memory).   The kernel is also (by a "define" in the bootloader C++ code) written to the SD card ... so a subsequent reboot will boot the newly installed kernel..  

Thus I don't ever swap the SD card out of the rPi (except when I have to work on the bootloader itself).

This all works with (javascript scripts written in) Komodo, which calls "make" to do various builds, and notices when something has been built which should be downloaded to the rPi.  This console program "knows" about the teensy-rPi relationship and communicates to the rPi via the Teensy.  Everybody gets their own "color" for messages coming in, or going out, over the serial port, so I can tell what's happening.  The teensy  can also reboot the rPi, and I can tell the rPi, via the console program, to do that with "ctrl-B' on my windows machine.

Why am I saying all this.

Because the devleopment environment context might be useful to understanding or reproducing this project.  And it goes to the hardware architecture (or why there's a teensy in my rPi box).

---

USB MIDI and HID DEVICES suck!

So after my Nov 2019 post, I did, in fact, develop a rudimentary Looper.   That required developing an event driven windowing system for Circle which took quite a bit of time.   So then I had a mouse and windows thing running on my 40" HDMI T.V.  Then I added a rudimentary USB MIDI event handling system, implementing controls for audio devices (things like a VU meter, volume control sliders, etc) that actually work, so that I could hook up some external MIDI devices to control the looper.

That's where I began to run into problems and the project took a fairly serious multi-month step in a different direction.

I started running into intermittent failures, crashes, or just noise in the system when using USB Midi devices (or a USB mouse).

A word about rPi sound cards.   The best sounding, as well as the most powerful, sound card I have is the AudioInjector Octo 6-in, 8-out sound card.  I have implemented 2-3 other sound cards in this code, including the AudioInjector Stereo, the Teensy audio card, and a HiFiBerry (or whatever it's called), and the Octo is the quietest of them.   It is also the most CPU intensive, inasmuch as the I2S bandwitdth used is actually a full 8 channel pseudo TDM format .   So the system is actually processing  8 channels of input data into 8 channels of output data, even though I am only using two of them (stereo) in this rudimentary Looper.

And yet the latency of the system is just right at 6ms ... which I am happy with and determined to keep.

But, for whatever reason, I found that, in the current code, the use of USB within Circle negatively affects the audio performance

USB Midi SUCKS.   Every USB Midi Device has to be POLLED (by the USB controller) at least 500 times a second!   Same for USB HID devices.  Polled! The USB (MIDI) architecture is the opposite of interrupt driven!   I guess "mature operating systems" like Windows can handle this load due to many years of optimization work, but the Circle environment appears to create some conflicts, at least with my implementation of I2S audio  (the port of the Teensy Audio library to this Circle bare metal rPi environment) and the 2835 BCM/PCM.

EVEN though I am, in bare metal, utilizing the mutli-core rPi 3B+ .... Core 0 is dedicated to handling interrupts ... no code, per se, "runs" on it, only interrupt handlers.   Core 1 is dedicated to the audio system, Core 2 is dedicated to the UI, and Core3 is currently unused (somewhat used for debugging probes).  When Core0 receives an I2S (DMA) interrupt, it sets up to read or write the next buffer via DMA, and passes the current buffer to Core1 via an inter-processor interrupt.  That's it.  Core1 then has the maximum chance to do what it can in the 30ms window per buffer (128 sample buffers at 44.1 Khz), which is where the "Looper" is implemented..

I found that if I use the mouse, or a USB midi device, that either (a) the program starts crashing intermittently, or (b) I start getting a bunch of noise in the audio buffers.   And none of it was for any "apparent" reason that I could suss out.   I spent almost a month trying to figure out what the heck the problem was, and came very close to canning this whole project.    Seriously, I was about to throw away 1.5 years of development effort here.

I was like, WTF don't I just write this for linux on the rPi, like everybody else?

But that little devil was in my ear.  He said "yeah, well you know you won't even be able to get the driver for the Octo to work reliably in Linux, and in the end, you'll have the same issue, but additional mountains, literal planetary masses, of code to sort through and question in the process" .....

So, I decided to blow off USB Midi (and USB HID device) support.

The Looper worked pretty well over the serial port (to the Teensy to the rPi).   I'll figure something out.

---

HENCE THE TEENSY EXPRESSION MIDI PEDAL

Thus I ended up going off in yet another direction for a couple of months, culminating in the production of the Teensy Expression MIDI controller project, which I hope to post today.

Huh?   I thought you just said that USB Midi was the root of all evil!

Yes, well, ahem, after my initial knee jerk thought to just create some kind of a switch-controller array for the "Looper" (teensy/rPi) thing, as I started really thinking about making a floor controller, and I started working with the teensy3.6, and one thing led to another. 

I mean, it's really easy to make a teensy3.6 based MIDI controller.    And I will need expression pedals, right?   And wow, look, I can host a USB device as well, like the Fishman FTP dongle, and continue reverse engineering and understanding that.   So I spent quite a bit of time experimenting with, and developing stuff completely unrelated to the Looper ... not the least of which was to implement the USB Host capability on the teensy3.6 and do some serious reverse engineering of the Fishman FTP (Triple Play) MIDI protocols.  

And since I've gone that far, might as well make it so that it can be EITHER a USB midi device, or (communicating via Serial data) a dedicated controller for this rudimentary "proof-of-concept" Looper thing.

So the TeensyEpression sub-project grew into it's own semi-huge thing, that actually WORKS with my current gig-rig, allows me to experiment with that, while at the same time, being able to be used as a controller (via serial data) for this rudimentary Looper project.

So, that project has to get posted.   I hope to post it today.

There was also some interesting experimentation into continuous controllers that I am continuing and not revealing at this time, but I am very excited about!

---

CROSSFADE NEEDED

Then (actually before, and at the same time as I was almost giving up on this whole mess) I realized that my rudimentary Looper was ... uhm ... TOO rudimentary.

It basically worked well at recording clips of time and playing them back, and if you just count "one two three four" into a microphone, you get nice loops.   But in actual use, like looping a guitar, or for test purposes, a continuous human voice, anything that crosses a loop boundary produces really noticeable "clicks" and other sonic artifacts.

I determined I needed to implement "cross-fades" between the ends, and beginnings of loops.

So, that took some time.  It was significantly complex to change the way I did everything in handing, recording, and playing back clips to provide cross-fading between clips.    Not the least of which was that, in my mind, to do it "right" you need to record MORE than the "user designated loop length".    Let's say I want a 30 ms cross fade.   That means that I have to record an EXTRA 30ms AFTER the loop, WHILE (in certain circumstances) starting to play the clip I am recording, back at the start, from beginning.     Thus I need to create, and maintain, two pointers into a given sound clip, and the resultant state machine is significantly more complex than the previous one.

More notes on that in the Looper project itself, but lets just say that added another bunch of time to the project.

---

END OF THIS LOG REPORT

It has been a tough year.  With the virus and everything, there have also been more than the usual amount of personal and work related issues over the last year.  It has been hard to make consistent progress on the project, and harder even still to find the time, and motivation, to post here on Hackaday, about it.

But I preserver.   I hope to be posting at least one or two projects in the "now" time-frame here on Hackaday, as I also project being able to hopefully be able to post the Looper project, as well as use it in some YouTube videos, in the next couple of weeks.

And please note that the Looper is only a rudimentary demonstration (though a needed part) of what this system can do.

The 3D printed looper box, which I will be showing you, even though only stereo in this incarnation, with it's rPi, teensy 3.2, and Octo 6x8 sound card, as well as a 7" touch screen, is actually, more or less, a complete development platform for arbitrary 6-in, 8-out audio applications.  Guitar effects?   Mixer?   Synthesizer?   All of em?

And note that the hardware (circuit boards, 3D printing stuff) could ALSO be used for any linux based applications that could take advantage of the sound card..

But, finally, to the point, what this project represents is a self contained, from the ground up, platform for developing rPi audio applications in C++, that does not require a linux distribution or a lot of development environment dependencies.

I hope someone besides me thinks this was at least worth some of the effort :-)

---

That's it for now.  Hopefully you will see some new project(s) very soon.

- Patrick