A hardware implementation of Fluxly, a software musical physics looper in which you create new instruments by spinning colorful fluxum.
Fluxly is written in C++ and uses the openFrameworks toolkit for creative coding. openFrameworks is aimed at making coding easier for artists and designers and provides convenient libraries or addons that provide an embarassment of riches for someone willing to take it on. Fluxly uses the built-in OpenGL capabilities of openFrameworks, and the OfxBox2d port of Box2D and danomatika's libPd addon.
There are a few parts to the patch: an input stage, the Yak Shaver engine, and an output stage, which handles panning and reverb. There are also the "scopes" that are arrays that hold the data of the sample so they can be rendered in the app. A block diagram looks like this ("Ofx" below refers to the openFrameworks app that the patch is talking to):
The main interface to the app has the four "subunits" at the top, an interface for testing the on/off and tempo controls, and a section at the bottom for loading the default values. Clicking on the messages to the right will load different samples into the 8 yak shaver channels:
The Yak Shaver engine makes extensive use of send and receive blocks (the "s" an "r" blocks) as well as throw~ and catch~ blocks, which are the audio signal equivalent. The "r" or receive blocks here will catch messages sent from within the patch or from the openFrameworks app.
Each of the Yak Channels (there's 8 of them) loads a WAV file into a buffer and plays it back as long as the on/off toggle is on. The tempo (marked "multiple" here) controls how fast the file is played back. The openFrameworks app sends a message to this inlet every time the angular velocity of one of the Fluxum circles changes. If the app sends a message to the "filename" inlet a new file is read into the buffer.
Each of the voices is panned in the final output depending on the X coordinate of the Fluxum looper. This subunit of the output stage handles the panning of each channel:
Each of the panned outputs of the 8 channels are added together and run through the Pure Data "Freeverb" processor. The inputs are sent as a message from the app depending on the rotation of the white Fluxum in each scene. The output is toggled on or off when the sample selector is chosen.
Recording is handled by the input stage, which uses Pure Data's analog to digital converter to read in microphone audio and save it to a buffer. When the stop button is pushed a message is sent and the buffer is trimmed to size and saved to the app's documents directory. Right now the limit is about 10 seconds (400000 samples at 44.1kHz, as seen in the "resize" command upper right), but that could easily be changed if there's interest in longer samples.
The audio output is also sent to the "scopes" subunit, a group of arrays that are sent back periodically to the main app and used to draw the waveform on the screen.
One of the benefits of using openFrameworks is that it provides a simplified framework for writing your app. Anyone familiar with the "sketching" approach of Processing or Arduino wil recognize an openFrameworks app: it consists of a setup() function called once, an update() function called repeatedly, and a draw() function called at a certain framerate. The draw() function should be as lean as possible, focused just on things that need to be done to draw a frame of the screen play. The update() function is where any number crunching should be done. There are also convenient callbacks for handling user interface events like touches.
Fluxly uses two openFrameworks addons: ofxPd as a wrapper around Pure Data and the Yak Shaver patch, and ofxBox2d which provides the physics model. Here's an illustration of how they interact:
Fluxly began in 2008 as a wizard duel game where you had to make your own game controller in hardware in order to play it. The first demo controllers were cigar boxes with pennies, an Arduino and a USB connection. It was originally written in Flash using the Box2d physics engine (the same one used by the original Angry Birds) and ran on a desktop computer.
There were about 20 different aspects of the game that you could control: gravity, wind, lightning, etc. The idea was that participants would use Arduino to make a controller (there was a Fluxly Arduino library), and whoever could make the best controller would have an edge in the game. It was pretty fun; we launched it at the first RI Maker Faire, Foo Fest, and a bunch of other events. It even got a mention in Boing Boing.
The original Fluxly character was based on a whiteboard drawing by my daughter Louise that I turned into a sketch. For the game Arley Rose Torsone turned it into lovely pixel art.
Arley and I and Brandon Edens began meeting on Tuesday afternoons to teach ourselves how to make apps. The first project we worked on was another Louise-inspired game called Space Wozzle, an episodic sci-fi collection of mini games riffing on classic arcade games. We made a few iterations but we realized the truth about making video games is that they have to be fun and they have to jell, and I'm afraid Space Wozzle ended up having too any moving parts.
Later, we used the same "meet every Tuesday for a year and see what happens" business model with Elliot Clapp and Shawn Greenlee to work on some musical apps. Elliot and I had just finished the first year of Fab Academy and had made a few hardware musical projects at that time, like the Fluxamasynth and the Fluxalodian. Meanwhile Shawn had taught a great three week workshop on Pure Data called the Pure Data Club, which was how I started learning the tool. Another happenstace was the release of Peter Brinkmann's libPd, which allows you to enbed Pure Data as a sound engine in embedded or mobile devices. Shawn, Elliot and I made about 6 libPd apps in a year, two of which we thought were interesting enough to launch in the app store: Noisemusick and DR-OM.
The Noisemusick app was based on a hardware kit I had made at AS220 that we used in Learn To Solder workshops.
After that I was at O'Reilly Media and Maker Media for a few years, during which I had the pleasure of working with Peter Brinkmann on a book about libPd called Making Musical Apps. I also worked with Ray Wilson on his book Make: Analog Synthesizers, which is when I spent more and more time exploring electronic musical instruments.
I made a bunch of little instruments and sonic appliances at this time, one of which was the Yak Shaver. The Yak Shaver was a resitive touch interface to a Pure Data patch that was inspired by the Rhythmicon. The Rhythmicon (below right) was essentially the world's first electronic drum machine, which Henry Cowell commissioned from Leon Theremin. It was optical and operated on spinning tone wheels; each key on the 12 note keyboard would play a pulsing tone at a multiple of the previous key. My Pure Data patch emulated this, but let you read in a WAV sound sample file and play it back at different multiples of tempo.
Since 2014 I've been at AS220 Industries, which is a Fab Lab, Printshop and Media Lab in Providence RI. It's a great place to make friends and meet interesting people. Fluxly is directly influenced by many members and coworkers at the Industries: Jacque Bidon's moire pattern books, Aisha Jandosova and Melita Morales' Maker Camp activities (mutant instruments, gear trains, and more moire!), Tycho Horan's risograph color experiments hanging next to my worktable, Chris Anderson's experiments with the Benjolin synthesizer we made in a workshop, and more. Singer Laurie Amat (and coordinator of the PVD Loop festival) is a frequent Industries...Read more »