Close
0%
0%

Real World Fluxly, Physics Looper

A hardware implementation of Fluxly, a software musical physics looper in which you create new instruments by spinning colorful fluxum.

Similar projects worth following
Fluxly brings a whole new approach to knob twiddling and sound making. Sound sample loops play back based on the angular velocity of the fluxum players.

This project is to build a hardware version of Fluxly based on the ESP32 microcontroller and gyroscope/accelerometer. The first iteration sends rotation data to a Pure Data-based server that plays back up to 8 channels of sound. The final version aims to render audio on the server and send audio back to each unit for local playback over Wifi.

Fluxly was built using openFrameworks, Box2d, libPd, Pure Data, and the ESP32 toolchain for the Arduino IDE.

In the app version of Fluxly, you can choose from a bank of built-in sound samples, or record your own samples and play them back in either direction at any speed. By touching two Fluxum and pushing them together you can link them into looping gear train machines. You can also enable movement and let the built-in physics engine take over as the Fluxum bounce around the instrument. 

The goal of making a Physical Fluxum is to simulate as much of the ideal software physics model with actual real world physics.

YakShaveriOS3.pd

Latest version of the Yak Shaver patch, which should work standalone as long as you have the freeverb~ extension.

pd - 40.15 kB - 10/02/2018 at 21:28

Download

freeverb~.zip

Freeverb~ is needed for reverb in the Yak Shaver. Unzip in the same directory or install in Pure Data using Deken.

Zip Archive - 31.90 kB - 10/02/2018 at 21:26

Download

samples.zip

The default set of WAV samples.

Zip Archive - 9.98 MB - 10/02/2018 at 21:25

Download

PhysicalFluxlyNetReceive.pd

This is the Yak Shaver patch with the inputs hooked up to a netreceive object to grab data from the gyro over UDP.

pd - 42.75 kB - 10/02/2018 at 21:23

Download

  • Making a Physical Fluxum, Part 2

    Fluxly10/01/2018 at 20:54 0 comments

    Now that the Fluxum controller and the Yak Shaver are talking to each other, the next step is to read the gyro sensor and send some actual data. I started with Kris Winer's MPU9250 example and added the Wifi and UDP broadcast code, this time calculating the angular velocity in degrees per second and sending that as four bytes. The gyro code hasn't been made into an Arduino library so the setup code is a little long to post here (it's in the files section). Here's the loop() code snippet:

    void loop() {  
      
        // If intPin goes high, all data registers have new data
        if (readByte(MPU9250_ADDRESS, INT_STATUS) & 0x01) {  // On interrupt, check if data ready interrupt
            readGyroData(gyroCount);  // Read the x/y/z adc values
            getGres();
     
            // Calculate the gyro value into actual degrees per second
            gx = (float)gyroCount[0]*gRes;  // get actual gyro value, this depends on scale being set
            gy = (float)gyroCount[1]*gRes;  
            gz = (float)gyroCount[2]*gRes;   
      
        }
      
        Now = micros();
        deltat = ((Now - lastUpdate)/1000000.0f); // set integration time by time elapsed since last filter update
        lastUpdate = Now;
    
        sum += deltat; // sum for averaging filter update rate
        sumCount++;
      
        if (!AHRS) {
            delt_t = millis() - count;
            if(delt_t > 50) {
                count = millis();
            }
        }
    
        byte val1 = 0;
        byte val2 = 0;
        byte val3 = 0;
        byte val4 = 0;
        int v = abs(gz);
        
        int thisVal = v;
    
        if (!((thisVal < (prevVal + 2)) && (thisVal > (prevVal-2)) || (thisVal < 5)) && connected) {
          //Send a packet
          udp.beginPacket(udpAddress,udpPort);
          packetBuffer[0] = command;
          packetBuffer[1] = dir;
          if (thisVal < 5) val1 = 0;
          packetBuffer[2] = val1;
          packetBuffer[3] = val2;
          packetBuffer[4] = val3;
          packetBuffer[5] = val4;
         
          udp.write(packetBuffer, 6);
          udp.endPacket();
      } 
      
      prevVal = thisVal;
      delay(20);
    }

    For the first iteration of this project I am only sending the absolute value of the z axis angular velocity. The next iteration will use the command field to indicate the direction of rotation so the sample can play forward or backward.

    Here's the first working iteration of the Fluxum controller:

    (Text from Richard Brautigan's All Watched Over By Machines of Loving Grace, copyleft 1967 by author):

    ...I like to think
    of a cybernetic forest
    filled with pines
    and electronics
    where deer roam peacefully
    past computers
    as if they were flowers
    with spinning blossoms

    Next steps:

    • Play samples forward and backward
    • Make custom PCB with 3 evenly spaced AA batteries for more of a flywheel effect
    • RGB LEDs to backlight Fluxum character
    • Use ESP32 streaming audio example to play sound on the actual device instead of the server

  • Making a Physical Fluxum

    Fluxly10/01/2018 at 20:54 0 comments

    The first step in making the Fluxly controller was to make a jig that would allow the microcontroller to spin. I laser cut a couple of acryliccircles with a cutout and a masonite base to hold the ESP32 module and a battery. A skate bearing is press fit in the center.

    The sensor is a MPU9250 gyro/accelerometer. It has to be centered on the circle so it sits on top of the ESP32 module. I'm using a module that I made called the Fluxamasynth-32; it has an additional sound-making chip on board that I'm not using here. 

    Unfortunately the EAGLE part I was using for the ESP32 module had the I2C lines marked incorrectly (I may have drawn it!) so I pulled the wrong pins out to the header. Instead I used prototyping wire soldered directly to the pins on the module.

    Here's what it looks like spinning:

    Once I had the microcontroller spinning, the next step was to get the ESP module talking to the Pure Data patch running on the computer. I decided to use Wifi rather than Bluetooth, since Pure Data is well equipped to handle messages over the network. I set up a router running OpenWRT as an access point, which each Fluxum will connect to. The laptop running the patch will also connect to the router.

    The idea is to broadcast UDP packets over wifi, using a simple 6 byte protocol: fluxum_id, command, data1, data2, data3, data4. I started with the Arduino example code that comes with the Wifi library and modified to send 6 bytes with random data, just to get them talking. 

     #include <WiFi.h>
    #include <WiFiUdp.h>
    
    // WiFi network name and password:
    const char * networkName = "dd-wrt";
    const char * networkPswd = "**********";
    
    byte packetBuffer[ 6];
    
    byte command = 1;
    byte id = 2;
    
    //IP address to send UDP data to:
    // either use the ip address of the server or 
    // a network broadcast address
    const char * udpAddress = "192.168.1.255";    // broadcast
    const int udpPort = 3001;
    
    //Are we currently connected?
    boolean connected = false;
    
    //The udp library class
    WiFiUDP udp;
    
    void setup(){
      // Initilize hardware serial:
      Serial.begin(115200);
      
      //Connect to the WiFi network
      connectToWiFi(networkName, networkPswd);
    }
    
    void loop(){
      if (connected){
        //Send a packet
        udp.beginPacket(udpAddress,udpPort);
        packetBuffer[0] = id;
        packetBuffer[1] = command;
        packetBuffer[2] = random(255);
        packetBuffer[3] = random(255);
        packetBuffer[4] = random(255);
        packetBuffer[5] = random(255);
        udp.write(packetBuffer, 6);
        udp.endPacket();
      }
      delay(30);
    }
    
    void connectToWiFi(const char * ssid, const char * pwd){
      Serial.println("Connecting to WiFi network: " + String(ssid));
    
      WiFi.disconnect(true);
      WiFi.onEvent(WiFiEvent);
      WiFi.begin(ssid, pwd);
      Serial.println("Waiting for WIFI connection...");
    }
    
    void WiFiEvent(WiFiEvent_t event){
        switch(event) {
          case SYSTEM_EVENT_STA_GOT_IP:
              Serial.print("WiFi connected! IP address: ");
              Serial.println(WiFi.localIP());  
              udp.begin(WiFi.localIP(),udpPort);
              connected = true;
              break;
          case SYSTEM_EVENT_STA_DISCONNECTED:
              Serial.println("WiFi lost connection");
              connected = false;
              break;
        }
    }
    
    

    On the Pure Data side, the netreceive object listens for data on a particular port, 3001 in this case. It receives 5 bytes and the list object splits them into individual outlets. The id byte routes the data to a particular input, command is as yet unused, and the 4 data bytes get added back together. Here's the netreceive patch added to the YakShaver engine:

    There's a packet counter at the lower left that calculates how many packets per second are coming in. The goal is to reach 60 packets per second (currently around 30). 

    Next step is to add the gyro sensor.

  • Making Fluxly, Part 2: The Software

    Fluxly09/08/2018 at 19:28 0 comments

    Fluxly is written in C++ and uses the openFrameworks toolkit for creative coding. openFrameworks is aimed at making coding easier for artists and designers and provides convenient libraries or addons that provide an embarassment of riches for someone willing to take it on. Fluxly uses the built-in OpenGL capabilities of openFrameworks, and the OfxBox2d port of Box2D and danomatika's libPd addon.

    All of the hard work of the sound synthesis is done in the embedded Pure Data patch, the Yak Shaver Engine called Yakshaver.pd, which you can download and open using the freely available Pure Data.

    There are a few parts to the patch: an input stage, the Yak Shaver engine, and an output stage, which handles panning and reverb. There are also the "scopes" that are arrays that hold the data of the sample so they can be rendered in the app. A block diagram looks like this ("Ofx" below refers to the openFrameworks app that the patch is talking to):

    The main interface to the app has the four "subunits" at the top, an interface for testing the on/off and tempo controls, and a section at the bottom for loading the default values. Clicking on the messages to the right will load different samples into the 8 yak shaver channels: 

    The Yak Shaver engine makes extensive use of send and receive blocks (the "s" an "r" blocks) as well as throw~ and catch~ blocks, which are the audio signal equivalent. The "r" or receive blocks here will catch messages sent from within the patch or from the openFrameworks app.

    Each of the Yak Channels (there's 8 of them) loads a WAV file into a buffer and plays it back as long as the on/off toggle is on. The tempo (marked "multiple" here) controls how fast the file is played back. The openFrameworks app sends a message to this inlet every time the angular velocity of one of the Fluxum circles changes. If the app sends a message to the "filename" inlet a new file is read into the buffer.

    Each of the voices is panned in the final output depending on the X coordinate of the Fluxum looper. This subunit of the output stage handles the panning of each channel:

    Each of the panned outputs of the 8 channels are added together and run through the Pure Data "Freeverb" processor. The inputs are sent as a message from the app depending on the rotation of the white Fluxum in each scene. The output is toggled on or off when the sample selector is chosen.

    Recording is handled by the input stage, which uses Pure Data's analog to digital converter to read in microphone audio and save it to a buffer. When the stop button is pushed a message is sent and the buffer is trimmed to size and saved to the app's documents directory. Right now the limit is about 10 seconds (400000 samples at 44.1kHz, as seen in the "resize" command upper right), but that could easily be changed if there's interest in longer samples.

    The audio output is also sent to the "scopes" subunit, a group of arrays that are sent back periodically to the main app and used to draw the waveform on the screen.

    One of the benefits of using openFrameworks is that it provides a simplified framework for writing your app. Anyone familiar with the "sketching" approach of Processing or Arduino wil recognize an openFrameworks app: it consists of a setup() function called once, an update() function called repeatedly, and a draw() function called at a certain framerate. The draw() function should be as lean as possible, focused just on things that need to be done to draw a frame of the screen play. The update() function is where any number crunching should be done. There are also convenient callbacks for handling user interface events like touches.

    Fluxly uses two openFrameworks addons: ofxPd as a wrapper around Pure Data and the Yak Shaver patch, and ofxBox2d which provides the physics model. Here's an illustration of how they interact:

  • Making Fluxly, Part 1

    Fluxly09/03/2018 at 03:09 0 comments

    Fluxly began in 2008 as a wizard duel game where you had to make your own game controller in hardware in order to play it. The first demo controllers were cigar boxes with pennies, an Arduino and a USB connection. It was originally written in Flash using the Box2d physics engine (the same one used by the original Angry Birds) and ran on a desktop computer. 

    There were about 20 different aspects of the game that you could control: gravity, wind, lightning, etc. The idea was that participants would use Arduino to make a controller (there was a Fluxly Arduino library), and whoever could make the best controller would have an edge in the game. It was pretty fun; we launched it at the first RI Maker Faire, Foo Fest, and a bunch of other events. It even got a mention in Boing Boing.

    The original Fluxly character was based on a whiteboard drawing by my daughter Louise that I turned into a sketch. For the game Arley Rose Torsone turned it into lovely pixel art.

    Arley and I and Brandon Edens began meeting on Tuesday afternoons to teach ourselves how to make apps. The first project we worked on was another Louise-inspired game called Space Wozzle, an episodic sci-fi collection of mini games riffing on classic arcade games. We made a few iterations but we realized the truth about making video games is that they have to be fun and they have to jell, and I'm afraid Space Wozzle ended up having too any moving parts.

    Later, we used the same "meet every Tuesday for a year and see what happens" business model with Elliot Clapp and Shawn Greenlee to work on some musical apps. Elliot and I had just finished the first year of Fab Academy and had made a few hardware musical projects at that time, like the Fluxamasynth and the Fluxalodian. Meanwhile Shawn had taught a great three week workshop on Pure Data called the Pure Data Club, which was how I started learning the tool. Another happenstace was the release of Peter Brinkmann's libPd, which allows you to enbed Pure Data as a sound engine in embedded or mobile devices. Shawn, Elliot and I made about 6 libPd apps in a year, two of which we thought were interesting enough to launch in the app store: Noisemusick and DR-OM.

    The Noisemusick app was based on a hardware kit I had made at AS220 that we used in Learn To Solder workshops.  

    After that I was at O'Reilly Media and Maker Media for a few years, during which I had the pleasure of working with Peter Brinkmann on a book about libPd called Making Musical Apps. I also worked with Ray Wilson on his book Make: Analog Synthesizers, which is when I spent more and more time exploring electronic musical instruments. 

    I made a bunch of little instruments and sonic appliances at this time, one of which was the Yak Shaver. The Yak Shaver was a resitive touch interface to a Pure Data patch that was inspired by the Rhythmicon. The Rhythmicon (below right) was essentially the world's first electronic drum machine, which Henry Cowell commissioned from Leon Theremin. It was optical and operated on spinning tone wheels; each key on the 12 note keyboard would play a pulsing tone at a multiple of the previous key. My Pure Data patch emulated this, but let you read in a WAV sound sample file and play it back at different multiples of tempo.

    Since 2014 I've been at AS220 Industries, which is a Fab Lab, Printshop and Media Lab in Providence RI. It's a great place to make friends and meet interesting people. Fluxly is directly influenced by many members and coworkers at the Industries: Jacque Bidon's moire pattern books, Aisha Jandosova and Melita Morales' Maker Camp activities (mutant instruments, gear trains, and more moire!), Tycho Horan's risograph color experiments hanging next to my worktable, Chris Anderson's experiments with the Benjolin synthesizer we made in a workshop, and more. Singer Laurie Amat (and coordinator of the PVD Loop festival) is a frequent Industries...

    Read more »

View all 4 project logs

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates