The system can be asked questions, told to remember things, made to serve information. Complex sequences of physical events can be arranged to occur without human interaction, or in response to human interaction. It makes coffee, and will deliver it. It will collect trash. It shows you the status of the stock market, or of the weather, or of a custom dataset. It can cleanly stop a machine in an emergency. It unlocks a chemical cabinet for an adult, but not for a child. It keeps the human safe, preserves data, and preserves itself to keep maintenance minimal and unobtrusive. It does not become obsolete in less than a year. It will do whatever it is enabled to do.

Almost every component in the system is optional, which makes it highly tolerant to failure. The presence or absence of devices, each with a small number of capabilities, determines the capabilities of the system as a whole.

I call my reactive machine units "Reactrons". These devices have a fairly simple interface that amounts to listing what small set of abilities they have, and control points to execute those abilities. These devices are classified into a handful of groups:

This project introduces Integrons and Recognizers as separate, discrete machines. The others are basically all familiar hardware, with a small control board added to provide them the ability to interface with the system.

What this system does:

The main idea is to reduce the complexity and increase the number of machine nodes that are constantly on and around us. Here is a system diagram of the network.  Note that it contains exemplars, and a multiplicity of units exist beyond this scope. The only unique and non-optional thing is YOU, and your time and experience, and that is the whole point.

They detect our position and do things asynchronously so that our needs are anticipated, for a high percentage of the time. Verbal commands and line-of-sight status indicators give a way to interact with the "culture" of nodes, but passive interaction is preferred.  In order of priority:

It is my hope that most of the interaction is #1 and #2, thereby allowing you to move through your life without interacting actively with the machines, most of the time, analogous to the way the doors just open for Maxwell Smart (https://www.youtube.com/watch?v=sWEvp217Tzw) without him breaking stride, only with more complex results than opening doors.

Here is a system diagram of the Integron unit itself, showing the internal components and how they are expected to interact. The only task of this unit is human integration with the network of nodes, as described above. This post describes what is currently working and what remains as of August 20th 2014.

The simplicity of each node is a huge factor, reducing MTBF for every node, and keeping costs low. Simple machines just work better and last longer. And you can have multiples so that when one breaks another steps in. This allows you to remove maintenance tasks from the critical path of your human activity. Save them up for the chore weekend, or delegate them. The simpler a machine is, the higher the chances are that one can build a machine to do the maintenance.   At critical mass, we have simple machines maintaining simple machines, and we can step away.  But until then, this system will allow you move those tasks to more convenient times, if configured with redundancy.

Some examples may illustrate how I try to leverage complexity from simplicity.

Locked cabinets: Collectors in the room detect my position and some physical characteristics. A set of Recognizers statistically calculates that it is me, or at minimum, an adult. A Processor continually looks at my position, and its relationship to the known static location of the chemical cabinet. If I come close, the associated Energizer is commanded to trigger a solenoid to unlock the cabinet. If the Recognizer does not think it is me, the cabinet will not unlock.

If I wanted to add extra security measures with identity authentication, one approach would be to add a verbal command, but that is not the approach I like to take. Instead, the Recognizer can take data from Integrons (which contain Collectors, audio data) to associate a voice of my frequency with my position. If I speak, the Recognizer can compare the frequency to my known voiceprint range and further statistically narrow the identity of the person approaching the cabinet. What if I don’t say a word? Is my car here? (It has a transceiver.) Face recognition at any point along my travels? The Recognizers’ only job is to identify me. High precision is not an easy job by any means, but it is not encumbered by having to do all the other tasks. Optimally, I don’t want to interact with the system at all - cabinet, just be unlocked when I pull on the handle. Lock it again when I am out of arm’s reach, as the door is no longer in my control.

All the data is local, and belongs to me. It is not shipped off to some Orwellian remote server.

If a helper is in the room, and does not have authenticated access to the cabinet, but I want him to bring me the ferric chloride, I say to an Integron, “open the cabinet”. The Integron parses the speech but that’s about it. It then queries the other nodes along the lines of “does any node know how to deal with this request?” There are several that do, and they are known ahead of time. The Recognizers identify me and my helper as distinct entities. The Processor sees that I am not near a cabinet, but that my helper is, and is not a child, according to criteria I have defined. It contacts the Energizer to open that cabinet. Alternatively, I could say to the Integron something like “grant chemical cabinet access”. The Integron language parse finds the right recipient, the Processor. The rule is changed, and the Processor treats the guest entity access to the specified cabinet as if it were my own, until the entity leaves the scope of the Recognizers. The door unlocks, but only that door - I specified chemical cabinet, not others. It is just a function argument at that point.

The two different workflows are not supported by modern interfaces such as key lock, or a thumbprint swipe. Intelligence would be built into the cabinet, and it forces you to take time to serialize your activity, bottleneck it at that interface. After fifty thumb swipes, twenty of which failed, you will probably just leave the cabinet unlocked rather than waste the time or suffer the annoyance. If you get a new cabinet, you’d have to replace the entire system. If you have ten cabinets, you have to buy ten individual systems. In the above scenario, you would need to replace only the one Energizer tied to the cabinet door.

Each component is actually very simple. The Integron needs to have a fair amount of processing power to process speech, and the Recognizer (which can be hosted on the same SBC Linux system) needs to have sufficient power to do near real-time statistical adjustments. The other components only need a small embedded microcontroller to handle its rules and actions. Inexpensive and reliable.

Coffee: I like to have coffee when I arrive at the lab. I can’t use a timer because my schedule is not predictable. A Collector detects the presence of my car when it arrives. A Recognizer understands that it is my car, and that it has arrived, and sends this information to the Processor, which uses this as a trigger to make coffee. It commands the Energizer. The coffee is ready by the time I get to the machine. No time lost interfacing with a machine, they can do all this without me.

Later in the day, I want more coffee, using a different workflow and a different combination of nodes. I can say to an Integron “coffee, immediately”. The Integron parses the speech and locates the same Processor. It may then command the Energizer to make the coffee.

Additional control paths may be supported. For instance, the Processor may query my biometric shirt, a Collector. If it is transmitting a pulse rate that I have previously deemed too high for more coffee, the Processor may send the interrogative to the Integron “Your pulse rate is high, are you sure you want more coffee?” whereupon the response is parsed and returned, and acted upon.

In a different scenario, the Processor may find that the coffeemaker is out of water or a coffee charge. It can alert me via the above workflow, or if material Transporters exist in the system, it may be able to simply refill the water or recharge the coffee and act upon the request. Those modules are optional, but if they exist and are online, they increase the utility of the system.

I don’t build a complex voice-controlled coffeemaker with yet another smartphone app to control it. I don’t want to serialize my time that way - these tasks the machines can do without me. And when my coffeemaker breaks, as it certainly will eventually, I don’t throw out a lot of tech. It’s just a small control board that controls the few abilities of a coffeemaker (basically, the brew button). It can be replaced easily. No need to retrain the whole system with my rules - the Processor is separate, and is also backing itself up, so that when it fails, it can be easily replaced.

Other optional components include other material transporters, such as a simple payload transport robot. It is solely focused on moving a payload from point to point. It has the ability to transport the coffee from point of brewing to my point of existence, but that is all; it is built to do one task - move stuff. It does not care about making coffee or parsing speech, and can’t do those things anyway. Once integrated with a Processor rule, it can be invoked by a finished coffee brew, and through a Recognizer, understand where to go to deliver the coffee. What it is not, is a complex, voice-controlled, coffee-brewing waiter that understands the whole task from request to fulfillment. It is an optional component that increases the utility of the entire machine culture. It is also not singular - if several are online, the closest one could be dispatched for the task (to optimize for speed), or the one that has been used least (to optimize for wear). If one is busy with a different task, or goes offline to recharge or for maintenance, another can do the task. If none are online, I have to get up and get the coffee, but I am still better off because the coffee made itself.

In the two examples, there are shared components: the very same Integrons, Recognizers, and Collectors. Other defined workflows have even more overlap of the same components. At a certain critical mass, the number of complex outcomes starts to dwarf the number of simple components. With further development, the utility of the system only improves.

An Integron need not be something home built - it can be any device that can run an app - laptop, tablet, phone, or otherwise. This system is currently being integrated with Google Glass. A third example illustrates usage.

Circuit diagrams: I used to waste a lot of time referring to diagrams and pinouts on datasheets. I probably still spend the same amount of time on the diagrams themselves, but now I do not waste the time obtaining them. I wanted to eliminate all the time wasted putting my soldering iron down, going over to the laptop, logging in, searching around, finding the datasheet excerpt I needed. By the time I have my iron back in hand, I forgot - was it pin 1 or 2? I have to refer back again. A minor annoyance, but when repeated ten thousand times, amounts to a large amount of my lifetime being wasted. Instead, I invoke my Integron, “Display 2N7002 pinout”. A Processor obtains the excerpt I want from my local storage. A very simple Glassware displays the result in a Glass “immersion”. I do not have to put the iron down, or even avert my gaze from the circuit I am working on. I don’t need to remember anything, it is just there. I can switch to the next component easily.

These examples are from my actual life. This leads me to:

Why I built this system

This system has been in development for about fifteen years, with the major acceleration in development in the last five, when small processors became powerful enough. Initially I needed a system for data acquisition in an environment hostile to small electronics (high voltage, high vacuum, rapidly changing electrical fields, particle bombardment, vibration). I came up with a system that had small nodes that would back each other up and continue as a system when one failed.

Eventually I realized I could use this system for other tasks, so began to add components to do various singular tasks, mostly control systems for my various equipment. At a certain point, I would seek to do something new and realize that I had all the hardware components necessary, I just needed some more software to tie the nodes together in a particular way to achieve the purpose.

The reason I decided to convert this system to life augmentation is to stop the waste of precious time. I have better things to do than to wait for machines to do tasks. Maybe I am not doing the task itself, but if I am waiting for it, I am spending my life that way. We used to talk about how many years of a person’s life is wasted commuting in a car. This kind of time waste is even more insidious, it is a small but constant drain, but still a drain.

The proliferation in recent years of websites with some service I wanted or needed was also making me crazy. It’s not so much the management of hundreds of logins, (for that we will have Mooltipass!) it is the time one is forced to take doing this repeatedly. Particularly maddening are sites that time out and force you to log in again. This human-machine interaction is not infinitely scalable into the future. At a certain point, you spend more time logging in than using the services. (Auto login? Sure, now the security is defeated by that compromise. The whole paradigm is wrong.) Now add to this less-than-optimal interfaces. Swyping a strong password on your five inch smartphone? Takes forever, or at least longer than I care to spend. Phones are great interfaces for some things, terrible for others. But finding a real keyboard somewhere, also potentially time consuming.

I want to spend my time differently. Not absorbed in a tiny screen while things happen around me. Not stressed out by the constant ding of notifications invoking my fight-or-flight response. I needed a machine interface that was minimally serializing or intrusive, and could cover 80% of what I needed machines for. I wasn’t looking for a general purpose interface for everything. I don’t want to write code on a phone, and certainly not via voice recognition, nor do I need an entire laptop just to show me a green light because the weather is good. I realized that an advanced annunciator light was basically all I needed, and with speech as an input method it could really be accomplished with very simple hardware.

All sorts of time wasting things exist, it’s not just the electronic stuff. I did a workflow analysis of my physical life and identified many places where time could be saved by letting machines not serialize my activity through a less-than-optimal interface, like cabinet locks, or through serial actions requiring human input where unnecessary.

So I built this system. It’s still expanding and developing. It has reduced my electronic burden, and personal stress. It gives me more time for the things I value. And it’s kind of cool. Is it “build the future”? Maybe. Right now it’s just a tool I use in the present. But I think there is potential...

For more on the system paradigm, see this post.