Tools and information to build an internet of things that doesn't suck. Do it yourself!
This is a video of how to create a 'hello world' for the device cloud--blinking an LED from a webpage over the internet:
Here's a deeper look at the services for the DIY device cloud:
This diagram shows a simple device cloud 'hello world' application that turns an LED on from a web page over the internet. You can see in the bottom left a web page which is hosted on the broker server. When a button is clicked (1) it communicates to a python web application which is hosted by gunicorn (python WSGI web app server) and nginx (main web server). That web application talks directly to the MQTT broker on the server (2), which will inform any connected clients of MQTT messages. A device runs a device service which is just a simple python script that talks to the broker (3) and turns on/off the LED with GPIO libraries (4). The device service is hosted by the excellent supervisor tool so it will automatically run in the background on the device--all the messy details about properly running a background process are abstracted away by supervisor.
This diagram highlights the two main extension points I see where you can plug in your own code to build your device cloud:
The ansible deployment scripts for both the broker server and devices take care of setting up all the services, configuring them, etc. You only need to place your web service and device service code in a certain location and follow a few simple conventions.
I made a video of creating this simple hello world application and will post it shortly when Youtube is finished processing it.
I've done some cleaning up and refactored the Ansible playbooks to use roles as suggested by Ansible's best practices. I'm not super happy with how the system is configured--right now global configuration is in the inventory file under the [all:vars] group. This is a good spot to configure the entire system in one file but feels a little unwieldily. I'll be looking more into other options here to see if there's a simpler and more flexible way to configure the system.
I've also added quite a few roles to form the base of the server, including:
One more major package I want to install is a Python WSGI app server that can host web applications written in Python. I plan to make a few simple web apps for reading and writing MQTT messages and would prefer to write them in Python with the Flask web app framework. I've looked into some options here and it looks like it comes down to either uWSGI or Gunicorn. From my testing uWSGI is kind of painful to setup so I might go for Gunicorn.
In the past couple days I've added playbooks to the project github source which allow you to provision an MQTT broker server on Amazon's EC2 cloud and set up a basic device cloud. Check out this video to see a walkthrough of setting up a server and communicating with a Raspberry Pi:
I'm still a little new to using Ansibile, but so far am really happy with how simple it's making the process of standing up a server from scratch.
Also I purchased the 'diydevicecloud.net' domain name and am making subdomains of it available to anyone who wants to host their device cloud from an easily reachable address with the excellent FreeDNS.afraid.org dynamic DNS service. You can find the domain here and just need a free account on FreeDNS to get a subdomain. I've even integrated into the Ansible playbook a step to register a regular cron job that will keep your broker server IP address up to date with your FreeDNS domain. Watch the video to see more details of the process.
You aren't limited to using this domain either--there are many other domains on FreeDNS, or you can even buy your own domain and get a static IP in Amazon's cloud.
The next area I want to focus on are client apps. As a part of this project I want to document a few end to end examples of MQTT applications, such as measuring sensor data or remotely controlling devices.
I've started a repository on github that will be the home for all the software output from this project. Since documentation will be an important part of this project I've also created a base for the documentation at http://diy-device-cloud.readthedocs.org/ using the excellent readthedocs.org service. The documentation is actually built from files in the github repository for the project (under the docs folder) based on the sphinx tool. Because the docs are in the repository it's easy to keep them versioned and up to date. For now I've put up a couple docs that recap what I've covered so far in the project logs here.
Here's a high level look at how I see different components fitting together to form a device cloud:
The MQTT broker, mosquitto, is at the center of all communication and lives on a server in Amazon EC2, Microsoft Azure, etc. Along with the broker other applications like visualizations or control apps are hosted on the same server and can communicate with the broker directly. I have some early thoughts on these apps and will save that for a followup post.
Devices in your home network, or really anywhere, connect to the MQTT broker using an encrypted SSL MQTT channel. Normally SSL is a tremendous pain to configure and manage (especially with self-signed certificates), however I've found it's easy to automate the certificate creation tasks using Ansible so it should be painless. I want the device cloud to be secure by default and not require someone to be an expert in security to setup or run the system.
Devices can be any hardware that's powerful enough to speak MQTT's protocol. Embedded Linux boards like the Raspberry Pi, Beaglebone Black, Arduino Yun, Intel Galileo, etc. will be very easy to configure and use with the device cloud. I hope to even provide Ansible tasks that can deploy and configure MQTT client tools and libraries on any embedded Linux device automatically.
What about devices which don't support SSL, like an Arduino & CC3000? In this case mosquitto has a concept of a bridge server which can act both as an MQTT broker and client. Any MQTT messages sent to the bridge will be relayed back up to the parent broker and vice-versa. With this setup it will be possible to build a small, low power Arduino + CC3000 device that periodically connects to the bridge over an unencrypted channel to send/receive commands. Because the bridge lives inside your home network (which is secured with wireless encryption, right?) it's still relatively secure. Certainly more secure than opening the broker in the cloud up to unencrypted MQTT traffic! Again Ansible tasks should be able to turn any Linux gadget into a bridge for the device cloud easily.
Ansible is part of a new generation of open source IT automation tools which aim to simplify the automation of tasks like installing software and setting up servers. You can see more about Ansible from its creator in this recent PyCon talk:
Ansible strives to be simple and requires very little support infrastructure or complex setup. Even though the project is only a few years old, it is already one of the largest on github and home to a healthy ecosystem of modules that can automate almost any server task.
I've been investigating Ansible and expect to make heavy use of it as the primary means of automating the setup and maintenance of the device cloud. The primary output of this project will likely be a collection of Ansible playbooks and tasks that help you build and manage your own device cloud.
MQTT is a publish-subscribe protocol that's perfect as a broker for device to device communication. Created by Andy Stanford-Clark and Arlen Nipper, it has been used in projects like Andy's own 'Twittering House'. You can see more from Andy in this TEDx talk:
Mosquitto is an open source MQTT broker implementation that's under active development, has an established reputation, and is well documented. I've done some informal testing with Mosquitto and it seems like the perfect broker to use in this project.
What is a device cloud?
I consider a device cloud to be a service that provides both a broker for communication between devices and a host for applications that interact with devices. The broker allows devices to communicate to other devices or applications without having to be tightly coupled or directly connected to each other. The device cloud is also an easily accessible host for applications that interact with devices, such as visualizations of data or control of device functionality.
Why do you want a device cloud?