This project has grown up to its own documentation website : 

https://www.homesmartmesh.com/ (Home Automation Frameworks, Microcontrollers, Wireless Networks, Applications, with tutorials and references to the documentation of protocol standards and hardware manufacturers)

If you would like to get support, give feedback or discuss new ideas related to this project ?

It's possible to directly discuss on the forum

https://homesmartmesh.discourse.group/

New Website

fluent website using hugo, very comprehensible Left and Right table of content

https://homesmartmesh.github.io/

https://homesmartmesh.github.io/docs/raspi/readme/

Follow up project

Hackaday Project

#Smart Home 3D Webapp

Control your home like a Video Game served from your local pi

This idea is to take advantage of the experience from the used frameworks in this project to combine them in a homogeneous 3d web app. The main concepts are :

• Full home status in singe blink !
• Any device control is one touch away !

more info on https://hackaday.io/project/169046-smart-home-3d-webapp

Back to this project : Used devices

• Aqara sensors are used with zigbee2mqtt.
• Hue lights are used with Hue bridge and a python API to access the REST service so that both vendor app and custom apps can be used at the same time. I found the python hue API more practicle than the zigbee2mqtt interface (Note that Hue allows the bridge usage with a smartphone app or through the REST in the local network without any account).
• The nRF52 sensors are DIY RF mesh open source HW/SW from #nRF5 Custom Mesh Network, that join the rest of thesensors at mqtt level.
• The bed heater is simply layin in gihub, the ESP32 code is in this repo and its mqtt python client is here.
• For miflora, I used this miflora-mqtt-daemon, which integration in the rest of the environment worked out of the box, but actually plants are boring and it didn't bring any interesting use case.
• The neopixels panel and its webapp are detailed in #Neopixels Animations on ESP32 MQTT Json Webapp.
• Shelly power control and monitors, but require a local time server, see https://hackaday.io/project/20388-home-smart-mesh/log/174162-happy-shellies-in-a-local-network

More details about deployment at https://hackaday.io/project/20388-home-smart-mesh/log/169353-sensors-to-services-deployment.

For any question or recommendation, feel free to use the comments section.

Aqara and Eurotronics hand in hand to save energy

and make life more convenient for users. In this home automation section, aqara window contact sensors are used to send a window open alert to the Eurotronic thermostat, once the alert is disabled, the thermostat resumes the previously set temperature.

The Spriit (google "eurotronic thermostat zigbee" ) has the spirit of devices compatibility introducing a zigbee product, which support is added by the zigbee2mqtt community.

In this code section below, multiple windows contacts ("apertures") are aggregated and a notification is sent when the state changes, the full code is available here.

Python Client injects selected MQTT traffic to influxdb

This project has been constatly evolving, from custom RF to zigbee sensors to DIY Thread as depicted below. It is a collection of ideas and sample code and not focusing on a single software or framework.

Thread, OpenThread, nRF52-dongle, firmware Coming soon

The evolution of WPAN (wireless private area network) has led to this moment where the 802.15.4 (already in use by zigbee) joins the ipv6 in the Thread standard managed by the threadgroup. Google, behind Nest provided an open source implementation : openthread.  Nordic has provided a very cheap and convenient nRF52840-dongle supporting all sorts of modern wireless protocols and with a direct usb connection (for bootloader sw update, logging, interfacing), sure you can get the nRF52 dev kit rather for development, but hacking the dongle is more fun and given the size and cost you can have many to test with real networks:

I prepared an OpenThread mind map here, have x5 of those dongle, managed to have the USB running to log the events of the MQTT-SN sample (to be shared soon in github)

Given the beauty of the tools and the framework, released around a raspberry pi border router, I'm glad to lead the next generation of custom low power wireless sensors toward the Thread standard, and they're likely to easily get compatible with other off the shelf future smart home products !

(You can contact me in case you'd like to help in this direction)

zigbee2mqtt 2 influx 2 grafana

• I hoped to replace my nRF52 low power sensors with off the shelf zigbee but that's not gonna happen any time soon as the ones I designed with the custom protocol are competitive in the logging use case. I have now an Aqara temperature sensor running over zigbee2mqtt logging side by side with my custom sensor on a grafana dashbaord (example snapshot here). 

The high resolution of the nRF52 custom sensor comes from a stunning 30 sec period log (1 min per parameter), that's been running for alomst 6 months already on a CR2032 with a +4dB configuration, so no wonder how long the CR2477 version can last.

• When it comes to buttons, all Aqara buttons and wireless wall switches are replacing the DIY buttons, as wireless button is becoming a less attractive topic of DIY project.

All the environemnt and latest Python scripts are under https://github.com/nRFMesh/nRF52_Mesh

The rest of this project description is not yet updated to the new zigbee generation.

Home Smart Mesh Website

Home Smart Mesh with comprehensible structure, menu for different boards, mesh concept,...

Node-red : from Aqara motion button to esp32 RGB Leds

Below, an esp32, like the one visble on the test board, is hidden in the white 3d printed tower. 7 WS2812B have been united for the purpose. An 18650 battery power module like the one visible on the right has been used inside the tower.

Everything becomes easy when iot unite around MQTT to talk the same language. The Aqara motion button has been paired with zigbee2mqtt and picked up directly from there by node-red. I developed an esp32 mqtt application that takes json payloads to turn the grb leds with on and off. The esp32 is developed using the esp-idf, and the source code is available on github : https://github.com/HomeSmartMesh/esp32_iot/tree/master/rgb_led

For those not familiar with zigbee2mqtt, the aqara button click produce such publish :

zigbee2mqtt/motion button {"click":"single","battery":"100.00","voltage":3055,"linkquality":92}

 and to turn all leds red by command line (broquer on localhost) :

mosquitto_pub -t 'esp/rgb led/all' -m '{"red":60,"green":0,"blue":0}'

And the rest of the rules can either be handled by node-red as the example above, or by python for a much more flexible rules. Note that direct rules connections as simpler with node-red, but as soon as some logic is needed, then even for beginners python is more readable than node-red. github for python rules example : https://github.com/nRFMesh/nRF52_Mesh/blob/master/raspi/esp32/esp32.py

nRF52 Environemnt description

I split the nRF52 in a separate Hackaday project and github repo as the potential is so huge that it can soon take this Home Smart Mesh to a whole new level

• nRF52 Mesh Hackaday project
• nRF52 Mesh Github repo : Fully functional using nRF SDK 15 with a clean application and boards declaration

nRF24 + nRF51 + nRF52 Mesh !

Hardware 1 : nRF52 Sensors

Hackaday Project	nRF52 Sensors
Official website	nRF52 Sensors in Home Smart Mesh
Github repo	nRF52 Sesnors firmware
Tindie Product page	Not to be deployed

Hardware 2 : nRF51 Sensors

The nRF51 sensors Tag is a low cost (8 ~ 9 €) sensors Tag that can talk Bluetooth and custom radio and equipped with sensors that allow very versatile applications. See further below on why custom radio is way more efficient than Bluetooth in the case of sensors logging.

• MPU6050 (x3 axis acceleration and x3 axis gyroscope)
• AP3216 (Ambient Light)
• BMP180 (Temperature, Pressure)

=> Finally a clean git integration based on the nRFSDK : although most of the projects are based on mbed-os, unfortunately the nRFSDK integration inside mbedos (of the time of the evaluation) only supports usage with softdevices which take control over the HW and only allow bluetooth, not esb custom radio, so I had to fall back over a bare nRFSDK usage and I was surprised how clean the experience was, the support is great (googling issues can be quickly found on the forum), so now a subset of the required sdk is integrated as a submodule in a an SDK based project.

=> Note that current available application on the github repo is only broadcasting x3 axis acceleration, other parameters are still to be added (should be fast forward).

Hardware 3 : nRF51 Beacon

The nRF51 is a market available bluetooth beacon , but I hacked it to turn it into a custom RF Mesh beacon. Main code already on github. I used the nRF SDK with the proprietary_rf examples esb_low_power_ptx. The modified code makes the beacon behave as a dash button talking to all other mesh RF nodes, every press ends up in the database and can feed any python MQTT script (actually every press, and every release are separate logged events). Given the low cost of around $4, this makes it an ideal IoT ally. I program this beacon with a jLink Edu and I designed for it this 3d printed needle adapter

Hardware 4 : STM32 IoT Board

This board issue of the #STM32 Blue pill IoT expansion boards  project, comes with multiple functions

• APDS9960 : Gesture and proximity, Light Color sensing
• WS2812B : RGB LED status
• BME280 : Temperature, Humidity Pressure
• nRF24L01+ PA LNA + Shield : Long range repeater
• HCI : Host Controller Interface, talking to Rasp server c++ App and to Python.
• mbedos : based application with arduino like c++ call backs and usage facility.

Hardware 5 : STM8L Cell Coin Sensor Tag

• STM8L151F3P6
• BME280 : Temperature Pressure Humidity
• MX44009 : Light
• MRMS211H : Magnet sensors
• Usage of modules very simple to solder, can reduce the cost vs on the shelf tags by soldering only the required module
• cost-less debugger and IDE
• power measures : to be provided during next week (beginning December)

Fully open source available on github 

• STM32 Gateway and repeaters, Raspberry pi server MQTT to DB, and react webapp on github
• The latest STM8L Coin Cell SW on github
• The program broadcasting BME280 and MAX44009 values over the nRF24L01+ on that repo is here 

Raspberry pi IoT Software

• C++ RF Gateway: RF Mesh network to MQTT
• Mosquitto: MQTT Broquer
• Mongodb: discarded
• InfluxDB: Timeseries Database for simple values over time
• Grafana: Generic exploration of time series data
• React+D3js: frozen status
• OpenHAB2: indirectly supported, not much relevant here
• NodeRed: Same, indirectly supported

Grafana Interactive dashboard snapshot

• Follow this Interactive snapshot link
• There you can experience an interactive dashboard showing one day of measures similar to the screenshot below
• Notice the logarithmic display of the light values that show similar curves for different light sensors on different rooms, the passing clouds patterns can be seen in the signal, as well as differentiation between natural and artificial light (sudden change). This is just the beginning as further sensors already deployed bring even the color of the ambient light.
• Installing your own InfluxDB and grafana server would get you the same for your daily real time measures, all of this owned by you, locally on your raspberry pi.

Mesh Protocol

MQTT as a sensors interface for time-series databases

MQTT as a Multi-level interface for Actions

nRF51 Dash button - 3d print your own

• Link to these modelling attempts

More buttons

How to program the nRF51 Button

• Link to the 3D Model of the needle adapter

Is this a mesh ? Sure, with End to End Acknowledge !

We can see two use cases here, a broadcast and a Peer to peer message that is made reliable with Acknowledge and retries. All the details in the nrf24l01 Flood Mesh Protocol log entry.

The Future of the Mesh

• The STM32 Do it all Nodes are low cost and competitive with solutions (ESP32, Rapberry pi Zero). Using PL LNA with shiel they can reach up to 2100 m (I did not test that), but I tested them with an impressive 4 stories building (x4 levels in between), and the signal was still usable (above 10%).
• The STM8L is being deployed, but batteries have to be changed every 4 months, not a huge success (still looking on low power tweaks).
• The nRF51 is an ideal dash button and promise to have a long life as it can to both BLE and custom Mesh.
• The nRF51 + PL LNA would be an ideal replacement for the STM32 + nRF24, but due to SW environment, the integration of the nRF SDK would require a big effort, the mbed supported SW did not include the custom RF unfortunately (hope to see it in there soon).
• The nRF51 sesnors is a market available TAG that would also be able to do both BLE and custom mesh, same issue with the sensors drivers and the mbed support.

Why not Bluetooth ? Waste of battery with either transfer way

Logging sensors data is different than collecting it inside the tag memory and then offloading them to a smartphone. Logging require transmission of the latest sample as soon as it is acquired, so that every user can access to the latest status from a database which is maintained up to date, from those clients are data analysis clients that would take an immediate action if something happens. While with custom radio, you can optimize the wake up sequence with the time spend on tx mode, using Bluetooth brings constrains that compromise communication robustness in an unknown environment over radio power optimization. Two methods are possible :

• Advertising: has the issue that you have to send many packets to get at least one received in one of the 3 listening channels.
• Connection: has a relatively big number of RF transaction on every wake up which make the RF balance a waste compared to the useful sent data (waste of battery life)

In addition to the battery waste misconception of bluetooth, its integration comes with some other expenses :

Custom RF vs Bluetooth : Simplicity Vs Complexity

• With Bluetooth, you need a bigger SW stack (hundreds of kilobytes, while it is few kilobytes here 2~ 3 KB)
• With Bluetooth, you need a more expensive uC or SoC
• With Bluetooth, you cannot customize the protocol to your needs (Mesh, Ack, ...)
• With a custom protocol you can have a simple mesh starting from few functions, easy to understand and optimize by yourself
• With a custom protocol you can have a huge amount of sensors, using a specific configuration.
• For simple sensors that do not control anything, no door opening, no light switching, security is not relevant as these sensors are beacon like and do not provide more information than what a neighbor could see out of your window.
• The internet security is handled by the raspberry pi that is beyond the custom RF gateway, so you can access your data safely from all over the world, for example I am using a vpn router, so the security is even beyond the rapsberry pi server's hacking.

Links to project Logs

The Logs consist of specific use cases based on this IoT_Framework going into some details of firmware and software :

• Open the bathroom window, Close the bathroom window : A useful use case to help know when to open the bathroom window depending on the humidity level.
• nrf24l01 Flood Mesh Protocol : The mesh concept on how do the nodes communicate with each other, including the protocol bits details.
• Completely Hackable IoT dashboard : The web technology behind the dashboard that allows you to hack the GUI while the access itself is highly secured. A good example on how you can own your IoT devices, private local network, no third parties, not even for remote access!!!

• Hello Mesh - RGB Colors Broadcast : Although the show goes over the broadcast, the focus is how the STM8 managed to control the WS2812B in assembler.

• 60 RGB LEDs shading with that tiny STM8 : Show details on how colors are shaded before being sent.
• Smart LED Panel - MQTT to x216 WS2812B : out of the box MQTT control of RGB Panel color with a smartphone app.

First generation STM8L Sensor Tag

• STM8L + PCB (x10 / 10) + nRF24L01+ : ~ $4 if you're unlucky
• STLink-v2 debugger : ~ $2 you being generous with the seller
• Professional IDE with breakpoints stepping without restrictions (for the full flash size which is already 8 KB covered completely by the demo) => FREE !