Close
0%
0%

Home Smart Mesh

Mesh of sensors, zigbee, custom RF, BT, wifi, all to MQTT Gateways on Raspberry Pi with database, dashboard and webapps.

Similar projects worth following
The custom RF mesh started with nRF24L01+, STM8L to STM32 with mbed-os, then nRF51, nRF52832, and nRF52840. Then Xiaomi off the shelf zigbee sensors kicked in. On server side, Raspberry pi, MQTT Mosquitto, json, influx time series database and Grafana as dashboard, also custom web with websockets, reactjs webapp, d3js and even 3d webapps with three.js. Python used for different services. The ESP32 is used for non low power sensitive projects.
All SW and HW is opensource on github.

This project has grown up to its own documentation website : 

https://www.homesmartmesh.com/ (Home Automation Frameworks, Microcontrollers, Wireless Networks, Applications, with tutorials and references to the documentation of protocol standards and hardware manufacturers)

If you would like to get support, give feedback or discuss new ideas related to this project ?

It's possible to directly discuss on the forum

https://homesmartmesh.discourse.group/

video demo of how to collect all webapps in a single one using hugo and markdown.

New Website

fluent website using hugo, very comprehensible Left and Right table of content

https://homesmartmesh.github.io/

https://homesmartmesh.github.io/docs/raspi/readme/

Follow up project

Hackaday Project#Smart Home 3D Webapp 

Control your home like a Video Game served from your local pi

This idea is to take advantage of the experience from the used frameworks in this project to combine them in a homogeneous 3d web app. The main concepts are :

  • Full home status in singe blink !
  • Any device control is one touch away !

more info on https://hackaday.io/project/169046-smart-home-3d-webapp

Back to this project : Used devices

More details about deployment at https://hackaday.io/project/20388-home-smart-mesh/log/169353-sensors-to-services-deployment.

For any question or recommendation, feel free to use the comments section.

Aqara and Eurotronics hand in hand to save energy

and make life more convenient for users. In this home automation section, aqara window contact sensors are used to send a window open alert to the Eurotronic thermostat, once the alert is disabled, the thermostat resumes the previously set temperature.

The Spriit (google "eurotronic thermostat zigbee" ) has the spirit of devices compatibility introducing a zigbee product, which support is added by the zigbee2mqtt community.

In this code section below, multiple windows contacts ("apertures") are aggregated and a notification is sent when the state changes, the full code is available here.

Python Client injects selected MQTT traffic to influxdb

This project has been constatly evolving, from custom RF to zigbee sensors to DIY Thread as depicted below. It is a collection of ideas and sample code and not focusing on a single software or framework.

Thread, OpenThread, nRF52-dongle, firmware Coming soon

The evolution of WPAN (wireless private area network) has led to this moment where the 802.15.4 (already in use by zigbee) joins the ipv6 in the Thread standard managed by the threadgroup. Google, behind Nest provided an open source implementation : openthread.  Nordic has provided a very cheap and convenient nRF52840-dongle supporting all sorts of modern wireless protocols and with a direct usb connection (for bootloader sw update, logging, interfacing),...

Read more »

PCBs_Design_gerbers_bom.zip

Eagle design sources, cam and bom with Hacker's license, own this design.

x-zip-compressed - 149.28 kB - 04/27/2017 at 18:42

Download

  • 1 × STM8S103F3 breadboard Microprocessors, Microcontrollers, DSPs / Microcontrollers (MCUs)
  • 1 × nRF24L01+ Nordic's 2.4GHz Radio Transceiver
  • 1 × STM8L151F3 Low Power 8bit Microcontrollers in TSSOP20 Package
  • 1 × BME280 Sensor for Temperature, Humidity, Pressure
  • 1 × MAX44009 Ambient Light Sensor

View all 12 components

  • Happy shellies in a local network

    Wassim02/23/2020 at 15:36 1 comment

    The diagram above shows how to achieve an isolated IoT environment that has no internet access and devices are running in a local network.

    An additional security measure is to isolate IoT devices in their own local wifi network with their dedicated router. This way, even if any IoT device gets hacked, it cannot access the rest of your main router network where you might have open drive shares or unprotected devices, and it cannot access the internet so no privacy concerns. Note that devices might be hacked before being introduced into a local network, and depending on the definition of hacking, some devices might want to report usage statistics or worse private information you would not agree with, such as location, activity, presence,... Note that internet access is equivalent to geo-localisation.

    When it comes devices the user would like to trust, such as the shelly smart devices, it is always possible to update their firmware by manually attaching a physical cable between the "isolated domain router" and the "main router" for the upgrade duration and then removing it. Keeping the IoT domain isolated is a better protection from hacks coming from the internet. An open ssh port with a weak or default password on any small iot device should not have consequences involving misbehavior of high power equipment.

    TP-Link TL-WR841N N300 WLAN Router

    This router is an IoT hero

    • Low cost ~ 15 €
    • Low Power ~ 1.5 to 1.7 W (yeah, really)

    Default DHCP settings can be used and address reservation can be configured in case you'd like to open each shelly device always on the same address. But as shelly devices run on MQTT using their device ID, fixed ip is not required.

    Shelly devices configuration

    simple and usual as described in the manual. A shelly device creates an access point with open security at first. Once connected to its wifi Access Point, opening the url 192.168.33.1 allows to provide it the "untrusted wifi router" credentials.

    Mosquitto server

    The raspberry pi MQTT broquer needs through to be in a fixed address, here "192.168.0.110" (no dns used), to be configured in the shelly devices

    Below is an example shelly device configuration

    Up to here, everything is fine and your shelly starts logging on its MQTT topic but one issue persists and we'll see in the following section how to solve it. The power has a correct value but the energy (kWh or Watt.min) stay at 0.

    Explanation

    The power is an instant measure, so no time required, but the energy is the accumulation of power over time. Someone could argue that the internal shelly oscillator should be accurate enough for energy accumulation, but admittedly, the energy error would be proportional to the time drift. Long story short, shelly decided to rely on time server for accurate calculation. The Network Time Protocol being too complex and an overkill in this situation, accurate decision was made to use the Simple variant "Simple Network Time Protocol". the protocol has identical queries for the subset it supports, so it can talk to a normal ntp server.

    Raspberry pi as time server

    sudo apt install ntp
    sudo nano /etc/ntp.conf
    sudo service ntp status | start | stop | restart

    installing and activating a time server is very simple with the commands above. Yet one additional configuration is important when providing the time in a local network

     You do not need google for that, just reading the config file can help

    # If you want to provide time to your local subnet, change the next line.
    # (Again, the address is an example only.)
    broadcast 192.168.0.255

    un commenting the line and adding the correct local network address is enough.

    Until here, it still not working as yet another important step is still missing

    The time server does not know your time zone

    Seriously ? a shelly connects to a time server and still does have no clue what time it is. Practically yes, as depending on which side of the earth you are, it could be 1am...

    Read more »

  • Understanding Zigbee2mqtt and viewing the Zigbee graph

    Wassim10/20/2019 at 13:41 0 comments

    zigbee2mqtt is a main module in Home Smart Mesh, yet it is not that easy to fulfill a reliable netwrok where a lot of situations do not match the intuitive expectations. As examples

    • manually removing devices before pairing again
    • manually activating joining if a router had to be out of power

    So understanding what's under the hoods allows to work around such issues and maybe even help improve it.

    I added the same diagrams in my raspi github as svg and original .graphml : https://github.com/HomeSmartMesh/raspi/tree/master/zigbee

    The svg format of the following diagrams can be found in the link above, unfortunately hackaday does not support/allow svg, which is important for zooming such diagrams:

    Startup

    Message Events

    Python to dot

    what can be found in the repo is also a python tool  that retrieves the graph info and store it into a file. The python is relatively unpractical in comparison to the superseding webtool of the next section

    Zigbee network web viewing

    This small tool is not anyhow thought to rival https://github.com/yllibed/Zigbee2MqttAssistant. But to be honest, I still did not manage to use the Zigbee2MqttAssistant due to the constraints it requires (prebuilt binaries, c sharp, lack of configuration of the prebuilt).

    A mosquitto command line interface is enough for most interactivity with zigbee2mqtt, adding some aliases alos helps, so this small tool fills the gap of the last missing needed GUI for my usage, and it also brings some advantages :

    • interactive SVG integration, can pan and zoom to see graphs that are huge with tiny text
    • can change the graph algo layout on the fly without asking zigbee2mqtt again which produces a network scan for nothing. Different algos are more adapted to different networks
      • neato : good organisation but non compact
      • fdp : best average result
      • circo : best for complicated graphs but explodes in large space
    • can add custom buttons for those who have multiple zigbee2mqtt instances
      • can add a specific algo per instance
      • can add a renaming so win space as graph viewing is all about space

  • Sensors to services Deployment

    Wassim09/29/2019 at 12:53 0 comments

    Servers

    • mini-server : We begin with deployment on hardware. The mini server is a x86 ~ 20W silent PC, living room friendly, hosting the main website for a rugged access from the outside, but all smart services (grafana,...) are local network serivces.
    • mano : no logic for naming is followed, just friendly host name for management. Raspberry pi have a samba share for easy logs check, the rest is ssh control. Mano has most services, standard ones (Mosquitto broquer, influxDB, Grafana, zigbee2mqtt, miflora2mqtt) and others custom developed within this project (Hue python ruler, nrf2mqtt custom python adapter, mqtt2influx simple python service and heat ruler that controls thermostat reactions)
    • hassio / lifo : more experimental raspberry pi servers. Hassio has the hassio service that I find nice as a backup, it is good as interfacing everything, but the rules are more complicated than vanilla python, the viewing is nothing to compare with grafana and the control e.g. lights is quite rudimentary, it is after all hard to do everything and good.
    • Hue gateway : used in local network, no account, no internet access enabled. The Hue API is quite clean and simple and python make it usage even easier. The Hue android app is also amazing and one of the main reasons why I kept the light with the Hue gateway although I could controll it through mqtt. Note that given the zigbee standard, all sorts of zigbee light bulbs can be attached to the gateway and not only those of the manufacturer, which is quite fair.
    • remote access : all services are remotely used through a vpn access, which relaxes all need for security that should not be mishandled by diy development rather collected in a (relatively) reliable interface.

    Wireless networks

    One of the most annoying feature of wireless network is the shared resources (the air - or the void). Users can rarely see on which channel is the wifi network, and it is left for automated selection by the router. Especially in case of intensive experimental environment, it is important to allocate frequency space for every network to avoid interference's. Pairing a zigbee device can be much less reliable in case of high interference, also battery usage as the zigbee is an acknowledged protocol (over IEEE 802.15.4) and the simple no-ack protocols would see more data drops.

    The wifi has a wider bandwidth, so it's good to leave more than one zigbee channel free around it. The diagram is only illustrative, the bandwidth depends also on the router configuration.

    Bluetooth does not appear here although the miflora devices are bluetooth. It is narrower than zigbee (shorter range smaller bandwidth) and always hopping so it freaking disturb a little everywhere. A further analysis could check detailed impact, more info on the matter can be found in references such as : https://help.turningtechnologies.com/PDF/Hardware/RFInteroperabilityDoc.pdf

    Devices deployment

    Aaround a hundred devices to be managed would make every day life more confusing than helping if not seamlessly integrated in the environment. Changing batteries is also required, luckily devices hold from 6 months (the custom made ones) to around 3 years at least for the others. What is important on the other hand is a low battery alert to avoid having to keep checking batteries level very often.

    Philips hue lighting are quire reliable, inter-operable with Innr, and the pure white ones (not colored) are quite cheap. Zigbee include devices association so that the dimmer can talk to the light directly without the gateway, but that is a useless feature. In about two years of usage, I did had a failure once when for some reason I had to re-pair a dimmer. Using independent buttons (aqara) to switch hue light through the hue api have a reliability of Ethernet network, in my case no wifi used for the raspberry pi controlling the light and luckily the hue gateway also has Ethernet plug. So other than de-bouncing bugs from early maturity, the system...

    Read more »

  • nRF52840-Dongle hacking

    Wassim01/13/2019 at 15:43 0 comments

    The nrf dongle has many stones on its hacking path.

    • UICR.REGOUT has to be set to 3v3

    Default to do so might brick it, and rescue needs high voltage jumpers switch explanation from Nordic here

    • UICR.NFCPINS

    Using the simplest pio, with my luck on falling over pio 0.10, of course it does not work, you need to set the UICR of NFCPINS PROTECT to disable.

    • LEDS don't blink

    although defined with the right functions LED1_G,... into pca10059.h, when using 

    bsp_board_led_on() indexes from 0 to nb leds have to be passed and not the led defines !!!

    • Ground mass

    without a bigger ground mass, the cpu keeps getting hard faults, so when debugging with swd I had to connect a usb charger without powering it on, just for bigger ground mass.

  • Node-red : from Aqara motion button to esp32 RGB Leds

    Wassim11/18/2018 at 19:45 0 comments

    Below, an esp32, like the one visble on the test board, is hidden in the white 3d printed tower. 7 WS2812B have been united for the purpose. An 18650 battery power module like the one visible on the right has been used inside the tower.

    Everything becomes easy when iot unite around MQTT to talk the same language. The Aqara motion button has been paired with zigbee2mqtt and picked up directly from there by node-red. I developed an esp32 mqtt application that takes json payloads to turn the grb leds with on and off. The esp32 is developed using the esp-idf, and the source code is available on github : https://github.com/HomeSmartMesh/esp32_iot/tree/master/rgb_led

    For those not familiar with zigbee2mqtt, the aqara button click produce such publish :

    zigbee2mqtt/motion button {"click":"single","battery":"100.00","voltage":3055,"linkquality":92}

     and to turn all leds red by command line (broquer on localhost) :

    mosquitto_pub -t 'esp/rgb led/all' -m '{"red":60,"green":0,"blue":0}'

    And the rest of the rules can either be handled by node-red as the example above, or by python for a much more flexible rules. Note that direct rules connections as simpler with node-red, but as soon as some logic is needed, then even for beginners python is more readable than node-red. github for python rules example : https://github.com/nRFMesh/nRF52_Mesh/blob/master/raspi/esp32/esp32.py

    It might be me, but it took me quite some effort to get through using node-red for a little bit advanced not too advanced examples, so I show some screenshot for very quick start with node-red.

    The input is an mqtt input with the broquer address configuration where mosquitto is running :

    I used json node with this config :

    Then a switch node that selects which event has happened in the button :

    The Flash green example :

    Then the mqtt output to the esp topic :

    Conclusion

    For connecting events to actions, node-red is really a wonderful tool that bring up visibility to your iot network.

    When playing with timing already, for generating 300 ms or 500 ms pulses, that's where node-red frontier stops. The ethernet and mosquitto jitter is much higher than such time scales, so ideally for next time, I'll implement a flash function in the esp32 directly and special topic for it so that node-red only need to send one message for flashing and not the start and the end of it.

    I wish I could rid of that "to json" box, and maybe use functions in the future, less boxes is always good.

  • nRF24L01 Flood Mesh Protocol

    Wassim05/14/2017 at 12:20 0 comments

    Mesh use cases

    before talking too much about mesh concepts, let's see end results what useful things can we get of this ?

    Flood versus Routing

    This is a Flood mesh type opposed to routing mesh types.

    • With a Flood mesh, Zero configuration is required, just throw nodes in, every node has its id.
    • A Peer to peer message just need to know the destination id, the stack handles an end to end acknowledge with retries if necessary.
    • This type of mesh is router less, which means it get advantages such as
      • the nodes do not need routing tables to define the shortest path
      • which is also hard to configure if the nodes move or would have to be identified on startup.
      • A routing mesh would also require a central server or duplication of management in every node acting as a server.
    • But there's no magic in here, a Flood Mesh have some compromises to do which might be completely irrelevant in some scenarios.
      • The mesh Range is as wide as you have nodes and capacity to live through jumps, yes but then why don't we use a huge time to live for an endless range ?
      • Well the problem is the echo, as the nodes have no routing info, they will keep repeating messages to each other, even back to those they got the messages from !!!
      • This Echo is born from the flood concept itself, it reaches all the nodes around who keep singing along for one single message until that message dies.
      • During a time to live group repetition, all the impacted network is busy with that single message. which highly increases collision chances and messages loss.
      • In the scenario of a home, this compromise is not of much relevance as you barely need a single repetition jump, one Time To Live can get you through a villa, and two through a huge building, this is not fit for a city network as its name says this is a #Home Smart Mesh.
    • As two nodes listen to an RF message at exactly the same time, they are likely to react with a repetition at exactly the same time as well. To avoid that, every node uses a per id delay configuration to repeat the message with a shift in time.

    Leaf versus Repeater Nodes

    Leaf Nodes are acting like beacons. They are wireless low power sensors that cannot keep listening, thus the frequent Broadcast for them is strategically a better compromise than a Message to a server. The unpredictable time listening waiting for the acknowledge can allow to re transmit much more frequently with the same power budget. Additionally, these nodes do not need to be configured to know the server Id.

    The Mesh Protocol

    • This Mesh protocol was custom designed to take advantages of the hardware it was designed with. The reason it is different than others is simply that the RF transceiver nRF24L01+ is much cheaper. With up to x10 times cheaper, well you get some side effects such as packet size limitation. With 32 bytes, if your protocol header is big, well then you have no more room for a payload.
    • The protocol above uses only 8 Bytes at max while fulfilling all conditions of end to end addressing, redundancy checks, acknowledgment and repetitions.
    • By design, messages do not have time stamps nor sequence ids, which let the multiple repetitions reception be filtered at the destination side. That means that yes, filtering time has to be slightly higher than the Flood echo time, and the sensors broadcast frequency must be slightly lower than the listeners filtering time.
    • The only complexity we get here is in the Pid that has been compacted to handle both the message type (e.g. Temperature, rgb color request,...) and the message status, if it is a broadcast or in case of peer to peer Request or response.
    • The source and destination being part of the user software protocol solves the problem of all radio transceivers that have hardware pipes with addresses. Those addresses have to be configured depending on the network and the number of pipes is limited for the nRF24L01 to 5. Well, it is nicer to have more than 5 nodes in your mesh !!!
    • The HW acknowledge of the shock-burst...
    Read more »

  • Smart LED Panel - MQTT to x216 WS2812B

    Wassim04/29/2017 at 13:32 0 comments

    Demo

    • This demo is just a picture where the RGB Panel is switched on next to its STM8S_RF_Dongle which is here an RF to WS2812B converter.

    How is this working ?

    • The STM8 firmware to control the WS2812B has already been presented in this Log and the LED series up to 60 here
    • Here we push the number of LEDs up to x216, it could go up to x220 but then it would not be aligned any more. The panel offer up to x256 LEDs but the STM8S memory of 1KB reaches its limit here. For a single color I could simply rework the function to lightup an unlimited number of WS2812B, but we would then loose the option of displaying figures or messages.
    • Every WS2812B consumes up to 60 mA, so for 256 LEDs that's no kidding 15 Amps !!!! Thus the transformer next to it can go up to 20 Amps !! note that this might be dangerous if you use less powerful transformers !! From safety perspective, I'm currently using a manual switch in addition, I would not recommend such an installation without a relay that fully cuts out the current when not in use!!!!
    • The radio protocol is very simple and provided in the libraries through the WS2812B.c it looks like this :
    PID - NodeID - R - G - B - CRC

    MQTT Control

    For convenience here, I used a ready Android application MQTT Dash which worked out of the box once I used the topic payload format for colors which is the #RRGGBB (e.g #FFFFFF for white). It is enough to program the right topic address (e.g. Nodes/10/RGB), this app is cool as it even provides colors pickers.

    By the way, we can see here a TVSet Power as well which is an off the shelf Wemo switch connected through OpenHAB2 and MQTT.

    Conclusion

    If you need just brighter light for your room, do not use this Panel which is more valuable than that, I plan to shit it to a central messaging notifiers and use it to display text and symbols.

  • Completely Hackable IoT dashboard

    Wassim04/26/2017 at 19:09 0 comments

    Demo

    • We can see here how the dashboard looks like, definitely not impressive but the good news is that it is completely hackable !!!

    So we can see there bullets going up with the temperature changing color accordingly, note that the color matches the gradient position of the rectangle it's in. The curves show the last 24 h, still planning to add dynamic real time animations, the technology allows but not the time. This page shows an ugly text debug window at the bottom helping me to check all received notifications.

    What's behind this ?

    • first of all, this is web, only web, html5 is helping, javascript is improving and the tools coming with it as well to make it a descent programming language.
    • I struggled looking for ready javascript libraries to show graphs such as highcharts or amcharts they look nice, but not only you get stuck with licence stuff but you cannot edit it to very customized display, the right choice ended on d3js

    D3js

    • d3js is a wrapper library around the svg standard which just makes selections easier and have a lot of helper functions and examples
    • The learning curve is long, but I highly recommend it to those who want to understand the web technology and not simply use it.

    Websockets

    • Another major improvement in the web, after struggling for server side events with comets or other alien like mechanisms, the websocket restart from scratch and provide an efficient tool already known and proved, the hack behind it is to start as an http request, and then upgrade to a websocket

    Poco

    • I know javascript is fancy, I know python is cool, sorry to say this but c++ is easier, here I'm not talking about personal preference. Not about writing code only which is obviously faster if you omit declarations, but I'm talking about the effort you spend from when you start writing your code till you get it running as you expect.
    • That is why I opted for a C++ application for netwroking where Poco marvelously does the trick. I hesitated between boost io and poco, then opted for poco as it is more than a library as it provides even application class, log and other helpers.

    Security

    • Some of you might wonder how would I secure this, with all polemics we read about every mentioning IoT devices being hacked, and I do yes access this remotely.
    • VPN : yes, the answer is simple, let us not re-invent the wheel, plus I have no time to spending updating every new patch of my authentication web interface. I admit here the solution was expensive, a VPN enabled router, but it's a single time payment for a multipurpose router and the result is independent from any 3rd party (other than simple domain name redirection).
    • It is important to note here how users can and must own their IoT devices by using tools that do not require third parties services that only work with registrations or even worse requiring active internet connections, here latency and confidentiality have to be taken into account.

    Some Javascript

    I learned how to split my javascript in more than one file (not trivial, yes, I'm serious), so I isolated the usage of the d3js graphic elements from a simple interface:

    var StatusParams = 
    {
    	svgID : "#StatusDisp",
    	data  :
    	[
    		{ x: 100, y: 100, r: 20, color: "Blue", 
    			Temperature : 26, Light : 500,
    			Name : "LivingRoom"
    		},
    		{ x: 300, y: 100, r: 20, color: "Blue", 
    			Temperature : 26, Light : 500,
    			Name : "BathRoom"
    		},
    		{ x: 500, y: 100, r: 20, color: "Blue", 
    			Temperature : 26, Light : 500,
    			Name : "Balcony"
    		}
    	]
    };
    
    var nmap = {6 : 0,
    			7 : 1,
    			16 : 2};
    
    //---------------------------------------------------------------------------
    //					Time Chart
    //---------------------------------------------------------------------------
    var now = Date.now();
    var start_disp = now - 24*60*60*1000;//1 day
    var ChartsParamsList = 
    [
    	{
    		svgID : "#ChartLivTemp",
    		scale_x_domain : [start_disp,now],
    		scale_y_domain : [10,25],
    		NodeId : 6,
    		SensorName : "Temperature",
    		Label : "Livingroom Temperature °C"
    	},
    	{
    		svgID : "#ChartLivHum"...
    Read more »

  • Open the bathroom window, Close the bathroom window

    Wassim04/26/2017 at 17:52 0 comments

    Demo

    • On the down right side of the video is the RGB Led status indicator that turns blue when the bathroom is humid (Humidity = Water = Blue) and green again when it is time to close it not to use much eating energy for example.
    • In this video, I closed the door to raise the humidity level, then opened the door and the window to evacuate it.

    Installation

    • STM8L wireless sensor node with its BME280 as a humidity sensor
    • An RF Repeater node that is visible on the down right side of the video which has an WS2812B as an RG led status indicator
    • A raspberry pi with an RF dongle as depicted in the Gallery HW + SW design

    Software : MQTT Ruler

    • The relevant software to fulfill this action in addition to the already installed HW+SW, is simply one small MQTT client, I wrote it in C++ but it could be in python as well, this client is nothing more than a ruler, it subscribes to an MQTT message (humidity level of the bathroom sensor id), and then produces a color depending on the level and send it again through another MQTT channel where the RF dongle is subscribed to send an RF signal to the right node id with the color to set.
    void mqtt_c::on_message(const struct mosquitto_message *message)
    {
        std::string msg(static_cast<const char*>(message->payload) );
        std::string topic(message->topic);
        if(topic.find("Nodes/") == 0)
        {
    		std::string Text = topic;
    		//the topic is "Node/6/Humidity"
    		utl::TakeParseTo(Text,'/');//remove first section 
    		std::string Id = utl::TakeParseTo(Text,'/');//take the second element
    		int NodeId = std::stoi(Id);
    		float humidity_val = std::stof(msg);
    		publish_humidity_status(NodeId,humidity_val);
    • This code extracts the node id from the Topic, converts the message payload from text to float and sends them together to the "publish_humidity_status()" function which is testing the node id, applying the rule and publishing the message.
    • The function applying the rule which converts the humidity float into a color can be seen below
    void from_50_Green_to_Blue(float humidity,unsigned char &red,
    									unsigned char &green,
    									unsigned char &blue)
    {
    	red = 0, green = 5, blue = 0;
    	if(humidity > 60)//60 -> 100
    	{
    		red = 0, green = 50, blue = 0;
    		float factor = (humidity - 60)/40;//0->1
    		float blue_f = 255 * factor;// 0 -> 255
    		blue = f2c_sat(blue_f);
    		float green_f = 50 - 50 * factor;// 50 -> 0
    		green = f2c_sat(green_f);
    	}
    	else if (humidity > 50)//50 -> 60
    	{
    		float factor = (humidity - 50)/10;//0 ->1
    		green = f2c_sat(5 + factor * 45);//5 -> 50
    	}
    }

    The complete MQTT humidity to color ruler can be found with the rest of the IoT_Frameworks repo here in github.


  • 60 RGB LEDs shading with that tiny STM8

    Wassim04/02/2017 at 10:03 0 comments

    Demo

    Yes, the STM8 less than $1 board can do it

    • shading colors of 60 LEDs by addressing every of them with the full 24 bits resolution.

    Firmware

    • The WS2812B bitbanging was detailed on the Hello Mesh Log
    • The 60 x 3 BYTEs info for RGB of every LED are first updated in the STM8 memory, then sent out.

    Shading colors

    What might look sophisticated for an LED application is a basic operation for a graphics application background

    //inputs: the leds section to use defined by which led to start with and which one to end before
    //inputs: The color of the first led to use and the color of the last led to use
    //action: updates the leds memory with the first and last led colors and all intermediate leds are interpolated
    //comment: This function does not send the led data bits on the bus, so that multiple operations can 
    //         be applied before sending the whole result together with SendLedsArray();
    void rgb_Shade(BYTE LedStart, BYTE LedEnd, RGBColor_t ColorStart, RGBColor_t ColorEnd)
    {
      int nbLeds = LedEnd - LedStart;
      BYTE LedId = LedStart;
      for(int iCount=0;iCount<nbLeds;iCount++)//0-10
      {
        RGBColor_t Ci = rgb_ColorScale(iCount,nbLeds,ColorStart,ColorEnd);
        rgb_SetColors(LedId,Ci);
        LedId++;
      }
    }
    • The rgb_Shade() function interpolates a range of LED colors

    //input : two colors, and ratio integers
    //output : the interpolaetd color
    //Sets the interpolated color between Colors Start and End, to the ratio of iCount/nbCount
    RGBColor_t rgb_ColorScale(int iCount,int nbCount,RGBColor_t ColorStart,RGBColor_t ColorEnd)
    {
        RGBColor_t resColor;
        
        int Red = (ColorEnd.R - ColorStart.R);//255
        Red = Red * iCount;//0 - 2550
        Red = ColorStart.R + (Red / (nbCount - 1));//0 - 255
        resColor.R = Red;
    
        int Green = (ColorEnd.G - ColorStart.G);//255
        Green = Green * iCount;//0 - 2550
        Green = ColorStart.G + (Green / (nbCount - 1));//0 - 255
        resColor.G = Green;
    
        int Blue = (ColorEnd.B - ColorStart.B);//255
        Blue = Blue * iCount;//0 - 2550
        Blue = ColorStart.B + (Blue / (nbCount - 1));//0 - 255
        resColor.B = Blue;
    
        return resColor;
    }
    • The rgb_ColorScale() interpolates one color to assign to every intermediate LED, which is interpolated between the start and the end color

    Animation

    • Although we address 60 Leds here, every with a different color, the information needed is parametric so very small and packed in few bytes to be sent in a small rf packet.
    • That could display smart information such as the Traffic conditions on the every day drive through path, or a heat map,...
    • The animation in the video has another loop on top, that interpolates the end color from Blue to RED, which is then fed as ColorEnd to the rgb_shade() function

    Source code

    • Just to mention that the complete project ready to flash on an STM8 with the IAR studio is part of the IoT_Frameworks github, namely in the rgb_led_bar folder

View all 11 project logs

  • 1
    BOM and components sourcing

    I'm trying a new experience with online BOM that could facilitate the production and purchase, otherwise the BOM is also available in the zip file, though to facilitate the DIY production, these boards are based on modules.

    • STM8S_RF_Dongle_v2 findchip BOM
      • Here you need 4x res of 1K in 1206 format
      • The STM8S103F3P6 is the small blue pill module (very easy to find online, but not available on raw electronics parts suppliers)
      • The nRF24L01+, same is the ready module.
    • STM8L_RF_Sensors_v2 findchip BOM
      • The BME280 based module is referenced as GY-BMEP
      • The MAX44009 module is referenced as GY-49 (both modules are also very easy to find online and not from raw components suppliers)
      • MRMS211H, STM8L151F3P6 and resistances are raw components.
  • 2
    STM8 Firmware
    • The firmware is located in every corresponding directory close to the Rpi SW in the main IoT_Frameworks repo.
    • This repo contains a main libs folder in the root that contains the drivers that are shared by all applications.
    • It is possible to use free compilers, but I opted for a fully integrated solution including debug capabilities with the IAR for STM8. It is free for non commercial and requires registration but fully worth it.
    • You need an ST-Link_v2 which is available online for very cheap.
    • I use the ST Visual Develop to configure the option bits, verification and EEPROM programming (that where the nodeds are)
    • You need to install any driver for your USB to serial adapter (e.g. CP2102)
    • I use putty as a terminal.
    • I use Visual Studio Code for firmware as well
      • Note that you can add the include of the IAR compiler to have automatic completion
  • 3
    Raspberry Pi SW

    I know that the Raspberry pi SW installation is not trivial, there's plenty of google entries for every topic, which I required for all steps I've done. I am also not an advanced Linux user and cannot improvise an install or compilation from source that is not described in another wiki somewhere, so here is simply the install list I performed :

    • boost (described further in the main github repo)
    • POCO : Network components for c++ HTTP servers, websockets and others
    • SAMBA : for a convenient access to the RPi and work on windows environment
    • MQTT-Mosquitto
    • OpenHAB2

View all 5 instructions

Enjoy this project?

Share

Discussions

indiantrainask wrote 09/08/2022 at 08:26 point

Thank you for sharing such a valuable piece of information which is of great importance for me as well as everyone. I was looking for this information, so thanks for giving us proper information regarding home automation.

  Are you sure? yes | no

Wassim wrote 09/10/2022 at 11:20 point

Thank you, I'm glad it helped.

  Are you sure? yes | no

Waquar Husain wrote 06/09/2020 at 17:24 point

Can anyone tell me how to make an automation project which retain its previous state (on/off) after power failure.

I stucked in this project in this step, rest everything is working perfectly fine.

  Are you sure? yes | no

nodemcu12ecanada wrote 08/21/2020 at 17:17 point

  Are you sure? yes | no

Wassim wrote 08/24/2020 at 18:43 point

I just seen your question, but sure I can help, as mentionned by nodemcu12ecanada some devices offer this function already, to be more specific, I can name the shelly plugs that can be configured to recover their state before power off. Even the lights, Philips hue has a setting from the app to configure power state recovery.

The non volatile memory is a basic principle in every embedded project, if you do it yourself, then you can use the EEPROM in a microcontroller, or with the ESP32 and arduino you can even use a file system that is non volatile.

You can maybe provide which system are you using in particular if you'd like more hints.

  Are you sure? yes | no

jacquesn4444 wrote 12/12/2019 at 10:08 point

Hello, as I can see, it's quite an old project as things change quickly :-) Would you do the same with new capabilities?

Thanks anyway, it's very interesting for me as I'm beginning. I'm wondering why don't you use a open software just like jeedom/domoticz or whatever?

I can see you're using hue as your engine for your domotic system, is there any reason for that? I'm very interested in your advices

Regards

PS: I've discovered miflora btw!

  Are you sure? yes | no

Wassim wrote 12/20/2019 at 21:09 point

Hi @jacquesn4444 , you are right, things evolve quickly and I have to follow. I started this when sensors were x10 time more expensive than Today and proprietary. When it comes to light, I admit I use hue as the main gateway for few reasons :

 (+) it is reliable 1000 %

 (+) the app is quite smooth

 (+) the API is simple, user friendly with a a lot of details

 (+) in runs completely local without any cloud account (which is optional)

 (-) on the down side, the non availability of server side events is a big minus, I love ikea for that, they opted for CoAP, ikea is on my TODO list.

MQTT on the other hand particularly zigbee2mqtt do have quite a pain when it comes to repairing devices each time, it is let's say 99% reliable, but frustrating the other 1% of times.

I do appreciate all the available frameworks and is a big user of them (openhab, hassio as my favourites) I did keep my python independenc for two reasons :

 * I like python and I find it much, much easier than any other reinvented configuration language. I like staying as native as possible, when it comes to issues, it is much easier to debug, so adding another layer of framework is for me just not necessary.

* When it comes to GUI and dashboards, I do not like the toy like gui's and unpractical dashboards. Grafana helped me fulfill most of my dashboard use cases.

So such home assistant frameworks other than the big ones (Google, Apple homekit), are just not good enough for a luxurious feel and are not that easy to configure.

Those might be not very solid arguments, my most solid one is that it's more fun when you do things yourself, that how someone can learn best :)

I'm glad you discovered Flora, I use it now for few months, but not very thrilling info, plants are quite boring :)

And by the way checkout my latest project, on how all this stuff merges into a 3d app : #Smart Home 3D Webapp 

I'm glad, if those projects inspire you, let me know also if you have more feedback.

  Are you sure? yes | no

Jacques W wrote 12/20/2019 at 21:48 point

Hello, 

thanks a lot for your answer, still interesting and complete. Will keep in mind any of your advices

Regards

  Are you sure? yes | no

stefan.schnitzer wrote 10/16/2019 at 05:12 point

That's one of the best smart home solutions I have seen so far. 

How long do the batteries on those Aqara devices last?

Do you use the CC2531 USB stick ?

I would really like to add zigbee devices to my smarthome:https://hackaday.io/project/14862-diy-scada-smart-home

  Are you sure? yes | no

Wassim wrote 10/16/2019 at 15:57 point

Thank you @stefan.schnitzer , I think this project is more liked by people who prefer simple and open scripts instead of ready but cumbursome solutions, but that does require more programming I think.

The batteries life in the Aqara devices is crazy, I just checked my orders, I got the Aquara TempHumPress (which report more often than buttons) exactly a year ago, all batteries of 3 of them is still higher than 3 V ! So I would say 2 ~ 3 years at least.

I started using CC2531 USB sticks, then I discovered the GB2530-H = CC2530 + CC2591, which is an additional PA-LNA, and that makes a huge difference in range and sensitivity, so I ordered a bunch of them. The CC2531 is a bad user experience in comparision, I would highly recommend investing on the GB2530.
I see that you do have a very elaborate project with an advanced HMI, I'll gladly help with any of your zigbee questions. After ignoring Zigbee for some time and developing my own RF mesh, now I'm back on studying zigbee from the specification, then the ZNP stack from TI and its serial protocol, then the zigbee-shepherd which is the host stack used by zigbee2mqtt, my goal is to take full advantage of zigbee and not get stuck with zigbee2mqtt limitations.

  Are you sure? yes | no

stefan.schnitzer wrote 10/18/2019 at 10:04 point

Thank you for the reply. This is a battery lifespan I could live with :)

In my case the best way to integrate Zigbee would be some Zigbee to Modbus-TCP/IP translator device. Maybe it is possible to build something like this with an Arduino and the GB2530-H.

  Are you sure? yes | no

Wassim wrote 10/18/2019 at 15:49 point

@stefan.schnitzer. With Arduino, I would not know how to add the zigbee app as the dongle is only a Zigbee-Network-Processor and needs a host in my case it is the zigbee2mqtt node sw. Or why not a raspberry pi, I just saw there is something like https://github.com/riptideio/pymodbus that could make a perfect zigbee to modbus translator with python, together with the mqtt python client.

  Are you sure? yes | no

Pedro Perez wrote 07/06/2018 at 20:03 point

This post is very interesting, If you are interested in receiving help on home atomatication, "http://www.logicbus.com/ is the best option, in your store you can find some products that mentioned. They helped me to automate my house. I recommend them

  Are you sure? yes | no

Wassim wrote 01/05/2019 at 10:30 point

Thank you for the link, but could you actually explain which product would allow creation of a mesh network ? and which protocol would it be using. It is a pity to see products limited in range and that do not have a open protocol that can interact with other products.

  Are you sure? yes | no

avtop2k wrote 07/05/2018 at 12:48 point

Thx Wassim , IoT_SIM8 rgb_leds is receiver only ?, how to make color change to rgb_leds_rf node , is this not latest ? I am not able to understand by the comment  "But actually, the STM8 was the very first version of the mesh HW"

  Are you sure? yes | no

Wassim wrote 01/05/2019 at 10:27 point

Hi @avtop2k , sorry I did not notice the comments notification here. The STM8 was developed few years ago, after which I started using STM32, then replaced both sensors and repeaters/dongles with nRF52 single chip components.

if you're interested in Neopixels, I suggest you have a look at my project #Neopixels Animations on ESP32 MQTT Json Webapp , the power of the ESP32 available libraries allowed me to easily make complex animations of RGB panels, so unless you need low power, and are stuck with STM8 for some other reasons, you can check that out.

  Are you sure? yes | no

avtop2k wrote 07/05/2018 at 12:47 point

Thx Wassim , IoT_SIM8 rgb_leds is receiver only ?, how to make color change to rgb_leds_rf node , is this not latest ? I am not able to understand by the comment  "But actually, the STM8 was the very first version of the mesh HW"

  Are you sure? yes | no

Ling wrote 02/13/2018 at 03:30 point

Hi Wassim,

I was wondering how does the 24L01 6 channel limitation applies here.  Does that mean I can't have one node talks to more than 5 nodes in the same level? If I have, let's say 50 leaf-nodes (STM8), how do I get all the sensor data back to RaspberryPI with minimal number of repeaters?

Thanks   

  Are you sure? yes | no

Wassim wrote 02/25/2018 at 15:09 point

Hi @Ling , in the nRF24L01, there is an RF channels frequency, which is here set to the same for all nodes, and there are TX and RX addresses which are only numbers sent in a protocol header to filter messages. I do not use these addresses in my user sw protocol, and I set them the same for the whole network nodes. These addresses could help avoid conflicts with your neighbor for example using the same protocol, not secure but avoids conflicts. As you can see in the description, I have a user SW protocol that defines a node id. This information is received by all listeners in the payload, so every listener can know which user node id have sent the message. There is also driver level, and application level, where you can filter to get a callback only for broadcasts or received messages. I have implemented the same user protocol in all of STM8, STM32 and nRF51, and the nRF51 has even SW control for some bytes that were in the nRF24 controlled by HW only.

So your example of 50 leaf nodes would all be listened to with one repeater or uart gateway.

The current limitation is 256 Nodes per mesh, as I have used 1 Byte for the Node Id. Currently I declared up to 44 Nodes only, but already thinking of having Id-less nodes, that would use an exception id where they would extend the address space to be unlimited, and depending on the exception, you could feed the complete serial number as identifier.

Another idea could be to make those 256 addresses dynamic, same as in the Ethernet, MAC addresses are long and unique, while ip addresses v4 are shorter and temporary retrieved through DHCP.

  Are you sure? yes | no

Ling wrote 02/26/2018 at 22:15 point

Hi Wassim,

Thank you so much for your detailed explanation. Initially I thought there are lot of limitations because of your light-weighted protocol (number of nodes, security)...   But  after I read though your project log again and digested what you posted in the comments. I realized the simplicity is the beauty.

The number of nodes and the range are not issue when the repeaters are used. The user defined encryption can be added in the payload data to prevent your neighbors to peek into your sensors and control your light. The use of Raspberry Pi connects the whole mesh to the internet.

You really did a great job!   

Thanks again!

  Are you sure? yes | no

liveej wrote 08/19/2017 at 03:28 point

What are the shortcomings of this Protocol ?How many nodes can be accessed at most?

  Are you sure? yes | no

Wassim wrote 10/21/2017 at 08:10 point

Hi @liveej, this protocol is designed for a home, not for wider buildings as in such case, the main shortcoming would be the scalability. For example, it cannot scale from a house with let's say 50 Nodes, to a complete building with 500. It is restricted by the depth of repeaters that exponentially increase the echo. You still can set up let us say 100 Nodes in every room with only one repeater for that room. I'm currently testing with a time to live of one, a single repeater, but a depth of up to two is reasonable.

  Are you sure? yes | no

electrobob wrote 04/27/2017 at 20:43 point

Have you considered RFM69 instead of the NRF24 ? It should have no problem covering a house without any repeaters. 

I have started my own home automation with the NRF (actually another clone, RFM75) but dumped it. I always thought mesh is very sensitive (seen it nice in research under the assumption that the whole place is littered with always on repeater nodes).  

  Are you sure? yes | no

Wassim wrote 04/27/2017 at 21:29 point

Well actually yes, but it's a complex decision. (+) Technically, a lower freq has better walls penetration for sure. (-) modules are kind of more expensive, (-) sub 1 ghz regulations are kind of more complex. (-) more power comes with more consumptions up to 130 mA, the nrf24l01p has also the pa lna version with 100mA up to 1000m, so that's not for the low power battery nodes. (+) the RFM69 has even encription.

I 'liked' your Stockie project. Have you considered using the stm32? stm32+RFM69 might be something.

Actually custom mesh network is very easy, even trivial if you stick with a depth (time to live) of one or two. An always on repeater within reach in a house is a reasonable assumption. Low power batteries repeaters is on my list but that's indeed a whole different story.

  Are you sure? yes | no

electrobob wrote 04/28/2017 at 07:01 point

The RFM is indeed slightly less efficient (comparing both at 0dB output power). Honestly, I find too few places where AAA are too big to use. 

In 'Stockie' i found the NRF to not be a problem - these modules can actually cover a pretty big area (with the central node being a PA/LNA). BUT: with tens of WiFis around, it sometimes takes a lot of attempts to send a packet - when they find a gap. This is fine for the application, not fine for others. 

I have at some point earlier decided to switch to ARM and did consider the STM32, among others. Coming from AVR/Xmega atmel familiarity did play a role, but in the end 2 things mattered most: 1. Atmel studio is way more convenient for a hobbyist to us (ST has no interest to provide a free compiler when they are collaborating with the expensive companies like IAR). 2. The low power STM32L are quite expensive. 

So I went with the SAMD21. Since the arduino zero now uses it, i am hoping that soon we'll get arduino clones for cheap (I don't care about the arduino part, just the cheap hardware). 

  Are you sure? yes | no

electrobob wrote 04/28/2017 at 07:02 point

My home automation project starts here http://www.electrobob.com/1000-1000/ 

  Are you sure? yes | no

Wassim wrote 04/28/2017 at 18:53 point

The interference is really a low power killer, I prefer to resend the measures more frequently rather than to waist energy on listening, ack waiting and re transmitting.

The micro controller choice is something I re consider from time to time. I started my hobby career with dozens of PICs from Microchip, technically now owning Atmel as well. It's weird to see how ARM is gaining field, so either you use an Atmel or an ST, it is the arm-gcc compiler or the armcc after all. When it comes to IDEs, I lost faith in them as they take part of the fun away, and I believe that if everyone does a good job of providing standard interfaces (compiler, gdb,...), you could keep using your favorite IDE independent of which CPU you chose, PlatformIO is going in that direction.

Thanks for the link, I'll have a scan of it, it is always good to have a Hackaday entry point to help people find such projects.

  Are you sure? yes | no

electrobob wrote 04/29/2017 at 20:10 point

There are different trade-offs between the 2 modules. I think that the lowest power can be achieved with the NRF, provided the communication distance is small. In my tests, modules close to gateway could have 90% chance of sending a packet, while those far off in my apartment were dropping to even less than 10%. With the nrf all of them stay at more than 90% success rate. 
I was saying in my blog post that the ideal module would be a NRF with higher power (>10dB) and RSSI, so one can turn the transmission power down when not necessary. 

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates