• Integrating results into home automation system

    David Woodhouse12/13/2015 at 19:52 0 comments

    I've put together some scripting for Domoticz which converts depth and temperature readings into virtual sensors for tank full percentage and volume — compensating for fuel expansion and reporting the volume at a canonical temperature so you don't get lower readings overnight.

    What I need to do next is automatically feed the data from rtl_433 into Domoticz. Some people have scripts for that, but I think Domoticz should spawn rtl_433 in its mode which outputs JSON, then just read data from its stdout.

    I also need to fix the volume sensor. Currently I'm abusing a water meter sensor type, since that was the only one with the correct units (albeit misspelled). But the graphs don't do the right thing there because they plot usage as the volume increases, and we want the opposite. And we also want to plot the absolute volume over time and project when it'll hit zero — not something we have to do with a water meter.

  • Even more temperature calibration

    David Woodhouse12/10/2015 at 20:58 0 comments

    This is probably the final short-term experiment to aid with temperature calibration. The transmitter was placed in the oven again until it reported a temperature reading of '4'. It was then placed in the same location as the previous freezer experiment, at about the same ambient temperature. Again the measurements were recorded as the unit reached room temperature, and the depth reading corresponding to each temperature reading was adjusted according to the speed of sound at the temperature we believe the unit to be reporting.

    These data strongly support the hypothesis that the temperature at a given reading t is intended to be interpreted as and that the observed discrepancy between that and the measured temperatures at the high end of the range is due to experimental error or miscalibration of the temperature sensor in the unit under test. The realisation that this even makes a little bit of sense in legacy units (3°F per value) contributes to the confidence in this conclusion.

    At this point we might perhaps do one more experiment, leaving it in the oven at a constant temperature for a long period of time and verifying a single set point — and comparing with other transmitter units. But since we have little use for the temperature reading anyway except to perhaps account for the expansion of fuel at high temperatures, perhaps this is good enough for now.

  • More temperature calibration

    David Woodhouse12/10/2015 at 11:59 0 comments

    More calibration at higher temperatures shows a distinct non-linearity in the results. Arguably, if the temperature of the oil tank is 60°C then we probably have bigger things to worry about than precisely how much oil is in it (unless it's on fire?). But it would be good to be reasonably calibrated at least up to 30°C.

    The new data were obtained by heating the transmitter (and the 1wire temperature sensor) in an electric fan oven to 65°C and then letting them cool. The oven door was kept closed so that cooling rate was relatively slow — so the fact that the unit itself has higher thermal inertia than the temperature sensor should not have too much of an effect. The unit was also disassembled, to have its circuit board exposed to the air in the oven. So although part of the resulting curve might have been due to this effect, hopefully it's not too pronounced.

    The spreadsheet at http://david.woodhou.se/watchman.ods has been updated with the new data. The result of the speed-of-sound compensation is also included in the graph.

    At this point, the best fit line for the temperature reading is at odds with the best fit for the speed-of-sound adjustments that the unit was observed to be making. The best fit for the latter, keeping the adjusted depth reading as consistent as possible, remains close to the calculation inferred yesterday — with a reading of 25 indicating -10°C and a reading of 17 indicating 20°C, as shown by the green line above.
    A best fit for the newer temperature readings, however, makes the speed-of-sound adjustments make a lot less sense:

    It's probably worth repeating the experiment to monitor how it compensates for the temperature that it perceives, as it cools down from a higher temperature.

    Alternatively, we could just put an independent temperature sensor next to its installed location, and compare temperature values over a longer period of time.

  • Temperature calibration

    David Woodhouse12/09/2015 at 11:04 5 comments

    The temperature reading is non-obvious, and appears to be inversely proportional to the temperature. Also, when the temperature is on the borderline between two values and is fluctuating, the depth reading appears to fluctuate with it — there is some adjustment being done according to the measured temperature. Two experiments were performed to help understand this.

    Firstly, the unit was placed in the freezer at -20°C until the temperature reading became stable (at 35). It was then removed and placed in a stable location at about 18°C and roughly 250cm from a wall. The readings were monitored as it warmed up to ambient temperature.

    Secondly, the unit was placed in a warm room and then the heating was turned off and the window opened. Readings were monitored overnight as the temperature dropped to 5.7°C and then rose again. The maximum and minimum ambient temperature at which each value from the sensor was seen, was charted.

    Using the results of the second experiment, a chart was plotted to illustrate an estimate of the meanings of the temperature values. A good fit is achieved by assuming that 35 indicates -10°C, which is the lowest rated temperature of the system, and 17 indicates 20°C.

    These assumptions are further validated by the results of the depth calibration in the first experiment. For each temperature reading, an assumed actual temperature is now available, and the speed of sound at that temperature can be calculated.

    Remember, in the first experiment the ambient temperature was actually about 18°C, with the speed of sound being ~342m/s. But the unit had just been taken out of the freezer and was still warming up, so its reading of the temperature was inaccurate. To start with, it was assuming that the temperature was -8.3°C and thus the speed of sound was only 326m/s, and thus it was under-reading the distance by 5%. If we adjust for the temperature misreading, a distance of 251.46cm is obtained.

    Similar calculations are seen in the table above, for each temperature value. It can be seen that the assumptions (35==-10°C, 17==20°C) give a reasonable fit for these data too, with a minimal variance between the minimum and maximum "adjusted" depth readings as the unit warmed up.


    Given the above, it seems reasonable to conclude that the transmitter is giving a pre-adjusted depth value which already takes into account the temperature and the resulting speed of sound, and that the temperature indicated by a given temperature reading t is calculated by:

  • Interpreting the data

    David Woodhouse12/09/2015 at 00:31 0 comments

    Having solved the RF part of the puzzle, we now have 8-byte data packets. As noted, byte 0 seems to be a constant 0x28 and byte 7 is a 1Wire CRC.

    Bytes 1-3 are a unit ID, which changes each time you rebind the device.
    Byte 4 is flags — bit 2 (0x04) seems to be the leak/theft alarm, which surprisingly is sent from the transmitter rather than inferred by the receiver. Bit 0 (0x01) is set when a magnet is held to the transmitter. We're still working out the others — it would be useful to be able to send packets and see how the receiver responds.
    The top 6 bits of byte 5 indicate a temperature. We haven't fully calibrated this, but a value of 35 (decimal) seems to be -10°C while 17 is around 20°C.
    The actual distance reading is 10 bits, from the low 2 bits of byte 5 along with byte 6. The value is roughly given in centimetres. The transmitter unit compensates for the fact that the speed of sound varies with temperature, to give a reading which should be relatively stable as temperature varies. A depth value of zero indicates a failed reading. The reading is quite unreliable under about 10cm.

    When a magnet is held to the transmitter, bit 0 of the flags byte is set. The depth reading also switches to a countdown — which counts from 0x51 to 0x5a before the unit ID changes to a new value. The new IDs seem to be time-based, increasing every time. If the receiver receives these countdown messages with a high RSSI, within two minutes of poiwering up, it will bind to the new unit ID. The PIC in the receiver has a few bytes of EEPROM, so presumably the currently-bound unit ID is stored in there.

    Support for decoding this has also been added to rtl_433, giving output as follows:

    2015-12-08 20:43:29 Oil Watchman c7724c 80 17 0 137
    2015-12-08 20:51:24 Oil Watchman 4ecc8c 80 23 0 57
    2015-12-08 20:59:58 Oil Watchman c7724c 80 17 0 136
    2015-12-08 21:23:38 Oil Watchman c7724c 80 17 0 137
    2015-12-08 21:26:21 Oil Watchman 4ecc8c 80 23 0 57
    2015-12-08 21:41:17 Oil Watchman c7724c 80 17 0 0
    2015-12-08 21:57:17 Oil Watchman c7724c 80 17 0 136
    2015-12-08 22:01:17 Oil Watchman 4ecc8c 80 23 0 57
    2015-12-08 22:19:52 Oil Watchman c7724c 80 17 0 137
    2015-12-08 22:22:24 Oil Watchman c7724c 81 17 80 0
    The fields are unit ID (hex), flags byte (hex), temperature (decimal, raw uncalibrated value), binding countdown, depth (raw value). The latter two are of course taken from the same byte of the packet, depending on whether bit 0 of the flags byte is set.
    The receiver unit has DIP switches which allow the size of the tank to be configured. It is possible that it also compensates for expansion of the contents.

  • Decoding the FSK transmissions

    David Woodhouse12/09/2015 at 00:05 0 comments

    Using RTL-SDR, we've managed to listen to the transmissions of the device and decode them in software. This is the FM-demodulated waveform of one of its transmissions:

    This is a Manchester encoding where high->low indicates a 0 and low->high indicates a 1, with a frame preamble of three high half-bit periods followed by three low half-bit periods, and a frame postamble which just maintains the level of the last half-bit (low for a 0, high for a 1) for a further two half-bit periods to make three. The "half-bit" period is about 1ms. The data in this particular packet are as follows:

    Or 0x28 0xc6 0xd6 0x1d 0x81 0x40 0x51 0x83. The first byte is always 0x28 and might be part of the framing or packet identification. The final byte is a 1Wire CRC. We'll get to the rest later.

    At this point it should be possible to configure the RFM01 or similar modules to receive the signal. Support for decoding this framing has also been added to rtl_433 which is probably easier to experiment with.

  • Update 2

    jlbrian706/25/2014 at 01:19 0 comments

    If I did my math right the SI4320 can be configured in billions of ways.  So I made a big mistake here.  When I removed the PIC processor from the board I damaged it, and having the arduino cycle through all possible configurations is going to take an absurd amount of time.  What I think I need to do from here is buy two more of the receivers.  One to use like it is supposed to so when I begin to doubt the transmitter I can verify, and the other to listen to the PIC processor (I will try soldering straight to the legs and see if I get valid data, but if this connection corrupts the communication I will cut the traces to the SI4320 and not remove the PIC) and hook it up to the arduino through SPI and capture what comes out.  

    I think that this will work because all of the setting commands have a defined header and are of the same length.  To get this far I wrote a python script that generated the hex for all possible bit combinations for each setting.  I am hoping that I can read the data from the PIC processor to the arduino, pass it to the python script, look up the value in the appropriate dictionary and finally make a working program. 

    As with all of the best laid plans of mice and men I have to keep my fingers crossed that I didn't make other blunders along the way and don't get receivers with different chip combinations.   In the mean time I will let this board combo plug away and see if I get lucky.

  • Update

    jlbrian706/23/2014 at 19:08 0 comments

    So far I have communication with the SI4320 through SPI, still haven't managed to listen in on the transmitter.  

    Pin Out is as follows:


    1 (SDI) ---------------------------------------------- D11

    2 (SCK) -------------------------------------------- D13

    3 (nSEL) ------------------------------------------- D10

    4 (SDO) ------------------------------------------- D12

    6 (floating input when using FIFO)-------- D8 (used to tie it down)

    11 (VSS) ------------------------------------------- Ground

    14 (VDD) ----------------------------------------- +5vdc (2.2-5.4VDC)