Close
0%
0%

DSC Neo Alarm Integration for Home Assistant

An ESP32 based virtual keypad integration for DSC Neo Alarm Systems

Similar projects worth following
The DSC Neo Alarm system uses an encrypted data bus that prevents third party home automation integrations from communicating directly with the panel.

I built an ESP32 based virtual keypad integration for the DSC Neo Alarm system which works by sniffing the keypad's lcd data bus and taking control of the the keypad's 5x5 button matrix.

Where it all started:

In late 2023, Chamberlain started blocking the MyQ garage door integration from Home Assistant. This move confirmed my concerns about relying on integrations that communicate through cloud based APIs. I begrudgingly overcame MyQ's blockade by soldering smart relays to the pushbuttons of a wired garage door opener and using contact sensors to receive the position status. However, MyQ left a sour taste in my mouth, and since I was also running the alarmdotcom HACS integration for my DSC Neo panel - an integration that warns it "communicates with Alarm.com over an unofficial channel that can be broken or shut down at any time" - I was determined to find a way to integrate my encrypted DSC Neo panel locally so that it would not become the next MyQ. 

Neo's encrypted bus blocks any direct integration from non-partners, but I learned from the MyQ blockade that if you can interact with a device physically, i.e. by reading a simple lcd screen and pushing some buttons, no matter how strong the encryption is between that device and its mothership, the information to and from your eyes and fingers must be in an unencrypted form that can then be intercepted or injected remotely. I focused on the keypads, and on building an ESP32 based virtual keypad integration that would sniff the Neo Keypad's lcd screen data bus and hijack the keypad's button matrix. I first documented my journey on home-assistant.io's forum, and now decided to document it here as I continue to refine this project.


Part 1 - Sniffing the LCD Databus:

I knew the H2SLCD Full Message Keypad contained a 16x2 lcd display. I suspected it contained the ubiquitous HD44780 LCD Controller.  On opening the keypad I discovered the lcd was on its own pcb which connects to the mainboard with an spongy flexible connector that when examined with a magnifier essentially contains a large number of parallel wires or plates sandwiched between two soft pads that connect the contacts on the mainboard  to the lcd when they are pressed together. 

I first mapped the contacts on the mainboard to the contacts on the lcd by simply looking where the two boards lined up across the connector. Then with my multimeter I started probing to find what I could. I was able to find a number of Gnd contacts, a +12V contact, a +3.3V contact, and a number of contacts reading varying + voltages between 0-3V.  I then turned to the various test points on the lcd pcb. I determined that 6 of the test points connected to an unused footprint labeled "IC3", leaving 12 contacts actively communicating with the mainboard. Of the 12 remaining test points, the first that were easily identified were the cathode/anode of the LCD backlight (labeled K/A at the top right of the board), Gnd, and +3.3V. That left 8 test points unknown. If I was dealing with an HD44780 LCD controller, it would have to be working in 4 bit mode, with the remaining 8 contacts likely being Vo (contrast control), RS (Register Select), RW (Read/Write), E (Enable), and D4, D5, D6, D7 (4 data lines).  

I ordered a 20$ USB logic analyzer off Amazon and started analyzing the signals. I used Logic 2's built in HD44780 analyzer and attempted to identify which signal might be which. I had a good idea which signals were likely RW, RS, and E based on the datasheet, but I was not sure the order of the 4 possible data lines. There could be 24 different combinations in the order of the 4 data lines, and through trial of error, I ran through all the combinations (I went through the 24 combinations a few times only to discover they were in the correct order on the pcb to begin with) and while changing various settings in the analyzer settings, eventually "\x80System Is \xC0Ready to Arm" appeared across the top of the E line! I had successfully decoded the LCD controller and was reading the keypads status with my logic analyzer. 

RW = white, RS = purple, E = brown, DB4-7...

Read more »

Gerber_Neb.zip

Carrier PCB Gerber Files

x-zip-compressed - 15.14 kB - 02/20/2024 at 07:01

Download

BOM_Neb.csv

Carrier PCB BOM File

Comma-Separated Values - 1.35 kB - 02/20/2024 at 07:01

Download

PickAndPlace_Neb.csv

Carrier PCB Pick and Place File

Comma-Separated Values - 5.29 kB - 02/20/2024 at 07:02

Download

  • CODE UPDATE - RW pin is back to keep it n'sync

    Valdez11/20/2024 at 04:46 0 comments

    The biggest changes were made in the readData() interrupt service routine. Previously, in the rare event data was randomly missed or misread, the messages or the reading of the high/low bits could get out of sync. The previous code relied upon always receiving exactly 32 characters per message to know when to end a message and begin a new one. This generally always works but after a long time it could randomly get out of sync. It is suspected that other internal messages causing interrupts to happen too close to the message we are trying to capture would cause a misread. It was discovered that most often the low bits of the last space character (0x20) in the message would then end up in the highbits of the first character of the next message, desyncing all messages thereafter. When the code was previously checking and ignoring any 0x0 reads in the high bits (as all ASCII characters expected in a message are within the range of 0x20 to 0x7E)  the issue was always avoided as the 0x0 from the end of the previous message pushed into the highbits would get ignored, re-syncing the message. Although this worked well, it was not an ideal solution as it relied upon one type of desync where the low bits of the last space character (0x20) was pushed into the next message. A different last character at the end of the message that did not end in 0 could ultimately cause a desync that would not be fixable by the highbits 0 check.  Ideally we want the ESP32 to be able to correctly delimit the beginning of messages, and not rely on case specific desyncs and receiving 32 characters to know when the next message starts. 

    This was solved by having the code check the internal register instructions to the LCD controller for an internal instruction indicating a new message. The controller sends 0x00000001 to clear the screen before each message is sent. The way the decoder is set up it reads this with the least significant bit first, as 0x10000000 or 0x80. We can use this internal register instruction 0x80 as a delimiter to indicate the beginning of new messages and correctly ensure the messages and the reading of the high and low 4-bit nibbles always remain in sync. 

    The way this is checked is when the enable pin triggers an interrupt, the code first checks if the keypad is writing data to the LCD (RS_PIN == HIGH). However, if it is not writing messages to the LCD, then we know RS_PIN is low, and we can add an ELSE IF to further check if the interrupt is an internal register instruction (RW_PIN == LOW). We can then check if the instruction is 0x80, indicating a new message is coming, and reset the necessary variables and flags to have everything synced for the new message. We have to check the RW pin to differentiate internal register instructions (RW == LOW) from busy flag checks (RW == HIGH).

    Some further optimizations have also been made to the code. One is that I have gone back to reading the data pins sequentially as it has been found more reliable. It is best advised to use the new code in its entirety. 

    New code posted below and also in the Instructions section.

    //ESP32 DSC NEO KEYPAD DECODER & BUTTON PUSHER BY VALDEZ - NOV/2023
    //CODE IS DESIGNED FOR USE WITH ESP32-S3-WROOM-2-DEVKITC-1-N32R8V AS PER SCHEMATIC
    //LIBRARIES MARKED #INCLUDE <*.h> BELOW SUCH AS PUBSUBCLIENT MAY NEED TO BE INSTALLED THROUGH YOUR LIBRARY MANAGER OF YOUR IDE
    //DECODER CODE MODIFIED FROM LEONARDO MARTINS 19/01/2019 HD44780 DECODER CODE
    //OTA UPDATING ENABLED AFTER FIRST FLASH
    //THANKS TO USER KUSHKI7 FOR ADDITIONAL HELP WITH CODE OPTIMIZATIONS & DEBUGGING  
    
    #include <PubSubClient.h>
    #include <Wire.h>
    #include <WiFi.h>
    #include <ESPmDNS.h>
    #include <NetworkUdp.h>
    #include <ArduinoOTA.h>
    
    //DEFINE HD44780 DECODER INPUT PINS (RS, RW, EN, DB4-7)
    # define RS_PIN 2 // RS (Register Select) pin
    # define RW_PIN 42 // RW (Read/Write) pin 
    # define EN_PIN 1 // EN (Enable) pin
    # define DB4 4
    # define DB5 5
    # define DB6 6
    # define DB7 7
    
    //Define...
    Read more »

  • CODE UPDATE - NOV. 8 2024

    Valdez11/08/2024 at 20:28 0 comments

    1. AP Mode has been removed as it was causing issues. Enter your wifi/mqtt credentials manually in the code before compiling. 

    2. OTA updating code added so code can be flashed remotely through wifi. Will require a first flash by usb before it will work. Default password is "admin" unless changed in the code. 

    3. @kushki7 provided further code optimizations to the interrupt service routine. The ISR now reads all 4 pins simultaneously using a register increasing the read speed. Bitwise & is used like a modulo to loop through the dataBuffer and set the dataReady flag as is much faster than a for loop. It takes minimal processing to compare two bytes with & which is why it increases speed. A further explanation is in the code comments.

    4. With @kushki7 optimizations, it appears the integration is fast enough to operate the interrupt on a falling edge and work. The actual HD4470 controller latches the DB4-7 data on a falling edge of the enable pin. However, the data is set up at the time the pin goes high, and triggering the interrupt on the rising edge adds a buffer of safety in case the reading of DB4-7 is delayed. Code is left on a rising edge for this reason, but can make that change if you want. 

    5. The interrupt function is set to run off ESP32s RAM instead of Flash using IRAM_ATTR. This is necessary for a properly functioning ISR.

    6. Changes to the dataSpill() function were also made. noInterrupts() must wrap any volatile variables changed in dataspill to prevent problems with the ISR trying to change them at the same time. In our case only dataReady flag needs to be wrapped with noInterrupts(). When it was not previously, synchronization errors were happening shifting the messages high/low bits over and out of sync. Lastly the string pointer to the volatile dataBuffer was replaced with a local char array with an extra null character at the end. The dataBuffer is copied into this array and the extra null at the end allows proper handling and sending it to MQTT as a recognized string. 

    7. A MQTT reset message has been included in case you ever need to remotely reboot the ESP32. If you send "reset" to the button pushing topic "esp32/output" the ESP32 will reboot itself.  

    8. Code was cleaned up in general and some variables and instances redefined to make things more clear so best to use the new code in its entirety

  • Update - New Code, RW pin, Power - November 4, 2024

    Valdez11/04/2024 at 18:39 0 comments

    User @kushki7 discovered some bugs when building my design for his own system. After a long debugging and re-programming session with him we have fixed these issues. 

    We discovered that the timing of the reading the data from the bus is critical and has to happen as fast as possible. We also found some code bugs in the interrupt routine that were causing issues. 

    For the timing issue, we removed the need to check the RW pin as the keypad never reads the data from the LCD controller RAM and so checking when the RW is low when RS is high is not necessary as RW will always be low when RS is high. It just isn't necessary for capturing the write data and the added complexity of checking the RW line adds processing to the interrupt where we want it the fastest as possible - right before reading the data lines.   An updated PCB design may come sometime in the future to reflect this.

    The code in instructions section corrects these bugs found, and does a few other optimizations for reading the data. 

    Power management - @kushki7 also had a greater strain on power running the ESP32 off his keypads 3.3V+. It ran, but he felt it was pulling a lot of power from the keypad causing his display to slow down and dim and decided to run his ESP32 off an external 5V USB power source. I never had this issue, and it likely depends on each individuals alarm panel setup and power demands. If you want to run the ESP32 off an external power source, disconnect the V+ line from the PCB to the keypad. Gnd must stay connected between the keypad and PCB or the data signals will not make any sense to the ESP32 (the signals are voltages or potential differences between Gnd so the ESP32 and keypad must share a common Gnd).  

View all 3 project logs

  • 1
    Keypad Connection Points

    Connection points on keypad LCD pcb:

    Connection Points on button Matrix (there are also corresponding connection points on the 100 pin micro controller on opposite side of pcb but requires very fine soldering). The solder points were chosen to cause least interference with the numpad and most used function buttons to allow continued use of physical keypad. Keep solder low and flat and wires running off to the side and away from center of button contacts to maintain physical keypad use:

  • 2
    Wiring Schematic

    Optocouplers = PC817

    Resistors = 210 Ohm

    Note the opposite direction of X vs Y connections to the optocouplers due to the opposite direction in current flow.


  • 3
    ESP32S3-DevkitC1 Code (UPDATED NOV. 19, 2024)

    AP-mode has been removed due to too many problems. Edit the code below to manually enter your WIFI/MQTT credentials. 

    On board RGB LED indicator status:

    Blue = WIFI and MQTT connected
    Flashing Purple = AP Setup Mode
    Flashing White = Attempting to connect to WIFI
    Flashing Yellow = Attempting to connect to MQTT
    Green = Rebooting device

    Will Attempt to connect to WIFI and MQTT for 30 seconds each and then reboot itself. 

    Can confirm the LCD decoder is working using serial monitor. 

    //ESP32 DSC NEO KEYPAD DECODER & BUTTON PUSHER BY VALDEZ - NOV/2023
    //CODE IS DESIGNED FOR USE WITH ESP32-S3-WROOM-2-DEVKITC-1-N32R8V AS PER SCHEMATIC
    //LIBRARIES MARKED #INCLUDE <*.h> BELOW SUCH AS PUBSUBCLIENT MAY NEED TO BE INSTALLED THROUGH YOUR LIBRARY MANAGER OF YOUR IDE
    //DECODER CODE MODIFIED FROM LEONARDO MARTINS 19/01/2019 HD44780 DECODER CODE
    //OTA UPDATING ENABLED AFTER FIRST FLASH
    //THANKS TO USER KUSHKI7 FOR ADDITIONAL HELP WITH CODE OPTIMIZATIONS & DEBUGGING  
    
    #include <PubSubClient.h>
    #include <Wire.h>
    #include <WiFi.h>
    #include <ESPmDNS.h>
    #include <NetworkUdp.h>
    #include <ArduinoOTA.h>
    
    //DEFINE HD44780 DECODER INPUT PINS (RS, RW, EN, DB4-7)
    # define RS_PIN 2 // RS (Register Select) pin
    # define RW_PIN 42 // RW (Read/Write) pin 
    # define EN_PIN 1 // EN (Enable) pin
    # define DB4 4
    # define DB5 5
    # define DB6 6
    # define DB7 7
    
    //Define Pushbutton Matrix Pins
    # define X1 10
    # define X2 11
    # define X3 12
    # define X4 13
    # define X5 14
    # define Y1 18
    # define Y2 8
    # define Y3 3
    # define Y4 46
    # define Y5 9
    
    # define LED_BUILTIN 38 //define onboard LED pin
    
    //declare functions
    void IRAM_ATTR readData(void); //the ISR is to run from RAM not flash
    void connect_wifi(void);
    void connect_mqtt(void);
    void callback(char* topic, byte* message, unsigned int length);
    void buttonPush(uint8_t X, uint8_t Y, int time);
    void dataSpill(void);
    
    
    //********************************************************************
    //******* ENTER YOUR WIFI AND MQTT CREDENTIALS MANUALLY HERE**********
    
    //WIFI SSID/Password credentials
    const char* ssid = "ENTER WIFI SSID HERE";
    const char* pass = "ENTER WIFI PASSWORD HERE";
    
    //MQTT SERVER CREDENTIALS
    const char* mqtt_server = "homeassistant.local";
    const int mqtt_port = 1883;
    const char* mqtt_username = "ENTER MQTT USERNAME HERE";
    const char* mqtt_password = "ENTER MQTT PASSWORD HERE";
    
    //************************************************************************
    //************************************************************************
    
    //DECLARE GLOBAL VARIABLES FOR HD44780 DECODER
    volatile byte data = 0; // Storage for data
    volatile bool highBits = true;  // flag to indicate high or low 4 bits being read 
    volatile bool dataReady = false; // Flag to indicate when data is ready to read
    volatile bool highBitsInt = true; 
    volatile char dataBuffer[32] = {0}; //databuffer to store the 32 char message sent to LCD
    volatile byte charCount = 0; // character counter
    volatile byte internalReg = 0; //internal register instruction storage
    
    //WIRELESS SETUP
    WiFiClient wifiClient; //create wifi instance wifiClinet
    PubSubClient mqttClient(wifiClient); //create mqtt instance mqttClient
    int countx = 0; //counter for reconnection attempts
    
    //MAIN PROGRAM SETUP
    void setup() {
    
      Serial.begin(115200); // Start serial monitor communication
      
      neopixelWrite(LED_BUILTIN,0,0,0); // initialize onboard led to off
      
      //initializes keypad matrix output pins
      pinMode(X1, OUTPUT);
      pinMode(X2, OUTPUT);
      pinMode(X3, OUTPUT);
      pinMode(X4, OUTPUT);
      pinMode(X5, OUTPUT);
      pinMode(Y1, OUTPUT);
      pinMode(Y2, OUTPUT);
      pinMode(Y3, OUTPUT);
      pinMode(Y4, OUTPUT);
      pinMode(Y5, OUTPUT);
      digitalWrite(X1, LOW);
      digitalWrite(X2, LOW);
      digitalWrite(X3, LOW);
      digitalWrite(X4, LOW);
      digitalWrite(X5, LOW);
      digitalWrite(Y1, LOW);
      digitalWrite(Y2, LOW);
      digitalWrite(Y3, LOW);
      digitalWrite(Y4, LOW);
      digitalWrite(Y5, LOW);
    
      //INITIALIZE INPUT PINS
      pinMode(RS_PIN, INPUT);
      pinMode(RW_PIN, INPUT);
      pinMode(EN_PIN, INPUT);
      pinMode(DB4, INPUT);
      pinMode(DB5, INPUT);
      pinMode(DB6, INPUT);
      pinMode(DB7, INPUT);
    
      // Setup the interrupt
      //note the HD44780 reads the data on the falling edge but the data is setup at the time of a rising edge which allows extra time to be captured
      attachInterrupt(digitalPinToInterrupt(EN_PIN), readData, RISING); 
    
      connect_wifi(); //connect wifi
      connect_mqtt(); //connect mqtt
      
      ArduinoOTA.begin(); //initialze OTA firmware updating
      
      // OTA UPLOADING DEFAULT INFORMATION (DEFAULT PASSWORD IS "admin" unless changed)
      // UNCOMMENT BELOW IF WANT TO CHANGE OTA AUTHENTICATION SETTINGS FROM DEFAULT
      // ArduinoOTA.setPort(3232);   // Port defaults to 3232
      // ArduinoOTA.setHostname("myesp32");  // Hostname defaults to esp3232-[MAC]
      // ArduinoOTA.setPassword("admin");   // No authentication by default
    
    }
    
    //MAIN PROGRAM LOOP
    void loop() {
        
      ArduinoOTA.handle(); //prioritize OTA updater
    
      //reconnect wifi if disconnected  
      if (!WiFi.isConnected()) {
        connect_wifi();
      }
    
      //reconnect mqtt if disconnected 
      if (!mqttClient.connected()) {
        connect_mqtt();
      }
      
      mqttClient.loop(); //mqtt message check
      
      //send LCD message to MQTT if ready 
      if (dataReady){
      dataSpill(); 
      }
    
    }
    
    //WIFI SETUP & CONNECT FUNCTION
    void connect_wifi() {
      neopixelWrite(LED_BUILTIN,0,0,0);
      delay(1000);
      Serial.println();
      Serial.print("Connecting to ");
      Serial.println(ssid);
      WiFi.begin(ssid, pass);
    
      while (WiFi.status() != WL_CONNECTED) {
        delay(500);
        Serial.print(".");
      }
      Serial.println("");
      Serial.println("WiFi connected");
      Serial.println("IP address: ");
      Serial.println(WiFi.localIP());
      neopixelWrite(LED_BUILTIN,32,0,32);
    
    };
        
    //MQTT SETUP & CONNECT FUNCTION
    void connect_mqtt() {
        mqttClient.setServer(mqtt_server, mqtt_port); 
        mqttClient.setCallback(callback);
        mqttClient.connect("esp32-DSC-Client", mqtt_username, mqtt_password);
        
        countx++; //increases counter each time it attempts a connection
        
        //attempt 15 connection attempts (30 seconds) then restart
        if (mqttClient.connected()) {
          Serial.println("connected to MQTT");
          mqttClient.subscribe("esp32/output"); //topic subscribed to for button pushing messages
          neopixelWrite(LED_BUILTIN,0,0,32); //if MQTT connects change LED to blue
          countx = 0;
        } else {
          if (countx >= 15) {
          ESP.restart();
          }
          Serial.print("MQTT connection failed, try again in 2 seconds");
     
          // Wait 2 seconds while blinking orange LED
          delay(500);
          neopixelWrite(LED_BUILTIN,0,0,0);
          delay(500);
          neopixelWrite(LED_BUILTIN,32,16,0);
          delay(500);
          neopixelWrite(LED_BUILTIN,0,0,0);
          delay(500);
          
        }
    }
    
    
    //CALLBACK FUNCTION TO RESPOND TO MQTT MESSAGES RECEIVED - PUSHES BUTTONS DEPENDING ON PAYLOAD RECEIVED
    void callback(char* topic, byte* message, unsigned int length) {
      Serial.print("Message arrived on topic: ");
      Serial.print(topic);
      Serial.print(". Message: ");
      String messageTemp;
    
      //PRINT OUT THE RECEIVED MSG TO SERIAL MONITOR
      for (int i = 0; i < length; i++) {
        Serial.print((char)message[i]);
        messageTemp += (char)message[i];
       }
        Serial.println("");
    
      //CODE TO CALL BUTTONPUSH CONDITIONAL ON THE PAYLOAD RECEIVED   
       if (messageTemp == "1"){
          buttonPush(X1, Y1, 100);
        } else if (messageTemp == "2"){
          buttonPush(X2,Y1,100);
        } else if (messageTemp == "3"){
          buttonPush(X3,Y1,100);
        } else if (messageTemp == "4"){
          buttonPush(X1,Y2,100);
        } else if (messageTemp == "5"){
          buttonPush(X2,Y2,100);
        } else if (messageTemp == "6"){
          buttonPush(X3,Y2,100);
        } else if (messageTemp == "7"){
          buttonPush(X1,Y3,100);
        } else if (messageTemp == "8"){
          buttonPush(X2,Y3,100);
        } else if (messageTemp == "9"){
          buttonPush(X3,Y3,100);
        } else if (messageTemp == "*"){
          buttonPush(X1,Y4,100);
        } else if (messageTemp == "0"){
          buttonPush(X2,Y4,100);
        } else if (messageTemp == "#"){
          buttonPush(X3,Y4,100);
        } else if (messageTemp == "STAY"){
          buttonPush(X4,Y1,2000);
        }  else if (messageTemp == "AWAY"){
          buttonPush(X4,Y2,2000);
        } else if (messageTemp == "CHIME"){
          buttonPush(X4,Y3,2000);
        } else if (messageTemp == "F4"){ 
          buttonPush(X4,Y4,2000);
        } else if (messageTemp == "<"){
          buttonPush(X5,Y1,100);
        } else if (messageTemp == ">"){
          buttonPush(X5,Y2,100);
        } else if (messageTemp == "NIGHT"){
          buttonPush(X5,Y3, 2000);
        } else if (messageTemp == "reset"){
          ESP.restart(); // ability to restart ESP32 from MQTT with "reset" payload
        } 
       
     }
    
    
    //BUTTON PUSHING FUNCTION
    void buttonPush(uint8_t X, uint8_t Y, int time){
      neopixelWrite(LED_BUILTIN,32,0,32); // blink purple each time a button is pushed
      digitalWrite(X, HIGH);
      digitalWrite(Y, HIGH);
      delay(time); //length of button push
      digitalWrite(X, LOW);
      digitalWrite(Y, LOW);
      neopixelWrite(LED_BUILTIN,0,0,32);
      delay(100); //IF PUSHING MULTIPLE BUTTONS QUICKLY BY AUTOMATION NEED TO HAVE A DELAY IN BETWEEN
    }
    
    
    //ISR FUNCTION TO READ THE 4 bit DATA WHEN THE ENABLE PIN TRIGGERS THE INTERRUPT 
    void IRAM_ATTR readData() {
    
       // Check that data is being written (RS == 1) 
       // store the 4-bit parallel data into the LSB (0000xxxx) of variable part
        if (digitalRead(RS_PIN) == HIGH) {
            byte part = 0; 
            part |= digitalRead(DB4); 
            part |= digitalRead(DB5) << 1;
            part |= digitalRead(DB6) << 2;
            part |= digitalRead(DB7) << 3;
    
    
        if (highBits) {   // This is the high bits (MSB) (xxxx0000) part of the 8-bit data
      
            data = part << 4; //assign and shift the 4 bit part to the most sig bit positions of the 8 bit byte
    
        } else {  // This is the low bits part (LSB) (0000xxxx) 
    
          data |= part; //copy the 4 bit part to the least sig bit end of the 8 bit byte using bitwise OR
          dataBuffer[charCount++ & 31] = data; //store the full 8 bit character data byte in the buffer array and increment the counter 
          if (!(charCount & 31)) dataReady = true, highBitsInt = true;  // once counter reaches 32 set dataready flag, reset internal register highbits flag to keep in sync. 
          //above statements uses bitwise & (X & (n-1)) similar to modulo (X % n) to repeatedly loop through the array with minimal processing; This only works when n is a power of 2.  
        }
        
        // Flip the highBits flag to alternate inputting the 4 bits from the high bits to low bits
        highBits = !highBits;
    
      //if controller is not writing to the LCD (RS == LOW), check if it is writing internal register instructions for the new message instruction (0x80) to keep messages in sync
      //maintain a separate highbits flag for internal register instructions so each type of write instruction can re-sync the other  
      } else if  (digitalRead(RW_PIN) == LOW) { 
            byte partInternal = 0;
            partInternal |= digitalRead(DB4);
            partInternal |= digitalRead(DB5) << 1;
            partInternal |= digitalRead(DB6) << 2;
            partInternal |= digitalRead(DB7) << 3;
    
        if (highBitsInt) {  
      
            internalReg = partInternal << 4; //shift the high bits into the internal register instruction storage
    
        } else {  
          internalReg |= partInternal; 
            if (internalReg == 0x80) { // if new message instruction detected resync the highBits flag and charCount to keep messages in sync. 
                charCount = 0; 
                highBits = true; 
            }  
        }    
    
        highBitsInt = !highBitsInt; 
      }
    
    }
    
    //FUNCTION TO WORK WITH THE STORED DATABUFFER AFTER INTERRUPT COMPLETES
    //SEND THE MESSAGE TO MQTT WHEN THE DATA READY FLAG IS TRUE
    void dataSpill(){
    
      noInterrupts(); //no interrupts must be set while manipulting volatile data that can also be changed by the ISR
      dataReady = false; //set flag to false so message is not re-sent until next message arrives
      interrupts();
    
      char *outputx = (char *) dataBuffer; // setup pointer to the databuffer char array to act like a string
      mqttClient.publish("esp32/alarm", outputx); //send the message to mqtt
      
        for(int i = 0; i < 32; i++){
          Serial.print(outputx[i]);                        // prints the lcd message to the serial monitor
        } 
          Serial.printf("\n");
    
      /*   
        for(int i = 0; i < 32; i++){             //for debugging purposes, prints the hex codes of each char to serial monitor.
          Serial.printf("%02x ", outputx[i] );  //for debugging purposes, prints the hex codes of each char to serial monitor.
          }                                    //for debugging purposes, prints the hex codes of each char to serial monitor.
          Serial.printf("\n");                //for debugging purposes, prints the hex codes of each char to serial monitor.
      */  
    
    }

View all 5 instructions

Enjoy this project?

Share

Discussions

taofineberg wrote 11/22/2024 at 21:13 point

I know this is not directly related to this project. I’ve purchased all the components and assembled the ESP32 side of things. I then bought a DSC PowerSeries Neo Wire-Free Keypad HS2LCDWFPV4 from eBay—it appears to be new. The keypad powers on and displays: "HS2LCDWFPV V1.30 ENG L81." I can initiate the adoption process using an HS2LCDRF - LCD RFK Keypad with firmware version V1.30.01.04 and hardware revision UA627, revision 03. The alarm board I’m using is an HS2064 E v1.3FN.

The system recognizes the correct 6-digit Device ID and successfully adds the keypad as a wireless Keypad on slot #2. However, the wireless keypad remains stuck on "Waiting for Confirmation."

It’s worth noting that I’ve paired many wireless sensors without any issues. Any assistance with resolving this would be greatly appreciated!

  Are you sure? yes | no

Valdez wrote 11/28/2024 at 17:27 point

Sorry I am not familiar with the installation process of that model of keypad. If it has a 16x2 display it can probably be read using the project, but the configuration may be different, and I would use a dedicated external power supply.

  Are you sure? yes | no

kushki7 wrote 10/17/2024 at 02:05 point

this is great Valdez. I am going to try this out with my Neo. I previously hooked up a powerseries system with EVL4 that one day just died. Rather than going down that path again I am thinking of hooking up my spare NEO which I have two wall panels for. I think what you have accomplished here is really phenomenal.  A quick question? Which ESP32S3devkitC1 are you using? I have some generic ESP32 boards but want to use the PCB you designed and want to buy the module you have. I was checking and there are several different models, some are based on ESP32-Wroom-1 some are Wroom-1U or Wroom-2. Looks like you re using Wroom-2 based board. I also found different flash sizes (N8R8/N16/N32R8V). I believe those last digits are flash sizes (check link https://www.mouser.ca/c/embedded-solutions/engineering-tools/?q=esp32-s3-devkitc-1&instock=y&sort=pricing%7C1 ). can you please shed some light on this?

  Are you sure? yes | no

Valdez wrote 10/23/2024 at 15:41 point

Hi kushki7, this is the exact model dev board used with link to digikey.

ESP32-S3-DEVKITC-1-N32R8V 

https://www.digikey.com/en/products/detail/espressif-systems/ESP32-S3-DEVKITC-1-N32R8V/15970965

  Are you sure? yes | no

kushki7 wrote 10/24/2024 at 04:38 point

Thank you Valdez. I set it up today and so far initial implementation is working. I have only set up three zones so far and will do more later. I don't know if you face this issue or is just me but several times the payloads are received garbled. For example instead of 'Entry Motion <>03' I receive 'b"\x97ntry Motion <>03 '. I am thinking of adding 5 microsecond delay after reading the highbits and lowbits in the code but the damn esp and wall panel is in the basement so need to bring the laptop there and change it on step-stool. Also I am thinking of adding ArduinoOTA code as well to avoid step-stool work in the future.

  Are you sure? yes | no

Valdez wrote 10/25/2024 at 11:09 point

I haven't had that issue. A rising edge on the enable pin is what triggers the interrupt to read each set of 4 bits. It looks like it's having an issue with the first characters or reading some data not intended to be a character coming before the characters such as internal register instructions or busy flag checks. I suspect that because if it was just a high bit/low bit misread it would still be the correct number of characters.

You can actually likely remove the code that tests for all 0s in the high bits as it was part of a debugging process and I never removed it in case it broke something when it was working. In my experience you want as minimal operations happening in an interrupt as possible. I know wired motions signal the panel extremely fast compared to other sensors that have a long open and closed state like a door. 

So...

if((part << 4) != 0b00000000){ 

data = part << 4; 

 } else{ highBits = false; 

}

Becomes simply:

data = part << 4; 

That would give the processor fewer clock cycles per interrupt and may help gain some speed on reading.

But that is a lot of extra characters your getting and I really wonder if you're somehow capturing busy flag or internal register instructions being sent as characters.

When RS and RW are low internal registry instructions are sent to clear the display for a new message or carriage return etc. When RS is low and R/W is high, the keypad is reading the busy flag to ensure it is safe to write. When RS is high and R/W is low the keypad is writing characters to the lcd and that is all you want to capture. The first conditional on the interrupt function captures only data when RS is high and RW is low.

Maybe re check RS and RW connectors to make sure they are wired correctly not shorted to something else that could be messing with their signals and pulling them to the state that allows data not intended for write operations to be read as characters.

  Are you sure? yes | no

kushki7 wrote 11/02/2024 at 20:58 point

Sorry for the delay in replying, for some reason I did not get notification of your comment this time. I had the same suspicion about the amount of processing in the interrupt. I tried removing the if statement like you demoed and the the code worked for a while and than stopped updating the messages inside home assistant (lcd was live and updating on the keypad). Message on HA side was stuck and would not update until manually reboot the esp. Seems like that if statement does some sort of error correction that i am not aware of so needs more troubleshooting. Other suspicion I had was the lcd was reading way too fast after activating the interrupt so tried adding delays(microseconds) at the start of the interrupt and tried different value such as 100 ms, 50 ms, 10 ms etc. but that just made things worse in all instances. It seemed as if the erroneous reading was shifting up and down the line. I.e. instead of the first character being misread no it was doing that with 3rd or 7th character. I finally removed all delays and replaced with a 'asm volatile nop' statements to give a small breather of nanoseconds just before reading the pins. This hasnt fixed the issue but has slowed down the number of occurrences significantly. I also tried to filter out the text sent by mqtt publish service to include only certain ascii characters so strings misread strings can be eliminated but that did not help at all. The characters that were allowed are alphabetical, numeric, asterisk, #, space, dash. 

Other than that I have implemented OTA and built in wifi info ( no portal mode). Also I have filtered certain messages such as 'Scroll to view open zones' and 'Secure or arm system'. When messages start with 'Secure or' or ;Scroll to', it will discard the message and not publish to HA. None of this made the problem above any better or worse, the errors happened before these changes and after as well. What is surprising is I never received a malformed message about the excluded strings such as I never received '/t34ecure or Arm System', which technically should not filter out. May be the issue is not with the code but with the mqtt publish but i cant imagine how that can happen.

  Are you sure? yes | no

Valdez wrote 11/04/2024 at 18:45 point

After a long debugging session with @kushki7 we found the issues. There was timing issues and code bugs in the interrupt. Code has been optimized and updated in instructions to reflect these changes. @kushki7 also made some power management changes and instructions on how to do so are in the Logs.  Please read the log:

https://hackaday.io/project/194448-dsc-neo-alarm-integration-for-home-assistant/log/234222-update-new-code-rw-pin-power-november-4-2024

  Are you sure? yes | no

spekkio wrote 04/25/2024 at 05:04 point

Just wanted to mention I implemented this project with some help from Valdez and it has been great!  Used it to get my installer code too.

Pro tip, you can mute the beeps in the user menu.

  Are you sure? yes | no

Dan Maloney wrote 01/19/2024 at 01:28 point

Outstanding hack! As a former owner of a DSC system, and a current Home Assistant fan, this is pretty slick. I love the fact that the control panel provides a complete end run around DSC's encryption. Wrote this up for the blog, should publish soon. Thanks for posting this here!

  Are you sure? yes | no

Valdez wrote 01/19/2024 at 22:28 point

Thanks Dan. I look forward to reading it!

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates