Close
0%
0%

LoRa + Neural Network Security System

Spot trespassers with a neural network and transmit basic results via LoRa

Public Chat
Similar projects worth following
A Raspberry Pi and camera is used to spot people using a Modivius neural compute stick and send the imformation via a peer to peer LoRa network to an Arduino MKRWAN 1300 for sounding an alarm.
Although the programming behind this project is extremely complex, if you know your way around a Raspberry Pi, using the files included in this blog it's relatively simple to implement.

If anybody wants to support any of our projects, check out our Amazon Wishlist here: https://www.amazon.co.uk/hz/wishlist/ls/3AW6R7V5BVU3R?ref_=wl_share

Patreon donations can be made here: patreon.com/Goat_Industries


You may well ask 'What's wrong with an ordinary security system?' ….. Well, the problems I had are listed below:

  • Very long cables were required due to distance of gateway from the house. Other cameras use WIFI, but this is not strong enough to cope with the distances involved.
  • Although the system had an alarm output, there were a lot of false positives which were vey annoying. The system would sound alarms for cars and, worse, shadows of lorries driving past the gate.
  • There's no ability to sense how close people were to the camera or whereabouts they were in the frame.

My eventual solution satisfied all my criteria - the system eradicates nearly all false positives and transmits 'intelligent' data with a range of about 15 km which means that I could monitor the system whilst walking the dog on the local beach!

How does it work? ……. There is a transmitter unit, shown above, powered by a Raspberry Pi and a receiver unit with an Arduino MKRWAN 1300. The Modividius USB stick helps speed up processing on the Raspberry Pi to such a degree that it makes it 'useful' and 'accurate' and a 'neural network' is used to 'infer' the position of a person in the frame of a camera. Python scripts are used to grab the data, process it and send relatively short lines of data over the radio frequency airwaves using the long range LoRa protocol.

This is the Arduino receiver below:


ino - 4.62 kB - 12/23/2018 at 12:41

Download

Intruder alarm 02.pcb

Opens with 'Design Spark' free PCB software.

pcb - 174.50 kB - 12/23/2018 at 11:47

Download

Intruder alarm Gerber files.zip

PCB files for manufacture.

x-zip-compressed - 42.14 kB - 12/23/2018 at 11:47

Download

security_camUSBCam.py

This file is for use with a USB webcam. Paste it into /home/pi/ncappzoo/apps/securityCam

x-python - 10.57 kB - 12/14/2018 at 17:58

Download

security_camPiCam.py

This file is for use with a Raspberry Pi cam V.2. Paste it into /home/pi/ncappzoo/apps/securityCam

x-python - 10.70 kB - 12/14/2018 at 17:58

Download

View all 8 files

View all 13 components

  • Testing the Device

    Capt. Flatus O'Flaherty ☠12/27/2018 at 11:28 0 comments

    After putting the system into headless mode and logging in via a laptop using SSH, I was able to test the system 'in the field'. Initially, there was a bug that caused the camera to turn off after 15 minutes and this was cured by installing 'screen' on the Raspberry Pi and typing 'screen' into the command line before initiating the python files. What screen does is it opens another terminal on the Pi, so it has it's own live terminal which does not get shut down when my laptop terminal shuts down. It's a really good solution and avoids having to mess about with other 'run on boot' solutions which can potentially wreck the whole system.

    A separate video camera was set up in my office, 200 m away, synchronised with the main camera and focused on the receiver gadget (bottom right of video). On testing, the system did not respond to the dog in the camera frame, but did respond to me (a person) ….. Success!

  • Detecting Other Objects and Animals

    Capt. Flatus O'Flaherty ☠12/23/2018 at 14:20 0 comments

    The 'security_cam.py' file is a generic 'live-object-detection' script and can be modified very easily to detect a total of 20 different objects.

    If we look at line 119:

    # "Class of interest" - Display detections only if they match this class ID
    CLASS_PERSON         = 15

    To detect dogs, just change this to:

    # "Class of interest" - Display detections only if they match this class ID
    CLASS_DOG         = 12

    Also, line 200 needs to be changed as well:

            # Filter a specific class/category
            if( output_dict.get( 'detection_classes_' + str(i) ) == CLASS_PERSON ):

    Although the person class worked really well and was quite impressive, the dog class was a little bit underwhelming and not nearly as good as some of the other models I've tested. Nonetheless, here's the full list of classes available with this model:

    1. Aeroplane
    2. Bycycle
    3. Bird
    4. Boat
    5. Bottle
    6. Bus
    7. Car
    8. Cat
    9. Chair
    10. Cow
    11. Diningtable
    12. Dog
    13. Horse
    14. Motorbike
    15. Person
    16. Pottedplant
    17. Sheep
    18. Sofa
    19. Train
    20. TVmonitor

  • Arduino MKRWAN 1300 Code

    Capt. Flatus O'Flaherty ☠12/23/2018 at 12:50 0 comments

    Nothing too fancy about the code, except I wanted the tone of the beeper to change according to how close the detected person was to the camera. This would be useful for discerning the difference between someone posting a letter in the postbox and someone actually walking up the drive. The code uses string analysis functions to, firstly, confirm that the data is coherent by searching for the word 'Box' and then finding the two pairs of coordinates that represent the detection box. If the detected person is close to the camera, the area of the detection box is greater and the resulting alarm tone is of higher frequency:

    #include <SPI.h>
    #include <LoRa.h>
    String myString =" ";
    String myStringReversed =" ";
    
    void setup() 
    {
      pinMode(4, OUTPUT);
      pinMode(LED_BUILTIN, OUTPUT);
      tone(5,1000,1000);
      digitalWrite(LED_BUILTIN, HIGH);
      digitalWrite(4, HIGH);
      delay(1000);
      digitalWrite(LED_BUILTIN, LOW);
      digitalWrite(4, LOW);
      Serial.begin(9600);
    //  while (!Serial);
    
      Serial.println("LoRa Receiver");
    
      if (!LoRa.begin(8690E5)) {
        Serial.println("Starting LoRa failed!");
        while (1);
      }
    }
    
    void loop() 
    {
      //delay (1000);
      // try to parse packet
      int packetSize = LoRa.parsePacket();
      if (packetSize) 
      {
        // received a packet
        Serial.print("Received packet '");
      digitalWrite(LED_BUILTIN, HIGH);
      digitalWrite(4, HIGH);
      delay(100);
      digitalWrite(LED_BUILTIN, LOW);
      digitalWrite(4, LOW);
        // read packet
        myString =" ";
        myStringReversed =" ";
        int i = 0;
        char c;
        while (LoRa.available()) 
        {
          //c[i] = (char)LoRa.read();
          //Serial.print((char)LoRa.read());
          myString = (char)LoRa.read() + myString;
          i++;
          //Reverse the string:
          c = myString.charAt(0);
          myStringReversed = myStringReversed + c;
        }
        processString();
        //Serial.print("My string: ");Serial.print(myString);
        // print RSSI of packet
        //Serial.print("' with RSSI ");
        //Serial.println(LoRa.packetRssi());
    
      }
    }
    void processString()
    {
        Serial.print("My string reversed:");Serial.print(myStringReversed);
        // print RSSI of packet
        Serial.print("' with RSSI ");
        Serial.println(LoRa.packetRssi());
        int len = myStringReversed.length();
        int j=0;
        char a,b,c,d;
        String coord1 = " ";
        String coord2 = " ";
        String coord3 = " ";
        String coord4 = " ";
        int k =0;
        char x = ',';
        int z=1;
        int y=1;
        int r=1;
        int s=1;
        int v=0;
        while (j < len) 
        {
          a = myStringReversed.charAt(j);
          b = myStringReversed.charAt(j+1);
          c = myStringReversed.charAt(j+2);
          if((a=='B')&&(b=='o')&&(c=='x'))                           // The word 'box' has been identified in the string - k is now greater than 0.
          {
            k = j+5;
            Serial.print("Character B was found at: ");Serial.println(j);
          }
          j++;
        }
        if (k>0)
        {
          v =0;                                                      // int V stops perpetual loops occurring.
          while((z==1)&&(v<200))
          {
            if(myStringReversed.charAt(k)==x)                        // Build up string 'coord' until a comma is reached.
            {
              Serial.print("k");Serial.println(k);
              z=0;
            }
            else
            {
            coord1 = coord1 + myStringReversed.charAt(k);
            k++;
            v++;
            //Serial.print("coord1: ");Serial.println(coord1); 
            }
          } 
          v =0; 
          k++;
          while((y==1)&&(v<200))
          {
            if(myStringReversed.charAt(k)==')')                        // Build up string 'coord' until a comma is reached.
            {
              Serial.print("k");Serial.println(k);
              y=0;
            }
            else
            {
            coord2 = coord2 + myStringReversed.charAt(k);
            k++;
            v++;
            //Serial.print("coord2: ");Serial.println(coord2); 
            }
          }  
          v =0;
          k=k+3;                                                     // Takes account of two brackets and a comma.
          while((r==1)&&(v<200))
          {
            if(myStringReversed.charAt(k)==x)                        // Build up string 'coord' until a comma is reached.
            {
              Serial.print("k");Serial.println(k);
              r=0;
            }
            else
            {
            coord3 = coord3 + myStringReversed.charAt(k);
            k++;
            v++;
            //Serial.print("coord3: ");Serial.println(coord3); 
            }
          }  
          v =0;
          k++;
          while((s==1)&&(v<200))
          {
            if(myStringReversed.charAt(k)==')')                        // Build up string 'coord' until a comma is reached.
            {
              Serial.print("k");Serial.println(k);
              s=0;
            }
            else
            {
            coord4 = coord4 + myStringReversed.charAt(k);
            k++;
            v++;
            //Serial.print("coord4: ");Serial.println(coord4); 
            }
          }  
        }
        Serial.print("coord1: ");Serial.println(coord1);
     Serial.print(...
    Read more »

  • Detector and Transmitter Unit

    Capt. Flatus O'Flaherty ☠12/23/2018 at 11:44 0 comments


    As can be seen, this unit is very easy to assemble and just needed to be located in a waterproof case with a transparent front ( https://uk.rs-online.com/web/p/general-purpose-enclosures/0104212/ ) . The above components were wedged tightly into the box using thick cardboard and a bit of judicious Origami:
    The camera MUST be up the right way round for the neural network to work properly.

  • Reciever PCB Populated

    Capt. Flatus O'Flaherty ☠12/23/2018 at 11:37 0 comments

    Other than the Arduino MKRWAN 1300, the PCB features a L293E chip for stepping up the voltage and current required for the alarm system, which is itself a block of 8 LEDs and 3 buzzer beeper chips. Attempting to run these devices directly off the Arduino would instantly frazzle the device!

    After assembly and testing, the whole thing worked perfectly and, after a bit of experimentation, the best resister for the red LEDs was 39 Ohms.

    Although most of the components are surface mount, they are all very large and no stencil is required. After checking the polarity of the LEDs, the PCB was pasted up with solder, populated with the surface mount components and chucked in the toaster oven for cooking to 260 degrees C.

  • Example of Captured Image

    Capt. Flatus O'Flaherty ☠12/15/2018 at 13:15 0 comments

  • Save Camera Snapshots to a USB Drive

    Capt. Flatus O'Flaherty ☠12/14/2018 at 18:35 0 comments

    The security cam Python file can be easily adapted to save the repeated snapshots of detected people by modifying line 235 to something like the following, where my USB drive is called 'KINSTON':

    photo = ( "/media/pi/KINGSTON" + "/captures/photo_" + cur_time + ".jpg" )

    In reality, using this USB stick slowed down the program rather drastically! Images can be transferred / deleted easily from the Pi if another PC is used to SSH into the Pi once it's deployed.

  • Upgrading Camera to Raspberry Pi NoIR V2

    Capt. Flatus O'Flaherty ☠12/14/2018 at 17:56 0 comments

    Obviously, we're going to want to use this gadget in the dark with infra red lights, so we need a decent camera with IR capabilities ie no IR filter. Fortunately, these cameras are a lot cheaper an more compact than the Logitech USB and they're also very easy to install:

    Firstly, check that camera is enabled in the RPi settings and then, after plugging it into the board, check it works with:

    $ raspistill -o image.jpg

    Next install the following Python dependencies

    $ sudo apt-get install python3-picamera
    $ pip3 install "picamera[array]"
    $ pip3 install imutils

     Lastly, use the Pi cam version of the security_cam file, downloadable from here: https://cdn.hackaday.io/files/1626676959544928/security_camPiCam.py

    Run the file using the following command:

    $ cd && cd /home/pi/ncappzoo/apps/securityCam && python3 security_camPiCam.py

    The security camera is now ready to test out in the wild, although obviously not in the rain! It'll be interesting to see what the transmitter range will be with a decent antenna :) 

    Other than waterproofing, another issue is where to collect the captured photos produced when the device spots a person - maybe a USB stick, to prevent filling up the raspberry Pi SD card?

  • Combining Two Python Packages to get LoRa and Security Cam Working Together

    Capt. Flatus O'Flaherty ☠12/14/2018 at 13:26 0 comments

    It's been a bit of a battle, with a steep Python learning curve, but finally I created a single Python file that enables a time stamp, detection class, confidence and bounding box coordinates to be sent to the Arduino base station. Obviously, there's still a number of dependancies in various directories - some close by and others deeply embedded in the Python system somewhere, but here's my 'top of the stack' code:

    #!/usr/bin/python3
    
    # ****************************************************************************
    # Copyright(c) 2017 Intel Corporation. 
    # License: MIT See LICENSE file in root directory.
    # ****************************************************************************
    
    # Detect objects on a LIVE camera feed using and send data over LoRa network
    # Intel® Movidius™ Neural Compute Stick (NCS)
    
    import os
    import cv2
    import sys
    import numpy
    import ntpath
    import argparse
    
    import mvnc.mvncapi as mvnc
    
    from time import localtime, strftime
    from utils import visualize_output
    from utils import deserialize_output
    from pySX127x_master import basic_test01
    
    import datetime
    import array as arr
    import numpy as np
    from time import sleep
    
    
    from pySX127x_master.SX127x.LoRa import *
    from pySX127x_master.SX127x.LoRaArgumentParser import LoRaArgumentParser
    from pySX127x_master.SX127x.board_config import BOARD
    
    #from pySX127x_master import Tegwyns_LoRa_Beacon
    
    BOARD.setup()
    
    parser = LoRaArgumentParser("A simple LoRa beacon")
    parser.add_argument('--single', '-S', dest='single', default=False, action="store_true", help="Single transmission")
    parser.add_argument('--wait', '-w', dest='wait', default=1, action="store", type=float, help="Waiting time between transmissions (default is 0s)")
    
    myString = ""
    #from pySX127x_master.Tegwyns_LoRa_Beacon import *
    
    class LoRaBeacon(LoRa):
    
        def __init__(self, verbose=False):
            super(LoRaBeacon, self).__init__(verbose)
            self.set_mode(MODE.SLEEP)
            self.set_dio_mapping([1,0,0,0,0,0])
            
        def on_rx_done(self):
            print("\nRxDone")
            print(self.get_irq_flags())
            print(map(hex, self.read_payload(nocheck=True)))
            self.set_mode(MODE.SLEEP)
            self.reset_ptr_rx()
            self.set_mode(MODE.RXCONT)
    
        def on_cad_done(self):
            print("\non_CadDone")
            print(self.get_irq_flags())
    
        def on_rx_timeout(self):
            print("\non_RxTimeout")
            print(self.get_irq_flags())
    
        def on_valid_header(self):
            print("\non_ValidHeader")
            print(self.get_irq_flags())
    
        def on_payload_crc_error(self):
            print("\non_PayloadCrcError")
            print(self.get_irq_flags())
    
        def on_fhss_change_channel(self):
            print("\non_FhssChangeChannel")
            print(self.get_irq_flags())
                
        def start(self):
            global args
            sys.stdout.write("\rstart\r")
            stamp = str(datetime.datetime.now())
            text=bytearray('PING LoRa Test PI: ' + stamp + ('  ') + myString,'utf-8')
            self.write_payload([0x00, 0x00, 0x00, 0x00] + list(text))
            self.set_mode(MODE.TX)
            
    lora = LoRaBeacon(verbose=False)
    args = parser.parse_args(lora)
    
    lora.set_pa_config(pa_select=1)
    #lora.set_rx_crc(True)
    #lora.set_agc_auto_on(True)
    #lora.set_lna_gain(GAIN.NOT_USED)
    #lora.set_coding_rate(CODING_RATE.CR4_6)
    #lora.set_implicit_header_mode(False)
    #lora.set_pa_config(max_power=0x04, output_power=0x0F)
    #lora.set_pa_config(max_power=0x04, output_power=0b01000000)
    #lora.set_low_data_rate_optim(True)
    #lora.set_pa_ramp(PA_RAMP.RAMP_50_us)
    
    #print(lora)
    #assert(lora.get_lna()['lna_gain'] == GAIN.NOT_USED)
    assert(lora.get_agc_auto_on() == 1)
    
    print("Security cam config:")
    print("  Wait %f s" % args.wait)
    print("  Single tx = %s" % args.single)
    print("")
                        
    lora.start()
    
    # "Class of interest" - Display detections only if they match this class ID
    CLASS_PERSON         = 15
    
    # Detection threshold: Minimum confidance to tag as valid detection
    CONFIDANCE_THRESHOLD = 0.60 # 60% confidant
    
    # Variable to store commandline arguments
    ARGS                 = None
    
    # OpenCV object for video capture
    camera               = None
    
    # ---- Step 1: Open the enumerated device and get a handle to it -------------
    
    def open_ncs_device():
    
        # Look for enumerated NCS device(s); quit program if none found.
     devices...
    Read more »

  • Getting 'Person Detector' Working using the Neural Compute Stick

    Capt. Flatus O'Flaherty ☠12/12/2018 at 16:47 0 comments

    A couple of key things to note before getting started:

    1. The correct Neural Compute stick for Raspberry Pi is the NCSM2450.DK1 and currently (2018) no other Intel sticks will work on the Pi.
    2. Be careful which version of the stick SDK or APi is downloaded - V2 and above is NOT for Raspbian stretch, only Ubuntu 16.04.

    Instructions:

    1. I installed the full version of the latetst version 1 of the SDK and the APi and it did not take too long to install:

    $ sudo apt-get update
    $ sudo apt-get install
    $ git clone https://github.com/movidius/ncsdk.git
    $ cd /home/pi/ncsdk && sudo make install
    $ cd /home/pi/ncsdk && sudo make examples

    2.  Test that the stick is working ok:

    $ git clone https://github.com/movidius/ncappzoo
    $ cd /home/pi/ncappzoo/apps/hello_ncs_py
    $ python3 hello_ncs.py 

     3. Download this file: https://cdn.hackaday.io/files/1626676959544928/graph and paste it into the /home/pi/ncappzoo/caffe/SSD_MobileNet folder. Do not change it's name or extention.

    4. Make and run the demo using the following commands:

    $ cd /home/pi/ncappzoo/apps/security-cam
    $ make run

    ..... Obviously a camera is required. Mine is a USB logitech and it worked straight away. 

View all 13 project logs

Enjoy this project?

Share

Discussions

Falk Schuetzenmeister wrote 10/11/2019 at 05:13 point

The code files seem to be incomplete and broken. Some python modules such as utils or pySX127x_master.basic_test1 imported in security_camPiCam.py are missing. Is there a chance to correct or even push the working stack to Github? Thank you

  Are you sure? yes | no

kylix34 wrote 07/08/2019 at 08:12 point

Would a Raspberry Pi 4 be able to manage by itself the image analysis and exclude this way the need of using a Movidius device?

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates