Neil K. SheridanELEPHANT AI SYSTEM
Here's a talk I did about the project
INTRODUCTION AND GOALS
This is an evolution of my 'Automated Elephant-detection system' that was a semi-finalist in the Hackaday Prize 2016. The current project differs substantially in that it makes use of more advanced machine vision techniques, and eliminates the usage of RF communication and village base stations. Alternatively using 4G/3G/EDGE/GPRS on each elephant-detection device, and includes elephant-deterrence devices to completely eliminate interaction between humans and elephants whenever possible.
* Thanks to www.nerdycute.com for drawing our logo!
So, let's get to the primary goals of Elephant AI:
- Eliminate contact between humans and elephants
- Protect elephants from injury and death
- Protect humans from injury and death
How will the Elephant AI accomplish these goals?
- Detect elephants as they move along their regular paths. These paths have been used by elephants for many years (perhaps centuries) and often cut through areas now used by humans. Humans will be warned that elephants are moving on the paths so they can stay away or move with caution.
- Detect elephants as they leave forested areas to raid human crop fields. At this point, elephant deterrence devices will attempt to automatically scare elephants. This will be using sounds of animals they dislike (e.g. bees and tigers, and human voices in the case of Maasai people in Kenya/Tanzania), and perhaps by firing chili balls into the paths of the elephants from compressed air guns.
- Detect elephants before they stray onto railway lines. This can be done via a combination of machine vision techniques and more low-tech IR (or laser) break-beam sensors. Train drivers can be alerted to slow-down and stop before hitting the elephants who are crossing.
Just how bad is it for humans and elephants to interact? This video, shot several months ago, in India, gives some idea. It is really bad indeed. It causes great stress to elephants, and puts both the elephants and humans at risk of injury or death.
That's why Elephant AI wants to take human-elephant interaction out of the equation entirely!
HARDWARE SETUP
We need a daylight camera (IR-filtered) and a night camera (NoIR filtered + IR illumination array) since elephants need to be detected 24hrs per day! In my original project I completely forgot about this, then decided to multiplex cameras to one Raspberry Pi. It was actually cheaper and easier to use two raspberry pi's; each with its own camera. Night-time and daytime classification of elephant images both need their own trained object detector anyway, so I don't think it's such a bad solution (for now).
METHODS FOR IMAGE CLASSIFICATION (older approaches)
This is the main part of the project. In my original automated elephant detection project I'd envisaged just comparing histograms!! Or failing that I'd try feature-matching with FLANN. Both of these proved to be completely rubbish in regard of detecting elephants! I tried Haar cascades too, but these had lots of false positives and literally took several weeks to train!
Initially with ElephantAI I worked with an object detector using Histogram of Oriented Gradients (HOG) and Linear Support Vector Machines (SVM). That had promising results; giving only 26% false-positives with a dataset consisting of 350 positive elephant images and 2000 negative non-elephant images (see https://hackaday.io/project/20448-elephant-ai/log/57399-4-result-for-object-detector-using-histogram-of-oriented-gradients-hog-and-linear-support-vector-machines-svm) and I would expect improved results with larger datasets. And it did. I got a result of 16% false-negatives with 330 positive elephant images and 3500 negative non-elephant images (see result #5)
At present, I am working on differentiating between types of elephants using deep convolutional neural networks for image classification vs. classical machine-vision techniques I had previously employed. This is important because different types, or classes, of elephants will exhibit different behaviours! Some aggressive, some defensive, some more placid!
I made a distinction initially between the following five types/classes:
- Elephant herds with no calves
- Elephant herds with calves
- Lone calves
- Lone adult females
- Lone male females
N.B. We can extend to a sixth class: i.e. humans carrying weapons, in order to add poacher-detection
So, I've cut these classes down now, since I could not obtain enough images to train a CN for sexes.
These are the classes we are using now:
- Elephant herds
- Lone elephants
- Calves
- Unknown elephants
An unknown elephant is just when you get an image of a trunk or leg very close to the camera! This happens very often!
Example of unknown_elephant class:
After the elephant detector software gives us a 'probability' for detection of elephant class, the detection device will:
- upload the image to a web server (via 4G/3G/GPRS modem - if connectivity exists)
- notify cell phones on list of detection (via SMS) and elephant type/class (which of 5 types)
- notify deter devices via bluetooth, activate these to deter elephants by bee sounds!
Step one is actually a great thing for education as these images could be shared with schools!
Image classification update logs:
#4 result for object detector using Histogram of Oriented Gradients (HOG) and Linear Support Vector Machines (SVM) which gave a false-positive rate of 26% and false-negative rate of 0% (n=50)
Retraining TensorFlow Inception v3 using TensorFlow-Slim (Part 2) (Retrained CNN with 5 classes of elephant) [remove]
#5 result for object detector using Histogram of Oriented Gradients (HOG) and Linear Support Vector Machines (SVM) which gave a false-positive rate of 16% (n=50) and false-negative rate of 0% (n=10)
So switching from classical machine-vision techniques to using CNs (see: https://en.wikipedia.org/wiki/Convolutional_neural_network)
1. Transfer learning on InceptionV3 model using TensorFlow. Added two new classes. I.e. herd_elephants and lone_elephants https://hackaday.io/project/20448-elephant-ai/log/68436-software-elephant-detector-transfer-learning-off-shelf-model-with-tensorflow
TESTING
Field testing of ElephantAI: with horses https://hackaday.io/project/20448-elephant-ai/log/68949-field-testing-of-elephantai-with-horses
Field testing of ElephantAI: with elephants https://hackaday.io/project/20448-elephant-ai/log/68950-field-testing-of-elephantai-with-elephants
Open Source
Any software written for the ElephantAI project is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License version 3 See http://www.gnu.org/licenses/
This is the license for OpenCV, since OpenCV libraries will be used in the project http://opencv.org/license.html
This is the license for Python, since Python will be used in the project https://www.python.org/download/releases/3.4.0/license/
TensorFlow is open-sourced as a standalone library and associated tools, tutorials, and examples with the Apache 2.0 license
Any hardware plans for the ElephantAI project, including 3d design files and stl files, etc. are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. See http://creativecommons.org/licenses/by-sa/4.0/
My GitHub code is under https://github.com/nksheridan/elephantAI/blob/master/LICENSE