Close

Plan for finished system

A project log for Machine-vision based wildlife-detection project

Camera-trap for larger animals, and hand-held (Pi, or run on smart-phone) for bugs and smaller animals (e.g. frogs)

neil-k-sheridanNeil K. Sheridan 10/30/2017 at 21:410 Comments

So, I've decided to turn this into a kind of digital naturalist system, with a rewards system for competing against other users to see who can spot the largest number of critters (and could add plants too).

SPOTTING AND CLASSIFYING COMPONENTS OF THE SYSTEM

1. Camera trap for larger animals

So we already have this, but not with a waterproof enclosure, and we need to alter the solar recharging circuit too. This is just the raspberry pi, or two, that will take images when PIR triggered, classify them, take video subsequently, and upload these along with labels returned from TensorFlow. We'll need two raspberry pi for night and day detection, unless we solve camera multiplexing. We will also need the IR illumination for night detection. So we'd want to build our own 5v one I think.

2. Hand-held unit for getting close to bugs, and smaller animals like frogs

So this will be a Pi with an LCD screen. So users can hold it close to the smaller critters before photographing them. We will need a GUI application to make everything easy. That'll start with a camera interface, then run the code for classification of the critters in the images acquired. Then it can return this to the user, and ask if they want to upload the spotted critters. This will need 3G/4G connectivity. We can start with the USB 3G dongles.

UPLOADING THE IMAGES AND REWARDS SYSTEM

So this is the most difficult bit for me.

What we need:

Digital naturalist system website.

The users of the system will sign up for this, register their spotting/classifying components, and here they will compete against each other to spot the most critters. I guess we could do a XP thing as per gaming, so they'd get not much XP for a cow, but quite a lot for a more illusive creature such as a pine marten. We could alter the XP given based on geo-location. So if user is in location with plentiful lions, they would not be getting so much XP for spotting those. I'd go ahead and add a feed (like the facebook old feed). So if other users like your spotted critter, you'd get more XP. We'd have a determined number of XP points to level-up too! The current easiest comparison is Overwatch, I guess. 

Digital naturalist system smart-phone apps.

We'd need the same thing, linked to the website, as smart-phone apps for iOS and Android too. Actually, we could do the hand-held unit just from a smart-phone.

Supervised re-training of CNs.

So this is most exciting part I guess, and has the cross-over to help researchers. We attract all these users with the competitive element, and then we use their feedback on classified images (i.e. was that actually a starling, or was it a blackbird?) to help retrain our CN. I touched on my idea for doing this with cats here https://hackaday.io/project/20448-elephant-ai/log/69834-building-a-daytime-cat-detector-part-2 . Eventually we will have enough human-labelled images of common wildlife to build a CN from scratch.

THINGS TO DO

1. So should I start by writing a GUI app for Pi in C# as needed by the hand-held unit? Then I can use C# to write the iOS and Android apps too? That seems ok.

2. I've no idea about how to go about doing the website!

Discussions