FugSlucks™ 2019

Description

Ever since Mankind began gardening, slugs have been one of the major problems in growing our own food. Over the last 70 years of the BBC program 'Gardener's question time' (GQT) nobody has yet come up with a suitable method of despatching slugs from our crops. The best contender so far, according to Bob Flowerdew of GQT, are nematode worms, but these are unfortunately prohibitively expensive unless you start breeding them yourself. Now, with modern computer technology, can we finally crack this problem?

Details

Ai slug detection involves using a high end web cam to capture live video images of the crops or seed trays and streaming those images into a specialist computer, not much bigger than a Raspberry Pi, which then uses a pre-trained 'model' to identify slugs and draw a boundary box around each and every one of them. The boundary box is essentially a set of four coordinates which can then be exported to another system such as a high powered laser to kill the creatures as quickly and humanely as possible.

There are other possible methods of despatching the slugs such as catching them in a trap, picking them up with a robotic arm or catapulting them off into the neighbour's garden and it would be interesting to get people's responses on the ethics of slug disposal. Is killing slugs with a high powered laser unsportsmanlike ?

Much of the work involved in this project is in creating the trained model and so far this has necessitated taking 5712 photos of the animals in the dark under the light of a head torch. The images are then cropped and labelled and the 'image sets' (images and labels) then uploaded to Amazon Web Services (AWS), via Google Drive, for processing on a Nvidia V100 graphics card using software created by Nvidia in one of their 'Docker' containers. AWS currently (2019) charge $3.04 an hour for this service and this works out being a lot cheaper, and more convenient, than trying to build one's own computer with a high end graphics card.

Files

Project Logs

Collapse logs

More tests with fixed camera focus and no zoom
Tegwyn☠Twmffat • 04/11/2019 at 10:33 • 0 comments

Performance of the slug detection was improved by disabling auto focus in the Logitech C290e camera and not using zoom as the zoom is digital rather than using lenses and dramatically reduces the pixel density. The detection results seem to be a lot better as shown in the video below:
Fully House Trained Slugs
Tegwyn☠Twmffat • 04/10/2019 at 13:49 • 0 comments

The slugs are now house trained. 10605 images of slugs have created a fairly good slug detector although there could be improvements by adding blurred images to get the training to focus more on the overall shape of the animals, including their tentacles, or antennae. In deployment, there are quite a few false positives, not really shown on the video below, which was created on a leaf background:

The training shows obvious convergence, even though testing in real life does not pick up on the overall shape of the slugs, but more readily on the patterns on their bodies. Training on AWS Nvidia V100 GPU took about 13 hours to reach 500 epochs:

Training slugs has proved to be more difficult than the common European wasps, probably because the wasps have much more obvious features whereas the slug is more of an amorphous blob with antennae. At this stage, the project is very possibly finished as the limitations of the object detection algorithm has been successfully explored and the conclusion is that more obvious features need to be apparent for effective detection.
Future work would benefit from a better camera eg Logitech Brio 4K rather than Logitech C930e, which would give 4x the pixel density and HDR. Also, more images, possibly another 10,000 of them!
Slug Detection Starts to Work in Deployment
Tegwyn☠Twmffat • 04/04/2019 at 09:23 • 0 comments

Deployment success! Not brilliant accuracy, but it's definitely working. Most inference is between 50 and 60 % accurate. Also detects snails, which is weird as I took all the snail images out of the image sets. Must be a relic of previous training as I always use a trained model from the previous AWS session (pre-trained network).
Finally the Training is Converging
Tegwyn☠Twmffat • 04/03/2019 at 19:43 • 0 comments

After a couple of weeks of hunting for slugs at night time and processing the images, 5,712 of them, the training is converging! It's been hard work, adding 1,000 images at a time and testing for convergence. Thank God it eventually worked!
Looking at the graph, it seems that training has got quite a bit further to go as the top red line is still sloping downwards and the loss coverage is not diminishing as it should. I'll test the model that this session created, but I think throwing another 1,000 images at the network would be a good idea. I really needed this result to boost my enthusiasm!
It's worth thinking now about the next stage in a bit more detail - how to catch / destroy the slugs? I think a Google questionnaire form might be appropriate with questions to probe people's views on the morals / ethics of killing slugs and the possibility of more 'humane' alternative solutions?
3498 Image Sets and Still No Convergence Using DetectNet !
Tegwyn☠Twmffat • 03/22/2019 at 18:17 • 0 comments

I added another 1,000 images and text files of slugs and about 1,100 'dontcare' image sets and still no signs of convergence. I don't feel disheartened because I've already had good success with custom images of the common European wasp. Another difference between the data sets is that with the wasp images, there were often over 10 wasps in one single jpg and the wasps all looked pretty much identical, whereas the slugs are mostly one per image and there's about 10 different sprecies. Will the model converge or will I have to go back to YoloV3 to get a deployable model?
Training 1600 image sets on Nvidia detectNet shows no convergence
Tegwyn☠Twmffat • 03/20/2019 at 13:44 • 0 comments

Even after adding another 600 images and associated text files, detectNet neural network failed to show any real signs of convergence, even after a couple of hours on AWS P100 GPU:
Since I want to make use of the recently released Nvidia Nano dev kit, it that to get this network to converge, more images are needed. This is probably because, as mentioned previously, there are at least 10 species of slug and 1 species of snail being photographed, which means approx. 160 images for each species which is a very small data set for this kind of thing.
If detectNet fails to converge, I'm sure it's possible to run YOLOV3 on the Nano without too much trouble.
Failiure to Deploy YoloV3 Model on Raspberry Pi
Tegwyn☠Twmffat • 03/15/2019 at 16:55 • 0 comments

In my other project, the Ai Wasp sentry gun, I successfully managed to deploy a model on the Raspberry Pi using MobileNet SSD, although the results were admittedly pretty poor.

This time I thought I'd try YoloV3 as, theoretically, there is a complete software toolchain to take the Yolo model to the Pi. Training 1,000 annotated images of slugs on AWS seemed to be successful:

and testing the detection accuracy on AWS gave goodish results, so the problem may well be the Intel model optimiser software. Who knows? There's a copy of my notes here, if anybody can understand this stuff: https://cdn.hackaday.io/files/1641377022437408/Yolov3%20train%20AWS%20p3.2xlarge%20instance.txt

Or here, below, but hard to read due to bad formatting on these narrow pages:

source activate python3
git clone https://github.com/paddygoat/darknet cd darknet make
conda install -c anaconda opencv
yolov3.cfg (236 MB COCO Yolo v3) - requires 4 GB GPU-RAM: https://pjreddie.com/media/files/yolov3.weights
To train on Linux: $ ./darknet detector train data/obj.data yolo-obj.cfg darknet53.conv.74
wget -P /home/ubuntu/darknet/build/x64 https://pjreddie.com/media/files/darknet53.conv.74 cd darknet/build/x64 ./darknet detector train data/obj.data yolo-obj.cfg darknet53.conv.74
ubuntu@ip-172-31-25-87:~/darknet$ ./darknet detector train data/obj.data yolo-obj.cfg darknet53.conv.74 yolo-obj Couldn't open file: yolo-obj.cfg ubuntu@ip-172-31-25-87:~/darknet$
./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74
Loading weights from darknet53.conv.74...Couldn't open file: darknet53.conv.74 wget -P /home/ubuntu/darknet https://pjreddie.com/media/files/darknet53.conv.74 cd darknet ./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74
Total BFLOPS 65.290 Allocate additional workspace_size = 200.40 MB Loading weights from darknet53.conv.74... seen 64 Done! Learning Rate: 0.001, Momentum: 0.9, Decay: 0.0005 If error occurs - run training with flag: -dont_show
(average loss:1907): Gtk-WARNING **: cannot open display: (python3) ubuntu@ip-172-31-25-87:~/darknet$
to see the mAP & Loss-chart during training on remote server without GUI, use command: ./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 ./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 -dont_show -mjpeg_port 8090 -map then open URL http://54.190.27.97:8090 in Chrome/Firefox browser)
source activate python3 cd darknet ./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 -dont_show ./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 -dont_show -mjpeg_port 8090 -map
Can't open label file. (This can be normal only if you use MSCOCO): data/obj/AB00001.txt
Wrong annotation: class_id = 2. But class_id should be [from 0 to 0]
Only for small datasets sometimes better to decrease learning rate, for 4 GPUs set learning_rate = 0.00025 (i.e. learning_rate = 0.001 / GPUs). In this case also increase 4x times burn_in = and max_batches = in your cfg-file. I.e. use burn_in = 4000 instead of 1000.
8.1. For training with mAP (mean average precisions) calculation for each 4 Epochs (set valid=valid.txt or train.txt in obj.data file ???? test = test.txt ??? Note: If you changed width= or height= in your cfg-file, then new width and height must be divisible by 32.
wget -P /home/ubuntu/darknet/data https://github.com/paddygoat/darknet/blob/master/data/obj.data wget -P /home/ubuntu/darknet/data https://github.com/paddygoat/darknet/blob/master/data/test.txt wget -P /home/ubuntu/darknet/data/cfg https://github.com/paddygoat/darknet/blob/master/cfg/yolo-obj.cfg
./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 -dont_show -gpus 0,1,2,3 AWS p3.2xlarge only has one Tesla V100 GPU !!!!!!
./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74...
Read more »
Slug Detection Using Raspberry Pi and Intel Neural Network Stick
Tegwyn☠Twmffat • 03/12/2019 at 17:11 • 0 comments

Watch this space for major update in the next couple of days!
Hunting for Slugs
Tegwyn☠Twmffat • 03/03/2019 at 11:46 • 0 comments

Training neural networks for detecting slugs requires hunting down slugs in the night time and photographing them under the light of a head torch with macro lens. Fortunately, they don't move too quickly, but photography is still challenging as the macro lens is very sensitive to movement so the aperture needs to be quite large which means getting focus correct is hard. An advantage is that the background gets blurred, especially when the slugs are silhouetted somewhat. A minimum of 2,500 photos are required and each one needs to be cropped, resized to 640 x 640 pixels, renamed and then labelled. I seem to be able to do an average of about 100 a day and have done 1,000 so far.
Slugs are amazing creatures and getting this close to them is quite revealing. I think most people would think of them as being pretty gross, but on close up they are truly beautiful animals. I think I identified about 10 different species, from the large black ones to the small dappled.

View all 9 project logs

Discussions

[deleted]

[this comment has been deleted]

Tegwyn☠Twmffat wrote 05/02/2019 at 08:43

Ok .... Gimme $50 ?

Are you sure? yes | no

Ivan Stepaniuk wrote 04/11/2019 at 11:15

Is direct color imaging the best input? What about IR, UV or even multi-spectral imaging? Cross-polarized imaging might be interesting as well, I imagine that it could reveal more features on slugs as their wet, ridged skin certainly produces a distinct specular reflection pattern not present in foliage. Not that AI wouldn't be enough by itself, but evolution made this bugs difficult to see in color; the way their predators see. Just thinking aloud here, nice project!

Are you sure? yes | no

Tegwyn☠Twmffat wrote 04/11/2019 at 12:31

I've tried IR and UV, but there seems to be no benefit at all. Not sure what multi-spectral imaging is nor cross polarising is but will google it. Camera resolution seems to be main problem at the moment ie 1080p versus 4k. Thanks for the tips!

Are you sure? yes | no

Ivan Stepaniuk wrote 04/11/2019 at 15:18

By multispectral I just mean combinations of wavelength bands at the same time by means of filters, multiple sensors, etc.

Cross-polarizing is using a linearly polarized light source while blocking the camera with another linear polarizer in such a way that certain reflections are filtered out. It's very common in microscopy and medical imaging.

These picture have been taken with such technique, in the right sample, you can basically "see through" the outermost layer of the skin. http://www.wilkoff.net/cross-polarization-photography/

Which brings me to the thought; Wouldn't an photo camera, likea DSLR be better at this job?

Good luck!

Are you sure? yes | no

Tegwyn☠Twmffat wrote 04/05/2019 at 09:36

So far, on balance, best solution for dispatching the slugs may well be salt shot out of some kind of gun. The salt, particularly if it is something like ammonium nitrate, does not necessarily cause harm to the plants although may not conform with organic soil association's regs. Thanks everybody for the comments and suggestions.

Are you sure? yes | no

Ivan Stepaniuk wrote 04/04/2019 at 12:33

The detection is useful by itself! A home gardener myself, if with a camera and this software I could be notified about the slugs eating my broccoli, I would be really happy. I have worked with CO2 lasers for medical applications, and as a side note, I can tell that there is no chance that a laser powerful enough to kill a slug would be safe to have in your garden.

Are you sure? yes | no

Tegwyn☠Twmffat wrote 04/04/2019 at 12:41

Thanks for your comment, Ivan. I guess if you've got a valuable crop an alarm system could alert you to the encroaching slugs. Laser safety is a primary concern and by no means assumed to be an acceptable solution. But the question has to be asked!

Are you sure? yes | no

Ivan Stepaniuk wrote 04/04/2019 at 12:53

True! And lasers are awesome! If it was up to me, I would just remove them.

I had a similar idea (use machine vision) to detect the pieris rapae, a white butterfly that lays eggs on the underside of broccoli, cabbage, etc. leaves. Being white, I think it would be easier than the slugs. A blow of compressed air would be enough to send the butterflies away with no damage to the plants or the butterflies. The worms can eat the whole plant in a few hours after hatching.

Are you sure? yes | no

PointyOintment wrote 04/04/2019 at 19:48

Salt darts?

Are you sure? yes | no

Tegwyn☠Twmffat wrote 04/04/2019 at 12:58

I'm planning to instigate Caterpillar detection in the Summer. I think it's too difficult to target the butterflies. The stripey black and yellow ones will be easy to detect. Not so sure about the green ones though :(

Are you sure? yes | no

Martin wrote 03/20/2019 at 15:46

Unfortunately these slimy creatures have a high water content so you need quite some energy to vaporize them. So you need probably a projectile solution (very fine shot) or a decent CO2 laser tube.

Are you sure? yes | no

Tegwyn☠Twmffat wrote 03/19/2019 at 12:04

Good point. I'm certainly not convinced laser is the right way to go. Might be better if laser can only ever point directly downwards and works autonomously in the middle of night when no people around. Reflections could still be a problem - not thought of that..

Are you sure? yes | no

Dan Maloney wrote 02/27/2019 at 17:47

Get ducks. They locate slugs infallibly, and as a bonus they eat them.

True, they make a mess and a racket. So I'm keen to seen how AI can do - lost a bunch of seedlings to slugs last year. Nasty ones, too - some as much as a half foot long. More like snakes. Yech!

Are you sure? yes | no

Tegwyn☠Twmffat wrote 02/28/2019 at 15:32

Hello Dan! Thanks for reply. I'm about 95% confident it will work, based on my experience with training for wasps. I believe the key is getting the photos right - no slugs in the distance or half occluded on the photo edge, none out of focus or grainy/noisy, try to get slugs straight on horizontal or vertical to get tight boxes, use large variety of backgrounds and at least 2,500 photos.

I was wondering what Hackaday's policy is on animal cruelty - obviously dogs, cats, rats, mice is out, but where is the line drawn? Is it ok to vaporize molluscs on Hackaday?

BTW, love your blogs!

Are you sure? yes | no

Dan Maloney wrote 02/28/2019 at 15:44

Thanks! Love your work too!

I don't know that we have a specific policy other than what would be covered by common sense and common decency. I'm sure the slugophiles out there will be put off by slug snuff films, but I imagine that's a fairly small constituency. Let's just play it by ear.

How exactly will the offending slugs meet their demise? Vaporization sounds - interesting. Lasers, perhaps? We use the "drowning in a cup of beer" approach. Old school, but they seem to die a happy death. We should all be so lucky.

Are you sure? yes | no

Tegwyn☠Twmffat wrote 02/28/2019 at 16:25

I'm planning to use a 15 watt laser - the biggest I could find on ebay - I tested it at 1 metre and successfully burnt a hole in my carpet! Currently, main problem is getting high enough resolution in gimbal. The proper motors are very expensive. The alternative is to get the gimbal closer to the target, which may be better anyway.

Oh, the problem with ducks is that they also love to eat the crops.

Are you sure? yes | no

David wrote 03/19/2019 at 11:25

nice idea, but lasers are really way too dangerous! I work professionally with lasers, and I would never use more than a few milliwatts out in the open. With 15 watts, you will blind anyone from the reflections off even a tiny piece of aluminium foil left on the ground!

If you do the math, you will see that lasers focus the light onto a tiny patch of retina, with is why a tiny 5 mW laser pointer actually focuses more energy onto a point in the retina than staring into the sun.

Infra-red doesn't help much, it will still be focused and burn out retinas.

if you went far IR, where the beam is absorbed by water, at least you would only 'cook' people's corneas, and maybe they could get a transplant...

Are you sure? yes | no

Neil K. Sheridan wrote 04/03/2019 at 20:07

I liked how the detection of slugs is going! It's interesting an all! But I just use sharp stones to protect my plants from slugs! I have literally never seen slugs or snails moving over my 1m of sharp stones! I know hedgehogs eat em, but they can get lungworm from the slugs.. Do ducks get it too?

Are you sure? yes | no

FugSlucks™ 2019

Description

Details

Files

Yolov3 train AWS p3.2xlarge instance.txt

Project Logs

Collapse logs

More tests with fixed camera focus and no zoom

Fully House Trained Slugs

Slug Detection Starts to Work in Deployment

Finally the Training is Converging

3498 Image Sets and Still No Convergence Using DetectNet !

Training 1600 image sets on Nvidia detectNet shows no convergence

Failiure to Deploy YoloV3 Model on Raspberry Pi

Slug Detection Using Raspberry Pi and Intel Neural Network Stick

Hunting for Slugs

Discussions

Similar Projects

PINTUS

Inexpensive Composite Propellers/Rotors

Data junkies part 1

FugSlucks™ 2019

Become a Hackaday.io member

Just one more thing

Description

Details

Files

Project Logs Collapse logs

Enjoy this project?

Discussions

Become a member

Similar Projects

Does this project spark your interest?

Report project as inappropriate

Send message

Remove Member

Project Logs

Collapse logs