Project | FugSlucks™ 2019

« Back to project details Sort by:

More tests with fixed camera focus and no zoom
04/11/2019 at 10:33 • 0 comments

Performance of the slug detection was improved by disabling auto focus in the Logitech C290e camera and not using zoom as the zoom is digital rather than using lenses and dramatically reduces the pixel density. The detection results seem to be a lot better as shown in the video below:
Fully House Trained Slugs
04/10/2019 at 13:49 • 0 comments

The slugs are now house trained. 10605 images of slugs have created a fairly good slug detector although there could be improvements by adding blurred images to get the training to focus more on the overall shape of the animals, including their tentacles, or antennae. In deployment, there are quite a few false positives, not really shown on the video below, which was created on a leaf background:

The training shows obvious convergence, even though testing in real life does not pick up on the overall shape of the slugs, but more readily on the patterns on their bodies. Training on AWS Nvidia V100 GPU took about 13 hours to reach 500 epochs:

Training slugs has proved to be more difficult than the common European wasps, probably because the wasps have much more obvious features whereas the slug is more of an amorphous blob with antennae. At this stage, the project is very possibly finished as the limitations of the object detection algorithm has been successfully explored and the conclusion is that more obvious features need to be apparent for effective detection.
Future work would benefit from a better camera eg Logitech Brio 4K rather than Logitech C930e, which would give 4x the pixel density and HDR. Also, more images, possibly another 10,000 of them!
Slug Detection Starts to Work in Deployment
04/04/2019 at 09:23 • 1 comment

Deployment success! Not brilliant accuracy, but it's definitely working. Most inference is between 50 and 60 % accurate. Also detects snails, which is weird as I took all the snail images out of the image sets. Must be a relic of previous training as I always use a trained model from the previous AWS session (pre-trained network).
Finally the Training is Converging
04/03/2019 at 19:43 • 0 comments

After a couple of weeks of hunting for slugs at night time and processing the images, 5,712 of them, the training is converging! It's been hard work, adding 1,000 images at a time and testing for convergence. Thank God it eventually worked!
Looking at the graph, it seems that training has got quite a bit further to go as the top red line is still sloping downwards and the loss coverage is not diminishing as it should. I'll test the model that this session created, but I think throwing another 1,000 images at the network would be a good idea. I really needed this result to boost my enthusiasm!
It's worth thinking now about the next stage in a bit more detail - how to catch / destroy the slugs? I think a Google questionnaire form might be appropriate with questions to probe people's views on the morals / ethics of killing slugs and the possibility of more 'humane' alternative solutions?
3498 Image Sets and Still No Convergence Using DetectNet !
03/22/2019 at 18:17 • 0 comments

I added another 1,000 images and text files of slugs and about 1,100 'dontcare' image sets and still no signs of convergence. I don't feel disheartened because I've already had good success with custom images of the common European wasp. Another difference between the data sets is that with the wasp images, there were often over 10 wasps in one single jpg and the wasps all looked pretty much identical, whereas the slugs are mostly one per image and there's about 10 different sprecies. Will the model converge or will I have to go back to YoloV3 to get a deployable model?
Training 1600 image sets on Nvidia detectNet shows no convergence
03/20/2019 at 13:44 • 0 comments

Even after adding another 600 images and associated text files, detectNet neural network failed to show any real signs of convergence, even after a couple of hours on AWS P100 GPU:
Since I want to make use of the recently released Nvidia Nano dev kit, it that to get this network to converge, more images are needed. This is probably because, as mentioned previously, there are at least 10 species of slug and 1 species of snail being photographed, which means approx. 160 images for each species which is a very small data set for this kind of thing.
If detectNet fails to converge, I'm sure it's possible to run YOLOV3 on the Nano without too much trouble.
Failiure to Deploy YoloV3 Model on Raspberry Pi
03/15/2019 at 16:55 • 0 comments

In my other project, the Ai Wasp sentry gun, I successfully managed to deploy a model on the Raspberry Pi using MobileNet SSD, although the results were admittedly pretty poor.

This time I thought I'd try YoloV3 as, theoretically, there is a complete software toolchain to take the Yolo model to the Pi. Training 1,000 annotated images of slugs on AWS seemed to be successful:

and testing the detection accuracy on AWS gave goodish results, so the problem may well be the Intel model optimiser software. Who knows? There's a copy of my notes here, if anybody can understand this stuff: https://cdn.hackaday.io/files/1641377022437408/Yolov3%20train%20AWS%20p3.2xlarge%20instance.txt

Or here, below, but hard to read due to bad formatting on these narrow pages:

source activate python3
git clone https://github.com/paddygoat/darknet cd darknet make
conda install -c anaconda opencv
yolov3.cfg (236 MB COCO Yolo v3) - requires 4 GB GPU-RAM: https://pjreddie.com/media/files/yolov3.weights
To train on Linux: $ ./darknet detector train data/obj.data yolo-obj.cfg darknet53.conv.74
wget -P /home/ubuntu/darknet/build/x64 https://pjreddie.com/media/files/darknet53.conv.74 cd darknet/build/x64 ./darknet detector train data/obj.data yolo-obj.cfg darknet53.conv.74
ubuntu@ip-172-31-25-87:~/darknet$ ./darknet detector train data/obj.data yolo-obj.cfg darknet53.conv.74 yolo-obj Couldn't open file: yolo-obj.cfg ubuntu@ip-172-31-25-87:~/darknet$
./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74
Loading weights from darknet53.conv.74...Couldn't open file: darknet53.conv.74 wget -P /home/ubuntu/darknet https://pjreddie.com/media/files/darknet53.conv.74 cd darknet ./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74
Total BFLOPS 65.290 Allocate additional workspace_size = 200.40 MB Loading weights from darknet53.conv.74... seen 64 Done! Learning Rate: 0.001, Momentum: 0.9, Decay: 0.0005 If error occurs - run training with flag: -dont_show
(average loss:1907): Gtk-WARNING **: cannot open display: (python3) ubuntu@ip-172-31-25-87:~/darknet$
to see the mAP & Loss-chart during training on remote server without GUI, use command: ./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 ./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 -dont_show -mjpeg_port 8090 -map then open URL http://54.190.27.97:8090 in Chrome/Firefox browser)
source activate python3 cd darknet ./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 -dont_show ./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 -dont_show -mjpeg_port 8090 -map
Can't open label file. (This can be normal only if you use MSCOCO): data/obj/AB00001.txt
Wrong annotation: class_id = 2. But class_id should be [from 0 to 0]
Only for small datasets sometimes better to decrease learning rate, for 4 GPUs set learning_rate = 0.00025 (i.e. learning_rate = 0.001 / GPUs). In this case also increase 4x times burn_in = and max_batches = in your cfg-file. I.e. use burn_in = 4000 instead of 1000.
8.1. For training with mAP (mean average precisions) calculation for each 4 Epochs (set valid=valid.txt or train.txt in obj.data file ???? test = test.txt ??? Note: If you changed width= or height= in your cfg-file, then new width and height must be divisible by 32.
wget -P /home/ubuntu/darknet/data https://github.com/paddygoat/darknet/blob/master/data/obj.data wget -P /home/ubuntu/darknet/data https://github.com/paddygoat/darknet/blob/master/data/test.txt wget -P /home/ubuntu/darknet/data/cfg https://github.com/paddygoat/darknet/blob/master/cfg/yolo-obj.cfg
./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 -dont_show -gpus 0,1,2,3 AWS p3.2xlarge only has one Tesla V100 GPU !!!!!!
./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 -dont_show -mjpeg_port 8090 -map
http://54.245.40.222:8090
Region 94 Avg IOU: -nan, Class: -nan, Obj: -nan, No Obj: 0.355157, .5R: -nan, .75R: -nan, count: 0 Region 106 Avg IOU: -nan, Class: -nan, Obj: -nan, No Obj: 0.242947, .5R: -nan, .75R: -nan, count: 0 Region 82 Avg IOU: 0.409650, Class: 0.478970, Obj: 0.430828, No Obj: 0.384286, .5R: 0.250000, .75R: 0.000000, count: 8
use $ nvidia-smi to check GPUs $ watch -n0.1 nvidia-smi
delete old jpgs and txts in darknet/data/obj wget -P /home/ubuntu/darknet/data https://github.com/paddygoat/darknet/blob/master/build/darknet/x64/data/obj.data wget -P /home/ubuntu/darknet/data https://github.com/paddygoat/darknet/blob/master/build/darknet/x64/data/obj.names wget -P /home/ubuntu/darknet/data https://github.com/paddygoat/darknet/blob/master/build/darknet/x64/data/train.txt wget -P /home/ubuntu/darknet/data https://github.com/paddygoat/darknet/blob/master/build/darknet/x64/data/valid.txt wget -P /home/ubuntu/darknet/data https://github.com/paddygoat/darknet/blob/master/build/darknet/x64/data/test.txt
https://www.kaggle.com/tegwyntwmffat/slug-detect/downloads/slug_images_and_txt_files_01.tar.gz
cd darknet/data/obj tar -xvzf slug_images_and_txt_files_01.tar.gz ./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 -dont_show -mjpeg_port 8090 -map
Tensor Cores are disabled until the first 3000 iterations are reached. (next mAP calculation at 1000 iterations) 149: 52.033905, 95.407059 avg loss, 0.000000 rate, 4.756684 seconds, 9536 images MJPEG-stream sent. Loaded: 0.000027 seconds Region 82 Avg IOU: 0.423813, Class: 0.523438, Obj: 0.195280, No Obj: 0.209835, .5R: 0.333333, .75R: 0.111111, count: 9 Region 94 Avg IOU: -nan, Class: -nan, Obj: -nan, No Obj: 0.103330, .5R: -nan, .75R: -nan, count: 0 Region 106 Avg IOU: -nan, Class: -nan, Obj: -nan, No Obj: 0.044443, .5R: -nan, .75R: -nan, count: 0 Region 82 Avg IOU: 0.438981, Class: 0.656168, Obj: 0.202069, No Obj: 0.208572, .5R: 0.375000, .75R: 0.000000, count: 8
darknet/backup/yolo-obj_last.weights darknet/backup/yolo-obj_1000.weights http://54.245.40.222:8090
./darknet detector train data/obj.data cfg/yolo-obj.cfg backup/yolo-obj_1000.weights -dont_show -mjpeg_port 8090 -map
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
source activate python3 conda install -c conda-forge tensorflow

git clone https://github.com/mystic123/tensorflow-yolo-v3.git
cd tensorflow-yolo-v3
git checkout ed60b90
source activate python3 cd tensorflow-yolo-v3
git checkout ed60b90

python3 convert_weights.py --class_names /home/ubuntu/darknet/data/coco.names.txt --size 416 --data_format NHWC --weights_file /home/ubuntu/darknet/backup/yolo-obj_1000.weights --output_graph /home/ubuntu/darknet/backup ......... Seemed to do something useful !!!!
python3 convert_weights_pb.py --class_names /home/ubuntu/darknet/data/coco.names.txt --size 416 --data_format NHWC --weights_file /home/ubuntu/darknet/backup/yolo-obj_1000.weights .................. WORKED!
RESULTS: (python3) ubuntu@ip-172-31-25-87:~/tensorflow-yolo-v3$ python3 convert_weights.py --class_names /home/ubuntu/darknet/data/coco.names.txt --size 416 --data_format NHWC --weights_file /home/ubuntu/darknet/backup/yolo-obj_1000.weights --output_graph /home/ubuntu/darknet/backup 2019-03-11 10:41:59.937583: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA Model saved in path: ./saved_model/model.ckpt (python3) ubuntu@ip-172-31-25-87:~/tensorflow-yolo-v3$
RESULTS: (python3) ubuntu@ip-172-31-25-87:~/tensorflow-yolo-v3$ python3 convert_weights_pb.py --class_names /home/ubuntu/darknet/data/coco.names.txt --size 416 --data_format NHWC --weights_file /home/ubuntu/darknet/backup/yolo-obj_1000.weights 2019-03-11 10:47:47.185367: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 1330 ops written to frozen_darknet_yolov3_model.pb. (python3) ubuntu@ip-172-31-25-87:~/tensorflow-yolo-v3$
coco.names.txt must contain 'person' only for one class. frozen model was found in /home/ubuntu/tensorflow-yolo-v3/ (.pb file)
//////////////////////////////////////////////////////////////////////////////////////////////////
0. source activate python3 1. Upload new yolo-obj.cfg file to 'cfg' directory 2. delete 'obj' folder in darknet/data/ 3. Upload slug_images_and_txt_files_02.tar.gz to darknet/data/ 4. cd darknet/data/ 5. tar -xvzf slug_images_and_txt_files_02.tar.gz ...... This will extract image and txt files to a new 'obj' folder 6. delete darknet/backup/yolo-obj_1000.weights 7. cd && cd darknet/
8. ./darknet detector train data/obj.data cfg/yolo-obj.cfg backup/yolo-obj_1000.weights -dont_show -mjpeg_port 8090 -map ..... or....... ./darknet detector train data/obj.data cfg/yolo-obj.cfg -dont_show -mjpeg_port 8090 -map ...... or ........ ./darknet detector train data/obj.data cfg/yolo-obj.cfg backup/yolo-obj_last.weights -dont_show -mjpeg_port 8090 -map
http://34.215.134.36:8090 ...... open in chrome only.
9. cd tensorflow-yolo-v3
10. python3 convert_weights.py --class_names /home/ubuntu/darknet/data/coco.names.txt --size 416 --data_format NHWC --weights_file /home/ubuntu/darknet/backup/yolo-obj_11000.weights --output_graph /home/ubuntu/darknet/backup ......... Seemed to do something useful !!!!
11. python3 convert_weights_pb.py --class_names /home/ubuntu/darknet/data/coco.names.txt --size 416 --data_format NHWC --weights_file /home/ubuntu/darknet/backup/yolo-obj_11000.weights
12. delete tensorflow-yolo-v3/frozen_darknet_yolov3_model.pb
13.cd && cd /opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/extensions/front/tf/ 14.sudo chmod 777 yolo_v3.json 15.ls -l yolo_v3.json 16. Edit yolo_v3.json accordingly.
17. cd && cd /opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/
18. python3 mo_tf.py --input_model /home/ubuntu/tensorflow-yolo-v3/frozen_darknet_yolov3_model.pb --tensorflow_use_custom_operations_config /opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/extensions/front/tf/yolo_v3.json --input_shape=[1,416,416,3] --data_type=FP16
19. saved bin and xml at /opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/ 20. cd && cd darknet/ && ./darknet detector test data/obj.data cfg/yolo-obj.cfg /home/ubuntu/darknet/backup/yolo-obj_11000.weights
/home/ubuntu/darknet/data/obj/AA00502.jpg /home/ubuntu/darknet/data/obj/AA00501.jpg /home/ubuntu/darknet/data/obj/AA00500.jpg /home/ubuntu/darknet/data/obj/AA00499.jpg /home/ubuntu/darknet/data/obj/AA00498.jpg /home/ubuntu/darknet/data/obj/AA00497.jpg /home/ubuntu/darknet/data/obj/AA00001.jpg /home/ubuntu/darknet/data/obj/AA00307.jpg /home/ubuntu/darknet/data/obj/AA00207.jpg /home/ubuntu/darknet/data/obj/AA00107.jpg /home/ubuntu/darknet/data/obj/AA00007.jpg

/home/ubuntu/darknet/data/obj/AB00497.jpg /home/ubuntu/darknet/data/obj/AB00001.jpg /home/ubuntu/darknet/data/obj/AB00307.jpg /home/ubuntu/darknet/data/obj/AB00207.jpg /home/ubuntu/darknet/data/obj/AB00107.jpg /home/ubuntu/darknet/data/obj/AB00007.jpg
/home/ubuntu/darknet/Test/test001.jpg /home/ubuntu/darknet/Test/test002.jpg /home/ubuntu/darknet/Test/test003.jpg
Loading weights from /home/ubuntu/darknet/backup/yolo-obj_11000.weights... seen 64 Done! Enter Image Path: /home/ubuntu/darknet/Test/test002.jpg /home/ubuntu/darknet/Test/test002.jpg: Predicted in 37.062000 milli-seconds. person: 61%
(predictions:2080): Gtk-WARNING **: cannot open display: (python3) ubuntu@ip-172-31-25-87:~/darknet$
Slug Detection Using Raspberry Pi and Intel Neural Network Stick
03/12/2019 at 17:11 • 0 comments

Watch this space for major update in the next couple of days!
Hunting for Slugs
03/03/2019 at 11:46 • 0 comments

Training neural networks for detecting slugs requires hunting down slugs in the night time and photographing them under the light of a head torch with macro lens. Fortunately, they don't move too quickly, but photography is still challenging as the macro lens is very sensitive to movement so the aperture needs to be quite large which means getting focus correct is hard. An advantage is that the background gets blurred, especially when the slugs are silhouetted somewhat. A minimum of 2,500 photos are required and each one needs to be cropped, resized to 640 x 640 pixels, renamed and then labelled. I seem to be able to do an average of about 100 a day and have done 1,000 so far.
Slugs are amazing creatures and getting this close to them is quite revealing. I think most people would think of them as being pretty gross, but on close up they are truly beautiful animals. I think I identified about 10 different species, from the large black ones to the small dappled.