Close

Failiure to Deploy YoloV3 Model on Raspberry Pi

A project log for FugSlucks 2019

Ai slug detector

Tegwyn☠TwmffatTegwyn☠Twmffat 03/15/2019 at 16:550 Comments

In my other project, the Ai Wasp sentry gun, I successfully managed to deploy a model on the Raspberry Pi using MobileNet SSD, although the results were admittedly pretty poor.

This time I thought I'd try YoloV3 as, theoretically, there is a complete software toolchain to take the Yolo model to the Pi. Training 1,000 annotated images of slugs on AWS seemed to be successful:

and testing the detection accuracy on AWS gave goodish results, so the problem may well be the Intel model optimiser software. Who knows? There's a copy of my notes here, if anybody can understand this stuff: https://cdn.hackaday.io/files/1641377022437408/Yolov3%20train%20AWS%20p3.2xlarge%20instance.txt

Or here, below, but hard to read due to bad formatting on these narrow pages:


 
source activate python3  
git clone https://github.com/paddygoat/darknet cd darknet make  
conda install -c anaconda opencv  
yolov3.cfg (236 MB COCO Yolo v3) - requires 4 GB GPU-RAM: https://pjreddie.com/media/files/yolov3.weights  
To train on Linux: $ ./darknet detector train data/obj.data yolo-obj.cfg darknet53.conv.74  
wget -P /home/ubuntu/darknet/build/x64 https://pjreddie.com/media/files/darknet53.conv.74 cd darknet/build/x64 ./darknet detector train data/obj.data yolo-obj.cfg darknet53.conv.74  
ubuntu@ip-172-31-25-87:~/darknet$ ./darknet detector train data/obj.data yolo-obj.cfg darknet53.conv.74 yolo-obj Couldn't open file: yolo-obj.cfg ubuntu@ip-172-31-25-87:~/darknet$  
./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74  
Loading weights from darknet53.conv.74...Couldn't open file: darknet53.conv.74 wget -P /home/ubuntu/darknet https://pjreddie.com/media/files/darknet53.conv.74 cd darknet ./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74  
Total BFLOPS 65.290  Allocate additional workspace_size = 200.40 MB Loading weights from darknet53.conv.74...  seen 64 Done! Learning Rate: 0.001, Momentum: 0.9, Decay: 0.0005  If error occurs - run training with flag: -dont_show  
(average loss:1907): Gtk-WARNING **: cannot open display: (python3) ubuntu@ip-172-31-25-87:~/darknet$  
to see the mAP & Loss-chart during training on remote server without GUI, use command: ./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 ./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 -dont_show -mjpeg_port 8090 -map then open URL http://54.190.27.97:8090 in Chrome/Firefox browser)  
source activate python3 cd darknet ./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 -dont_show ./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 -dont_show -mjpeg_port 8090 -map  
Can't open label file. (This can be normal only if you use MSCOCO): data/obj/AB00001.txt  
 Wrong annotation: class_id = 2. But class_id should be [from 0 to 0]  
Only for small datasets sometimes better to decrease learning rate, for 4 GPUs set learning_rate = 0.00025 (i.e. learning_rate = 0.001 / GPUs).  In this case also increase 4x times burn_in = and max_batches = in your cfg-file. I.e. use burn_in = 4000 instead of 1000.  
8.1. For training with mAP (mean average precisions) calculation for each 4 Epochs (set valid=valid.txt or train.txt in obj.data file  ???? test = test.txt ??? Note: If you changed width= or height= in your cfg-file, then new width and height must be divisible by 32.  
wget -P /home/ubuntu/darknet/data https://github.com/paddygoat/darknet/blob/master/data/obj.data wget -P /home/ubuntu/darknet/data https://github.com/paddygoat/darknet/blob/master/data/test.txt wget -P /home/ubuntu/darknet/data/cfg https://github.com/paddygoat/darknet/blob/master/cfg/yolo-obj.cfg  
./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 -dont_show -gpus 0,1,2,3 AWS p3.2xlarge only has one Tesla V100 GPU !!!!!!  
./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 -dont_show -mjpeg_port 8090 -map  
http://54.245.40.222:8090  
Region 94 Avg IOU: -nan, Class: -nan, Obj: -nan, No Obj: 0.355157, .5R: -nan, .75R: -nan,  count: 0 Region 106 Avg IOU: -nan, Class: -nan, Obj: -nan, No Obj: 0.242947, .5R: -nan, .75R: -nan,  count: 0 Region 82 Avg IOU: 0.409650, Class: 0.478970, Obj: 0.430828, No Obj: 0.384286, .5R: 0.250000, .75R: 0.000000,  count: 8  
use $ nvidia-smi to check GPUs       $ watch -n0.1 nvidia-smi  
delete old jpgs and txts in darknet/data/obj wget -P /home/ubuntu/darknet/data https://github.com/paddygoat/darknet/blob/master/build/darknet/x64/data/obj.data wget -P /home/ubuntu/darknet/data https://github.com/paddygoat/darknet/blob/master/build/darknet/x64/data/obj.names wget -P /home/ubuntu/darknet/data https://github.com/paddygoat/darknet/blob/master/build/darknet/x64/data/train.txt wget -P /home/ubuntu/darknet/data https://github.com/paddygoat/darknet/blob/master/build/darknet/x64/data/valid.txt wget -P /home/ubuntu/darknet/data https://github.com/paddygoat/darknet/blob/master/build/darknet/x64/data/test.txt  
https://www.kaggle.com/tegwyntwmffat/slug-detect/downloads/slug_images_and_txt_files_01.tar.gz  
cd darknet/data/obj tar -xvzf slug_images_and_txt_files_01.tar.gz ./darknet detector train data/obj.data cfg/yolo-obj.cfg darknet53.conv.74 -dont_show -mjpeg_port 8090 -map  
 Tensor Cores are disabled until the first 3000 iterations are reached. (next mAP calculation at 1000 iterations)  149: 52.033905, 95.407059 avg loss, 0.000000 rate, 4.756684 seconds, 9536 images  MJPEG-stream sent. Loaded: 0.000027 seconds Region 82 Avg IOU: 0.423813, Class: 0.523438, Obj: 0.195280, No Obj: 0.209835, .5R: 0.333333, .75R: 0.111111,  count: 9 Region 94 Avg IOU: -nan, Class: -nan, Obj: -nan, No Obj: 0.103330, .5R: -nan, .75R: -nan,  count: 0 Region 106 Avg IOU: -nan, Class: -nan, Obj: -nan, No Obj: 0.044443, .5R: -nan, .75R: -nan,  count: 0 Region 82 Avg IOU: 0.438981, Class: 0.656168, Obj: 0.202069, No Obj: 0.208572, .5R: 0.375000, .75R: 0.000000,  count: 8  
darknet/backup/yolo-obj_last.weights darknet/backup/yolo-obj_1000.weights http://54.245.40.222:8090  
./darknet detector train data/obj.data cfg/yolo-obj.cfg backup/yolo-obj_1000.weights -dont_show -mjpeg_port 8090 -map  
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////  
source activate python3 conda install -c conda-forge tensorflow  
 
 
git clone https://github.com/mystic123/tensorflow-yolo-v3.git  
cd tensorflow-yolo-v3  
git checkout ed60b90  
source activate python3 cd tensorflow-yolo-v3  
git checkout ed60b90  
 
 
python3 convert_weights.py --class_names /home/ubuntu/darknet/data/coco.names.txt --size 416 --data_format NHWC --weights_file /home/ubuntu/darknet/backup/yolo-obj_1000.weights --output_graph /home/ubuntu/darknet/backup  ......... Seemed to do something useful !!!!  
python3 convert_weights_pb.py --class_names /home/ubuntu/darknet/data/coco.names.txt --size 416 --data_format NHWC --weights_file /home/ubuntu/darknet/backup/yolo-obj_1000.weights .................. WORKED!  
RESULTS: (python3) ubuntu@ip-172-31-25-87:~/tensorflow-yolo-v3$ python3 convert_weights.py --class_names /home/ubuntu/darknet/data/coco.names.txt --size 416 --data_format NHWC --weights_file /home/ubuntu/darknet/backup/yolo-obj_1000.weights --output_graph /home/ubuntu/darknet/backup 2019-03-11 10:41:59.937583: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA Model saved in path: ./saved_model/model.ckpt (python3) ubuntu@ip-172-31-25-87:~/tensorflow-yolo-v3$  
RESULTS: (python3) ubuntu@ip-172-31-25-87:~/tensorflow-yolo-v3$ python3 convert_weights_pb.py --class_names /home/ubuntu/darknet/data/coco.names.txt --size 416 --data_format NHWC --weights_file /home/ubuntu/darknet/backup/yolo-obj_1000.weights 2019-03-11 10:47:47.185367: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 1330 ops written to frozen_darknet_yolov3_model.pb. (python3) ubuntu@ip-172-31-25-87:~/tensorflow-yolo-v3$  
coco.names.txt must contain 'person' only for one class. frozen model was found in /home/ubuntu/tensorflow-yolo-v3/  (.pb file)  
//////////////////////////////////////////////////////////////////////////////////////////////////  
0. source activate python3 1. Upload new yolo-obj.cfg file to 'cfg' directory 2. delete 'obj' folder in darknet/data/ 3. Upload slug_images_and_txt_files_02.tar.gz to darknet/data/ 4. cd darknet/data/ 5. tar -xvzf slug_images_and_txt_files_02.tar.gz  ...... This will extract image and txt files to a new 'obj' folder 6. delete darknet/backup/yolo-obj_1000.weights 7. cd && cd darknet/  
8. ./darknet detector train data/obj.data cfg/yolo-obj.cfg backup/yolo-obj_1000.weights -dont_show -mjpeg_port 8090 -map ..... or.......   ./darknet detector train data/obj.data cfg/yolo-obj.cfg -dont_show -mjpeg_port 8090 -map  ...... or ........     ./darknet detector train data/obj.data cfg/yolo-obj.cfg backup/yolo-obj_last.weights -dont_show -mjpeg_port 8090 -map  
http://34.215.134.36:8090 ...... open in chrome only.  
9. cd tensorflow-yolo-v3  
10. python3 convert_weights.py --class_names /home/ubuntu/darknet/data/coco.names.txt --size 416 --data_format NHWC --weights_file /home/ubuntu/darknet/backup/yolo-obj_11000.weights --output_graph /home/ubuntu/darknet/backup  ......... Seemed to do something useful !!!!  
11. python3 convert_weights_pb.py --class_names /home/ubuntu/darknet/data/coco.names.txt --size 416 --data_format NHWC --weights_file /home/ubuntu/darknet/backup/yolo-obj_11000.weights  
12. delete tensorflow-yolo-v3/frozen_darknet_yolov3_model.pb  
13.cd && cd /opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/extensions/front/tf/ 14.sudo chmod 777 yolo_v3.json 15.ls -l yolo_v3.json 16. Edit yolo_v3.json accordingly.  
17. cd && cd /opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/  
18. python3 mo_tf.py --input_model /home/ubuntu/tensorflow-yolo-v3/frozen_darknet_yolov3_model.pb --tensorflow_use_custom_operations_config /opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/extensions/front/tf/yolo_v3.json --input_shape=[1,416,416,3] --data_type=FP16  
19. saved bin and xml at /opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/ 20. cd && cd darknet/ && ./darknet detector test data/obj.data cfg/yolo-obj.cfg /home/ubuntu/darknet/backup/yolo-obj_11000.weights  
/home/ubuntu/darknet/data/obj/AA00502.jpg /home/ubuntu/darknet/data/obj/AA00501.jpg /home/ubuntu/darknet/data/obj/AA00500.jpg /home/ubuntu/darknet/data/obj/AA00499.jpg /home/ubuntu/darknet/data/obj/AA00498.jpg /home/ubuntu/darknet/data/obj/AA00497.jpg /home/ubuntu/darknet/data/obj/AA00001.jpg /home/ubuntu/darknet/data/obj/AA00307.jpg /home/ubuntu/darknet/data/obj/AA00207.jpg /home/ubuntu/darknet/data/obj/AA00107.jpg /home/ubuntu/darknet/data/obj/AA00007.jpg  
 
/home/ubuntu/darknet/data/obj/AB00497.jpg /home/ubuntu/darknet/data/obj/AB00001.jpg /home/ubuntu/darknet/data/obj/AB00307.jpg /home/ubuntu/darknet/data/obj/AB00207.jpg /home/ubuntu/darknet/data/obj/AB00107.jpg /home/ubuntu/darknet/data/obj/AB00007.jpg  
/home/ubuntu/darknet/Test/test001.jpg /home/ubuntu/darknet/Test/test002.jpg /home/ubuntu/darknet/Test/test003.jpg  
Loading weights from /home/ubuntu/darknet/backup/yolo-obj_11000.weights...  seen 64 Done! Enter Image Path: /home/ubuntu/darknet/Test/test002.jpg /home/ubuntu/darknet/Test/test002.jpg: Predicted in 37.062000 milli-seconds. person: 61%  
(predictions:2080): Gtk-WARNING **: cannot open display: (python3) ubuntu@ip-172-31-25-87:~/darknet$  


Discussions