Yeah it looks pretty ridiculous definitely will be a short lived project.
This will use a Pi 4 with 2 steppers to auto zoom a trained model of an rc plane. The Pi 4 and lens will be on one side, while the batteries will be on the other side. Designed to attach to the sides of the hat.
The ideal plan would be to have presets eg. delta, flying wing or standard. You would film your plane and label photos of it for training.
Voice command would also be nice but for now I will go with an OLED display and some tactile buttons for field control.
This will be a long-term project. I just started designing the zooming bit.
This is just a 8-50mm lens so I'm not sure how far away I can zoom, my model airplane has a 1.5m wingspan so it's not small but even a 200mm lens on a full frame sensor is not enough to zoom in.
So I need a way to control the zoom with my voice.
I tried some different libraries initially eg. sphinx and vosk but they were bad in terms of accuracy.
Also I was probably using them for the wrong thing (not intent).
Someone mentioned picovoice to me and that's what I'm using now.
I got a model of two intents: ZoomIn and ZoomOut (for the utterance Zoom in and Zoom out)
I got this running locally, verified it works offline (since it requires an access key) by turning wifi off.
Then I paired it with the stepper control
See recent video/demo below
My code flow/architecture is still garbage at this point, still bridging things.
The display for example is messed since it needs a single instance and I have to propagate that all the way down.
I added battery life tracking too (based on max measured uptime and CRON 5min increment). I also went away from the box drawn around the active choice in the menu to color, since the box has to be drawn one piece at a time (4 draw calls) with the current code I'm using.
I also fixed an audio issue, in the above you can kind of see that the 3.5mm audio jack plug does not have a ground going to the speaker, that was causing my constant buzzing/hissing issue.
It got really bad when the speech intent listener loop was running.
Made good progress on modeling the parts so I can design the case that holds everything together.
I also designed and printed the parallel battery box (pictured above middle).
I have to make a model of the Pi 4B still and then do the designing... aiming to get the entire body done by end of this week if no printing/measuring mistakes.
I also cut the first holes into the hat for the bolt mounts.
And... I already made a mistake, the part I printed is the wrong thing. I used the black battery cell holder plastic things instead of the wrapper design that goes around it... so I gotta redesign that and print it again... great. Was only a 2 hour print though.
Initially I was trying to use canny edge but it was suggested to me to use laplace variance (and others). I just went with this one for now since it's a basic one-liner from OpenCV.
It works pretty well of course this is one test case (my desk). I need to try it out and add catches for the field case (clear blue sky).
This is good for now, I was able to integrate the camera stream with the steppers/make a control loop.
I'm going to proceed with new parts of the project eg. the OLED disply, speakers and d-pad mostly because I need to make sure I have enough pins and then design the case that holds all these parts together.
That'll be nicer to work with than a box of parts and a breadboard.
So.... this took me a bit longer than it should have. I also had an oversight where I did not add a zero-position bumper. I still need to make sure I have enough pins left for the d-pad and display.
I decided to store the positions of the steppers so they could be resumed/rezeroed on boot. Also allows you to make sure you don't go too far.
Had some problems with Python to split a class/have everything work still.
So at this point I'm going to move forward with the frame sampling and based on "zoom intuition" write an algorithm to auto zoom.
The steppers are slow, so I have a basic goal of being able to zoom/keep up with some motion.
I'm largely relying on narrow aperture and far zooming to keep something in focus.
I have not sampled a frame from a video feed before and need to limit processing under 30ms I believe... so it'll be interesting.
So far it seems the farthest the plane can be at 1/3 frame is around 100ft... which is not great... but maybe 1/3 is too big anyway... so you could do 200 ft... which your ceiling is 400ft and I normally fly around 200' or so from a launch.
This was a quick design and had problems. I overlooked the middle open/close ring being taller than the diameter of the lens barrel. So the telephoto ring gear I had to sand... I also had to sand the front-lens support part. I had to sand it a lot... so it would fit.
Then I had problems with the stepper GPIO's interfering with the camera detection for some reason. If they're plugged (ground connected specifically) on boot, the camera cannot be detected, I don't know why.
So I'll put a switch in for that, I know you could do an electron switch, maybe even a relay but not worth it for me.
Now I can actually do the frame by frame analysis to try and autozoom this thing.
So... I started printing parts and getting rough sizing... I was planning on using some left over MG90D servos... however they only rotate 180 deg (with this pwm code I'm using) and also the gearing cuts it in half... the focus ring rotates the most on the 8-50mm lens. So I have to switch to these steppers that I already have and they are kind of big... oh well. This thing will be chonk.
After this I'll design the camera mount which also supports the lens and the steppers will attach to that.
I will have a programmable unit sometime this week. The software is the hardest part about this project.