The project is back on track after losing a year to Covid19 & other distractions life throws at us.

Bottom line up front: Achieved 75Hz full processing including SBUS data read, frame grab, convert to numpy array, process image to track, generate PID control quad command, and finally SBUS write to the quad's flight controller.

Details:

For now, I've abandoned the use of USB cameras due to cost and other reasons.

Switched to Pi 4 model b.

Currently switched to Pi Camera Module V2 noIR via CSI ribbon cable.  Achieved high frame rates at 640x480 finally, 90fps in mode7.  In order to perform manual image processing, I don't need the regular jpeg compressed frames, rather need frames as numpy arrays, which was also achieved with very low latency. 

The SBUS update rate on the remote control system I'm using is 75Hz (Radiolink AT9S transmitter with R6DSM receiver on the quad).

Remember folks, the quad's flight controller (Omnibus F4 running Betaflight) would normally be commanded via SBUS from radio receiver.  But in this project we run the SBUS through the Pi.  The Pi can be in pass through mode or autopilot mode where the Pi can inject pitch, yaw, roll commands.  I have not implemented 'throttle' yet, currently the Human controls throttle at all times. 

Testing using my ceiling fan with a hair-scrunchy strung between blades using fishing line was conducted.  To get the best loop rate, I shut off any (cv2.imshow) screen updates of the image. But I still need to see what the camera and processing was doing, so I store the images(numpy arrays) and X & Y tracker (box around target) output to local memory (just a python list).  For test purposes, I can set the script to run for some number of frames ( I used 700 frames ), upon completion it writes .jpg of each image and plots determined X & Y track positions using matplotlib pyplot.

The PID flight control of pitch and roll is implemented and checked out by monitoring the Betaflight GUI for the values being sent to the flight controller.

Outdoor tests with very limited flight while tracking another quad (DJI Mavic mini) have also been conducted a few times.

The target drone was set up to fly around with a hard acceleration in one direction at first and then some minor movement in other direction as you can see in the plots of tracker position.