Commercial tracking options

A project log for Auto tracking camera

A camera that tracks a person & counts reps using *AI*.

lion mclionheadlion mclionhead 12/07/2021 at 07:090 Comments

A home made tracker by now is surely a huge waste compared to a DJI robot.  Maybe it wasn't when it began in 2019.  It's hard to believe a junk laptop which finally allowed pose tracking at a reasonable speed didn't arrive until June 2019, so lions have only had 2.5 years of tracking of any serious quality.  3 years of total pose tracking including the low quality tablet camera.  The final tablet source code was lost despite only dating back to June 2019.

Despite inflation, the cost of webcam quality video with tracking plummeted since 2019 & we have the 1st decent look at the user interface.

Fortunately, the picture quality is still horrible.  There are tracking speed options & a composition option to change how high the head is in the frame.  There's a differentiation between resetting the gimbal & starting tracking.  That's quite difficult on the servocity because it has no center PWM.  Apple introduced a tracking feature called centerstage in 2021 which crops the full phone cam in on a face. 

These all track the face instead of the full body.  No-one knows how well they track 2 faces.

The DJI RS2 is the only tracker which uses a DSLR.  Now that more detailed reviews abound, it's more limited than expected.  It tracks a feature rather than a body or a face.  It's not reliable enough to track a deforming feature for 30 minutes, unattended.  It receives servo commands from a phone where the tracking is performed.   The phone receives video from an HDMI transmitter sold separately.

The only thing making the gimbal relevant is the proprietary wireless protocol between the phone & the motors.  The phone app could use any video source to control any 2 servos, but is locked into using just a DJI transmitter & a DJI gimbal.  It's not clear if the HDMI transmitter requires the gimbal to function.

The stabilization of the gimbal is worthless if it's always mounted on a tripod, but its stabilization is equivalent to the lion kingdom's hacked Feiyu.  They all rely on camera stabilization to achieve perfect smoothness.  It would be a true waste to buy a DJI RS2 just for tracking, although lions could use the gimbal for photographing goats.

Tracking a mane as well as a full body is the holy grail.  The confuser would need to run 2 models simultaneously.  

There was an interesting bug where the stdin & stdout handlers for ffmpeg held the same mutex.  It deadlocked since ffmpeg couldn't empty stdin without writing to stdout.  The easy solution was using a file descriptor instead of a file pointer to access the streams.  That didn't require the mutex.

To handle gimbal centering, it saves the last manually dialed position as the center.  Going in & out of tracking mode doesn't reset the center & it doesn't rewind the camera to the center.  It's essential to not have anything else change to go between portrait & landscape mode.  The user can press a center button when it's not tracking.

Some portrait mode tests showed how much junk is required to feed the flash & remote control.  It needs a better place to install the flash & a place to install the phone.  The wireless GUI allowed some mane shots without looking at the laptop.  There might be a case for manual pointing with the GUI, but carrying around the phone & the camera remote would be a big deal.

The openpose model couldn't track a lion bending over with his mane untied.  It could track the mane tied up but ran away straight up when the mane was let down.  Thus, limits on PWM were still needed.  

This alignment made it appear to tilt up looking for a head.

That was about all that could be done without a jetson nano.  The rep counter is still trending towards a dedicated LCD panel.