Portrait mode & HDMI tapping

A project log for Auto tracking camera

A camera that tracks a person & counts reps using the same code.

lion mclionheadlion mclionhead 05/17/2020 at 19:100 Comments

The tracker needs to support still photos in portrait mode.  Portrait mode is never used for video, at least by these animals.  A few days of struggle yielded this arrangement for portrait mode. The servo mount may actually be designed to go into this configuration. 

It mechanically jams before smashing the camera. Still photos would be the 1st 30 minutes & video the 2nd 30 minutes of a model shoot, since it takes a long time to set up portrait mode.

News flash: openpose can't handle rotations.  It can't detect a lion standing on its head.  It starts falling over with just 90 degree rotations. The video has to be rotated before being fed to openpose.

Also installed were the HDMI video feed & the remote shutter control.  These 2 bits achieved their final form.  Absolutely none of the EOS RP's computer connections ended up useful.  Note the gears gained a barrier from the cables.

News flash: the EOS RP can't record video when it's outputting a clean HDMI signal.  The reason is clean HDMI causes it to show a parallel view on the LCD while sending a clean signal to HDMI.  It uses a 2nd channel that would normally be used for recording video & there's no way to turn it off. 

Your only recording option when outputting clean HDMI is on the laptop.  Helas the $50 HDMI board outputs real crummy JPEG's at 29.97fps or 8 bit YUYV at 5fps.  It's limited by USB 2.0 bandwidth, no good for pirating video or any recording.

The mighty Tamron 17-35mm was the next piece.  It was the lion kingdom's 1st lens in 12 years.  The lion kingdom relied on a 15mm fisheye for its entire wide angle career.  It was over $500 when it was new.  It was discontinued in 2011 & used ones are now being sold for $350.  Its purchase was inspired by a demo photo with the 15mm on an EOS 1DS.

Defishing the 15mm in software gave decent results for still photos, but less so for video.  There will never be a hack to defish the 15mm in the camera.

With the HDMI tap, it could finally take pictures & record video through the camera.  The tracker did its best to make some portraits.  The tracking movement caused motion blur.  Large deadband is key for freezing the camera movement.  Portrait mode still needs faster horizontal movement, because it has less horizontal room.

Openpose lacks a way to detect the outline of an animal.  It only detects eyes, so the size of the head has to be estimated by the shoulder position.  It gets inaccurate if the animal bends over.  Openpose has proven about as good at detecting a head as a dedicated face tracker.

The tracker has options for different lenses.  Longer lenses make it hunt. 50mm has been the limit for these servos.   Adding deadbands reduces the hunting but makes it less accurate.  It's definitely going to require a large padding from the frame edges.  For talking heads, the subject definitely needs to be standing up for the tracker to estimate the head size.

A corner case is if the entire body is in frame, but the lower section is obstructed.  The tracker could assume the lower section is out of frame & tilt down until the head is on top.  In practice, openpose seems to create a placeholder when the legs are obstructed.