The Depth Quandary

A project log for P43D - Ping Pong Pixel Painter

Track the size and angle of a lighted ping pong ball and use this to draw in 3D virtual space.

Daren SchwenkeDaren Schwenke 01/22/2021 at 20:191 Comment

The resolution of the depth information you can get just from object size is going to drop off really quickly as you move away from the camera.

When you are relatively close to the camera, the ball will occupy a lot of pixels and the depth of the object within the frame is high resolution.  

As you move away from the camera however, the number of pixels the ball is occupying in your captured image rapidly decreases.  

As that happens, the possible position of the ball along a line in space drawn from the camera to the center of the ball, gets larger and larger.  Eventually a single pixel error in calculating the ball size will translate to a massive positioning error along that line!

Some Solutions

  1. We use a larger ball.  
    1. This moves the useable range of the sensing out further at the expense of near
    2. A larger wand is more cumbersome. 
  2. We use a camera with a narrower FOV. 
    1. This decreases the usable width and height of the performance area, but then extends the length.
    2. The camera will need to be further away from the performance area to start with.
  3. We use a camera with a really high resolution.
    1. The speed of OpenCV is inversely proportional to the number of pixels you are feeding it.  Bigger images are a lot slower to process.
    2. Using the very low latency integrated CSI-2 camera interface of the PI or Jetson Nano, we end up maxing out at 8MP or about 1080P@30fps of RAW image data.  To do much better than this we would need to use a USB-C camera and the cost skyrockets.
  4. We use stereo imaging and determine the depth via triangulation.  
    1. We immediately lose half of the capture rate.
    2. We lose the ability to track multiple objects without more work as we can't triangulate each object without being able to differentiate them.  
      1. Flashing each ball works, but then your capture rate goes down really fast as only one ball per color can be lit per captured frame and then the flashing must to be coordinated.  
      2. Using different colored balls works, but then you lose the ability to paint in the color you are seeing as the ball.

For now I think the solution I am going with is to use a narrower FOV CSI-2 camera that can do 1080P@30fps and feed it to a Jetson Nano.


marypeck369 wrote 02/03/2021 at 05:45 point osm site to all in one

  Are you sure? yes | no