Head Tracking Virtual Window

Tracks the user's head and displays the appropriate view of a virtual scene.

Similar projects worth following
The goal of this project is to make a regular display work like a window into the virtual world. This is done by tracking the user's head with a camera and rendering a distorted image to account for the user's perspective of the display. The effect becomes more convincing at higher framerates, so performance is critical. This could be used for entertainment, productivity, or even as decoration.

Written in Python, using OpenCV for image manipulation, Mediapipe for face tracking, and Pygame and OpenGL for 3D rendering. Face tracking works at a wide range of distances, from a few inches to at least 6 feet away, and at angles you probably wouldn't use a display at. The effect is best for producing small amounts of inward depth; scenes that "pop out" or are very far away suffer greatly from a lack of stereo depth.

A long-term goal is for this to be a software tool that would allow any display and camera to be calibrated for use as this type of virtual window.

View project log

Enjoy this project?



Colton H Sandwich wrote 03/30/2023 at 00:21 point

Have you been able to continue this project? Been looking to do this exactly 

  Are you sure? yes | no

alan.ramsay wrote 01/07/2023 at 11:19 point

I've been contemplating something like this with 3d data for a little while, after doing something similar with video feeds a while back, nice work!

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates