This project will have five separate audio tone generators controlled by the X,Y voltages. I wanted the volume of these tones to be controlled by the distance of the fingers from the screen (Z). I'm not very musically inclined, but I can tap a pattern with my fingers. Using the distance of the fingers as volume should translates the tapping motions to a sequence of notes.

It wasn't possible to get a true Z-axis value like it was with X and Y. Those aren't effected by anything other than the physical phototransistor position provided there's enough light to trigger the detector circuit. The Z-axis is based on the amount of light hitting the phototransistor, so it's effected by both the distance of the phototransistor from the monitor and the brightness of the monitor.

When the raster beam hits the phototransistor it produces a short pulse that increases in voltage based on the brightness and distance of the sensor. The phototransistor pulse is fed into a peak detector to get a consistent voltage based on the pulse height. The pulse occurs at a frequency of 60Hz (monitor refresh rate) and this shows up in the peak detector output. It's filtered out with a low pulse filter so there isn't a 60Hz hum in every audio signal that passes through the voltage controlled amplifier.

Schematic showing the Z-axis output, audio mix, and detection pulse circuit. There are five of these (one per finger). All five audio outputs connect directly together to form a single mixed channel.

Using the Z-axis to manipulate video was an afterthought so the Z output to the video synth is not properly buffered or filtered. That had to be placed on the video synth board. Making some changes to fix that is on my todo list.

Here's me playing with the X,Y, and Z axis. The video effect circuit used to generate this comes in the next step.