After some heavy refactoring of the CUDA code the software can now keep up with the framerate! The biggest bottleneck was the pow function in CUDA. I replaced it with a log / exp combo and now it is flying!

I also 3d printed a contraption that controls the magnification of the lens. A NEMA 8 motor is connected to the 3d printer as extruder motor. This means smooth and coordinated moves for X, Y, Z and E. The lens has some drawbacks: it won't keep focus when zoomed, so each zoom is accompanied by a small change in Z to try and keep the lens focused during zooming.

Finally, I made a video and posted it in the details section!