The PanoCama is a design for taking stereoscopic panoramic photos. By panning and tilting the cameras, the device can reconstruct a single wide-angle image through image stitching. Through its dual-camera design, the system can create rudimentary disparity maps as a precursor 3D environmental mapping. Image stitching and computer stereo vision are both computationally expensive tasks. Thus, to accelerate processing and prolong battery life, images are processed by a dedicated external server. While the server handles image processing, The onboard microcontroller, MCU, is responsible for interfacing with both cameras, controlling the pan-tilt servos, and maintaining web connection with the server. Rather than directly hosting a web server on the microprocessor, a separate device acts as proxy and issues commands to the device.

The main parts of this project fall into a few broad categories: ESP32 firmware, Tornado web server, panoramic stitching, camera calibration, stereo correspondence, PCB designs, and 3D printed parts. I made a working prototype with an ESP32 DEVKIT V1, a pan-tilt servo kit, and two SPI cameras. The main focus of this project is further exploring PCB design and image processing. Thus, the design uses a prebuilt pan-tilt system from Adafruit rather than custom 3D printed or machine pieces. Similarly, rather than multiplexing parallel buses from two OV2640 image sensors, the design uses prebuilt ArduCAMs- 2MP SPI camera modules. So far, I produced two PCBs: one closer to the breadboard prototype and another using the ESP32-WROOM-32 module that includes some additional previously unsupported features. 

The image processing portion of the project uses the Python OpenCV library. The library supports both image-stitching and stereo correspondence but does not contain any state-of-the-art algorithms out of the box. For verification of some OpenCV results, I used the MATLAB image processing toolbox. While image stitching works well in test cases, generating stereo disparity maps proved more difficult. Thus, computer stereo vision and 3D environmental mapping are areas of future work.

Hardware Design

Overall, the hardware design is pretty simplistic. The Mini Pan-tilt system is from Adafruit and uses two servo motors with one controlling yaw and the other controlling pitch. While incapable of full 360-degree motion, the kit is sufficient during prototyping. The servos are controlled with pulse width modulation (PWM) from the microcontroller. Since the design is only concerned with panoramic pictures, the jerkiness of servo motion is irrelevant. If I work further on the project, building a more custom pan-tilt setup would be a good direction to investigate.

Electronics-wise, the design uses two ArduCAM SPI cameras and an ESP32 to handle processing and WiFi communications. The picture above shows a working prototype built on a breadboard that was used during testing.  The camera modules greatly simplify things: they improve the frame rate with an internal frame buffer, and allows easy addition of more cameras. Implementing these additions would have been a substantial effort worthy of an additional project. The interface for the cameras is a little unusual, it seems that commands to the camera are issued over I2C while the image data is sent over SPI. What's more, since both cameras have the same I2C address, I had to initialize an additional I2C bus on the ESP32. 

From this design, I created a pretty simple breakout board for the ESP32 and the other parts. Originally, I had issues interfacing with the cameras on the breadboard due to SI issues from the jumpers. The SI issues were solved when I replace the jumper with wire cut to the proper lengths. So, this PCB is mostly intended to fix any lingering SI issues and to stop any wobble of the cameras that might comprise the calibration...