A project log for VR Camera: FPGA Stereoscopic 3D 360 Camera

Building a camera for 360-degree, stereoscopic 3D photos.

Colin PateColin Pate 09/18/2017 at 03:000 Comments

I've made quite a bit of progress since the last log, and run into a few road blocks on the way. Here's where I'm at right now.

Image Capturing

Using the OV5642 software application guide, I copied the register settings from the "Processor Raw" into my custom I2C master and used this to successfully get Bayer RGB data from the sensor! This outputs a 2592*1944 array of 10-bit pixels. In case you aren't familiar with Bayer image formatting, here's how it works:

On the actual image sensor, each pixel captures one color. Half of them are green, and the rest alternate between blue and red. To get the full RGB data for a pixel, the colors of the nearest pixels are averaged. Looking at a Bayer image like the ones I captured at first, it just looks like a grayscale photo. However, if you look closely, you can see the Bayer grid pattern. OpenCV includes a function called cvtColor that makes it incredibly easy to convert from Bayer to regular RGB.

Memory Writer

In order to take the streaming image data from the OV5642's Digital Video Port interface and write it to the Cyclone's DDR3 Avalon Memory-Mapped Slave, I had to create a custom memory writer in VHDL. I might have made it a little more complicated than I needed to, but it works fine and that's all that matters. I will upload it in the Files section as mem_writer.vhd.

Saving the Images in Linux

To read the images from DDR3 and save them to the MicroSD card, I re-purposed one of the OpenCV examples included on the LXDE Desktop image. I didn't feel like messing around with creating a Makefile, so I just copied the directory of the Houghlines example and changed the code to do what I wanted and rebuilt it. Here's the most important parts:

        int m_file_mem;
	m_file_mem = open( "/dev/mem", ( O_RDWR | O_SYNC ) );
	void *virtual_base;
	virtual_base = mmap( NULL, 0x4CE300, ( PROT_READ | PROT_WRITE ), MAP_SHARED, m_file_mem, 0x21000000 );
	if (virtual_base == MAP_FAILED){
		cout << "\nMap failed :(\n";

 I set up my OV5642 stream-to-DDR3 writer module to write to the memory address 0x21000000, and simply used the Linux command mmap() to get access to the DDR3 at this location. The size of this mapping, 0x4CE300, is simply the size of the imager (2592 * 1944 pixels).

Then, the OpenCV commands to save the image from DDR3 onto the MicroSD card are very simple.

Mat image = Mat(1944, 2592, CV_8UC1, virtual_base);
imwrite("output.bmp", image);

This doesn't include the Bayer-to-RGB conversion, which was a simple call to cvtColor.

In my next log, I will discuss the parts of the project that have gone less smoothly...