The basic job of this project is to take the video input from a camera, add a filter to it and then display it on the screen using a VGA cable. This basic job is divided into multiple small tasks. In the description of the design of this project I will first explain the tasks which will be divided into subtasks which the explained in the later paragraph.
The video input from the camera cannot be directly processed or used, it has to be converted to the right format. The camera is attached to the composite NTSC port which sends the video in analog form to the analog-to-digital converter, which converts it to digital format. This data is then sent to the Video-In decoder which converts the data to small packets of data which are called Avalon streaming packets. The Avalon streaming packets send data in a format such that each frame is divided into 720 columns and 244 rows which has 16 bits per pixel in 4:2:2 YCrCb color space. This data is then converted to data in a format where each frame is divided into 320 columns and 240 rows which has 16 bits per pixel in the RGB color space. This data is then sent to the pixel buffer direct-memory-access (DMA) core which writes the video stream into the SRAM/SSRAM. The data is read from the SRAM/SSRAM by the pixel buffer DMA core which sends the frames to the filter system where a filter is added to the video. This processed data is then sent to the VGA controller which sends timing and data signals to the VGA port to display the video on the screen.
I will now explain the data conversion processes mentioned in the previous paragraph. The data sent by the Video-In decoder has 720 columns and 244 rows for every frame with 16 bits per pixel in 4:2:2 YCrCb color space, but the pixel buffer DMA requires the data to have 720 columns and 244 rows for every frame with 16 bits per pixel RGB color space. This conversion is done in multiple steps. The Chroma Resampler converts the pixel from the 4:2:2 YCrCb to the 4:4:4 YCrCb formats, while maintaining the frame resolution of 720 x 244. This is then converted by the Color Space Converter into 24 bit RGB pixels with the frame resolution of 720 x 244. The RGB Resampler then converts this data to 16 bit RGB pixels with frame resolution of 720 x 244. The Clipper trims the stream from the 720 x 244 resolution to a 640 x 240 resolution by dropping the columns and rows around the exterior of the frame. Lastly, the Scaler reduces the stream to 320 x 240 by dropping every other pixel. This data is then saved in the SRAM/SSRAM. The Resampler core then converts this data from 16 bit RGB with frame resolution of 320 x 240 to 30 bit RGB with frame resolution of 320 x 240. Then finally the Scaler core replicates each pixel 4 times to get data in the format of 30 bit RGB pixels with frame resolution of 640 x 480 which is the only format accepted by the VGA controller.