Close

JPEG Decoding

A project log for The Slowest Video Player with 7-Colors!

Watch your favorite films at 1 frame per minute

Manuel TosoneManuel Tosone 02/17/2021 at 15:530 Comments

The frames to be displayed will be stored on an SD-Card as a sequence of jpeg files. Considering the time it takes to refresh the display and that I want to maximize battery life, I decided to update the display every 24 minutes, showing only one every 24 frames. This will make the battery last 24 times longer. The display resolution is 600x448 and, since only one every 24 frames must be stored on the SD-Card, the storage requirement is 144MB/hour (with 60% quality). An average length of 90 minutes movie can be stored in 200MB.

Time to implement a jpeg decoder and... I wrote my own from scratch. You see I want to learn something new with each project, this was the perfect opportunity for learning how jpegs work. The best resource that I can find on the subject is a video series entitled Everything You Need to Know About JPEG.

If you are not familiar with how jpeg works here is a brief overview. The first step in the encoding process is to convert the image from RGB to YCbCr, each color component is encoded independently. Next, each color component is divided into blocks of 8x8 pixels, each block goes through to the encoding process independently. The encoding starts with the forward DCT that converts the 8x8 block into a frequency domain representation. After that, the quantization step removes high-frequency data, and then the entropy coding is used to more efficiently store the coefficients.

JPEG Encoding/Decoding

The decoding process works the same way only backward, it starts with the compressed data stream, the first step is to undo the entropy coding then the coefficients are dequantized and, after that, the inverse DCT returns an 8x8 block of pixels. The final step is to convert back from the YCbCr color space to RGB. 

The tricky part is how to interpret the bytes in the file since to decode the compressed data more information is needed other than the compressed bitstream. To undo the entropy coding the decoder must have a copy of the Huffman table that has been used by the encoder and to dequantize, the decoder must have a copy of the quantization table. This information is encoded into the file and is separated by markers. Markers are two-byte codes that precede and identify a block of data.

The implementation of a decoder on a microcontroller poses some challenges, there is not enough RAM for a frame buffer and so the pixels must be sent to the display as they are decoded. Fortunately, the way 8x8 blocks from each color component are interleaved makes this straightforward.

The problem is that the pixels to the display must be sent in order from left to right top to bottom and the decoder produces 8x8 blocks of pixels instead. The solution is to have a small buffer, with the height of a block and the width of the screen. The decoder fills the buffer with pixels and, when it's full, it is sent to the display.

To make the images look good with only 7 colors, before sending the buffer to the display, the Floyd Steinberg dithering algorithm is applied.

The following video shows the display updating with some test images. It is noted that even if the display has only 7 colors, the dithering algorithm does a good job. If viewed from a couple of meters away the image looks almost perfect.

Discussions