Close

Lepton AGC

A project log for Lepton 3.5 Thermal Imaging Camera

Documenting my experiments with the FLIR Lepton 3.5 thermal imaging camera.

dan-julioDan Julio 07/18/2018 at 03:230 Comments

Most demo code, including my lepton_test6 sketch, use a simple linear transformation from the raw 14-bit count or 16-bit radiometric temperature data to 8-bit values that are then transformed to RGB values via a lookup table (colormap).  To do this the maximum delta between all pixels in the frame is computed and then each pixel value scaled as follows:

  8-bit pixel value = (Pixel value - minimum pixel value) * 255 / maximum delta

However this simple algorithm fails with scenes that contain both hot and cold regions because it tends to map most pixel values to either the maximum or minimum values resulting in images with little contrast between the two temperature extremes.

The Leptons have an AGC (Automatic Gain Control) function that can be switched on to try to ameliorate the shortcomings of a simple linear transformation for temperature data to be displayed.  FLIR has an entire - slightly confusing - section (3.6) in the engineering data sheet describing their AGC implementation - actually two different forms of AGC they call histogram equalization (HEQ) and linear histogram stretch.  They claim improvements over traditional histogram techniques with their HEQ algorithm and include many parameters to tune the algorithm but don't deeply describe it.  They don't describe their linear histogram stretch algorithm.

Enabling AGC mode (and disabling radiometric calculations - something that took me a few weeks to figure out) generates 8-bit data out of the selected AGC algorithm.  There are many parameters but they claim default values should produce good results.  There is an additional, somewhat mysterious, AGC Calculation State Enable as well.  The description states "This parameter controls the camera AGC calculations operations. If enabled, the current video histogram and AGC policy will be calculated for each input frame. If disabled, then no AGC calculations are performed and the current state of the ITT is preserved. For smooth AGC on /off operation, it is recommended to have this enabled. "


Since reading the AGC section I have been interested in how well their implementation performs.  I wrote a sketch, lepton_test8, that allows cycling between the original linear transformation and 4 variations of FLIR's AGC function.  They are

  1. "AGC disabled" - Radiometric mode enabled, my code linearly transforms 16-bit data to 8-bit data.
  2. "AGC linear C" - AGC mode enabled, linear histogram stretch mode, AGC Calculation State enabled.  Lepton outputs 8-bit data.
  3. "AGC HEQ C" - AGC mode enabled, histogram equalization (HEQ) mode, AGC Calculation State enabled.  Lepton outputs 8-bit data.
  4. "AGC linear" - AGC mode enabled, linear histogram stretch mode, AGC Calculation State disabled.  Lepton outputs 8-bit data.
  5. "AGC HEQ" - AGC mode enabled, histogram equalization (HEQ) mode, AGC Calculation State disabled.  Lepton outputs 8-bit data.


I then compared a scene with hot and cold components (soldering iron and ice-water with a room-temperature background) using the various modes and various color maps.  I also did some informal testing with less dynamic scenes.  The TL,DR summary is that the AGC modes generate distinctly better images in scenes with large dynamic range.  They do only slightly better with more mono-temperature scenes.  I couldn't see much difference between the linear histogram stretch and HEQ modes.  I also couldn't see much difference with the AGC Calculation state enabled or disabled (perhaps I saw artifacts left behind when the camera panned).

Following are some pictures showing output with different modes.  You can see the soldering iron and glass in the first image.  The mode and current colormap is shown at the top of the camera's LCD display.

A big difference with the "Iron Black" and "Golden" colormaps.


The advantages of the built-in AGC modes was less pronounced when the temperature range of the scene was less.  The soldering iron was removed for the first two sets of pictures.  It was replaced and the glass removed for the last set of images (however you can still see the cold spot on the desk where the glass sat).


Solder iron only.


Disadvantage of built-in AGC

FLIR touts the benefits of their algorithm for the display of temperature data.  My testing bore that out.  However because the AGC algorithms remove temperature data and scale the image to the full 8-bit range, they cannot be used if you want to display the current temperature scene inside of a larger temperature range.

One annoying artifact of scaling the Lepton's output to the full color map range is that the image will completely change when something with a higher- or lower-temperature is brought into the scene.  It can be handy to scale the data to a larger range so the image doesn't change so dramatically as you pan across scenes with different temperature components.  For example, assuming you were looking at the heat leakage of your house on a cool winter day, you might scale the data to a 0-40C range.  Then the image won't change radically as you pan across various hot and cold parts of the house.

Ideally, with enough processing power, one could take the 16-bit temperature data and run that through a software AGC algorithm that matches the performance of FLIR's algorithm but lets the user select the displayed temperature range.  Unfortunately that's beyond me at the moment.

Discussions