Introduction

Thermal Imaging technology used to be priced prohibitively outside the budget of hobbyists until recent years. Late 2012 when I started working on my 1st Thermal Imaging project, the lowest cost thermal camera available on the market was the Flir i3 (priced at $995 back then). Moreover, there was (to my knowledge) absolutely no lower cost, open-source alternative solution on the market to these expensive, proprietary devices - that mostly had to do with the high development and manufacturing costs of high/medium performance IR FPAs and lenses and with the timid availability of low cost, low performance IR modules (FPA + lens + ASIC), still not implemented in any low-cost Thermal Imager.

I was therefore highly motivated to explore the possibility of developing a Thermal Imaging platform that was low cost ( < $200 to manufacture) and could offer sufficient performance for some of the less demanding hobbyist level tasks. That's how I've started working on Thermal Imaging devices.

A bit of background

1st attempt: Mechanically Microscanned Thermal Imager (MMTI)

I've started prototyping a device around the MLX90620BAD 16x4 IR array. The idea was to select a low-cost (implicitly low performance) IR FPA that had a low pixel FOV (2.5° x 2.6° FOV / pixel) and a satisfactory NETD (0.08°K at 1 fps in its datasheet; found out the hard way during testing that it was more like 0.4-0.5°K - still have no clue how they got to that figure).

Then, by means of a stepper-motor controlled pan-tilt platform (that had its rotation axes centered as well as possible to the FPA's surface center in order to simplify pixel position calculations) produce a pattern of controlled sub-pixel shifts (microscanning) of the whole IR module.

Each thermal data frame would be sequentially fed via WiFi to a connected iOS device that would form in realtime a composite high resolution image from the individual low-res frames (in the end a 176 x 44 final high resolution image would be obtained out of 121 frames of 16x4).

The prototype consisted of:

Melexis MLX90620BAD 16x4 IR array
PIC32MX795F512L based USB Starter Kit II + Starter Kit I/O Expansion Board
MRF24WG0MA Wi-Fi G PICtail Daughter Board
3-axis stepper control board based on 3 x A3977SED (self designed / manufactured board that I had lying around since my early 20's) - naturally, only 2 axis used
Vexta PK246PA stepper motors that I had lying around (1.8° / step - had to use 1/8 micro stepping to obtain 0.225° steps to fulfill my sub-pixel shifting resolution demands)
3S 1600mAh LiPo battery
cheap laser module (from a laser pointer) - to evaluate microscanning pattern accuracy and orientation
custom pan-tilt platform (self designed / CNC manufactured) - to allow precise centering of MLX90620BAD FPA surface center to the pan-tilt platform rotation axes

Here's an image of what the Hardware Prototype of MMTI looked like and to the right is the high-res composite image 176 x 44 (formed out of 121 frames of 16x4) - 'self portrait' - yes, I stood frozen for 121 seconds :)

Conclusions drawn from the MMTI prototype testing:

Successfully verified the validity of the microscanning concept of this particular setup - I was actually surprised by the rather good alignment of pixels and frames
Sufficient spatial resolution
Much higher NETD than expected (specified 0.08°K at 1 fps in MLX90620BAD datasheet; found out the hard way during testing that it was more like 0.4-0.5°K - still have no clue how they got to that figure)
High pixel non-uniformity (although all MLX90620 are factory calibrated to a certain degree of non-uniformity); I had to implement a pre-scan calibration to get to satisfactory uniformity
High measurement noise for MLX90620BAD (as sensor frame rate is increased, the noise increase too, rendering any frame rate higher than 4 FPS useless for this application)
Concluded that the primary performance...

TJ development takes a break while new development possibilities are being evaluated
Marius Popescu • 09/27/2014 at 16:48 • 2 comments
On Thursday 27th, Seek Thermal launched a $199 thermal camera accessory for iOS and Android. It boasts an absolutely impressive 206 x 156 microbolometer array, -40 to 330°C temperature detection range and 36º FOV. Yeah, it puts the unborn $99 TJ to shame, as well as it's more expensive competitor, the $350 Flir One.
I must admit that the product launch timing came as a total surprise for me, even while Seek Thermal first surfaced in August declaring they've been hard at work for a couple of years already and that their camera product is in development; and I have a feeling that it came a bit surprising for the big players in the thermal imaging industry as well. What they've achieved (Seek Thermal in collaboration with Raytheon Vision Systems and Freescale) with the FPA design and manufacturing in terms of cost reduction is truly amazing!
Now, work on TJ and it's ancestor, MMTI was directed at making thermal imaging more affordable and more open to hobbyists and everyday people. But now as a $199 thermal imager is already here, and one that sets new price/performance standards, I think there would be little incentive to buy a $99 imager that's so far-off performance-wise.
That being said, my motivation for continuing the development of TJ took a big hit :) Therefore, I've halted development for the time being and I'm the process of evaluating new development possibilities. I also feel there's little motivation left for fulfilling the requirements of the Hackaday Prize Finals.
I'll be abroad for a week but I'm planning to update the project with the latest unpublished developments after I return: updated hardware, firmware and iOS software sources.
TJ Gen2 Prototype status - as of this update
- TJ Gen2 PCB prototypes successfully tested (both the D6T-1616L populated version and the AMG8832 populated version)
  - minor changes needed (1 added capacitor, minor pin order changes on the 12-pin expansion connector to adapt changes on the new v900 ESP8266-based module, microUSB connector placement)
- Added TJ firmware and iOS software support for the AMG8832 version also (in addition to D6T-1616L version support)
  - common TJ firmware for both TJ versions (TJ firmware detects the connected sensor automatically and reconfigures accordingly)
- Added firmware and iOS software support for the ESP8266 WiFi module
  - ESP8266 module presence on TJ 12-pin expansion header is detected automatically and TJ reconfigures for wifi-streamming instead of audio-based communication
- iOS application UI changes
  - added temperature bar
  - tap pixel for temperature measurement
  - cleaned UI of test parameters
  - partially implemented gesture based manual ranging
TJ Gen2 and modules / shields hardware prototypes; iOS App.
PCBs ordered!
Marius Popescu • 09/09/2014 at 09:01 • 2 comments
After another two weeks of work evaluating and implementing different features / improvements / changes to TJ, I present you the TJ Gen2 prototype and shields:
- TJ Gen2 thermal imager - baseboard (lower right corner)
  - 37.5 x 20.5 mm
  - 2 versions depending on the included thermal sensor
    - Omron D6T-1616L (16x16 FPA) --- $99
    - Panasonic AMG883x (8x8 FPA) --- $49?
  - audio-jack interface (iOS devices)
  - USB interface (Android / PC / Mac)
  - 12-pin shield-connector
- BLE shield (upper left corner)
  - based on the popular $6 HM-10 module
  - allows low-power remote connections to TJ
  - can be piggy-backed on the battery shield for battery-operation
- WIFI module
  - no-shield needed (simply plug a $4.5 ESP8266-based module in TJ's shield-connector)
  - allows longer-range remote connections to TJ
  - can be piggy-backed on the battery shield for battery-operation
- Battery shield
  - provides power to TJ through a standard 2-pin JST LiPo battery connector
  - charges LiPo batteries
aaaand... what's that on the lower left corner? It's a...
- Display shield
  - uses a $10, 1.5" OLED 128x128 color display
    - an <$4, 1.44" LCD 128x128 color display will also be evaluated
  - contains an uSD card slot for saving thermal images and video
  - 2 buttons
  - a LiPo charger and connector
  - Should allow TJ to become a stand-alone ultra low-cost thermal-camera :)
The PCBs have just been ordered today at a China fab and should probably arrive late next-week.
TJ versions and 'shields'
Marius Popescu • 08/31/2014 at 17:00 • 0 comments
TJ is all about making entry-level Thermal Imaging as open and affordable as possible. With that in mind, I've started on a journey of designing a device that's not only as inexpensive as possible but one that still squeezes the best available performance, convenience and flexibility, by means of clever hw/sw co-design :)
Affordability is paramount:
- The $99 version (using the D6T-1616L thermal sensor) will most probably offer the absolute best thermal imaging performance available at that price-point
- A $49-59 version (using the AMG883x thermal sensor) will also be offered for applications less-demanding performance-wise
Convenience is another aspect that I specifically targeted:
- You have an iOS device (iPhone, iPad, Ipod Touch) -> You simply plug TJ in your headphones jack and it's ready for use
- You have a device with an USB port (Android smartphone / tablet with USB OTG; Windows tablet, PC; Mac) -> You connect TJ to the USB port using a microUSB cable and it's ready for use
How about Flexibility:
- You want to connect remotely to TJ or want to connect to more than one device (think room occupation detection / people-counting applications etc.)
  - TJ will have a small extension header that you can plug a 'dirt-cheap' $4.5 WIFI module like this ESP8266-based one (thanks Hackaday for the article on this module!)
    - just stack the WIFI module, apply power on TJ's USB connector / external powering pins or use a battery shield and it's ready to stream thermal data
- You want to power TJ by battery?
  - A simple shield containing a LiPo charger IC, a power switch, an LED and a standard battery connector will be designed and made available
Few words on current status and future plans
Marius Popescu • 08/26/2014 at 17:08 • 0 comments
- I'm hard at work designing an actual PCB prototype which will stack nicely under the D6T-1616L sensor PCB; This one will really be thumb-sized :) ~40x20mm - roughly the same size as the D6T-1616L sensor PCB
- I'm planning on designing a simple, low cost, ultra-small WiFi shield ($25 is a target price); This would simply plug into the low-level 4pin expansion header and allow TJ to function autonomously (without the need of a smartphone / tablet / PC) and stream thermal data to a remote device (use cases: thermal monitoring in hazardous or special conditions, people detection, people counting etc.)
  - In addition to this, I was thinking of adding a feature to the smartphone / tablet app. that would allow to broadcast video and/or raw thermal data to a remote device (locally or over the Internet)
- A re-design of the iOS app. UX / UI is scheduled after the work on the PCB prototype is complete (I have some nice ideas on how to greatly improve usability)
- I'm considering the possibility of also offering a lower spec'd version of TJ, based on the Panasonic GridEYE 8x8 sensor, priced somewhere between $49 and $59; Both versions (D6T1616L 16x16 @$99 and GridEYE 8x8 @$49-$59) would share a common PCB and the same components except for the thermal sensor and a few passives
  - Even upgrading at a later time would be possible for the ones with minimal soldering / de-soldering skills :)
What do you think?
A few quick facts about the TJ Gen1 Prototype
Marius Popescu • 08/20/2014 at 08:22 • 0 comments
Here's a few quick facts about the TJ Gen1 Prototype:
- Communicates through the headphones jack using a custom modulation scheme that I've developed (5 Symbol, double polarity, Pulse Amplitude Modulation); It allows an effective bandwidth of 13714 bps (sufficient for sending the thermal data produced by the D6T-1616L uncompressed) and much more than the Hijack project modulation scheme limitations permitted - See Project Log 3 for details
- While the overall concept was inspired by the Hijack project and the TJ Gen0 prototype still used the Hijack iOS comm. library, the TJ Gen1 Prototype is an all-together different beast that bears no relation in hardware or software architecture to the Hijack project, furthermore boasting significant performance improvements in communication and power harvesting areas
- All electronic components of the TJ Gen1 Prototype (except for the Omron D6T-1616L - which is by far most the most costly component) amount for less than $4 (@1K quantities) with further cost savings possible
- The measurable temperature range that the D6T-1616L offers (or at least the engineering sample I've tested) is from -25°C (or even less) up to just shy of 300°C
Rolling 'my own' Data Modulation Scheme!
Marius Popescu • 08/16/2014 at 20:39 • 0 comments

After countless frustrating hours spent trying to optimize my hardware / software implementation (of the Manchester Encoding scheme that the Hijack project proposed and that I've previously used in the TJ Gen0 prototype) in order to be able to increase the communication frequency (therefore data bandwidth), I've convinced myself I've hit a decisive hardware limitation.
Long story short:
The D6T-1616L sensor that I'm using on the TJ Gen1 prototype produces ~12366 bps of data ( (16 x 16 pixels + 1) x ~12bits x 4FPS)) that TJ has to relay to the iOS device through the microphone input.
To send that amount of data (without using thermal data compression, which I've decided would be the last resort), I'd need to increase the frequency at which the Manchester modulated data is sent through the mic. input from 8820 Hz to something like 14700 Hz or 16000 Hz (divisor of 44100 Hz or 48000 Hz sampling rate).
Put short, sparing you of details, neither of those frequencies worked not even with minimal reliability in spite of my efforts of accurately deriving the sampling clocking from the output sine wave used for power harvesting and using that to clock my output signal.
Bottom line is that Manchester encoding could only take me this far... :)
Then, an "AHA moment" struck me... There's already a 5bit DAC in the micro controller I've chosen for the design, why not try to use a form of PAM (Pulse Amplitude Modulation) ???
This would allow me to keep the communication frequency low and still multiply the data bandwidth: WIN WIN :)
After another few days of painstaking testing I've arrived to a compromise between comm. frequency and different amplitude levels that seems to work best:
- 5 pairs of voltage levels (coding 2bits; 1 pair is for sync) clocked at 8000 Hz
Let's take a look at the following example:
The red test signal was captured (with a super-useful app. called SignalScope Pro) by the iOS device on the mic. input.
The encoding scheme is as following:
- there are '7 blocks' per data packet (1 sync block, 6 data blocks), so each data packet is delimited by a start and an end sync block
- on each block the positive and negative parts are driven identically in absolute amplitude (forming pairs) - this is to assure transitions on every bit (to pass AC coupling on mic. input) and because only the falling edge is measurable (the rising edge is completely useless for measurement because that's where the transition between different voltage levels happens) - see the rising edge on the transition from 0b11 to 0b00
- there are 5 different voltage pairs (symbols), expressed in 5bit DAC input value (where 0 is 0V and 31 is 3.0V):
- SYNC symbol (HS = 16, LS = 15)
- '0' symbol (H0 = 18, L0 = 13)
- '1' symbol (H1 = 21, L1 = 10)
- '2' symbol (H2 = 25, L2 = 6)
- '3' symbol (H3 = 31, L3 = 0)
- The SYNC symbol encodes the sync blocks
- The '0', '1', '2' and '3' symbols encode the data blocks (each block is a 2bit value)
- 12bit values are encoded in 7-block data packets ( SYNC block, 2-bit data block 1, 2-bit data block 2, ... , 2-bit data block 6)
Decoding is done as following:
- The difference in amplitude on each falling edge of the signal is measured by the iOS app. (the bar graph on the right shows the differences measured on the left signal)
- The differences are decoded to symbols by certain thresholds that separate them (raw thresholds are hard-coded in source code but on each thermal data frame (4 times per second) there's a special data packet that the app uses to finely calibrate the thresholds; the differences are really stable over time but there seem to be variations from an iOS device model to another (e.g. iPad 3 to iPhone 5s)
- The detected symbols are transformed to a 12-bit value (by simply merging the 2-bit pairs) that represents the thermal measurement data of a pixel
The modulation scheme is currently used in the TJ Gen1 prototype and works 'flawlessly' (less than 1 corrupt data packet in >100000...
Read more »
D6T-1616L teaser :)
Marius Popescu • 08/08/2014 at 21:59 • 0 comments

Today I've received the D6T-1616L sample from Omron. Here's a quick snapshot from preliminary testing (looking at the same Parallella Gen1 board, for comparison).
Low-cost IR Array performance characterization
Marius Popescu • 07/27/2014 at 16:47 • 2 comments

I've just finished editing a video that compares the performance of the low-cost IR Arrays I've tested so far (Omron D6T-44L-06, Panasonic AMG8832, Melexis 90620ESF-BAD):
* if you open the video on YouTube, there's a 'Table of Contents' in the "Description" field and you can quickly jump to any part of the video

On the TJ Gen0 prototype:
- I've made a minor change to the Dickson Charge Pump topology that I use in the Power Harvesting circuit (audio output channels are not wired in parallel anymore but as separate clocks phase-shifted 180°
---> This results in much better performance of the Power Harvesting circuit
---> Less ripple on the regulated 3.3V
---> More margin to increase the current consumption if necessary
- I've experimentally changed the clock source for the Manchester encoded signal (the one carrying thermal data, sent by TJ's microcontroller trough iDevice's Mic. input)
---> Signal is not clocked locally on the TJ anymore (by microcontroller's internal oscillator) but obtained through sampling the two 180° phase-shifted sine waves (to double the frequency) outputted through the audio channels
- all sampling is done in hardware by the microcontroller (by a combination of internal comparators, S-R latch and T flip-flop)
- this should result in much more precise clocking and alignment to iDevice's sampling -> hopefully allowing me to further increase the frequency (at which the thermal data is sent - currently 8820Hz) without communication errors
I've refrained from posting more technical details such as hardware schematics and source-code as I'm still experimenting (with somewhat radical changes) in the pursuit of squeezing the best performance out of this Gen0 prototype before settling on a design for the Gen1 prototype.