TJ - $99 Thermal Imager

On Thursday 27th, Seek Thermal launched a $199 thermal camera accessory for iOS and Android. It boasts an absolutely impressive 206 x 156 microbolometer array, -40 to 330°C temperature detection range and 36º FOV. Yeah, it puts the unborn $99 TJ to shame, as well as it's more expensive competitor, the $350 Flir One.

I must admit that the product launch timing came as a total surprise for me, even while Seek Thermal first surfaced in August declaring they've been hard at work for a couple of years already and that their camera product is in development; and I have a feeling that it came a bit surprising for the big players in the thermal imaging industry as well. What they've achieved (Seek Thermal in collaboration with Raytheon Vision Systems and Freescale) with the FPA design and manufacturing in terms of cost reduction is truly amazing!

Now, work on TJ and it's ancestor, MMTI was directed at making thermal imaging more affordable and more open to hobbyists and everyday people. But now as a $199 thermal imager is already here, and one that sets new price/performance standards, I think there would be little incentive to buy a $99 imager that's so far-off performance-wise. 

That being said, my motivation for continuing the development of TJ took a big hit :) Therefore, I've halted development for the time being and I'm the process of evaluating new development possibilities. I also feel there's little motivation left for fulfilling the requirements of the Hackaday Prize Finals.

I'll be abroad for a week but I'm planning to update the project with the latest unpublished developments after I return: updated hardware, firmware and iOS software sources.

TJ Gen2 Prototype status - as of this update

• TJ Gen2 PCB prototypes successfully tested (both the D6T-1616L populated version and the AMG8832 populated version)
  • minor changes needed (1 added capacitor, minor pin order changes on the 12-pin expansion connector to adapt  changes on the new v900 ESP8266-based module, microUSB connector placement)
• Added TJ firmware and iOS software support for the AMG8832 version also (in addition to D6T-1616L version support)
  • common TJ firmware for both TJ versions (TJ firmware detects the connected sensor automatically and reconfigures accordingly)
• Added firmware and iOS software support for the ESP8266 WiFi module 
  
  • ESP8266 module presence on TJ 12-pin expansion header is detected automatically and TJ reconfigures for wifi-streamming instead of audio-based communication
• iOS application UI changes
  • added temperature bar
  • tap pixel for temperature measurement
  • cleaned UI of test parameters
  • partially implemented gesture based manual ranging 

TJ Gen2 and modules / shields hardware prototypes; iOS App. 

After another two weeks of work evaluating and implementing different features / improvements / changes to TJ, I present you the TJ Gen2 prototype and shields: 

• TJ Gen2 thermal imager - baseboard (lower right corner) 
  • 37.5 x 20.5 mm
  • 2 versions depending on the included thermal sensor
    • Omron D6T-1616L (16x16 FPA)    --- $99
    • Panasonic AMG883x (8x8 FPA) --- $49? 
  • audio-jack interface (iOS devices)
  • USB interface (Android / PC / Mac)
  • 12-pin shield-connector
• BLE shield (upper left corner)
  • based on the popular $6 HM-10 module 
  • allows low-power remote connections to TJ
  • can be piggy-backed on the battery shield for battery-operation 
• WIFI module 
  • no-shield needed (simply plug a $4.5 ESP8266-based module in TJ's shield-connector)
  • allows longer-range remote connections to TJ
  • can be piggy-backed on the battery shield for battery-operation
• Battery shield
  • provides power to TJ through a standard 2-pin JST LiPo battery connector
  • charges LiPo batteries

aaaand... what's that on the lower left corner? It's a...

• Display shield
  • uses a $10, 1.5" OLED 128x128 color display
    • an <$4, 1.44" LCD 128x128 color display will also be evaluated
  • contains an uSD card slot for saving thermal images and video
  • 2 buttons
  • a LiPo charger and connector
  • Should allow TJ to become a stand-alone ultra low-cost thermal-camera :) 

The PCBs have just been ordered today at a China fab and should probably arrive late next-week.

TJ is all about making entry-level Thermal Imaging as open and affordable as possible. With that in mind, I've started on a journey of designing a device that's not only as inexpensive as possible but one that still squeezes the best available performance, convenience and flexibility, by means of clever hw/sw co-design :)

Affordability is paramount:

• The $99 version (using the D6T-1616L thermal sensor) will most probably offer the absolute best thermal imaging performance available at that price-point

• A $49-59 version (using the AMG883x thermal sensor) will also be offered for applications less-demanding performance-wise

Convenience is another aspect that I specifically targeted:

• You have an iOS device (iPhone, iPad, Ipod Touch) -> You simply plug TJ in your headphones jack and it's ready for use
• You have a device with an USB port (Android smartphone / tablet with USB OTG; Windows tablet, PC; Mac) -> You connect TJ to the USB port using a microUSB cable and it's ready for use

How about Flexibility:

• You want to connect remotely to TJ or want to connect to more than one device (think room occupation detection / people-counting applications etc.) 
  • TJ will have a small extension header that you can plug a 'dirt-cheap' $4.5 WIFI module like this ESP8266-based one (thanks Hackaday for the article on this module!)
    • just stack the WIFI module, apply power on TJ's USB connector / external powering pins or use a battery shield and it's ready to stream thermal data

• A Bluetooth Low-Energy shield based on the rather popular HM-10 $6 BLE module will also be designed and made available for low-range / low-power consumption interfacing

• You want to power TJ by battery?
  • A simple shield containing a LiPo charger IC, a power switch, an LED and a standard battery connector will be designed and made available 

• I'm hard at work designing an actual PCB prototype which will stack nicely under the D6T-1616L sensor PCB; This one will really be thumb-sized :)  ~40x20mm - roughly the same size as the D6T-1616L sensor PCB
• I'm planning on designing a simple, low cost, ultra-small WiFi shield ($25 is a target price); This would simply plug into the low-level 4pin expansion header and allow TJ to function autonomously (without the need of a smartphone / tablet / PC) and stream thermal data to a remote device (use cases: thermal monitoring in hazardous or special conditions, people detection, people counting etc.)
  • In addition to this, I was thinking of adding a feature to the smartphone / tablet app. that would allow to broadcast video and/or raw thermal data to a remote device (locally or over the Internet)   
• A re-design of the iOS app.  UX / UI is scheduled after the work on the PCB prototype is complete (I have some nice ideas on how to greatly improve usability)
• I'm considering the possibility of also offering a lower spec'd version of TJ, based on the Panasonic GridEYE 8x8 sensor, priced somewhere between $49 and $59;  Both versions (D6T1616L 16x16 @$99 and GridEYE 8x8 @$49-$59) would share a common PCB and the same components except for the thermal sensor and a few passives
  • Even upgrading at a later time would be possible for the ones with minimal soldering / de-soldering skills :)

What do you think? 

Here's a few quick facts about the TJ Gen1 Prototype:

• Communicates through the headphones jack using a custom modulation scheme that I've developed (5 Symbol, double polarity, Pulse Amplitude Modulation); It allows an effective bandwidth of 13714 bps (sufficient for sending the thermal data produced by the D6T-1616L uncompressed) and much more than the Hijack project modulation scheme limitations permitted - See Project Log 3 for details
• While the overall concept was inspired by the Hijack project and the TJ Gen0 prototype still used the Hijack iOS comm. library, the TJ Gen1 Prototype is an all-together different beast that bears no relation in hardware or software architecture to the Hijack project, furthermore boasting significant performance improvements in communication and power harvesting areas
• All electronic components of the TJ Gen1 Prototype (except for the Omron D6T-1616L - which is by far most the most costly component) amount for less than $4 (@1K quantities) with further cost savings possible
• The measurable temperature range that the D6T-1616L offers (or at least the engineering sample I've tested) is from -25°C (or even less) up to just shy of 300°C  

After countless frustrating hours spent trying to optimize my hardware / software implementation (of the Manchester Encoding scheme that the Hijack project proposed and that I've previously used in the TJ Gen0 prototype) in order to be able to increase the communication frequency (therefore data bandwidth), I've convinced myself I've hit a decisive hardware limitation. 

Long story short:

The D6T-1616L sensor that I'm using on the TJ Gen1 prototype produces ~12366 bps of data ( (16 x 16 pixels + 1) x ~12bits x 4FPS))  that TJ has to relay to the iOS device through the microphone input. 

To send that amount of data (without using thermal data compression, which I've decided would be the last resort), I'd need to increase the frequency at which the Manchester modulated data is sent through the mic. input from 8820 Hz to something like 14700 Hz or 16000 Hz (divisor of 44100 Hz or 48000 Hz sampling rate).

Put short, sparing you of details, neither of those frequencies worked not even with minimal reliability in spite of my efforts of accurately deriving the sampling clocking from the output sine wave used for power harvesting and using that to clock my output signal.

Bottom line is that Manchester encoding could only take me this far... :)

Then, an "AHA moment" struck me...  There's already a 5bit DAC in the micro controller I've chosen for the design, why not try to use a form of PAM (Pulse Amplitude Modulation) ???

This would allow me to keep the communication frequency low and still multiply the data bandwidth: WIN WIN :)

After another few days of painstaking testing I've arrived to a compromise between comm. frequency and different amplitude levels that seems to work best:

- 5 pairs of voltage levels (coding 2bits; 1 pair is for sync) clocked at 8000 Hz

Let's take a look at the following example:

The red test signal was captured (with a super-useful app. called SignalScope Pro) by the iOS device on the mic. input.

The encoding scheme is as following:

- there are '7 blocks' per data packet (1 sync block, 6 data blocks), so each data packet is delimited by a start and an end sync block

- on each block the positive and negative parts are driven identically in absolute amplitude (forming pairs) - this is to assure transitions on every bit (to pass AC coupling on mic. input) and because only the falling edge is measurable (the rising edge is completely useless  for measurement because that's where the transition between different voltage levels happens) - see the rising edge on the transition from 0b11 to 0b00

- there are 5 different voltage pairs (symbols), expressed in 5bit DAC input value (where 0 is 0V and 31 is 3.0V):

- SYNC symbol (HS = 16, LS = 15)

- '0' symbol (H0 = 18, L0 = 13)

- '1' symbol (H1 = 21, L1 = 10)

- '2' symbol (H2 = 25, L2 = 6)

- '3' symbol (H3 = 31, L3 = 0)

- The SYNC symbol encodes the sync blocks

- The '0', '1', '2' and '3' symbols encode the data blocks (each block is a 2bit value)

- 12bit values are encoded in 7-block data packets ( SYNC block, 2-bit data block 1, 2-bit data block 2, ... , 2-bit data block 6)

Decoding is done as following:

- The difference in amplitude on each falling edge of the signal is measured by the iOS app. (the bar graph on the right shows the differences measured on the left signal)

- The differences are decoded to symbols by certain thresholds that separate them (raw thresholds are hard-coded in source code but on each thermal data frame (4 times per second) there's a special data packet that the app uses to finely calibrate the thresholds; the differences are really stable over time but there seem to be variations from an iOS device model to another (e.g. iPad 3 to iPhone 5s)

- The detected symbols are transformed to a 12-bit value (by simply merging the 2-bit pairs) that represents the thermal measurement data of a pixel

The modulation scheme is currently used in the TJ Gen1 prototype and works 'flawlessly' (less than 1 corrupt data packet in  >100000 data packets...) !!!

It allows me a maximum practical data throughput of ~13714 bps, more than sufficient to send the ~12366 bps that the D6T-1616L produces.

More details on the TJ Gen1 prototype will be published towards the 20th of August :) 

Today I've received the D6T-1616L sample from Omron. Here's a quick snapshot from preliminary testing (looking at the same Parallella Gen1 board, for comparison).

I've just finished editing a video that compares the performance of the low-cost IR Arrays I've tested so far (Omron D6T-44L-06, Panasonic AMG8832, Melexis 90620ESF-BAD):

          * if you open the video on YouTube, there's a 'Table of Contents'  in the "Description" field and you can quickly jump to any part of the video

On the TJ Gen0 prototype:

- I've made a minor change to the Dickson Charge Pump topology that I use in the Power Harvesting circuit (audio output channels are not wired in parallel anymore but  as separate clocks phase-shifted 180° 

          ---> This results in much better performance of the Power Harvesting circuit

          ---> Less ripple on the regulated 3.3V

          ---> More margin to increase the current consumption if necessary 

- I've experimentally changed the clock source for the Manchester encoded signal (the one carrying thermal data, sent by TJ's microcontroller trough iDevice's Mic. input)

         ---> Signal is not clocked locally on the TJ anymore (by microcontroller's internal oscillator) but obtained through sampling the two 180° phase-shifted sine waves (to double the frequency) outputted through the audio channels

                - all sampling is done in hardware by the microcontroller (by a combination of internal comparators, S-R latch and T flip-flop)

                - this should result in much more precise clocking and alignment to iDevice's sampling -> hopefully allowing me to further increase the frequency (at which the thermal data is sent - currently 8820Hz) without communication errors  

I've refrained from posting more technical details such as hardware schematics and source-code as I'm still experimenting (with somewhat radical changes) in the pursuit of squeezing the best performance out of this Gen0 prototype before settling on a design for the Gen1 prototype.