Close
0%
0%

Lepton 3.5 Thermal Imaging Camera

Documenting my experiments with the FLIR Lepton 3.5 thermal imaging camera.

Similar projects worth following
I bought a FLIR Lepton 3.5 mounted on the Pure Engineering breakout after using a friend's thermal imaging camera to analyze heat generation on a PCB I designed. The Lepton 3.5 can output absolute temperature data (Radiometric) at 160x120 pixel resolution. I decided a thermal imaging camera would be a good tool to have and decided to build my own. I started with a Teensy 3.2 based test platform, turned to a Beaglebone Black with a short stop in Raspberry Pi land, and finally developed a set of thermal imaging cameras based around the ESP32. I hope the documentation in this project is helpful to others who might also want to play with these amazing devices.

My original long-term goal was to create a capable thermal imaging camera using the Beaglebone Black and a 7" LCD cape as the platform matching some of the features of high end commercial products.  However it soon became obvious that I'd need a simpler platform to learn how to use the Lepton module when I started reading the documentation and playing with the various demo codebases.  This is the story of my thermal imaging camera journey.

I read about a lot of other great projects online to get going.  Pure Engineering and Group Gets are to be commended for making these devices available to makers.  Max Ritter's DIY Thermocam and Damien Walsh's Leptonic are really well done and instructive.  Both Max and Damien were very gracious when I sent them various questions when I was getting going.

I ended up starting with a Teensy 3.2 based prototype with a Lepton breakout board, before moving on the Raspberry Pi, then the exciting world of PRUs in the Beaglebone SBCs before finally settling on the ESP32 for my final design (actually designs).

teensy_schematic.pdf

Test platform schematic

Adobe Portable Document Format - 26.67 kB - 07/10/2018 at 19:11

Preview
Download

View all 18 components

  • tCamViewer app in Google Play Store

    Dan Julio04/06/2023 at 13:04 0 comments

    I'm very excited to announce that the beta release of the Android tCamViewer app by Jim Turner is available in the Play Store.  There may still be a few issues here and there so please let Jim know via email on his website or log an issue at the github repository.

  • tCam kit available at Group Gets

    Dan Julio03/02/2023 at 22:30 0 comments

    Group Gets has created a store entry for a tCam kit.  You can find it here.  It includes

    • tCam-mini Rev4 (onboard antenna)
    • gCore (pre-flashed with tCam-firmware)
    • 3D Printed Enclosure

    • Hardware:
      • 4x M2x10 machine screws
      • 4x 4-20 x 1/4 screws
      • 8-Pin Female Header
      • 10-Pin Male Header
      • 2-Pin Jumper
    • Lithium Ion Battery 2000mAh

    You'll have to add a Lepton.  I recommend the Lepton 3.5 instead of the Lepton FS because you're guaranteed accurate radiometric (temperature) data from the 3.5.  But the FS is a little cheaper.

    You can also add a Micro-SD card for local image storage.  

    There's also a link to the video showing how to assemble the camera.  Instructions for use and links to all the supporting software in the github repository.

  • Android App Alpha (finally!!!!)

    Dan Julio02/24/2023 at 23:11 1 comment

    Edit 2/25/2023 - The developer decided getting the app into the Play Store alpha program was too complex and will be providing an APK for download.

    Summary

    An Android app capable of displaying images from the tCam cameras is ready for some alpha testing.   If you're interested please reach out to me with that, either in a message here on hackaday.io, or to my personal email address which you can find at my website or via my github page.  I'll get you on the list.  When the developer is ready he or I will notify you when it's ready to download and include instructions how to do that and how to provide feedback.

    Initial limitations

    The app is pretty full featured however it has a known issue when streaming.  The streaming stutters.  The developer knows about this and plans to rewrite the socket handling code to deal with this.  Other than that please submit feedback about other issues or features.

    TL;DR about the Android App

    People have asked for an Android App since tCam-Mini was first released and I don't blame them.  Lots of Android phones out there.  If you've ready back you might see that I originally wanted to use Xojo to take the iOS app I developed last year and release a version for Android when Xojo finally supported sockets on Android.  Xojo's Android effort is moving at a snail's pace and sadly I don't imagine that I can start that port anytime soon.  Unfortunately I don't have the time - and perhaps the high-level language expertise - to learn native Android app development.

    Last summer a software developer I had been working with on another project, Jim T, kindly offered to write an Android app.  It was his way to relearn Android development and he enjoyed playing with the tCam cameras.  It was much more effort than he originally imagined but to my amazement, he stuck with it in his spare time.  I have helped him test the app and given advice and suggestions and he is finally ready for other people to start testing. 

    So if you have a tCam-Mini or are building a tCam, I hope you'll consider playing with this app.

    Some more pics!

  • tCam Assembly video

    Dan Julio02/08/2023 at 22:37 0 comments

    Just finished a video showing how to assemble a tCam.

    Right now you have to buy parts separately (gCore from Group Gets or Tindie), tCam-Mini and the Lepton 3.5 from Group Gets and source the misc parts but hopefully soon there will be a full kit available from Group Gets.

  • tCam/tCam-Mini serial programmer utility

    Dan Julio01/30/2023 at 20:32 0 comments

    I wrote a simple utility program that runs on 64-bit Linux x86, OS X or Windows to make it easy to load the most recent firmware (downloaded from my website) for either tCam (gCore hardware) or tCam-Mini/tCam-POE.  I did this primarily to make it easy to program a gCore for those people who are building tCams but it can also be used to load the most current tCamMini firmware as well.

    It simply requires connecting the camera's USB serial port to the computer and programming.

    It can be downloaded from my website.  Look for the tCam Family Serial updater section.

    The application itself is pretty simple xojo.  It's able to program the ESP32 on the different platforms because it includes compiled versions (for each platform) of the Espressif esptool.py python program.  I just use xojo's shell object to run the binary version with the selected serial port and paths to the downloaded firmware binary files.

    I was led to this idea when I found that the Espressif ESP32 Arduino package included compiled versions for Windows and Mac OS X.  I dug around their github repositories until I found a script that uses pyinstaller to compile the python esptool.py.  I only had to create a binary for Linux x86 which I did with the following command after installing pyserial and pyinstaller.

    pyinstaller -F --add-data="esptool/targets/stub_flasher/*.json:esptool/targets/stub_flasher/" esptool.py

    This left a built executable called esptool in a dist directory.  Very cool.

  • tCam-POE design pushed to github

    Dan Julio01/22/2023 at 19:46 0 comments

    I finally had a moment to add the tCam-POE hardware design to the tCam github repository.   tCam-POE is basically tCam-Mini but with the ability to connect via ethernet instead of Wifi and also supports IEEE802.3af POE power.  No word yet if Group Gets is planning to build this.  Let them know if you would like to see it as a product.  :-)

  • tCam Video

    Dan Julio12/13/2022 at 04:06 0 comments

  • tCam! Finally (and a github change)

    Dan Julio11/06/2022 at 20:18 0 comments

    gCore is going to be available through a campaign at Group Gets.  This means tCam can also finally become a reality for anyone who would like to make one.  Finally.  It has felt, at times, that I'd never get to this point.

    There's a dedicated gCore github repository.  

    And I moved all the ESP32 thermal imaging stuff (tCam-Mini, apps, etc) to its own github repository so that people who are interested only in that don't have to download everything else I did with the Lepton.  The tCam firmware has been uploaded there as well as instructions for use and the enclosure designs.  The firmware binaries which must be loaded onto gCore and the enclosure designs can also be directly downloaded from my website.

    I realize that gCore might seem a bit pricey at the moment.  Neither I or Group Gets is making much money on this campaign.  It's really a factor of the parts cost (higher than some dev boards because gCore has a lot of high end features, and those take extra silicon), the manufacturing cost as Group Gets is building the boards with a USA contract manufacturer and the initial low volume of only 100 boards.  Even so I think it allows building a full feature, open-source thermal imaging camera for a reasonable price.  Over time I am working to bring this price down.  That will be discussed on the gCore project page.

  • Big Software Release

    Dan Julio08/08/2022 at 21:00 0 comments

    Hardware designs for gCore and tCam-POE have gone to Group Gets for quoting and buying.  I've noticed Mouser has a lot more parts recently so maybe the great partageddon is finally showing signs of easing.

    I also just pushed a bunch of new code up to github and my website.

    tCam-Mini Firmware Revision 3.0

    Big changes so this code base can also support tCam-POE.  I also added a mDNS (bonjour) responder so that the IP addresses of cameras on a network can be found by the various applications.  Rounding this release out were a few bug fixes plus I changed the internal flash access to use 80 MHz QIO accesses in preparation for some future firmware enhancements.

    Sadly the day after I made the release I found another minor bug.   A camera that has had its WiFi reconfigured to use STA mode with a static IP address may not correctly restart the WiFi stack until the next reboot.  Easy fix that will be rolled out in a bit but odd that I didn't see this before.

    A very kind soul got the previous firmware to compile on Espressif IDF 4.4.1 (instead of my ancient 4.0.2) and sent a pull request so I'll be trying to integrate his work for the next release too.

    Desktop Application 3.0.0

    A whole bunch of changes so I'm just copying the release notes here.

    1. New IsoTherm palette
    2. Added ability to change palettes by clicking the top or bottom of the color bar
    3. Added support to identify new Ethernet camera model
    4. Added Camera Info window (opened by new Camera->Camera Info... menu item)
    5. Application no longer displays firmware version of connected camera in main window
    6. Added new Discover function for bonjour discovery of cameras from Preferences Window
    7. Wifi/Network Settings tab understands Ethernet camera model and disables irrelevant controls
    8. Added Help window accessed through Application->Documentation menu
    9. Added ability to check for updated application automatically at startup or from Application Menu
    10. Added ability to check for new tcam/tCamMini firmware versions from Camera Menu based on connected camera type
    11. Enabled close button on About window
    12. Main Window toolbar connect button toggles between "Connect" and "Disconnect" captions
    13. Improved error message when application can't connect to camera
    14. Internal changes with window management in preparation for multi-camera sessions
    15. Signed Mac version with Apple developer ID

    Here's a screen shot of the new mDNS camera discovery in operation (accessed from the Preferences Window where  you set the camera address).

    Perhaps the neatest feature of the new version is the ability of the application to tell you if there is an updated version of itself or an updated version of the firmware for the attached camera and then download a zip file with the new code.

    iOS App

    The iOS app got the new IsoTherm palette as well as camera discovery.  The IsoTherm palette is great for quickly finding both the hot and cold spots in an image.

    Python Driver

    Last, but not least,  my friend bitreaper and I have been working on the Python driver to allow it to support the tCam-Mini Hardware Interface on a Raspberry Pi.  One of the things I've noticed over the years (and saw for myself) is that user space programs on the Pi have a hard time keeping up the the Lepton VoSPI pipeline.  Lots of users complaining in the forums about that. tCam-Mini with the python driver is a great solution to that problem.  In addition to communicating over a network socket the Python driver can now use the Pi's serial and SPI ports to communicate directly with a tCam-Mini.

    My very first tCam-Mini PCB got new use with wires soldered directly to the ESP32 module for testing.  Here it is on a Raspberry Pi 4 running the streamtest_hw.py demo and a full speed streaming image.

    The driver makes it easy to process data from the camera.  After setup the following two routines handle getting the radiometric data from the stream and converting it to an image to display using Tk.

    def convert(img):
    
        dimg = base64.b64decode(img[...
    Read more »

  • Introducing tCam-POE

    Dan Julio07/28/2022 at 22:01 0 comments

    I've been working on a new tCam model that helps deal with the limitations of WiFi in some situations, for example monitoring temperature inside metal cabinets or other industrial environments.  The tCam-POE design is based on the tCam-Mini design but also includes support for wired ethernet through the inclusion of an ethernet PHY chip and magnetics.  In fact it and tCam-Mini will run the same firmware starting with the upcoming version 3.0 release.  tCam-POE can connect using either 10/100 ethernet or WiFi (switch selectable at power up) and it can be powered via a USB-C 5V power source or, with the additional of a commonly available IEEE 802.3af DC-DC converter module, via power-over-ethernet.

    Shown here are my two prototypes sporting Winchen and Silvertel POE DC-DC converters.

    tCam-POE has the same command set as tCam-Mini with one exception.  The set_wifi command doesn't need to include WiFi-specific arguments such as SSID and Password and tCam-POE cannot be configured to serve DHCP addresses over ethernet (it still can act as an AP when running with the WiFi interface).  It 's ethernet interface either gets a DHCP served address from an external router or it is set with a static IPv4 address.  It also doesn't have the hardware interface that tCam-Mini Rev 4 PCB has.

    Porting the code to support it was pretty easy as Espressif has built-in support for an ethernet interface and after initialization the same network stack can be used for either WiFi or ethernet.

    Small changes are being made to the other apps so they can identify the new model number sent by tCam-POE.

    All code will be put up in the next couple of weeks.

    The plan is for Group Gets to build and stock these units.  Hopefully in the next 2-3 months - pending parts availability (I'm getting better about specifying parts with multiple sources...)

View all 53 project logs

Enjoy this project?

Share

Discussions

Nathan Sinclair wrote 02/01/2023 at 00:06 point

Hello Dan. having limited knowledge about sockets, I have a general query.  What is the reason for using ZeroMQ? From the little research I've done so far the message size for even 1 frame would be way too high... am I missing something? 

  Are you sure? yes | no

Dan Julio wrote 02/24/2023 at 22:26 point

Hi Nathan.  Sorry for the tardy reply - hackaday.io doesn't send me notifications even though I've turned them on.  I'm not sure which camera you are referring regarding ZeroMQ.  I did use it with the linux computers (Pi & Beaglebone) for experimentation because I wanted to interoperate with Damien Walsh's leptonic stuff which used it originally.  For the image data I was sending it seems to be efficient enough.

I use raw sockets for the ESP32-based tCam family of cameras and simply treat them as a full duplex communication channel between the camera and any application connecting to it.  There are special out-of-band start-of-packet and end-of-packet delimiters around the json data packets which make it very easy to always have the two ends synchronized and because it's all riding on top of TCP the communication channel is reliable.

  Are you sure? yes | no

sullivan wrote 08/09/2021 at 18:53 point

Hi Dan. Love the project. I have been playing with a Lepton 3.5 on the breakout board 2 and Raspberry Pi 4B. The thermal images I am getting are in the range of 8300, not the radiometric, T-linear mode I am expecting. Do you get images in the 30000 range where the formula: T(degC) = pixval/100 - 273.15? If you are also in the range of 8300, how are you converting to temperature? Thanks in advance for the advice.

  Are you sure? yes | no

Dan Julio wrote 10/06/2021 at 03:15 point

Hey Sullivan, I'm so sorry for this tardy reply.  For some reason I no longer seem to get notifications when someone posts here.  Are you sure you're switching the camera into radiometric mode with TLinear?  If so then what gain level is your lepton running at?  In High gain mode the resolution is 0.01° which means the data is in the 30000 range (e.g. 25°C -> count of 29815) but in Low gain mode the resolution is 0.1°C (25°C -> 2981).  In my cameras I set Radiometry, Radiometry TLinear, AGC Calcs all true.

  Are you sure? yes | no

leoshmu wrote 01/12/2021 at 03:23 point

Thank you for this excellent project. I am curious about pushing the limits of frame rate. Are you able to achieve a full 9Hz frame rate on the thermal video stream, or do you find that the frame rate is limited to less than 9Hz in practice. Could you imagine getting 2 leptons integrated together and offsetting the thermal stream so that they are off by 55ms, allowing for a combined 18hz signal? Perhaps something with a master clock circuit and then controlling each lepton separately based on the clock timing? Also I'm curious, why do you prefer the pure thermal board to the lepton v2 breakout board? 

  Are you sure? yes | no

Dan Julio wrote 02/04/2021 at 03:26 point

Hey Leo, thanks for the kind words.  I have been able to achieve the full 9Hz frame rate using the Beaglebone's PRU (in AGC mode) and with the ESP32 (both radiometric and AGC modes).  With the ESP32 I more-or-less dedicate one of the two CPUs to getting data out of the Lepton (and I use the VSYNC signal).  I will shortly be posting the code for tCam-Mini which is capable of reading the full frame rate (but not quite capable of sending that frame-rate over the WiFi).

Regarding your interesting idea of using two devices:  I'm not sure how to synchronize them.  I think there is a lot of internal processing that ends up affecting when the device is capable of sending a video packet - and things like a FFC also impact its ability to output a valid frame.  I suspect - but have no proof - that it is capable of a higher frame rate but held back in firmware (the sensor response time seems to be around 50 mSec).  I'm not up to it but I've wondered if someone could hack it.

I like the pure thermal board because it's smaller - I don't think there are any performance related differences.  Now, with tCam-Mini, I'm moving to directly mounting the sensor on my own boards.  

  Are you sure? yes | no

Ailsamly wrote 08/18/2020 at 10:30 point

Hi Dan,

I also plan to build a small thermal camera with a  suitable thermal sensor module, I want to connect it to a mobile platform and do building inspection (like water damage and air leakage and other thermal anomalies) .  The thermal sensor will be 5-10cm in front of the wall. I want to follow Max Ritter's project DIY-Thermocam. But I am not sure whether the resolution of Flir lepton can fulfull the requirements of mine. Do you have some ideas? Thank you!

Best,

Leyuan

  Are you sure? yes | no

Dan Julio wrote 12/26/2020 at 19:11 point

Hello Leyuan, I apologize for this tardy reply.  For some reason I never saw your post.  I think the Lepton should work from a distance of 5-10 cm.  However I can't answer your question about the resolution as I'm not sure what the characteristics of the things you are looking for are.  I can only suggest seeing if you could borrow a camera from someone and do some experimentation before committing to your own build.  Good luck though!  Sounds like a cool project.

  Are you sure? yes | no

Richard wrote 05/30/2020 at 16:15 point

Hi Dan,

According to Flir's datasheets, the Lepton uses 3.0V for IO functions, but the Teensy's on board Vreg runs at 3.3V. Did you do any level translation on the VOSPI and I2C interfaces or modify the Teensy to run at 3.0V, or just put 3.3V on the buses? I'm about to start a Lepton project of my own and want to be sure I don't kill my expensive sensor!

Thanks!

  Are you sure? yes | no

Dan Julio wrote 05/30/2020 at 16:49 point

Hey Richard,

You're right, they specify a 2.8-3.1 volt VDDIO.  But they also specify that the maximum on an IO pin is the lesser of VDDIO + 0.6 volts or 4.8 volts (table 18 - Absolute Maximum Ratings).  The Pure Thermal breakout boards have a 2.8 volt regulator feeding VDDIO so the maximum input voltage should be 3.4 volts.

To be honest I didn't pay attention to this spec when I first started.  I saw that the Pure Thermal boards were used with many different 3.3V SBCs so I jumped right in.  It wasn't until later I saw the specs you are referring to.

Although it doesn't seem like good engineering practice, I have run several Lepton 3.5 modules for long periods of time and this does not seem to change their functionality.  One could design their own breakout board for the Lepton and provide 3.1 volts to VDDIO which would lessen the over-voltage condition even more or provide resistor dividers on the SCLK and CSN signals (and pull SDA/SCL to the VDDIO rail).  As I work on what I hope is my final thermal imaging camera I am contemplating a full custom PCB (eliminating the breakout boards) and this is probably what I'll do, especially after your timely email :-)

  Are you sure? yes | no

Avi Cohen wrote 11/14/2019 at 11:45 point

Hi Dan, 

Is the code open to download?

  Are you sure? yes | no

Dan Julio wrote 11/14/2019 at 17:09 point

Hey Avi, 

yes, it's in the github repository.

https://github.com/danjulio/lepton

  Are you sure? yes | no

Sophi Kravitz wrote 07/10/2018 at 17:21 point

hi Dan! This is a cool project! This is happening pretty soon: https://hackaday.io/contest/159585-battery-powered-radiometric-lepton-contest

  Are you sure? yes | no

Dan Julio wrote 07/10/2018 at 18:09 point

Hi Sophi!  I saw that.  I'm not sure I'll have anything for the Lepton 2.5 but I will contribute a port of FLIR's LeptonSDKEmb32OEM library for the Teensy which might be applicable to the ESP32.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates