Project Logs

Collapse logs

• tCam Web prototype

  Dan Julio • 02/25/2021 at 20:10 • 0 comments

  One long-standing goal I have had for my thermal cameras was to record events such as a wildlife in our driveway at nights (we think it's pretty busy out there) by detecting parts of the image in certain temperature ranges.  Sort of like a very souped up PIR detector.  The cameras would act like a security webcam.

  I've taken the first steps toward this vision with a prototype of tcam_web, a recording web server for tCam cameras.  It's written in xojo (which has some limitations described below) and can run on OS X, Windows and either x86 or ARM7 Linux hosts (my ultimate server will be a RPi 4).

  It can connect with one or more tCam cameras via a socket interface and one or more browsers via its web server interface.

  It can display an image from a camera or be configured to record images that meet certain detection criteria.  To do this it enables the cameras into streaming mode and analyzes the images it receives.  Right now the detection criteria is very simple.  It will record an image if at least certain percentage of pixels are within a specified temperature range.  I did try, unsuccessfully, to create a fast blob detector and hope to return to that at some point.  The detection criteria, along with some parameters limiting how many images can be stored based on a detection event, the streaming interval and camera configuration may be setup through the web interface. 

  The program also allows viewing images it has stored.  I intend for this application to be used in conjunction with the desktop tcam application.  The desktop application can be used to analyze example images to determine the selection criteria for tcam_web as well as view and analyze images it has recorded.

  The xojo development environment allowed me - someone who is not skilled in web techologies at all - to write this application fairly quickly and reusing code  originally created for the desktop application.  Awesome!  Unfortunately while xojo is very mature and effective for writing desktop applications, its new "Web 2.0" API is still slow and a little buggy.   The way it renders may be good for controls but is not so good for images.  I had hoped to be able to stream images from the camera to the web browser but without learning a bunch of web stuff and digging into their Web SDK that doesn't seem possible.  It takes a significant fraction of a second (and, apparently a couple of communication trips back and forth between the browser and application) to load a single image.  So right now you can just manually load one image at a time.  Also the code is currently single threaded and I suspect to handle more than a couple of cameras I will have to break that task into its own thread or process using their worker class.

  Eventually I want the application to be able to email or text (via twillio probably) an image when it has a detection event.

  Once the application has been debugged for a bit I will post binaries for the different platforms to github.
  

  I also plan to marry a tCam-Mini with my Solar MPPT charger for deployment in our lower driveway as soon as some snow melts.  I'll have the tCam-Mini report the charger's status as its own battery status.

• tCam-Mini is a Group Gets campaign!

  Dan Julio • 02/18/2021 at 20:13 • 0 comments

  Interested in a tCam-Mini and don't want to wire one up yourself?  I've been working with the folks at Group Gets and tCam-Mini is a campaign!   If successful then they'll build a run of boards with their US-based CM for the backers.  If super successful then it might become an on-going product available in their store.
  
  Work continues on software support including a simple web server that can detect images with certain temperature ranges and record images and a python library to make interfacing with the tCam cameras easy.

• Protecting the Lepton Outdoors

  Dan Julio • 02/14/2021 at 00:20 • 0 comments

  I have been researching lenses for thermal imaging cameras recently because one goal is to put a camera outdoors in a weatherproof enclosure streaming to software looking for heat signatures of wildlife crossing in front of the camera.  But because traditional glass and plastic materials are opaque to LWIR radiation the thermal imaging camera senses, other materials must be used.  Expensive materials!  Searching for lenses made with the materials Flir recommends in the data sheet  such as Germanium and Gallium Arsenide return results that cost far more than the Lepton itself.  Many of these lenses also can magnify or change the focal length, capabilities that may not be needed for simple applications where all one wants to do is protect the sensor.

  Further research took me to the EEVblog forums and some of the incredibly smart members there.  They were discussing this stuff way back in 2013 and 2014 if not sooner.  There I learned that materials like Polyolefin (shrink-wrap) film and even common kitchen materials such as cling wrap or plastic sandwich bags are very transparent to LWIR.

  A very quick experiment was somewhat promising.  My "super scientific" setup is shown below.

  The camera can still take a good image but computes a different radiometric temperature.  I suspect reflection from the camera's own internal heat from the plastic bag.  The more expensive lenses have anti-reflective coatings.

  A single layer of cling wrap resulted in a slightly smaller difference, less than 4°F.

  Obviously one wouldn't want to use cling wrap to protect against flying debris but with some kind of shroud or other structure could be quite useful for an outdoor thermal camera.

  I didn't test but read that Flir sells Polyolefin protectors.  I plan to obtain and test some Polyolefin sheets as well, preshrinking them over a frame that could be mounted in front of the sensor.
  

  Finally, an interesting link I found while researching.
  

  https://infraredtraininginstitute.com/thermograpahy-information/
  

• Easier to build tCam-Mini

  Dan Julio • 02/07/2021 at 18:42 • 0 comments

  I found the TTGO V7 V1.4 ESP32 WROVER based development board long after I started tCam-Mini but it's a good way, along with a Lepton Breakout from Group Gets, to even more easily build a camera without having to make and load a PCB. The PSRAM is already part of the WROVER module simplifying the build considerably.  Just need to add a button, dual-color LED, pair of resistors and an electrolytic capacitor.

  Wiring is very straight forward.

  Two things to note about using the TTGO board. 
  

  1. It may have only 4 MB of Flash.  This is sufficient for the code but I use an 8 MB version on the PCB.  If you use the Espressif Programming tool to load precompiled firmware you will need to correctly specify the Flash size (see instructions in the "programming" directory on github).
  2. It may have an older revision ESP32 chip.  My TTGO board had a Rev 1 chip.  I specified a module for the PCB that has a Rev 3 chip and I built for that revision.  If the board won't boot (you can see an error message in the serial output) then you can try the version of the precompiled firmware I made for the Rev 1 chip also in the "firmware/precompiled" directory at the github repository.

• tCam-Mini and Desktop App pushed to GitHub

  Dan Julio • 02/04/2021 at 22:02 • 0 comments

  Big news today.  I pushed the design and code for tCam mini to the github repository  (new section: ESP32).

  I've also been busy putting the finishing touches on the companion desktop application and binaries for it are also available.   Lots of bug fixes and three useful new features.

  1. Support for video files comprised of multiple radiometric images from the camera.  The tCam stream function can now stream at different rates enabling time lapse thermography.
  2. Radiometric markers to monitor temperature in more than one location in a scene.
  3. Graphing of the spotmeter and radiometric markers over time.

  The menu bar on the main window also got a slight layout change so it looks better on Linux (the app runs on OS X, Windows and Linux).

  Graphs can be made from real-time streaming data coming from a camera or they can be made from a video file.  In this case from the file "pi0_5sec_heading.tmjsn" that was created from a streaming session with an image every 5 seconds of a Pi 0 mounted on my Pi Platter heating up and then cooling down.

  Finally I added a U.FL connector to my prototype board and tested with an external antenna.  It provides better performance (higher streaming rates).  I can now get between 6-7 frame-per-second which may be near the limit of the ESP32 WiFi/network stack.

• Software progress & Antenna questions

  Dan Julio • 01/17/2021 at 20:09 • 0 comments

  Some time over the holiday let me make quite a bit of progress with the tCam-mini firmware and desktop application software.  A little more testing and I'm ready to post the v1.0 tCam-mini firmware to github, along with the rev 1 hardware design.  Along with some cleanup the main change was to add the ability for the streaming enable command to include a mSec period between images and a maximum number of images.  This change was made to facilitate the upcoming graph function in the desktop application.

  The desktop application got a lot of work.  Aside from slight UI modifications and various bug fixes the main changes include

  • Support for radiometric data file vides and a new video file format (.tmjsn extension) that allows real-time playback of series of radiometric images.
  • Up to four radiometric markers allowing sampling the temperature in more places than just the spot meter alone.  These will also be used for the graphing function.
  • A more accurate frames-per-second readout for streaming and video playback.
  • Application and file icons
  • Filetype registration with the OS so double-clicking an image or video file opens the application and displays the file.
  • Drag and drop support.  Dragging an image or video file onto the application automatically opens it.

  I hope to be able to add a graphing function over the next couple of weeks that will complete the version 1.0 release of this application so that the temperature of multiple points can be plotted over time.
  

  I also did a lot of analysis of the firmware running on tCam-mini to try to understand why streaming performance varied so much.  I moved the Lepton readout task to be the only task running on the the APP (CPU 1) processor.  I also had to increase its priority but it is capable of keeping up with the Lepton's ~9 fps output rate with no problem.  The command processing and response tasks running alongside the WiFi stack on the PRO (CPU 0 processor do not require a lot of processing time.  The largest chunk of time is spent converting the binary radiometric and telemetry data to Base64 for inclusion the the json packet sent over the interface but that's only a few tens of milliseconds.

  The biggest variability comes from the system as it tries to send the data over the WiFi interface.  I see times from about 100-400 mSec per image (~55k).  This lead me to wonder about the antenna on the ESP32 module.  Sure enough things like position of the device in space and my finger touching the module made huge differences.

  The ESP32 hardware design guide has this to say about PCB layout.

  I had knowingly violated that spec with my layout to keep the board size down.  However chopping off the two little PCB wings designed to help mount the board had no effect - unlike my finger as can be seen in the almost double frame rate shown below.

  I am still at a loss but think that I will use an external antenna in the future connected via a U.FL connector on the ESP32 module.

  Up next is to try to finish version 1.0 of the desktop app, more testing and then upload everything to github in case anyone also wants to build a tCam-mini.  I will also have a couple of extra assembled PCBs that I'll either sell directly or on tindie.

• Lepton flat-field measurements

  Dan Julio • 12/26/2020 at 18:56 • 0 comments

  The histogram output lets us look a little at the Lepton 3.5's internal operation.  I put the sensor just above a flat black metal surface (emissivity approaching 1.0) that has a reasonably constant temperature (~21.9°C at the time of measurement).

  The default operation is Radiometry enabled - TLinear output so that every pixel value from the Lepton is an absolute temperature with resolution of 0.01 K.  After letting the sensor run for about 30 minutes the output looks like this.
  

  There appears to be a temperature gradient across the surface of the sensor.  Perhaps the outer areas are prone to absorb radiant heat being generated by the Lepton and the ESP32 module on the other side of the board.  I won't claim that this setup is incredibly accurate but the temperature measured by the thermistor on the surface is different than the temperature reported by the Lepton by more than the +/- 5°C accuracy claimed in the spec sheet.  At some point I'd like to make a temperature controlled black-body radiator for some more precise accuracy measurements.  This might allow additional calibration factors to be applied on a per-sensor/camera basis for improved accuracy.
  
  The software's data processing algorithm scales the radiometric data to 8-bits to simplify display through the various look-up tables.  This accounts for some of the quantized look of this histogram data as the 120 possible radiometric values for a 1.2°C delta between minimum and maximum pixel temperatures is being displayed in a 256 point field.  However it appears the Lepton's internal algorithm may be slightly reducing the resolution as we only see 60 bin values.
  
  The placement of the ESP32 behind the Lepton maybe a source of error in this design if it causes a thermal gradient to appear across the camera.  I'm not sure this is the case however because over the course of many images I have not seen a constant gradient and usually the coldest part of the image is more-or-less centered.
  

  Interesting, also, to see what effect AGC has on the output data.  This image was taken only seconds after the one above.
  

• tCam-Mini PCBs arrived!

  Dan Julio • 12/16/2020 at 18:10 • 0 comments

  Rev 1 of the PCBs arrived and seem to work great with one caveat.
  

  I built up two boards.  Both work, running the existing code, but one streams at a slower rate than the other.  With this design I moved to modules using the v3 ESP32 chip and even after recompiling for that version the slow-down on one module persisted.  I have noticed that one can get the FreeRTOS scheduler stuck in a pattern and I wonder if something like that is happening.  Or perhaps something with one module's wifi interface.  I'm going to add debugging code to see what part of the code is slowing down on one module.
  
  

  The board has the ability for the ESP32 to reset the Lepton and I'll add code for that and do some clean-up.  Then I should be able to post the code and hardware design on github.
  

  Probably also eventually rev the board to add a reset button (useful to restart while the serial monitor is attached) and tighten up the cutout for the LED to make assembly easier (or hopefully find a cheap side-looker RG SMT LED with lens to poke through a 3D printed enclosure (also have to design the enclosure...)

• tCam-Mini

  Dan Julio • 11/30/2020 at 15:54 • 0 comments

  A socially distanced Thanksgiving holiday finally gave me some time to work on tCam.  Specifically I finished getting a new version that I’ve been working in spurts for a while called tCam-Mini debugged and running.  I also put to bed some bugs and memory leaks in the companion desktop application.

  My current idea is that there will be at least two models of tCam.  The full tCam with a local LCD display and file storage and tCam-Mini designed to be a small, simple streaming thermal imaging camera.  My work with the full tCam (now ported to the Fire Cam hardware for access to its 4-bit Micro-SD interface) shows that the ESP32 probably lacks sufficient horsepower to simultaneously display, record and stream over wifi the full 8.7 frames-per-second from the Lepton.  I plan to use two ESP32 chips for the full tCam and this will be the subject for a future project log.  However one ESP32 is sufficient for a small camera that forgoes the LCD and local file storage.

  All tCam cameras will work with the desktop application.  It will grow to include the ability to graph the temperature of selected points in the image over time (I intend to use this feature to monitor temperatures on circuit boards).  I’ve also started to contemplate another computer application.  This one will be a recording web server, hopefully with the ability to recognize some thermal signatures so the cameras can be left running and pictures or videos recorded when “interesting” things happen.  Access will be from any web browser.

  tCam-Mini acts as an access point by default but can be put on another network using the utility.  It can return single pictures or stream images.  It is based on the code from the full tCam code-base although some of the code was optimized for operation as a streaming camera.  The code base has also been ported to the IDF 4.0.2 release.  Although I am getting the full frame rate from the Lepton in one task, I can only stream 5-6 frames-per-second over wifi from another.  I hope further debugging will improve that number.
  

  The prototype is based around a Sparkfun Thing dev board.  Initially I hoped that I could get by without extra RAM but found I couldn’t fit the necessary buffering in the limited on-board space so I bodged a PSRAM on top of the module.

  Over the holiday I also laid out a circuit board for tCam-Mini and plan to mate it with a 3D printed enclosure.  It uses a WROVER ESP32 module and forgoes the Lepton breakout board to include the Lepton circuitry directly on board.  Aside from the Lepton, single unit price for the components and PCB is about $20.  I hope to build up a unit in December.
  

  Ultimately - after further optimization and cleanup - tCam-Mini will be open sourced.  I may try to sell them on Tindie but pricing will be tricky as I currently pay around $200 for each Lepton.  That makes the camera fairly expensive with even only a moderate mark-up.  Research on quantity pricing for the Lepton is in order…

• tCam video

  Dan Julio • 06/24/2020 at 15:58 • 0 comments

  I entered tCam into a contest sponsored by Tech Briefs Media Group.  Here's a video made for the entry that shows the camera in operation.

View all 27 project logs