Close
0%
0%

mlx90640 sensor works w 800 bytes

Everywhere i read on the web people are excited to get the mlx90640 working. i have examples that use less than 800bytes ram

Similar projects worth following
This is an offshoot off another project for a thermal cam pointer gun, there is interest both in the amg8833 sensor and the mlx90640. the mlx90640 is interesting because out of box it has physically 32x24 thermopile cells that detect temp. however it is a beast to setup and even sparkfun recommends more than 20k ram. the libraries included with it are ram intensive too. I decided to change that. for now, the examples i include read the internal settings , change mode settings and also can read the entire sensor (still working on adding calibration data)this should be enough for those wanting to try it out that only have arduino uno, or those that want to know how it functions.https://www.youtube.com/watch?v=LLbCHUhbZf4 example here https://github.com/jamesdanielv/thermal_cam_mlx90640<

this project is to show how to get mlx90640 working without first needing a powerful micro controller with more than 20k of ram. the idea is to have projects be able to be tested and workin on Arduino uno, and if more performance or features are needed then similar code can be used.

also i have code that is not include with melexis examples, such as update ram with sensor data and wait for mcu to initiate it. this allows ram to be read from sensor rather than needing to store it all in mcu, also reading of sensor calibration files data is done in example. however i do not use the calibration data yet. the goal is to have that data stored in mcu eeprom, and read when needed for calc.

MLX90640-Datasheet-Melexis.pdf

good bedtime reading of sensor specs and how to program it and get data from it. lots of floats, complex math and other things. don't blame me if your mind goes numb

Adobe Portable Document Format - 2.80 MB - 10/16/2018 at 21:45

Preview
Download

MLX90640 driver.pdf

this doc makes more sense, regrettably they didn't include commands that change device into step mode over continuous. my example programs show you how here. https://github.com/jamesdanielv/thermal_cam_mlx90640

Adobe Portable Document Format - 900.11 kB - 10/16/2018 at 21:45

Preview
Download

getEEpromDump.zip

this file extracts eeprom calibration data from mlx90460 chip, sends it to terminal where it can be loaded into progmem of Arduino. instructions will be avail on how to do this.

Zip Archive - 383.35 kB - 09/29/2018 at 08:49

Download

  • optimizing temp readings of sensors. reading one line at a time a lot faster because of i2c overhead

    jamesdanielv12/16/2018 at 14:49 0 comments

    i2c overhead is a lot per request of data from mlx90640 sensor

    here is i2c protocol

    from what i understand, is that after address is sent to sensor, and read mode sent, as long as no stop command is issued, each data space is read back one byte at a time and  increments the address after each data byte transferred. 

    this means that requesting more than one byte at a time is how i can make more efficient use of the i2c bus, also it will work up to 800khz on arduino. 

    examples of efficiency are here

    i need to request at least 10 memory addresses of data for each thermal pixel

    overhead with no bytes is 3140 microseconds. i will work on reducing this. but it shows over head for requests

    requesting 1 at a time is with total processing time of 4900-3140= 1760 microseconds per cell read

    requesting 8 at a time is with total processing time of 9448-3140=6408/8= 801 microseconds

    requesting 32 at a time is 28132-3140 =25092/32= 784 microseconds per cell read time

    time of return diminishes after about 8 bytes at a time, 

    for example using an atmega chip with 8k of ram, 

    and reading all 768 cells for thermal data at a time 

    requesting 768 604568-3040=601528/768=783 microseconds

    so reading up to 32 at a time for connivence is best, but 8 at a time is reasonable as well with reading only taking 801 microseconds.

    there are several tricks to improve performance, this just shows that optimum reading of data peaks at about 32 sensors reading at a time. this is about 640 bytes of data and does not include calculations time, you can see that that overhead has been removed from calculations. 

    there are some efficiencies in the fact that at least 70% of calculations are applied to all sensor, and 20% are applied to at least 2 or 4 cells for calibration. leaving only 10% of math that needs to truly be done independently, and some of that is for noise and hysteresis.

    i guess what im getting at is that 5000 microseconds read time per cell will be quickly reduced to at least half that, or more. it could be as low as 500microseconds per cell.

    and the big kicker, we only need 3-5 cell reads to a tempurature, rest of data can be raw values without the To calculations to show color changes, we only need 3-5 temp readings for a reference so we can figure out color range for displaying images, and show temp values of high, low, and center.

  • array of 768 sensors seems to work with temp in celsius and calibrated

    jamesdanielv12/09/2018 at 19:30 0 comments

    here is a 32x24 sensor read for terminal. seems to be calibrated, and using 808 bytes of ram and working on Arduino!

    https://github.com/jamesdanielv/thermal_cam_mlx90640/blob/master/32x24calibratedsensorRead.zip

    not all rows fit on screen but you can see the degrees in . currently each sensor read routine is about 5000 microseconds. this is far from optimized yet, it just seems to be working.

    temp seem to be calibrated, will test and put it into a color map to verify.

    the only wrench for this is i seem to only get these values when i have sensor run in continuous mode, not single shot mode. this causes a small error in gain calculations because vdd varies +/- 0.01 v between reads.

    i will try to resolve why, because some of these measurements are a frame behind the others, however the rate of change for sensor may not mean it makes much difference, especially with multiple samples to reduce noise, but i will try to fix it either by getting single shot mode fixed (possible bit setting) or have code that figures out how old sensor data is and apply correct Vdd sample to calculate temp with more precision. the example i list above is with 19 bits of resolution so in theory with a resolution of 

    -40°C-300°C ~340 range, and 524288 (19bit) levels sensor is capable of sensing difference of 0.000648 degrees. amg8833 is 0.025 degrees. the noise suppression is superior in the mlx90640 sensor making it excellent for thermal imaging at a distance! 

  • mlx90640 sensor reads degrees on Arduino! uses 852 bytes ram.

    jamesdanielv12/09/2018 at 17:57 0 comments

    https://github.com/jamesdanielv/thermal_cam_mlx90640/blob/master/MLX906040_1tempcell.zip

    this is a test code setup that reads degrees c on arduino. sketch uses about 20k , and it uses 852 bytes of ram. code is not optimized or shrunk in size yet. it is just working. next i will get an array of sensor data to be output to a ram buffer so it will be compatible with my amg8833 code. the goal is to pretty much allow my drivers to replace the amg8833 code as a drop in replacement and build in features that allow more resolution detail. 

    a few things. this code is only a test code and it only converts 1 cell to a degree in temp, but it can do it for any cell.

    CalculateTo(12,10); for example stores temp data in float value To, you can change CalculateTo(x,y) to any pixel value from 0,0 to 32,24 the full resolution of sensor.

    CalculateTo does not return value yet, it is stored in To

    there may be a higher amount of noise from this method as it reads and stores

    1 cell at a time. this can be corrected in 64 samples per second mode, by taking 3 samples and averaging them over a short time. 

    don't worry i will reduce noise, and fix an errors eventually. this is just an update for those that want to try sensor on Arduino.

    next code will have settings to change from 1cell to reading an array of cells for example code.

  • extract gain values done as well using few bytes

    jamesdanielv12/03/2018 at 08:44 0 comments

    gain was easier, and it is across all pixels evenly. 

    i use a global variable SensorGaincommon to store value in it is a float.

    void  ExtractGain(){
    worddata[0]= pgm_read_word_near(factoryCalData+0x0030);//we can get this value from eeprom, faster and more reliably

    if (worddata[0]> 32767){worddata[0]=worddata[0]-65536;}//per document 11.2.2.4

      SensorGaincommon=worddata[0];
      MLX90640_I2CRead(MLX90640_address, 0x070A, 1, worddata);//we get from ram

     if ( worddata[0]> 32767){worddata[0]=worddata[0]-65536;}
      SensorGaincommon=  worddata[0]/SensorGaincommon;
    }

    from terminal image you can see gain is calculated as well. this seems to not change much so i would say it needs to be read at least 1 time after changing analog resolution detail.

    i have opted to just show 8x8 sensors (even though 32x24 is possible) using the middle of display to make finding issues and moving drivers into thermal cam project easier. after testing successfull will start making 32x24 work, probably by 16x16,24x24, and then 32x32 using an advanced interpolation method.) 

    here is the math. this is same for all pixels. it also does not need to be updated that much

  • i now have ambient temperature of silicone measurements

    jamesdanielv12/03/2018 at 07:37 0 comments

    i have the ambient temperature of silicone measurements. this is important for calibration of gain and using difference to cal a individual cells temp. so i'm getting closer. below i show code used, and an image of output to terminal. this is an important step to having sensor work on arduino. 

    i will also show the math required to get it working. i don't know why they made it so complex. it didn't need to be. i did also reference Melexis driver code to speed things along, so i'm grateful that they did document it. the Melexis code however uses heavy use of ram, and there is little reason to do so. most of the values are from calibration and several values are fixed once calculated. some calc and conversions are done on the compiler side and never done on Arduino.

    to get ambient temp of silicone these values are needed to be calculated

    KVPTAT

    KTPTAT

    VPTAT25

    ALPHAPTAT

    ptat

    ptatArt

    Vdd (we use the stored value we have for this already)

    the sensor assumes its base temp to be +/- from 25 deg C so offsets to calculate it are from there.

    here is my ruff code. i try to use reads from stored data from flash whenever possible but this code needed several reads. good news is that this code can run every frame or only twice per full scan of sensors.

    void ExtractAmbientTemp()
    {

    worddata[0]= pgm_read_word_near(factoryCalData+0x0032);//we can get this value from eeprom, faster and more reliably

       worddata[0]=worddata[0] & 0xFC00;// as per 11.2.2.3
       KVPTAT=worddata[0]>>10;
       if (KVPTAT<31) {KVPTAT-=64;}   
       KVPTAT = KVPTAT/4096;
       
    worddata[0]= pgm_read_word_near(factoryCalData+0x0032);//we can get this value from eeprom, faster and more reliably

        KTPTAT = worddata[0]& 0x03FF;
        if(KTPTAT > 511)
        {
            KTPTAT = KTPTAT - 1024;
        }
        KTPTAT= KTPTAT/8;

    VPTAT25= pgm_read_word_near(factoryCalData+0x0031);//we can get this value from eeprom, faster and more reliably
      

    worddata[0]= pgm_read_word_near(factoryCalData+0x0010);//we can get this value from eeprom, faster and more reliably

        ALPHAPTAT=(worddata[0] & 0xF000)/pow(2, (double)14) + 8.0f;//as per documentation. wil simplify later on
        float ptat;
        float ptatArt;
        float vdd;
        float ta;   
        vdd = Vdd;//Vdd has already been calculated or should have been
        MLX90640_I2CRead(MLX90640_address, 0x0720, 1, worddata);//we read register memory
        ptat= worddata[0];
        if (ptat> 32767 ){ptat = ptat - 65536;}//documented method
        
        MLX90640_I2CRead(MLX90640_address, 0x0700, 1, worddata);//we read register memory
        ptatArt =worddata[0];
       if(ptatArt > 32767){ptatArt = ptatArt - 65536;}
           ptatArt = (ptat / (ptat * ALPHAPTAT + ptatArt)) * pow(2, 18);
        
        ta = (ptatArt / (1 + KVPTAT * (vdd - 3.3)) - VPTAT25);

        
        ta = ta / KTPTAT + 25;
        AmbientTemp=ta;
    }

    and the math

  • voltage calibration of sensor working.

    jamesdanielv12/02/2018 at 11:58 0 comments

    here is the calibration routing i am using for the mlx90640 sensor

    i have already stored values in eeprom, and only need to read ram value of voltage to sensors 1 time per pass. 2 passes to read all sensors. so two voltage values.

    this code extracts the kVdd, and the Vdd25 which are stored values, and then applies calculations to get the VDD at time of sensor read. i use a 2 byte word worddata to get data when i need it from sensor. i'm trying to reduce the reads from the chip and use the stored values that are copied to eprom flash of arduino

    void ExtractVDDParameters()
    {
       worddata[0]= pgm_read_word_near(factoryCalData+0x0033);//we can get this value from eeprom, faster and more reliably
        kVdd =worddata[0];
        kVdd =kVdd & 0xFF00;//we do some required conversion (11.2.2.2 in document)
        kVdd =kVdd >>8 ;//we divide by 2^8 
        if (kVdd >127){kVdd -=256;}//if >127 we subtract 256 as per document
        kVdd =kVdd *32 ;//we now multiply by 32    
        Vdd25=worddata[0];
        Vdd25=Vdd25 & 0x00FF;
        Vdd25=  ((Vdd25 -256)<<5)-8192;
        //we need to get this from ram on sensor because it changes all the time!
     MLX90640_I2CRead(MLX90640_address, 0x072A, 1, worddata);//we get vdd used real time from senso
        Vdd=worddata[0];
        if (Vdd>32767 ){Vdd=Vdd-65536;}//we do this to normalize the value
    uint16_t storedRes= pgm_read_word_near(factoryCalData+56);//we can get this value from eeprom, faster and more reliably
     storedRes= storedRes& 0x3000;
      storedRes= storedRes>>12;
    //we need analog resolution
     MLX90640_I2CRead(MLX90640_address, 0x800D, 1, worddata);//we read register memory
     worddata[0]=(worddata[0]& 0x0C00)>>10;//we get ad resolution detail
        
    float resolutionfix;
    if (worddata[0]==0){resolutionfix=4.0;}
    if (worddata[0]==1){resolutionfix=2.0;}
    if (worddata[0]==2){resolutionfix=1.0;}
    if (worddata[0]==3){resolutionfix=0.5;}
    Vdd= (resolutionfix* Vdd - Vdd25) / kVdd + 3.3;
    }

    here are the results in terminal. for current troubleshooting i am only using center 8x8 sensors (even though it is capable of 32x24). you can see the extracted vdd25 and kvdd values. also note that for each frame the voltage is different slightly. this will effect gain and proper reading if both values are not used.

    the sensor reads currently are raw values and are currently meaningless except to know that they change values.

    here is the calc references from documentation. seems simple but a lot of troubleshooting. also i found a better way to calc the different resolutions. at least one that does not need POW

    now on to the next set of needed calc for getting temp from sensor. 

    here are the calculations in all that i need to work on getting coded for Arduino to work, now that i know the format and the offsets for eeprom data, and how the register on this product works, hopefully things will go quickly.

  • added mirroring feature to mlx90640

    jamesdanielv11/08/2018 at 17:17 0 comments

    i'm now testing the mlx90640 for use as a thermal cam sensor. first thing i notices when i mounted it was that the image i saw in terminal needed to be inverted. so this feature has been added to test file setup and can be downloaded here: 

    https://github.com/jamesdanielv/thermal_cam_mlx90640/blob/master/fullsensorreadToterminal.zip

    it is a simple setting

    //#define MLX90640_mirror false

    the code for it is almost as simple as we just reverse order memory is read from single 32 word line buffer

       for (int i=0; i<24;i++){//this samples and draws entire page of data. sensor is 
      MLX90640_I2CRead(MLX90640_address,  0x0400+32*i,  32, mydata);//read 32 places in memory
      #if MLX90640_mirror == false 
      for (int x = 0 ; x < 32 ; x+=1) //this is normal sensor mode
      #else 
      for (int x = 32 ; x >0 ; x-=1) //this is mirrored sensor mode
      #endif
      { //we compare active one row to see at output
        int testx=(50+mydata[x]-startvalue[x]);
        char outputx=46;
        if (testx>60){outputx=44;}//+
        if (testx>70){outputx=111;}//o
        if (testx>80){outputx=79;}//O
        if (testx>100){outputx=64;}//@
        Serial.print  (outputx);Serial.print(F("  "));
      }

  • how to buffer data for display writes from more complex sensors such as mlx90460 or flir 80x60

    jamesdanielv10/19/2018 at 05:01 0 comments

    i have several methods to increase Arduino performance. i use non float interpolation. I use spi bursting, i sometimes used half pixel processing to allow every other line on big displays, and i use cell buffering to reduce changes to screen 2x-5x, and finally i control data bandwidth to allow most changed pixels priority because noise is also a factor otherwise. here is how i'm thinking some of this will play into the mlx90460

    One of the ways i'm getting good performance from Arduino (in many cases on par with arm processors) is that i incorporate buffering at the level of the analog world. the input side, not the display resolution. we only write areas on the display that change, and when a lot is going on, only the areas that change the most because analog is noisy. to keep performance on a small memory footprint, we need to think of areas on screen as cells. this is similar to though processes of jpg imaging, and mpeg encoding. look for areas that change. however we will look at areas as a group of cells. 

    on amg8833 we use 2 bytes for each of 64 cells. but we only see a range of 1024 values.

    the code i use for that sensor is if values are different in cell area we write cell. now with the more complex mlx90460 sensor we have 32x24 sensors. we still want to break up cells data into 8x8 areas for buffering, so what we do is add all 12 cells together, and store a 2 byte number of those values per cell. if number is different it will be updated on screen. this allows us to use the same buffer space as before, only difference is 12 numbers are higher. adding numbers together doesn't use much processing power and comparing number values with stored values is very low cost to code time. we are still doing the same thing looking for changes at input. 

    on another note, we will not be sub sampling as far up to get more detail with this sensor, so there may actually be less processing time to scale data. my goal with this sensor is to have it working at least at 64x48, and then at 128x96 resolution. and possibly higher.  this only requires 2 levels of upscale which is very easy to do. (or so i think so far).

  • it is less work than i though to get mlx90460 sensor usable for thermal cam.

    jamesdanielv10/16/2018 at 21:44 0 comments

    i've been trying to get the energy and time to convert all the commands to run on Arduino. the evil scientists at Melexis want you to purchase supercomputers to process float thermopile calculations (or at least a high end arm cortex). i think they did this as a joke. well i brought the bag to the doorstep, and lit fire a few weeks ago  (just watched Billy Madison) with my no float test setup here:

    https://github.com/jamesdanielv/thermal_cam_mlx90640

    in data sheets for mlx90460 it states:

    13.2. Using the device in “image mode”

    In some applications may not be necessary to calculate the temperature but rather to have just and image (for instance in machine vision systems). In this case it is not necessary to carry out all calculations which would save computation time or allow the one to use weaker CPU.

    In order to get thermal image only following computation flow is to be used:

    Supply voltage value calculation (common for all pixels) - 11.2.2.2 

    this means for now as a start all i would need to convert are the now epprom saved results from factory, and gather the latest voltage to chip readings from sensor for frame1, and frame 2.

    the process is this. capture 1st frame, store voltage to sensor, capture second frame, store voltages to sensor, read 32 columns, display results, read 24 rows down. and by the way. the memory can be read over and over again, as long as the sensor is in step mode, and you haven't ask it to sample sensors again. by the way i had to create my own command to do this, as melexis examples did not have a way to do it out of box. if i'm wrong about it let me know. one of the status registers is below. 

    eeprom results no longer need to be in ram and can be put into Arduino thru a table cut and paste with program i created here: 

    https://github.com/jamesdanielv/thermal_cam_mlx90640/blob/master/getEEpromDump.zip

    and the following calculations below:

    i'll upload the complete spec sheet into the files section of this project.

    this looks scary, but it is really just AND compare and shift rights and a few bits of subtraction and manipulation of resolution 8bits to left for simulation of what would be float. Ok.. it is scary, but i've got my big boy pants on this week! :)

    once this is tested, it will be added to setup files i have for the mlx90460 here:

    https://github.com/jamesdanielv/thermal_cam_mlx90640

  • added ability for changing ad sensitivity, hz and single shot mode

    jamesdanielv09/30/2018 at 06:38 0 comments

    https://github.com/jamesdanielv/thermal_cam_mlx90640

    i modified the code to the files in the fullsensorreadToterminal file. now the file has settings at top that will be checked and if different rewritten into the sensor.

    //below lines change important settings of sensor
    #define continuousmode true //true is default when sensor is bought, however we want step mode. this is checked by the code and only written to if it is different than value here. it is here for experimentation.
    #define hzMode 2//0=0.5hz,1=1hz,2=2hz,3=4hz,4=8hz,5=16hz,6=32hz,7=64hz 
    #define adSensorResolution 2 //0=16bit,it 1=17bit, 2=18bit, 3=19b

    the sketch now can store the calibration data of the sensor in progmem. this will be useful after i setup to extract the calibration values. currently the flash has my sensor settings in it, but you can load yours by first installing the included sketch getEEpromDump. and in terminal copy the output (command c, or cntrl c) after highlighting everything it asks you to, then pasting into fullsensorreadToterminal sketch.

    the code in sketch reads the sensor data and compares to what it finds in progmem to the data on sensor, if it is not a match it will throw out errors at you, but it will allow you to process currently.

    after that it checks the hz setting and the analog resolution presets as well as the step or continuous scan mode. if settings are different on sensor it writes the settings in firmware sketch and verifites that they match after a quick reboot of Arduino.

    I would like to add that adafruit and melexis do not include code for switching camera modes, so i'm excited to have it included in this sketch. next update will have things hidden away into functions and some simpler commands.

    anyway currently using 9344 bytes flash and 630 bytes ram. flash can shrink about 1-2k , bytes can go down about 50-100 with placing new features into functions, and reducing the debug flash usage.

    after this small clean up, ill begin making use of the calibration data. i don't expect much more ram usage but the sketch might go up to 14-15k short term.

    these are the functions i need to reimplement for calibration to work on arduino:

    void ExtractVDDParameters(uint16_t *eeData, paramsMLX90640 *mlx90640)

    void ExtractPTATParameters(uint16_t *eeData, paramsMLX90640 *mlx90640)

    void ExtractGainParameters(uint16_t *eeData, paramsMLX90640 *mlx90640)

    void ExtractTgcParameters(uint16_t *eeData, paramsMLX90640 *mlx90640)

    void ExtractResolutionParameters(uint16_t *eeData, paramsMLX90640 *mlx90640)

    void ExtractKsTaParameters(uint16_t *eeData, paramsMLX90640 *mlx90640)

    void ExtractKsToParameters(uint16_t *eeData, paramsMLX90640 *mlx90640)

    void ExtractAlphaParameters(uint16_t *eeData, paramsMLX90640 *mlx90640)

    void ExtractOffsetParameters(uint16_t *eeData, paramsMLX90640 *mlx90640)

    void ExtractKtaPixelParameters(uint16_t *eeData, paramsMLX90640 *mlx90640)

    void ExtractKvPixelParameters(uint16_t *eeData, paramsMLX90640 *mlx90640)

    void ExtractCPParameters(uint16_t *eeData, paramsMLX90640 *mlx90640)

    void ExtractCILCParameters(uint16_t *eeData, paramsMLX90640 *mlx90640)

    int ExtractDeviatingPixels(uint16_t *eeData, paramsMLX90640 *mlx90640)

View all 11 project logs

Enjoy this project?

Share

Discussions

jamesdanielv wrote 01/18/2019 at 16:29 point

just wanted to post that i'm in the middle of updating my upsample drivers so they are separate from the other code. this allows upsampling outside of my project, and makes code a lot simpler to understand. for example, you would just set size of area to update, and then input at lower resolution data into buffer. it will do all the optimizations at greater resolution and use very little memory.

after the upsample code rewrite and documenting it as a separate article, i can then again focus on these sensors. 

  Are you sure? yes | no

jamesdanielv wrote 12/04/2018 at 01:23 point

there are issues in how the calculations are done currently. they require 20k or more ram. as for non float in library you will need some floats for accuracy at least for comparing voltage and offsets, you can however choose to repurpose needed floats, as long as you do several setup routines within a function (vdd, gain, Ta, ect) .. i have broken down some routines and how they process in comments log. the files need to be zipped for now. but i will try to make an unzipped version available that is updated less frequently.

 they include all libraries to run code, and in some instances they are different than standard drivers for performance, for example  i have other code for amg8833 and lcd adafruit library can only get 1/10 performance because i can not use unrolling loops and combining tasks during spi processing is not possible with the way drivers are written, or the way i have optimized them. 

Also my drivers for melexis will be different than standard driver. possibly in the end an entire rewrite with different documentation. melexis drivers for example don't include functions to switch sensor into a frame and store mode, they just assume continuous operation at speed the i2c bus can send the data. this is not needed. 

if you can wait a week or two, ill have drivers completed enough at least for imaging that will reduce write calls to ram by about 20000%, yes about 2000 times less writes to ram. everything you do with melexis driver currently thrashes ram, even values that are suppose to be static. currently you could fix the float point issues but the over writes to ram far outweigh the float calculations. again the way the drivers are written seems unnecessarily evil.

  Are you sure? yes | no

Yevhenii wrote 11/15/2018 at 14:03 point

Please, do not place zip files to git :) unpacked sources :)

I also have 90640, planning to make fixed point library to speedup calculations (FYI, but it is almost empty now https://gitlab.com/riconec/mlx90640_fp_libc)

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates