Lepton's use an output-only slave SPI interface called VoSPI (Video SPI) to output pixel data with a maximum 20 MHz clock.  It has the feeling of something that has evolved over time as FLIR added newer models.  It definitely does not feel like something that would be designed from scratch.  The Lepton's basic unit of data transfer is a 164- or 244-byte packet.  The 164 byte packets contain 80 16-bit pixels (of which 8-, 14- or 16-bits of data may be valid for the Lepton 3.5 depending on its operating mode).  The first 4 bytes are a 16-bit ID word and a 16-bit CRC word (that I have not, to-date, attempted to use).  The 244-byte packets contain 80 24-bit (8-bit each for R, G and B) pixels and the ID and CRC words.  The ID word carries a line number (0-59 or 0-62).

The 80x60 pixel Leptons (2 and 2.5) output 60 packets per frame (or 63 packets if telemetry data is also included with the pixel data).  This turns out to be about 2 Mbits/second for a 100% dedicated interface.

The 160x120 pixel Leptons (3 and 3.5) modify the protocol a bit in order to carry 4x the data.  They add a segment number to the ID word of packet 20.  Segment numbers 1-4 indicate that the entire set of packets contain a valid segment.  It definitely would have been easier (less buffering and easier processing) if the segment number was in the first packet.  This family of device requires a minimum of 8+ Mbits/second although I found that with the Teensy I had to use 16- or 20-MHz SPI clocks.  I noticed most of the other demo programs also use much higher SPI clock rates.

All Lepton's have a hard requirement that the host must get the data out within three lines of when it is generated in the Lepton or it will lose synchronization and be unable to output valid data.  All Leptons also output what are called "Discard packets" that are indicated by a specific bit-pattern in the ID Word.  The host is to ignore these packets but keep reading for good data later.

In my testing I also found that until synchronized they may output nonsensical non-discard packets too.

I wrote a quick Teensy sketch that enabled VSYNC and polled it until asserted.  It then read packets until it saw a complete segment, or it saw invalid data (invalid line number) or a timeout was exceeded (maximum VSYNC period).  The program could usually sync with the Lepton within a handful of VSYNC periods.  The output from a typical run is shown below.  Once synced you can see a new valid frame every twelve frames (or 1/3 the data frame rate).  These are the frames that get displayed.  Each line number (0-59) represents 160 bytes of pixel data.

It is interesting to see that alternating segments have discard packets.  I don't have an explanation for this although I wonder if this has to do with the timing of my SPI bus (16 MHz) and the Lepton's internal processing rate.

Getting this code running meant I could get valid data out of the device.

The core of the code:

void loop() {
  bool curVsync;

  curVsync = (digitalRead(pin_lepton_vsync) == HIGH);
  
  if (curVsync != prevVsync) {
    Serial.printf("%3d : ", counter);
    ProcessSegment();
    Serial.println();
    counter++;
  }
  prevVsync = curVsync;
  
  if (counter == 106) {
    WaitForChar();
    counter = 0;
  }
}

void WaitForChar() {
  while (!Serial.available()) {};
  while (Serial.available()) {
    (void) Serial.read();
  }
}

void ProcessSegment() {
  uint32_t startUsec;
  bool done = false;
  
  line0count = 0;
  discardCount = 0;
  startUsec = micros();
  while (!done) {
    if (ProcessPacket()) {
      done = true;
    } else if (AbsDiff32u(startUsec, micros()) > LEP_MAX_FRAME_DELAY_USEC) {
      done = true;
    }
  }
  Serial.printf(": %d (%d)", AbsDiff32u(startUsec, micros()), discardCount);
}

bool ProcessPacket() {
  uint8_t line = 0;
  uint8_t seg;
  bool done = false;

  SPI.beginTransaction(SPISettings(16000000, MSBFIRST, SPI_MODE1));

  //Start transfer  - CS LOW
  digitalWrite(pin_lepton_cs, LOW);

  SPI.transfer(frame, 164);

  //Repeat as long as the frame is not valid, equals sync
  if ((frame[0] & 0x0F) == 0x0F) {
    discardCount++;
  } else {
    line = frame[1];

    if (line == 0) {
      line0count++;
    }
  
    //Get segment when possible
    if (line0count > 1) {
      done = true;
    } else if (line >= 59) {
      Serial.printf("%d ", line);
      done = true;
    } else if (line == 20) {
      seg = (frame[0] >> 4);
      Serial.printf("%d(%d) ", line, seg);
    } else {
      Serial.printf("%d ", line);
    }
  }

  //End transfer - CS HIGH
  digitalWriteFast(pin_lepton_cs, HIGH);

  //End SPI Transaction
  SPI.endTransaction();

  return(done);
}