I was having SPI issues that seemed to be exacerbated by setting a clock speed faster than 2MHz.  I was getting junk data from the TDC chip sometimes.  So, I kluged around this by discarding values that were outside a reasonable range, but I wanted to address the underlying issue.  So, lets dig into what is actually happening here.  

Looking at the SPI transactions with a scope showed a "weak" MISO.  

The magenta trace should be going to 3.3V but sometimes doesn't even manage to get to 1V.   Just as an aside, always remember to put in test points for signals.  The first revision didn't have easy places to probe and it was a massive pain to capture these traces on that board.  

To make things even weirder, probing the traces cause more problems.  I could see better results when not probing.  Just as a test, if I set the clock speed to where it just barely worked, and then when I touched the TDC chip with my finger, it would fall apart.  The extra capacitance from the probe or my finger would degrade the signal enough that it would stop working.  This is a pretty short, well routed trace so it shouldn't be so borderline.   

So does SPI need a pull-up to work? In theory no, but it sure seems like this chip could use some help.  But what value to pick? 10K did nothing, but using 3.3K did improve things:

You can see the multiple slopes on the leading edge and no flat top.  The pull-up drags the MISO up to 3.3V, but it takes a few 100ns.  This is with a clock set to 5MHz (the RP2040 seems to not quite make it there).  But if we bump up the clock to the advertised 20MHz

Here you can see the leading edge takes too long to rise up to the 3V level and the clock already gets to a falling edge where the measurement is made.  So, lets try a 1K pull-up.  

Now at 20MHz (RP2040 only makes it to 15.6MHz), even the last single bit on MISO can make it up to about 2.7V in time for the falling clock edge.  You can also see the TDC starting to struggle in bring the low level down to 0V, but the 0.5V level seems to be good enough to be treated as logic low.  

Things still aren't perfect.  The TDC should be able to drive the MISO from 0V to 3.3V without that 1K pull-up, but at least this gets it working at 15MHz without kluging around it in software, which is much faster than where it was before.  More importantly, it seems more stable, so board to board or chip to chip variations won't mess it up.  I've also seen a board that worked at one point start to fail after a few months with SPI errors.  Now if I could only figure out why the WiFi sometimes won't reconnect...