I had a chance to prototype a quick test of the mixer-right-at-the-photodiode idea. It's a mess both aesthetically and electrically, but it'll do as a PoC.
The prototype implements this idea for the mixer using a 74HC4066 quad analog switch and 1 uF integrating capacitor (C1):
It's the same image from the last log, just here for convenience. On the breadboard, A and C are connected to the input of a signal generator, while B and D are see an inverted copy of the input waveform created by a simple transistor inverter. A second transistor drives the LED either 0% of the time, 100% of the time, or in or out of phase with the input signal. I used a BPW34 photodiode and a somewhat random IR LED I had around.
Here's the output with the LED off:
The yellow trace is the input from the signal generator, in this case a 11 kHz square wave. The cyan trace is the voltage across the integrating capacitor. In this case, the output voltage is very near zero. This is despite the fact that the shop lights are very bright, and the BPW34 generates 6.4 uA of short-circuit current when tested with an ammeter.
With the LED on constantly, the output is the same, very close to zero:
When the LED is driven in-phase with the mixer, the output jumps to around 130 mV:
and with the LED driven out-of-phase, the output is -150 mV.
I didn't have a LFSR generator handy for these tests, so this simple prototype would cause all sorts of interference, but it shows that the mixer idea is sound. It already seems pretty immune to interference from other sources, though. An IR TV remote right next to the BPW34 just produces a little jumpiness on the output, but no appreciable DC shift.
I still don't know about the small-signal performance, which I can't really evaluate with the lousy layout of the prototype. Way too much noise and crosstalk.
I'm experimenting with this front-end design, but I still intend to start with Vishay's VSOP98260 amplifier. I also found that they have receivers with photodiodes and this receiver circuit integrated into a single package. I'm going to evaluate the TSMP6000TT and TSMP77000TT modules.
I also found a part made by Broadcom, the HSDL-9100-021, that integrates a matched IR LED and photodiode and includes shielding to reduce crosstalk between them. This sounds like it might work well with either the CMOS switch mixer idea or the VSOP98260 amplifier.
Now, I have a few small PCBs to design to get these things mounted on something for testing.