I don't know why I haven't thought of this before.
There's no reason to be so fussy about figuring out the precise voltage for the displays. The high side driver chip is perfectly happy to be fed by 5v and use 3.3v input levels from the controller. Given that, there's no reason not to just put series resistors in the anode lines. In that universe, there's also no reason we can't just use an LDO instead of a switching supply for the controller and GPS receiver - we use the same arrangement in the talking clock and it works fine.
Better yet, they make 30 mA current limit "diodes" that we can use instead of resistors. That way we don't have to tailor the resistors for the LEDs' Vf - we can source any color we can get in the correct pinouts and let the current limiter dynamically tailor itself to the Vf of the displays.
I'm going to try this architecture and if it works, it'll be the follow-on version to v3.1 in the store.
EDIT: One difficulty I had not fully considered was that the current regulator has an overhead voltage of 1.8 volts. Given a 5v input, that means a maximum Vf of 3.2v, which is right where the blue LEDs sit. The 5V input voltage was necessary because it was the maximum voltage of the 3.3v LDOs I've got handy, but it turns out you can get pin-compatible wide input LDOs. One factor to consider going that route is the maximum power dissipation of the LDO, but back-of-the-envelope calculations with the NCP718SN330T1G show that the input voltage can go all the way up to about 9 volts before power dissipation becomes a concern. That suggests that running with blue LEDs would be best with a 6 volt supply, but there are folks who have asked about using very large LEDs with much higher Vf. It turns out that's not a problem at all with this new design. The high-side switch and current limiters can handle an input of up to something like 45 volts, and if you remove the on-board LDO and supply 3.3v through the PDI header on the board, it'll work just fine. In this configuration you'd probably also want to solder the antenna power jumper to the 3.3v side or open it and use an external bias-T so as to not feed too high a voltage to your antenna.