At work we have a little Arduino and electronics breadboard kit on a random table to play around with. No one uses it for much, but it had a 9V battery and a 2n2222. I wired up the circuit, and using a 100k resistor for Rbc, and 330 ohms for Rled, recreated the same circuit. The battery read out about 9.2 volts on the multimeter. I feel more confident in the circuit since the LM317 is no longer a factor (long-shot that the power supply had some interaction with the transistor), and that it works with at least a different transistor number, sourced from somewhere completely different.
The next thing I need to try is removing the LED. It is, after all, a diode, and as we've seen from other projects on this site (the DIODE clock!) diodes like to be 'active' devices from time to time.