At work we have a little Arduino and electronics breadboard kit on a random table to play around with.  No one uses it for much, but it had a 9V battery and a 2n2222.  I wired up the circuit, and using a 100k resistor for Rbc, and 330 ohms for Rled, recreated the same circuit.  The battery read out about 9.2 volts on the multimeter.  I feel more confident in the circuit since the LM317 is no longer a factor (long-shot that the power supply had some interaction with the transistor), and that it works with at least a different transistor number, sourced from somewhere completely different.

The next thing I need to try is removing the LED.  It is, after all, a diode, and as we've seen from other projects on this site (the DIODE clock!) diodes like to be 'active' devices from time to time.