The inside of the foundation is the sides of the heat storage area. Once the foundation has cured and the forms are removed use vapor barrier to make a liner around the inside of the foundation. Tape all the seams so it is a continuous barrier. If it rains it would fill up like a swimming pool.

Cut and fit foam insulation board around the bottom and sides of the heat storage area.  Use 2" thick 25 PSI or better High Density Expanded Polystyrene Foam Board that's rated for below grade under slab and sidewall foundation use. Don't use Extruded Polystyrene, the blue, green or pink stuff, the blowing agents used to produce it have a high global warming potential.  Consult with an architect or structural engineer for the PSI rating required for your structure. 

Seal the insulation boards together with tape and/or expanding spray foam.

The heat storage medium in this house is #1 crushed limestone. The stones should be 4 inches in diameter. Ask for it washed. I didn't and it took a lot of effort to wash the grit off.

The bottom of this photo is south, where the sunspace will be built.