This log is to detail the requirements and challenges required of the wireless module. Fundamentally it is a webserver serving files from a microsd card and/or it's own flash memory. It also is sharing the SD card with another device, that the ESP8266 can power up and down.

There are several simple webserver examples out there: e.g. https://randomnerdtutorials.com/esp8266-web-server/

Or the much better: https://github.com/marcoskirsch/nodemcu-httpserver .

Nevertheless, there are several things the webserver needs to be able to do reliably, that the simple examples don't necessarily cover. (though Marcos Kirsch's one is pretty complete on the webserver front and does several of these things perfectly)

• Serve images larger than 2-4 kB
• Server images fairly fast
• Allow the module to do some server side scripting
• Use that scripting to make an index that updates
• Control a camera module to get live images though webserver
• Not interfering with the sd card communication (i.e. have the pins float when not in use)
• Keep on operating after 100's of refreshes.
• keep on operating with 100's of images.
• Pull time from NTP and edit SD card with time updates before enabling camera
• Still provide a responsive webpage despite waiting on a slow camera.
• Provide file upload to modify the scripts wirelessly. 
• Allow file deletion, and bulk file deletion
• Be able to handle a fresh formatted microSD card in the camera.
• Limit connections on the ESP8266 to within it's limit of ~4

Some of the solutions here can be hacks, it's OK if say, the module resets after every picture, it's never going to be a fast experience. Some things set requirements for later on - for instance everything apart from images is staying on the ESP8266's flash.

The overall methodology I originally planned was to have lua commands embedded in html files, for some kind of .htmlua file. In that manner one file would be responsible for everything. However, this makes sending files quite complicated in that you have to scan for and buffer for commands every time the file is requested... it could get quite tricky.

Instead the preferred technique as used in the nodemcu-httpserver project is to have .lua files which individually do the whole connection (and send some dynamic HTML), and some static html pages. Any hyperlink can also have a GET request easily, that info can go into the scripts, but the bulk of the work will be with a set of static .html files and using .lua files as dynamic pages. They'll get a bit unwieldy, but should be readable. Headers and stuff repeated often can be hidden in a .lua file, and called as needed.

One obvious hard part is keeping the connection alive if the camera's taking many seconds to turn on. I don't want to go into dynamic web page content - the ESP8266 doesn't like having many simultaneous connections...instead it may simply have to either be slow, or show the previous image. Making a webpage auto-refresh is easy though.

For now, the best bet appears to be https://github.com/marcoskirsch/nodemcu-httpserver. If I do manage to fix any generic problems with it, it's open source so I could submit a pull request.