Summary

It is time to write the code to control the Nixie clock, and provide a service to do some things.

Deets

Now that the hardware is apparently stable, I can start to work on the software side.  The software in this case is written in Lua, and run in an event-driven execution environment.

A Little About Lua

I won't explain Lua too much except to say that it scripted, compiled to byte-code for execution, and has very few fundamental data types (notably number, string, boolean, nil, function, and 'table').  The sole structured data type is the 'table', which is an associative array.  The special case of integer keys is used to realize conventional arrays, but they are meant to be 1-relative.  To me, Lua feels a little bit like Javascript.

When Lua code is executed, it is immediately translated into a 'byte code' form that can be executed by a virtual machine.  I say 'can be', because some statements such as 'function' are only compiled, and create an object named as declared in the source code that contains it's byte code representation.  Statements at file level are executed immediately.

Lua scripts can be in files, but they can also be in strings.  A section of Lua code (file or string) is called a 'chunk'.

A quirk of Lua is that named objects are global by default, unless declared 'local', or in the special cases of parameter names, etc.

A Little About the NodeMCU Execution Environment

The execution environment in NodeMCU is a little different than what is more commonly found in Lua environments in that it is intended to be used to define a mainly asynchronous system.  This is similar to NodeJS which was the inspiration, hence the name.  In this asynchronous environment, you try to do as little as possible in sequential steps of execution (that is a synchronous model), and rather break up your activity into a bunch of handlers that will be invoked when relevant events come in.  As such, your program when run really just defines and registers a bunch of handlers, then immediately returns control back to the system.

This style of authoring can be a little disorienting if you are mostly used to the step-by-step style (i.e. 'synchronous') of coding, but you you will get used to it.  The existing modules are pretty rich, so your code in Lua often is rather small.  But it's definitely not a sequential step of execution from the top of your source file to the bottom.

My first attempt at an application will by structured like this:

(No Visio for me, tee-hee.)

There will be three files:

1. init.lua
   This is a specially named file that is automatically executed after the NodeMCU board has gotten the Lua environment up, just prior to running the interactive shell.
   You could put your entire program in here, but I'm not going to for reasons I'll explain later.
2. kosbo.lua
   This will be the program itself.  It will load configuration, declare all the event handlers, utility functions, and have a little immediate code that causes all that to wire together.  This is a fast process, immediately exiting and returning to the system (which will then run the interactive Lua shell on the USB serial port).
3. kosbo.cfg
   This will contain configuration settings.  Things like my wireless router's SSID and passphrase.

And that's it!  'init.lua' and 'kosbo.cfg' are simple, so I'll explain them first.

init.lua

As mentioned, init.lua is a specially named file that works a little like autoexec.bat of olden days.  You can put your whole program here, but I actually like to put my program in a second file that is invoked from this one.  I find this handy for development.  If I have a bug in my program, I would prefer the board boot to the shell and not run my program.  Then I can manually run my program and see any sort of debugging output on the terminal.  If I had autoexec'ed my program, all that output would be lost by the time I connected the terminal to the serial port.

My init.lua is a one-liner:

pcall ( function() return dofile("kosbo.lua") end )

'pcall' is a 'protected call' and is roughly equivalent to a 'try' in other languages.  It will catch any errors raised and return, instead of giving them to the runtime, which will simply abend.  It returns at least two values (yes, Lua functions can return multiple values), a boolean indicating the function ran, and a textual message (or nil if no message) that may have been part of where the code error()'ed.  It may return additional values, which are the return values of the function called.

Here, we define an anonymous function inline which runs dofile() on 'kosbo.lua'.  So, if I have init.lua on the board, it will run my program on boot, and if I don't have it, it will drop to the shell.  At the shell, I can manually execute that same one line, and run the program and see any important output to help me debug.

kosbo.cfg

The configuration file is just slightly more interesting.  Here is a skeleton version:

wifi_sta
{
    ssid = "myrouterssid",
    key = "myrouterwpapassphrase",
}

 To a human that is intelligible:  a 'section' of stuff named 'wifi_sta' ('sta' for 'station' mode -- arbitrary name), followed by some stuff in curly braces that are name-value pairs separated by the'=' symbol, and they themselves can have several separated by the ',' symbol.

Fact of the matter is that this is actually Lua code.  So to 'parse' your configuration file, you merely need to 'execute' it.  That idea will totally freak out security-conscious folks, but it was considered cool in the 90's and in fact that is what JSON was all about as well.  In this case, the 'code' interpretation is 'call a function named 'wifi_sta' and pass it a parameter which is a table which has two entries with the key of 'ssid' and 'key'.  Then party on that.'

So to process configuration, one needs to implement a (global) function named 'wifi_sta' (in this example), and then merely call dofile() on the configuration file.  Your wifi_sta() function will take one parameter:  a Lua table, and it will contain all the key-value pairs listed.  It will be invoked when you 'execute' the config file.  Tada!  No special config file parser.

Here's a minimal example:

-- config function; set the wifi station
-- this creates a named function to be executed later, but this has to be
-- global, because it has to be reachable when executing a different file
function wifi_sta ( tuple )
    print('in wifi_sta()...')
    print('the SSID is:  ' .. tuple.ssid )
    print('the key is:  ' .. tuple.key )
    -- do other interesting things
end

-- 'read' configuration file by executing it.
-- this creates a named function that we know will be references by code
-- in this file only, and so it can be 'local'.  It will go away when this
-- file's execution ends
local function configure ( confname )
    print('reading configuration...')
    local ok, msg = pcall ( function() return dofile(confname) end )
    if ( ok ) then
        print('configuration loaded!')
        return true
    else
        print("configuration not loaded from file '"..confname..
                "' message = "'..msg.."'")
        return false
    end
end

-- the following is at file level and is executed immediately
if ( configure("kosbo.cfg") ) then
    -- ... do more things
else
    print("failed to process configuration; ending...")
end

The above file is named 'kosbo.lua', and it what will eventually be auto-exec'ed via 'init.lua' as mentioned earlier.  But for now it's handy to manually execute it so that I can see the debug output.

There's aspects of the Lua execution environment that is useful to understand.  When the file is executed, what is happening is that it is being compiled into byte-code, and either executed immediately, or stored for later.  The first two sections define a function object for later use, under the names 'wifi_sta', and 'configure'.  Nothing gets executed there at this time.  The last section is at file level, and so it gets executed immediately.  When the end of the file is reached, control is passed to whatever invoked it.  This might be the Lua shell when we do it manually, or back to init.lua if via that mechanism.

When that happens, the byte code that was generated for the third section is (eventually) reclaimed by the garbage collector, and anything declared as 'local' is as well, if there were no other references to it.  This is the case with the second function, configure(), since it was declared as 'local'.  However, the first function wifi_sta() was not declared as local (i.e., it is global), so it sticks around and takes up memory.

There is a reason that wifi_sta() is global.  The reason is that the configuration file 'kosbo.cfg' needs to be able to reach the wifi_sta() function.  Since it is in a different file, it would not otherwise be visible to the kosbo.cfg unless it was global.

The downside is that wifi_sta() is only needed for a moment, when configuring, after that is just a waste of RAM.  That's easily remedied, though, simply by setting the function name (which is really a variable name containing a function object) to 'nil'.  Then it will effectively be deleted, and it's memory available for garbage collection.  A good place to put those is right after the dofile() call.  E.g.:

local function configure ( confname )
    print('reading configuration...')
    local ok, msg = pcall ( function() return dofile(confname) end )
    -- now we can delete the global config functions from memory
    wifi_sta = nil
    if ( ok ) then
        print('configuration loaded!')
        return true
    else
        print("configuration not loaded from file '"..confname..
                "' message = "'..msg.."'")
        return false
    end
end

Connecting to the Network

For the next amazing feat, we will connect to the WiFi.  This involved doing something useful in the wifi_sta() configuration function, and then writing some Node-style code that registers callbacks that are invoked when connection has been successfully made.  First, fleshing out the wifi_sta() function:

-- config function; set the wifi station
--(this has to be global; we delete it when we're done with it)
function wifi_sta ( tuple )
    print('in wifi_sta...')
    -- set the ssid and password if different from what is already in flash
    -- oh, and set auto connect
    local ssid, password, bssid_set, bssid = wifi.sta.getconfig()
    -- retained in flash, so avoid writing unnecessarily
    if ( tuple.ssid ~= ssid or tuple.key ~= password ) then
        print('setting wifi parameters to ssid='..tuple.ssid..', 
                key='..tuple.key)
        wifi.sta.config ( { ssid = tuple.ssid, pwd = tuple.key, auto = true, 
                save = true } )
    end

    -- static IP setup, if desired
    if ( tuple.ip and tuple.netmask and tuple.gateway ) then
        wifi.sta.setip( { ip = tuple.ip, netmask = tuple.netmask, 
                gateway = tuple.gateway } )
    end
end

This is fairly straightforward:  take the configuration parameters and stuff them into the wifi library.  We do a little optimization in that we avoid setting them redundantly, because these are stored in flash, and we want to avoid wearing it out needlessly.

The next part is a function that will make repeated attempts to connect, and invoke notification functions on success or failure.  Failure means that the maximum number of attempts has been reached without successful connection.

It is forbidden in NodeMCU to take 'too much' time processing without yielding control back to the 'system', so things like spin-waiting in a delay loop are straight out.  But I don't want to hammer the wifi checking for connectivity, so I use a timer.  The timer will have a registered callback function that will be invoked by the system periodically, and this will function similar to what I would otherwise do in a for loop, with a sleep-like function.

local function connect_and_run()
    -- try to connect to the access point; check 10 times, 3 sec between check
    if ( (wifi.getmode() == wifi.STATION) or (wifi.getmode() == wifi.STATIONAP) )
            then
        -- we use a timer instead of a loop so that we yield to the system
        -- while we're waiting for a delay to pass between attempts.
        local joinCounter = 0
        local joinMaxAttempts = 10
        local joinTimer = tmr.create()
        joinTimer:alarm ( 3000, tmr.ALARM_AUTO, function(t)
            local ip = wifi.sta.getip()
            if ( ip == nil and joinCounter < joinMaxAttempts ) then
                print('Connecting to WiFi Access Point ...')
                joinCounter = joinCounter + 1
            else
                -- relinquish this timer now
                t:stop()
                t:unregister()
                -- we either succeeded or failed...
                if ( joinCounter == joinMaxAttempts ) then
                    -- sorrow
                    print('Failed to connect to WiFi Access Point.')
                    connect_failed()
                else
                    -- joy
                    print('Connected!')
                    connected()
                end
            end
        end )
    end
end

 Of note here if you're not familiar with Lua's syntax is the use of the colon ':' operator.  This is syntactic sugar to make Lua look more like an object-oriented language.  It simply passes a hidden parameter is the first argument.  So the following are equivalent:

-- semantic sugar to look OO
t:stop()
-- functional equivalent without sugar
t.stop(t)

It's useful to note that this function connect_and_run() exits immediately.  It is the anonymous function that is registered during joinTimer:alarm() that is run later (and repeatedly, as we have set it up).

It's also useful to note that the variables joinCounter, and joinMaxAttempts are accessible within the body of that function, even though they ostensibly have gone out-of-scope when connect_and_run() exited, which was long before the anonymous function was called for the first time.  That is because Lua binds those variables to the function as what it calls 'upvalues'.  You don't have to do anything special to make this happen, it's just good to be aware that it is available.

The 'loop' created will try 10 times, waiting 3 seconds between each attempt, before giving up.  If during this time it was successful, the connected() function is invoked, and if the maximum attempts are reached, the connect_failed() function is invoked.

To kick this process off, the file-level immediate code is modified to invoke the connect_and_run() method:

print("processing configuration...")
--setup the environment as per config
if ( configure("kosbo.cfg") ) then
    -- the mode is retained in flash, so avoid writing it unnecessarily
    if ( wifi.STATION ~= wifi.getmode() ) then
        print('setting station mode...')
        wifi.setmode(wifi.STATION)
    end

    -- explicitly request connection to happen if we aren't already connected
    if ( wifi.STA_GOTIP ~= wifi.sta.status() ) then
        print('trying to connect...')
        wifi.sta.connect()
    end

    print("connecting to access point...")
    connect_and_run()
else
    print("failed to process configuration; ending...")
end

The connect_failed() function will reboot the system, restarting the process:

local function connect_failed()
    -- we simply reboot to start it all up again
    node.restart()
end

The connect() function is invoked on successful connection.  I print out a little status info, and then start a SNTP synchronization process.  This will register even MORE callback functions:

local function connected()
    -- emit some info
    print("Wireless mode: " .. wifi.getmode())
    print("MAC: " .. wifi.sta.getmac())
    print("IP: "..wifi.sta.getip())
    print("Hostname: "..wifi.sta.gethostname())

    -- now that we have network, sync RTC
    sntp.sync(nil, sntp_syncsuccess, sntp_error, true)

    -- XXX other things

end

 And the the callbacks for handling the SNTP activities:

local function sntp_syncsuccess ( seconds, microseconds, server, info )

    local sec, usec, rate = rtctime.get()
    local tm = rtctime.epoch2cal(sec)
    print ( "sntp succeeded; current time is:  " .. 
    string.format("%04d-%02d-%02d %02d:%02d:%02d", 
            tm.year, tm.mon, tm.day, tm.hour, tm.min, tm.sec) )

    -- XXX more things

end

local function sntp_error ( code, text )
    -- code:
    -- 1: DNS lookup failed (the second parameter is the failing DNS name)
    -- 2: Memory allocation failure
    -- 3: UDP send failed
    -- 4: Timeout, no NTP response received
    print ( "sntp failed!  code:  " .. code );

    -- XXX do we need to retry?  or will it retry automatically?

end

So, all this was tested out.  I couldn't stimulate an SNTP error, so I'm not sure if the library will keep retrying, though I think it will.  As written above, the SNTP sync() will repeatedly synchronize every 1000 seconds.  There is not a provision to change this interval.  The internal implementation of sync() will set the rtc of the ESP8266.  Ultimately, we'll use that to set the Nixie clock time and date.

Next

Controlling the Nixie clock.