In this log I'll outline the structure of VoiceBox.py. In later logs we'll look at features in more detail.

if __name__ == "__main__":
    global mode, flask_thread, assistant
    mode = 'hotword'                   # 'button' / 'hotword'
    
    # Try to connect to Assistant before starting sub-threads
    credentials = aiy.assistant.auth_helpers.get_assistant_credentials()
    try:
        assistant = Assistant(credentials)
    except Exception as e:
        print(e)
        aiy.audio.say("Couldn't connect to Assistant, see error log.")
        exit(-1)
        
    flask_thread = threading.Thread(target=flask_run_thread)
    flask_thread.start()
    
    button_thread = threading.Thread(target=button_wait_thread)
    button_thread.start()
    
    assistant_loop()

At the start of the main program we set the global variable mode to the default value, 'hotword', meaning that the microphone is on and the Google Assistant will be listening for the keywords, "Hey Google" or "Hello Google" to trigger speech processing. (In 'button' mode, speech processing will only start when the button is pressed.)

We then try to connect to Google Assistant. Last night, I spent so much time tinkering with the program that I burned through my entire 500 query Google Assistant API daily quota.  This caused the Assistant() call to fail with Error:429: hence the try/except wrapper.

After creating an Assistant client, we start a thread, flask_run_thread(), to process web requests. This just runs the app Flask() object as in the three previous servers we've looked at: sysinfo, piHole and Blinkter.

def flask_run_thread():
    app.run(host='0.0.0.0', port=9011, debug=False)
    exit()

 We then start another thread, button_wait_thread(), to wait for button presses.

def button_wait_thread():
    button = aiy.voicehat.get_button()
    while mode != 'quit':
        button.wait_for_press()
        print('Button pressed')
        response = requests.get('http://localhost:9011/button')

The clue's in the name. This thread calls the aiy API to wait for a button press, and then  triggers the flask thread's button_get(). Note that if the mode global variable is set to 'quit', the button thread will end.

@app.route('/button', methods=['GET'])
def button_get():
    global assistant
    print('Button press requested, enabling mic')
    assistant.set_mic_mute(False) # enable microphone
    assistant.start_conversation()
    os.system('aplay src/VoiceBox/static/sounds/glass_ping.wav')
    return 'button requested'

This switches on the microphone,  nudges the Assistant, and rings a bell to let the user know that VoiceBox is ready to listen.

We mention a 'quit' mode, how is the mode set? Via flask routines:

# Change mode to 'quit', send wake-up message to assistant and shutdown flask thread
def quit_flask():
    global assistant
    set_mode('quit')
    assistant.send_text_query('Goodbye.') # send text to wake Assistant loop
    func = request.environ.get('werkzeug.server.shutdown')
    func()

# When quitting from assistant thread, post rather than get
@app.route('/quit', methods=['POST'])
def quit_post():
    quit_flask()
    return 'quit' # return simple text so calling thread doesn't hang when flask thread stops

@app.route('/quit', methods=['GET'])
def quit_get():
    quit_flask()
    return redirect(url_for('root_post'))

 The routine quit_flask() can be called either in response to a browser 'GET' at url /quit, or to a programmatic 'POST' to the same url. 

If /quit is requested from a browser, quit_flask() is called and the browser is redirected to the VoiceBox root page. (This prevents restart loops where a browser directed to /quit constantly shoots down the VoiceBox server after every restart whenever the browser refreshes.

The /quit 'POST' routine is triggered by a request from the Google Assistant thread - when the user says "Hey Google, quit".

And here is the quit() routine in the Assistant thread:

def quit():
    assistant.stop_conversation()
    global flask_thread
    if mode != 'quit': # No need to send quit request if already triggered
        response = requests.post('http://localhost:9011/quit') # n.b. post, not get see below
    else:
        aiy.audio.say('Terminating program')
        GPIO.cleanup()
        button_thread.join()
        flask_thread.join()
        os._exit(0)        # sys.exit() doesn't shut down fully

 This quit routine can be called in two places in the Assistant thread: either in response to the user saying "quit", or when the assistant processing loop discovers that the mode is set to 'quit'. The actual setting of this mode is done in the quit_flask() routine, either due to a web request, or a programmatic request from quit().

It's a bit fiddly, but this was the only way I could get the three threads to exit gracefully. Flask has so many hooks into the OS that it can't be shot down by brute force.

Almost done for this log. We just need a quick look at the structure of what I've been calling the Assistant thread (it's actually the continuation of the main program after launching the flask and button threads.)

def assistant_loop():
    global assistant
    initDisplay();
    displayText(0, 'VoiceBox 0.1')

    for event in assistant.start():
        process_event(assistant, event)

 Just initialize the Nokia display, then loop forever (until the os._exit(0) call), processing assistant events.

The process_event() routine is just a hacked version of the Google demo code in assistant_library_with_local_commands_demo.py, so get your head around that first.

That's enough. Next log we'll look some functionality.