We're up at the vacation villa this week and I installed the watchdog and set up the Pi that's in residence to take care of business. Well, last night in the middle of the night the router wedged itself and the Pi and box did their job flawlessly and brought everything back.

I've decided to run the script every 2 hours instead of 4 (so the cron spec is 0 */2 * * * $HOME/watchdog.py), and I've reduced the testing timeout to 30 minutes from 60 (a firmware update should take no longer than 15 minutes including time to get the system back up).

Of course, it would be better if this sort of thing just simply didn't happen and the router was reliable, but I don't think any router is going to be reliable enough that I'd not want to keep this system in place, frankly.