This major effort replaces a 20 year old HCS_C Home Automation System.
I got the flash programming working, and was able to successfully load up a baseline set of web pages with embedded control codes into flash on my NXP4357 PC board. Then I set up the base address of my pseudo-USB disk to point to this flash, and voila! I was able to view the webpages with a browser. Later, I will set up the Apache webserver to point to the NXP4357 board, which will look like a disk to Apache, which will then serve these web pages to the internet.
In the meantime, though, this is just the baseline web page list. Inside these webpages are control codes. When a web page is requested, the NXP4357 will dynamically replace these control codes with actual data read by the NXP cpu--for example, the control codes for the weather station will be replaced with actual data to view. I don't have to have any code running on the web server to bog it down--the USB disk will just look like web pages that are constantly changing. Cool, no?
Now the task is to write that code that will do the dynamic substitution of control codes--currently it just displays the control codes as if they were text. This will take a little while...
I replaced a blown power supply for all the remote modules, but it looked like the supply failure killed most of the remotes. The venerable main board, which has been running continuously for about 23 years, is still running like a champ--but the time has come. I can't compile for it without pulling out my old XP system--none of the new machines can run the old compiler or FPGA tools (the new FPGA synthesis doesnt support the ancient FPGA on board). Now I have a lot of incentive to get the new system up and running.
The USB code is working like a champ--so much much better than RS232. Now I'm trying to write the serial flash code. There are 6 serial flashes on the processor board, one of them to hold the firmware, one to hold working html pages, and 4 to hold log files for home automation events--should be good for the next 20 years!
Since the remote modules have probably been damaged, I've been doing a lot of research on new cameras, even commercial ones like Nest and Canary. I got some Raspberry PI Zeroes and 2s, will look at using the Zeroes as remote modules using the Tiny Core linux. Might save me a lot of work...
I freely admit it. I know nothing about USB, one of the many reasons I took on this project was to stave off dementia by learning how it works. It's been a nice holiday break learning a lot about how USB operates--a crucial step to implementing my vision of an ideal Home Automation interconnect. Making a USB mass storage device that holds dynamically updated webpages is a win in so many ways, so I have to make it work. Easier said than done though, USB isn't a simple serial port. Fortunately, the NXP LPC4357 part has a USB ROM library that takes most of the work out of the process, but there's still a lot of details. I laid out the HCS_IV processor board just guessing what needed to be there for USB, so of course I left off the inductor and diode ESD protection. Never fear, I'll just hack it onto the board.
Uhh, those ESD parts are *tiny*. I was a micro-surgeon for a day, but modified two boards with in series common mode inductors and a diode array. I'd suggest if you do a USB board, it's a lot easier to include these parts in the layout.
As I've started in on the firmware side of the project, the old HCS has begun to fail. This has really motivated me to get moving--the old HCS has been indispensable, it runs the house! One of the power modules failed--fortunately I still have a replacement--but now the cameras no longer reliably capture pictures. Other things are starting to get flaky as well--a huge departure from the bullet-proof reliability I have enjoyed with HCS.
I had a really great idea for improving the FW design of the new system. The old HCS is a kludge on a kludge on a kludge--the main board ARM processor runs the interpretation of the programmable token list of commands and timers, which communicates over serial at 115200 to a netbook running a host program written in C, which deposits HTML and CGI files in the right places for an Apache web server. The new HCS uses a completely redone main board with about 100 times as much memory (lot of improvements in chip technology in 25 years) and should run about 10 or 20 times as fast. In addition, I decided to simply write home control programs in C rather than the old HCSXPress token interpreted language--I had been maxing out the performance of the old ARM7 processor running at 29Mhz! In addition, the 115200 baud serial port severely limited the stream of data from the main board to the netbook.
But what a huge accumulated pile of firmware and software, none of which is easy to change or update without breaking everything! I've stopped adding features or even changing the program because of the fear of losing the whole thing--a really bad state of affairs for someone who loves to tinker with Home Automation.
In a brilliant (if I may say so) flash of inspiration, I found a great solution. I knew I had to learn USB to get over the problem of not enough data bandwidth--so why not remove the host program on the netbook altogether and have the main board act as a USB mass storage device--and generate the HTML and cgi itself??!! The worst part of the original HCS was trying to get the main board to talk to the netbook--the master/slave serial port streaming thing just doesnt work because I really needed a peer to peer interface--just a huge kludge--then the host program also had to communicate with the apache server and handle CGI--what an ugly mess I could eliminate altogether just my making the main board look like a disk drive and serve the web pages directly!
It took awhile, but finally solved a nasty little problem with the NXP processor talking to the FPGA on the HCS_IV control board. That interface is asynchronous since the clock from the NXP is not available. It worked the vast majority of the time, say for about 500 accesses, but then got a hold fail on the NXP address. This was fine at first because I just needed diagnostic tracking that is on the FPGA, I was working on other parts of the HCS_IV system. But now I'm starting to turn on the HW thread manager which resides in the FPGA, and I had to fix this.
The solution I used changed the interface to latch rather than clock the NXP signals. Latches are verboten in production designs (makes testing very hard among other things) but carefully applied I think it can solve some interface problems like this one. All of the synchronizing solutions required handshakes and made the interface run too slow. This solution just latches the address, control, and write data on the NXP read/write strobe, then synchronizes the appropriate strobe (reads often trigger an event, but the read data has to be supplied asynchronously to meet MXP hold timing). Complicated problem to solve, facilitated by both my Ghz Logic Analyzer--see one of my other projects. What a priceless gizmo that thing has been. Small, easy to connect up and use--and I've got four of them, making it easy to leave one hooked up to any of my several projects I'm working on.
Now I can finally move forward. The thread manager I designed automates all timed events and UART sequences so that all the NXP processor has to do is run interrupt service routines when a timed event occurs. In the old HCS_C, the processor got a system tick every second and checked all timers and other streamed devices to increment time counts and see if servicing was needed. This severely limited what the processor could do, because for a large events file this firmware could consume a large percentage of available processor time. This new HW scheme puts the timer tracking info in the FPGA SRAM, increasing the number of available concurrent events from 256 to around 65000. The FPGA can now do the checks 100 times a second, thus allowing much more precise timing control of whatever the HCS_IV is managing (I'm thinking using one of my boards for robot control). It never bothers the processor with an interrupt unless something finished or timed out and needs attention. If I need that many events, I probably need to get out more...
What? wait... Yes, I do need to get out more. But this is a lot of fun--kewl to be back working on HA.
After some work on the new logic analyzer probe, now I'm back to turning on the new HCS_IV Home Automation boards. The CPU board looks really good, just a few haywires and is up and running. I can emulate, run from flash, run from boot code in the FPGA, and output diagnostics to an LCD monitor. The RS485 driver board also has turned on well so far--I've loaded the three big FPGAs and a DDR2 as well as the voltage regulators and configuration circuitry. Next is to plug the CPU board into the RS485 driver board and attempt to drive an RS485 channel and store data into the DDR2. Why so many FPGAs? Because HCS_IV drives a *lot* of pins--each of 12 RS485 channels, the DDR, 64 inputs, 64 outputs, Insteon, 10 serial ports, 6 serial flash devices, LVDS video memory, SD?MMC, and so on. Each RS485 gets its own DMA channel into memory, thus allowing rapid independent picture/sensor data capture. I realize that RS485 can have many taps, and in fact this is how the old system worked--but this severely limited simultaneous receive from all remote RS485 modules--which in turn severely limited the number of pictures that a given channel could transmit. I'm also hoping to now support video, so having independent RS485 channels will allow parallel operation as well. Exciting times turning everything on!
It's so interesting to see the emerging interest in Home Automation--Wall Street Journal and other newspapers all have articles talking about Google Nest and all the other commercial HA products now available. Other than the Insteon network, mine is all homebrew, and is based on a system approach that really works and is extremely useful--not just gee-whiz technology. The cameras and the intercom timer announcements, and the logging of every event that happens in our house for years, just makes the existing system so valuable. The new revision builds on what I've learned over the last 20 years--and is a lot of fun!
Well, I'm off! Actually, I've been working on this for about a year, and decided to post occasionally about it. I currently have the NXP CPU board running. It can run from the Keil emulator or from boot. The processor boots in EMC mode from the FPGA, which holds the boot code. I got the the diagnostic video port working so that I can dump debug statements to the LCD monitor. This is just a diagnostic port, the actual installation will drive a low power 4D Systems display from Adafruit. Next up will be configuring the CPU for faster operation and to allow direct OS downloads into memory. Right now, I compile and link on the LPCXpress IDE, convert to binary, convert the binary to an Altera MIF file, compile the FPGA, and program the FPGA config device... yucko! The new boot code will allow an RS232 port to write directly to the NXP CPU internal program SRAM, bypassing the FPGA programming step. What great fun!