I got everything working to support multiple Arduinos and multiple control modes tonight. Probably the hardest thing I've worked on, just because of how complicated the data structures have gotten. It's all working now. Definitely needs a lot of testing with multiple configuration modes, but that will come with time.
In any event, I got it all working with the Raspberry PI so I now have a functional wireless controller!
Next up I'll probably take a look at creating a web based configurator/viewer.