I had a post up here the other day to show how to augment an existing wireless access point with the encoded SPS SSID data (on a Billion BiPAC 7800VDPX). As it turns out, the method I was using (URL encoding) had issues with sending the correct data through the web form. This is because "for a non-ASCII character, it is typically converted to its byte sequence in UTF-8".
So I tried another method by trying to apply the SSID byte data via ssh/telnet to the router:
sh wlctl -a wl0 down wlctl -a wl0 ssid -C 1 `echo -ne "\x53\x50\x53\x3a\x41\x55\x53\x01\x01\x63\x3c\x15\x0b\x07\x3c\x76\xff\x31\x07\x5b\xcd\x15\x9c\xb0\x07\x01\x01\x20\xd5\x01\x32"` wlctl -a wl0 upUnfortunately any values greater than 0x7F were coming back as what looked like UTF-8 encoded multibyte values (in fact, the SSID being returned was an invalid length of 38 characters with the above), but in fact they weren't:
0xff => (-17,-93,-75) 0xcd => (-17,-65,-67) <= SAME!? 0x9c => (-26,-71,-76) 0xc4 => (-17,-65,-67) <= SAME!?
This is certainly not standard.
After wasting a day on this, I decided instead to change the schema and coder hopefully for a last time in the interests of becoming close to 100% compatible with all devices. While the nodes support the old schema without problems, the process of getting existing devices to support SubPos should be trivial, rather than trying to hack them to work.
The new coder source is here.
I have removed the country code as it was slightly redundant (as well as the colon on the tag). I have also shifted the layout around to make decoding easier to read. I increased the range and precision of TX power values (most chipsets only give 0,1 decimal points of precision), increased the number of coefficient values and reduced the altitude bits to give a range of +-67km (deepest crust depth is ~50km and next smallest value possible is 33km; if you removed another bit).
The coding mask is now a two stage process, first we use the 21 effective bits out of 24 bits to encode out >0x7f values (by removing removing the MSB of each byte and placing it into the coded data). These 21 bits are spread over 24 bits so they never become greater than 0x7f, i.e. this shifting step automatically codes them out.
There are now 32 bits with 28 effective bits used to remove unsupported characters. This has to now cover a total of 168+24+32=192bits (the SPS data, 0x7f mask and the unsupported mask).
For the entire SPS dataset, we could do things in 7 bit chunks instead of the 0x7f mask, but it makes it harder to be neater as 16/24/32 bits factor to 8 really well. So we sacrifice processing speed for readability (which it turns out to be relatively simple anyway in the code).
Notice the the coding masks become neat factors of 7. Although we do lose 7 bits in the entire process because as we can't use the MSB bits (since it might violate the 0x7f rule), but this is unavoidable as we have no way to code them out (you would need more bits to code them out then those bits might violate the rule, etc.).
Interestingly, now that we have done this, we no longer have to use the hidden wifiSsid Android library and we can just use the SSID string directly as it doesn't have issues with UTF-8 coding. This will be good if they decide to deprecate it at some stage and might come in handy when developing the iOS App.
All in all I am pretty happy with this update, I had thought about it earlier, but I thought most devices would have been more standard than not. It turns out most things aren't programmed to correctly accept proper octet arrays as they should, but this method allows us to err on the side of caution when it comes to compatibility and hopefully will now work with nearly every device on the market today.