One of the most important factors that can contribute to a project falling into obscurity is - does it work as well for calls/SMS/data connections as a real phone would? Right now, it doesn't, and it's immensely important it does. I want to make it a point to have the GSM part, both in hardware and software, stable, and make it one of ZeroPhone project's promises. It's pretty much a pre-crowdfunding priority - that is, until I know it's possible to solve it without changing the project in some major way, I don't feel safe launching the crowdfunding. What's the plan?
[TODO: insert photo/gif of current phone UI]
First of all, the 2G modem works with ZeroPhone. I can send AT commands through the serial port to initiate or accept calls, as well as receive or send SMS. Apart from some bugs in the modem's firmware and an issue with the microphone noise (only affects current revision of PCBs), the hardware is accessible and working - but, obviously, you can't use the terminal to call others on a daily basis, and PDU encoding of messages is not that easy to decode by just looking at it =D Now, it needs GSM backend software to be usable.
Well, it turns out that writing a decent GSM backend is hard. I have already attempted doing it, when I was still working on beta boards - as a result, pyLCI has a small UI for making calls and receiving calls, and the backend for it. Writing a working base meant accounting for a lot of special cases, however - modem doesn't respond to AT commands instantly, you need to track its states, properly route URCs (info messages that can come from the modem at any moment, such as "OH MY GOD UNDERVOLTAGE") and handle them. All of that involves having at least one more monitor connected, with the modem's AT command documentation open on it, and Ctrl+F-ing a lot through English where you'd need a thesaurus to find anything, since you can't rely on the translator having picked the obvious term for the function you're looking for (and not going for the least obvious one, and misspelling it to in a way that makes your eyes bleed).
I implemented URC handling (and limited processing), and all was going well. But then, when it comes to, say, SMS handling, it all needs an SMS storage system, with a separate API, provisions for adding various hooks, as well as decent input UI elements for editing SMS (basically, writing text editor logic in Python - doable, but not as simple). A contributor was going to implement this, but then I assessed the situation and decided that telephony is one of the last parts that would ever be have to be written from scratch. For example, what about GSM data? Whatever GSM backend is used, when GSM data mode is enabled, the GSM backend needs to create a network connection, while still keeping track of when the connection is broken (from either modem or user side), so that it can resume handling calls and SMS messages (you wouldn't like to miss a call or an SMS because tethering was enabled!). Can this be done in Python, in a way that is quick enough to not induce nostalgy about times when dialup was still widespread? It probably can... Well, with the most important parts written in C, that's my guess. So, the experimental UI and backend will remain there - but only for testing purposes.
What about existing backends?
Most useful link (that, for some reason, I only found recently) is going to be the description of telephony stack that Jolla uses, and the most interesting from all of that is ofono. I've actually seen ofono mentioned in Hackernews comments about ZeroPhone, even back in January/February, so I've tried it back then, and couldn't launch it - the documentation for a "start from scratch" situation just wasn't there. However, I've subscribed to their mailing list, have been reading conversations and patches, and I'd like to think that I started understanding how it works, and about ZeroPhone-relevant aspects of it =)
There is also ModemManager. Their documentation is much better than ofono's documentation is, but they're not actually that well-suited for, say, handling voice calls - networking seems to be their stronger side. However, I'm still looking on its capabilities, as a backup plan - and reading their mailing list, of course.
I've asked a couple of questions on the ofono mailing list a couple of days before, and got answers that were more than satisfactory:
>> Hi! >> >> I'm interested in running ofono on a Raspberry Pi (BCM2835, 1GHz), connected to a SIM800 modem over UART (the stable Pi UART, not the MiniUART). I also have RST, DTR and RESET signals connected to the Pi, and for sound I'm using analog audio outputs and inputs of the modem. Basically, I'm talking about using ofono in ZeroPhone project - https://hackaday.io/project/19035 . >> >> For now, I'm interested in 1) notifications about voice call status (not concerning audio for now, as it's handled in hardware) 2) SMS and MMS support 3) 2G data connections (whatever available connection types are on 2G networks) 4) USSD support. My questions are: >> >> 1) Is ofono going to be suitable for my needs? I'm more than OK with using DBus and rolling my own user interface to it. It supports everything except MMS. That requires a separate MMS stack and is out of scope for oFono. I don't know of any full-featured open source MMS stacks, but I'm probably not paying enough attention. Maybe someone here can give you pointers. You can take a look at the (now undeveloped) mmsd project to see how the MMS stack should interact with oFono: https://git.kernel.org/pub/scm/network/ofono/mmsd.git/ >> 2) How do I run ofono from command-line and tell it to connect to a specific serial port and use a specific ofono driver? You write a driver for it and since it is a serial port modem, use udev rules to setup the TTY port accordingly. See doc/sim900-modem.txt for an example. The device detection magic is inside plugins/udevng.c and most modem drivers reside in plugins/*. See the plugins/sim900.c, it may be close enough to the sim800 to get you started. >> 3) Do you foresee any necessary changes to ofono? For example, need to make a sim800-specific driver? The most likely answer is yes. All new modems integrated into oFono required 'some' vendor/firmware specific handling. However, it is not possible to answer that unless you have the modem control protocol specifications in hand. If it is a standard AT command based modem, then the changes should be minimal. >> 4) I see that Sailfish phones are using ofono as the GSM backend. Did they upstream their changes, or do they have a fork? They have a fork. >> 5) I'm planning to work with SIM5320 modem in the future, and attempt using its USB connection. I know it creates multiple serial ports, and supports streaming audio over one of them. Does ofono come into play here, does it have some kind of provisions to convert this kind of audio input/output into actual sound devices under Linux, or is that usually handled by other software? oFono does not handle the audio bits. We can take care of sending the necessary AT commands to the device to configure the codec, etc. But the actual audio processing belongs in your audio daemon. E.g. PulseAudio or whatever. -------------------------------- >> Thank you, that was a great answer. I'll focus on one part for now: >> >> Is it possible for me to run ofono from command-line, without involving udev? I'm asking this because, in my application, I'm using the UART as a debugging UART for first 30 seconds of phone's boot, then, if there's no activity (or other trigger set), I enable the GSM modem and run the GSM backend. Is there a reason why ofono is not capable to be run this way (provided it's not capable of this)? Is there a way I can just do the setup that udev would normally do, but manually? If so, where should I look for pointers - code, documentation or somewhere else? (doc/sim900-modem.txt? plugins/udevng.c?) oFono can be started any time you wish. udev is just normally used to detect the hardware. Either via probing the USB bus, or in the case of serial devices via external configuration (e.g. udev rules). Without udev oFono would not know what TTY corresponds to the modem. E.g. on some platforms it may be ttyS0 and others it would be /dev/ttySAC0, or whatever. The oFono plugins / modem drivers are free to do anything they want. So if you want to hardcode the behavior, that is fine. You can always implement your own detection logic. For example plugins/phonesim.c uses a config file, plugins/rildev.c uses getenv. The possibilities are endless. udev is the preferred approach, other approaches might not be palatable to upstream. In terms of documentation or pointers, it really depends on your hardware setup. If you have a traditional AT command based modem with a multiplexer, then take a look at doc/calypso-modem.txt and plugins/calypso.c. This is the modem used on the original Freerunner. If you have multiplexing handled by the kernel, then the SIM900 docs / plugin might be applicable.
So, now at least I have an idea on how to launch it. What's more, ofono even ships Python code examples! The plan is as follows:
- Make ofono run with a known-working modem, see if it's fully compliant to what ZeroPhone needs, determine requirements for mobile data support
- Write a driver for SIM800 (can be based off SIM900 driver which already exists for ofono)
- Write the pyLCI-side UI for calling
- Add SMS and call history storage - either use an existing one, or roll our own (only in case of fundamental incompatibility, and borrowing as much architectural decisions as possible)
- Add UI elements for SMS viewing and writing
- Add SMS support to ZeroPhone
- Work on mobile data support
- Work on hooks (allowing users to add features like automation, blacklists, remote control, etc.)
What about 3G? The 3G modem I'll likely be using for the 3G mode board is SIM5320, it's slightly worse than SIM5360 but it has analog audio built-in (which ZeroPhone currently uses), so I don't need to add additional circuitry. With a single UART, it's going to be trickier to make it work - if you have a 3G modem, you'll definitely want data connections to work reliably! However, SIM5320 has a USB port, and it exposes not even one, but 3 UART connections over it - so, provided that SIM5320 will be connected through USB on ZeroPhone, the software should work with minimal changes. Of course, that also means having to add a USB hub chip onto ZeroPhone (or a SPI-USB host chip, like MAX3421E), which will mean slightly increased power consumption and inability to use the USB port in slave mode (not in the SPI case, though). I'll see about whether there's a better way to implement all of it.
About those hardware glitches I was talking about. First problem with the current GSM setup on ZeroPhone is that the GSM microphone, all of a sudden, has a lot of noise with the current boards (the previous revision didn't have any noise). It's either a result of changes that I made to microphone traces, or that decoupling is not good enough. So, next revision will have better microphone traces and more advanced decoupling, too - once ZeroPhone gets a landing page, I'll use a @zerophone.org address to finally be eligible for free samples, and sample some better capacitors =D
Second problem is that the modem doesn't always register when the call has ended, so, say, the person you're calling has hung up, but the modem remains in sort of a "limbo" state, still thinking that the call is long finished - and not accepting any commands to finish it, either =( It might be solve-able with a firmware update - thankfully, there are Linux tools for updating the modem, and the update protocol is documented, so, provided that there's a firmware image that works well, there's even a possibility for fwupd compatibility in the future.
Also, one more popular question - what about open-source GSM modems? Well, stay tuned - soon, I'll be describing one promising open-source (but not libre) option that is suitable for ZeroPhone usage, its licensing status and what's going to be done to make sure it works with ZeroPhone =) It'll most likely be a part of long awaited "licensing status of ZeroPhone components" worklog.