With the hardware decided on I looked to making a mobile application to control the different phone functions. Since I was using a Serial Bluetooth device, my app would have to process that data and then call the commands on its own, instead of the bluetooth module having the power to do this.

Features

My goal for this project is to make it so the user can do as many things with their phone without having to look at their phone as possible. The features that I had in mind were:

• Controlling Music
• Making and Receiving Calls
• Handling Texting

This is a large array of features and are among the most common distractions while driving. Before I started implementing these features, there was one big question to answer first...

Android vs iOS vs Windows

Since i'm making a mobile app, I need to decide what systems my app is going to be on .

Android is pretty easy to get up and running with. You can develop for Android on Windows, Mac, or Linux OS so no problem there. The Android SDK is completely free and you can develop using Eclipse or Android Studio (both are cross-platform). To put your app on the market costs a one time fee of $25 which is not much.

iOS poses a couple of problems for me. For one the iOS SDK is only allowed for use on a Macintosh computer. I don't have a Mac to use so I already cannot develop for iOS. If I did have a Mac the iOS SDK is free to use, however in order to test on a physical device I would need to pay a licensing fee of $99 per year.

Windows is fairly easy to get up and running with. To start developing for it, you need Visual Studio (which the community version can be downloaded for free) and the Windows Phone SDK (which is free). Visual Studio is only available for Windows (Visual Studio Code will not work). If you're on Mac you can use BootCamp to run Windows and develop on there. Otherwise you can use VMWare or VirtualBox to run a Virtual Machine and develop using that (which is available for Mac and Linux). To deploy your application on a physical device, you only need to register your device (which is free). However to deploy to the Windows Store you need to apply for a Developer's License. The Individual License is most likely what I would use, which only costs $19, however it also comes with limited features, but thankfully the limited features are more for Business Applications and this project won't utilize any of them. The only problem with Windows Phone Development is that I don't have access to a Windows Phone. I can use the simulator, but I wouldn't be able to physically test it with my device.

There's also a fourth option of using a multi-platform tool like Appcelerator. However this still doesn't solve the problem that I wouldn't be able to test on an iPhone Device since I don't have a Mac computer, and I wouldn't be able to test on a Windows Phone because I don't have access to one.

As a result, I ended up settling with Android.

Controlling Music

With the platform decided, the first feature to implement is being able to control the music. Since i'm using gestures, I want to try and make the gestures as intuitive as possible. The different gestures that are available are UP, DOWN, LEFT, RIGHT, NEAR, and FAR. For controlling the music the different features we want to use are PLAY, RESUME, SKIP, and PREVIOUS.

If we look at a generic music player, we'll see a couple of things that could give us hints as to which gestures we should use. The Skip Button is facing toward the right and the Previous Button is facing toward the left, so it would make sense if these two buttons were mapped to a RIGHT Gesture and a LEFT Gesture. The play button is in the center so any gesture that is in the middle would work out. So our options are UP, DOWN, NEAR, or FAR. I'm not super sure which one to pick so i'll leave that for later.

Another thing to consider is what music we are playing. To play music that is saved onto the device, we can just use the default Music Player, however a lot of people use different music players such as Spotify, iHeartRadio, or Pandora so it would be advantageous to implement one of those as well. I decided to use Spotify because I am most familiar with it. With Spotify there is one other feature that the default Music Player doesn't have, which are Playlists. We would need an gesture to shuffle through the different playlists.

Looking at the playlists on Spotify, they're layed out in a vertical order, so switching between playlists would make sense with either an UP Gesture or a DOWN Gesture. That leaves the NEAR or FAR Gestures for the Pause/Resume actions. I did a little bit of testing with the FAR gesture and it is a little inaccurate. Sometimes if your hand goes back and pass through it the sensor will think you meant a FAR gesture. So the NEAR gesture seems the best for Resume/Play action. With the UP and DOWN Gestures we can use these to go to the next playlist or the previous playlist in Spotify.

These actions are very generic so if later I needed to implement iHeartRadio or Pandora I could easily do it. All I would need to do is update the app.

Making and Receiving Calls

Receiving phone calls is actually pretty easy. The only actions we need to handle are:

• Accepting a call
• Declining a call
• and Ending a call

Declining and Ending a call are actually handled the same way, so we really only need two gestures to handle this.

Looking at the incoming phone call screen, we can see the user has to move the middle button either left or right to decline or accept the call, so using the same gestures would be very intuitive. We are already using these gestures, but the user won't be listening to music while they're on the phone (hopefully!) so these gestures are essentially available!

Making a phone call is much more complicated. The user would do a gesture, and then need a way to call a person from their contacts. One way we can do this is to use Speech to Text and make the user say which user they want to call. A couple problems arise from using this method though:

• Speech to Text could not be able to translate what the user said
• If there are contacts with the same name how can the app distinguish which person to call

The first problem can be solved if the user messes up, then just prompt the user again. The second one we would need to prompt the user again saying that there are multiple entries and tell the user the names of each entry. Then the user could use Speech to Text again to call the correct user.

As far as the gestures go, we can use the same gestures for the phone call as we did for recieving a phone call, but we need a new gesture for being able to call a user. Since the user could make a phone call at any time the only have the inaccurate FAR Gesture. In order to add in this feature, we need to forgo a gesture from one of the music functionalities. Looking at the features we absolutely need Resume/Play, Skip Track, and Previous Track, so that leaves us with changing playlists with UP and DOWN Gestures. Looking at it, the user will probably only have a few playlists so having the ability to navigate backwards and fowards is not a huge necessity. I chose to use the DOWN Gesture for this.

After testing, I noticed that the DOWN Gesture is very easy to accidently do in the car because are arms tend to move down. What I ended up doing was making the user do the DOWN gesture twice in order to intiate the Speech to Text prompt. With this, its much more reliable and the user won't accidently trigger the prompt.

Handling Texting

The last thing to do was to add texting functionality. I had a hard time trying to figure out how to implement texting functionality so I looked at some other devices that implemented this feature for ideas. These devices either use Text to Speech and Speech to Text for handling text messages or display the text message on a screen in the car. The idea of my project is to make it so the user doesn't have to look at a screen so that's out. I tried using a device that used Text to Speech and Speech to Text for handling text messages and it just didn't work that well. Reading a text message was fine, but Speech to Text for an entire text message rarely worked out how the user wanted it to. I decided to hold off handling texting for later since I didn't have a great way of implementing it.