Introduction

In the late 1980’s Bill Gates founder of Microsoft designed and implanted an integrated media centre solution into his newly built home near Lake Washington, which used small pin badges to detect the presence of users and re-direct the media accordingly to the nearest appliance as the user moved around the home. Unfortunately there it is little literature to support how Gates’ achieved his project, however Gates (1995) published a video documenting the media centre, which is available on YouTube[ CITATION Nig07 \l 2057 ]. This project was coved in a number of publications at the time. U.S News (2012) reported that Bill Gates had completed construction of his house at cost of $100 million, and detailed that it had a number of features such as high-definition television monitors that displayed art digitally, computer controlled music, lighting and climate settings that are linked to individual users location within the mansion. Again, the article did not state how this was achieved.

Since 1997, when the Gates home was completed, technology has significantly advanced in a number of linked technologies such as computing, televisions, radio-frequency tags, CCTV cameras as well as media centre technology. Alongside these developments many users have cited privacy concerns, relating to some detection methods, such as technologies processing images taken, tracking movement and even logging played content on the home automation platform. A further question is whether a user would want the fact that they’re out of the house recorded by others.

Part of the literature review will focus on possible ways of reliably detecting room occupancy. This can be achieved in a number of ways, such as radio sensors which users must carry, face detection, movement or co2 sensors. This Literature review will offer a critique each approach, as well as looking at how the technologies are applied in different settings. The literature review will then go on to focus on media centre technology, possible hardware solutions and effective ways to synchronise all aspects of the project to allow the Media Centre to act as one.

Ways of Detecting Room Occupancy:

Radio

Sensors:

There are existing projects similar to the Bill Gates project (Gates, 1995) that use pins or devices to track room occupancy for the purpose of redirecting media content to the nearest appliance. Linux MCE (2015) is a whole home automation suite that uses also uses radio sensors, namely Bluetooth, as a method to track users around the home from mobile devices. There are limitations associated with Bluetooth, namely the inability to obtain signal strength between two devices around without first paring them together using standard hardware. On the other hand, there are some devices that are able to communicate to all Bluetooth devices without paring, such as Ubertooth (Chai, Deardorff and Wu, 2012) which allows the interception and transmission of Bluetooth packets from and to any mac address, as well as to detecting the single stretch of remote devices. Ossman (2015) has developed the product Ubertooth to allow users to directly access the Bluetooth protocol Hardware such as the Ubertooth One makes tracking Bluetooth without paring possible without excessively expensive hardware Hay and Harle (2009) discuss the process involved, in their paper, and conclude that connection-based tracking which permits tracking of a previously identified handset within a field of fixed base stations is a viable alternative to inquiry-based Bluetooth tracking. Although the paper is from 2009 the research is still relevant today.

Although Bluetooth tracking could be seen as a convenient approach for tracking individuals, this would not be suitable for this project due to usability issues. Users would be required carry a phone or an object to be detected as occupying a room. Although this approach suitable for the project discussed in this paper, there are commercially available products.

Tracking in Retail:

There are a number of products that have designed to allow companies to track shoppers uniquely using the signals from their Bluetooth device such as phone. Specially designed hard and software is used to triangulate users as they move around an environment such as a shopping centre. Solutions like Rapidblue Solutions (2014) offer real world analytics about to customer shopping patterns and their product uses a number of listening devices to triangulate a mobile phone as they move around a retail outlet. Case studies by Rapidblue show that through using the shopper analytics platform companies can increase revenue for shopping centres clients by understanding the flow of consumers between retailers in a shopping centre. Again, this approach will not be adopted in this project as it requires the user to carry their mobile phone at all times.

The problem of identifying individual customers with in the retail space has existed for a number of years and has been addressed in a number of ways. Retailers have historically used loyalty cards to track the buying habits of their customers. If a customer presents his or her loyalty card during the transaction they receive an incentive, the data collected is then calculated to make very accurate projections on the demand for personal items. This data allows retailers to better reduce spoilage. Companies such have Tesco’s have used incentives cards such as the club card from as early as 1995 to collect buying habits [ CITATION Roh06 \l 2057 ]. Through using buying habits data retailers such as Tescos, are able to make informed decisions about targeting promotions to a selected group of people or predicting how much skimmed milk is needed for the upcoming day. However this method of tracking does not aid in the process of tracking customers around the store. Tesco are moving to further innovations in the retail space by using “facial recognition software at some 450 of its petrol stations in the UK” [ CITATION Mil15 \l 2057 ] with the directly targeted adverts on the video screen in the store changing based on their gender, age or another unspecified characteristics. Tesco’s are using an Amscreen product called OptimEyes (Huffington Post, 2013) that only collects statics and allows targeted adverts, addressing concerns that users may have when it comes to privacy in terms of recording images.

Optical

Detection:

Following in the footsteps of retailers the literature review will move away from looking at radio sensors as a way of tracking users to looking optical recognition. There are a number of different approaches that can used process an optical images to detect room occupancy within an image. These methods include tracking movement, lighting conditions or using a process to count the number of human faces in an image taken. There are a number of commercial products that perform the task of processing human faces, as well as Opensource library which is able to detect shapes and patterns. In the following paragraphs existing products will be critiqued to see if a suitable solution can be found for this project.

OpenCV

OpenCV is an Open Source project that offers a number of simple really time image processing libraries, it was first developed by an Intel Research Initiative (Bradski and Kaehler, 2008). and is written in C++. There os also an API that can be accessed using Python, Perl and Ruby (Bradski and Kaehler). OpenCV can be used in a number of ways from stereo correspondence to eye and mouth tracking and even gesture recognition. OpenCV also allows for face recognition similar to OptimEyes (Huffington Post, 2013). OpenCV is open source which allows developers are able to modify and extend the sourcecode allowing for innovation after innovation.

When compared to other comparable products this would be a suitable way of detecting room occupancy for the purpose of this project, as researchers Belaroussi and Milgram (2012) have already investigated the success and failure rate of OpenCV’s face tracking and facial recognition libraries and came to the conclusion that OpenCV only had a small, acceptable percentage failure rate. The study also investigated the processing power required to detect faces in a real time environment. One of the key issues highlighted by Belaroussi and Milgram is that “lighting conductions can dramatically change a faces tracking process” and the orientation of the subjects face also has an effect on the success of positive face detection.

Existing

work (Bill

Gates project)

Linking back to the original inspiration for this project, due to limited literature on Bill Gate’ (1995) project, it can only be speculated as to the technologies used in the original project based on available products of the time. For comparison purposes it would be interesting to know what technology was used for the Gates project for the purpose of using similar the modern day alternatives to create a system that works in a similar way. The only different comparison we can conclusively draw is that we need a way to track unique individuals around a large environment. The next part of the literature review will compare possible modern approaches and technologies that could be used in a modern system of this kind, which are within the budget constraints of this project. Following on from this a conclusion will be drawn as to how the improved media centre automation system that will be developed should be deployed and what hardware, software and technologies should be used to fulfil the requirements outlined in the project aims and objectives.

Existing

work (LinuxMCE Home

Automation Project

One comparable product to the project that will be created is LinuxMCE (MCE) which is based on Ubuntu. The product contains everything needed to setup a home automation system. This system supports many of the modem automation protocols such as X11 and Z-Wave. What makes LinuxMCE noteworthy is that is that is outdated thanks to developments in both hardware and software. The system uses Windows CE or XP tablets that allow the user to control their home automation system and media centre from a single location. MCE has also an implementation that allows the computer or tablet to detect the proximity of Bluetooth device. This feature is called “follow you” and allows lights and media devices to be turned on and off, but this does required Bluetooth to be turned on and the computer or tablet to be on your person. As mentioned previously with other Bluetooth devices these requirements make it unsuitable for this project.

RFID/NFC

In relation to the Gates project (1995) it can be hypothesised that some form of RFID technology such as a product called HITAG by Philips that was commercially available around the time could have been used in the home automation system at the Gates mansion. As early as 1996 the HITAG [ CITATION Phi98 \l 2057 ] used protocol and technology simpler to modern RFID/NFS. These small tags could have small amounts of data written to them and contained unique identifiers that could be read from a distance of 200 mm to 1000 mm according to the data sheet [ CITATION Phi98 \l 2057 ]. It’s fair to theorise that a network of devices such as these could have been deployed in a grid layout to accurately track the movements of users around a large environment, this theoretical approach would most likely be expensive but the resulting accuracy and redundancy would most like offset the cost when taking into account that money wasn’t an obstacle in the Bill Gates mansion, but cost is a consideration in this project.

At present products available include RFID tags (Amtel, 2005) and NFC tags (ECMA International, 2013). These are identification tags that can be read using passive tags using different low frequencies, both forms of these tags are powered using an inductive style. RFID contains a hardcoded ID, which would make them unsuitable for this project, whereas some NFC tags are writeable or can implement cryptography. NFC tags can contain a number of different types of data such as phone numbers, email/addresses, business cards even URL addresses (ECMA International, 2013).. This feature of NFC is useful for the project outlined in this paper because the NFC tags could be read by a mobile device and points a user’s device (Phone, tablet or reader) to a specially designed URL that could instigate playbacks of a certain film or TV channel. NFC and RFID tags need to be within a range of 20cm to be able to be read, although there are products that use specially designed to be read as far as up to 10 meters away. IPICO have developed a product called Sports Tag (Active Network, 2015) that have been developed for the purpose of communicating timings for sporting events by reading RFID tags attached to runner shoes

Another point for consideration is that researchers have been able found ways to extend the range of NFC and RFID tags. Paget demonstrated one of theirs approaches at (Defcon, 2009) tags by creating “RFID Door Frame Skimmer” with attached the RFID reader antenna to a door frame allowing every tag that passes through door to be read. Although this would be an accurate way of tracking user movement around the home, the price of the components would be out of the budget for this project.

Media Centres

Although the purpose of this project is not to simply create a media centre, there will be a high level of integration required to control the media centre, therefore the software used needs to be taken into consideration. The end product needs to query information from the media centre as well as compare this information with sensor/data, then instruct the media centre to change its state.

Samsung

Samsung currently offer a number of unorthodox control surfaces or their latest range of smart TV that includes voice control and hand gestures (Samsung, 2009). Although theirs approach works sufficiently well under some circumstances, it still requires direct user input, as well as requiring users to see and read the feedback from the media centre.

XBMC

As the media centre is not the main focus for the project there the literature review will only briefly mention the research studied to underpin the decision to XBMC (Kodi, 2015). From the literature provided about XBMC on the Kodi website it is suggested that media centre supports an extensive set of API integration using JSON, as well as having the ability to run on a wide range of devices from a PC to an Apple TV and on Raspberry Pis.

Frontend Hardware

Another consideration is that the hardware for this project must be durable as a home is a twenty-four hour setting. Power consumption and stability are important factors as well as a responsive user experience. HD Video playback is also sensible requirement as a majority of modern televisions support HD content, as well as supporting audio over HDMI to allow for the use of the built in speakers. Following research a further key requirement for the media centre hardware is that the frontend must have CEC support (HDMI Licensing, 2013) to allow the automated media centre to control the television. The below list contains 3 different options for frontend platforms with their specifications:

• Raspberry Pi with an 800 MHz processor and 512mb memory costing approximately £35.
• Shuttle XS35GS V3 with Intel® Atom™ Processor D2700 with 128 GB SSD and 4GB of ram built in Wi-Fi (+ USB - CEC Adapter + £30) = £600.
• M8 Amlogic S802 Quad Core Android TV BOX, with built in wi-Fi (+ USB - CEC Adapter + £30) = £129.99.

All of the options apart from the Raspberry Pi require an extra device (USB - CEC Adapter) to allow for interaction with the television CEC. The Shuttle and M8 both require power adaptors ranging from 12-19 Volts at 2 amps. The Raspberry Pi only requires between 700-1000 mA, some power supplies that in theory can be supplied by the TV USB service port, or even PoE using (POE: http://www.raspberrypioneer.com/2013/06/13/how-to-run-your-raspberry-pi-from-poe-for-under-15/) as developed by Peter (year) using off the shelf parts allowing a Media Centre to be powered using only one network cable and a PoE injector on the other end. The Raspberry Pi is the only device from the selection that does not have Wi-Fi built-in as well as benign slowest device when compared to the other devices under consideration although all devices have the same speed 10/100 mbits network devices.

From the comparisons drawn the Raspberry Pi hardware platform is a well suited platform for this project, even considering its drawback in regards to connectivity and processing power. This is because it meets the budget constraints of the project, without compromising on meeting key project outcomes.

Conclusions

After reviewing a range of modern, available technologies and approaches a selection will be made identified based on cost and accuracy. The main goal of this project is to instigate similar approach to that of Gates (1995) which is affordable to the mainstream consumer, whilst still having the same sense of automation. From the research discussed in the course of the literature review it can be concluded that there are number of ways to detect occupancy of a room. Having critically reviewed the positive and negative aspects of ways of tracking room occupancy this project will use face detection as a means of establishing room occupancy. A further positive aspect of this method is that it does not require direct interaction from the user. From the research discussed there can be a number of considerations to ensure effective face detection these will be evaluated in line with research in the methodology section of this documentation. From research into Bluetooth communication, RFID and NFC tags this project will focus on using NFC tags for the user to control the media centre. The aim is to mitigate problems and better the user experience for those with eyesight or literacy issues.