Close
0%
0%

Grandma-proof face-controlled telepresence

Project focused on enabling a caller to move the camera around whilst holding a video-conference call using only a phone

Similar projects worth following
Nowadays, a lot of video calls are made via Watsapp, or face-call or whatever you're used to on your side of the world. These tools are great since they're simple enough that they allow an older person to use them easily. Nevertheless, if we want to enable them to "look around the table" or "look my way" whilst we're cooking, we need something really simple that can be controlled remotely with no additional hardware or middleware.

This project aims at bridging the telepresence gap at a low cost and complexity by adding basic telepresence functionality to a phone or table without resorting to:
* Heavy lifting SBC such as Raspberry Pi or PC
* Internet connection
* Machine learning
* No remote control interface

The project was inspired by a robot featured recently in Hackaday.

It is by no means a finished product, but it goes to show telepresence implemented using readily available components.


Telepresence is really cool. Being able to have my mum, who lives some 7000km away, look in the direction of one of my kids whilst we're on the dinner table is just priceless. Nevertheless, the solutions available are either expensive, require a lot of complex ML trickery or a remote control. Each one of this solutions has it's pros and cons, I'll focus on the why I didn't want any of it below.

Telepresence robots

They're so cool. Put simply, I cannot afford one.

AI-driven solutions

There are really great projects out there which capture images or sound, analyse them using ML algorithms and trigger an action on the remote host. These solutions are really fantastic, but they rely on a computer to do the heavy lifting, some of them need to interface with platforms that sit outside our control and are hard to implement from my POV. Also, people don't always have a spare computer or Raspberry Pi to leave on the telepresence robot to digest the input.

Remote control

Personally I think this is the worst of all the solutions. When calling someone on the phone, I need the solution to be operated remotely without resorting to another machine, or another app on the phone. This is a must since it's not so easy to ask someone to go fetch a laptop or a tablet to control the robot.

Also it's hard to come by any APIs to for example overlay remote control buttons over the Watsapp screen.

The simplest solution

First, the constraints:

  • I only have LDRs
  • A couple of Arduinos and a few NodeMCUs
  • I'm not very good with motors
  • I don't have a 3D printer
  • I don't have a workshop

Following upon the robot project from a recent Hackaday blog post, I realised I could maybe use the image on the screen to control the robot only by looking at the brightness of a certain area of the screen. This idea came from a different project of mine that used LDRs to measure the intensity on a screen to recognise numbers.

The objective was to put two LDRs, one at each side of the screen, and check the brightness using a photoresistor in a voltage divider configuration.

If the edge of the phone changes brightness, i.e. lighter or darker, like for example when someone moves their head to the side, the phone will turn in that direction. The result can be seen in the video below with a dark background one one side and a lighter background on the other side.

This solution has its limitations, namely:

  • The background has to be lighter than the caller, but I'm planning on improving upon this.
  • The caller cannot move around. If the brightness in the background changes, then the phone will start moving.
  • This version can only turn around the Z-axis, but I'm already thinking of ways to get this off its place.

I know, another tele-presence project, really? But this one is different.

circuit.cddx

Circuit Diagram file.

cddx - 3.05 kB - 04/27/2020 at 14:15

Download

Stepper_Sensor_BD.ino

Updated Arduino sketch. This one allows for the background to be either lighter or darker than the face of the caller.

ino - 1.59 kB - 04/27/2020 at 11:34

Download

Stepper_Sensor.ino

Arduino file that controls the motor based on the input from the sensors

ino - 1.42 kB - 04/26/2020 at 23:49

Download

  • 1 × Arduino Uno R3
  • 2 × Resistor 1k Used for the voltage divider
  • 1 × 28byj-48 Stepper motor
  • 1 × uln2003 Control board
  • 1 × Lego and Duplo

View all 7 components

  • Build instructions

    ciplionej04/27/2020 at 12:38 0 comments

    The goal of this report is to describe the instructions to build the telepresence platform.

    Working principle

    The working principle is: as the caller moves, it obscures (or lightens) an area of the screen which is being sensed by the LDR. As the LDRs detect a change in brightness on the screen, they change in resistance. As the resistance changes, a voltage divider transforms the change onto a voltage change which is then read by the ADC on the Arduino.

    The Arduino then compares the change in voltage with the calibration values upon startup and if there is a change bigger than a 0.2V offset, instructs the stepper motor to move in the direction in which the caller moved.

    Circuit diagram

    The setup is included below and was made using Circuit Diagram Editor.


    The power source was made using two 18650 batteries, though any other solution will do.

    Platform build

    The platform was build on Legos and Duplo, but any other solutions can be made that would also work.

    The platform is divided into the following functional components:

    Rotating platform

    This is holding the phone in place, the sensors and rotating as the motor is activated.

    An interesting finding is that the shaft from the stepper motor fits snugly into the hole of a Lego. The fit is so good that once the platform reaches the edge, the shaft slips safely, preventing the whole contraption from falling apart.


    Phone support

    This is the support proper made out of a brick wall of Lego on which the phone is supported. On the same wall, the sensors are mounted and positioned as below.

    Putting it together

    The rest is really some parts to hold everything in place. This is by no means a finished product, but it goes to show that telepresence can be implemented very easily using low cost components widely available and a simple Arduino sketch with no connection to the internet.

View project log

Enjoy this project?

Share

Discussions

Saabman wrote 04/30/2020 at 03:39 point

that’s really neat. 

  Are you sure? yes | no

Dan Maloney wrote 04/27/2020 at 20:09 point

In the video it looks like the camera is panning in the opposite direction than it should to keep the subject centered. Am I missing something?

  Are you sure? yes | no

ciplionej wrote 04/27/2020 at 22:26 point

Very good question and now I realize that the objective of the project can be taken in two ways.

The goal of this project is for the remote caller (A) to be able to control (pan) the camera of the other person (B) remotely by moving to one side or the other side of the screen. In the video, the remote caller is the mobile phone on the right and the camera being controlled is the one on the left.

Another implementation for this would be having the local camera (B) pan around to follow the (B) caller. This was not the goal of the project but it can be made to do this as well, but it'd be tricky. The reason for this is that this is a simple but really dumb system. The background serves as a reference and once we move the camera, we lose that reference. We could wait for the stepper motor to stop and then take a new reference snapshot from the new location. Nevertheless, if the subject is still on the edge of the image, the system would behave erratically.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates