Intro

The HandHolo project uses a bluetooth mouse that connects with an Android smartphone with Android O. A Unity app translates the mouse movements and the phones gyroscope to an explorable scene which is projected on a mirror/transparent plastic sheet.

The mouse provides positional tracking, and the gyroscope of the phone the rotational tracking. You move the device to view different sides/places of the AR scene.

Demo video: 

Steps

This is an easy and fun project, especially if you're a game developer of sorts.

There's only a few steps:

  1. Check your phone is on Android O. Android O has support for relative mouse movements with cursor lock.
  2. Acquire a Bluetooth mouse (compatible with your phone, but should be most of 'em). Do not use a flat design mouse, rather one conventional. Not too big, not to small, just plain. Cheap mouses work, but tracking is better if you spend a bit more.
  3. Create the HandHolo cutout out of cardboard, and find a piece of reflective material for the screen. Acrylic plastic works (darker reflects screen better) or another mirrored surface. 
  4. Fire up Unity, import the project from Github. Else just include the scripts and set it up manually.
  5. Connect the mouse. Turn brightness to max and disable screen filters. Insert mouse with buttons facing hand, so reversed. Run the Unity app.
  6. Finally tune the mouseSpeed variable on the CameraScript to match real world movement to hologram movement.

Cutout

The cutouts dimensions are described on this picture, where the lengths are measured in cm:

Total cardboard area is 28x29cm. The longer broad bottom part is where the phone rests on. There are two incisions on there of 3cm, where the topleft and topright extensions (2.5cm each) insert into.

The right flap next to the phone rest, is the part that holds the screensheet upright. It is folded so that the 2 flaps (bottom-right) insert into the other 2 flaps (bottom left, 1cm) and can be strengthened with some tape.

The bluetooth mouse goes into the mid top. I made the mistake here by using a very flat designed mouse. The 2 flaps (9cm) hold the mouse in place by two folds that have a spring because cardboard is just not very bendable. The width notated as 4.5cm between these 2 flaps is a bit more than the width of your mouse.

Also, any other places where you have trouble folding, cut an one-sided incision along the outside of cardboard fold. Don't do this for the folds holding the mouse; these need to spring.

Attach the screen sheet with some glue or double sided tape. I used a pure mirror, a dark transparent acrylic sheet gives the best holographic experience.

Result:

Unity description of scripts

- FlipCamera.cs: Applied to the Main Camera. Flips the camera image so if it reflects on the screen sheet it will be correct.

- SelectionScript.cs: A cursor selection square demo that can be used with the left mouse button (right when inserted in cardboard).

Needs a cursor quad and a quad for the selection grid. Doesn't do any selecting of units yet however.

- CameraScript.cs: The script that interfaces with Android to call for a pointer lock, and catches the relative movements, as well as gyroscope.

CameraScript and SelectionScript are supposed to be parent of the Main Camera. Place the Camera so that the distance to focus point is about the same as distance from face to screen, and looking down in the same angle (~-45 degrees).

The Unity files can be found on my Github:

https://github.com/Kjos/HandHolo