VR/AR interaction devices

Wand and Haptic Exoskeleton Glove Devices for VR/AR IO.

Similar projects worth following
This project will aim to be 1) A tutorial to create some devices which could be used for different types of interactions in AR/VR 2) A Starting point for those who may want to build their own unique VR/AR input device .
This project is built keeping in mind the user interactions to be enabled at an affordable pricepoint.

This project will be further extended to include applications in telerobotics and motivate alternative methods for robot control and programming.

Problem Statement:

An immersive experience in Virtual Reality requires a method of input that is intuitive and natural. There are two components to this requirement- use of natural gestures and movements to represent the virtual act and some kind of haptic cue as feedback to the user.

This project will be further extended to include applications in telerobotics and motivate alternative methods for robot control and programming.

My solution:

I am going to attempt to make 2 types of devices:

1) Wand Type Controller

2) Exoskeleton Glove

The user will typically hold the wand in the left hand and the glove in the right hand. (Right handed user)

I believe this kind of hybrid scheme will offer features that are not possible with only wands, joysticks or gloves. Refer to DesignDoc1 for more information and justification on how my project addresses the problem statement.

How will I do it?

I intend to primarily use IMU's for tracking. I do understand that errors in IMU's can accumulate very quickly when it comes to position tracking. However I suspect that we can still get data that will allow us to guess user intent for the interaction, this data may be orientation or acceleration data which you can use in a clever way to define an "action or move" in VR.

See Potential Problems for more discussion. I posted this text here because I don't want you to read IMU and then abandon reading the rest of this project which might still be useful...

I will use a DC motor on the wand and probably a motor-spring combination for the glove impedance control. The advantage of using IMU is that it is computationally less expensive than computer vision methods. This makes it more suitable for mobile VR applications. See project log 3 for initial thoughts on glove actuator design.

Distinguishing features:

1) Force Haptic Feedback on glove and simulation of squishy or elastic virtual objects and interactions

2) Use of Hybrid Control Scheme for VR input ( all others use only glove or only controller)

3) Suitability of mobile VR as well as PC/Console based VR.

4) Cheap, Open Source and Easy DIY

I think this project will be useful to:

This project will hopefully be useful to novices in this field. Another reason for keeping it simple and low-cost is that I want to involve more people with a diverse background. Hopefully this will provide some relevant material for Artists,Product Designers, Game Creators and different types of engineers.

So if you have a cool idea for VR but the current set of available controllers is not ideal and you need another set without having to design every component yourself OR if you want to spend time on your own idea of the ideal combination for VR input but need some starting input then this project may be useful to you.

I can also think of some Non VR/AR applications in field of assistive technology, new training methodologies etc. For example: with the glove you can use it to teach someone to play the flute. You can take a song and preprogram a key sequence for the fingers. Thus the learner will get cues while playing the song. This is useful because being able to play a song somewhat well in the early stages of learning will be very encouraging and will keep the person motivated in the early stage when they are most likely to give-up.

Possible Problems:

I have not made a similar project before but IMU drift may be a problem.

Possible Solutions and Future Projects:

I believe an external on-the-fly re-calibration system using a leap motion device or the light-house type system could be used to periodically obtain a more accurate position of the system to correct for the drift. I do not yet know how exactly to do that but I think this could be possible by getting the leap motion to recognize a feature on the device as a tool and then using that reading for the accurate position thereby resetting the error periodically.

Valve which is behind the light-house system intended to make this technology available to hardware manufacturers. From what I understand...

Read more »

JPEG Image - 5.15 MB - 06/04/2016 at 23:16


JPEG Image - 2.68 MB - 06/03/2016 at 20:46


JPEG Image - 2.53 MB - 06/03/2016 at 20:46


JPEG Image - 2.19 MB - 06/03/2016 at 20:46


JPEG Image - 2.80 MB - 06/03/2016 at 20:46


View all 19 files

  • 1 × Arduino Micro
  • 1 × Lithium Ion Battery 3.7V 1200 mAh
  • 1 × IMU BNO055
  • 1 × Bluetooth Transmitter
  • 1 × Wires

View all 9 components

  • Wand Fabrication Day2 and Modular Design

    Gaurav Ghorpade06/03/2016 at 20:42 0 comments

    Most of the parts seem to fit together. I made a silly mistake with one of the dimensions as I do not use the arduino in the CAD and I did not notice one of the assembly issues .

    Overall it is pretty compact and I think this is ok for a first prototype. Improving the form factor could be a good project to build upon. This will be atleast partially a job for someone interested in electronics as we might want to move away from using an arduino and breadboard to something more compact.

    You will need a drill and drill set for the assembly. Most of the features used to align and fix the pieces are cylindrical in nature to allow you to make the small adjustments with a drill( as opposed to a mill)-this is required when the holes don't print perfectly.

    I am also going to have to re-design the IMU mount as I have a new idea to mount it which can result in a smaller device.

    I am attaching some pictures which will help explain the assembly ( full instructions will be available only after I reach a somewhat final design. I am posting these updates because some of you may want to suggest improvements on the design). My hands are shown for reference. (I have small hands).

    For the modular design of the wand, it will be more than just swapping out buttons or components. I am thinking along the lines of attaching additional frames. So for example, you can take the wand and take a frame for a two handed submachine gun and attach the wand as one of the handles for the gun.(We will now use the orientation data from the wand which is more reliable).

    Then because our exoskeleton glove can haptically render squishy features, it can be used to create a virtual trigger(haptic feedback) -(not possible with braking mechanism).

    So why do we need to attach that frame? Because when you hold a physical gun, it places constraints on your hands. I.e. Both hands need to be at a fixed distance from each other in physical space. If we do not have a frame to grip with both arms then there will be a mismatch between what you feel and what you see. This is not only bad for immersion but creates real problems for input. Since the gun frame is supposed to be rigid, both palms end up with a fixed orientation with respect to each other. If we do not have the frame, the arms can move with respect to each other. Now how do we take the input? The orientation of only one? That is a one handed gun. It will cause a mismatch. Do we use relative position to calculate the angle? That is somewhat better but position tracking is unreliable. Furthermore, even with position tracking, it is still un-natural. Therefore we need the gun frame.

    So why not just create a gun VR device? That might be a good project for someone to work on. I am trying to keep it modular because I want people to be able to take this one device and do lots of things with it. VR is already expensive and it will be nice if we can re-use components as much as possible.

  • Wand Design Extension and IMU setup

    Gaurav Ghorpade05/30/2016 at 21:26 0 comments

    Explanation of developments in wand design and setup of IMU

    I am first trying out the BNO055 IMU. I have managed to get some sensor readings( one of my arduinos broke). I was hoping to have good orientaton readings even if the position is going to drift. I note that the damping is not amazing, so if I disturb my IMU and then leave it alone, it takes a while for the orientation to settle down.

    Nonetheless I am pleased with the data so far because I believe that the VR application can handle all of these issues. Example- in a shooting game you will you some kind of autoaim and therefore even if the reading is not perfect we are just trying to make sure we can get reasonable data to understand user intent.

    Also, just a note to dummies like me who are following the guide online- use the PDF for the IMU instructions, the wiring is somewhat different if you dont use the same arduino as the one on the webpage(uno).

    On the wand design front, I am posting some pictures which will explain the design. I can already see that the breadboard holder and battery holder can all be integrated into the frame. This will allow me to shave some mm to get more space/smaller device. (Do note that I am designing the CAD files so that you get pretty stiff parts which will not warp and will still fit after cooling shrinkage and therefore many of the thicknesses may seem too large. We can certainly use different materials and manufacturing processes for a more compact build but hopefully this is something which people can make with the tools they have at home)

    Update: I am still working on that IMU setup, everything does not seem to be calibrated perfectly.

  • Glove Design 1

    Gaurav Ghorpade05/30/2016 at 20:57 0 comments

    Initial design concept for the glove:

    For a glove exoskeleton, ideally we would like to be able to separate the finger into two links and give actuation for both links. However since I need to make a wearable, using the current set of commonly and cheaply available actuators, I will only be providing a single link to apply the force on the index finger.

    Since I want to be able to apply a variable impedance at various points in the coordinate space for the finger. I am currently working along the lines of a motor and spring combination.

    The reason why I am using the spring is because of time response and ease of control. One of the main use-cases in VR is currently gaming. Gaming interactions are very quick. This means we need to be able to apply forces on the hand very quickly. If I use only a servo and combine it with a force or pressure sensor at the finger tip(or something along those lines) then there will be a time lag between when the force or movement is detected by the sensor, this information is sent to Micro-controller, then either processed by VR device then sent back to controller and then actuated by the motor.

    However with the spring device, when the finger is trying to move, the physics of the spring will automatically apply a force on the finger. I will sense this change in force and then control the motor to change the deformation of the torsional spring. This will ensure a responsive actuator with fast feedback and at the same time allow for generation of impedance in a more natural way.

    I am thinking of using a rotation sensor to sense the deformation in the spring. It looks like some mechanical encoders are available cheaply however these encoders have only 24 pulses per revolution. I think this is a somewhat low resolution. I am thinking of either using a gear between the shaft and encoder to give me a higher effective resolution OR to use an optical encoder.

    Do let me know if you have any suggestions on cheap,compact rotation sensor with atleast 4 degrees resolution.

  • Wand Fabrication -Day 1

    Gaurav Ghorpade05/26/2016 at 05:33 0 comments

    Checked out the 3D printer available to me. Noticed a lot more shrinkage than I am used to. I have made available the parts that work after accounting for shrinkage.

    The part is a holder for the breadboard. This part will attach onto the wand handle part. I am not sticking the breadboard directly onto the wand handle because I do not want it to come lose when the wand is moved around. I also want to be able to re-use all components on different projects. I like to avoid getting double sided tape stuck to the bottom as many times a little residue builds up when you remove the tape, do this a couple of times and you change the effective thickness of your breadboard.

  • Project Aims/Goal Definition

    Gaurav Ghorpade05/23/2016 at 06:38 0 comments

    I believe that my system will be able to or should attempt to enable the following types of interactions:


    1) Using a bow and arrow with some kind of haptic cues

    2) Using a sword and shield or a knife( This is a classic case of where a full exoskeleton system is more immersive but more on this later on)

    3) Using different types of guns- pistols,shotguns, shoulder fired rocket, submachine guns etc.

    4) Boxing?

    Non gaming:

    1) Pointing, pressing/touching, grasping

    2) Button oriented input

    Locomotion problem:

    Hopefully I will be able to enable a system which works better than teleportation and overcomes some issues with treadmills.

    Edit: Note that simulator sickness is supposed to be caused by mismatch visual, vestibular and proprioceptive senses. The treadmill allows you to trick your proprioceptive sense but your vestibular sense is less affected. My proposed method of locomotion also allows you to stimulate your proprioceptive sense(to a smaller extent). I intend to test whether this is sufficient to deal with the simulator sickness issue.

    I am going to try to keep some parts of the wand device modular so that people can customize it to use only the parts that they need for their specific implementation.

View all 5 project logs

Enjoy this project?




[this comment has been deleted]

Gaurav Ghorpade wrote 06/04/2016 at 21:06 point

Thank you! I hope it works out :)

  Are you sure? yes | no

Gaurav Ghorpade wrote 05/26/2016 at 05:35 point

Please feel free to leave a message here about your thoughts on my project. If you are enthusiastic about VR and would like to see specific interaction features enabled on my project feel free to leave a comment and I will try to see what can be incorporated.

Thank you for taking the time to see my project :)

  Are you sure? yes | no

Does this project spark your interest?

Become a member to follow this project and never miss any updates