Close
0%
0%

Room Based VR Positioning

My idea for a room based VR positioning system.
The goal is for tennis court coverage to play "real" virtual tennis!

Similar projects worth following
I love the thought that well-made VR environments are just around the corner, with the endless possibilities it brings to engineering, science, the arts, medicine and, of course, gaming.

As a kid I imagined it how cool it would be to be able to go to a Niven style “Dream Park” where people could game. Then, after I became an engineer I quite often would look at the technologies I came across and they would slot in place in my head as part of a system to make it happen. This is one of those things I’ve had rattling around in my head for about the past 17 or 18 years.

I’m putting it on here because I don’t think I’m ever likely to get round to producing my own VR arcade :-) and because I don’t think this approach is patented. I’m hoping that someone would want to take it forward to an open source VR system to push the VR adoption that little bit faster.

If you do use it please comment so I know!

THE PRINCIPLE

The principle behind it is a pair of laser lines, separated by a known angle and rotating at a constant rate which then sweep a pair of sensors with a known separation about the axis of rotation.

For a laser pair (L1 and L2) sweeping down around the horizontal axis, the time between laser L1 hitting the top and bottom sensor is a factor of how far the sensor pair is from the centre of rotation due to the constant rate and variable, relative, angle.

The time between laser fan L1 and L2 hitting the same sensor is always the same, regardless of distance, because the angle and the rate of rotation within the timeframe of the sweep are very constant. Taking the ratio of these two times will remove the rotational speed from the equation and give you a relative distance between the sensor pair and the lasers. Knowing the exact vertical separation between the sensors and also knowing the exact angle between the laser fans will fix the distance to a known scale.

I've re-done some of the calcs I did a while ago (at that time I was thinking 8MHz clocks!) and added a link to the google sheet as well as an image of the chart. At 5m you're looking at a single distance measurement LSB accuracy of 2mm (20mm sensor seperation, 32MHz counter, 25Hz/120deg laser sweep). Implementing the logic and counters within a CPLD/FPGA would be more development (for me at least) but could push the accuracy by up to x10, proportional with the clock frequency increase.

Fusing multiple readings and a 6DoF sensor and the accuracy for the sensor cluster should be very good.

THE REALITY OF IMPLEMENTATION

That’s the fundamental principle, and would work fine if the sensor were just sliding around a table with the lasers at the edge. However, in reality, the sensor is going to be attached to a head which has six degrees of freedom and as soon as the head tilts or moves above or below the laser plane (as in the diagram below,) the perspective angles (and thus the time differential) change. This will throw off the simplistic view of the world above.

This is where we need pattern recognition; a way of recognising the alignment of the sensor head by the order and timing that the laser line hits the various individual sensors. I’ve chosen a circular band with sensors around the top and bottom for this example because it is the easiest to visualise. I’d likely have chosen this configuration as my proof of principle too because it’s a lot easier on the maths to fit to a known curve.

As you can see, with the swipe of a single laser across the sensor head, it is not possible to differentiate between a side view with no roll, and a rolled head lower down in the vertical axis. However, add a 6DoF sensor to the head and the system now knows the down-centric attitude of the head, it also knows the flat line the laser describes indicates that the sensor head is in line with the roll giving a relative attitude with respect to the laser. Now add a second laser on the opposite wall, and the 3D picture becomes easier as the attitude which gives a shallow curve on one side, will describe a greatly exaggerated curve from the new laser.

I believe the relative angles between the sensors given by the system can be used in a modified perspective n-point computer vision algorithm to give the distance and orientation to each of the known laser bases. Fuse that with the measured attiude of the sensor cluster from the 6DoF and it should be enough to accurately pin-point the cluster in 3D space. The more bases the better the position and attitude estimation.

SCALING IT UP TO WAREHOUSE SIZE

As the only critical time for interference is that limited time it is sweeping over the cluster, it should be easy to have multiple bases sweeping without the need to sync them all together. Indeed, you could enforce a minimal interference policy by giving lasers a slightly different sweep frequency. Each sweep frequecy could also be used to differentiate between...

Read more »

3dPositioningHVScanning-V2.zip

Java Netbeans project to give the relative angles in Az and El from a base to a cluster of sensors. Plot filters based on a (very rough) estimate of detection.

x-zip-compressed - 210.19 kB - 11/07/2016 at 00:07

Download

View file

  • About turn - The Mathematics (done?)

    Lee Cook01/18/2017 at 12:30 0 comments

    After taking a detour (and dragging a couple of others with me) through the realms of ePnP, I’ve done an about turn and gone back to my original idea of using circles and their apparent ellipses for tracking position and attitude. I’ve already implemented most of the steps and I’m reasonably confident that this approach will work, though I still need to see how well it will sit on a 300MHz Cortex M7. Anyway, here’s the approach as it stands at the moment:

    Take three sensor points in 3D space which have been lit with laser light (i.e. base angles are known). Construct a circle in 3D space using those three points and determine: the centre of the circle, the radius of the circle, the relative angles of the sensors around the circle and, the normal to the plane the circle sits on (use a combination of the three sensor normals to determine the outward facing side).

    At the end of the process you should have something like this which shows a circle constructed from A, B and C:

    Once the 3D circle information is known then start working on the information from the laser sweeps. Use the relative angle/bearings between the sensors to determine the angular magnitude between the sensors (AB, AC and BC). These distances relate to a series of chord lengths around an ellipse with the defined theta angles between them:

    The following equation can be used to determine the length of a chord on an ellipse:

    The problem easier due to the fact that, at this stage, there is only the need to match the ratio of the chord lengths with the ratio of the major/minor axis at those angles. This allows the removal of the minor axis as an unknown as well as the root. However, the problem is made more difficult in that the sensor group may be at any theta angle on the ellipse depending on the attitude of the HMD with respect to the base. This has the effect of “sliding” the sensor points around the ellipse as shown below:

    This basically boils down to a system of three equations where only R (axis ratio) and S (slide angle) are unknown:

    Unfortunately my mathematical skills failed me at this point and I have had to solve this part through iteration. I’m still hopeful that a colleague will take a look at it and come through with a less computationally intensive solution.

    Perspective errors in the ellipse can be reduced by taking advantage of the fact that there are actually two “slide” angles in the ellipse which will fit the chords - exactly PI radians apart. Solve for the slide angle in 0<S<PI and also in the PI<S<2PI ranges then average the two sets of results (still to do in the code).

    Once solved, this gives a major/minor axis ratio, chord lengths (with the minor axis set to one) and, the “slide” angle at which the chain of chords starts, and we start relating the two sets of figures, circle and ellipse together to figure out the pose:

    • The ratio of the major/minor axis gives the angle the circle has been tilted from perpendicular to the base.
    • The “slide” angle will determine how much the HMD has been rotated around the centre of the circle (in the plane of the circle), use an accelerometer to determine up and the correct angle
    • The ratio of the original chord length vs the calculated one gives actual length of the minor axis
    • Scale the minor axis up by the major/minor axis ratio to get the major axis length
    • Use the major/minor axis to determine the positions of the points wrt to a zero and then work back to determine the centre of the measured ellipse from the scan.
    • The centre of the ellipse is a bearing to the centre of the circle.
    • The major axis is the angle created by the circle radius length – use both to determine the distance between the base origin and the centre point of the circle.

    This is about as far as I have come so far. Through with use of external sensors and the sensor normals it should be possible determine the position and orientation of a base wrt to...

    Read more »

  • First iteration of a simulator

    Lee Cook10/30/2016 at 23:15 1 comment

    If there's one thing I've learned over the years it's that you need a decent simulator of the real-world characteristics of your system to develop & debug the control aspects. I've attached a zipped Java Netbeans IDE (8.2) project as the start of mine which will hopefully help with the developement of the maths algorithms...

    It's the first iteration which at the moment defines a spherical sensor cluster:

    Imagine a six sided die blown up to a sphere. At the centre of what was each face, the middle of each edge and the 3-point corners is a sensor.

    The cluster can be placed in a XYZ space, be given an attitude vs down (the output from an accelerometer) and then rotated on the horizontal plane for a heading.

    The program then prints an Az and El angle which would be read from a system similar to Lighthouse and taking the differences would be the output from my system:

    The next iteration, hopefully done by the end of next weekend will fix the inevitable bugs. I'll also expand the functionality to look at the normals of the sensors vs the angle of the base along (potentially taking in to account distance too) to try and estimate if the sensor would detect the laser and, if it does, the duration of the pulse. I'll refine the estimates with real-world data as the hardware comes along.

  • The maths?

    Lee Cook10/19/2016 at 23:50 6 comments

    I’ve just put my thoughts down elsewhere while trying to explain what I see the maths problem behind this project, and I thought I’d share them here too. It’s quite late at the end of my third 15hr work day so a little slack is appreciated. That said, if what I’m proposing is a complete load of bo**cks feel free to point out the errors – with corrections please! ;-)

    My system can't produce exact bearings/'pixels' like the lighthouse system does, but it can measure the relative angles between the various points and, in theory, relate those angles to an absolute distance.

    It's similar, or perhaps even the same(?), as the n-point problem I think.

    There is only one orientation (and from a perspective point of view, distance) of the target with respect to the laser emitter, which when scanned with the laser line, will give that particular ratio of angles. The problem arises because that orientation could be anywhere on the sweep of the laser, like a toroid in the axis of rotation. However, if you know the attitude of the target with respect to the axis attitude of the laser sweep, it becomes fixed against that axis – there is only one orientation of the target wrt to the laser axis which has that particular target pitch and roll.

    The "orientation" reduces down to a relative translation of the target (x,y,z) with a rotation in target yaw/heading - and the whole thing potentially rotated around the vertical axis.

    Having a second set results from another characterised laser scanner (characterisation done during the room calibration routine) and it should be possible to calculate the yaw/heading component of the target and fix the target on the XY plane.

    The whole reduction thing relies on having known attitudes for the emitters and the targets so they can all relate in the same axis.

    Anyway, that's my theory. I've talked to one of the mathematicians at work and he seems interested enough in the system to want to give me a hand getting it working. Perhaps when I give him this talk he'll laugh in my face at my naivety and point out I need 30 scanners to make up for the lack of real bearings.

    I hope not.

  • Shelved no longer!

    Lee Cook10/07/2016 at 19:08 0 comments

    Ok, I have come to the conclusion that in order for this to become a non-virtual reality (sorry, couldn't resist) I would need to progress it to the point where it generates interest. So.. I’ve shelved other projects and this will now take all of whatever time is left after work, a horrendous commute and family of four kids. (i.e. don't expect rapid results!)

    I’ve ordered a load of line lasers, MEMS units and other bits which will be coming to me on a slow boat from China.

    While I’m waiting for those I've been looking at photodiode detector circuits. The parts for this circuit:

    (originally from <here> and copied later <here>) are on order as it looks the most promising starting point for the input stage initial design. Hopefully it will bias out a lot of the ambient light current but let the modulated laser current through.

    It has to be said, I’m nervous about getting back in to proper analogue electronics.

View all 4 project logs

Enjoy this project?

Share      

Discussions

Mike Turvey wrote 12/16/2016 at 16:24 point

Hey Lee-- glad to hear you're still looking at PnP.  Are you specifically looking at ePnP or a different algorithm?  Looking to write the code from scratch, or adapt one already out there?

  Are you sure? yes | no

Lee Cook wrote 12/16/2016 at 20:22 point

Thanks.

I'm still open to any approach. Though I'm going to see if I can do my own initial estimator direct from the sensor data and try different scemes for cherry picking sub-sets of lit sensors for speed/performance.

Initially though I just expect to implement the off the shelf EPnP example code to see how the M7 performs and how it could be optimised.

  Are you sure? yes | no

Jonathan Kelly wrote 10/01/2016 at 00:16 point

Wondering, have you looked at https://hackaday.io/project/15496-precision-indoor-positioning ?

Curious to see how that goes.  

What sort of precision and error are you wanting?

If there was something low cost that could reliably fix a position in 3 dimensions +/- 5cm over around 5m I would be pretty interested.

  Are you sure? yes | no

Lee Cook wrote 10/01/2016 at 13:21 point

Yeh, I've been trying to chase him down to see if he is willing to either sell, or specify the BoM and circuit for, the modulated laser that goes with the TS3633 modules he's using.

I've re-done some of the calcs I did a while ago (at that time I was thinking 8MHz clocks!) and added a link to the google sheet as well as an image of the chart on the sheet to the project images.  At 5m you're looking at a single distance measurement LSB accuracy of 2mm (20mm sensor seperation, 32MHz counter, 25Hz/120deg laser sweep).

This is a *really* simplistic view, in reality you'd be able to average over multiple sensors and you'd be fusing it with the 6DoF data - think of how GPS gets more accurate over a period of time.

Doing the logic and counters within a CPLD or FPGA would be more development (for me at least) but, on these, you could push to x4 -> x10 times the counter frequency - with proportional effect on accuracy.

  Are you sure? yes | no

Mike Turvey wrote 10/01/2016 at 17:20 point

Hey Lee,

The TS3633 are designed to go with the Vive lighthouses (A.K.A. Base Stations).  While the Vive controllers use dedicated FPGAs for some of the critical timing work and a full computer for translating those timings into position, the Vive is capable of sub millimeter precision in 2 axes, and about 2cm precision in the depth axis.  If you use two lighthouses, I believe it becomes submillimeter in all axes.  All that my project is doing is trying to replace the dedicated Vive VR controllers and full computer with a microcontroller and a custom tracked object (in the form of a constellation of sensors).

  Are you sure? yes | no

polyhistor10 wrote 09/26/2016 at 13:28 point

My idea for localizing is as follows;

- have two laser beacons of known coordinates sweeping the area. Line lasers are very cheap now and they can be modulated to 10kHz.

- each beacon is fitted with a compass module. The sweeping beam is modulated with the bearing value from the compass.

- the target has a light sensor with demodulating software. As the beam hits the target it is able to read the bearing from that beacon.

- the target can be continuously triangulated with the bearings from the two beacons.

I have built this scheme with inconclusive results, mainly because I am struggling with a laser receiver and demodulating software.

  Are you sure? yes | no

Lee Cook wrote 09/26/2016 at 20:38 point

How are you modulating the signal?  You would also need to add at least one (two if the item can go above and below the plane the beacons are on) more transmitter in order to fix the item in space.

  Are you sure? yes | no

polyhistor10 wrote 10/20/2016 at 01:09 point

Lee,

I am only working in 2D. I agree you would need vertical scanning for 3D.

Have you had a look at this system?;

http://www.hizook.com/blog/2015/05/17/valves-lighthouse-tracking-system-may-be-big-news-robotics

  Are you sure? yes | no

Lee Cook wrote 10/20/2016 at 09:04 point

Yeh, I've been following vr fairly closely for a while. It was lighthouse that tipped me over the edge to go public with my idea as I figured it was only a matter of time before someone else patented the technique.

Lighthouse is very cool, and they're making the tech free for hackers. Have a look at Mikes project (above), he's using relatively cheap COTS items to track with the lighthouse base station and sensors (TS3633). Lighthouse method allows him to look at the system as a high resolution camera problem so, in theory, there is a huge amount of examples to draw from.

The downsides to Vive are however the need for very accurately matched clocks with minimal drift, or the ability to drift sync them against a master station (similar to the 1PPS embedded in GPS). It also requires those clocks to precisely matched with rotational speed of the lasers and the timing of the sync pulses. Finally, from what I've read, because of those sync pulses they're pretty much at the limit of what they can achieve. In order to scale up the working volume they believe they would need to add additional sync repeaters around the room to match the 20m working distance of the lasers.

My end goal is to have two or three of my base stations around half an *open* tennis court to play full size virtual tennis!

  Are you sure? yes | no

polyhistor10 wrote 10/20/2016 at 14:38 point

My end goal is to make a robotic lawn mower for my rectangular lawns!

  Are you sure? yes | no

Jonathan Kelly wrote 09/24/2016 at 03:28 point

I have also been thinking about ways of localising things in space as part of another project I am working on.

A thought I had is to perhaps borrow ideas from how aircraft VOR navigation beacons work.

This is a bit simplified but basically aeronautical VORs have a ground station that sends out 2 radio signals, one is a highly directional signal that rotates like your idea and sweeps 360° 30 times a second.  

The other is a reference signal.  The sweeping signal changes its phase from the reference depending on where it is in the sweep (0° and no phase change, when aimed 90° it has a 90° phase shift etc).  

The aircraft VOR receiver picks up the beacon and compares the phase of the 2 signals and calculates what angle it is from the beacon.  Aircraft use that to determine what bearing they are from a known point on the ground.

Now...back to your idea.  Suppose you had a beacon that sent quickly pulsed light signals - a reference pulse of one colour that pulsed at a constant frequency (say 1kHz) and a rotating pulse (that rotated say at 10 times a second) of a different colour but that adjusted its pulse frequency between say 1kHz to 1.359 kHz, depending on the angle it was currently aimed at.

The receiver would detect the 2 light signals and compare the pulse frequency between say the red constant frequency pulse with the green variable pulse one and thus know the angle it was from the beacon when it detected the green signal.  eg If it was 200Hz different then it would be 200° from the beacon.

Now suppose you had 3 of these beacons distributed to known points in the room.  

Assuming the receiver could differentiate between the light signals of each beacon, it could work out what angle it was from each beacon and from that could determine by triangulation exactly where it was in  the room.

Potentially you could do away with the reference pulse from the beacons if the 0° frequency was able to be accurately measured by the receiver.

My idea is not very thought through and may have big flaws or not be practical but hey... thought it worth saying.

  Are you sure? yes | no

Lee Cook wrote 09/24/2016 at 14:53 point

I think any idea is worth saying.  Thanks for the comment.

The main problem with angular triangulation is that the angles need to be very accurate, around 0.5deg for 1cm accuracy at 1m.  Take that to 5m for a reasonable sized room and you're looking at 0.1deg of accuracy.  I don't think you'd be able to encode that amount of accuracy at 10Hz with a digital signal.  That'd be 18000 different encodes per second (if you doubled the emitters, 36000 if you had a single with half facing the wall), with 1800 unique over the half sweep - perhaps double if you wanted to gaurentee full reception.

Really, I think you'd have to provide an analogue signal, perhaps multiple lasers with the power modulated for each over the sweep - say RGB with each transmitter sending a combination of RG, GB, RB ramp down one while ramping up the other.

Reception hardware would be quite expensive and would need very good calibration ADCs with 4096 levels.

Thanks for looking!!!

  Are you sure? yes | no

Lee Cook wrote 09/30/2016 at 11:35 point

I've been looking at the Triad Semi TS3633 since you last posted.  Lighthouse modulate their lasers at around 2.5 MHz according to the spec, I'm assuming for noise rejection.

Say a 2mm beam width at 10Hz sweep, 5m distant would light the photo detector for (I think?) around 6us.  In the 2.5 MHz range would that give enough of a signal in order to accurately detect the delta changes in the frequency?

  Are you sure? yes | no

Mike Turvey wrote 10/20/2016 at 21:25 point

Interestingly, 6us is almost exactly how long I'm seeing the pulse as the lighthouse laser sweeps across my sensor.  The TS3633 are sensitive to a 1-5 MHz carrier frequency.  Certainly this helps with noise rejection, but somewhere I saw that the long term plan for supporting many lighthouses in a single space was to not just do TDMA (as is done now, but doesn't scale well beyond about 2 lighthouses), but to add in FDMA.  So, you could have multiple lighthouses operating in the same space, but using different carrier frequencies.  With the sweep pulse lasting 6us, that should give plenty of time for a dedicated ASIC to determine the carrier frequency "channel."

  Are you sure? yes | no

Lee Cook wrote 09/30/2016 at 11:42 point

Sorry, just thinking some more...  How about something even closer to the VOR you originally looked at.  Two lasers, different wavelengths, modulated at the same frequency but the phase changes.  Phase differences may be easier to detect, though at those frequencies the board layouts (or their calibration) become more problematic.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates