Game modifications for using haptic instead of visual senses

Public Chat
Similar projects worth following
Most games work with visual clues, which results in an decreased fun for visual impaired people. As a sidetrack of the Personal Photonics project (, funded by the german ministry of education and science for developing an haptic toolkit, these game modifications happens due to inspirations of the community. They are still mostly first steps at the moment, just to check if these ideas are feasible and make fun.

So far we tried out 4 different types of game to modify: A haptic memory game with included RFID tags to find pairs of braille and black letters, a scorecard for an (german) Yatzee game, translation of building game cards into haptic senseable 3D-sculpturs and reliefs and using a haptivision belt (, Sean Bensons would of course also work) as haptic dispaly for computer games. Of course, thats not HD resolution, so we go back to the computer stone age and tried out pong on it.


Final version with a magnetic sheet as backplate

scad - 9.79 kB - 09/30/2019 at 11:06


  • 1 × PLA or other materials for a 3D-printed form
  • 22 × 3mm sphere magnets ... for the scorecards, better more as reserve...
  • 1 × magnetic foil or piece of metal ... for the scorecards, 190*140mm
  • 1 × Arduino, MP3 Player shield, RFID reader, Speaker, RFID-sticker (dependent on number of cards)) ... for Braille-Memory
  • 1 × Arduino Nano, 5V Relais, Washer pump, bottle, 2 4AA battery holder, Sparkfun EMG kit ... for Water gun

View all 6 components

  • Log 10 - Current intermediate results

    JanThar09/30/2019 at 11:17 0 comments

    Results so far: From a usefulness perspective at the moment the big buttons are the most ready parts (driving around the world on a wheelchair in use), but have little to do with the intend of "haptic" games due to the original personal photonics project. The scorecard gets also good feedback, but the last version still needs testing on the long run if it works as intended.

    The water gun might become interesting, at the moment we are using (and built) them for ourself, might be interesting if there is an actual more "serious" use case... as serious as one can get with a water gun :)

    While pong on the belt works nice, buyilding it just for ithis purpose is overkill (there the 1D-games might be easier to built and use.

    The construction game pieces fail due to stability issues or (when cirmcumventing these throug bigger build size) space constraints, while memory is not as funny as intended (as one could have already known beforehand).

  • Back to Scorecards

    JanThar09/30/2019 at 10:51 0 comments

    From all the ideas so far the scorecards are the most useful thing (both from the usage as well as production-wise ^^). The biggest problems there is the (minor) problem of the magnets flipping over to another line, which can be reduced by using deeper holes or bigger distances between the checkboxes,abacus rows. The latter will make the card more clumsy, thats why we will go for deeper holes, which will also be used for clearer distingushing of the abacus positions.

    While the washer systems gave overall a great locking feature, when the magnets clicks into its hole, some of the washers glueing failed over time, as well as the production process of gluing washers in would be rather expensive (but with reusing common materiels it's of course great for single rebuilding processes as an open-source project). Instead we are now testign the use of a single sheet of metall behind as a base plate. The 3D- cover is then glued on top of it as a single piece, with its form allows even to use the most basic injection molding, thermoforming or casting.

    This will furthermore elevetas the issue with the not-fingertip-friendly 3D-printed braille letters, since in this case the right half-sphere form can be used for each dot. For a 3D-printed version it's better to leave the braille letters away and later glue common embossed braille tape with the text on the cards.

    Which one need just a (magnetic) sheet (183,7*140) mm, either a magnetic foil or just metal, then a bunch of 3mm magnet spheres and  the 3D-printed (or otherwise manufactured) form, glued on top of the magnetic sheet and you are done.

  • Back to Haptic Games...

    JanThar09/30/2019 at 09:53 0 comments

    ... sort of :) Here we got inspired from the prosthetic Nerf gun (There exist also the idea of a mind controlled one .... hmm, looking at the mind flex controller ...). And since the CCCamp this year was a bit dusty, we considered that the world needs also more inclusive water guns:

    Using somthing like the EMG sensor (but basically anything else will work) as a trigger on an analogue input an Arduino Nano and a relais connected to a digital output to drive 12V with a "little bit" of current (delivered by 8 AA batteries in row in two 4 AA batterie housings) to control a wiper pump. It will suck the water out of a bottle and soak adversaries through a nozzle at a wristband. The pump is attached with a bunch of hot glue on the bottle cap, otherwise 3D-printed housings and textile strips (reusing lanyard and similar things) are used to attach the parts on belt (Batterie pack with arduino/relais below and the bottle) and arm (EMG sensor and noozle  - but places might vary depending of personal preferences/needs). The also provide a bit of water spray cover, otherwise the good old hot glue is used for encapsulation.

    At the beginning we thought that we also need a valve at the bottle to let air in, but at the and, as long as only short pulses are sprayed, the water/air will be sucked back through the pump in between shoots, even preventing/reducing leakage of water through the noozle while not shooting.

    Ok, still a little bit far fetched from the original haptic games idea, which turned out to go more into a games for impaired direction), but  actually there exist a paper wher there used water pressure in a sort of old crt-monitor-way as an haptic feedback device ^^.

  • 1D Games

    JanThar09/30/2019 at 08:51 0 comments

    Another interestign approch are 1D-games, which can be played on a led strip. Examples are the 1D dungeaon crawler, pong versions, racing gameeven a two player combat game. Since the output space is reduced, it makes them also more feasible to be used with less controls, therefore our few buttons which can be used with the head are enough (and we have as additional benefit nice demos for them for maker faires :). But that gives another interesting idea: we can also use a single row vibration belt as game field, going away from our overkill HaptiVest to the more common and easier to built 1D navigation belts. Another interesting approach would be (going away form the limitations of the vibration motor things) using a braille line for this type of games. Of course, the player wouldn't have complete overview over the game area and needs to scan the line with the fingertips, but giving a more precise and informative overview over the game area. E.g. the start of the braille line is the player position, and scanning the line would be the search for upcoming obstacles (dungeon), the ball (pong) or the street layput (racing game), to react at the right time.

  • How to play with limited movements

    JanThar09/30/2019 at 08:26 0 comments

    For playing games with the big buttons, they have to be connected to the computer instead of the wheechair. Here we could be lazy: As described here, one can just use a Arduino Micro or Leonardo as a joystick or gamepad (or just an ordinary keypad). So we just need to make a nice housing, add mono phone jackets for each button and we would be done. But this leaves room for more ideas: Since we can use basically every type of sensor as an input, we might be able to go away from the buttons and add additional sensor types, since game control with just a very few buttons is limited. Simple options are: A gyroscope as kind of joystick (analogue input), and/or air flow sensors as additional input control... But in this case we are in the "its start to get complicated", so that will be thing which is still ongoing in development.

  • BigButtons for Wheelchair driver

    JanThar09/30/2019 at 08:17 0 comments

    Originally intended fo another purpose, stearing a wheelchair with just a ring of buttons behind the head, we designed big buttons, based upon a standard small button in a 3D-printed housing. Theey should be relative cheap, reliable and needed to be activated with a slight touch. As it turns out, they can be also used for playing games. Not as a arcade button, which would be also possible, but for the wheelchair driver. In this case, a version with a slightly harder button, but a better click-sound for activation was made... which leads to other ideas. But first: Instructions can be found here.

  • BrailleMemory

    JanThar06/07/2019 at 11:31 0 comments

    The idea here was finding the right pairs of Braille and Black Letters for learning Braille. Of course, since the Black letters are not readable for people who need Braille letters, for them an adapted version of Braille-Braille pairs would be better. To support learning we added an RFID tag in each card, such that the letter can be also read out loud. This allows also additional game features, like bets if thats the right pair from every player be using a dedicated bet card in future versions. For reading out we used first an arduino, progranmm by Rene Niewianda, with an Text-To-Speech-synthesizer, sounding like old school computer voices (a.k.a. horrible). Later we switched therefore to an MP3 player shield, again connected to the arduino with the RFID reader on top, code is still work in progress. For reducing movement of the cards by scanning the game field we added a grid structure. Of course there is still room for improvement: Our first idea with addtional hints on the grit is unnecessary overkill. Furthermore, the round edges of the card allow still a rotation of the cards within the grid. Then, just a 4*4 grid is complicate enough and should be also mounted on a base plate for handing the game field around. Main issue is of course that memory is mostly fun for the younger children. There is also an ongoing discussion if enlarged Braille letters as in this game are actually useful: People who can read braille are actually confused when the letter is bigger than the fingertip, where their normal reading approach didn't work.
    Preliminary design files and code can be found here.

  • HapticPong

    JanThar06/07/2019 at 10:05 0 comments

    Using the HaptiVision Belt as base we tried out if we can use it also as Display for playing games on it. Of course, with 16*8 pixel that's not high resolution, but we thought that at least games like Pong might work. Adaption was rather easy: Using an Arduino as controller, we added an 16*8 WS2812 LED matrix at pin6 as a display and two linear variable resistors as controllers at pin A0 and A1. The haptic matrix was then connected with the I2C port. So writing the code was for us the biggest issue (not for me, since Moritz Messerschmidt did the work), since we had such a belt already at hand. The Code can be now also found at the belts github page.

  • Construction Game

    JanThar06/07/2019 at 09:50 0 comments

    Another idea was transalting gamecards for a construction game into sculptures and reliefs - translating the image what to build into an haptic senseable object. Construction was of course rather easy, but both versions didn't work well in real life: The sculptures broke easily, and the reliefs were hard to read, since details where to small. Of course, by printing them bigger you can circumvent these issues, but then you need a very big box for the cards. Design files for OpenScad can be found here.

  • Yahtzee score cards

    JanThar06/06/2019 at 19:51 0 comments

    For the scorecard we tried first a complete 3D-printed version. For counting numbers we used an abacus system on the right side (seperasted into upper and lower part score because of bonus points) and then haotic check boxes on the left for each type of score. In between is the text field for score line description, which can switched between Braille and Black letter usign a different font.
    Since the 3D-printed check stones slide eiter to easy or even break apart we adapted it by using round magnets with glued in washers.
    The haptic feeling was good regarding the magnets, but some issues remain at the moment: The 3D printed Braille is not fingertip-friendly, here might be a blank card and using common embossed Braille sticker better. Furthermore, the magnets tend to flip to easy to the neighbouring magnet, and for the abacus the washer didn't gave optimal locks at one number. For this issues deeper holes will work in the next version.
    Design files for OpenScad can be found here.

View all 10 project logs

  • 1
    Build instructions for the Scorecards

    At the moment the most useful (and even easy-to build) system are the Yatzee scorecard. You just need to print the openscad scorecard file (modify the text corresponding to your languange or comment this part out and use embossed sticker instead for better haptics). Afterwarts glue a sheet of magnetic material behind and add the magnets. And you are already done and ready to play. Really lowtech but useful.

View all instructions

Enjoy this project?



Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates