Close
0%
0%

The BYTE

A hands free universal interface

Similar projects worth following
The BYTE is an extremely affordable, safe and responsive hands free interface.

Since it is controlled with the mouth even those with profound mobility limitations will benefit from this intuitive and accurate pointing solution.

In addition to the obvious computer use cases, I can envision applications including wheelchair or vehicular navigation, prosthetic controls, smart home controls, robotic arms...
Who knows what else - maybe this is the ideal interface for people who hang glide?

Project Log Direct Links:


Overview Video




Sensor Overview

A durable sensor package is encased with food safe silicone rubber. 

The stem portion contains a shunt-mode force sensitive resistor.  It is held between the teeth to stabilize the BYTE while measuring the applied bite force.

Beneath the tongue-operated navigation button is an intertwined array of four thru-mode force sensitive resistors. These are measured 125 times per second to determine the pointer speed and direction.

One important change from earlier prototypes is that all of the digital interface electronics have been moved from the sensor pack to a separate controller module. This approach has dramatically reduced the cost and complexity of the sensor assembly, to the point that it can nearly be considered a disposable item. 


Basic Operation

Light bite pressure enables mouse movement and triggers a small haptic vibration.

Firm bite pressure is applied for clicking. Clicks are confirmed with a slightly more intense haptic indication.

Longer clicks are also detected and can trigger any desired function.

Evenly pressing the whole navigation button at once triggers yet another click type, the "press click" and again can be mapped to any desired action. Examples include opening specific programs, running macros, and triggering discrete outputs for control of other devices.

Scrolling is done by pressing forward or back while applying no bite pressure.

Regardless of the default behavior, any particular motion, pressure combination, sequence of actions, etc. may be re-mapped to any desirable output - be it IR for TV controls, or complex WiFi/Bluetooth commands to water plants, activate coffee makers,  or update social media.


Per-User Calibration

On first use, and whenever else it is desired, the BYTE can be re-calibrated to match the user's ability and preference for the minimum and maximum pressure they are comfortable applying to each of the sensors.

The calibration routine takes advantage of the keyboard emulation function of the controller.  It automatically opens a new text document and guides the user through the different calibration steps, typing out the instructions and providing feedback along the way.

There is no need to install an app or otherwise change settings on the host.


Controller

The controller emulates a regular USB mouse and keyboard, allowing it to work seamlessly with nearly any computer or tablet. A Bluetooth version is in the works for even wider compatibility.

I chose a standard RJ45 jack (or more accurately an 8P8C modular connector) to connect the sensor package to the control board. This provides a reliable, latching attachment and has enough conductors for all of the required functions.


A Fresh Take:

The BYTE supersedes my previous project, The Bit. The BYTE is a complete overhaul and redesign, and re-uses none of the electronics or code. It approaches the same problem in a radically and substantially different way and has undergone many, many revisions - the 3D model is beyond version 100!

BYTE-master-2.zip

Updated Full Design Package (3D Model, PCB Design, Firmware, Schematics, BOM, Testing Procedure...)

Zip Archive - 4.15 MB - 10/05/2020 at 09:58

Download

BYTE-master.zip

Full Design Package (3D Model, PCB Design, Firmware)

Zip Archive - 1.44 MB - 08/31/2020 at 07:45

Download

View all 29 components

  • Summary of the Hackaday Prize Criteria

    oneohm10/05/2020 at 08:04 0 comments

    Direct links relating to the required contest criteria:

    Concept-
    Is the project creative, original, functional, and pushing boundaries?
      - To my knowledge there are no similar devices currently available
    Does the project effectively address the selected challenge?
      - The UCPLA challenge seeks "new designs for adaptive tools... like trackballs or joysticks" The BYTE definitely fits the bill.

    Design-
    System design:
      Design Process - Evolution!
      Discussion and diagram of main sensor design
      Controller Design Details
    CAD models:
      Discussion of the sensor CAD model
    Project test methods:
      Testing procedure document
    Functional block diagram:
      System block diagram
    How user-friendly is the design?
      See the overview video for a usage demonstration

    Production-
    Is the project realistically reproducible?
      - With the information provided, the BYTE can easily be replicated
    Are the manufacturing processes detailed?
      Video showing the assembly process
    Are those processes realistic for scalability?
      Path from prototyping -> manufacturing

    Benchmark-
    How well is the project impact and viability demonstrated?
      Discussion of the viability of the design
      Advantages of the pressure sensor approach
    Are estimated costs realistic?
      Discussion of estimated costs
    How well does the project improve upon other currently available solutions?
      Discussion of available solutions

    Communication-
    How thoroughly have the Final Round requirements been completed?
     -Hopefully this summary addresses this point
    How well documented is the project?
      Full design package available on Github
    How “open” is the design?
      - Firmware license:
        GNU General Public License v3.0
      - Hardware license:
        Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License
        Once the design is finalized the hardware will receive a more permissive license.
        And of course UCPLA is permitted to use the design.

     

    Additional Final Round Requirements:

    Working prototype ✓
    Video:
    - between two (2) minutes and five (5) minutes in length ✓
    - shows a working prototype ✓
    - describe the challenge it addresses and demonstrate how it facilitates the solution ✓
    High-resolution photos of the project inside and out ✓
    At least ten (10) Project Log updates ✓
    Components list that is complete with a bill of materials for one unit ✓
      Complete Bill of Materials
    Complete schematics ✓
      Full System Schematics
      XIAO Controller Schematics
    Documented input and output requirements and specifications ✓
      Just in case it is not clear, The only IO requirement is a standard USB C cable

  • Testing Procedure

    oneohm10/05/2020 at 05:42 0 comments

    The testing procedure for the sensor pack is now available in the Github repository.

    This procedure should be performed after initial assembly, but before sealing into the enclosure.

    It can be downloaded here: Testing Procedure

  • Full System Schematics

    oneohm10/05/2020 at 03:39 0 comments

    Also available in the Github repository here: Full Schematics
    The schematics for the XIAO board are available here: XIAO Schematics

  • Functional Block Diagram

    oneohm09/21/2020 at 23:42 0 comments

    Here is the basic architecture of the current prototype. Additional interface options are in the works, including WiFi & Bluetooth connectivity and discrete outputs for replacing existing buttons, switches or other controls.

  • ​The Logic of Tongue Control

    oneohm09/13/2020 at 22:43 0 comments

    Cerebral palsy, spinal cord injuries, stroke, and other neuromuscular disorders may lead to tetraplegia, a condition that results in loss of sensory and motor functions of all four limbs and the torso.

    Many approaches are used to help leverage the remaining functional parts of the body for computer interfacing including:

    • head tracking
    • eye gaze tracking
    • speech recognition
    • mouth/chin-operated mouse/joysticks
    • sip-n-puff
    • brain-controlled interfaces
    • tongue-operated interfaces

    Some limitations of existing approaches:

    Eye trackers use eye movements to control the mouse cursor on the computer screen. Unfortunately, they interfere with visual functions because the computers have difficulty determining whether an eye movement was a control input or looking at an object, they require a camera mounted in front of the user, which can block the field of view, and they are susceptible to lighting conditions and need frequent adjustment if the user’s head position is changing.

    Voice recognition systems are efficient for typing but are slow and not intuitive for controlling a computer mouse or a powered wheelchair. They are also susceptible to ambient noise, and therefore, not suitable for outdoor environment. 

    Head trackers and mouth/chin joysticks require head and neck movements, which do not cover tetraplegics who have limited or no head control, or they cause fatigue in weakened cervical/shoulder muscles, and cannot be used over extended periods. 

    Sip-and-puff (SnP) is one of the most popular ATs for driving powered wheelchairs due to its low cost and ease of use, but offers only four commands. It is to slow for computer access and offers limited options. Moreover, the SnP straw and tubing require frequent cleaning or replacement.

    EEG-based BCIs are too slow because of the limited EEG bandwidth, and susceptible to motion artifacts and interference. They also have a cumbersome setup procedure and need headcap/gel for good electric contact to the scalp.
    The Neuralink has some promise, although not everyone will be willing to have a sizable hole drilled into their skull...

    Advantages of tongue-operated interfaces:

    Since the tongue is connected to the brain by the well-protected hypoglossal cranial nerve, it generally retains functionality in those with tetraplegia.

    The tongue has many inherent capabilities that make it an excellent control modality for those with tetraplegia. The area of the motor cortex dedicated to the tongue and mouth rivals that of the fingers and hand, providing the tongue with sophisticated motor control and manipulation capability. 

    Tongue movements are natural, intuitive, and do not need thinking or concentration. Hence, assistive devices that operate based on tongue motion are easy to learn and use. Additionally, the tongue can move both quickly and accurately and since the tongue muscle is similar to the heart muscle it does not fatigue easily. 

    Moreover, its position inside the mouth gives the users of tongue-operated devices a certain degree of privacy. 

    The dexterous, intuitive, rapid, precise, and tireless motion of the tongue is the ideal interface for those with limited mobility to access computers/smartphones, drive wheelchairs, and interact with their environments.

    Summarized from this paper on tongue Interfaces out of Georgia Tech:
    Tapping into tongue motion to substitute or augment upper limbs

  • Prototype and production cost estimates

    oneohm09/11/2020 at 05:53 0 comments

    I believe that an initial production batch for early adopters should be possible for around $150 - $200 each. This would help to cover the injection mold tooling and manufacturing equipment costs. Later batches could be offered at a lower price, hopefully dropping below $100.

    Initial estimates for pricing at different quantities:

    ~ 100 units
    ~ 500 units
    ~ 1000 units
    $150-200$120-150< $100

    Throughout development of this project, my objective has been to create something innovative that will improve the quality of life for people.  But in order to realistically make my project available to any who will benefit, I must also prioritize affordability.  

    Rather than working to add features and complexity, I have put a lot of effort into simplifying the design, being careful to preserve core functionality. A reduced part count not only lowers cost but, more importantly, increases reliability. 

    The cost for the individual parts of my prototypes includes quite a bit of overhead: shipping charges, larger-than-required minimum order sizes, and non cost optimized part selection.

    For additional pricing details the costed BOM is available on Github: Prototype BOM

    Material costs for production quantities (on the order of 1000 units) will be much lower, but there are also additional considerations that need to be taken into account when transitioning to production. Assembly, testing, packaging, shipping, and support costs are often overlooked or underestimated when bringing a new product to market.

    While I would love to be in a position to make the final product available for free to all, it is important to be realistic. In order to introduce a quality product to the market, back it up with responsive support, and still be able to invest in ongoing development/improvements, the sale price must be at least somewhat higher than the raw manufacturing cost.  That said, I hope anyone who desires to make their own BYTE at home would be able to follow my published instructions.

  • Currently Available Solutions

    oneohm09/10/2020 at 03:12 0 comments

    As far as I can tell, there are no commercially available tongue operated controllers currently on the market. This seems to be an important unmet need.

    The most common competitive devices are mouth controlled joysticks. There are several models commercially available. I was very surprised that they all seem to be extremely expensive; especially when considering that many of their users have limited budgets. 

    Two common models are the QuadJoy and the Jouse, both ringing in at well over a thousand dollars. The TetraMouse and QuadStick are less expensive, but still cost many hundreds of dollars each. 

    While mouth controlled joysticks are in some ways similar to the BYTE, there are important differences. One differentiator is the type of muscles involved in using these devices. 

    The lip, jaw and neck muscles involved in operating these joysticks tire much more quickly than the tongue (Think of the last time you had to hold a smile just a little too long for a slow photographer...). The tongue is very similar to the type of muscle comprising the heart, rendering it nearly tireless. 

    Another advantage of a tongue controlled interface device has to do with the robust and direct connection of nerves from the tongue to the brainstem. Most people with spinal cord injuries, and many with otherwise paralyzing neurological diseases, can still move the tongue.

    Several tongue operated controllers have been demonstrated, although mostly limited to one-off prototypes and academic research. A well known example comes out of Georgia Tech's Bionics Lab, the Tongue Drive System, using a magnetic tracking approach.  It requires the user to have their tongue pierced with a magnetic stud; for those with a strong stomach a paper detailing the procedure is available here

    Aside from no piercing requirement, the BYTE has a major advantage over a magnetic tracking approach, especially for those with cerebral palsy - since it is pressure controlled, there is no need to accurately position the tongue within the mouth or to make small precise movements. 

  • Advantages of Force Control

    oneohm09/09/2020 at 23:27 0 comments

    Instead of responding to tongue position, the BYTE is force controlled. This is an important distinction as some people with cerebral palsy find it challenging to perform fine and precise articulation of their tongues. 

    With the BYTE, the whole tongue can be pressed against the nav-button and rocked or tilted in the desired direction. There is no need to hold the tongue in a specific position or to make small precise movements; in fact, the exact position of the tongue is largely unimportant.

    For users who may suffer from spasms of their tongue or chin muscles, additional averaging and filtering can be implemented in the controller. This approach will still provide accurate pointing since following a spasm the tongue need not return exactly to its previous position, but simply re-apply pressure in the desired direction.

  • Controller Details

    oneohm08/31/2020 at 04:53 0 comments

    The controller acts as a bridge between the sensor module and the host system.  A set of resistor dividers measure the five force sensitive resistors while a transistor drives the haptic actuator.

    I am using a SAMD21 based micro-controller module (the SeeedStudio XIAO). It incorporates an oscillator, voltage regulator, USB connector, EMI shield and more for less than $5. In production these components will be directly incorporated into the control board design, but while prototyping it would be difficult to beat the cost.

  • Assembly Process

    oneohm08/30/2020 at 22:18 0 comments

    Putting the sensor together:

View all 14 project logs

  • 1
    Sensor Assembly Video

View all instructions

Enjoy this project?

Share

Discussions

Phil Malone wrote 06/25/2021 at 02:59 point

Hi.  Are you any closer to having "production" sensor modules available?

  Are you sure? yes | no

Adalberto Caldeira Brant Filho wrote 11/16/2020 at 18:27 point

That is a great project. I woud like to propose a new assembly for a wireless version, as a dentist I think I could help. Did you try to use the tongue and put the device in a more confortable place inside de mouth , so the guy could speak and at the same time use it?

  Are you sure? yes | no

Phil Malone wrote 11/15/2020 at 18:05 point

Congratulations.  I'm glad you've opened the discussion board up.  I've been trying to message you for months.  I'm eager to try adapting your device to my https://hackaday.io/project/170932-hoverboards-for-assistive-devices project.  When you are ready to make your device available, I'd love to purchase just the sensor.  I'd adapt the interface to my own HUGS protocol.

  Are you sure? yes | no

unardia wrote 11/12/2020 at 06:46 point

Help me to buy one. I have a person who need it. Thanks!

  Are you sure? yes | no

Ammar wrote 11/11/2020 at 19:57 point

this is amazing. Where can someone buy one?

  Are you sure? yes | no

Rik wrote 11/11/2020 at 18:05 point

My first reaction was: 'Fuuuuuuuuuu! mind blown!

This is amazingly clever! I And it has a lot of other applications... Could be used by divers too with a bit of tinkering?

You deserved the prize for sure!

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates