Close
0%
0%

Empower speech with AssistiveTech

Assisting speech requests through touch enhances communication and inclusiveness for those in need.

timTim
Public Chat
Similar projects worth following
Many people face communication challenges due to brain injuries, degenerative diseases, and other causes. This can lead to frustration and isolation, but assistive technology and tailored communication approaches can help them connect with loved ones and caregivers, improving their quality of life.
We aim to give them a voice, allowing customised messages to be replayed from touch gestures. In future iterations gesture recognition from video machine learning or muscular flex sensing is intended to be developed.

Currently, 14 touch sensor phrases are planned for the initial release, but as each client may have varying abilities to activate them, we are developing with the view to offer significantly higher input variations with unique messages to accommodate these differences.

Some software exists for tablets offering similar concepts, but lack the ability to use interchangeable finger guides or expand to other input methods that can cater to the client's unique abilities.

Firstly, I want to address the issue of my spelling - I'm dyslexic and while I rely on assistive tech to perform read-aloud and spell check, it is not perfect. So please accept my apologies for poorly worded sentences or mistakes.

The #AssistiveTech solution I am presenting was initially designed for a specific client and started as a large joystick with 8 possible positions connected to an Arduino that played pre-recorded MP3 messages. A second client leant of this and wanted one with 4 large push buttons instead of the joystick. Talking to various carers of clients with degenerative muscular disorders It became apparent that there is a need for this kind of support device but it needs to be easily adapted to individuals' needs based on their movement capabilities. To accommodate this, it was decided to make a voice control board that was independent of the input sensing board so variations could be more readily adapted.

Original Joystick based version
Original Joystick based version

Planned Capacitive Touch Version

The currently planned design is to use capacitive touch sensors that can sense through a 3D printed case that is customised to meet the clients requirements, Initial tests were performed using off-the-shelf touch sensors but they had varying sensitivity results so a custom PCB was designed with 14 buttons. The number of buttons was based on the inputs available on the Microchip MTCH108-I/SS sensing chip (7 inputs).

The sensitive areas were made to be large enough to easily contact, even with limited movement control, and to aid the finger placement, 3D printed guides were designed in Fusion 360 as shown below. While all 14 positions can play a custom message, the larger positions are intended for common answers such as yes/no. 

Interchangable finger guides
Interchangeable finger guides

All messages are recorded to match the user's specific requirements and can be in a voice and language to suit. In some cases, the message is repeated in two languages. One for the user and the other for the carer, which aids both parties.

The device is housed in an enclosure that has inter-changeable finger guides to allow easy modification to meet the users' needs. 

Currently, the electronics operate from a battery that will last around 4 days between charges. There is a plan to incorporate wireless charging to make it easier for a mobility-impaired person, avoiding the need to plug in small plugs. This will also improve the splash resistance and cleaning of the device. 

The recording of the messages is in an MP3 format and this can be performed on many recording devices including a mobile phone. Message length can be any length but it has been found that less than 10 seconds is ideal. Once recorded, the MP3 are named numerically to correspond to the button position and uploaded to the processor via a computer where it is seen as a USB flash drive. 

Because the target user often has poor movement control, the message will only play once per button every 15 seconds, avoiding repeat messages if they hold their finger on the button or press multiple times.

For the prototype, several shortcuts have been made including:

  • The processor board is currently using discrete inputs for each channel but is planned to be SPI
  • The voice board is an off the shelf unit from DFrobot 

Processor PCB.pdf

Processor PCB

Adobe Portable Document Format - 63.08 kB - 05/04/2023 at 10:36

Preview
Download

Touch PCB.pdf

Capacitive touch PCB

Adobe Portable Document Format - 181.04 kB - 05/04/2023 at 10:36

Preview
Download

Processor board.pdf

Processor board to determine which MP3 to play

Adobe Portable Document Format - 112.20 kB - 05/04/2023 at 10:31

Preview
Download

Touch board.pdf

Schematic for capacitive touch PCB

Adobe Portable Document Format - 118.15 kB - 05/04/2023 at 10:26

Preview
Download

  • 1 × MP3 module DFR0534

  • The finished product

    Tim05/22/2023 at 12:28 0 comments

    This video shows the finished prototype and discusses its operation and future plans.

  • An overview of the functional design

    Tim05/15/2023 at 10:45 0 comments

      In this video I look at how the different boards interact and some of the design decisions made to help the intended user not get multiple triggers due to not being able to accurately control their hand movement.

    • Message recordings can be different lengths and disable additional presses while playing
    • A button has a repeat delay to prevent shaky fingers from pressing multiple times
    • Finger held down delay prevents the message from repeating if the finger is not removed

  • Fully functional prototype

    Tim05/15/2023 at 10:25 0 comments

    In the past week, I have built up the new touch PCB, the processor PCB and the voice board as a complete system.

    Initial results are that it works well, has good sensitivity with no cross-talk like the first prototype touch PCB. There may still be an issue with false activations that needs further work.

    The video shows it working as a board out of the case, with some random phrases recorded, and then in the case (with some very dodgy sticking together)

  • Case design

    Tim05/04/2023 at 10:57 0 comments

    Here is the case design with interchangeable finger guides

  • New capacitive touch PCB comparison

    Tim05/04/2023 at 09:15 0 comments

    This is the second revision of the capacitive touch PCB, where I have made the guard rings much thinner and not a closed loop. On the other side of the board, I have removed the ground plane from behind each capacitive touch area. 

    Fingers crossed it performs more reliably than the first after reading the datasheet a second time.

    Side note: I am dyslexic and find reading large amounts of text extremely difficult, along with spelling correctly. As such you are likely to see numerous mistakes in my writing. I can't tell you how much I rely on #Assistivetech to help me with reading through read-aloud plugins on the computer, and spell checkers. 

  • Prototype 1 PCB built but has sensitivity issues

    Tim05/02/2023 at 12:42 0 comments

    The first version of the capacitive touch board has major sensitivity issues, where it is false triggering in some areas and not triggering at all in others.
    To experiment, I ground sections of the capacitive pads away as shown in the following video. This gave a significant improvement so off to PCB ways to get revision 2.

View all 6 project logs

Enjoy this project?

Share

Discussions

Tim wrote 04/29/2023 at 05:55 point

In 2022 I was asked by the carer of a person with a muscular disorder that prevented them from talking and limited their hand control to basic movements, to give them a way of communicating responses to questions. 

To accomplish this, a joystick was interfaced to an Arduino to detect up to 8 positions, which would trigger a voice module to speak a phrase or word such as 'Yes', 'No', 'Toilet', 'Drink' etcetera.

This worked extremely well for a period of time, but as the person's disorder became worse the need to identify new input methods became apparent leading to the need to develop a solution that is adaptable to multiple trigger requirements.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates