Close
0%
0%

Eye-to-Speech Module

We want to help people who are unable to communicate verbally and cannot use sign language by providing a wearable “eye-to-speech" solution.

Similar projects worth following
As part of our student project called Speak4Me we want to help people who are unable to communicate verbally and suffer from severe physical disabilities by providing a wearable “eye-to-speech” solution that allows basic communication. By using our device, users can express themselves with customizable phrases using situational and self-defined profiles that allow them to communicate with their environment with 128 phrases.
Through simple eye movements within certain timeframes and directions, Speak4Me will output audible predefined phrases and allow basic communication. A simple Web-Interface allows the creation of profiles, without any limitation on word complexity or length.
Following a simple step by step video tutorial Speak4Me can be built for around 100$, which is way below given solutions. All product related information regarding needed materials and construction manuals are provided via a central homepage.

1.Introduction

As part of our student project called Speak4Me we want to help people who are unable to communicate verbally and suffer from severe physical disabilities by providing a wearable “eye-to-speech” solution that allows basic communication. By using our device, users can express themselves with customizable phrases using situational and self-defined profiles that allow them to communicate with their environment with 128 phrases.

Through simple eye movements within certain timeframes and directions, Speak4Me will output audible predefined phrases and allow basic communication. A simple Web-Interface allows the creation of profiles, without any limitation on word complexity or length.

Following a simple step by step video tutorial Speak4Me can be built for around 100$, which is way below given solutions. All product related information regarding needed materials and construction manuals are provided via a central homepage.

We won the 2021 Hackaday Prize challenge 03: Reimagine Supportive Tech Finalist Award, which is endowed with $500:  WINNERS OF THE HACKADAY PRIZE SUPPORTIVE TECH CHALLENGE

Project GitHub page including step-by-step video tutorials and the no-code customization tool: https://speak4me.github.io/speak4me/

1.1 Motivation

The UN has defined 17 Sustainability Goals (SDGs) that all members of the UN have adopted in 2015. Main focus of the SDGs is the desire to create a shared blueprint for more prosperity for the people and the environment by 2030 (cf. UN, 2015). Speak4Me embraces SDG 10 which aims to remove inequalities within and among countries, especially in poorer regions. Within this SDG our solution can bring value to people with speech related disabilities by giving them a way to communicate. Worldwide over 30 Million people are mute and therefore unable to communicate using their voice. (cf. ASHA, 2018)

Our main focus are patients with serious mental and or physical disabilities, that are not only verbally mute, but are also unable to communicate using gestures and other means. This includes patients with ALS, Apraxia and other degenerative diseases that lead to a slow loss of control over body functions, as well as patients affected by spine damage that makes it impossible to communicate via body language and other means. Birth defects, damage to vocal cords and accidents that damage relevant organs and many other conditions, can also lead to muteness.

Physical muteness is rarely an isolated condition. Most commonly it is just the result of other underlying conditions like deaf-muteness which is the most common reason, for people to be unable to communicate verbally (cf. Destatis 2019). 

Speak4Me could also provide help to patients with temporary conditions. Just in Germany over 250.000 people suffer from strokes every year (cf. Destatis 2019). During recovery our solution can help strongly affected patients to communicate with their environment, which otherwise might be impossible. Other nursing cases like patients suffering from the Locked-In syndrome could also benefit, as only control of the eyes is required.

1.2 Objectives

Speak4Me wants to provide an affordable, customizable, easy to build and use device to support handicapped people to communicate with their environment. Language synthesizers and speech computers exist, but are very expensive which can make them unaffordable depending on socio economic background including the country of residence. Our target is to deliver a solution below 100$ in total cost, to reach as many affected people in the world as possible. By providing blueprints and code basis of the entire solution we want to encourage others to build upon our work an improve or adapt it.

1.3 Background

Our entire solution is based on the open Arduino platform, which is built around standardized hardware and a basic coding language. Previous projects based on the Arduino platform, have shown promising results...

Read more »

3d-printing-stl-files.zip

All required 3d printing files you need to print/order components to better adjust the sensors on your glass and have a custom fit box for the Arduino and the circuit board. For more information check our DIY video tutorial: https://speak4me.github.io/speak4me/step-1-what-you-need.html

Zip Archive - 111.27 kB - 08/07/2021 at 21:06

Download

Gerber.zip

All required Gerber files if you like to like professional designed circuit board instead of messy cables. For more information check our DIY video tutorial: https://speak4me.github.io/speak4me/step-1-what-you-need.html

Zip Archive - 34.76 kB - 08/07/2021 at 21:06

Download

architectural_overview_EUeKgRyTmt.png

Architecture schema of our solution

Portable Network Graphics (PNG) - 457.22 kB - 08/07/2021 at 21:04

Preview
Download

arduino_code.ino

The coding for the Arduino. To customize your expression, checkout our customization tool on our project, you do not need any coding skills to customize your required phrases. #noCode

ino - 26.31 kB - 08/07/2021 at 21:04

Download

  • 4 × QTR-1RC Reflectance Sensor
  • 1 × Parallax Emic 2 Text-to-Speech Module
  • 1 × 3.5mm angle/straight jack cable (AUX, 1.5 m)
  • 1 × Insulated braided copper wire (3m)
  • 1 × USB 2.0 cable (EASY male A > male mini-B, 0,5m)

View all 14 components

View all 2 project logs

  • 1
    Instruction overview

    To assist you in making your own Speak4Me glasses, we have provided a list of necessary steps. Since we know how challenging written instructions can be, we have also created How-To videos in which we show the steps. The steps are divided into the following parts:

    Hardware Setup

    2. Soldering and cutting the resistances
    3. Trimming the AUX connector
    4. Making the AUX connector fit
    5. Soldering the AUX connector
    6. Soldering the USB connector
    7. Soldering the connection for the internal speaker
    8. Soldering the transistor for the keep alive circuit
    9. Cutting the leftovers of the transistor for the keep alive circuit
    10. Placing the Arduino and EMIC
    11. Soldering the Arduino and EMIC
    12. Cutting the leftovers of the Arduino and EMIC
    13. Soldering and cutting the leftovers of the LED

    Glass setup

    14. Remove the USB-cable insulation
    15. Marking the connections
    16. Preparing the sensors
    17. Mounting the sensors

    How to use the Profile Customization

    18. How to customize your profiles
    19. How to customize the settings
    20. How to use the upload and download functionalities

    Load the software on your Arduino

    21. Load the pre-configured code to the Arduino

    Advices on how to use the glass

    22. Practical advice's to use your Speak4Me glass

  • 2
    Soldering and cutting the resistances

    To start with the assembly of the necessary hardware, prepare a solid workplace and have the following tools ready to use:

    • Pliers
    • Rubber band
    • Wire stripper
    • Voltmeter
    • Nail file
    • Sandpaper
    • Soldering iron
    • Solder wire
    • A helping hand or something to fix it

    You will also need the circuit board and most of the components you had to order before.

    In case you never soldered something before, we highly recommend that you first familiarize yourself with the process by asking a hardware-addicted friend or starting with one of the various tutorials to be found on youtube and solder a few things to get familiar with the process. You will not need any advanced soldering skills for this project, tho, so everyone should be able to perform the soldering steps on his own in a manageable amount of time. Once you feel confident, get back to the board and start soldering.

    Start by consecutively positioning the resistors, fasten them using the rubber band and then solder them to the board. Afterwards, cut off the protruding legs of the resistors.

    The whole pcb assembly process has been documented in the following video (Click here to go to the first step of the video.):

  • 3
    Trimming the AUX connector

    Most aux connectors have small brackets on their bottom side, which shall help to keep the connector in place. We do not need them, so please just clip them off using your pliers for example.

    Click here to go to the corresponding part of our video.

View all 22 instructions

Enjoy this project?

Share

Discussions

LowellBode wrote 10/28/2022 at 15:48 point

The project is aimed at solving one of the major challenges in the healthcare industry: communication between caregivers and those who need their help. The wearable technology will assist caregivers during home visits by translating eye movements into text or speech based on a person's preference. Reading https://trendblog.net/how-to-create-an-awesome-junior-software-engineer-resume-in-2022/ article also helps you to know how to create an awesome resume.

  Are you sure? yes | no

nick vuono wrote 08/31/2021 at 21:21 point

thank you for submitting this. I’d tried looking into similar technology a couple years ago but didn’t have a lot of luck.


one thing to consider as a point of extension is using the eye movement as input to a “switch control” device that would be used to control existing AAC speech apps on ios or android devices. Most people don’t know how easy it is to go to “IOS settings > accessibility > switch control” and start using switch control inputs.

The simplest implementation for a Bluetooth “switch control” device is just a Bluetooth keyboard input.  You would just be mapping eye movement commands and sending a keypress of 7 for “move to previous item”,  8 for “move to next item”, 9 for “tap”

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates