Neltume: Language learning device for blind people

Language learning device for blind people

Similar projects worth following

Learning a foreign language is a challenging experience. During my different experiences trough the language acquisition I had struggles, at the time of memorize new vocabulary and grammar and I could only imaging how difficult could it be for blind people.

There is a lot of useful methods and digital tools available to make easier that challenge, but unfortunately almost every tool out there requires the vision of the user, in order to see the content itself, even the simplest ones like Flashcards.

That's why I decided to design Neltume, which means in Mapudungun to become free, an Open Source Hardware tool that will help people with low vision or blindness to memorize new words and phrases on a foreign language.

On the first phase of this project I am designing the device for Spanish speakers, who wants to learn English.

The main functions of the devices in an early stage will be

  • Learn and review the pronunciation of the alphabet
  • Learn and review of vocabulary in English.

The Software has been programmed with Python 3.x, which will be execute it in a Raspberry Pi A+ but tested through a Raspberry Pi 3b taking advantage of the on build wifi.

The voices of the menu, options and vocabulary (Spanish and English) are from IBM Bluemix Text to Speech Software.

plain - 1.03 kB - 03/21/2017 at 10:18


  • 1 × Raspberry Pi 3b For prototyping and test through ssh
  • 1 × Raspberry Pi A+
  • 7 × Arcade Button (Sanwa alike)
  • 1 × USB Audio adapter with volume regulator

  • Update on Audio Interface Navigation and reviewing system

    Hernán Valdés03/28/2017 at 13:36 0 comments


    • In the moment that you access to a menu/submenu it is said the name of that menu and goes back to the first option of that menu, exception in the submenus where you go directly to the second option (the first one lets the user go back to the previous menu)
    • When you press button 4 or 5 (top buttons) you can directly go back to the previous menu (it was a pain in the ass to go to the first option to go back)
    • Add the option to review the button enumeration

    Reviewing system (there is a lot of work to be done):

    • Before it was used a very basic system, where the worst word was always reviewed first, now I am implementing the Leitner System for spaced repetition, which now contains 4 reviewing options:
      • 1: Review 2 times the day
      • 2: Review 1 time the day
      • 3: Review 1 time every 2 days
      • 4: review 1 time in the week
    • When the user turn the device on, it is checked if the day is the same as the last reviewed word and check which words need to be update (change the leitner box

  • Prototype B - First test

    Hernán Valdés03/21/2017 at 10:32 0 comments

  • Prototype B

    Hernán Valdés03/21/2017 at 10:30 0 comments

    After reading documents about the situation of blind people on Latinamerica and specifically in Chile, where only between 15-20% of the total blind population can actually read in braille or other systems, a rethink and redesign was necessary at least for an early Prototype.

    After rethinking the actual design and button distribution of the device I changed the distribution to 3 buttons (bottom) for navigation (Right - Confirmation - Left) and 4 buttons (up) for multiple alternative selection.

    Prototype B is implemented in a Raspberry Pi 3B, because I can easily transfer the file through SSH via WiFi.

    The sound of the Raspberry Pi via Jack 3.5mm is just horrible and noisy, one alternative is to use a USB Audio Card.

  • Sample of the structure of the learning system

    Hernán Valdés03/21/2017 at 10:13 0 comments

    A lot of variables aren't really considered for the first prototype of the learning and reviewing system.

    One of them (and very important) is time and the generation of reviewing frequency.

    Until the first prototype and the arrival of the RTC there is no option for the tracking of time on the raspberry pi.

    Also, there is a difference between the actual learning system of the alphabet. I was dealing with the amount of letters to learn per session, and the exercises were a randomly distribution of those letters. After coding the word learning algorithm, I realize that a big exception (when you were in letter 24 you only have 2 letters left instead of 6 for the rest of the code) could be skip.

    So, the person learn one letter, and the other five (to complete the six buttons) are randomly selected from a the alphabet (which doesn't include the letter that the person is learning)

    Also, there is still some general problems with the conception of the reviewing code. One of them, is that I'm actually not tracking the number of right/wrong answers. Should it be in the same json file or another?

  • Example of the audio interface

    Hernán Valdés03/21/2017 at 10:10 0 comments

    Example of how is going the development of the audio interface and word learning process. Everything is still in my computer, waiting for the arrival of the buttons,

  • Rethinking the Prototype

    Hernán Valdés03/21/2017 at 10:07 0 comments

    I have already generated some of the basic functions available in Prototype A Software

    • Learn Alphabet
    • Review Alphabet
    • Learn new words

    In order to keep logs (with dates) of how many times the user had used Neltume it is necessary to use a Real Time Clock (RTC) that raspberry doesn't include in his original design. So, until the RTC arrives we are only using this format make a log of the word:

    log –> #id_word:[#goods, #total]

    In this first prototype (A) I want to develop a laptop only interface. Using the keyboard distribution previously shown on Log2. Prototype B will consist of a model inspired on the braille-keyboard, with 9 main buttons a.k.a. Neltume XL and Prototype C will have different configuration of buttons, and possibly some of them will have an arrow shape to navigate through the menu. Maybe it could be an easier interface use only four option button (instead of the six actual), two for navigation and one for confimation. Only tests can confirm which one is easier.

    Also, thinking of other possible uses for Neltume, it came to mind the possibility to use it like an interactive audio book and for test preparations.

  • Beginning of the Software adventure

    Hernán Valdés03/21/2017 at 10:03 0 comments

    I have started developing a simple script and tried how to play sounds with python. I found MPG123, an open source alternative that plays sounds trough the terminal. One of the available alternatives is to use the Subprocess library and the function Popen to run it from Python. I pretend to use for Prototype A arcade buttons, specially those alike to Sanwa buttons. They arrived in 3 weeks, in the meantime I will use the following setup, simulating the buttons. So, I need to be able to identify every time when the user type a character on the Keyboard. And I found, after some iterations, the excellent library py-getch, which can be download with pip ($pip install py-getch).

    In the meantime there is no Raspberry Pi prototype, this will be the configuration of the keyboard.

    I'm going to save all the data on JSON files, which are directly human-legible with a simple text editor.

    On the other hand, the structure of the menu will follow the structure of the chart. Before some one could learn new words, one needs to learn the alphabet in order to keep going. But how are going to be processed the words and characters after learned?

  • Prototype A

    Hernán Valdés03/21/2017 at 09:59 0 comments

    Prototype A is inspired in the setup of the braille Keyboards.

    The Braille Matrix is represented by 6 Dots. Those dots are projected under 6 different buttons. Additionally are 3 more buttons, one for spacing/confirmation and two for the devices navigation. Generating a Prototype with the same setup of a braille Keyboard, allow the person to learn how to write the word as well as to memorize using a multiple selection option.

View all 8 project logs

Enjoy this project?



Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates