Close
0%
0%

Neltume: Language learning device for blind people

Language learning device for blind people

Similar projects worth following
Neltume helps blind people to memorize and exercise words and phrases in a foreign language through a multiple selection system.
The whole system is based in an Audio Interface manipulated through buttons, so the person doesn't need to see the device in order to use it.
A bast majority of the blind population in Third World countries are living in poverty conditions and are unemployed, that means that this device needs to be as cheap as possible.
In the first iteration of the project the whole system is in Spanish, intended for blind people who want to learn English.

The Problem

Learning a foreign language is a challenging experience. There is a lot of material available to solve different learning difficulties, not only digital material like Applications for  Smartphones or webbrowser (like Memrise, Duolingo, etc) but also written material like Books and Flashcards. Almost every single material available is fundamentally based in visual elements. So, what happened if the person is not capable to see the content itself? Which tools and materials are already in the market for blind people?

That's why I decided to design Neltume, which means in Mapudungun to become free, an Open Source Hardware tool that will help people with low vision or blindness to memorize new words and phrases on a foreign language.

On the first phase of this project I am designing the device for Spanish speakers, who wants to learn English

How the solution helps the World

In Latin America (and mostly in every Third World Country) there isn't a lot of opportunities for blind people in education, work and recreation (among other areas). Most of them haven't the opportunity to go to University or Institutes or even finish High-school, leaving them at the "mercy" of the State or their families.

I want to contribute creating new tools adapted for them. Learning a new Language can give you new Jobs opportunities, access to untranslated material and of course (and no less) a lot of fun and the possibility to meet new people and to form new relationships.

I have used tons of really useful material and applications (most of them open source or free) and now I want to give others an adaptation of what has really helped me.

You can use Neltume with earphones or speakers

How it works

The whole system is based in an audio interface and the learning method is through multiple selection.

The buttons 1, 2,3 and 4 are for multiple selection and the buttons 5, 6 and 7 are for navigation of the interface. Button 6 is for confirmation and it works for the questions and the navigation itself.

To the date are working the following functions

  • Learn and review the pronunciation of the alphabet
  • Learn and review of vocabulary in English.

How the menu works

How is learned a new word

How is exercised a word

Educational principle (not completely implemented) : Spaced repetition learning


Neltume is currently using a different system, but the idea is to use in the future a system called Spaced Repetition. In this system every box symbolized the frequency in which every word needs to be exercised and reviewed.

For example, for the words in the first Box, they are need to be exercised one time during the day. For the words in the second Box, they are need to be exercised one time every 2 days, etc.

What is currently used is reviewing the words through a list of conflictive words. The conflictive words are those whose ratio between the good answers/total answers is lower in the list of the words already learned.

Specs

The Software has been programmed with Python 3, which will be execute it in a C.H.I.P. Pro (Previously I was using Raspberry Pi A+ ).

The voices of the menu, options and vocabulary (Spanish and English) are from IBM Bluemix Text to Speech Software.

Test Video

Standard Tesselated Geometry - 611.21 kB - 09/04/2017 at 01:13

Download

Standard Tesselated Geometry - 146.57 kB - 09/04/2017 at 01:13

Download

gzip - 26.70 MB - 08/09/2017 at 01:23

Download

plain - 1.03 kB - 03/21/2017 at 10:18

Download

  • 1 × C.H.I.P. Pro
  • 7 × Arcade Button (Sanwa alike)
  • 1 × Headphones or speakers
  • 1 × Lithium Ion Battery 1000 mAh

  • New 3D Printed Case and test of new buttons

    Hernán Valdés08/21/2017 at 01:30 0 comments

    After testing and carrying the device for some days with me, I realized that the size of the second prototype was too much and not really comfortable, so I decided to test not only decreasing the size of the case itself, but also changing the buttons.



    And this is the comparison with the second prototype.



  • 3D Case

    Hernán Valdés08/10/2017 at 18:01 0 comments

    I tried yesterday to print the second part for the case, but there were some problems with the filament during the night and today the print work wasn't totally finished. It was good enough to use some tape to close it and now I can made some test with the different sensations and dimensions. I am planning to cut with a CNC the next prototype.

  • 3D Printed Enclosure and migration to C.H.I.P. Pro

    Hernán Valdés08/09/2017 at 01:49 0 comments

    I moved back from the country were I was studying abroad and developing Neltume during the last year (Germany) to my Homeland (Chile) and during the process I had to turn apart the prototype that I initially made out of a Christmas Box.

    Last Week I decided that I wanted something more elaborated and in the FabLab of my University I was allowed to 3D print a new Prototype. It was much much slower than my expectations and totally unpractical for futures test prototypes, so I plan during the week to use a CNC Laser Cutter in order to create a new enclosure.

    Also I tested the C.H.I.P. Pro and after some changes in the the button controller script (that was initially thought for the Raspberry Pi) I could make it work perfectly.

    To make work the GPIO ports of the C.H.I.P. Pro I used the library of xtacocorex and it works perfectly.
    Some friends of the Fab Lab gave me good feedback about the interface and usability, like the size and position of the buttons. I am reconsidering the form factor, it could be a good idea to use a Joystick to control the navigation and selection option and it would be interesting to design a prototype with the form a usability of a NunChuck (Nintendo) so the user can control the whole system with only a hand.

    Other possibility is to replace the actual arcade buttons with small micro push switch. I like how it looks the design that the people of Adafruit created for the BMO 3D printed project and it could decrease the size significantly.

     

  • Change in the microprocessor

    Hernán Valdés08/03/2017 at 22:43 0 comments

    After trying unsuccessfully to improve the audio output of the Raspberry and seeing that I was adding more and more components to the project, I found a very good alternative to the Raspberry Pi, CHIP Pro.

    With CHIP Pro I don't need to  buy an external LiPo Charger (like the Powerboost 500) and an external USB audio output, because it has a digital audio output. This combined with a lower price, makes the CHIP Pro an excellent alternative to the Raspberry Pi.

    Now I am doing the migration to the new micro processor and a new Enclosure for the prototype.

  • Update on Audio Interface Navigation and reviewing system

    Hernán Valdés03/28/2017 at 13:36 0 comments

    Navigation:

    • In the moment that you access to a menu/submenu it is said the name of that menu and goes back to the first option of that menu, exception in the submenus where you go directly to the second option (the first one lets the user go back to the previous menu)
    • When you press button 4 or 5 (top buttons) you can directly go back to the previous menu (it was a pain in the ass to go to the first option to go back)
    • Add the option to review the button enumeration

    Reviewing system (there is a lot of work to be done):

    • Before it was used a very basic system, where the worst word was always reviewed first, now I am implementing the Leitner System for spaced repetition, which now contains 4 reviewing options:
      • 1: Review 2 times the day
      • 2: Review 1 time the day
      • 3: Review 1 time every 2 days
      • 4: review 1 time in the week
    • When the user turn the device on, it is checked if the day is the same as the last reviewed word and check which words need to be update (change the leitner box

  • Prototype B - First test

    Hernán Valdés03/21/2017 at 10:32 0 comments

  • Prototype B

    Hernán Valdés03/21/2017 at 10:30 0 comments

    After reading documents about the situation of blind people on Latinamerica and specifically in Chile, where only between 15-20% of the total blind population can actually read in braille or other systems, a rethink and redesign was necessary at least for an early Prototype.

    After rethinking the actual design and button distribution of the device I changed the distribution to 3 buttons (bottom) for navigation (Right - Confirmation - Left) and 4 buttons (up) for multiple alternative selection.

    Prototype B is implemented in a Raspberry Pi 3B, because I can easily transfer the file through SSH via WiFi.

    The sound of the Raspberry Pi via Jack 3.5mm is just horrible and noisy, one alternative is to use a USB Audio Card.

  • Sample of the structure of the learning system

    Hernán Valdés03/21/2017 at 10:13 0 comments

    A lot of variables aren't really considered for the first prototype of the learning and reviewing system.

    One of them (and very important) is time and the generation of reviewing frequency.

    Until the first prototype and the arrival of the RTC there is no option for the tracking of time on the raspberry pi.

    Also, there is a difference between the actual learning system of the alphabet. I was dealing with the amount of letters to learn per session, and the exercises were a randomly distribution of those letters. After coding the word learning algorithm, I realize that a big exception (when you were in letter 24 you only have 2 letters left instead of 6 for the rest of the code) could be skip.

    So, the person learn one letter, and the other five (to complete the six buttons) are randomly selected from a the alphabet (which doesn't include the letter that the person is learning)

    Also, there is still some general problems with the conception of the reviewing code. One of them, is that I'm actually not tracking the number of right/wrong answers. Should it be in the same json file or another?

  • Example of the audio interface

    Hernán Valdés03/21/2017 at 10:10 0 comments

    Example of how is going the development of the audio interface and word learning process. Everything is still in my computer, waiting for the arrival of the buttons,

  • Rethinking the Prototype

    Hernán Valdés03/21/2017 at 10:07 0 comments

    I have already generated some of the basic functions available in Prototype A Software

    • Learn Alphabet
    • Review Alphabet
    • Learn new words

    In order to keep logs (with dates) of how many times the user had used Neltume it is necessary to use a Real Time Clock (RTC) that raspberry doesn't include in his original design. So, until the RTC arrives we are only using this format make a log of the word:

    log –> #id_word:[#goods, #total]

    In this first prototype (A) I want to develop a laptop only interface. Using the keyboard distribution previously shown on Log2. Prototype B will consist of a model inspired on the braille-keyboard, with 9 main buttons a.k.a. Neltume XL and Prototype C will have different configuration of buttons, and possibly some of them will have an arrow shape to navigate through the menu. Maybe it could be an easier interface use only four option button (instead of the six actual), two for navigation and one for confimation. Only tests can confirm which one is easier.

    Also, thinking of other possible uses for Neltume, it came to mind the possibility to use it like an interactive audio book and for test preparations.

View all 12 project logs

  • 1
    Print the parts for the enclosure
    1.stl

    2.stl

  • 2
    Add the buttons (Super Glue requiered)

    You can use a different glue, but for me Super Glue was very effective The dimensions of the microswitch are 12x12mm and 6mm height.

  • 3
    Solder a common wire for the buttons


    A and C, B and D are connected, so what we need to do is to connect every C with the A of the next button. After that we can give with only one cable 3.3v to the whole system.

View all 15 instructions

Enjoy this project?

Share

Discussions

dearuserhron wrote 05/23/2021 at 08:55 point

It is a general purpose computer. I wish this thing has some programming capabilities for blind people. You know, input some code using buttons and then execute it.

upd: or maybe monofonic melody sequencer, like in old Nokia phones.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates