close-circle
Close
0%
0%

Neltume: Language learning device for blind people

Language learning device for blind people

Similar projects worth following

The Problem

Learning a foreign language is a challenging experience. There is a lot of material available to solve different learning difficulties, not only digital material like Applications for  Smartphones or webbrowser (like Memrise, Duolingo, etc) but also written material like Books and Flashcards. Almost every single material available is fundamentally based in visual elements. So, what happened if the person is not capable to see the content itself? Which tools and materials are already in the market for blind people?

That's why I decided to design Neltume, which means in Mapudungun to become free, an Open Source Hardware tool that will help people with low vision or blindness to memorize new words and phrases on a foreign language.

On the first phase of this project I am designing the device for Spanish speakers, who wants to learn English

How the solution helps the World

In Latin America (and mostly in every Third World Country) there isn't a lot of opportunities for blind people in education, work and recreation (among other areas). Most of them haven't the opportunity to go to University or Institutes or even finish High-school, leaving them at the "mercy" of the State or their families.

I want to contribute creating new tools adapted for them. Learning a new Language can give you new Jobs opportunities, access to untranslated material and of course (and no less) a lot of fun and the possibility to meet new people and to form new relationships.

I have used tons of really useful material and applications (most of them open source or free) and now I want to give others an adaptation of what has really helped me.

How it works

The whole system is based in an audio interface and the learning method is through multiple selection.

The buttons 1, 2,3 and 4 are for multiple selection and the buttons 5, 6 and 7 are for navigation of the interface. Button 6 is for confirmation and it works for the questions and the navigation itself.

To the date are working the following functions

  • Learn and review the pronunciation of the alphabet
  • Learn and review of vocabulary in English.

Specs

The Software has been programmed with Python 3, which will be execute it in a C.H.I.P. Pro (Previously I was using Raspberry Pi A+ ).

The voices of the menu, options and vocabulary (Spanish and English) are from IBM Bluemix Text to Speech Software.

Test Video

sla - 112.00 kB - 08/10/2017 at 03:23

download-circle
Download

sla - 123.91 kB - 08/10/2017 at 03:23

download-circle
Download

gzip - 26.70 MB - 08/09/2017 at 01:23

download-circle
Download

plain - 1.03 kB - 03/21/2017 at 10:18

download-circle
Download

  • 1 × C.H.I.P. Pro
  • 7 × Arcade Button (Sanwa alike)
  • 1 × Headphones or speakers
  • 1 × Lithium Ion Battery 1000 mAh

  • 3D Case

    Hernán Valdés08/10/2017 at 18:01 0 comments

    I tried yesterday to print the second part for the case, but there were some problems with the filament during the night and today the print work wasn't totally finished. It was good enough to use some tape to close it and now I can made some test with the different sensations and dimensions. I am planning to cut with a CNC the next prototype.

  • 3D Printed Enclosure and migration to C.H.I.P. Pro

    Hernán Valdés08/09/2017 at 01:49 0 comments

    I moved back from the country were I was studying abroad and developing Neltume during the last year (Germany) to my Homeland (Chile) and during the process I had to turn apart the prototype that I initially made out of a Christmas Box.

    Last Week I decided that I wanted something more elaborated and in the FabLab of my University I was allowed to 3D print a new Prototype. It was much much slower than my expectations and totally unpractical for futures test prototypes, so I plan during the week to use a CNC Laser Cutter in order to create a new enclosure.

    Also I tested the C.H.I.P. Pro and after some changes in the the button controller script (that was initially thought for the Raspberry Pi) I could make it work perfectly.

    To make work the GPIO ports of the C.H.I.P. Pro I used the library of xtacocorex and it works perfectly.
    Some friends of the Fab Lab gave me good feedback about the interface and usability, like the size and position of the buttons. I am reconsidering the form factor, it could be a good idea to use a Joystick to control the navigation and selection option and it would be interesting to design a prototype with the form a usability of a NunChuck (Nintendo) so the user can control the whole system with only a hand.

    Other possibility is to replace the actual arcade buttons with small micro push switch. I like how it looks the design that the people of Adafruit created for the BMO 3D printed project and it could decrease the size significantly.

     

  • Change in the microprocessor

    Hernán Valdés08/03/2017 at 22:43 0 comments

    After trying unsuccessfully to improve the audio output of the Raspberry and seeing that I was adding more and more components to the project, I found a very good alternative to the Raspberry Pi, CHIP Pro.

    With CHIP Pro I don't need to  buy an external LiPo Charger (like the Powerboost 500) and an external USB audio output, because it has a digital audio output. This combined with a lower price, makes the CHIP Pro an excellent alternative to the Raspberry Pi.

    Now I am doing the migration to the new micro processor and a new Enclosure for the prototype.

  • Update on Audio Interface Navigation and reviewing system

    Hernán Valdés03/28/2017 at 13:36 0 comments

    Navigation:

    • In the moment that you access to a menu/submenu it is said the name of that menu and goes back to the first option of that menu, exception in the submenus where you go directly to the second option (the first one lets the user go back to the previous menu)
    • When you press button 4 or 5 (top buttons) you can directly go back to the previous menu (it was a pain in the ass to go to the first option to go back)
    • Add the option to review the button enumeration

    Reviewing system (there is a lot of work to be done):

    • Before it was used a very basic system, where the worst word was always reviewed first, now I am implementing the Leitner System for spaced repetition, which now contains 4 reviewing options:
      • 1: Review 2 times the day
      • 2: Review 1 time the day
      • 3: Review 1 time every 2 days
      • 4: review 1 time in the week
    • When the user turn the device on, it is checked if the day is the same as the last reviewed word and check which words need to be update (change the leitner box

  • Prototype B - First test

    Hernán Valdés03/21/2017 at 10:32 0 comments

  • Prototype B

    Hernán Valdés03/21/2017 at 10:30 0 comments

    After reading documents about the situation of blind people on Latinamerica and specifically in Chile, where only between 15-20% of the total blind population can actually read in braille or other systems, a rethink and redesign was necessary at least for an early Prototype.

    After rethinking the actual design and button distribution of the device I changed the distribution to 3 buttons (bottom) for navigation (Right - Confirmation - Left) and 4 buttons (up) for multiple alternative selection.

    Prototype B is implemented in a Raspberry Pi 3B, because I can easily transfer the file through SSH via WiFi.

    The sound of the Raspberry Pi via Jack 3.5mm is just horrible and noisy, one alternative is to use a USB Audio Card.

  • Sample of the structure of the learning system

    Hernán Valdés03/21/2017 at 10:13 0 comments

    A lot of variables aren't really considered for the first prototype of the learning and reviewing system.

    One of them (and very important) is time and the generation of reviewing frequency.

    Until the first prototype and the arrival of the RTC there is no option for the tracking of time on the raspberry pi.

    Also, there is a difference between the actual learning system of the alphabet. I was dealing with the amount of letters to learn per session, and the exercises were a randomly distribution of those letters. After coding the word learning algorithm, I realize that a big exception (when you were in letter 24 you only have 2 letters left instead of 6 for the rest of the code) could be skip.

    So, the person learn one letter, and the other five (to complete the six buttons) are randomly selected from a the alphabet (which doesn't include the letter that the person is learning)

    Also, there is still some general problems with the conception of the reviewing code. One of them, is that I'm actually not tracking the number of right/wrong answers. Should it be in the same json file or another?

  • Example of the audio interface

    Hernán Valdés03/21/2017 at 10:10 0 comments

    Example of how is going the development of the audio interface and word learning process. Everything is still in my computer, waiting for the arrival of the buttons,

  • Rethinking the Prototype

    Hernán Valdés03/21/2017 at 10:07 0 comments

    I have already generated some of the basic functions available in Prototype A Software

    • Learn Alphabet
    • Review Alphabet
    • Learn new words

    In order to keep logs (with dates) of how many times the user had used Neltume it is necessary to use a Real Time Clock (RTC) that raspberry doesn't include in his original design. So, until the RTC arrives we are only using this format make a log of the word:

    log –> #id_word:[#goods, #total]

    In this first prototype (A) I want to develop a laptop only interface. Using the keyboard distribution previously shown on Log2. Prototype B will consist of a model inspired on the braille-keyboard, with 9 main buttons a.k.a. Neltume XL and Prototype C will have different configuration of buttons, and possibly some of them will have an arrow shape to navigate through the menu. Maybe it could be an easier interface use only four option button (instead of the six actual), two for navigation and one for confimation. Only tests can confirm which one is easier.

    Also, thinking of other possible uses for Neltume, it came to mind the possibility to use it like an interactive audio book and for test preparations.

  • Beginning of the Software adventure

    Hernán Valdés03/21/2017 at 10:03 0 comments

    I have started developing a simple script and tried how to play sounds with python. I found MPG123, an open source alternative that plays sounds trough the terminal. One of the available alternatives is to use the Subprocess library and the function Popen to run it from Python. I pretend to use for Prototype A arcade buttons, specially those alike to Sanwa buttons. They arrived in 3 weeks, in the meantime I will use the following setup, simulating the buttons. So, I need to be able to identify every time when the user type a character on the Keyboard. And I found, after some iterations, the excellent library py-getch, which can be download with pip ($pip install py-getch).

    In the meantime there is no Raspberry Pi prototype, this will be the configuration of the keyboard.

    I'm going to save all the data on JSON files, which are directly human-legible with a simple text editor.

    On the other hand, the structure of the menu will follow the structure of the chart. Before some one could learn new words, one needs to learn the alphabet in order to keep going. But how are going to be processed the words and characters after learned?

View all 11 project logs

  • 1
    Settings for the Microprocessor (C.H.I.P. Pro)
    1. Connect the C.H.I.P. Pro to a computer via the micro USB cable
    2. Flash C.H.I.P. Pro with a sample image or Debian Linux using the Flasher in the Internet Browser Chrome
    3. If you are using a Mac or Linux computer you can use Screen in order to control the microprocessor. Follow this instructions
    4. Connect to WiFi following this instructions
  • 2
    Installing programs and libraries

    We are going to install the Software necessary for Neltume. The Super User is required and the default password is chip.

    1. sudo apt-get install mpg123 python3 git
    2. git clone https://github.com/hfvaldesg/neltume

    For the next step you are going to connect through SSH with the C.H.I.P. Pro

    1. Download the following .deb file
    2. Send it through SSH to the C.H.I.P. Pro
    3. sudo dpkg -i python-chip-io_0.5.9-1_armhf.deb
  • 3
    Wiring the buttons

    There is going to be a common cable for every single button, so you need to connect one of the terminals of the button with each other.

    You need also a cable for every button to connect with a GPIO of the CHIP.

View all 4 instructions

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates