Lighting Score

Description

Visual impairment has always been one of the disability conditions that have a severe negative impact on a person's life. According to research data recently, the number of people who have visual impairment has been increasing rapidly these years. Therefore, the issue of how to help improve the experience of these visually impaired people has become an issue that cannot be ignored.
Lighting Score is designed to help people with visual impairment to learn and practise piano music better. It works with a light bar indicator and an APP to help improve the experience of visually impaired people searching for piano scores, learning new scores and practicing continuously. Lighting Score will enrich the daily lives of people with visual impairments, meet their musical entertainment needs and strengthen their connections with the community and the world.

Details

For our project, we decide to design a assistive product which can help visual impairment people in their daily life. After some research we found that entertainment is an interesting area that we can work on to help them. So we did some interviews and After some group discussion and talking with our stakeholders, we found that they had more needs in the music area. They mentioned many problems while learning and practicing when they spoke about music. This means that our stakeholders have special needs in music.

After we researched on existing products, we found the main problems for visual impairment people are find music resources, how to identify the score and hard to play/practice music. Because music is a big area, we decided to focus on the piano which our stakeholders want to learn to design our project.

After summarizing the pain points, we brainstormed different ideas for solving the problems. We have 2 directions: the first one is to improve the experience of the VI community in finding electronic scores and reading them by means of an interface. The second direction is to find other ways of interaction which allow them to practice without having to read the score.

So we came up with the idea: why not use lights to tell the user the position of the keys they need to play? The light can cover a larger area and for the VI it would be more efficient to identify the keys that need to be played. In addition, the use of lights is the most feasible and inexpensive way to locate the keys without disturbing the user's playing experience. For the general user, even beginners can play well with lights. This would be a product with an inclusive design. We have considered the possibility of displaying a flow of notes in the form of an interface for the user while playing. This would allow the user to see their progress as they play.

The next coming process will post in project logs, thank you for your reading. :)

Files

Lighting score poster.png

Our project display image

Portable Network Graphics (PNG) - 23.25 MB - 05/17/2023 at 10:35

Preview

Download

Components

1 × Freaduino UNO

1 × Keypad Membrane dot Matrix 4 X 4 Arduino Matrix Keypad Keyboard Key

1 × Pixel Led Strips

Project Logs

Collapse

The next step of our project
TianxinXue • 05/17/2023 at 11:00 • 0 comments
We invited two stakeholders to have a test on our prototype. Both of them were happy with current using experience with a little suggestion gave our for future developing. They think we can add some feedback while users press the wrong key and have some instruction of before use the product.
Our prototype turned out to be success and testers were satisfied with using it. However, there still have some issues could be optimized to make the proto- type perfect.
For the Light Bar Indicator:
- Add tutorial of how to use the keypad to select different functions
- Try to achieve function of scanning printed score, so users can scan their physical score in music library for practice
- Misc lick notification on keyboard(audio & visual feedback) for better practice
For the UI:
- Improve workflow for better using experience
- More functions for inclusive design could be added
- Achieve connection of APP and indicator in real
Process of our product
TianxinXue • 05/17/2023 at 10:33 • 0 comments
After we figured out our final solution, we spent several days building our first prototype. The main parts of our prototype are Pixel LED strips and an Freaduino Uno. To ensure the LED strip can map the light on keys precisely, we divided them into different parts and then welded them based on the distance between each key. My teammate Diwen and Lilin made the code for the Arduino. They found many example codes online with the functions we wanted and combined them based on our needs.

Although this prototype looks very rough, the main function that uses the light to tell users which key to play works and is ready for the first user test.

1st Prototype

The first user test's goal is to determine if our stakeholders can see the light to find the key precisely and can they use this way to learn the new music without read the score.
We contacted several stakeholders and successfully invited them to join in this prototype test. All stakeholders showed great passion for testing and positively shared their idea on the prototype. They provided some useful feedback for us to improve our project and what they also need.

Shortages:
- Still want to see the music sheet while using this product
- The brightness of the LED is too high
- There is no audio feedback to tell users when they interact with it
- The color of the LED make not have enough contrast for different kind of people(color blind)
Thanks to the feedback we got from the user test, we understand the needs of our stakeholders. Based on this information, we decided to add more functions to our prototype.

So in our next couple prototype, we added a keypad which allow users to select different modes while practice music.

We made and printed the stickers on each key to ensure our users could understand it easily.
- We considered the users’ learning experiences and separated the music into different hand modes. Users can practice the music either left or right hand based on their habits(key#1,2).
- After they feel confident, they can press both hands mode to practice the original music(key#3).
- To help users practice music easily, we added functions that allow them to change the piece's speed (key#4,5), so they can follow the light at a slow speed at the beginning and then speed it up later.
- Because our stakeholders mentioned that the brightness of the LED is too high, we allow users to adjust the brightness of the LED by pressing keys #6 and #7. We hope it can help users have a better practice experience in different environments.
- The keys with two different colors are different color modes for the LED. We chose colors with high contrast to ensure most users can use our product easily.
The interface part:

We also made an interface to show the music sheet while practicing the music and provide audio feedback based on users' actions.

To make the user experience better and more inclusive for both VI and Not VI, we establish a UI workflow to try to add different functions to achieve. And we decided to focus on the home page, music selection page, practicing page, and setting page.

Based on the user flow, we built the information architecture, which gave us a clearer understanding of the design architecture of the UI interface.

With architecture information and workflow, we created the UI prototype using Axure. We simplified the content in the interface and made the buttons and loop marker large enough to ensure that our users could click on the information accurately. In addition, to assist people with different visual impairments in distinguishing the content, we have set up functions to adjust the light, contrast, and background color shift.

We invited our colleagues and visually impaired users to test it. After testing, most users found that our second prototype version satisfied their...
Read more »