HBChen-094
For disabled people to live independently, they must be able to independently control their home environment. Through the introduction of tools like Alexa and remote app control, modern smart home technology has already taken disabled people into account. But after talking to actual users, we discovered that there is still a certain demographic from which smart home technology excludes them. For instance, we met a woman who had a stroke and is unable to speak clearly and fully extend her arms and fingers. Her interactions with household items are influenced by her physical health. She nevertheless always maintains a positive attitude and tries her best to live independently. This inspired us to create more accessible smart home technology so that people like the lady can better control their household appliances and empower their independence and confidence.
How to achieve it?
1. On the whole, the underlying logic of this project is to identify human body expression signals, convert them into signals that can be recognized by machines, and transmit them to corresponding devices.
So in this project, the body posture recognized by the six-axis sensor and the muscle movement recognized by the EMG signal is used as input; the original infrared level signal and BLE mouse are used as the output. Finally, an infrared and Bluetooth communication device that realizes cross-protocol communication through body movements is formed.
This is based on the research and design results in the context of the Internet of Things.
The entire project uses the Arduino programming environment
Language: C++
The hardware motherboard has undergone many iterations and replacements, from the simplest Arduino uno basic version at the beginning to Leonardo and then to Nano and Pro mini (the whole device is expected to be compact), and finally, considering factors such as cost, function, and volume, I chose a more ESP32 Wroom board with wifi and BLE functions suitable for IoT development.
2. At the beginning of the project, we used the most basic 38khz ir receiver and transmitter to complete the receiving and sending functions of infrared signals.
We tried two kinds of TV set-top box remote controllers and various simple infrared communication content and found that the existing library or remote under Arduino does not completely cover all protocols, and because the TV set-top box consists of multiple protocols (such as RC5, RC6, NEC) plus a custom protocol, so we finally decided to use the transmission of original level signals to achieve cross-protocol collection and transmission. This allows us to ignore the decoding of the protocol and control more appliances using infrared communication.
3. Solved the problem of the output terminal. We started working on sensor capture. We first used Walfont's Muscle Sensor, which is a sensor that requires two external 9v batteries to provide power and uses 50mm electrode patches. We use it to read the muscle contraction of the arm and complete the stable reading and judgment of the EMG signal. When people bend their arms, there will be a peak in the EMG signal, and we can use this peak to determine the condition to determine the output. Later, we updated the myoelectric sensor, using myoware2.0, which is smaller and has an anti-shielding shell, which can effectively avoid noise interference when it is connected with other sensors at the same time.
4. We also expect the user's physical posture to affect the output, which is also in line with their expectations: they hope to continue to use their damaged limbs to interact, although in the traditional product interaction mode (click, slide, press) Under these conditions, these limbs can no longer bear the interaction. So we chose the MPU6050 - a basic six-axis sensor. What we are currently implementing is the detection of the attitude angle, and the roll angle and a pitch angle of the sensor are obtained for judgment. We tried the quaternion and Kalman algorithms and finally chose the DMP algorithm that comes with the MPU because we don't pursue high precision, it can already read the attitude angle well and effectively save the computing space of the chip.
5. After completing the development of the basic functions, we combined the three, conducted multiple stability experiments, and modified the code logic. Eventually, we modified and iterated on the interactive form. And at the same time realize the Bluetooth mouse function and the electric appliance control function based on the infrared signal.
Product appearance and material
We have identified our current basic components based on our previous technical implementation process. They are a six-axis sensor, infrared sensor, and EMG sensor. How to make our currently implemented technology more comfortable for the disabled population to wear is the core of what we need to consider in our current product design. Our product is divided into three parts, each module can be connected and can also be directly disassembled. In order to facilitate them to better wear according to their own situation, but also can be applied to different physical impairments of the disabled groups. In order to wear it more firmly and comfortably, we use a softer material that fits on the skin in the form of a strap for control.
Further research directions and vision
GMC is the beginning of our research into the use of an infrared sensor, a six-axis sensor, and an EMG sensor for people with disabilities, with the aim of making products and devices that serve people with disabilities with intelligent technology more common and simpler. We hope to further investigate technologies such as speech and image recognition to develop the re-use of different senses to cope with different levels or areas of functional deficit.