For disabled people to live independently, they must be able to independently control their home environment. Through the introduction of tools like Alexa and remote app control, modern smart home technology has already taken disabled people into account. But after talking to actual users, we discovered that there is still a certain demographic from which smart home technology excludes them. For instance, we met a woman who had a stroke and is unable to speak clearly and fully extend her arms and fingers. Her interactions with household items are influenced by her physical health. She nevertheless always maintains a positive attitude and tries her best to live independently. This inspired us to create more accessible smart home technology so that people like the lady can better control their household appliances and empower their independence and confidence.

How to achieve it?

1. On the whole, the underlying logic of this project is to identify human body expression signals, convert them into signals that can be recognized by machines, and transmit them to corresponding devices.

So in this project, the body posture recognized by the six-axis sensor and the muscle movement recognized by the EMG signal is used as input; the original infrared level signal and BLE mouse are used as the output. Finally, an infrared and Bluetooth communication device that realizes cross-protocol communication through body movements is formed.

This is based on the research and design results in the context of the Internet of Things.

The entire project uses the Arduino programming environment

Language: C++

The hardware motherboard has undergone many iterations and replacements, from the simplest Arduino uno basic version at the beginning to Leonardo and then to Nano and Pro mini (the whole device is expected to be compact), and finally, considering factors such as cost, function, and volume, I chose a more ESP32 Wroom board with wifi and BLE functions suitable for IoT development.

2. At the beginning of the project, we used the most basic 38khz ir receiver and transmitter to complete the receiving and sending functions of infrared signals.

We tried two kinds of TV set-top box remote controllers and various simple infrared communication content and found that the existing library or remote under Arduino does not completely cover all protocols, and because the TV set-top box consists of multiple protocols (such as RC5, RC6, NEC) plus a custom protocol, so we finally decided to use the transmission of original level signals to achieve cross-protocol collection and transmission. This allows us to ignore the decoding of the protocol and control more appliances using infrared communication.

3. Solved the problem of the output terminal. We started working on sensor capture. We first used Walfont's Muscle Sensor, which is a sensor that requires two external 9v batteries to provide power and uses 50mm electrode patches. We use it to read the muscle contraction of the arm and complete the stable reading and judgment of the EMG signal. When people bend their arms, there will be a peak in the EMG signal, and we can use this peak to determine the condition to determine the output. Later, we updated the myoelectric sensor, using myoware2.0, which is smaller and has an anti-shielding shell, which can effectively avoid noise interference when it is connected with other sensors at the same time.

4. We also expect the user's physical posture to affect the output, which is also in line with their expectations: they hope to continue to use their damaged limbs to interact, although in the traditional product interaction mode (click, slide, press) Under these conditions, these limbs can no longer bear the interaction. So we chose the MPU6050 - a basic six-axis sensor. What we are currently implementing is the detection of the attitude angle, and the roll angle and a pitch angle of the...