Close

Rough outline of modules for software control and software->actuator control

A project log for BCI-based assisted living and robot avatar system

This is a brain-computer interface system to allow people suffering from locked-in syndrome to experience a higher quality of life

neil-k-sheridanNeil K. Sheridan 04/06/2016 at 20:170 Comments

* Communication Module

p300 or SSVEP speller

* Wheelchair Module

Can be a flickering light box for direction (SSVEP) and stop; p300 menu for fave locations (photo of kitchen, front door, lounge, find friend* etc]

*friend can have RFID tag so we can find them in the house easily

* Assistive Living Environment Module

Rooms broadcast to BCIALERAS (brain computer interface assisted living entertainment and robotic avatar system) so we know where user is

-> Adjust lighting / turn on off including lamps (photo menu of lights)

-> Text-to-voice using SSVEP or p300 speller to answer door

-> Adjust temperature either home-wide or local room (e.g. AC / fans)

-> Fave food p300 menus to communicate with friends which food/drink would like easily! These could be sent to smart phone of friend so they can be purchased or brought as required.

* Robotic Avatar Module

When confined to one room and only able to move slowly in wheelchair (if at all when in bed) it could be frustrating to hear noises etc. and wonder what they are. Or perhaps may wish to explore the garden and look at plants, ponds etc. Or perhaps watch TV with friend whilst they are in sitting room and user is in bed. This is accomplished using a robot avatar. Simply a tracked robot with a 1m vertical extension featuring video cameras (including zoomable) and voice synth (for SSVEP/P300 speller to speech) and microphone. It could additionally feature a screen broadcasting video of the user.

- Robot can be controlled directly, or move semi-autonomously to fave locations.

- An automated rescue feature for robot to reach safety in case of getting stuck in corners and frustrating the user.

The robot could feasibly leave the house and be accompanied by friend on visits to shops, parks, galleries, museums car rides. Communicating via 4G to user stuck at home.

* Entertainment Module

Top Menu with links [Reading / Video / Audio / Internet ]

-> Reading -> Control of Kindle App using KDK [turn page/close/select book]

Net-> Web Browsing -> Cursor for Hyperlinks + pop up keyboard for URLs/fields

Net-> Twitter -> + pop up keyboard

Net -> Video -> Control using Amazon Video API and YouTube

Net -> Audio -> local library, iTunes?, streaming services

? keyboard option of p300 speller?

? p300 speller to choose books

Net -> Email control

Discussions