Open assistive controller

Details

Playing video games is a very common activity practiced by many people around the world. If you are disabled, this activity can be difficult to perform or even impossible.

If assistive gaming solutions exist and whether new solutions arrived on the market in recent years (see dedicated log), they remain incomplete, aren't compatible between different gaming platforms, aren’t fully configurable and don't support many specifics inputs.

We want to rely on the strength of communities (makers and users) and open source to design a solution to these lacks and problems and share the plans so that as many users as possible can benefit from it.Our project will build on the great work undertaken by the Open stick community and its GP 2040-CE project's which is a multiplatform gamepad firmware for the Raspberry Pi Pico and others RP2040 based microcontrollers.

The goal of our approach is to use the properties of this superb work to adapt it to the needs of a maximum of users, design and create an amazing universal controller.

FIRST TEST :

To check the feasibility of the project, I quickly made a prototype to explore the possibilities offered by GP2040-CE.

To start, I followed the very detailed documentation of the GP project to install the firmware on a pi pico and carry out the first tests.

A breadboard, some wires, switches and an analog joystick allowed me to add inputs very easily to a Raspberry Pi Pico.

I was also able to test a power wheelchair digital special command unit as a D-PAD via the GPIO and everything works well, on Nintendo switch and PC. I’ll test it on the other platforms later.

With these first observations, and the proof that it can work, it was time to sit down and think about the development, list user needs and determine specifications to create an assistive device adapted so that the greatest number of users can play.

IDEAL SPECIFICATIONS :

The device will act as an interface between user and the gaming device, It will allow players connecting specifics inputs corresponding to their accessibility needs : Switches, joysticks and buttons

Our goal is to design a dedicated hardware, will wich embedded the GP2040-CE and allow the use of specific inputs with ergonomics that adapt to different types of disabilities.This will go through the design of a box housing the various connectors, a dedicated electronic PCB and a battery to power everything.

We will not neglect different methods of accessibility and beyond the motor aspects, we will work on visual and sound accessibility (sensorial feedbacks implementations) and we will make the device easy to use.

Ideally, we should be able to connect the following elements to the device :

- 16 external switches (TRS jack cable 3.5mm) for game buttons

- 2 gaming joysticks ( USB) for in game sticks left and right

- 2 analogs sticks (TRRS cable 3.5mm) for in game sticks left and right

- 1 power wheelchair special control unit (SUB D9) for in game sticks or D-PAD

- The device will be able to connect to the game console via USB cable and a bluetooth connection would be ideal.

- The device must have a front panel layout of big buttons fors bascics control (with a D-Pad, an 6 button at least)

- Compatibility : Unlike marketed solutions, our solution will offer cross-platform compatibility, which will allow to play on PC, PS3 and PS4, Nintendo Switch, Steam Deck, MiSTer and Android. This is enabled by the GP 2040-CE

The device might look something like this. Here is what to get an idea of the different connections, do not take into account the appearance, layout big bouton isn't present and a real design work is to be carried out !

FEASIBILITY

GP2040-CE offers a solid base for our work. Its customization possibilities via its built-in web-based configuration are quite crazy and the possibilities of creating hardware are just as crazy.

Let's take a look at the technical feasibility ...

GP 2040-CE
Julien OUDIN • 05/25/2023 at 09:59 • 0 comments
At first, this project was made by a small group of arcade video game enthusiasts. Now it's an awesome community project released under the MIT license and offers a bunch of cool features. It’s a gamepad firmware that allows you to create from a 2040 microcontroller a fully customizable gamepad that works on different gaming platform like PC, PS3 and PS4, Nintendo Switch, Steam Deck, MiSTer and Android.

From a firmware and hardware point of view, these customization possibilities seems impressive and GP2040 could be a solid base for our project. It may help us a lot to design and create the adaptive controller we want to make

Here are the features listed on the projects page that this firmware brings :
- Select from 5 input modes: XInput, Nintendo Switch, PS4, PS3 and Keyboard
- Overclocked polling rate for less than 1 ms of input latency in all modes.
- Multiple SOCD cleaning modes - Up Priority (a.k.a. Stickless), Neutral, and Second Input Priority.
- Left and Right stick emulation via D-pad inputs as well as dedicated toggle switches.
- Dual direction via D-pad + LS/RS.
- Reversed input via a button.
- Turbo and Turbo LED with selectable speed
- Per-button RGB LED support.
- PWM Player indicator LED support (XInput only).
- Multiple LED profiles support.
- Support for 128x64 monochrome I2C displays - SSD1306, SH1106, and SH1107 compatible.
- Custom startup splash screen and easy image upload via web configuration.
- Support for passive buzzer speaker (3v or 5v).
- Built-in, embedded web configuration - No download required!
This project is constantly evolving with regular updates, and the community I have come into contact with seems willing to help implement accessibility features.

Have to test it !
Existing assistive gaming solutions
Julien OUDIN • 05/23/2023 at 13:24 • 0 comments
Users with special needs use a wide variety of inputs to act with their devices and environment. The control interface and its suitability to the users possibilities is essential when you have a disability. It conditions the effective use of the assistive device whatever its nature (wheelchair, computers, call bell...) For gaming purposes most commons and usefuls interfaces are switches and joysticks.
- Switches
The switch is a momentary push button which closes an electrical circuit which will be detected by the device to be used. In game, it can replace the direct press on a button on the controller. It has a 3.5 mono male jack connector and the device must be equipped with a 3.5 mono female jack

There are different types of switch which have different sizes and sensitivity and adapt to many motor possibilities. They are placed opposite in a zone or a reliable, repeatable and non-fooling movement. Typically, they can be installed near the hand, a finger, but it can also be at the head or anywhere else where the user can produce an exploitable movement.
- Joysticks
Joysticks are used as interface to produce movement in game, there are numerous in type, shape, sensitivity, connectivity and activation force. Typically additional joysticks are set up on a table or a specific mounting system to fit users needs and possibilities.

It is also possible to use the power wheelchair's joystick as a stick in game. With bluetooth connectivity if the wheelchair is equiped and a specific device the TITAN or TITAN 2. This device allow you to recover the mouse signal from a wheelchair joystick equipped with bluetooth but it requires a computer and a dedicated software to be programmed.

Or with an extra hardware which detect the movement made on the joystick and translate it in stick signal : JoytoJoy

For more info on adaptive gaming joysticks, read the nice MakerMakingChange joysticks documentation
- Hubs
To play video game with these types of specific inputs, you have to use a dedicated interface which act as a gateway between the interface devices and the videogame console.

The arrival on the market of Microsoft's Xbox Adpative Controller (XAC) was a big revolution and made life easier for many gamers by offering many possibilities to adapt external switches and joysticks to a console for a price comparable to a standard game controller ($99).

This aid works perfectly on PC and Xbox, and could also adapts to other consoles (like Nintendo Switch and Playstation) buy buying a specific conversion adapters.

The Hori Flex controller that recently arrived on the market is a similar aid that allow to play with assistive external devices on a Nintendo switch, for a $249 cost.

The disadvantage of this solution is the compatibility with different joystick, only a few available on the assistive device market are compatible and work properly with the hori flex.

To play on Playstation, there isn't dedicated solution yet, and Sony has recently announced working on a adaptive gaming hub call project Leonardo but its still in developement.

Another problem, common to Xbox adptive and hori flex is that they don't allow the connection with specific control interfaces for power wheelchairs inputs such as joysticks with serial connection (sub D9) . It's however a standard in assistive devices field.

Considering these issues, a high price for a game accessory, limited cross-platform compatibility and limits in connection possibilities, we thought it might be interesting to solve these problems with a little bit of open source, co-design and digital fabrication.