Close
0%
0%

SERPENTINE

Hand Gesture Recognition
using Self-Powered Stretchable Multi-Purpose Vibration Sensor
based on Triboelectric Nanogenerators Principle

Public Chat
Similar projects worth following
Rethinking about the materiality of tangible input interfaces and reinforcing them with sensing capabilities is the beauty and intent of our innovative approach in this research. We have invented Serpentine, a highly stretchable self-powered sensing interface empowered with signal processing for gesture recognition.


Team Members:
Fereshteh Shahmiri, Shivan Mittal, Chaoyu Chen, Yuhui Zhao

Collaborators:
Dingtian Zhang, Yi-cheng Wang, Steven Zhang

Advised By:
Dr. Gregory Abowd, Dr. Thad Starner, Dr. Z.L. Wang, Dr. Thomas Ploetz

CHALLENGE & OUR PROPOSE SOLUTION

We propose Serpentine, a highly stretchable self-powered sensing interface empowered with signal processing to recognize the deformations in its shape. This interface is fabricated in a coil-like structure with multiple coaxial layers of silicone, copper and conductive nylon thread. Proposing its structural design and specific choice of materials allow mechanical deformations of the interface create time-varying charge distributions between the conductive nylon and copper thread that generate electric signals. Such signal generation happens based on triboelectric nanogenerator (TENG) phenomenon that works on the conjunction of electrostatic induction and triboelectrification [37]. Since, the deformations and consequently, charge distributions lead to power generation, the working principle itself eliminates the need of external power to sense deformations caused by gestural interactions. Self-generated signals are then processed through an implemented system including signal processing pipeline that uniquely maps them to actions that created the deformations.Our proposing system enables us fulfill our vision to create an interface that is not only self powered but also sensitive to a wide range of expressive input modalities.


The cylindrical, cord-shape and highly stretchable form factor of Serpentine’s interface enables three way force application: towards the axis (radial), along the axis (longitudinal), and tangential to the cross section (tangential). These elementary modes of force application build interactions of touch, twist, tap, press, stretch, slide, and many more complex combinations. High sensitivity to low-frequency signals (e.g. less than 100 Hz) appropriates Serpentine as input technology for various computing devices including gestural sensing and recognition. The simplicity of the physical structure of the sensor, universal availability of tribo-materials (all materials available in our daily life like paper, fabric, PDFE, PDMS and many more) and DIY approach for its constructions allow such a sensing interface to be scaled for variety of applications. Serpentine eliminates the need of bulky or rigid sensing instruments worn on different parts of the body. It’s specific material properties, form-factor and physical structure allows gesture interaction naturally and unobtrusively. Our evaluations show that our sensing platform perform with 85 % accuracy in gesture classification and is consistent for different users.

HOW SYSTEM WORKS

Link to any repositories (e.g., Github)

https://github.gatech.edu/sshahmiri3/cordUI

Final Appearance of the Sensor

Self-powered Stretchable Coil-Shape Vibration Sensing Interface - Single Electrode  ( Copper Wire encapsulated with silicone as dialectic) Mechanism based on TENG Principles
Self-powered Stretchable Coil-Shape Vibration Sensing Interface - Double Electrode ( Copper Wire & Nylon thread encapsulated with silicone as dialectic) Mechanism based on TENG Principles


  • 1 × Hardware - Sensing User Interface - Vibration Sensor Check material for sensor fabrication in instruction part
  • 1 × Adafruit Feather M0 WiFi with ATWINC1500 Hardware - Circuit Design and Micro Controller
  • 1 × INA118 Precision, Low Power Instrumentation Amplifier Hardware - Circuit Design and Micro Controller
  • 1 × Wires and Resistors, Proto Board Hardware - Circuit Design and Micro Controller

  • APPLICATION

    fereshteh08/27/2018 at 06:31 0 comments

    APPLICATION: 

    Necklace for unobtrusive eyes-free interaction

  • Self-Power Sensing Mechanism

    fereshteh08/26/2018 at 03:52 0 comments

    SELF_POWERED SENSING MECHANISM 

    The following diagrams depict the TENG working principles of triboelectrification and electrostatic induction when sensor is under compression in tap and under tension in stretch gesture.

View all 2 project logs

  • 1
    SENSOR FABRICATION PROCESS

    MATERIAL: 

    The material we need to fabricate the self-powered stretchable force sensor are:

    • Conductive Copper Wire
    • Silver-coated Nylon Yarn
    • Silicone Rubber (Commercial Ecoflex 0050, Smooth-on, Inc)
    • PDMS (Commercial Sylgard® 184 silicone elastomer, Dow Corning Corporation)
    • Plastic Tube with any thickness you desire for your sensor

    DIY FABRICATION PROCESS: 

    Our novel self-powered stretchable cord-shape sensor has low-cost, cheap, easy and safe fabrication process. To fabricate the sensor:

    • Conductive copper wire and commercial silver-coated nylon yarn were chosen as inner and outer electrodes. You can choose other types of conductive threads by your choice. It is important to choose such threads with maximum negative and positive polarities in compare with each other.
    • For the proper stiffness, silicone rubber (Ecoflex 0050, Smooth-on, Inc) and PDMS (Sylgard® 184 silicone elastomer, Dow Corning Corporation) were mixed as the dielectric, supporting and encapsulating materials.
    • The frication of the core fiber:
      • Silicone rubber solution was prepared by mixing its two parts, in a 1:1 weight ratio and then blended.
      • PDMS solution was prepared by mixing its two parts, in a 10:1 weight ratio and then blended.
      • Silicone rubber solution and PDMS solution were mixed in a 4:1 weight ratio and then blended.
    • Pour the solution into a plastic tube and solidified at 60℃ for 5 minutes. You can use your oven at home or you can leave it in room temperate to be solidified. In the latter it needs more time for solidification.
    • Peel off the tube and the core tube-shape silicone is ready.
    • Roll the copper wire at a density of your design (The denser, the better since it creates more charge distribution) around the core silicone.
    • Use the same solution in step 3 to cover the copper wire. put it in oven for 5 minutes. We repeat this step three times to make sure the conductive thread is covered appropriately with silicone.
    • The sensor is ready as single-electrode one in this step. In case if you want to have more stable and more amplitude in your electrical output signals, we suggest fabricating the sensor in double electrode mode. For doing such, continue with next steps.
    • Roll nylon yarn around the three fabricated layers of silicone - copper wire – silicone. We suggest keeping the density of Nylon rolling the same as copper layer.
    • Use the solution in step 3 and repeat step 7 one more time.
    • The sensor is ready in double electrodes mode. 
  • 2
    CIRCUIT

    The sensor signal, because of the mechanism of its generation is bidirectional in nature (AC). In order to preserve the negative half of the signal, we offset the data by 1.6V using DC bias. The electrodes of the sensor are connected to “V In +” and “V In -” of the amplifier as shown in the circuit diagram below. The amplified signal is fed to the ADC of the MCU. The signal is converted with a 12 bit resolution, before being wirelessly transmitted to a computing device for real-time classification.

    Schematic diagram of connection sensor to microcontroller

    Collecting Analog data from differential potential between two electrodes in circuit
  • 3
    HAND GESTURES & SIGNAL PROCESSING

    GESTURES

    In a general categorization there are three types of interactions with sensor. 1. Towards the axis (Radial), 2. Along the axis (Longitudinal) and 3. Tangent to the cross-section (Tangential). These three ways of applying force to the sensor make up simple to complex gestural inputs. We have tested our implemented signal processing pipeline with 7 gestures; Tap, Press, Slide, Twist, Stretch, Bend, Rotate. 

    to see how we interact with such gestures check the following link:

    https://youtu.be/T_SP5k-Zab0

    SIGNALS

    to provide more insight on how signals collected from microcontroller look like, we added following images:

    Tap

    Tapping

    Pressing

    Rotating

    Twisting


    Sliding


    Stretching


    Bending

View all 4 instructions

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates