Electromyographic interfaces with proportional control for bionic prostheses with a large number of degrees of freedom

Nowadays, the problem of hand movement decoding based on myographic (muscle) activity data is relevant for the creation of bioelectric (bionic) prostheses of the hand. Even the most advanced prostheses are controlled using the simplest classification of tension and relaxation of two muscles of the shoulder or forearm, and an increase in the number of grips is achieved by sequential patterns of contraction of these muscles or by switching grips on a smartphone. Studying the possibility of restoring precise movements by muscle activity in people with amputation and congenital underdevelopment of the upper limb is a necessary part for creating a new generation prosthesis control system. The creation of such devices and control systems requires the collection, processing, and careful analysis of myographic data in both healthy individuals and those with congenital absence or amputation of the hand.

Existing solutions for classifying and decoding finger movements in people with disabilities focus primarily on gesture classification, that is, the joint movement of multiple fingers. However, modern machine learning methods can not only improve gesture classification, but also recover free, natural finger movements, handwriting (Okorokova et al., 2015) or movement intentions by electromyography (EMG). The goal of this project is to study the possibility of such decoding in people with congenital absence or amputation of the hand in comparison with healthy people. This project implements different approaches to the analysis and collection of myographic data and its subsequent decoding using neural networks.

Data collection methods include high-density EMG in the form of grids (8x8 cm and 4x4 cm) shown in the figure below, as well as bracelets allowing to collect information from the whole circumference of the forearm.


Figure 1. Electromyographic grids with high electrode density.

The project uses motion-capture technologies to detect finger positions: the MediaPipe framework for video recordings and algorithms inside virtual reality helmets such as Oculus Quest 2. The output contains 21 points of finger position at each moment of time in 3 spatial coordinates (x, y, z).

Figure 2. Output data of hand tracking and Oculus Quest 2 virtual reality helmet.

Preliminary results of the experiments and decoding show high accuracy in restoring finger flexion angles on healthy participants (prediction error: 15-20 degrees in angles, correlation of real finger kinematics and predicted values: 0.7-0.9). At the moment the data collection and analysis on people with a missing limb is underway, hypotheses involving a person in the training of the neural network (decoder) in real time are being tested. The project was supported by the Foundation for Assistance to Innovations under the Student Startup Program, which is used to develop software (virtual environment and algorithms) for bionic prostheses, as well as proprietary EMG bracelets and electronic circuits for data collection.

Figure 3. Virtual reality environment for EMG data collection and training of the study participant.

The images above represent a virtual reality environment for training and data recording. Instructions and gestures to be performed by the person are displayed on the screen. The cameras on the helmet follow only one hand, while the visualization of the second hand is a reflection of the movements of the first. This is necessary for training amputees - both hands and their movements are represented for them. After training the model, the coordinates of the fingers of the second (reflected) hand begin to depend on the signal from the muscles - so the person understands how the model predicts his intentions, and learns to control the patterns of muscle contraction.

Published article on the project: Okorokova, E., Lebedev, M., Linderman, M., & Ossadtchi, A. (2015). A dynamical model improves reconstruction of handwriting from multichannel electromyographic recordings. Frontiers in neuroscience, 9, 389. https://doi.org/10.3389/fnins.2015.00389