Why has there been so much recent attention of electromyography (EMG) within myoelectric control? EMG as a control input for working in myoelectric applications has been around since the 1960s. Myoelectric control can be described as the control of a device using the electrical signal generated during muscle contractions. These changes in muscle excitation properties can be measured via EMG wearable sensors.
However, with emerging curiosity within extended reality and metaverse type applications, big tech and researchers have begun to explore less conventional uses of EMG. When Mark Zuckerburg starts to offer his views on motoneurons, you can tell there is some serious interest.
EMG presents a promising modality for human-machine interfaces due to its non-invasive, intuitive, and constantly available nature. Whereas previous efforts have been made before to establish a toolbox for working with the myoelectric signal, such as EMGBody and BioPatRec, none have been widely adopted by the community. With the resource of open-source data sets, such as the NinaPro which includes 180 data acquisitions to promote machine learning research on human, robotic, and prosthetic hands, a toolbox that can be widely used and developed across the field would be hugely beneficial to the advancement of myoelectric control.
This is where LibEMG has now come in. LibEMG is an open-source Python library that aims to facilitate the development of myoelectric control systems, particularly for non-domain experts. To alleviate the barriers hindering the exploration of this technology, LibEMG provides detailed instructions to the many challenges associated with developing myoelectric control systems, including data processing, hardware interfacing, feature extraction/selection, classification, post-processing, and evaluation.
The Delsys Trigno system is one of the supported hardware options within LibEMG.
“This project looks to provide resources to make myoelectric control an accessible tool for researchers and interested parties outside of your classic prosthetic hand community.” – LibEMG Research Team
This blog looks to understand some of the further uses and aims of LibEMG. We discuss future goals and areas that hold promise within myoelectric control with insights coming from part of the LibEMG research team of Dr. Erik Scheme, Dr. Scott Bateman, Dr. Angkoon Phinyomark and PhD students Ethan Eddy and Evan Campbell from the University of New Brunswick. Expand the toggles (+) to see what the team had to say on each topic!
Ethan stated as part of his ongoing studies that “This is a project that has been of personal interest as we look to evolve ways to use EMG for prosthetic control and mixed reality research. It then took off from there to a toolbox that can provide an open-source, centralised, repository for standardising methodologies to be used in offline and online classification and data processing.”
space
Diagram of LibEMG’s Core Modules. Dashed lines represent modules that are optional to implement.
space
The aims go further than creating a tool box though as Erik built upon “This project looks to provide resources to make myoelectric control an accessible tool for researchers and interested parties outside of your classic prosthetic hand community.”
“The other main aim is to allow for replication of studies across groups that aids collaboration and sharing of knowledge. The project provides multiple pathways for groups to choose how to process their data, but is centred around a framework in the repository that allows for comparisons between hardware and to corroborate results and findings between groups.”
As we know with EMG, there are certain barriers to being able to use and implement the hardware, but Erik spoke as to what steps LibEMG have taken to make this more accessible. “Whilst working with EMG, there is so much domain knowledge that is required especially when working within prosthetic control when every patient’s muscle architecture is different, and the socket is customised meaning that every user is unique. However, we can’t expect every user to be skilled in this domain especially if looking to use EMG as a modality for extended reality, for example.”
space
The combined Raw EMG Data from the OnlineDataHandler
space
Space
Evan added that “There has also been the notion from human-computer interaction groups that EMG is a noisy modality due to the stochastic nature of the EMG signal. And this is where LibEMG can play a big role. We have been able to reduce the barrier to entry by putting best practices in place to help remove the domain expertise by allowing users to simply call the API from the library. We feel LibEMG can act as an accelerant to the field by showcasing the precise control you can obtain by using an embodied system, such as EMG, that is directly couped to user’s intent.”
space
Multichannel view from the Raw EMG Data of the OnlineDataHandler
space
Speaking more towards how this platform can work for multidisciplinary collaboration, Ethan noted that “The real end goal of the project is to bring myoelectric control prosthetic researchers and merge their knowledge with the resources and expertise of human-computer interaction scientists who understand the way in which software can be leveraged to form an intuitive and fun neural interface.
This then allows for real collaboration and everyone contributing towards a shared resource that can make EMG a much more readily accessible tool within this space”.
EMG really is just at the beginning of it’s journey within the human-computer interaction (HCI) space as Erik was keen to emphasize “There is a burning need for new tools within extended reality and human computer interactions. In prosthetic control we have seen work with force myography, with ultrasound, but EMG has been the main tool over the last 60 years and the main question that remains is “how good can we make it?”
However, now we are seeing this demand within HCIs because there hasn’t been anything new as a controller since the inception of computing. As humans, we have used mice and keyboards but now there is a real push for hands free ubiquitous control as the demand for something new grows.
space
Continuous myoelectric control of the popular game “Snake”
space
“For this to occur though, there must be a merging and sharing of knowledge across the two disciplines. Many times, across the wider departments, I will have students who are skilled in machine learning and EMG analysis but have never considered extended reality applications. Whereas conversely, I have the human-computer interaction researchers who are skilled in building an interface in Unity to implement models but have little expertise in deep learning and algorithmic work”
space
Cross platform control in Unity for mixed reality applications with the Hololens 2
space
This is then when Ethan’s cross department education was relied on for helping to bridge the gap between the two disciplines “This is where my passion came in as LibEMG allows me to work on my experience with VR and AR applications and couple this with the knowledge that Evan has within signal processing and machine learning. We developed LibEMG so that it can allow for inter-disciplinary research to happen quicker. LibEMG enables us to fail faster, see what works and what doesn’t, but also to learn at a speed that wasn’t accessible before. We believe this is where the real advancements for myoelectric control can then happen.”
There’s so much work left to do and these are just some of the aspects that Ethan found most exciting for the future of myoelectric control “There really is a lot of excitement around this space at the minute when you look at some of the big tech companies who are looking to leverage the EMG signals to control a heads-up display or to help interact with something in a virtual reality. This has to be some of the ultimate goals for this area, in my opinion. But it’s also important to consider the form factor of something that’s convenient to wear and intuitive to use so that it doesn’t become a gadget that’s only fun rather than practical.”
space
Mixed reality application using Hololens 2 to navigate a menu with EMG
space
Whilst Ethan spoke to the wider implications for XR and HCI applicaitons, Evan was very much focused on the algorithmic side “I’m still relatively firmly rooted within the algorithm side and what excites me is to continue active development. What gets a lot of attention at the minute, and is quite a hot topic within the literature, are the models that don’t require extensive, or any, training period. When we consider transfer learning approaches, user in the loop learning, and some other approaches around adaption and active learning, we believe these methods hold a lot of promise and excitement for pushing the field forward”
space
Proportional Control example of using the mouse interface (via myoelectric control) to play an online game
space
The library and toolkit opens up opportunity for so much and Erik spoke about the exciting future potential of the tool “What this current work has enabled as well is that we can now very quickly spin-out new research ideas and test these within the lab without having to create a whole new interface. The current library, in the way that it has been designed using Python, is built for expansion to grow and remain up to date with state-of-the-art research. A lot of this will of course depend on how widely adopted it may become.
We also have a lot of groups around the world who don’t have access to EMG equipment. If they gain access to an EMG dataset, such as the NinaPro project, and create a pipeline without knowing the inner workings then this can be harder to work with and understand. However, with LibEMG, we provide detailed code and information with regards to working with these data sets. If this could reduce the barrier to entry and inspire a few more early career researchers to go further with myoelectric control, then this would be amazing!
The future certainly holds some incredibly exciting opportunities for EMG to be used for myoelectric control with prosthetics, human-computer interactions, and XR applications. To find out more about how Delsys, and the EMG technology that we provide, can help you with your myoelectric control research, please get in touch with us at contact@delsyseurope.com.