Wearable sticker turns hand movements into communication

WASHINGTON — Imagine wearing a thin flexible sticker that can turn your hand or finger movement into communication without you having to say a word or tap a touch screen. Researchers have developed a new type of wearable sensor that can accomplish this futuristic feat and could open new possibilities for rehabilitation applications and helping those with disabilities to communicate more easily.

The new sensor combines a soft and flexible material called polydimethylsiloxane, or PDMS, with an optical component known as a fiber Bragg grating (FBG). The researchers designed it to be comfortable for long-term wear while also having the ability to detect movements with high accuracy.

“For someone recovering from a stroke, these sensors could monitor wrist, finger or even facial movements to monitor their rehabilitation progression,” said Kun Xiao from Beijing Normal University in China. “For individuals with severe mobility or speech impairments, the sensors could translate gestures or facial expressions into words or commands, enabling them to communicate with others or interact with technology more easily.”

In the Optica Publishing Group journal Biomedical Optics Express, a cross-disciplinary team of researchers in optical, biomedical, software and electrical engineering from Beijing Normal University, Sun Yat-sen University and Guilin University of Electronic Technology, all in China, describe the new sensor. The sensors showed a high level of sensitivity and accuracy during tests involving gesture recognition and communication assistance.

“Beyond detecting movement, these adaptable sensors could be tailored for applications such as monitoring other health indicators like respiratory or heart rate by detecting subtle body movements,” said Rui Min from Beijing Normal University. “They could also be useful for athletes or fitness enthusiasts to monitor and improve their form or technique in real time or be integrated into gaming systems for more immersive and interactive experiences.”

A personalized approach

This new work is part of a larger project aimed at developing innovative assistive technologies and was inspired by the challenges faced by people with disabilities and those recovering from conditions like strokes, who often struggle with basic movements and communication. “Traditional methods are either too cumbersome, lacked accuracy or weren’t versatile enough to cater to individual needs,” said Zhuo Wang from Beijing Normal University. “Our goal was to develop a wearable solution that was both precise in detecting gestures and comfortable for everyday use, offering a more personalized and adaptive approach to rehabilitation and assistance.”

To do this, the researchers created patches made from PDMS, a type of silicone elastomer that is very flexible and skin friendly. This allows people to wear them for long periods without irritation or discomfort. To give the patch its movement-sensing capability, the researchers embedded the PDMS with FBGs, a type of reflector that is etched into a short segment of optical fiber to reflect specific wavelengths while transmitting all the others. The sensor makes it possible to detect slight changes in the way light propagates through the fiber optic during movement, allowing the system to detect specific movements by analyzing the alterations in light behavior.

“The real magic comes from the combination of the PDMS with FBGs,” said Chuanxin Teng from Guilin University of Electronic Technology. “We found that using a thicker PDMS patch caused a more pronounced wavelength shift. Leveraging this sensitivity-enhancing effect of PDMS allows these optical sensors to detect even the slightest bend of a finger or twist of a wrist.”

The sensors can be applied to various parts of the body for a wide range of applications. The researchers also developed a precise calibration method that allows the sensors to be tailored to each user, making them adaptable to various applications.

Transforming movement into communication

To demonstrate the capabilities of their PDMS-embedded wearable FBG sensors, the researchers conducted a series of tests focusing on gesture recognition and communication assistance. After calibrating the sensors for individual participants, they attached the sensors to different parts of the body, such as the wrist and fingers, to detect various movements. They also developed a system that enabled the sensors to translate simple gestures into commands or messages. For example, they used finger movements to spell out words based on Morse code.

For both tests, the sensors demonstrated a high level of sensitivity and accuracy in recognizing a wide range of gestures. They could detect subtle movements that might be difficult for some traditional sensors to pick up. In the communication assistance experiment, the sensors successfully translated gestures into words, showing their potential as an assistive technology for individuals with speech or mobility impairments.

The researchers are now working to further improve and refine the technology to make it ready for practical use and for further studies through clinical trials. This includes making the sensor system even smaller and more integrated so that it can be easily worn, as well as enhancing the sensors’ ability to communicate wirelessly with smartphones, computers or medical devices. This will allow users to interact with the technology and help caregivers or medical professionals to monitor progress or data in real-time. They are also making improvements to the robustness and durability of the sensors to make sure they can withstand daily wear and tear, including exposure to moisture, heat and stretching.

Paper: K. Xiao, Z. Wang, Y. Ye, C. Teng, R. Min, “PDMS-embedded wearable FBG sensors for gesture recognition and communication assistance,”

Biomed. Opt. Express, Vol. 15, Issue 3, pp. 1892-1909 (2024).

https://doi.org/10.1364/BOE.517104

About Biomedical Optics Express

Biomedical Optics Express serves the biomedical optics community with rapid, open-access, peer-reviewed papers related to optics, photonics and imaging in biomedicine. The journal scope encompasses fundamental research, technology development, biomedical studies and clinical applications. It is published monthly by Optica Publishing Group and edited by Ruikang (Ricky) Wang, University of Washington, USA. For more information, visit Biomedical Optics Express.

About Optica Publishing Group

Optica Publishing Group is a division of Optica, the society progressing light science and technology. It publishes the largest collection of peer-reviewed content in optics and photonics, including 18 prestigious journals, the society’s flagship member magazine, and papers from more than 835 conferences, including 6,500+ associated videos. With over 400,000 journal articles, conference papers and videos to search, discover and access, Optica Publishing Group represents the full range of research in the field from around the globe.