Engineers at the University of California San Diego have developed a smart glove that is able to wirelessly translate sign language into text, while also controlling a virtual hand to mimic sign language gestures.
The smart glove, dubbed “The Language of Glove”, currently works with the American Sign Language alphabet. The engineers built it for less than US$100 using stretchable and printable electronics that are cost-effective, commercially available and easy to assemble.
Recognising the potential of their work, the researchers are now further developing the glove for use in a variety of other applications, from virtual and augmented reality to remote surgery, technical training and defence.
According to Timothy O’Connor, a nanoengineering Ph.D. student at UC San Diego, gesture recognition is just the beginning.
“Our ultimate goal is to make this a smart glove that in the future will allow people to use their hands in virtual reality, which is much more intuitive than using a joystick and other existing controllers,” he said.
“This could be better for games and entertainment, but more importantly for virtual training procedures in medicine, for example, where it would be advantageous to actually simulate the use of one’s hands.”
The glove is unique because it has sensors made from stretchable materials, combined with being commercially viable for manufacturing. According to Darren Lipomi, a nanoengineering professor, this is due to the use of off-the-shelf components.
“Our work could enable other researchers to develop similar technologies without requiring costly materials or complex fabrication methods,” he said.
The glove itself is a leather athletic glove, to which the researchers adhere nine stretchable sensors at the knuckles. Two sensors are positioned on each finger, and one on the thumb. The sensors themselves are made of thin strips of a silicon-based polymer coated with a conductive carbon paint, and secured onto the glove with copper tape.
Using stainless steel thread, the researchers connected each of the sensors to a low power, custom-made printed circuit board that is attached to the back of the wrist part of the glove.
When stretched or bent, the sensors change their electrical resistance. The glove is a laid out to allow for detection of the positions of all nine knuckles, translating a straight or relaxed knuckle into a “0”, while a bent knuckle is read as a “1”. This results in a nine-digit binary key, which algorithms match to the letter.
For example, the code for the letter “A” (thumb straight, all other fingers curled) is “011111111,” while the code for “B” (thumb bent, all other fingers straight) is “100000000.” The glove also has an accelerometer and pressure sensor to distinguish between letters like “I” and “J”, whose gestures are different but generate the same nine-digit code.
The electronics on the glove then converts the nine-digit key into a letter and transmits the signals via Bluetooth to a smartphone or computer screen. The glove can wirelessly translate all 26 letters of the American Sign Language alphabet into text. Researchers also used the glove to control a virtual hand to sign letters in the American Sign Language alphabet.
[Image: 'The Language of Glove': a smart glove that wirelessly translates the American Sign Language (ASL) alphabet into text and controls a virtual hand to mimic ASL gestures. Credit: Timothy O'Connor/UC San Diego Jacobs School of Engineering.]