The world is your touchscreen Thursday, 14 April 2016

Electrical engineers at the University of Washington are using sonar to track finger movements around a smart device without the user needing to touch the device at all.

You can’t type very easily onto a smartwatch display, so we wanted to transform a desk or any area around a device into an input surface,” said lead author and doctoral student Rajalakshmi Nandakumar.

I don’t need to instrument my fingers with any other sensors — I just use my finger to write something on a desk or any other surface and the device can track it with high resolution.”

The system called FingerIO turns a smartwatch or smartphone into a sonar system using the device’s own speaker to emit an inaudible sound wave. That signal bounces off the finger, and those echoes are recorded by the device’s microphones and used to calculate the finger’s location in space.

The team demonstrated that FingerIO can accurately track two-dimensional finger movements to within 8mm, which is sufficiently accurate to interact with modern mobile devices. And because sound waves travel through fabric and do not require a line of sight, users can even interact with a phone inside a front pocket or a smartwatch hidden under a sweater sleeve.

Nandakumar said, as mobile and wearable devices such as smartwatches become smaller, it is getting tougher for people to interact with screens the size of a matchbook.

The researchers employed a type of signal typically used in wireless communication — called Orthogonal Frequency Division Multiplexing — and demonstrated that it can be used to achieve high-resolution finger tracking using sound. Their algorithms leverage the properties of OFDM signals to track phase changes in the echoes and correct for any errors in the finger location to achieve sub-centimetre finger tracking.

Next steps for the team include demonstrating how FingerIO can be used to track multiple fingers moving at the same time, and extending its tracking abilities into three dimensions by adding additional microphones to the devices.

 

 

FingerIO allows users to interact with mobile devices by writing or gesturing on any nearby surface. Photo: Dennis Wise, UW