[Infowarrior] - Your Smartphone Spies On What You Type

[Richard Forno]

Your Smartphone Spies On What You Type 			
Written by Mike James	   
Sunday, 29 September 2013 00:00

http://www.i-programmer.info/news/105-artificial-intelligence/6430-your-smartphone-spies-on-what-you-type-.html

All you have to do is place your phone next to your keyboard to provide a direct channel for anyone to read what you are typing - and it's all down to the vibration of the keys.

We all do it - place our phones down on the desk next to the keyboard. This might not be such a good idea if you want to keep your work to yourself. 

A team of researchers from MIT and the Georgia Institute of Technology have provided proof of concept for logging keystrokes using nothing but the sensors inside a smartphone - an iPhone 4 to be precise.  

The first interesting finding is that the accelerometer inside an iPhone 3GS wasn't good enough to pick up the vibrations of the keys but the iPhone 4 produced a clear signal.  This demonstrates not only that sensors are getting better, but the improvement makes a real difference. 

The iPhone 4 was placed a few inches away from a keyboard and the keys were pressed while a sensor data was fed to two neural networks. The data was processed to produce a feature vector based on a range of temporal and frequency information. One network  was trained to recognize the horizontal position of the key and the other the vertical position. The scan rate was on the low side, 100Hz, due to the limitations fo the hardware. 

Just detecting raw keypresses didn't produce a very accurate result, but switching to picking up pairs of keywords and then using a word dictionary did produce useful data extraction. Accuracies of around 80% were achieved, but the accuracy reduced with the number of keypresses. Word recognition only achieved a 46% accuracy, but this increased to 73% if second choice words were included. Clearly semantic analysis could push the accuracy up. 
 
There have been experiments that recover text by listening to the noise a keyboard makes and these have achieved slightly higher accuracies that this vibration-based approach. The key difference is that this experiment used a smartphone rather than a desktop machine with a big microphone and high-speed sampling. The method may be less accurate, but it could be packaged into smartphone malware and be used without anyone knowing it was going on.  

The paper describes some problems with the method - if the phone is too far from the keyboard then the vibrations aren't strong enough, if  the user taps on the desk or plays a radio then there is too much interference and if the user types too fast the processor can't keep up.

So how to stop such eavesdropping?

Leave the phone outside the room or keep in in a bag next to the desk rather than on it. The researchers also suggest that in future APIs could be modified to restrict the sampling rate to below the Nyquist rate for the data being listened into - say 30Hz in this case. 

It is interesting to speculate on how much better an engineered design might do than the neural network learning approach. The accelerometer acts much like a seismometer and geologists have a lot of techniques for using such data to work out where an earthquake happened. Perhaps the same signal processing techniques could pinpoint the location of the key vibrations without the need to use supervised learning.

Whatever the final outcome, it is clear that with a smartphone in your possession secrecy and privacy are far from certain. Be careful the next time someone leaves their phone on your desk. 

---
Just because i'm near the punchbowl doesn't mean I'm also drinking from it.