[Infowarrior] - Snooping: It's not a crime, it's a feature

[Richard Forno]

Snooping: It's not a crime, it's a feature

New apps hijack the microphone in your cell phone to listen in on your life

Mike Elgan
 
April 16, 2011 (Computerworld)

http://www.computerworld.com/s/article/print/9215853/Snooping_It_s_not_a_crime_it_s_a_feature

Cellphone users say they want more privacy, and app makers are listening.

No, they're not listening to user requests. They're literally listening to the sounds in your office, kitchen, living room and bedroom.

A new class of smartphone app has emerged that uses the microphone built into your phone as a covert listening device -- a "bug," in common parlance.

But according to app makers, it's not a bug. It's a feature!

The apps use ambient sounds to figure out what you're paying attention to. It's the next best thing to reading your mind.

Your phone is listening

The issue was brought to the world's attention recently on a podcast called This Week in Tech. Host Leo Laporte and his panel shocked listeners by unmasking three popular apps that activate your phone's microphone to collect sound patterns from inside your home, meeting, office or wherever you are.

The apps are Color, Shopkick and IntoNow, all of which activate the microphones in users' iPhone or Android devices in order to gather contextual information that provides some benefit to the user.

Color uses your iPhone's or Android phone's microphone to detect when people are in the same room. The data on ambient noise is combined with color and lighting information from the camera to figure out who's inside, who's outside, who's in one room, and who's in another, so the app can auto-generate spontaneous temporary social networks of people who are sharing the same experience.

Shopkick works on both iPhone and Android devices. One feature of the app is to reward users for simply walking into participating stores, which include Target, Best Buy, Macy's, American Eagle Outfitters, Sports Authority, Crate & Barrel and many others. Users don't have to press any button. Shopkick listens through your cellphone for inaudible sounds generated in the stores by a special device.

IntoNow is an iOS app that allows social networking during TV shows. The app listens with your iPhone or iPad to identify what you're watching. The company claims 2.6 million "broadcast airings" (TV shows or segments) in its database. A similar app created for fans of the TV show Grey's Anatomy uses your iPad's microphone to identify exactly where you are in the show, so it can display content relevant to specific scenes.

While IntoNow is based on the company's own SoundPrint technology, the Grey's Anatomy app is built on Nielsen's Media-Sync platform.

Obviously, the idea that app companies are eavesdropping on private moments creeps everybody out. But all these apps try to get around user revulsion by recording not actual sounds, but sound patterns, which are then uploaded to a server as data and compared with the patterns of other sounds.

Color compares sounds between users to figure out which users are listening to the same thing. Shopkick compares sounds to its database of unique inaudible patterns that identify each store. The SoundPrint- and Media-Sync-based apps compare sound patterns to their database of patterns mapped from all known TV shows.

Who else is listening?

Apps that listen have been around for years. One type of app uses your phone's microphone to identify music. Apps like Shazam and SoundHound can "name that tune" in a few seconds by simply "listening" to whatever song is playing in the room.

A class of alarm clock apps uses your phone's microphone to listen to you sleep. One example is the HappyWakeUp app. If you're sleeping like a log, the app avoids waking you. When HappyWakeUp hears you tossing and turning near the scheduled time, it wakes you up with an alarm.

Of course, the use of your microphone with these apps is well understood by users, because that's the main purpose of the app.

The new apps are often sneakier about it. The vast majority of people who use the Color app, for example, have no idea that their microphones are being activated to gather sounds.

Welcome to the future.

Coming soon: A lot more apps that listen

What you need to know about marketing and advertising is that data is king. Marketers can never get enough, because the more they know about you and your lifestyle, the more effective their marketing and the more valuable and expensive their advertising.

That's why marketers love cellphones, which are viewed as universal sensors for conducting highly granular, real-time market research.

Of course, lots of apps transmit all kinds of private data back to the app maker. Some send back each phone's Unique Device Identification (UDI), the number assigned to each mobile phone, which can be used to positively identify it. Other apps tell the servers the phone's location. Many apps actually snoop around on your phone, gathering up personal information, such as gender, age and ZIP code, and zapping it back to the company over your phone's data connection.

Most app makers disclose much of what they gather, including audio data, but they often do so either on their websites or buried somewhere in the legal mumbo jumbo.

It turns out that, thanks to sophisticated pattern-recognition software, harvested sounds from your home, office or environment can be transformed into marketing demographic gold.

You should know that any data that can be gathered, will be gathered. Since the new microphone-hijacking apps are still around, we now know that listening in on users is OK. So, what's possible with current technology?

By listening in on your phone, capturing "patterns," then sending that data back to servers, marketers can determine the following:

	• Your gender, and the gender of people you talk to.
	• Your approximate age, and the ages of the people you talk to.
	• What time you go to bed, and what time you wake up.
	• What you watch on TV and listen to on the radio.
	• How much of your time you spend alone, and how much with others.
	• Whether you live in a big city or a small town.
	• What form of transportation you use to get to work.
All this data and more, plus the UDI on your phone, could enable advertising companies to send you very narrowly targeted advertising for products and services that you're likely to want.

The future of marketing is contextual. And listening in on your life will enable marketers to deeply understand not only who and where you are, but also what you're paying attention to.

How do you feel about cellphone apps listening in on your life? If you'd like to tell me, I'm listening, too.

Mike Elgan writes about technology and tech culture. Contact and learn more about Mike at Elgan.com, or subscribe to his free e-mail newsletter, Mike's List.