Your phone has ears and you might not know about it.
Your phone has ears
In many organizations you would have seen that they cover your phone’s camera with a sticker and sometimes may be laptop camera too. The reason to do this is to avoid being spied on. But what about the microphone, can’t that be used for spying?
The answer is Yes, it can be used for spying. Your phone is not listening to you in the most strict manner , but your activities, your media choices are definitely being watched out. Pressing “ALLOW” button at the start of every app does so.
The point is we can’t really avoid that, as it is required for the proper functioning of any application. But we do need to care about which app actually required mic permissions. Let’s get in detail of it
All this is done with a technology called ACR(Automatic Content Recognition).
Automatic content recognition (ACR) is an identification technology to recognise content played on a media device or present in a media file. This enables users quickly obtain detailed information about the content they have just experienced without any text based input or search efforts.
ACR can help users deal with multimedia more effective and make applications more intelligent.
How ACR works
Take the most popular music recognition and discovery app Shazam for example. The ACR workflow is as follows.
The fingerprinting algorithm processes the signal of audios and extracts digital features called fingerprints for each audio.
Fingerprints are very discriminative so the system can use them to identify the audio they belongs to.
Fingerprints are also robust which means they can resist the environment noise and this make it possible to identify recorded audios in rather noisy environments.
If the system finds matched fingerprints of the query snippet, it can determine the most like audio in database and gives the position of the snippet in the source audio.
The mission of ACR is completed when the system finds the match.
And This is how you see the most required Advertisements and video suggestions
Who is doing all this?
As the internet expands on the horizon is spatial search (think Google Maps) and even bigger, voice search (think Google Voice, Apple’s Siri Window’s Cortana and Amazon’s Alexa). Almost as a corollary, a bunch of companies is betting on voice blooming on its own helping better profile consumers. Advertisers globally spend some $650 billion annually. This is analyzing our ambient sounds and eventually helping target advertising better. This group includes Chinese company ACRCloud, Audible Magic from the US, and the Netherlands’s Betagrid Media and Zapr from India.
Zapr claims to have the largest media consumption analytics database in India. This helps television channels and brands to earn a better bang for their advertising buck. To be sure, advertising – even with the internet’s promise of better targeting – still is an inaccurate business with proxies, at best, helping measure its return on investment.
Zapr’s ACR technology uses the device’s microphone to monitor the audio playing on a device or the ambient surroundings and then creates a profile of the user’s media consumption habits. The data is then shared with the app’s developer to push out personalized content recommendations, ads, or is resold to other advertisers for targeted marketing.
Once ACR technology picks up an audio signal from a device’s microphone, it converts it into an audio fingerprint and then matches it against the database to create a profile of a person’s content preference. Once it is done, the app’s developer can push recommendations that are more likely to be picked up by the user.
Is it legal?
“That is totally a breach of privacy. Unless they give me clear cut indication that it is going to be used only by them for a specific purpose. And they won’t share with any other party, that amounts to breach”, retired Supreme Court judge B N Srikrishna said. However, he added that the absence of comprehensive data privacy laws such as GDPR makes it difficult to categorize Zapr’s technology as illegal or legal.
Sandipan Mondal, Zapr’s CEO, claims that their ACR technology would help brands and advertisers recommend the right kind of content with higher accuracy and efficiency.
When asked , if users are aware of the fact that they are being monitered , he said,
“I don’t know. I think as technologists we have to try our best to be as transparent as possible and not hide things behind legalities and jargon”.
Who are doing this in India?
Some of India’s leading apps and games – including Hotstar, Dainik Jagran, Chhota Bheem Jungle Run and NewsDog – use Audio Content Recognition (ACR) to listen to ambient sound in your vicinity, including snatches of what you are watching on TV, or any videos or music that is playing around you.
Developed by Bengaluru-based Zapr Media Labs for these apps, plenty of popular Indian games developed by Nazara also use ACR. Zapr is backed by the likes of Star (which develops Hotstar), Flipkart, Saavn and Micromax, among more than a dozen other investors. All these apps monitor us and observe what we are watching!
Here is the list of Indian Apps that uses ACR for advertisement suggestion:
How to stay safe from this, if you dont want these apps to monitor you?
One way users can block such apps from tapping into what your microphone is listening to is by blocking the permission in Android. Simply deny the app access to microphone audio, and you should be fine in most cases. The fact that none of above mentioned apps actually need your microphone to work is a blessing.
Of course, Zapr is not the only service which lets app tap into what you are listening to or watching around you.