Forget spy pens and recorders. An enterprising electrical engineering and computer science graduate has developed a smart algorithm that can reconstruct any kind of sound by reading the vibrations of a bag of chips placed near it.
Abe Davis, in collaboration with MIT, Adobe and Microsoft researchers, has created an innovative algorithm that can analyze surrounding visible vibrations recorded with the help of a camera, and translate them into intelligible speech.
These experiments attempt to retrieve audio information from video feed, through the subtle vibrations, for instance on the surface of a bag of chips, aluminium foil, leaves of plants and even water. Some of the experiments have employed high-speed cameras, with frame rates ranging from 2000 to 6000 frames per second, to capture the minute vibrations of the neighboring objects.
The team has also successfully used standard digital cameras, placed at a distance of 15 feet from the origin of the sound. With the help of a special software developed by MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), the researchers have been able to amplify these minute motions. The data collected is then utilized to distinguish the gender of the speaker, the number of people talking in the room and even the exact words uttered. Talking about the nature of the project, soon to be presented at the 2014 Siggraph conference, Davis says:
When sound hits an object, it causes the object to vibrate…The motion of this vibration creates a very subtle visual signal that’s usually invisible to the naked eye. People didn’t realize that this information was there…..We’re recovering sounds from objects…That gives us a lot of information about the sound that’s going on around the object, but it also gives us a lot of information about the object itself, because different objects are going to respond to sound in different ways.
Although immediate applications of this invention include law enforcement and forensics-related espionage, one cannot help but wonder if it’ll eventually prove to be lethal when it comes to encroachment of personal privacy.
Check the video that showcases how the researchers successfully extract sound from near objects –how MIT researchers extract audio how MIT researchers extract audio
Source: MIT News