From the New York Times:

“Meijer, a research scientist in the Netherlands, has developed a technology called the vOICe, which allows you to represent visual information – to “see” – with sounds. The device is a tiny camera, a laptop and headphones. The camera is mounted on your head and the laptop takes the video input and converts it into auditory information, or soundscapes. The scene in front of you is scanned in stereo: you hear objects on your left through your left ear and objects on your right through your right ear. Brightness is translated as volume: bright things are louder. Pitch tells you what’s up and what’s down. The image refreshes once a second.”

Read more

This is a wonderful application for 3D audio.

A scientific article I stumbled upon scanned subject’s brains using fMRI whilst they listened to voices being played over headphones. The voices were either played in stereo (inside the head) or using 3D audio (outside the head). The aim was to find the neural substrate for the perception of voices outside the head (in what we might call external auditory space) in an effort to better understand auditory hallucinations of that type. You can find the article for free here. Very interesting.

One other large emerging application of 3D audio that you might not be aware of is for hearing aids. People who require hearing aids at the moment have sounds simply amplified and they are unable to know the direction of where the sounds came from. This makes situations such as a dinner party almost impossible. Normal listeners at a dinner party are able to follow one conversation and ignore the other voices due largely to the fact that the brain knows where the sounds are coming from (this is the so-called Cocktail Phenomenon). Once this ability is lost, as with a hearing aid, all the voices are mashed together and it is impossible to follow a conversation. A hearing aid with 3D audio amplifies the sounds around the listener and at the same time provides a sense of where the sounds are coming from.

Have a listen to this demo (USING HEADPHONES ONLY) to see what it is like have a hearing aid with and without 3D audio. The demo starts without 3D audio and then 30 seconds in it switches to 3D audio, you will hear it don’t worry (think of what it’s like to have hallucinations of people talking while you listen). Focus on trying to understand the different talkers. Again, you must use headphones:

If you make it to Orlando, Florida, you will be able to experience a Disney attraction called Sounds Dangerous that relies heavily on 3D audio. Guests wear special earphones as they watch a short film starring comedian Drew Carey. At a point in the film, the screen goes dark while a 3D audio soundtrack immerses the guests in the ongoing story. It should be noted that this is not a generated effect but a binaural recording (see here).

The novel is called ‘The Death of Bunny Munro’, and you can find more info here: www.thedeathofbunnymunro.com

The explanation from the site for the audiobook goes as follows, “The audiobook uses a groundbraking 3D audio spatial mix, specifically designed for listening on headphones, creating a fully immersive experience for the listener”.

Have a listen here USING HEADPHONES! Notice that as the sounds behind Nick Cave’s voice are heard, they tend to move around your head. Listen carefully…

This is a great application of 3D audio as most audiobooks are destined for mp3 players and thus listened using headphones. It really takes listening to audiobooks to a new level, and according to the publishers is “unprecidented”.

The good news is that with the technology from the ‘listen with your own ears’ project you will be able to, in the not-too-distant future, have any audiobook that you may have on your mp3 player (that is not recorded in 3D audio) instantaneously converted to 3D audio and have the sound personalised according to the shape of your ears (see New York Times article on the project).

The aim of the first stage of the project has been to predict where in this space you the listener would be. We are in the process of refining a tool that will be able to predict your position in this space based on a few key morphological parameters (mainly related to the size of your ear). Once your position has been calculated we can then choose the closest neighbour (best set of ears) that is best suited for you. Have a listen here (or click on ‘Project Demo’ at the top of the page) to experience what it is like to listen with another person’s ears. See which set of ears you like more: the red, green or purple?

People Space

Is 3D here to stay? Could 3D audio be the next technology production companies roll out in order to lure customers back to cinemas and away from free downloads over the internet?

A recent article in The New York Times has suggested that the estimated $58.2 million in tickets at North American theatres over the opening weekend of ‘Monsters vs. Aliens’ “indicates that 3-D is the story”. The article goes on to say that a very significant number of theatres are being equipped for future 3D releases suggesting that the movie format could be here to stay this time.

Given the success of ‘Monsters vs. Aliens’ and the willingness of audiences to wear 3D glasses, could it be imaginable that they would be happy to wear headphones at the movies for a 3D audio effect? Using headphones at the cinema would allow for the ultimate surround sound experience, given that speakers can only create a convincing effect for a small number of seats in the cinema at the so-called ‘sweet spot‘. Movies could also be played in many different languages at the same time.

It’s an interesting concept, only time will tell.

What is interesting about Microsoft’s vision is the focus on tactile technologies, and in particular augmented and virtual reality. This is particularly relevant for the ‘listen with your own ears’ project (see About) given that the technology is all about creating a personalised virtual audio environment. Thus it could be used to compliment the technologies of the future in a big way.

Have a look at the montage and see what you think.

Visit the site and watch the short movie (or watch it below) for some amazing audio effects. Remember to wear your headphones.

Enjoy

The GSMA Mobile World Congress (formerly 3GSM World Congress) represents the world’s largest exhibition for the mobile industry with a stimulating and insightful congress that brings together prominent leaders and personalities from mobile operators and equipment vendors, as well as Internet and entertainment professionals. Arkamys will have a stand at the GSMA Mobile World Congress showing off its latest technology from the 16th to the 19th of February. http://www.mobileworldcongress.com/overview.shtml