A dummy head recording has the downside that it's no longer fully suitable for playback over anything besides headphones. Also, it is dependent on the HRTF of the dummy matching that of the listener.
I'm not sure Ambisonics [1] would work for 5.1 sound. In most cases, 5.1 isn't produced by mic'ing in the directions of the ideal 5.1 speaker placement (at least for movies and music, I don't know anything about 5.1 production for video games). Typically, the L and R channels are used for directional sound, the C is used for dialog and other things that might be corrupted by panning, and the surround and bass channels are used for (weakly directional) ambient sound. So trying to 3-D pan the surround channels will probably just muddle the effect.
Really, the best thing would be to actually record or synthesize in the Ambisonic format, and resynthesizing based on the playback setup. But a 5.1 recording, even one created with a 5.1 rig, is poorly suited for this.
[1] For those who aren't aware, Ambisonics is a a format for storage of 3-D recordings, independent of the mic and speaker placements at the time of recording and playback. If the mic positions are known, you can project the raw recorded signals from multiple mics onto a neutral format. Likewise, if the speaker positions are known at playback time, the neutral format can be projected onto the proper signals for the playback setup to best simulate the recorded soundfield. Ideally, the recording and playback setups are optimized to capture the full 3-D soundfield.
As in, it wouldn't be a cohesive audiofield in of itself, it merely would be like standing in front of a 5.1 system.
I think the trick was using headphones as a very wide stereo dipole system, and just doing all the timing correctly (ie, lesser than full scale HRTF). Due to how 5.1 systems are positioned (30 degrees between L and C, C and R, and 60 between L and R), there isn't much you can do to get this correct.
I'm not trying to be an ass, but do you have any hard evidence or citations for that claim?
I have been a skeptic of the technology as well, but just 2 weeks ago I tried a demo [1] that was _very_ convincing. In short, I wore ~$30USD headphones while a device with some positional sensors was moved around my head and music played. The only sound emitted was from the drivers in the headphones, but I do not exaggerate one bit when I say that my brain was tricked into believing the sound originated from the mystery box moving around my head.
Perhaps you have some scientific reason for why in theory (or fact) it doesn't 'work' in the physical sense, but my anecdotal evidence supports that there is a way to do a 'good enough' job where the brain is successfully tricked.
I admit to feeling there's still a bit of a snake-oil salesmen aspect to this technology, but the guys at Visisonics spun off their company from research at University of Maryland. They're not just another ragtag shop implementing your last decades' positional audio algorithms. They really seem to be pushing the space forward.
Scientific theory doesn't mean shit if the resulting product doesn't work.
And I've heard working audio cues on natural audio, but as I said, they've all been binaural recordings done with dummy heads, so you can't claim its just something defective with my hearing.
I want headphone downmixing of 5.1 surround to work, but I've yet to find one that works without butchering the audio.
That's the thing, I'm claiming that the implementation I tried DID work. Now, questionable YouTube videos, however, haven't worked for me before. Nor do the settings in the software for my discrete sound card. But I have experienced it with one particular company's implementation.
What was explained to me is that they have a generic HRTF that works for ~90% of the population, but there's a minority of the population for which a customized HRTF must be determined. They claimed that certain differences in the spatial characteristics result in their generic HRTF not being a good fit; perhaps you are part of that unlucky 10% who would need to be measured in a lab.
This was actually the topic of one of the questions I asked of the presenters, and their answer was to concede that unfortunately those 10% would be out of luck unless they got their personal HRTF derived. Who knows if the numbers are real or not.
Sorry you're getting downvoted for having a dissenting opinion/experience.
Dummy head binaural recordings work perfectly for me. I can hear forwards, backwards, up, and down perfectly.
The only 5.1 -> headphone HRTF implementations I've heard that properly give directional cues mangle the sound so bad that it isn't worth it.
The technology I tried is intended to be used for real-time games. Individual game objects' audio samples and their positions are run through their filter, accounting for position and the material properties of nearby reflective surfaces (hard vs soft, solid vs semi-solid).
Is this fundamentally different to the implementations you've tried? It seems to me you're referring to '3D' implementations that take an already-mixed 4 channel source and try to jam that into a spatial 2 channel mix. That's the kind of technology that I think is pretty lame/useless as well...
If you have a chance to try out Visisonics tech, I think it'd be worth giving it a go if games are your thing. Seems like you definitely know a lot about the subject.
Problem is, I rarely meet a game that does (I think maybe I own one or two, but I can't remember which ones, which tells you what sort of impact those games made on me).
Being able to already have positional queues on a per sound basis is much different than downmixing 5.1. The technology isn't useless when all I want to do is watch a movie and be able to enjoy it with headphones without garbled audio.
Like I said above, you can downmix left, center, and right, and throw in a little sub, and get a stereo/headphone mix, but positional cues provided by surrounds are gone, and you can't just mix those in normally, the mix gets too confused.
You could use Soundman OKM 2 microphones to measure out your ears and then I can put your measurements into my algorithm and it'll work just fine :)
For me surround sound is only amazing when used in video games or virtual reality in Head Phones only and not speakers.
Same with 5.1 sound for music also falls flat on my ears. The 5.1 mixes are impossible to do well since the engineer is mixing to his 5.1 and your 5.1 will certainly be different especially with sound reflections. On a head phone with surround sound it works out much better. I find the 2.0 head phone mixes are usually much better.
Trying to reprocess 5.1 surround into either a) making it sound like I'm there, or b) making it sound like I'm listening to a 5.1 audio system, has never worked for me. Dolby Headphone is the least objectionable one of them, but it just mangles audio.
I wish someone would try to do Ambisonics-like calculations to try to bake 5.1 into working surround sound. I'd even pay for it (for, say, a faux passthrough driver on Windows that outputs to your default sound card).