News for hackers. Stuff that matters.
Re:Optical microphone can separate multiple instruments from... by anigbrowl on Monday June 27, 2022 @02:12AM (#31891146)
I've been doing pro audio stuff or 25 years and this is a landmark paper, biggest breakthrough I've seen in years. I'm astonished at the quality of extracted signals. Biggest thing I've seen since deconvolution became good enough for realtime or near-realtime adaptive noise reduction.
Re:Optical microphone can separate multiple instruments from... by ackbar03 on Tuesday June 28, 2022 @07:57PM (#31914685)
I thought something similar was done with the laser on window pane thing already
Re:Optical microphone can separate multiple instruments from... by TrainedMonkey on Tuesday June 28, 2022 @09:08PM (#31915251)
Laser stuff was a single point. The advantage here is capturing a set of 2d pixels which let you see how signal travels through space. This essentially means a large array of microphones which are next to each other and in perfect sync (which I hear is hard when you try to do this with multiple microphones).
Re:Optical microphone can separate multiple instruments from... by williamcotton on Tuesday June 28, 2022 @09:26PM (#31915383)
It is hard due to phasing, that is, overlapping audio signals from the same source when summed during mixing can cancel out various parts of the signal. Channels on some mixers have a phase inverter and audio engineers will also move microphones around while looking at a phase monitor.
Re:Optical microphone can separate multiple instruments from... by cma on Wednesday June 29, 2022 @12:04AM (#31916370)
Re:Optical microphone can separate multiple instruments from... by rcxdude on Wednesday June 29, 2022 @07:13AM (#31918629)
This isn't how the system works. It basically can give you a small number of contact microphones (you need to aim a laser at each point you want to record).
Re:Optical microphone can separate multiple instruments from... by pishpash on Wednesday June 29, 2022 @04:27AM (#31917703)
Interesting interpretation of this as a high-density high-resolution array! But does it need to be isolated well from movement? Since sound wavelengths are long, so maybe not?
Re:Optical microphone can separate multiple instruments from... by infogulch on Wednesday June 29, 2022 @12:13AM (#31916426)
This is better quality (see comparisions in the video demo) and it's done with mostly normal cameras, not specialized high framerate cameras.
Re:Optical microphone can separate multiple instruments from... by anigbrowl on Wednesday June 29, 2022 @01:29PM (#31923208)
It has been, but the quality here is much better. Like, that example might have been good enough for intelligence or legal purposes, this is goon enough for commercial/entertainment purposes.

The main headache I see is that it still requires a somewhat expensive and complex camera setup, but I can see that coming within the realms of affordability/standardization quite soon.

Re:Optical microphone can separate multiple instruments from... by de6u99er on Tuesday June 28, 2022 @04:52PM (#31912858)
I hope this can be used to objectively distinguish between real audiophile equipment and audiophile snake oil.
Re:Optical microphone can separate multiple instruments from... by capableweb on Tuesday June 28, 2022 @05:57PM (#31913595)
Then please read the submitted article, the press release (https://www.cs.cmu.edu/news/2022/optical-microphone) or the paper itself (https://www.marksheinin.com/_files/ugd/a41a28_7d370603fafd41...) again, as it has nothing to do with audio quality.
Re:Optical microphone can separate multiple instruments from... by Dan_Sylveste on Tuesday June 28, 2022 @10:09PM (#31915660)
'Real' audiophile equipment is the stuff that sells to recording studios like powered studio monitors and rackmount solid state transports. It costs a lot less than the consumer audiophile stuff, too. If you're buying 'consumer' audiophile products like huge floorstanding speakers made of whatever wood of the day is in favour then you're buying status symbols, not audio reproduction quality.
Re:Optical microphone can separate multiple instruments from... by justinclift on Wednesday June 29, 2022 @09:12AM (#31919624)
> It costs a lot less than the consumer audiophile stuff, too.

That's the first time I've heard anyone say that studio monitors cost _less_ than consumer audiophile stuff.

Saying that as a person that buys and uses studio monitors, they are not cheap in any way.

Even "affordable" entry level stuff (eg JBL Pro LSR 305P/310S) is going to set you back a good chunk of change compared to consumer gear.

Re:Optical microphone can separate multiple instruments from... by Dan_Sylveste on Wednesday June 29, 2022 @10:08PM (#31928021)
Hmm from my perspective a pair of JBP LSR 308P MkII and a transport matches the price of entry-level consumer audiophile speakers and a cheap (Topping/SMSL) AMP/DAC pairing yet will outperform it any day.
Re:Optical microphone can separate multiple instruments from... by justinclift on Thursday June 30, 2022 @12:39PM (#31935068)
Heh Heh Heh. You also need to add the matching 310S sub (https://jblpro.com/products/lsr310s) for things to sound "right". :)

They're engineered to work together, and while you _can_ go without the sub for a while... it's a shit setup and only useful in the short term.

Wouldn't want to be doing production stuff (or even end user setup) without the sub. ;)

Re:Optical microphone can separate multiple instruments from... by Dan_Sylveste on Friday July 01, 2022 @06:19AM (#31944470)
I thought that the sub was less necessary with the 8" monitors? Maybe I misread or misremember.

One of the thing that's interesting to note with monitors is how 'biased flat' is very much a thing and that consumer audiophile speakers are very much not always biased flat, I think the varying companies have a sound profile which they aim to target.

Re:Optical microphone can separate multiple instruments from... by totetsu on Tuesday June 28, 2022 @07:52PM (#31914643)
This is for espionage not hifi audio
Re:Optical microphone can separate multiple instruments from... by macgyverismo on Monday June 27, 2022 @07:23AM (#31893025)
I love the creative use of the rolling shutter, instead of seeing it as a downside, they turned the line-by-line nature of the sensor into sample rate multiplier.
Re:Optical microphone can separate multiple instruments from... by dougabug on Tuesday June 28, 2022 @10:55PM (#31915941)
The use of rolling shutter to increase effective sampling rate was present in the original SIGGRAPH 2014 Davis et al paper from Bill Freeman’s group at MIT, “The Visual Microphone: Passive Recovery of Sound from Video” (https://dspace.mit.edu/handle/1721.1/100023).

The authors of the current paper cite this and other prior work. The key innovation is the use of both a rolling shutter and a global shutter reference.

Re:Optical microphone can separate multiple instruments from... by okwubodu on Tuesday June 28, 2022 @07:12PM (#31914310)
That blew me away before I even finished the paper.
Re:Optical microphone can separate multiple instruments from... by anfractuosity on Tuesday June 28, 2022 @04:55PM (#31912893)
Ooh that's really cool that they're using the laser speckle pattern. I like the fact they exploit the rolling shutter too. Something which https://people.csail.mit.edu/mrub/VisualMic/ also does.

There are devices which are called laser doppler vibrometers, which might also be able to do this by pointing at the strings/base of the guitar?

There do seem to be videos of laser doppler vibrometers being used with guitars on youtube, but I'm not sure if the soundtrack that goes along with them is just from a normal mic.

I had a little play with laser speckle patterns to detect keypresses, as they can help find very subtle changes to a surface - https://www.anfractuosity.com/projects/fun-with-speckle-patt... (by 'diffing' the patterns)

Re:Optical microphone can separate multiple instruments from... by zbrozek on Wednesday June 29, 2022 @12:47AM (#31916597)
I came here to post about LDVs but was beaten to it.

https://en.m.wikipedia.org/wiki/Laser_Doppler_vibrometer

They're sensitive to very small vibrations. A friend of mine used them while working at a hard drive manufacturer to better understand head and platter vibrations.

Re:Optical microphone can separate multiple instruments from... by ssfrr on Tuesday June 28, 2022 @10:06PM (#31915644)
LDVs are the gold standard for noncontact vibration measurement and are widely used in acoustics. The main problem is that they’re pretty expensive (I think on the order of a few 10s of k$)
Re:Optical microphone can separate multiple instruments from... by klabb3 on Tuesday June 28, 2022 @06:26PM (#31913881)
Combined with sophisticated noise cancelation and other relatively mature tech, this could make intentional focus listening possible, analogous to looking at something, as well as closing your eyelids, but for hearing.

Imagine being able to shut off specific ambient noises (and sometimes.. people) without losing spatial awareness. Or tune in a source you're paying attention to (the cocktail party problem).

The issue with super-hearing would be to re-adjust expectations of who can reasonably hear us. Could be used for creepy things, obviously..

Re:Optical microphone can separate multiple instruments from... by TheHideout on Tuesday June 28, 2022 @04:59PM (#31912930)
This seems like an interesting optical solution to the "cocktail party problem", which can be solved using Independent Component Analysis [0]

[0] https://en.wikipedia.org/wiki/Independent_component_analysis

Re:Optical microphone can separate multiple instruments from... by 38_9503-94_5201 on Friday July 01, 2022 @11:52AM (#31948200)
check your PI mail, syn9; the game is on
Re:Optical microphone can separate multiple instruments from... by thanatos519 on Tuesday June 28, 2022 @07:18PM (#31914361)
I'm imagining Q delivering this technology to 007, except that it would shatter the audience's suspension of disbelief. Truly astounding sorcery, yet at the same time completely straightforward.
Re:Optical microphone can separate multiple instruments from... by orbital-decay on Tuesday June 28, 2022 @04:01AM (#31905296)
Incredible separation, I don't think it's attainable by any other means. Should be super useful for speech in noisy environments.

I have a question though, is capturing lateral movements of a single spot on the instrument enough to represent how it sounds for a human ear? I think it's equivalent to a polarizer filter as it doesn't seem to be capturing depth axis vibrations.

Re:Optical microphone can separate multiple instruments from... by jcims on Tuesday June 28, 2022 @06:12PM (#31913749)
These example audio at the bottom. Seems the answer is no. But you could potentially use it as a means to isolate the sound from the recorded audio. You would have to synchronize the phases though because the optical speed of sound is c.
Re:Optical microphone can separate multiple instruments from... by pishpash on Wednesday June 29, 2022 @04:30AM (#31917715)
Be careful what you wish for, I can see a future where it's like EM today, the entire environment saturated with spread-spectrum (audio) noise and everyone has to wear a filtering+array gain device in their ear.
Re:Optical microphone can separate multiple instruments from... by infogulch on Tuesday June 28, 2022 @05:41PM (#31913387)
> it doesn't seem to be capturing depth axis vibrations

Good point, the paper mentions x-axis and y-axis, but doesn't mention z-axis. Maybe depth vibrations could be resolved as changes to the interference pattern?

Re:Optical microphone can separate multiple instruments from... by pbhjpbhj on Tuesday June 28, 2022 @06:23PM (#31913860)
Do human ears resolve sound in more than one dimension? I've always considered that, per ear, we only get a sequence of compressions and rarefactions on the ear drum, that other aspects of hearing are through combination or 'cheating' (skin sensitivity and such). So, would it matter?
Re:Optical microphone can separate multiple instruments from... by kimburgess on Wednesday June 29, 2022 @01:20AM (#31916742)
There’s binaural localisation from phase and amplitude differences, but importantly also monoaural cues from frequency response. Your pinna act as a set of directional filters - which is part of why some binaural recordings can quite literally feel as though they’re in your head.

What’s interesting with this is that while you can’t get those directional filters, you could use a system like this to provide 3 or more ‘sensors’ (eg the chip packet demo) within a scene and isolate signals the same way as an array mic.

Re:Optical microphone can separate multiple instruments from... by infogulch on Wednesday June 29, 2022 @12:14AM (#31916437)
No, but a thing vibrating in any dimension will still produce sound waves.
Re:Optical microphone can separate multiple instruments from... by taeric on Wednesday June 29, 2022 @12:59AM (#31916649)
You can pinpoint with your two ears where sound is coming from, fairly well. Takes some training, of course.

That said, I get the impression this is more complicated than the autocorrelation of multiple sensors?

Re:Optical microphone can separate multiple instruments from... by pontifier on Wednesday June 29, 2022 @12:31AM (#31916527)
I was just wishing for something like this yesterday.

I wanted to figure out how to detect some very very low infrasound reliably, and no conventional microphone technology seemed like it could do what I needed.

This feels like it could form the basis of a new wave of scientific vibration measurement systems.

Re:Optical microphone can separate multiple instruments from... by tnjm on Wednesday June 29, 2022 @05:50AM (#31918134)
Not only is the tech here astonishing, but full credit to the authors for producing such a clear, concise video accessible to just about everyone explaining what they did, how they did it, and why it matters. Excellent storytelling.
Re:Optical microphone can separate multiple instruments from... by 323 on Tuesday June 28, 2022 @08:46PM (#31915099)
Can this be used for crowd audio surveillance?

Lasers on light poles next to the already there high resolution cameras and you could record what everybody is speaking while walking on street.

Re:Optical microphone can separate multiple instruments from... by its_bbq on Tuesday June 28, 2022 @10:00PM (#31915600)
An interesting thought experiment. Are there vibrations on our throats or is it all taking place hidden inside our bodies?
Re:Optical microphone can separate multiple instruments from... by yjftsjthsd-h on Tuesday June 28, 2022 @10:24PM (#31915748)
Throat mics exist, so probably? (If I'm understanding your question correctly)
Re:Optical microphone can separate multiple instruments from... by imchillyb on Tuesday June 28, 2022 @10:47PM (#31915899)
When you speak, even your chest vibrates with the sound. Our bones conduct the vibrations in our skulls. This system could eventually be tuned to read those vibrations.
Re:Optical microphone can separate multiple instruments from... by ben_w on Wednesday June 29, 2022 @06:08AM (#31918237)
Ok, this is bizarre. A few days ago, and with no connection at all to any of this work, I had a similar idea. Not the same idea, just similar, and mine is just at the “hmm” stage (which also means mine may turn out to not work), but this still feels really weird.

Anyway, my thought was: laser goes to semi-silvered mirror between camera CCD and camera lens, passes through lens to diverge outward into environment, reflects off environment in the same way as a normal laser microphone, separate return signal now exists for all pixels on CCD.

Point this at a wall, do the right transformation (is a Fourier transform sufficient?) and the entire wall can be used as a computational phased array of microphones to listen to a specific (possibly moving) target.

Possibly mix with the original laser light to get beat patterns due to red/blue shift, like radar speed guns, but that feels like a separate application entirely.

I’m really impressed by this result in particular:

> Combining 63-fps video from two cameras, one with a global shutter and one with a rolling shutter, allows the researchers to recover a sound signal at 63,000 Hz

Because shutter speed was my first concern about possible limitations when I had my other idea.

Re:Optical microphone can separate multiple instruments from... by kazinator on Thursday June 30, 2022 @03:27PM (#31937330)
> a 63-fps limit on input data would seem to place a 63-Hz upper limit on the sound this device can "see."

Obviously not because it's not taking one amplitude sample. It's analogous to taking 63 sliding FFT's per second, which may be based on on thousands of samples and capture high frequency content. This speckle pattern being sampled is some kind of FFT-like transform of the signal containing lots of information.

But the 63 Hz sampling will have to show up as a limitation. I would expect it to be excellent for periodic signals, but to struggle with transients, like the attack of a percussion instrument such as a snare drum.

Re:Optical microphone can separate multiple instruments from... by wanderingstan on Tuesday June 28, 2022 @06:56PM (#31914165)
This tech could be applicable to ticketing loud vehicles, as discussed here earlier this year: https://news.ycombinator.com/item?id=30364669
Re:Optical microphone can separate multiple instruments from... by iguana_lawyer on Monday June 27, 2022 @04:19AM (#31891884)
Could an optical microphone be used to identify materials using resonance frequencies?
Re:Optical microphone can separate multiple instruments from... by crispyambulance on Wednesday June 29, 2022 @06:45AM (#31918457)
I just watched the video and am a little confused about the laser illumination.

Does the laser have to be "aimed" at each object of interest or is it just laser illumination of the whole scene?

The graphics suggest there's a laser point on each object, but that means it has to be aimed and thus follow the subject, right?

Moreover many objects, like guitars, have complex oscillation modes so if you are "listening" to just "one point" on the surface, you're not picking up the sound from the other parts of the guitar which are oscillating differently.

Re:Optical microphone can separate multiple instruments from... by rcxdude on Wednesday June 29, 2022 @07:12AM (#31918624)
The laser needs to be aimed at each point of interest. The system can track multiple laser points simultaneously with one pair of cameras, but there's a tradeoff with quality (because essentially the camera's field of view is being divided into slices, one for each laser point). So you've got a small number of virtual contact microphones you can point at a surface, not a video where you can see the vibration at each point.
Re:Optical microphone can separate multiple instruments from... by IndySun on Sunday July 03, 2022 @08:11AM (#31967107)
Since childhood I have been waiting for vinyl turntables that use a laser instead of a needle to 'play' records, without any wear and tear. This may be the breakthrough.
Re:Optical microphone can separate multiple instruments from... by yodon on Tuesday June 28, 2022 @05:08PM (#31913031)
Is technique this significantly different from the laser microphones[0] that intelligence services have been using for many decades?

[0]https://en.m.wikipedia.org/wiki/Laser_microphone

Re:Optical microphone can separate multiple instruments from... by medler on Tuesday June 28, 2022 @06:40PM (#31914013)
It builds on the technology behind laser microphones. According to the paper, visual vibrometry has historically required expensive cameras, and their method removes this need and appears to have other advantages over using a high-speed camera. They say they contribute “a novel method for sensing vibrations at high speeds (up to 63kHz), for multiple scene sources at once, using sensors rated for only 130Hz operation. Our method relies on simultaneously capturing the scene with two cameras equipped with rolling and global shutter sensors, respectively.“
Re:Optical microphone can separate multiple instruments from... by anewpersonality on Tuesday June 28, 2022 @09:23PM (#31915363)
Does the laser have to track the instrument in space?
Re:Optical microphone can separate multiple instruments from... by knolan on Wednesday June 29, 2022 @03:18AM (#31917369)
They discuss this in their video:

https://youtu.be/_pq0d1oxtA0