It's basically a terrible place to be a developer in it for the money. Really fun work otherwise. The cool gigs are the ones where you build custom plugins for someone's crazy idea.
In consumer applications, plugins are used all the time for prototyping before you go to hardware. MATLAB is way too slow for anything useful.
The price points really aren't horrendous if you consider how expensive the engineering is, how little demand there is, and how long you need to maintain a product. You aren't being ripped off by spending a couple hundred bucks on a plugin. I think we'll end up at a place where everything is a subscription, but I can tell you from experience that it creates friction for the users.
I’m curious if anyone has any direct knowledge about that.
There are so many professional activities similar to that where no one makes any money and people really just do it for the love, and then there are seemingly similar things like that where people make surprisingly large amounts of money.
There are also several solo/small shop developers that do make a living from selling plug-ins. Here are a few that I can think of off the top of my head.
Auburn Sounds: https://www.auburnsounds.com/ Valhalla DSP: https://valhalladsp.com/ Kilohearts: https://kilohearts.com/
It's hard to tell how much Duda is an outlier, though, and how many other people could succesfully follow his path.
One of the things I've also heard from labels is that not only there's money in the VST world (it's also very crowded, piracy is rampant as noted, etc.), a lot of plugins are ported over to iOS and are sold as "virtual pedals". The number of sales and revenue there was noted as being very interesting.
I made fun of him and we wouldn't have trusted it to be used live, but damn it worked impressively well
But you have to be willing to put in the time and make phenomenal products, because no one wants average instruments and effects, we can get those for free.
($1200) https://www.native-instruments.com/en/products/komplete/bund...
($300) https://www.soundtoys.com/product/soundtoys-5/
($500) https://www.arturia.com/products/analog-classics/v-collectio...
However, piracy is also pretty big when it comes to plugins.
>Do solo or small shop vst plugin developers make any money?
It doesn't have their breadth, but the tones it does have are nearly as good as it gets without serious air movement.
So realtime might match your definition, but it is consistent in audio production.
For humans, you can start to notice the lag @ 50ms. (A selection of experimental results summarized here https://gamedev.stackexchange.com/a/74975)
You can learn to play it. Pipe organs routinely have more than 50ms latency just from the distance the sound has to travel from the pipes to the organist. Add the time needed to set up steady oscillations in large pipes, and the slow pneumatic actions found in some organs, and >200ms latency is nothing unusual. The important thing is that the latency is consistent.
(jk)
One meaning is just that you can guarantee specific deadlines. So if your programme can react within an hour guaranteed, that would be real-time. (Though usually we are talking about tighter deadlines, like what's needed to make ABS brakes work.)
For 'real time' music usage you wouldn't need strict guarantees, but something that's usually fast enough.
“Usually fast enough” are three words that guarantee failure in a live show/MIDI environment, which is a large use case of VST and its peers beyond production. By extension, “usually fast enough” further guarantees nobody will ever use your software. That’s noticeable right away.
The question isn’t about compsci real-time theorycrafting, it’s “here’s a buffer of samples, if you don’t give it back in a dozen milliseconds the entire show collapses.” That’s pretty clearly meant by “real time“ contextually.
That said, I agree that the question of what "real-time" might mean is irrelevant given the context.
I’ll repeat again that any compsci theorycrafting is not the concern here, and real-time has a very specific meaning in DSP. Computer science does not own the concept of real-time, and the only people tripping over the terminology are those with more compsci experience than DSP. I appreciate everyone trying to explain this to me, but (a) I understand both, and (b) this is like saying “no, Captain, a vector could mean anything like a mathematical collection, air traffic control should learn a thing or two from mathematics.”
It's not "theorycrafting" to say that real-time music software running in a preemptive multitasking operating system without deterministic process time allocation will have to suffer the possibility of occasional drops. It happens in practice and audio drivers have to be implemented to account for the bulk of it, and the VST API is designed in such a way that failure to fill a buffer on time needn't be fatal.
Of course audio is block buffered over (mostly) USB, and as long as the buffers are being filled more quickly than they're being played out, the odd ms glitch here and there is irrelevant.
As real-time systems Windows, MacOS and Linux are terrible from a theoretical POV, and they're useless for the kinds of process control applications where even a ms of lag can destroy your control model.
But with adequate buffering and conservative loading they work well enough to handle decent amounts of audio synthesis processing without glitching - live, on stage.
> Of course audio is block buffered over (mostly) USB, and as long as the buffers are being filled more quickly than they're being played out, the odd ms glitch here and there is irrelevant.
As I've noted earlier in the thread. In fact, that the only thing you can offer under such circumstances is that "it usually doesn't happen" because "it's usually fast enough" is my entire point.
> As real-time systems Windows, MacOS and Linux are terrible from a theoretical POV, and they're useless for the kinds of process control applications where even a ms of lag can destroy your control model.
You could employ the same strategies to process control problems where latency is not a problem so much as jitter. You don't, because unlike a music performance an occasional once-in-a-week buffer underflow caused by a system that runs tens to hundreds of processes already at boot can actually make lasting damage there.
The medium answer is "this is a wavenet model, so inference is probably really expensive unless the continuous output is a huge improvement to performance".
Although the unusual structure of the net here may mean you're doing original and possibly publication-level work to adapt that stuff to this net structure.
If you were really interested in this, there could also be some profit in minimizing the model and then figuring out how to replicate it in a non-neural net way. Direct study of the resulting net may be profitable.
(I'm not in the ML field. I haven't seen anyone report this but I may just not be seeing it. But I'd be intrigued to see the result of running the size reduction on the net, running training on that network, then seeing if maybe you can reduce the resulting network again, then training that, and iterating until you either stop getting reduced sizes or the quality degrades too far. I've also wondered if there is something you could do to a net to encourage it not to have redundancies in it... although in this case the structure itself may do that job.)
(This goes in both directions on the spectrum too. You can have your hearing damaged by infrasound as well.)
Are you speaking from some experience with which I’m unfamiliar where it’s okay for DSP code to fail hourly? Trying to understand your viewpoint.
The article and your comments inspired in me the idea of a wave-net based VST learning wrapper. If the real plugin fails, substitute a wave-net based simulation of the plugin.
The behavior you describe (zero signal on underruns) is a common mitigation. The DAW or the driver itself initializes that'll eventually be handed to the sound card to zero before the host application requests the plugins to process, and if it doesn't have time to mix the plugin outputs it'll play back the initialized buffer instead.
From aea12 one might think that it's normal for an underrun to be fatal. Because underruns are not an exceptional occurrence during production (where you might occasionally load one plugin too many or run a different application with unpredictable load characteristics like a web browser) it really isn't an unexplored area and although they're are a pretty jarring degradation I've never experienced crashes that directly correlated with underruns.
If your plugins are crashing because of an underrun you have a much more serious problem than underruns. Then you have plugins writing to or reading from memory that wasn't either handed to them by the host or allocated by themselves. That bad code running in your process can cause it to crash is an orthogonal problem to buffer underruns causing skips or stuttering in audio.
The author says it works in real-time, but to non music/audio folks this could mean '100 ms latency is real-time enough, right?'
Generally, I think the audio VST business is a really fun space to be in for a lifestyle business, as it is way too small to be attractive for VCs. It seems like a space that provides many niches for lots of small players to thrive in.
As an aside, it's really quite interesting that a lot of cutting edge tech is now used to emulate the hardware-based tech of yesteryear. Think film filters for photoshop, and about 90% of all audio plugins that emulate high end hardware, compressors, pedals, etc etc.