News for hackers. Stuff that matters.
Re:Digital Audio Workstation Front End Development Struggles by jupp0r on Tuesday May 16, 2023 @01:53AM (#35958145)
It's kind of funny how the author dismisses web technologies while complaining about redundant rerenders in Rust GUI libraries. This is all solved in browsers, both on the rendering engine and on the web framework library levels.

Yes you need to pay attention to performance, but the premise that "web is slow" has been disproven so many times already. Just because it's really easy to write apps that are slow (and thus these apps being ubiquitous) doesn't mean that you can't write performant apps if you are so motivated.

Re:Digital Audio Workstation Front End Development Struggles by geodel on Tuesday May 16, 2023 @02:05AM (#35958218)
> but the premise that "web is slow" has been disproven so many times already.

No it is not. Though it seems proven that even with very powerful hardware users in general have been pushed to minimal expectations. So GUIs that take seconds to start or respond on multi Ghz processors and multi GB RAMs are considered fast now.

This argument is web GUI is fast is more of "It's been proven multiple times meal kits are cheaper than buying groceries from store". Yeah, someone can find really cheap meal kits if they spend many hours on hunting a deal. But in general case grocery in store is plain cheaper for most of reasonable shoppers.

Re:Digital Audio Workstation Front End Development Struggles by jupp0r on Tuesday May 16, 2023 @04:06AM (#35958908)
Define fast. Do you care about frame rates? input latency? startup times? memory usage?

There are trade offs, but if you care about high frame rates and low input latency, web apps can totally deliver that.

Look at https://lighttracer.org/app.html, for example.

Re:Digital Audio Workstation Front End Development Struggles by zigzag312 on Tuesday May 16, 2023 @08:32AM (#35960942)
I tried that app expecting responsive app, but the viewport lags significantly, and when rendering, the whole UI lags.
Re:Digital Audio Workstation Front End Development Struggles by jcelerier on Tuesday May 16, 2023 @01:00PM (#35964638)
I tried opening the link and did not even have the patience to wait until "compiling shaders" was done. When I saw the buttons slowly coming up one at a time I literally laughed. Is this supposed to be a remotely good example?
Re:Digital Audio Workstation Front End Development Struggles by jupp0r on Tuesday May 16, 2023 @08:22PM (#35969668)
I don't know, it was something that I randomly picked that was really performant on my phone. YMMV.
Re:Digital Audio Workstation Front End Development Struggles by andrewmcwatters on Tuesday May 16, 2023 @01:58AM (#35958178)
The amount of effort it takes to create high-performance applications in a web environment is significant, though, and the benefits you get simply working in a faster language in a desktop environment are lost on most web developers, I think, simply because most of them don't actually do any desktop or graphics API work.

They simply wouldn't know.

Also UI compositors are not a solved space. If they were, people would absolutely not being using Electron. But no one is saying, "Hey I have a CSS 2.1 compliant rasterizer and compositor that you can use in your C++ or Rust environment!" are they?

Re:Digital Audio Workstation Front End Development Struggles by rahkiin on Tuesday May 16, 2023 @02:02AM (#35958198)
Please, if someone has just that, let me know. It would be excellent for games. I do believe there are implementations of flexbox in rust.
Re:Digital Audio Workstation Front End Development Struggles by andrewmcwatters on Tuesday May 16, 2023 @02:05AM (#35958217)
Yeah, it's a problem space I'd like to try to tackle again some day. Yoga et al. aren't enough.

I think people are dying for a web-based compositor in other host language environments, but don't know how to elucidate this concept because they don't know anything else.

Re:Digital Audio Workstation Front End Development Struggles by suby on Tuesday May 16, 2023 @02:04AM (#35958211)
Right. The point of wanting to avoid redraws is for performance. Sure, you avoid redundant redraws in web browsers, but it defeats the point because the browser is slow compared to native, even with those avoided redraws.
Re:Digital Audio Workstation Front End Development Struggles by chrismorgan on Tuesday May 16, 2023 @03:04AM (#35958558)
> But no one is saying, "Hey I have a CSS 2.1 compliant rasterizer and compositor that you can use in your C++ or Rust environment!" are they?

There’s actually quite a lot of interesting work going on in that general space, has been in various forms for some years. A couple that immediately spring to mind:

• Azul <https://azul.rs/> builds on WebRender, as used in Firefox. I haven’t looked at it for a few years, but it looks to have grown quite interesting now.

• Blitz <https://github.com/DioxusLabs/blitz> is based on from-scratch implementations of CSS layout and rendering, and wgpu rendering. It’s not usable yet, but is a very interesting concept. If one happens to be familiar with React Native: it’s kinda like that, or React Native Web.

Re:Digital Audio Workstation Front End Development Struggles by ricardobayes on Tuesday May 16, 2023 @03:49AM (#35958799)
React Native Web is a bit like scratching your left ear with your right arm :)
Re:Digital Audio Workstation Front End Development Struggles by chrismorgan on Tuesday May 16, 2023 @04:06AM (#35958903)
Actually, forget the React Native Web thing, I think I was mixing it up with something from a few years back that was more the other way round (running React web stuff on React Native, not running React Native on web).

React Native Web… eh, it has a potential place. Maybe like scratching your left ear with your right arm, because your left arm is partially paralysed, or something!

Re:Digital Audio Workstation Front End Development Struggles by mike_hearn on Tuesday May 16, 2023 @11:36AM (#35963301)
Technically, you could ship a minimal link of a JVM and use JavaFX. It has a compositor, a CSS 2.1 rasterizer, layout managers and renders to D3D or OpenGL surfaces. I think they're adding Metal at the moment.

The weakness is interop with C/C++. There's a new Java FFI in the works that makes that a lot better by auto-generating bindings from header files and avoiding JNI, but it hasn't shipped as stable yet. It's usable. You just have to accept that the API might change a bit.

You can also compile the whole thing to a native library that doesn't use a typical JVM at all. Memory usage and startup time is a lot better if you do that. You can also expose functions natively to C, or bind to C natively, using a different approach.

If I were to do use that approach I'd write a generic binding layer that lets you connect JavaFX observables to their C++ or Rust-side data structures. FX has an architecture that's very based on observable variables, which all have a uniform API, so if you can control those from the native side then you can do anything. Either updating state objects that are then in turn connected to UI objects, or directly updating UI.

Re:Digital Audio Workstation Front End Development Struggles by usernew on Tuesday May 16, 2023 @02:16AM (#35958274)
Alright, here's I want you to do. It'll be interesting and fun, you'll make something cool, it's free, and then you'll come back and look at your comment and reply with a "/s."

Download pirated copies of protools and melodyne, and a bunch of virtual instrument libraries from torrents. Download a midi file for a song you like, dump it into protools. Record yourself singing, right on the PC or headset mic - no need to hit the right notes. Dump the voice instrument and your voice recording into Melodyne and drag the voice to match the notes on the voalc tone track. Now set up a link from your Melodyne track into a protools track.

Then go wild, play around, apply filters, do fun stuff. You might get stuck in the fun for a week and it'll keep you awake late.

Once you're done, think about doing all that on a web page or a web page pretending to be a local app. Then reread your comment.

Oh by the way, you will need to install a new raw driver for your soundcard. You see, something like a 5ms audio latency you get from you normal windows sound driver, is too slow. Give some thought on how slow doing anything in JS is going to be, if a native hardware driver is too slow.

Re:Digital Audio Workstation Front End Development Struggles by jupp0r on Tuesday May 16, 2023 @03:57AM (#35958849)
Until last year, I worked on a team that built an electron app for video conferencing with a double digit million user base. We did audio processing (mixing, RNN noise reduction) via WebAudio in WASM. This is in production right now as far as I know and adds very little latency.

That being said, this is about GUI. No reason not to do as much native processing as your heart desires outside of your GUI layer.

Re:Digital Audio Workstation Front End Development Struggles by TheOtherHobbes on Tuesday May 16, 2023 @06:21AM (#35959812)
A DAW is going to be doing far more audio processing on tens or hundreds of channels simultaneously, while playing back some of the audio from disk, some from memory, some from live inputs, and also receiving, processing, and recording/playing MIDI events. And hosting both the audio and GUIs of third party plugins.
Re:Digital Audio Workstation Front End Development Struggles by jupp0r on Tuesday May 16, 2023 @08:18PM (#35969636)
That makes sense, but what does that have to do with how GUI is implemented?
Re:Digital Audio Workstation Front End Development Struggles by zigzag312 on Wednesday May 17, 2023 @04:52AM (#35972743)
DAW's GUI needs to remain responsive with minimal and consistent latency. If DAW users complained because of DPC latency introduced by Nvidia's video driver, you can imagine how critical low-latency requirements are.

DAW can use pretty much all of system resources (RAM, CPU, disk IO) during playback. If any other thread steals CPU for for too long during playback, like GC of an UI thread, this can introduce audio clicks/pops.

Video conferencing app has tons of network latency which helps to hide other latencies. That kind of latency would be unacceptable in a DAW. Video conferencing app also doesn't starve system of resources like DAW can, so it's UI can be more resource-intensive.

Re:Digital Audio Workstation Front End Development Struggles by akdor1154 on Tuesday May 16, 2023 @07:30AM (#35960317)
> This is in production right now as far as I know and adds very little latency.

How much? As GP said, > 5ms is instant disqualification. And that is for your whole app, not just the effects pipeline.

GP is right, go use a DAW in a serious setting for a while to build up an intuition of the needs of these users.

Re:Digital Audio Workstation Front End Development Struggles by jupp0r on Tuesday May 16, 2023 @08:20PM (#35969651)
How is audio processing relevant to how you build the GUI?
[dead]
Re:Digital Audio Workstation Front End Development Struggles by jupp0r on Tuesday May 16, 2023 @08:18PM (#35969633)
I have not heard a single argument why DAW software can not have a web GUI. Do you have any? I'm genuinely curious.
Re:Digital Audio Workstation Front End Development Struggles by EyeSayUntoThee on Tuesday May 16, 2023 @09:06PM (#35970060)
There have been various toy DAWs the have web GUIs. I've never seen a production-capable DAW with a web GUI.

I suspect that existing web-as-native-GUI frameworks make presumptions about threading that won't work where a realtime audio callback thread piping data to other audio processing threads and back is the real show, and where the GUI may need things like updated zoom levels dirtying many waveform drawings at once requiring recalculation of peak values in viewable windows to draw the little lines that make the waveform, among other things that may make simple partitioning of systems more challenging...

Re:Digital Audio Workstation Front End Development Struggles by throw_m239339 on Tuesday May 16, 2023 @09:03AM (#35961295)
> This is in production right now as far as I know and adds very little latency.

You're not running 20 different tracks with compression, reverb, chorus, delay + send/return tracks + sub mixes in your app. Your app has absolutely nothing to do with a DAW. You're not taking MIDI input or running virtual instruments either. Try to do that in a browser with a latency < 10ms like Cubase or Logic and see how it goes.

Again, your product has absolutely nothing to do with a DAW and its constraints, so don't try to claim it's an achievement of some sort.

Re:Digital Audio Workstation Front End Development Struggles by jupp0r on Tuesday May 16, 2023 @08:10PM (#35969573)
I'm not sure how any of this is relevant with regards to how you build your GUI. If you have special requirements that require you to do your audio processing in native code using low level system sound APIs then there is nothing preventing you from doing so while still building web based GUI on top of it.
Re:Digital Audio Workstation Front End Development Struggles by throw_m239339 on Thursday May 18, 2023 @12:19AM (#35984086)
I'm not sure how any of your video app doing some light sound processing is relevant to the performance required for a DAW at first place either.
Re:Digital Audio Workstation Front End Development Struggles by duped on Tuesday May 16, 2023 @10:31AM (#35962411)
> This is in production right now as far as I know and adds very little latency.

Latency in a DAW isn't just at a premium, it's configurable and needs to be reported and compensated for. That means if any audio processing adds latency, all other processing units must compensate with their own internal (or external) delays to avoid the signal going out of phase.

Re:Digital Audio Workstation Front End Development Struggles by jupp0r on Tuesday May 16, 2023 @08:19PM (#35969645)
What do you think happens with multiple audio and video streams in a video conferencing use case?

Apart from that, I don't see how any of this is relevant to how you build the GUI for a DAW.

Re:Digital Audio Workstation Front End Development Struggles by outsomnia on Tuesday May 16, 2023 @02:20AM (#35958297)
Yeah. The OP should spend a day or two making HTML5 prototypes for the key aspects and see if it's really true that it's "slow".

The UI is not the hot code, it's the audio mixing. So long as the audio mising is faultless, it doesn't matter too much if it takes an extra video frame to update the dB display.

Because there are thousands of people working to maintain the code in the browser stack every day that are on his side then.

Re:Digital Audio Workstation Front End Development Struggles by richrichardsson on Tuesday May 16, 2023 @03:14AM (#35958620)
I can assure you that users complain when the GUIs of DAWs start to run below 30fps, and really the expectation these days is that "performant" GUIs run at 60fps.
Re:Digital Audio Workstation Front End Development Struggles by outsomnia on Tuesday May 16, 2023 @04:39AM (#35959113)
I am suggesting confirming if it is actually "slow" for the "key aspects".

You and the OP are just assuming it can't do whatever it is at "30fps"... if you're right, it will only take ten minutes if it's so bad, or an hour or two if it's subtle, to actually confirm whatever the worst thing is.

The "extra frame" I was thinking of is graphics pipeline latency, not redraw time.

Re:Digital Audio Workstation Front End Development Struggles by richrichardsson on Wednesday May 17, 2023 @05:40AM (#35973111)
Well now you're assuming what I'm assuming, which is fair enough because I assumed by your comment that you were saying the UI responsiveness isn't really a concern in DAWs. It's definitely a secondary concern, but people are accustomed to buttery smooth UIs and complain when they're not.

That said, I'd argue that even latency is still a concern for things like meters. If they are noticeably out of sync with what you hear then users will again complain.

Re:Digital Audio Workstation Front End Development Struggles by lelanthran on Tuesday May 16, 2023 @04:33PM (#35967375)
> but the premise that "web is slow" has been disproven so many times already.

That's news to me. Do you have a link comparing local non-electron apps with local electron apps?

I mean, a gui chat app I wrote a while back using libpurple(or something with that name, could have been libpigeon), without even trying, started up and loaded all cached chats faster than I could measure at the time (under 10ms).

None of the current electron-based chat apps I use right now take less than 5s even on hardware (M1 MB) that is literally 15 years ahead.

Re:Digital Audio Workstation Front End Development Struggles by jupp0r on Tuesday May 16, 2023 @08:25PM (#35969702)
Is latency of loading the app really relevant in the digital audio workstation use case? (no). As long as you get good frame rates while the GUI is running and latency is reasonably low, startup performance is not that important.
Re:Digital Audio Workstation Front End Development Struggles by lelanthran on Wednesday May 17, 2023 @01:14AM (#35971506)
I was responding to this:

> but the premise that "web is slow" has been disproven so many times already

So, yeah, startup duration absolutely counts.

More to the point, even while using (for example) slack you still see +500ms latencies while doing things like typing

Native chat apps had instant (sub 10ms) responses.

But, you made the claim that web being slow has been disproven, which really is news to me, so I want to see benchmarks you are referring to.