Transformers.js: State-of-the-art Machine Learning for the web

Re:Transformers.js: State-of-the-art Machine Learning for th... by jsheard on Thursday April 11, 2024 @08:55AM (#40001582)

Basically the same problem that's plagued games on the web ever since the first Unreal/Unity asmjs demos a decade ago, and pretty much no progress has been made towards a solution in that time. You just can't practically make a web app which needs gigs of data on the client because there's no reliable way to make sure it stays cached for as long as the user wants it to, and as you say, even if you could reliably cache it the download and storage would still be duplicated per site using the same model due to browsers cache partitioning policies.

[ Parent | on HN ]

Re:Transformers.js: State-of-the-art Machine Learning for th... by jampekka on Thursday April 11, 2024 @09:18AM (#40001792)

There are ways to do it. E.g. https://developer.mozilla.org/en-US/docs/Web/API/File_System...

[ Parent | on HN ]

Re:Transformers.js: State-of-the-art Machine Learning for th... by pjmlp on Thursday April 11, 2024 @10:33AM (#40002630)

Actually more like the first Java Applets, Flash, and initially Unreal targeted Flash on the Web, and PNaCL, before asmjs came to be.

The Unreal 3 demo using Flash is still on YouTube.

And this is why most game studios are playing wait-and-see with streaming instead, proper native 3D APIs, with easier to debug tooling (Web still hasn't anything better than SpectorJS), and big size assets.

[ Parent | on HN ]

Re:Transformers.js: State-of-the-art Machine Learning for th... by binarymax on Thursday April 11, 2024 @09:53AM (#40002167)

Been beating this drum for years. Wrote this 9 years ago! https://max.io/articles/the-state-of-state-in-the-browser/

[ Parent | on HN ]

Re:Transformers.js: State-of-the-art Machine Learning for th... by fauigerzigerk on Thursday April 11, 2024 @09:20AM (#40001828)

The File System Access API seems promising.

[ Parent | on HN ]

Re:Transformers.js: State-of-the-art Machine Learning for th... by jsheard on Thursday April 11, 2024 @09:32AM (#40001956)

I'm not sure more APIs are the solution, LocalStorage could already theoretically fill the role of a persistent large data store if browsers didn't cap the storage at 5-10MB for UX reasons. Removing that cap would require user facing changes to allow them to manage the storage used by sites and clean it up it manually when it inevitably gets bloated. Any new API which lets sites save stuff on the client is going to have the same issue.

[ Parent | on HN ]

Re:Transformers.js: State-of-the-art Machine Learning for th... by fauigerzigerk on Thursday April 11, 2024 @09:58AM (#40002220)

>Any new API which lets sites save stuff on the client is going to have the same issue.

I don't think it would have the same issues, because the files could be stored in a user specified location outside the browser's own storage area.

Browser vendors can't just delete stuff that may be used by other software on a user's system. And they cannot put a cap on it either, because users can store whatever they like in those directories, bypassing the browser entirely.

But I have never used this API, so maybe I misunderstand how it's supposed to work.

[ Parent | on HN ]

Re:Transformers.js: State-of-the-art Machine Learning for th... by jsheard on Thursday April 11, 2024 @10:09AM (#40002337)

If that's how it works then it would avoid the problem I mentioned, but the UX around using that to cache data internal to the site implementation sounds pretty terrible. You click on "Listen to this article" on a webpage and it opens a file chooser expecting you to open t2s-hq-fast-en+pl.model if you already have it? Users won't be able to make any sense of that.

[ Parent | on HN ]

Re:Transformers.js: State-of-the-art Machine Learning for th... by fauigerzigerk on Thursday April 11, 2024 @10:40AM (#40002719)

The API (or at least the Chrome implementation) appears to be unfinished, but the plan seems to be to eventually support persistent directory permissions.

So the web app could ask the user to pick a directory for model storage and henceforth store and load models from there without further interaction.

[ Parent | on HN ]