News for hackers. Stuff that matters.
Re:SQLite in the browser with WASM/JS - OFFICIAL by modeless on Friday October 28, 2022 @05:07PM (#33377940)
This is good but it can never be as good as Web SQL could have been, because it can't be a truly shared library. Like all WASM modules, It has to be downloaded and JIT compiled separately for every site that uses it. Not for technical reasons, but privacy reasons, so it's basically unfixable: https://developer.chrome.com/en/blog/http-cache-partitioning...

Maybe if half the sites on the web start using it, browsers can finally be convinced that it would be OK to add SQLite to the base platform.

Re:SQLite in the browser with WASM/JS - OFFICIAL by samwillis on Friday October 28, 2022 @05:18PM (#33378057)
I disagree, this is much better than WebSQL. WebSQL would have been tied to one specific version of SQLite, and developers would have to work to the lowest supported version. There would have been no extension mechanism.

WASM SQLite is the correct solution. It’s extendable by the developer using it, they can uses SQLite extension modules, and build their own.

But almost more so it proves the idea of a WASM db engine backed by a low level block FS api. We will see other db engines uses this architecture. DuckDB have already done it. I’m sure MongoDBs Realm and CouchBase Mobile will do the same soon too.

We are in for an exciting time in the next few years.

Re:SQLite in the browser with WASM/JS - OFFICIAL by modeless on Friday October 28, 2022 @05:26PM (#33378136)
> WebSQL would have been tied to one specific version of SQLite, and developers would have to work to the lowest supported version. There would have been no extension mechanism.

Why not? Web SQL would not be different from WebGL/GLSL or even JavaScript itself in that respect. Developers use feature detection and and work to the lowest supported version. APIs and languages can evolve, but in (mostly) backwards compatible ways. Extension and versioning mechanisms can be made. That's how the web works and Web SQL could have worked that way too.

More broadly, you could apply arguments like this against everything in the entire web platform. Maybe everything should be a WASM module that developers could choose themselves! Image loading, video decoding, JS engine, why not? It's actually a beautiful vision and I'd be all for it if not for cache partitioning. Every site would have to re-download and re-JIT an entire browser engine before it could do anything.

Re:SQLite in the browser with WASM/JS - OFFICIAL by samwillis on Friday October 28, 2022 @05:34PM (#33378208)
WebGL/GLSL is a low level API the equivalent of the OPFS api. Standardising WebSQL would be like browsers standardising on Three.JS rather than WebGL/GLSL.

WebGL/GLSL give the developer a low level api to the graphics hardware.

Video decoding apis talk to the hardware video decoding hardware.

OPFS gives developers a low level API to the persistent file system / HDD / SSD.

Re:SQLite in the browser with WASM/JS - OFFICIAL by modeless on Friday October 28, 2022 @05:43PM (#33378315)
The web platform has tons of very high level stuff in it, much higher level than SQLite, and more is being added. Even specifically on the topic of 3D, they're trying to add a <model> tag and standard 3D model file format to HTML right now. It doesn't make sense to reject Web SQL, which is much more foundational, on the grounds of being too high level. Not now, but even less so back when the decision was made in 2010, when the web itself was all higher level and the lower level APIs you mentioned didn't even exist.
Re:SQLite in the browser with WASM/JS - OFFICIAL by nl on Saturday October 29, 2022 @02:08AM (#33381434)
> they're trying to add a <model> tag and standard 3D model file format to HTML right now.

To clarify, Apple made a proposal[0] which got no real traction. It has migrated[1] to a Javascript API[2]

[0] https://github.com/WebKit/explainers/tree/main/model

[1] https://www.w3.org/community/immersive-web/

[2] https://github.com/immersive-web/webxr

Re:SQLite in the browser with WASM/JS - OFFICIAL by modeless on Saturday October 29, 2022 @02:29AM (#33381531)
This is wrong. It moved to the immersive web CG but that doesn't mean it changed form to a JS API. It's just a venue change. The WebXR repo you linked is a different project entirely. The new location for <model> is here: https://immersive-web.github.io/model-element/
Re:SQLite in the browser with WASM/JS - OFFICIAL by haimez on Friday October 28, 2022 @06:48PM (#33378944)
All of these APIs are going to be very reliable fingerprinting opportunities, especially in combination. Keep that in mind when you think it’s going to be great for your web browser to also be a full featured application runtime.
Re:SQLite in the browser with WASM/JS - OFFICIAL by nl on Friday October 28, 2022 @11:19PM (#33380666)
Don't hold back the one open platform there is (the web) with fingerprinting concerns when its competition (mobile platforms) require an identity to use them.

The enemy of progress is perfect.

Re:SQLite in the browser with WASM/JS - OFFICIAL by haimez on Saturday October 29, 2022 @12:50AM (#33381085)
I’m supposing that the problem is that the web browser is the universal platform for all applications. There’s a benefit for information consumption (web pages) being separate from functionality rich and infinitely fingerprintable “native capabilities”
Re:SQLite in the browser with WASM/JS - OFFICIAL by nl on Saturday October 29, 2022 @02:02AM (#33381410)
I've been using the web since 1994. It's always been an application platform and anyone who says otherwise is misremembering.

> I’m supposing that the problem is that the web browser is the universal platform for all applications

This is a feature not a problem.

> There’s a benefit for information consumption (web pages) being separate from functionality rich and infinitely fingerprintable “native capabilities”

What exactly is that benefit supposed to be? If you want a read-only publishing platform, put PDFs on a FTP site.

Re:SQLite in the browser with WASM/JS - OFFICIAL by FerociousTimes on Friday October 28, 2022 @06:08PM (#33378589)
> Every site would have to re-download and re-JIT an entire browser engine before it could do anything.

Isn't this like a revival of the dreadful era of browser plug-ins from Adobe Flash Player to Java applets?

Re:SQLite in the browser with WASM/JS - OFFICIAL by masklinn on Friday October 28, 2022 @11:48PM (#33380793)
No.

The issue with flash and applets was not that you had to download stuff.

It was that the security was non-existent (leading to the embedded runtime routinely crashing your browser), the interactivity was divergent from its surroundings, and the accessibility model was MIA.

Also downloading 500K over 56k and over fiber or 5G are rather different propositions.

Re:SQLite in the browser with WASM/JS - OFFICIAL by ilyt on Friday October 28, 2022 @11:13PM (#33380633)
I'd still rather have basic SQL (that is enough for most) to just be there, for the apps where including SQL engine would be vastly more code than app itself.

If we had block API and SQL (it could be just a concrete version running from WASM itself, to not bloat browser itself) it could save a lot on app size. App then could look at browser version and decide to use that or to download newest one to run.

Re:SQLite in the browser with WASM/JS - OFFICIAL by sgbeal on Saturday October 29, 2022 @12:51AM (#33381088)
> WebSQL would have been tied to one specific version of SQLite, and developers would have to work to the lowest supported version. There would have been no extension mechanism.

Not only that, but WebSQL disables a lot of mundane functionality, like the instr() SQL function and it's impossible to VACUUM because WebSQL requires explicit transactions and VACUUM cannot run in a transaction. i had the "pleasure" of having to work with WebSQL over the past couple of months for purposes of comparing its performance to the new sqlite features, and IMO, WebSQL is, as the kids say today, _weak sauce_. Its API is far too limited.

Re:SQLite in the browser with WASM/JS - OFFICIAL by croes on Saturday October 29, 2022 @12:53AM (#33381097)
Unless frameworks and libraries start depending on it and each one of them uses different extensions.

All of a sudden you have multiple slightly different instances running just because there isn't a standard.

Re:SQLite in the browser with WASM/JS - OFFICIAL by yarg on Friday October 28, 2022 @05:37PM (#33378238)
Dependency resolution for the web would fix this.

It can be done, and needs to be - and more securely than the half-assed efforts of the likes of NPM and Maven.

All dependencies signed - let's encrypt has made this a viable option.

I don't see how there's anything approximating an unresolvable privacy concern here.

Re:SQLite in the browser with WASM/JS - OFFICIAL by coder543 on Friday October 28, 2022 @05:53PM (#33378425)
Signing dependencies is related to integrity, which is orthogonal to privacy.

Sharing a cache between websites has proven to be a privacy issue.

Read more here: https://developer.chrome.com/en/blog/http-cache-partitioning...

or here: https://www.peakhour.io/blog/cache-partitioning-firefox-chro...

Re:SQLite in the browser with WASM/JS - OFFICIAL by yarg on Friday October 28, 2022 @06:06PM (#33378577)
As per your second link:

> This means if you visited a website and it loaded the resource:

> https://www.somesite.com/foo.js

> and you then visited a second website, and it also included the same resource, then the resource would be loaded from the shared cache rather than being downloaded from the internet a second time. Cookies set by these resources would also be shared.

The privacy problem is not a result of the shared dependency, it's a result of the shared cookies.

Yes, if you share the execution space between multiple programs running the same lib, there's a privacy concern.

No shit - don't fucking do that.

Re:SQLite in the browser with WASM/JS - OFFICIAL by coder543 on Friday October 28, 2022 @06:10PM (#33378611)
> The privacy problem is not a result of the shared dependency, it's a result of the shared cookies.

No, it's not the result of the shared cookies. You just ignored all of the timing attacks and fingerprinting which a shared cache allows, as those articles discuss.

The cookie thing is honestly completely irrelevant to the topic of privacy, if you understand how the shared cache used to work. If you loaded the same library from separate CDNs on different websites, the shared cache didn't come into play at all. The library was loaded twice anyways. There was no chance for cookies from different CDNs to accidentally cross the streams. Browsers weren't attempting to heuristically determine if you were trying to load the same asset from different hosts.

The shared cache only came into play if you loaded the same asset from the same third-party CDN on multiple websites. The host serving an asset is the one responsible for setting the cookies, and sharing the cookies is helpful in that case, since the same CDN is the host serving the asset to the browser for both websites. These aren't cookies controlled by separate websites, they're cookies supplied by the CDN, and they should be the same for both requests anyways, outside of maybe some remote possibility of a theoretical attack involving a malicious CDN intentionally setting some weird request-dependent cookies, but I can't see how that would even do anything harmful anyways. So, the cookies get shared because the browser is serving a cached response for the same asset from the same host to both sites, which makes sense.

So, cookies aren't the problem here, and they're not the reason the shared cache was partitioned. If you want browsers to undo that in any form, you would have to solve the actual privacy problems here.

Re:SQLite in the browser with WASM/JS - OFFICIAL by yarg on Friday October 28, 2022 @06:31PM (#33378804)
Fingerprinting is possible if you know what's in the cache before you use it.

If I make a request for a given dependency, and am allowed to so much as time how long it takes to resolve, I can detect if it was already there and there's an information leak.

Sure. At some point though, a malicious site's gonna end up making some very weird requests - obviously polling the cache.

You could specify the dependency set in a static context and limit the ability of a site to measure how long dependency resolution takes.

Is there still an information leak? Yes.

Do I think it should stand in the way of a functional internet? Not really.

Re:SQLite in the browser with WASM/JS - OFFICIAL by coder543 on Friday October 28, 2022 @06:32PM (#33378807)
> Do I think it should stand in the way of a functional internet? Not really.

We have a functional internet. I'm using it to communicate right now.

Re:SQLite in the browser with WASM/JS - OFFICIAL by yarg on Friday October 28, 2022 @06:38PM (#33378850)
OK. But it's far from efficient, and I think that you're exaggerating the difficulty associated with improving the situation.
Re:SQLite in the browser with WASM/JS - OFFICIAL by coder543 on Friday October 28, 2022 @06:50PM (#33378954)
Google, Apple, and Mozilla apparently all failed to find a solution to this problem, even though lots of people wanted there to be a solution. Certainly, any browser that shipped this feature in a privacy-respecting manner would have bragging rights for awhile, so there is an incentive. Given that, I don't think I'm exaggerating the difficulty of the problem. This problem is also very similar to the challenges brought by Spectre-class vulnerabilities, and that one has been enormously costly for the whole industry.

I think it is completely fair to say that privacy-respecting shared caches are not simple.

Some solutions can be imagined, but they come with weird trade-offs or they do nothing for majority of the web. A new manifest format like you describe falls into the latter, since it would only apply to new websites using the new feature, and that’s without digging into the other problems it would pose.

In practice, people often visit the same websites repeatedly; they aren't constantly visiting new websites only once. A partitioned cache works just fine for the normal scenario. It's slightly less efficient for the first day someone uses their browser, but then things are honestly fine after that. It's unfortunate that we can't eek out the last tiny bit of performance for this, but I think the difference would be hard to measure in practice.

In my opinion, if websites would more commonly use brotli, that would make a far larger difference in efficiency than returning to a shared cache, and if browsers could have a standardized means of downloading only the bytes that changed in an asset like a javascript library instead of downloading the new version from scratch, that would make a much bigger difference too.

Re:SQLite in the browser with WASM/JS - OFFICIAL by wruza on Friday October 28, 2022 @08:30PM (#33379714)
I think it is completely fair to say that privacy-respecting shared caches are not simple.

Couldn’t they make an exception for some domains and create a registry of really popular or fundamental links to packages like jquery et al? I have read on this topic before, but it sounded like all or nothing no shades of grey maximalism. Fine, partition those memes from imgur cdns, but let common libraries with known hashes to be shared at least. The potential attack is based on leaving a cdn-pixel and dl-time-testing it on other sites. But there is no big data in who has the 10 most popular releases of wasm-sqlite, dayjs or bootstap.min.css in their cache. These could be warmed up from literally anywhere, or even synced in background by an idle browser thread.

Re:SQLite in the browser with WASM/JS - OFFICIAL by coder543 on Friday October 28, 2022 @09:54PM (#33380215)
I feel like Google Chrome shipped an experiment at one point that was going to include some of the most popular libraries with the browser, so they would be equally cached for all Chrome users, for all sites. I'm having trouble finding any announcements about this, so maybe I dreamed this up.
Re:SQLite in the browser with WASM/JS - OFFICIAL by thruflo on Friday October 28, 2022 @05:12PM (#33377989)
Interesting — so basically an overhead until it’s fully native? I was reading the Chrome initiative as a pathway to native. Would that mean it can’t be WASM?
Re:SQLite in the browser with WASM/JS - OFFICIAL by modeless on Friday October 28, 2022 @05:15PM (#33378017)
That tweet is not implying that it will ever be native. Quite the opposite; Web SQL is native today and they are going to remove it since it was rejected by other browsers.
Re:SQLite in the browser with WASM/JS - OFFICIAL by Beltalowda on Friday October 28, 2022 @06:15PM (#33378660)
> It has to be downloaded and JIT compiled separately for every site that uses it

I don't really see the problem with that. Looking at the sql.js demo[1] the WASM binary is 610K (305K compressed/transferred), and it seems to run pretty fast even on my slow laptop.

> Maybe if half the sites on the web start using it

Most websites have no reason to use it; simple key/value localStorage is enough for many sites or apps that need some sort of storage. It's kind of a niche thing. Many regular desktop applications have no need SQLite, either.

Re:SQLite in the browser with WASM/JS - OFFICIAL by wruza on Friday October 28, 2022 @08:12PM (#33379623)
It’s an ubiquitous need for any app that has a dynamic collection view. You can’t take all sites, see that 99% of them are static documents developed as degenerate apps and then conclude that “most apps” have no reason to use it. It’s like saying that most cats are on jpegs, so cat food is kind of a niche thing for a cat owner.
Re:SQLite in the browser with WASM/JS - OFFICIAL by Beltalowda on Friday October 28, 2022 @09:46PM (#33380162)
> “most apps”

Please do not put words in quotes as if I said them when, in fact, I did not. Thank you.

Re:SQLite in the browser with WASM/JS - OFFICIAL by wruza on Saturday October 29, 2022 @05:04AM (#33382299)
What would that trivial correction change, in your opinion?
Re:SQLite in the browser with WASM/JS - OFFICIAL by ilyt on Friday October 28, 2022 @11:15PM (#33380644)
> I don't really see the problem with that. Looking at the sql.js demo[1] the WASM binary is 610K (305K compressed/transferred), and it seems to run pretty fast even on my slow laptop.

...and that is small by your standards ?

Re:SQLite in the browser with WASM/JS - OFFICIAL by Beltalowda on Saturday October 29, 2022 @11:29AM (#33384879)
It's not small; it's also not huge. For the type of website that has use for this it's not really all that much.
Re:SQLite in the browser with WASM/JS - OFFICIAL by sgbeal on Saturday October 29, 2022 @12:53AM (#33381095)
> Most websites have no reason to use it; simple key/value localStorage is enough for many sites or apps that need some sort of storage.

In case you hadn't noticed it, the new sqlite features include using localStorage and sessionStorage as db backend storage :).

<https://sqlite.org/wasm/doc/trunk/persistence.md#kvvfs>