News for hackers. Stuff that matters.
Re:Jazz – Apps with Distributed State by softfalcon on Monday October 07, 2024 @04:40PM (#41770794)
I'm more than a little worried about the following all being `Coming Soon`:

- Cursors & carets

- Two way sync to your DB

- Video presence & calls

    - both the Group and BinaryCoStreams
All of these are the key reasons I would be evaluating this framework to handle my data. All of these are not fully implemented yet.

It is these key topics of live reloading/updating data that make or break an app. In my opinion, if you haven't concretely solved these problems, you haven't really built a viable state framework for 2024.

I really like the patterns they've implemented though, looks a lot like the same framework I just built on top of MobX, websockets, and React for a recent project. They're headed in the right direction, but I'm not sure they realize how much more work they have to go before this is fully fleshed out.

Re:Jazz – Apps with Distributed State by theanzelm on Monday October 07, 2024 @04:47PM (#41770869)
It is already persisting and syncing data (fully encrypted) between the cloud and other users. You can think of Jazz as a distributed database itself.

The coming-soon badges are about interop with traditional systems and higher level features we will add later.

The fundamentals are solved and you can build full multiplayer, local-first apps with it.

Does that make sense?

Re:Jazz – Apps with Distributed State by aboodman on Monday October 07, 2024 @06:26PM (#41771761)
Does it store data persistently anywhere (on servers?). If so where?
Re:Jazz – Apps with Distributed State by KPGv2 on Monday October 07, 2024 @09:59PM (#41773125)
It looks like it stores it on Jazz Flex, a paid service.

Free tier is 1GB of storage, and it goes up from there.

Re:Jazz – Apps with Distributed State by theanzelm on Tuesday October 08, 2024 @03:02AM (#41774488)
The Jazz Mesh stores everything you sync through it in geographically distributed and redundant servers (with backups to S3)
Re:Jazz – Apps with Distributed State by softfalcon on Monday October 07, 2024 @10:28PM (#41773275)
That sounds a lot more promising!

Which brings in even more questions. What is the performance of these multiplayer experiences? Can I have 1000+ users all connected to the same chat session? What about 10,000?

(these numbers might seem high, but they're what I'm expected to deliver in my day-job)

Re:Jazz – Apps with Distributed State by theanzelm on Tuesday October 08, 2024 @04:24AM (#41774959)
Eventually, yes! The origin story of Jazz is that I did a lot of experiments and research to convince myself of satisfying performance characteristics. Right now, the pure-TypeScript implementation is not very optimised and I'm very intentionally optimising it as needed for early adopter apps instead of predicting where bottlenecks are. So if you start building a chat app with 10s of thousands of users interacting, we'll make sure that's possible by when you want to launch.

This lets us more easily prioritise features vs performance.

Re:Jazz – Apps with Distributed State by merlindru on Tuesday October 08, 2024 @04:35PM (#41781591)
this is amazing, keep up the great work. been wanting something like this forever
Re:Jazz – Apps with Distributed State by bilekas on Tuesday October 08, 2024 @04:48AM (#41775120)
Not to go off topic but isn't performance metrics your responsibility ? Those are small numbers in reality, but if you are not performing your own tests and trusting the maintainer/publisher instead then you're not doing your 'day-job' properly.
Re:Jazz – Apps with Distributed State by softfalcon on Tuesday October 08, 2024 @11:49AM (#41778694)
Yeah, I hear you. Which is why we assessed and designed our own platform for handling the above stated traffic (and much more) for multi-user real-time chat sessions.

The commenter seems to know a lot about Jazz, so I took the opportunity to ask further questions.

Re:Jazz – Apps with Distributed State by theanzelm on Tuesday October 08, 2024 @11:50AM (#41778715)
I made Jazz, that's why I know a lot about it ;) (I now have people helping me)

Let me know if you'd like to know anything else.

Re:Jazz – Apps with Distributed State by bilekas on Tuesday October 08, 2024 @08:26PM (#41783265)
I like the idea, I have some implementation questions but I’ll catch you on GitHub. Nice work so far though.
Re:Jazz – Apps with Distributed State by bilekas on Tuesday October 08, 2024 @08:22PM (#41783250)
Good stuff, and I hope I didn’t come across too rude but 10k parallel is not a problem for even basic setups to manage, it’s a problem solved many different ways some out of the box some not. Maybe your question was phrased wrong. But hit into 10kk parallel, there’s an interesting metric to discuss optimization.

I’m reminded of the quasi meme “You have designed an infinitely scalable service for 10 clients “

Re:Jazz – Apps with Distributed State by softfalcon on Wednesday October 09, 2024 @02:33AM (#41785107)
10k parallel users I agree, but 10k receiving all updates from the other 9999 and vice versa can become quite a complex problem to solve.

Especially within time (latency), monetary hosting cost, and accuracy constraints.

For example, an MMO town with 10,000 connected clients all viewing the same complex session state of all other users and entities in one town.

That’s a lot of room for error. Or am I missing something?

That scenario is complex to me and has remained complex to me despite my various times building it out for various products.

Re:Jazz – Apps with Distributed State by boffinAudio on Tuesday October 08, 2024 @04:19AM (#41774929)
What plans do you have for a third-party audit/review of your backend to verify the claims being made?

Its one thing to use this service on the basis of encryption claims - its another thing to have to clean up the mess from a forgotten API key being leaked somewhere... is there, therefore, a third party involved in an audit of Jazz?

Re:Jazz – Apps with Distributed State by theanzelm on Tuesday October 08, 2024 @04:22AM (#41774943)
First it's all open source so even without knowing that my paid service uses the same open source libraries, you can verify yourself that nothing leaves the client unencrypted.

Of course we will also do public audit(s) with security companies to make sure our cryptographic protocols are sound and implemented correctly.

Re:Jazz – Apps with Distributed State by boffinAudio on Tuesday October 08, 2024 @07:17AM (#41776071)
Thanks for the info. Of course, "its all open source" is a fair answer, but its also a dangerous one, since it shifts culpability for verification to anyone who has the time/resources to audit the open source components that you're using.

But, it also has to be stated, there is no guarantee that you're using only the open source components you've declared, which is why an impartial audit by a third party would be necessary before this project can be used to build products in some industries.

Anyway, I see that you have other concerns, so no worries and thanks for the honest answer.

Re:Jazz – Apps with Distributed State by theanzelm on Tuesday October 08, 2024 @07:40AM (#41776228)
Just making sure that you saw the point where we will do an impartial third party audit!
Re:Jazz – Apps with Distributed State by boffinAudio on Tuesday October 08, 2024 @08:16AM (#41776483)
I did, and I hope you will promote that fact a bit more overtly, because its kind of important for some .. especially in Europe ..
Re:Jazz – Apps with Distributed State by jimmywetnips on Monday October 07, 2024 @05:32PM (#41771287)
Seems like you have lots of real world mobx experience.

Have you ever read this article, basically saying that every front end state (including mobx) basically ends up being a worse version of a standard database?

https://sqlsync.dev/posts/stop-building-databases/

I ended up finding that article after running into lots of the challenges with mobx State tree. I ended up trying to use watermelondb, a sqlite wrapped for react native, but gave up entirely on offline due to bugs and project abandonment

Re:Jazz – Apps with Distributed State by yard2010 on Monday October 07, 2024 @07:33PM (#41772261)
Not GP but isn't it like saying that every database ends up being a worse version of a standard filesystem?
Re:Jazz – Apps with Distributed State by desdenova on Monday October 07, 2024 @09:45PM (#41773025)
Every filesystem ends up being a worse version of writing to the disk by hand with a magnetized needle.
Re:Jazz – Apps with Distributed State by enugu on Tuesday October 08, 2024 @04:49AM (#41775126)
The analogy is more like an easier to implement version of a system(Custom Frontend Data Store) is worse than the standardized system (Database) rather than one system(database) being implemented on top of another system(filesystem).

The database has more features relative to filesystem, so we wouldn't miss the filesystem whereas a powerful indexing system is a feature in database missing in frontend data stores.

Some other examples below(Firefox plugins which are file browsers, FTP clients, C programs informally implementing subset of Common LISP features).

https://en.wikipedia.org/wiki/Inner-platform_effect

https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule

https://wiki.c2.com/?GreencoddsTenthRuleOfProgramming

Re:Jazz – Apps with Distributed State by softfalcon on Monday October 07, 2024 @10:34PM (#41773316)
Yes, I have read and found similar articles to what you've posted!

It is an ongoing exploration for us to find the best way to store/consume reactive state data within our client sites/apps. So far, MobX has given us a lot of leeway since we can create the data store we want for our data.

Going forward, I foresee us switching to something closer to WatermelonDB or maybe even just SQLite with a thin wrapper in React to create observability. I'm not 100% sure where it's going to go, but I agree with you that flexible state management for large platforms evolves fast and the difficulty of creating fast look-ups rapidly approaches "reinventing the DB" client-side.

I've had to build similar on top of SQLite for various other mobile/desktop apps, the problem hasn't really changed, the only difference is we're now using React + JS instead of C++, C#, or Java.

All roads so far have led to SQLite though.

Re:Jazz – Apps with Distributed State by svilen_dobrev on Tuesday October 08, 2024 @02:00AM (#41774219)
i have built similar thing long time ago, using couchdb as replication-storage mechanism, and it's variants on non-server platforms - touchdb/android/ios <-> pouchdb/js-browser etc , but yes, except serverside, all other platforms still had sqlite underneath.

Long live sqlite :)

Re:Jazz – Apps with Distributed State by bhl on Tuesday October 08, 2024 @02:09AM (#41774255)
I've done the same with mobx-state-tree, and realized I was essentially building a relational database with cache-ing in JavaScript.

The only thing SQLite is missed, which is important when integrating with React, is incremental view maintenance: how do we maintain as much immutable state on an update as possible to avoid wasteful re-renders?

Re:Jazz – Apps with Distributed State by whizzter on Tuesday October 08, 2024 @09:29AM (#41777160)
I would retort with, stop trying to chuck a SQL database everywhere.

The issue imho is that multiplayer synchronization (like Jazz) should rely on primitives(CRDT or similar) that you really can't hide under abstractions but should be part of the datamodel to allow for custom synchronization protocols for different parts of the datamodel or even exposing synchronization faults to the user.

An analogy would perhaps be how wrongly abstracted systems like CORBA and DCOM lost out (Even if you can run into them in crevices of enterprise systems) to things centered around HTTP calls or simpler message oriented systems like gRPC.

Re:Jazz – Apps with Distributed State by bbor on Monday October 07, 2024 @08:15PM (#41772525)
Plus the sync only works between clients AFAICT — literally all the listed backends are also “coming soon”. Which is troubling, I’d guess that almost all local-first applications need some sort of trusted verification server, at the very least. But maybe I’m biased/unimaginative; I suppose some kinds of collaboration apps (figma, google docs, etc) might work fine serverless? Certainly not games or AI agents, right?

This seems awesome, but my natural game-stopping questions are;

1. When will backends be supported?

2. Why doesn’t the site mention “local-first”?

Oh and just for fun;

3. Why “Collaborative Values” instead of what I learned in school, “Shared Memory”?

Re:Jazz – Apps with Distributed State by theanzelm on Tuesday October 08, 2024 @03:01AM (#41774480)
1. The sync is not just between clients, it by default works over a central backend infrastructure that syncs and persists. Plus you can create server workers (with jazz-nodejs) that also are clients to the same infrastructure and can do side-effects like API calls and interacting with “normal” databases in response to react state changing. And they can also mutate Jazz state.

2. The next version will. I like to think of Jazz as distributed state, of which local-first is a special case.

3. Shared memory to me sounds like concurrency is handled with locks, while Jazz uses CRDTs

Re:Jazz – Apps with Distributed State by armandososa on Tuesday October 08, 2024 @11:48AM (#41778669)
As a MobX user, now I'm curious about your Mobx + websockets framework.