News for hackers. Stuff that matters.
Re:Skip the API, Ship Your Database by jokethrowaway on Wednesday September 13, 2023 @08:20PM (#37503363)
Imagine you're building a SaaS which allows your users to do create a website, hotel booking platform and channel manaager.

User can open the application, get the database, the frontend does all the offline updates the user want to perform. The user can update their website, add bookings they received on the phone, check what prices they set. This is all blazingly fast with no loading time, no matter your internet connection, because no further communication with the server is needed.

At some point they press a magic Publish button and the database gets synced with upstream and uploaded. The upstream service can take the data and publish a website for its users, update availabilities, update prices, etc.

It would be a better user experience than 99% of the SaaS out there.

Re:Skip the API, Ship Your Database by q7xvh97o2pDhNrh on Wednesday September 13, 2023 @08:42PM (#37503496)
Most of the Internet spent the last 10-20 years moving away from this because business metrics generally benefit from realtime updates and background autosaves. By and large, users expect systems to autosave their work-in-progress nowadays.

You might remember horror stories from the late 1900s when people would lose critical data and documents because they "forgot to hit Save and the computer crashed." Or, maybe you've never experienced this — because a couple decades of distributed-systems engineers and UX researchers made that problem obsolete.

So now we've... reinvented the Save button, but somehow needed a Paxos implementation to do it?

Everything old is new again, I guess.

Re:Skip the API, Ship Your Database by Merad on Wednesday September 13, 2023 @09:34PM (#37503860)
Heh, the last company I worked for had a (very popular, very successful) 20 year old desktop app that worked by downloading the client's entire database on login, then doing regular syncs with the back end and sending batch updates as the user made changes. It was an awful system that everyone hated, and the company was desperately trying to figure out how to move away from it without doing a complete rewrite. Maybe I should let them know that they're actually ahead of the curve if they can just hold out for a few more years...
Re:Skip the API, Ship Your Database by throwawaaarrgh on Wednesday September 13, 2023 @09:39PM (#37503899)
Hell, even with word processors we have auto save enabled by default for at least a decade.
Re:Skip the API, Ship Your Database by Freedom2 on Wednesday September 13, 2023 @10:46PM (#37504345)
I feel like I'm taking crazy pills. As HNers, we should want more control, not less over our work. Auto-save means that the company, no matter how nefarious, decides on their terms when my work is saved - regardless of what I've entered into the document (I write documents in a very stream of conscious manner).

I want to decide when to save, and how, not the application. I am not a product, I am a user!!!

Re:Skip the API, Ship Your Database by kristianp on Wednesday September 13, 2023 @10:50PM (#37504373)
This makes me smile, because it's satire, right?
Re:Skip the API, Ship Your Database by porker on Thursday September 14, 2023 @03:47AM (#37506133)
No. I value a meaningful "Last updated at" timestamp.

If I open a word document, redact a few bits, save to PDF and close it - I don't want my changes saved but they are (real scenario from last week. I'd only just moved the file into Onedrive so autosave had turned itself on)

Ditto sorting and filtering spreadsheets.

Software engineers: By all means save in the background so I never lose anything, but don't assume I want the latest changes.

Re:Skip the API, Ship Your Database by rmbyrro on Thursday September 14, 2023 @04:43AM (#37506436)
What you want, then, is versioning of your documents. MS Office and Google Suite do that.

This is not trivial to implement, though. Complexity will depend a lot of the application and the data. I'd say it'll only make sense for mature apps. A startup will most likely don't consider this in its roadmap.

Re:Skip the API, Ship Your Database by lukevp on Thursday September 14, 2023 @01:27AM (#37505306)
Seems like on-by-default auto save with the option to disable, and a separate save button, satisfy all your requirements. That’s a pretty common set of customizations on saves from what I’ve seen.
Re:Skip the API, Ship Your Database by vosper on Wednesday September 13, 2023 @08:39PM (#37503482)
Thanks for the example, it helped me understand the idea.

The thing that I'm unclear on is how do I figure out what data to ship to the user? Like if I don't already have a database-per-user then I have to extract only the data the user has permission to see, and ship that as a database?

That would be the case even if I had database-per-customer - not every user is necessarily able to see all the data owned by the organization they're a part of.

It seems like a lot of extra work, and error-prone, too (what could be worse than accidentally shipping the wrong data _as an operational database_ to a client?)

Edit: the article covers this at the bottom, but IMO it's a show-stopper. What application of any real complexity can actually implement database-per-user (database-per-tenant is probably not enough, as I mentioned above).

Re:Skip the API, Ship Your Database by hx8 on Wednesday September 13, 2023 @08:36PM (#37503453)
Once upon a time, this is how applications worked and this was a pretty good experience.

Now, you'll introduce a huge amount of user frustration around why their changes never made it to the central database. If you have users closing a form, especially if it's on a webpage, they are going to expect the write to happen at the same time the form closes. Making a bunch of changes and batching them in a single sync is going to be confusing to a ton of users.

Re:Skip the API, Ship Your Database by hamandcheese on Wednesday September 13, 2023 @08:43PM (#37503507)
What happens if two users want to make changes at the same time? What if their changes conflict? How do you even detect conflicts?
Re:Skip the API, Ship Your Database by Ambolia on Thursday September 14, 2023 @04:59AM (#37506540)
It could be done with "select for update" and etags+triggers checking the etag received + triggers generating a new etag on every create/update.
Re:Skip the API, Ship Your Database by mrbadguy on Thursday September 14, 2023 @05:28AM (#37506719)
If everyone's working on their own local copy of the database then the select for update isn't going to do anything. The issue is later syncing and detecting conflicts. It's actually easier to do this with a centralized DB, hence why everything works this way. If an app is mostly offline then, yeah, ship it with a SQLite DB and life will be good, but for something like a hotel booking app that doesn't actually solve many problems and makes existing ones harder.
Re:Skip the API, Ship Your Database by whalesalad on Wednesday September 13, 2023 @08:27PM (#37503404)
A lot of SPA’s already operate this by leveraging things like localstorage in the users browser.
Re:Skip the API, Ship Your Database by EMM_386 on Thursday September 14, 2023 @09:55AM (#37509076)
To do what? Isn't this limited to 5 MB?

I just checked a heavy user for our enterprise SPA. localStorage is using ~12 KB.

I don't know what people are just throwing in localStorage but they certainly aren't pulling down much of the database (which still would have to be checked to ensure cached data in there isn't stale).

Re:Skip the API, Ship Your Database by EMM_386 on Wednesday September 13, 2023 @08:47PM (#37503534)
> User can open the application, get the database, the frontend does all the offline updates the user want to perform.

"get the database"

How small do you think these databases are?!

You're going to download the entire hotel booking platform's database?

For how many hotels? One at a time, and then get another databse? Or are you getting a Sqlite booking database for every hotel in the world? And you're going to send them to each user? For what date range?

And even if that were possible, you then have to commit your offline updates. What if someone else booked the date range prior to you? Now your Sqlite copy is stale. Download the entire thing again? There could have been countless changes from other users in the time since you last got your copy.

This explanation just leaves me even more confused. It's illogical.

Re:Skip the API, Ship Your Database by layoric on Wednesday September 13, 2023 @10:20PM (#37504164)
Web applications aren't going to get bigger just by themselves! /s

Jokes aside, this extreme optimization for development does have impacts on user experience. The amount of bandwidth/storage etc used by a "just ship the whole db" type applications would surely suck for most people outside of the 'I measure my bandwidth in Gbps' crowd?

Re:Skip the API, Ship Your Database by yyyk on Wednesday September 13, 2023 @11:38PM (#37504700)
I have some experience with a comparable platform. In a typical CouchDB/PouchDB design, syncs are common and easy and it's pretty close to database-based design if you get fancy with views and javascript-based logic.

For this project, I'd do:

* A database for each user (this is natural on CouchDB, a setting one can enable to create the DB automatically. The platform won't choke on many databases) - for some per-user data, like comments, if the user has already reserved a booking we can mirror booking data there + some info regarding the hotel.

* Common synced databases - some general info regarding the platform and hotels. Preferably not too large (IIRC there's no limit to attachments with 3.x, but it sounds risky). Anything too large we'd do online or use data provided with the application.

* A large booking database which isn't synced and must be modified online with a REST call - we don't have to allow every action offline. This obviously needs the online component for making sure bookings don't conflict. Could even be a regular relational database. (Something like allowing a provisional offline reservation to the user database followed by a possible rejection on the server when syncing is doable, but I'm don't think there's much value to the user here).

It sounds very doable, rather similar to a project I was involved with, but I miss SQL a lot with that platform. A sqlite-based approach is an interesting alternative.

Re:Skip the API, Ship Your Database by throwaway290 on Thursday September 14, 2023 @02:25AM (#37505627)
You did not read the use case. It's not equivalent of Booking. More like equivalent of Wix for hotels.
Re:Skip the API, Ship Your Database by jokethrowaway on Thursday September 14, 2023 @12:36PM (#37511221)
Not the entire database, just your own data for your own hotel.

It doesn't sound that unreasonable.

Even if you have 3-4 hotels your data won't be significant.

Re:Skip the API, Ship Your Database by RhysU on Wednesday September 13, 2023 @10:08PM (#37504104)
Wasn't this Lotus Notes?
Re:Skip the API, Ship Your Database by Takennickname on Thursday September 14, 2023 @01:03AM (#37505164)
What is this? A scifi novel?

By the time your dude downloads the database half the hotels on your imaginary website have changed status.