Heck, as much as I don't like them, even Facebook got this right last I knew when I still used their app years ago. It never required a network connection. If you weren't actively receiving updates, it just showed you the cached last known feed, accepting that it wasn't up to date. And if you tried to post something, it would just cache that too and wait to send it. They didn't invent eventual consistency, either. It's been a basic operating principle of distributed organizations, especially armies, for thousands of years.
> I always end up duplicating almost all of the backend logic into the clients as well
Yep, this is a pain I feel acutely. I'm using dotnet (C#/F#) because it allows me to ship & run DLLs on the browser with Blazor, leading to significantly less duplicate code. F# can transpile to Javascript with Fable (F# + Babel), so that's also an option. I haven't fully vetted Blazor yet, but it seems good. Clojure can also run on the server and on the browser with ClojureScript.
The only other option I see is going fullstack Javascript, and I hate Javascript.
Yes it looks like event sourcing. But most of the classic event sourcing implementations I've seen are mostly backend only.
The project I'm currently hacking on has an MQTT broker that is used by both clients and backend and the events can come from anywhere. For example when a client sends a `location_updated` event, the backend reverse-geocodes this into an address and possibly sends out `place_entered` events, or sends out notifications for Tasks that are now relevant on the new location, or "completes" a "go to location X" task resulting in a `task_completed` event.
Enabling all this in an offline-first paradigm is hard and requires a lot of duplication, I am very close to just saying screw this and requiring network connectivity for most of the features.
you might like the architecture of holo-chain (stupid name, imo), it's still mostly going under the radar (also because they chose to refactor their library from go to rust to improve security):
"Holochain combines ideas from BitTorrent and Git, along with cryptographic signatures, peer validation, and gossip. Holochain is an open source framework for building fully distributed, peer-to-peer applications. [It's] purpose is to enable humans to interact with each other by mutual-consent to a shared set of rules, without relying on any authority to dictate or unilaterally change those rules. Peer-to-peer interaction means you own and control your data, with no intermediary"
good dev intro: https://medium.com/holochain/holochain-reinventing-applicati...
"Holochain is [...] like a decentralized Ruby on Rails, a toolkit that produces stand-alone programs but for serverless high-profile applications."
hmm no i think it does. the 'Resilience and availability' section is relevant for your question: https://developer.holochain.org/concepts/4_dht/
holochain is completely distributed, so there is no 'offline' and 'online' because there is no third party. it sounds weird writing that but i'm hoping it might challenge you to dig a bit deeper into the docs, because it's there. if not, please come back and let me know so i can pass on feedback about the docs that were unclear or unsatisfactory for you.
> does not solve conflict resolution
from the docs: "Holochain DNAs specify validation rules for every type of entry or link. This empowers agents to check the integrity of the data they see. When called upon to validate data, it allows them to identify corrupt peers and publish a warrant against them. [...] (the DNA is simply a collection of functions for creating, accessing, and validating data.)"
> All the DHT does is accumulate all these [updates] and present them to the application.
Yes, people will want conflict-resolution for contention on scarce resources, and it'd be lovely to have that baked into the system.
Two options (and yes, both still leave conflict resolution in the app dev's hands):
- Have the zome function resolve the conflict after it retrieves conflicting metadata from the DHT
- In the future we'll bake conflict resolution right into the network layer, so DHT authorities can resolve the conflict automatically using CRDTS or manually by pinging the nodes that published conflicting information so they can resolve it themselves.
It's also worth mentioning Syn, a library which uses operational transforms (a precursor to CRDTs) for conflict-free collaborative document editing. https://github.com/holochain/syn. I'm looking forward to seeing someone produce something similar, but with CRDTs.
How does it relate or differentiate with the GNU/Net project? What naming schemes does Holochain support and could it theoretically interop with the GNU Name System? [0]
so for all the people who might struggle with this (like my mom), but who still want to enjoy these new apps, Holo (again, stupid name, but this time it's the org who stewards the open source holochain library [1]) came up with a strategy that allows people to rent out some spare processing power on their computer/home-server, to host holochain applications for the above-mentioned people (so sort of like an Airbnb for AWS). they even ran a successful crowdfund that sold a million dollars worth of hardware for committed early adopters (plug-and-play home servers) [2].
so essentially the Holo hosting network is IPFS (and Holofuel is like Filecoin), but instead of hosting someone's files, you are running encrypted app code for them so that they can take part in the app/network, without having the required technical chops (the holofuel currency measures/represents processing power). the FAQ does a good job of explaining it a bit more: https://holo.host/faq/
i'm not super interested in the whole holo thing (they did a filecoin-type crowdfund to pre-sell hosting credits), yet i am glad they did it because it meant the team could ramp up development. they are already alpha-testing now.
about marx and engels. i am personally most excited about the potential for http://valueflo.ws on top of holochain (library called hREA) [3], because it will allow us to move away from today's Enterprise Resource Planning (ERP) software, into a new paradigm of Network Resource Planning (NRP) software. i hope it will have a big impact and enable the growth of the democratic and transparent supply chains of the next (socialist) economy.
> How does it relate or differentiate with the GNU/Net project? What naming schemes does Holochain support and could it theoretically interop with the GNU Name System? [0]
i'll take a look at it, i'm not too familiar with GNU/Net.
[1] https://github.com/holochain/holochain
[2] https://www.indiegogo.com/projects/holo-take-back-the-intern...
While it is true that you have to duplicate the mutations in the basic setup, you do not have to share the querying/reading code as it lives primarily on the client.
Also, if your backend happens to be javascript/typescript, then you can share the majority of the mutation code between client and server and the result is quite sweet.
My daily environment is mostly JVM-based (desktop/android/server) so your project is probably not a great fit for me but I'm definitely going to look into it for some inspiration.
I discovered this problem in my domain about two months ago. My solution was to split my Aggregate Root into smaller pieces, to use DDD terminology. In my domain, each user/client owns their own data. However they may choose to publish that data and have it be publicly visible so others may view/comment/copy/pull-request it. It's basically Github for flashcards. So, I have an Aggregate Root for publicly visible flashcards, and another Aggregate Root for a user's personal flashcards. It helps that there are distinct behaviors for each Aggregate Root - there's no real point to leaving a comment on a personal flashcard, and there's no real point to "studying" a public flashcard (because that implies logging your study history to the public card). It does mean, however, that there needs to be a translation layer - it should be possible to convert a private flashcard to a publicly visible one, and it should also be possible to copy a public flashcard to your own personal collection.
Obviously this is very domain specific.
It's based off something I call "source derivation"....
It has data sources where things get stored, repos that handle syncing and pulling data, and ents which represent the data but add functions to it.
Doing offline-first well implies that the app has a (sqlite or similar) local database, does work on that database and periodically syncs the changes to the backend. This means you have N+1 databases to keep in sync (with N=number of clients). Essentially distributed database replication where clients can go offline at any time. Potentially with different tech (for example sqlite on the client and postgres on the backend).
When the backend isn't very smart it's not too hard, you can just encapsulate any state-changing action into an event, when offline process the events locally + cache the chain of events, when online send the chain to the backend and have the backend apply the events one by one. Periodically sync new events from the backend and apply them locally to stay in sync. This is what classic Todo apps like OmniFocus do.
The problems start when the backend is smarter and also applies business logic that generates new events (for example enriching new entities with data from external systems, or adding new tasks to a timeline etc). Obviously the new server-generated events are only available later when the client comes back online.
When trying to make the offline experience as feature-rich as possible I always end up duplicating almost all of the backend logic into the clients as well. And in that case, what is even the point of having a smart backend.