rodocite

data is still stored somewhere but any routing to the data is disabled and the entity is disassociated.

if the references don't point to actual data, you don't need this.

- minimum 2 parts. a relay (reference hash) and the cold/true storage portion. you can break the reference hash up and reassemble only for secret holders. which brings us to:

- content-based routing

- there are also deterministic vaults for rolling keys

i'm actually not sure in what high-level situation "crypto-erasure" would work in because being able to re-key a reference means you have complete control. so why would you need to erase the "bad" key when you can just switch the reference?

Eth draft for enabling cold storage relay https://github.com/ethereum/EIPs/blob/master/EIPS/eip-1077.m...

Decentralized ID https://www.w3.org/TR/did-core/

^ both generally use the same concept i described above and solve GDPR "delete" issue. actually, it solves GDPR completely if you can just rely on the DID. "hard delete" is a separate issue, though-- no other way to get around that but to fork/version + replay your store and re-reference anyone who wants to hard delete if you didn't use reference hashes.

I don't quite understand this comment. Are you looking for a confirmation? Is it good enough to just have the most updated data for the state you want to track? I'm curious about what the specific use case was.

if you are waiting for ACK, you could alternatively create another stream that informs you of whatever write you were waiting for and therefore pushes you the most updated value or triggers the read. you can use the correlationID to make sure it's the set of changes that are bound to the original event you're tracking.

the "correctness" issue will crop up in whatever model you use since concurrency is the primary means for scaling a system. wrong order writes will have "correctness" problems with any db.

if you need strict ordering, the solution in ES will probably be the same as in pg-- separate the streams that need to be ordered and optimize them up front as much as you can. then post process when you no longer can't.

if data corruption is the issue, a changeset validation prior to the write probably works better than a rollback. if the validation is bad, catch it and send it to an exception stream you can track. event sourcing allows you to track by event/causation/correlation ids so you'll probably have an easier time debugging that.

I think the article is too scattered and doesn't actually discuss how event sourcing works. Key concepts are missing. People who actually want to learn it will end up getting even more confused or misguided-- which seems like the trend with this thing.

Excuse the list formatting. I don't post here that much. Just scroll if the list item is cut off.

TLDR; Use Redux-Saga-- it matches all the bird-eye view event sourcing concepts closely.

What is it?

  - Structure: In its simplest form, an append only file (aka a log).
  - Events -> { type, metadata } -> {type: "UserUpdated", data}
  - Projections -> The read model. Think of them as .reduce() over your log. But because all they need to do is look at the newest event to perform a reduction, they act as “realtime” queries for aggregated data.
  - Separation of read and write models aka CQRS
  - Read Model: Projection
  - Write Model: Event Dispatch
  - You can have CQRS without event sourcing (GraphQL, etc) but you cannot have Event Sourcing without CQRS. Event Sourcing is implicit CQRS.
  - Separation allows you to scale your reads and writes independently.
  - Event sourcing, since it is just a log, allows you to replay your data
  - For communication to a service that’s supposed to perform an action for you:
    - Command (dispatch) -> PresentTenseVerbNoun (UpdateUser)
    - Event (write) -> NounPastTenseVerb (UserUpdated)

The gist of what you do:

  - You dispatch events to the event store and create contextual "realtime" data via projections that your services read.

Why do you want to use it?

  - It scales and plays well with distributed infrastructure.
  - You are already using bits of the concepts if you are scaling or doing logging.
  - You have uniform communication between services.
  - Event sourcing is extremely good at modeling state in your system. You’re forced to think of state (via events) and how those events “eventually” resolve. If you’re purely on SQL, on the other hand, you need a log or triggers on top of your commits to keep track of state. Eg — Customer goes down the shopping isle and puts an XBOX in their cart. Then they decide to put it back and put a PS4 in their cart. If you had bound that behavior to events, you would be able to run complex projections on them.
  - ^ On that note, you essentially get free logging and metrics with Event Sourcing (though you need to build out the projections).
  - Event sourcing actually makes writing sequence diagrams to optimize or design your system very easy.

Why don’t you want to use it?

  - If it is overly “complex” for a small-mid sized CRUD project.

Misconceptions

  - Redux / Elm did not “popularize” event sourcing. There was a small snippet in the “Prior Art” section in the old Redux docs that mentioned event sourcing, but no one was thinking “event sourcing, yay!” as they were using Redux w/ Thunks.
  - Redux w/ Redux-Sagas is almost 1:1 the event sourcing model, however. If you want to learn event sourcing, instead of reading the article above, just learn how to use Redux-Sagas. Sagas, in event sourcing terms, is what is more generally known as a “Process Manager”.
  - Event Store DB vs PG DB — No need to go one or the other. Use the best of both worlds. ES for your event sourcing, PG for your read model and fully scoped write models.
  - “Event sourcing is so much more complex than using ORM” — No. The concepts are pretty standard whatever you use when you get into distributed systems modeling. Event sourcing is actually less complex but the tooling and verbiage we are used to is too highly focused on ORM, PG, etc.
  - Acid-compliance and eventual consistency are not mutually exclusive. Eventual consistency does not refer to the DB itself. It refers to the infrastructure. If your infrastructure is not brain splitting and is always “eventually consistent”, everything will be OK for most applications. There will be eventual consistency issues in any large scale system.
  - As soon as an event hits an event store, that event is "logged". It won’t be lost.
  - Streams in a proper event store are very cheap to create and not computationally expensive.

Things to Note

  - Blockchains are event sourcing implementations.
  - Read Smart Contracts are essentially “projections”
  - Write Smart Contracts are your write model, obviously
  - Blockchains replay “events”. That’s what syncing is with wallets. You can see that state changes.
  - The big structural difference between blockchains and a regular event store is a blockchain stores the events as a cryptographically verifiable log (the merkle tree).
  - The consensus algorithm in PUBLIC blockchains are also very different from your typical event store. Public blockchains use BFT consensus algorithm that’s very slow by design. Hyperledger, with its leader consensus model looks very similar to clustered event stores.
  - Redux-Saga matches the event sourcing flow and implementation so closely that it is probably the best place to start.
  - If you have really good modeling and event sourcing in place, you’ll start to see that Redux begins to disappear from your frontend; especially if you use GQL and caching.
  - Domain-driven Design really helps model an event sourcing infrastructure.

I'm pretty sure you can "soft-delete" for GDPR compliance. So this concern is sort of a non-issue.

Besides that:

1. If you're using an immutable structure, as long as you use references, you can obfuscate data. Blockchains ran into this problem before GDPR requirements and that's essentially all they do.

2. ^ "Update" strategy for event sourcing is the same as above. Essentially a copy of the log or the log slice then a re-indexing to remove/update streams, events, projections, etc. Greg Young talks about the indexing internals (for Event Store) in the 2012 video.