News for hackers. Stuff that matters.
Re:Ask HN: WebSocket data architecture strategies: idempoden... by bem on Wednesday April 28, 2021 @04:45AM (#26967154)
I think you could find some inspiration in cryptocurrency exchanges. Most of them expose public websockets for prices, order books, trades, etc. They're high volume with lots of subscribers.

For example, check out the docs for the Binance websockets (they use both snapshot and delta messages): https://github.com/binance/binance-spot-api-docs/blob/master...

If you could tell me a little more about the data format, data volumes, number of subscribers, and how you get the data on the backend, I can try to give you some more concrete advice.

Re:Ask HN: WebSocket data architecture strategies: idempoden... by aranchelk on Wednesday April 28, 2021 @03:30AM (#26966726)
If I understand the question: you don’t have to choose between idempotence and efficient incremental updates if you can give your application data a revision number. Send your incremental update(delta) with a reference to the app data’s last revision. Increment the revision number after applying each delta and reject any delta that doesn’t reference the current version on the server, they’ll only be applied at most once, so they’re idempotent.

There are fancier versions of this scheme that can be devised based on the business rules for your application.

Re:Ask HN: WebSocket data architecture strategies: idempoden... by 2upmedia on Wednesday April 28, 2021 @04:08AM (#26966946)
Happen to know of any places where I could find high level data architecture concepts like that? Or did you know that just by experience?
Re:Ask HN: WebSocket data architecture strategies: idempoden... by aranchelk on Wednesday April 28, 2021 @01:12PM (#26971738)
I’m searching now and I see multiple books on designing distributed systems. They’re probably worth looking at, but I haven’t read any of them so I can’t make a specific recommendation.

I’ve learned a lot from reading about the architecture of existing distributed systems e.g CouchDb, Git, and Bitcoin.

Re:Ask HN: WebSocket data architecture strategies: idempoden... by 2upmedia on Wednesday April 28, 2021 @03:21PM (#26973037)
Awesome. Thanks.
Re:Ask HN: WebSocket data architecture strategies: idempoden... by 2upmedia on Wednesday April 28, 2021 @04:01AM (#26966902)
Wow. Never heard of that one. Very useful thanks.
Re:Ask HN: WebSocket data architecture strategies: idempoden... by colesantiago on Tuesday April 27, 2021 @08:41PM (#26963993)
How are you currently doing this?

did you try going for a 3rd party provider like Firebase, Ably or Pubnub?

Re:Ask HN: WebSocket data architecture strategies: idempoden... by 2upmedia on Tuesday April 27, 2021 @10:16PM (#26964677)
Planning stages, so no 3rd parties yet. Considering strategies because I need live connections with as low latency as possible so I'm probably going to be managing the websocket connections myself on the server side, unless there's something else that will give me the same benefits and nearly no latency.