News for hackers. Stuff that matters.
Re:Introducing Workers KV by noelwelsh on Friday September 28, 2018 @10:13AM (#18093496)
Holy return of tuple spaces! Are there any features for controlling consistency, or is it just YOLO last write wins?
Re:Introducing Workers KV by steego on Friday September 28, 2018 @10:35AM (#18093694)
I had to upvote you just because I plan on getting some mileage out the phrase "YOLO last write wins." It has just the right amount of snark to remind junior developers to not trivialize or discount the value of consistency.
Re:Introducing Workers KV by biesnecker on Friday September 28, 2018 @03:44PM (#18096436)
And it's really "YOLO last write recorded wins," because who knows what order anything arrives in.
Re:Introducing Workers KV by kjeetgill on Friday September 28, 2018 @10:57PM (#18098459)
"yolo a latter write probably wins ... I guess?"

Seriously, if you're new to that domain go read:

https://aphyr.com/posts/333-serializability-linearizability-...

https://aphyr.com/posts/313-strong-consistency-models

Re:Introducing Workers KV by kloepper on Friday September 28, 2018 @10:23AM (#18093580)
Last write wins. There are several features for controlling consistency that we have been prototyping. Based on how storage is used we will enable those as the product matures.
Re:Introducing Workers KV by zzzcpan on Friday September 28, 2018 @10:30AM (#18093642)
So why not strong eventual consistency and CRDTs? As they fit perfectly for such functions running on edge nodes and no silly limitations of 1 write per second necessary.
Re:Introducing Workers KV by kentonv on Friday September 28, 2018 @10:35AM (#18093685)
Indeed. This is just stage 1 of our storage plans. We have some really cool stuff in the works but we wanted to get basic KV functionality out there for people to start using ASAP.
Re:Introducing Workers KV by _pmf_ on Friday September 28, 2018 @03:02PM (#18096100)
OT: we need to read more about your website-in-kv-thingy setup.
Re:Introducing Workers KV by kentonv on Friday September 28, 2018 @04:30PM (#18096787)
It was pretty trivial. I just stored each page into a value and served them out of there, with some stupid code to infer content-type from file extension. I meant it as a dumb demo because I wasn't feeling creative. I didn't know John would mention it in his blog. :)

Currently this isn't a great way to serve content because the values are limited to 64k, you have to infer content-type (which makes your URLs ugly), etc. So I don't recommend it. We'll come up with something better in the future. :)

Re:Introducing Workers KV by kloepper on Friday September 28, 2018 @10:35AM (#18093689)
Yes, that is one of the models we have been considering.
Re:Introducing Workers KV by falcolas on Friday September 28, 2018 @11:02AM (#18093922)
So, with a global consistency window of 10s, and 1 write per second, would consistency ever be reached if a key is written to every second? What value would be visible?

One great example of why this would matter to me - could workers ever institute a coordinated rate limiting function using this KV store?

Re:Introducing Workers KV by zackbloom on Friday September 28, 2018 @11:21AM (#18094110)
One thing to remember is you have at least two other ways of storing values. Each worker has global memory which persists between invocations (local, atomic), and you have access to the Cache API (per-data center, ephemeral).

By combining those with KV to do global coordination on a more course time scale, I think you could.

Re:Introducing Workers KV by zzzcpan on Friday September 28, 2018 @11:11AM (#18094002)
Rate limiting doesn't have to be coordinated to work. But you do need a PN-Counter, which they are only considering at this point.
Re:Introducing Workers KV by falcolas on Friday September 28, 2018 @11:22AM (#18094117)
A PN-Counter is a method of coordination. And if the writes were coordinated with the consistency time, that would be sufficient to implement consistent counters (or even a naive counter in a store given atomic operatons).
Re:Introducing Workers KV by zzzcpan on Friday September 28, 2018 @12:41PM (#18094773)
PN-Counter is coordination-free counter, it can't be used as a method of coordination. But it's ok for global rate limiting, as it doesn't have to guarantee exact rate at any point in time, as long as it eventually averages on specified rate.
Re:Introducing Workers KV by zackmorris on Friday September 28, 2018 @05:29PM (#18097214)
For anyone curious about this, it probably all comes down to an atomic compare and swap request:

https://en.wikipedia.org/wiki/Compare-and-swap

It might be straightforward to do this with an ETag like Firebase:

https://firebase.googleblog.com/2017/07/introducing-conditio...

Not sure what this would require on their backend though.

Re:Introducing Workers KV by kentonv on Saturday September 29, 2018 @01:49PM (#18101720)
Yes, you'll be able to use etags with if-match for optimistic concurrency. When reading and writing from a Worker, if you do a get() followed by a put() on the same key in the context of the same request, we'll take care of the if-match automatically under the hood, so the put() fails if there was a concurrent write.

(This logic is not quite in production yet but should be within a week or two.)

Re:Introducing Workers KV by panic on Friday September 28, 2018 @03:52PM (#18096495)
Can you implement IPC with this system? It seems wrong to relate this to tuple spaces otherwise.
Re:Introducing Workers KV by _pmf_ on Tuesday October 02, 2018 @06:16AM (#18120233)
A poor man's IPC, like HTTP and HTML is a poor man's application platform ...
Re:Introducing Workers KV by zackbloom on Friday September 28, 2018 @05:20PM (#18097158)
Put the target channel name and a message id at the front of each key and you can use a tuple space for IPC.
Re:Introducing Workers KV by panic on Friday September 28, 2018 @07:35PM (#18097849)
What I mean is: can a process block until a particular key appears, then resume execution after receiving the associated value? That's the main differentiating feature of a tuple space, and it wasn't clear to me whether this system supports that or not (or even has a concept of a process).
Re:Introducing Workers KV by kentonv on Saturday September 29, 2018 @01:53PM (#18101737)
No, that's not supported at present, but that's an interesting idea.

Workers don't have the concept of processes. The unit of compute is an individual event -- e.g. handling an individual HTTP request. So we'd have to augment it a bit, but we can probably come up with something useful.