trebecks

yeah i like this model. then you can use whatever synchronization junk you prefer to share state between the threads, if you have any.

your library looks well written and clean. thanks for sharing.

if anyone else wants to go coroutine spelunking, these were interesting to me:

https://github.com/higan-emu/libco/tree/master

https://github.com/Tencent/libco

https://github.com/hnes/libaco

https://kernel.googlesource.com/pub/scm/virt/kvm/qemu-kvm/+/...

https://tia.mat.br/posts/2012/09/29/asynchronous_i_o_in_c_wi...

https://www.cs.uml.edu/~bill/cs516/context_paper_rse-pmt.pdf

the scheduler is probably simpler if coroutines can't bounce between cores. you can have a single thread per core that runs the scheduler to multiplex a bunch of coroutines on the single thread, which lines up with the example linked in the repo to a redis clone. redis runs a single thread (i think technically it has some multithreading stuff now but that was the model for a while) but can concurrently process a bunch of requests like when 1 or more requests are blocked on blpop, a pubsub thing, xread with block arg given, etc. nginx does something similar with forking a process per core that runs single threaded (ignore the threading for stuff that reads from disk).

i got caught up on that statement too. i interpreted it as they can spin up more servers with their own volumes but thats not really 1 ebs volume anymore. maybe at their current load they only need 1? the op mentioned copying stuff to s3 at a certain size so it sounds like the disk isn't very big at any moment. i don't think there would be much to do if another server joins.

> Amazon EBS offers a higher durability volume (io2 Block Express), that is designed to provide 99.999% durability with an annual failure rate (AFR) of 0.001%, where failure refers to a complete or partial loss of the volume.

if they take snapshots often enough to feel comfortable with that low failure rate, it does seem kind of reasonable to me. really low risk of a given volume failing.

if i'm reading the op right, they kind of use ebs as a buffer for fresh data until it ages out to s3. they use a "local" disk to hold the stuff used by the queries that people actually make and the queries run quick. they let the old stuff rot in s3 where its almost never used. that sounds like a good idea to save money plus the stuff that's done often is fast.

the ebs slas look reasonable to a non expert like me and you can take snapshots. it sound like you need to be careful when snapshotting to avoid inconsistencies if stuff is only partially flushed to disk. so you'd need to pause io while it snapshots if those inconsistencies matter. that sounds bad and would encourage you to take less frequent snapshots...? you also pay for the snapshot storage but i guess you wouldn't need to keep many. i like that aws defines "SnapshotAPIUnits" to describe how you get charged for the api calls.

with aurora, it looks like you can synchronously replicate to a secondary (or multiple secondaries) across azs in a single region. it sounds nice to have a sync copy of stuff that people are using. op says the'yre ok with a few seconds of data loss so i'm wondering how painful losing a volume right before taking a snapshot would be.

i wonder if anything off the shelf does something similar. it sounds like people are suggesting clickhouse. i saw buffer table in their docs and it sounds similar https://clickhouse.com/docs/en/engines/table-engines/special.... it looks like it has stuff to use s3 as cold storage too. i even see geo types and functions in the docs. i've never used clickhouse so i don't know if i'm understanding what i read, but it sounds like you could do something similar to whats described in the post with clickhouse if the existing geo types + functions work and you are too lazy to roll something yourself.

yeesh i didnt realize there was a patent on that. your link lines up with the authors predictions of cdns coming up with and patenting different ways to accomplish this. i guess it makes sense and maybe it isn't a bad thing, but i never thought about it. i wonder how anyone infringing on that is even caught? is something like that ever enforced?

edit: i was blind and missed that it is marked as abandoned "ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION". i never look at these so maybe thats a normal ending, but it is interesting to me.

this guy sounds uhhh exactly the same https://patents.google.com/patent/US11706292B1/en...? reading these brings up the duration of a connection as a relevant dimension. in my bubble i've only dealt with relatively short lived conns. long lived conns must be more likely to get caught up in routing changes so a redirect to unicast can help.

the "Stupid DNS Tricks" section says using dns to map clients to a nearby pop is a trick. they predicted this trick would be used for decades and it seems like they were right. i know cloudfront uses this in some fancy form.

i don't know if i'd call it a trick though. if you have multiple pops, dns feels like a natural place to control what traffic goes to which pop. you will need resolvers to be well behaved which will never be the case. not all will respect ttls and use the client subnet extension but a lot do. dns gives you a nice knob and hooks to apply rules to control the traffic to each pop. this paper i think describes the idea well https://www.sigcomm.org/sites/default/files/ccr/papers/2015/....

if using dns like this is a trick, what is the right way to map a client to 1 of multiple pops? anycast?

the SHED_LOAD thing offered by ns1 looks cool. anyone have experience with it? https://help.ns1.com/hc/en-us/articles/360020683413-Configur.... seems like something juicy missing from route53. ns1 even has an integration with cloudwatch (not to say cloudwatch is great) as a source for the load stats.

it seems obvious at the level of like an individual load balancer to consider the load of the nodes backing an upstream when it decides which one to send reqs to. i didnt think of lifting that to the record that points to the load balancers too.

some nyc bridges and tunnels charge both ways. for the bridges and tunnels connecting ny and nj, you only need to pay going nj -> ny.

i use https://github.com/giltene/wrk2 pretty regularly.

it has decent lua hooks to customize behavior but i use it in the dumbest way possible to hammer a server at a fixed rate with the same payload over and over.

i run it by hand after a big change to the server to make sure nothing obviously regressed. i used to run it nightly in a jenkins job but 99% of the time no one looked at results. it was nice to see if assumptions on load a single node could handle didn't hold anymore.

flink uses akka for control plane kind of stuff. data plane stuff, like piping data between tasks, doesn't. check out https://flink.apache.org/2019/06/05/flink-network-stack.html for some (maybe outdated) details.

nginx too http://nginx.org/en/docs/stream/ngx_stream_proxy_module.html

yeah, that is what refresh tokens are used for and it works well for access tokens with a short ttl. in practice i have seen access tokens with ttl of a few hours. i think it might be done to cut down on the refresh calls? in that case, if you want to deny requests with a revoked token sooner than the potentially few hours it takes for it to expire, you could keep the set of revoked tokens server side.

that feels like a weird place to be in with server side state for stuff that is supposed to be stateless. i guess the revoked set is smaller than the full set of active tokens so maybe that is a win? curious if anyone has experience with this.

the user isn't signing anything. the server is signing something it sends to the user. when the user sends it back to the server, the server checks the sig to make sure the user didn't alter/forge it.