pryz

Jared: "We are sending out a proposal for Stacked Diffs on @GitHub to trusted design partners to gather initial feedback over the next few days. From there we’ll iterate and share the gameplan"

https://github.com/cloud-hypervisor/cloud-hypervisor or something else?

Love Zed! Would love it even more with Helix/Kakoune style binding :)

This is great! Are you considering opening a proposal for GORANDSEED? Could be really useful to generalize the approach!

> By embracing DynamoDB as your metadata layer, systems stand to gain a lot.

Yes yes yes. However, DynamoDB can be expensive very quickly :]

Like many developers, we've built our fair share of workflows that export data to 3rd-party services. They always start simple: pull data, hit an API, job done! Then the problems show up. We hit API limits, services go down, and those quick-and-dirty workflows become a major source of headaches.

The knee-jerk reaction is often to add a queue! Sure, it helps for a while. But queues introduce their own complexity: handling failures, managing retries, creating visibility... It's a band-aid, not a cure, and we've been wrestling with this problem for too long!

In this blog post, we'll break down:

- Why queues fall short when building truly resilient integrations - The core principles behind building scalable, fault-tolerant async workflows - Practical techniques that go beyond the limitations of queues

If you're done with fragile systems and want to level up your integration game, this one's for you!

Distributed coroutines are a perfect fit for Python! We're excited to explain how they work and what you can do with them!

Also happy to answer any questions :)

Hi HN,

We've had enough of traditional orchestration frameworks. That's why we created dispatch.run, aiming to streamline coding by integrating resilience more naturally.

The core of our solution? Distributed Coroutines. These aren't your typical tools; they're designed to enhance flexibility and reduce complexity in distributed systems.

We've detailed our approach and the potential of Distributed Coroutines in a new blog post. It's about making development smoother and more intuitive.

Let's discuss the future of distributed computing.

Except that way many many AWS services can't do ipv6.

The post from Polar Signal blog https://www.polarsignals.com/blog/posts/2023/12/13/embracing...

Exciting!

Forking Go means that you have to maintain a copy. That's a lot of work. Instead, the idea is to implement coroutine as a library and work toward an integration with the Go stblib.

https://research.swtch.com/coro might be an interesting read.

Congrats on the launch!

Continuous profiling is a game changer and once you use it on your production systems you can't do without it.

This is cool! Profiling interpreted languages is no easy task and it's nice to see eBPF leveraged for it.

The answer is: it's complicated. Which is most of the time the answer in the WASI world.

For this case it's complicated because some runtime supports https://github.com/WebAssembly/threads which mostly contains things like the spec for atomic but not the actual "threads" specs and then some runtimes (i.e wasmtime) also supports https://github.com/WebAssembly/wasi-threads which is one version of the threads. But a new proposal came into play https://github.com/abrown/thread-spawn so ... it's complicated.

The idea is that we are running few multi-tenant Kafka clusters to amortize the operation cost. These clusters host many topics with different topology depending on different requirements, cost profile and scale.

Some clusters are hosting hundreds of topics, thousands of partitions and used (consumers/producers) by hundreds of services.

I guess we just haven't got the need to use ACL yet. This may be something we will look later on. We have been started to implement what's needed to leverage things like SASL in our instrumentation but we are not there yet (see https://github.com/segmentio/kafka-go).

At Segment we manage clusters which are both on ECS and EKS. We didn't want to tie our Kafka management to the container orchestration.

If I'm right the Strimzi operator is also managing the brokers. Here we only focus on the topics.

Segment employee here.

Yes, we looked at Terraform to manage our topics. We are big Terraform user to manage our infrastructure. We mostly didn't go that way because we didn't want to have to manage the Terraform state. We also wanted the tool to be extensible and writing custom TF providers can sometime be tricky.