afc

+1. This is a very real problem in practice.

A technique we've used to deal with this situation: in the overloaded backend (that has to reject some percentage of incoming requests), group the incoming requests by the parent request (the one with the 1:15 fan out) and reject according to the parent request. One way to put it, simply (though somewhat inaccurately), would be: reject 100% of traffic from 20% of users, rather than 20% of traffic across all users (causing essentially full failure for all users).

We typically implemented this by propagating an ID of the parent request down to the backend. I'm simplifying a lot in this description (e.g. have to deal with the parent requests landing on different backend tasks; also rotate the IDs gradually to introduce some fairness).

Haha, tell me about it! I spent more than a decade mostly just deploying these systems across just one company. It's far from trivial!

Google's (or any similar large company's, really) efficiency requirements would never allow for such levels of waste. It would be incredibly irresponsible to allow just any random small server to unexpectedly use the entirety (or even just a non-trivial percentage) of a Google-scale datacenter's capacity. Do you evict other jobs? Do you run your datacenters cold to prepare for this eventuality? Both are terrible options.

Suddenly you push an update to your tiny server or one of its thousands of clients and, because of some bug or misconfiguration, traffic or per-request-resource-consumption grows 1000X or more. Given the size of the looong tail of small servers at Google, this would be a daily occurrence.

> Both incidents were capacity failures at their core. We failed to scale critical components before demand exceeded their capacity.

This is the wrong way to think about this because there's no such thing as infinite capacity. A large distributed system will be simultaneously mostly idle and (in some subcomponents) overloaded. The root cause is not "a component didn't have enough capacity (because of auto scaling failures)", but rather "this complex system collapses (rather than degrade gracefully) when demand exceeds capacity".

When components reach capacity limits, the excess traffic of the lowest priority should be rejected. Rejected traffic should not be retried — in fact, not only should clients not retry these errors, these errors should cause client-side throttling. Traffic isolation should be applied — if the cause of the overload is a single client/customer system, no other system should be affected.

Nearly a decade ago I wrote about some of the techniques we applied at Google to implement these protections: https://sre.google/sre-book/handling-overload/ Most other large internet services have since copied them, afaik.

+1. I also felt this huge gap between knowing (intellectually) what parenting is and actually experiencing it. It really changes you, to the point that now, when I talk to childless people my age, I feel like we're fundamentally differently. I think I can perceive fairly accurately whether someone my age is a parent after just a few minutes of general conversation (obviously, without having to visit adjacent topics).

I experienced this so strongly I spent some time actually writing a poem (in Spanish) about it: https://alejo.ch/39l

Interesting perspective. Your first paragraph also describes my experience, perfectly.

However, even though I also think nothing comes even close to parenthood for me, my perspective is that society strongly encourages —I'd even say strongly pressures— people to have kids, to feel that if you don't, it's almost like something is wrong with you. Maybe we're just different generations? :-)

Thank you for taking the time to type that, I found it informative and interesting. Like we like to think of it as a single idea that worked well, but it's actually a ton of work to get a million details just right, and failure in just a few of them would just have made the whole endeavor (of the printing press) fail.

Per Wikipedia, in 2018:

* Median household income in Mississippi: $44,717

* Median wage in Germany: €5,370 per month, equals $73,565.

So even the individual median wage in Germany is more than 50% higher than the median household income in Mississippi.

Sources: https://en.wikipedia.org/wiki/List_of_European_countries_by_... and https://en.wikipedia.org/wiki/List_of_U.S._states_and_territ...

I went back to a senior IC role after managing for 5 years and I think there are big differences. Yeah, some aspects have to do with leadership (regardless of whether or not you manage) and there's overlap, but... there are also huge differences. I wrote about one I experienced: https://alejo.ch/3g9

The approach we're taking to deal with this very real context rot is using a bunch of related techniques which we call transposing the agent loop: https://alejo.ch/3jt

In essence, we run many short agent loops, generating their prompts dynamically from structured data. Each loop advances the state in a small step towards the final goal.

Oops, you're right, I put the wrong url (Android auto complete fail, oops, suggested a link and I assumed it was the right one). I meant to link to https://alejo.ch/3gk

I guess I'll hide this submission and do another one with the right url.

Sorry for that and thanks for pointing it out.

Good luck!

In case you find it useful, I wrote what I learned in my first year of fatherhood (the little guy is now 17 months): https://alejo.ch/3hj

As you can see, I never found anything as scary as the possibility, however remote, of my baby being unhealthy. So, yeah, best of luck!

But filtered coffee is the most flavorful! No other method extracts the subtle nuances as well! P.s. I know it's subjective, just cringe on this claim of "the most flavorful" starting with espresso. :)

You're literally saying "an airline, booking a flight 6 months out, 6 days out, or 6 hours out" is not "charging two people for the same route differently", completely missing the point of alex43578's excellent question.

You're probably thinking of Walter Gropius, the founder of the Bauhaus school.

> by someone who maybe isn't so great at clear communications

I don't think that's fair. If you're familiar with the programming language, his writing is fairly clear. If you're not, maybe you're just not in his target audience.

IOW, optimizing his text for people familiar with the language is probably a better choice than teaching the language, which would distract him from his goal.

She has a cool blog, I liked her articles!

How does she keep track of what row she's on, as she knits? I wonder if she'd be interested in specifying her patterns in a format my knitting software (visualization) could consume.

Anyhow, cool stuff!

This is indeed much better. I couldn't really follow the original, but this one made it click. Pretty cool!

That's exactly my understanding as well. This is, essentially, the LLM hallucinating user messages nested inside its outputs. FWIWI I've seen Gemini do this frequently (especially on long agent loops).

I don't know if it's niche, but I like making granola for my friends and family. I give them a big jar and tell them free refills are included ("just bring me the empty jar"). I get pretty good nuts and tend to make largish batches (around 2 kg), and, because of the refills, I get a good sense of who appreciates it — always happy to make more for them. My recipe is here: https://alejo.ch/365

I got into designing my own knitting patterns. I enjoy that I can customize everything — the yarn material, color (including marling, helix knitting, double knitting), yarn weight, needle size (e.g., resulting in "airy" vs "packed" textures), knit textures (e.g., stockinette, linen, miss, etc.), construction process (e.g., can I figure out a way to knit in the round vs flat?), cables, gradual increases/decreases, selvedge/cord, desired ease, etc..

I wrote software to generate patterns given configurations and keep track of which row I'm on. https://news.ycombinator.com/item?id=40307089

I am sharing some of my patterns here: https://alejo.ch/2s0

I'm currently working on my second ruana.

I was also baffled. "No formal specification"? Two minutes of browsing is enough to find it: https://github.com/sqlite/sqlite/blob/master/src%2Fparse.y

Looks pretty cool. I think it's great that you support light/dark mode. FWIWI, I'm also a huge fan of Excalidraw.

I wanted to ask you: is there's a reason you use a separate svg file for each (light/dark) mode?

A single SVG file using CSS can change it's own colors based on the user's preference. I have an example here: http://alejo.ch/3jj - the 3 plots should honor your mode (I put the generator code here: https://github.com/alefore/mini_svg)

Just figure I'd ask. If you have a good reason for using separate files, I'd love to hear it (because it probably would also apply to what I'm doing). :)

Waiting for a few days of design review is a pain that is easy to avoid: all we need is to be ready to spend a few months building a potentially useless system.

How long have you been tracking? Can you share an insight you've had from your data?

I've been weight lighting for ten years and initially tried to track things (down to how many reps I did of which exercise, with how much weight) and quickly came to the conclusion that is want worth it for me.