When Robustly Tolerable Beats Precariously Optimal

Re:When Robustly Tolerable Beats Precariously Optimal by advael on Friday March 08, 2024 @03:34AM (#39639096)

This is a good evaluation of the way risk posture can inform design decisions, but I think it sort of ignores the elephant in the room: A short term strategy wins more on average in a competitive context, especially when existential risk is on the table for entities who lose a competition. Most solutions to this problem involve solving hard coordination problems to change the balance of incentives or lower the lethality of competition. Figuring out systems that work for your level of risk tolerance is very achievable and can be incrementally improved, but designing for robustness is something that needs fractal alignment at higher meta-layers of incentives to be sustainable

[ Parent | on HN ]

Re:When Robustly Tolerable Beats Precariously Optimal by nonrandomstring on Friday March 08, 2024 @07:57AM (#39640638)

> especially when existential risk is on the table for entities who lose a competition.

This really just explains externalities. Competition creates apparent short-range existential risk to a business. Real existential risk (to people or things damaged by your defective product because of "precarious perfection") land somewhere else - usually where their impact is much larger.

[ Parent | on HN ]

Re:When Robustly Tolerable Beats Precariously Optimal by advael on Friday March 08, 2024 @05:59PM (#39647557)

If something is considered an externality by some system, decisions that prioritize it are by definition hard or impossible to make within that system. In other words, to change that, we have to rewrite systemic priorities to some degree

[ Parent | on HN ]

Re:When Robustly Tolerable Beats Precariously Optimal by nonrandomstring on Saturday March 09, 2024 @06:10AM (#39650954)

That's an interesting thought I'll give some chin stroking to. Sounds like something Kurt Godel and John Gall would have hatched together. Is it a personal insight, or from experience, or something you think might be attributable. cheers.

> we have to rewrite systemic priorities to some degree

maybe Forrester or Meadows has an insight?

[ Parent | on HN ]

Re:When Robustly Tolerable Beats Precariously Optimal by advael on Sunday March 10, 2024 @03:14PM (#39661719)

I mean I am sure I'm not the first person to have this insight - as I said I view it as baked into the definition of "externality" - but I'm also not pulling it from any specific source I can consciously recall

[ Parent | on HN ]

Re:When Robustly Tolerable Beats Precariously Optimal by 6LLvveMx2koXfwn on Friday March 08, 2024 @09:27AM (#39641310)

> but designing for robustness is something that needs fractal alignment at higher meta-layers of incentives to be sustainable

This is why I remain an engineer and nothing more. If this sentence ever became meaningful I guess I would have been over-promoted.

[ Parent | on HN ]

Re:When Robustly Tolerable Beats Precariously Optimal by advael on Friday March 08, 2024 @02:09PM (#39644879)

lol okay I'll admit upon reading it again that that was a more verbose way to make that point than it needed to be

How about "prioritizing robustness and other kinds of long-termism need to be at least somewhat protected by the system of incentives they operate in to succeed"? I invoked fractal self-similarity there because I think you best protect long-termism by prioritizing long-termism in decision-making at scopes further out

[ Parent | on HN ]