jefffoster

I’ve been thinking about this too.

There’s this notion of variety popularized by cybernetics folks a long time ago. Variety is like the state space of the system. Then there’s a law that says “only variety absorbs variety”.

So if a method has high variety then it must have an equally complex implementation to handle the variety.

When there is a mismatch it means that either the method has parameters that aren’t useful, or that the body of the method isn’t covering cases it should.

https://fffej.substack.com/p/only-variety-can-absorb-variety was my attempt to write it up more fully.

AI is an expert in everything you are not.

I did a PhD in the late 90s.

What was the motivation? Honestly, I was too lazy to get a job and staying in academia for another 3+ years seemed amazing (probably not recommended, but it worked out OK for me).

What helped get me through it:

1) Doing something I genuinely enjoyed - I approached the Computer Vision professor who gave me some ideas. I super enjoy writing code, and the idea of processing gigabytes of video to produce answers seemed cool. I treated it as a super difficult programming project.

2) Breaking my leg - Just before starting, I broke my leg badly. And that meant working from home with a weekly visit from the professor with a stack of reading papers. That time spent understanding state of the art was super useful.

3) Funding - At some point, DARPA gave enough money for me not to worry about funding, so I never had to work a job or get distracted.

4) Marriage - The final straight of writing a thesis was tough and I was super lucky to have a supportive wife who pushed me to get-shit-done.

I remember attending a tech event at MSR Cambridge, and a speaker made some disparaging comment about older developers not being able to keep up in this modern world of programming.

An older gentleman stood up and politely mentioned they knew a thing or two.

That was Tony Hoare.

At a guess an Enterprise API account. Pay per token but no limits.

It’s very easy to spend $100s per dev per day.

It’s not only the low obesity rates, the article states it suppresses impulsiveness.

Does anyone have any intuition about the how reasoning improves based on the strength of the underlying model?

I’m wondering whether this seemingly underwhelming bump on 4o magnifies when/if reasoning is added.

A great quote that’s stuck with me is that “LLMs are experts in subjects you are not”

Mostly functional programming does not work (https://queue.acm.org/detail.cfm?id=2611829)

Interestingly they used to attach a sponge to the end. You might think that was because it doesn’t break the glass, but really it was to ensure the nearby houses don’t get woken up for free!

Not just cost reduction but allowing more investment. A good architecture can enable more people to work on your product.

This is an absolutely fantastic book!

Read it to learn how to program, learn a bit about AI as an added benefit.

Gives me a chance to reference the best Harvard Business Review title ever:

How Earlobes Can Signify Leadership Potential (https://hbr.org/2011/11/how-earlobes-can-signify-leadership-...)

Once upon a time I found string interning useful in conjunction with IdentityHashMap for quick lookup of lots of strings.

There's also a positive framing for the sunk cost fallacy.

Invest your money in something (e.g. exercise equipment) and you'll be more willing to spend time using it.

Reminds me of how hard it is to write a screensaver by jwz https://www.jwz.org/blog/2015/04/i-told-you-so-again/ (and follow the links)

I found Paradigms of AI Programming (https://en.wikipedia.org/wiki/Paradigms_of_AI_Programming) an amazing book to work through.

It takes you through implementing some classic AI examples (such as Eliza) and teaches you how to write code in a data-first style through Lisp.

I worked through the whole thing building it in Clojure and learnt loads (not just about AI, but about writing readable code in general). I can't recommend it highly enough!

I agree this is an important metric and a kind of technical debt that's hard to capture.

How do you discriminate this from RenovateBot or Dependa-Bot? Both of those will not only track the drift but they'll also generate the PR to fix it.

There's some research in the "Accelerate" book that talks about this. For example, having an automated CI/CD pipeline decreases stress measures (iirc).

I think there's a lot to learn from the aviation industry. I did a talk at my companies internal conference on this (turned into words at https://medium.com/ingeniouslysimple/why-dont-planes-crash-1...).

For me it's the mindset that differs. Too often as software engineers we find a bug and just fix it. Aviation goes a step deeper and finds the environment that created the bug and stops that.

Unfortunately, the recent 737 MAX incidents seem to have changed this. From what I understand the reaction to the problems sounds more like what I'd expect a software business to do, rather than the airline industry!

I haven't read the Stephen Pinker book, but it sounds similar to one of my favourite books "Factfulness: Ten Reasons We're Wrong About the World--and Why Things Are Better Than You Think" which sounds like it covers similar topics.

As douglaswlance says, the world is getting better faster than ever!

Not quite the same, but https://www.osti.gov/biblio/10117359 discusses a long term (10000 years!) system for marking areas of high radiation.