testtesttesst

Don't forget the 0.5% infant mortality rate

https://www.cdc.gov/nchs/fastats/infant-health.htm

Number of infant deaths: 19,582

Deaths per 100,000 live births: 541.9

Leading causes of infant deaths:

– Congenital malformations, deformations and chromosomal abnormalities

– Disorders related to short gestation and low birthweight: not elsewhere classified

– Sudden infant death syndrome

An abelian group of prime order has no nontrivial subgroups by definition. How can you "attempt to avoid" cofactors -- there are no subgroups?

Great ad to not buy your coin.

Project succeeds = good.

Project fails = bad.

If a bunch of projects given to NG + LLM fail, the company will say at some point "this is no good, let's not do that anymore". If they mostly succeed, they'll do it more.

From a middle manager's perspective, where they're funding 10-20 projects, this kind of measurement is feasible and carries at least some signal. So I don't agree it's difficult to measure.

> The experienced engineer also gets an LLM though

Say experienced engineer Edna was working on a project and the new grad Nelson replaced her because he was cheaper and about as good if both have LLMs, i.e.,

Edna + LLM ≈ Nelson + LLM

even though

Edna >> Nelson.

So Edna needs to find a project to work on where LLMs don't equalize her arbitrage, i.e.,

Edna + LLM >> Nelson + LLM

and Nelson needs one where

Nelson + LLM >> LLM.

>Software engineers in aggregate think and act like they’re working in a meritocracy (we aren’t)

Not a meritocracy, a market. "Meritocracy" implies some kind of value judgement (ex: "they are paid fairly" or "paid what they're worth"). The fundamental question is about the labor market and its efficiency.

I'll argue: your value in the market -- as an experienced engineer worth $230K or whatever ridiculous thing it is now -- is if you can go somewhere else that's willing to pay you $230K+ for your labor. The reason salaries are this number (despite antitrust things like [1]) is that some people actually exercise this option and move to higher paying jobs. And most employment is at-will -- you can leave whenever you want. So the market is at least somewhat efficient in this sense.

But efficiency works both ways. If you think your labor is worth more than what your employer is paying you, then you have to prove it by asking the market.

[1]: https://en.m.wikipedia.org/wiki/High-Tech_Employee_Antitrust...

Then those projects will fail and they won't do that anymore. Or they'll succeed and engineers will have to find harder problems to solve.

If a new grad + LLM can do task X about as well as an experienced engineer can, then the experienced engineer will have to find other harder problems to solve. Personally I think labor efficiency is good; why waste your labor on easy/solved problems?

That's capitalism. After you get fired, if you can still produce useful labor there will ideally be more employers waiting; you can freelance if your visa situation allows it; or you can start your own business and outcompete.

The cause of the inefficiency and "great filter" you're describing is the labor protectionism currently in the U.S.

Theoretically a good programmer has arbitrage via their skills and expertise. If you can get things done people will pay you for it, assuming you're allowed to work in the U.S. (or EU, etc.).

>people misunderstand and anthropomorphize LLMs

I agree but there is a certain "god of the gaps" process that's happening here.

To me it seems like what's been achieved so far is "encoding" into natural and programming languages. These models can't reason and e.g., can't pass a Turing test. If you play with GPT-3 or ChatGPT enough you can definitely get it to fail on a task a human would succeed at.

When models so far have achieved "reasoning" behavior, usually it's in conjunction with a classical algorithm -- for example AlphaZero used a model to evaluate chess positions but then ran a tree search algorithm to decide what move to play.

I think if LLMs can be combined effectively with some kind of computation process like in AlphaZero or some kind of "chains of thought" [1], then we'll see them do things we'd call reasoning.

Whether to be optimistic or pessimistic about AI depends on the final "gap" -- whether there is an irreplicable/noncomputable/etc. property of human consciousness other than qualia (i.e., if "understanding" is hard). If not humans will become obsolete, so I pray the answer is yes.

[1]: https://ai.googleblog.com/2022/05/language-models-perform-re...

I think people are more impressed than they should be because of a misconception about what programmers really do.

For most big software projects the bottleneck is not writing code and test boilerplate that runs correctly locally. Which of course LLMs can do reasonably well. Most time is spent on design, on integrations with other internal services, scaling concerns, and reasoning about other things that these models can't really reason about yet.

I'm obvious impressed that these models can generate simple client/server websites and generate code to solve competitive programming problems. But I don't think they've "solved" software engineering; they've only solved "what you boss thinks software engineering is".