coffeemug

We can make planes and engines because we kept the expertise, and we can make rockets because of herculean efforts of a few people. This simply isn't good enough, we need to make creating new physical things as easy as possible.

He is not arguing that. He is arguing that these programs obviously aren't conscious, and that it's dangerous to tilt the weights toward emulating conscious beings for reasons outlined in the essay.

I agree this is science fiction-y, but given the pace of progress I don't think it's so easy to dismiss.

> And you could still ban specific, known malicious models/labs instead of blanket open-source bans

With publicly available models today, absolutely. With models 10000x more powerful, we would need to be extremely conservative. (In that world everything is upside down, though; I don't know if "banning" is even a meaningful concept in that reality)

I don't have a view on whether this is a sufficient argument to ban models from potential adversaries, but it's not as trivial as "regulatory capture".

EDIT: ok, here are two ways:

1. if it's merely a voice, I want to hear it. If it's slop, I want it taken out.

2. voice is signal, slop is noise; thus low-signal sentences are slop.

The second flaw is that it eats perfectly capable minds for years and the results don't justify the time investment. Python or whatever is fine. I wish I took most (but not all) of the time I invested in Lisp and put it into something else instead.

EDIT: I agree that it should require an act of Congress to explicitly delegate this power.

As an aside, you made me curious if Trump made this constituency materially better off. Here's what Claude thinks (tl;dr: it's a wash): https://claude.ai/share/36233694-3729-4758-b2e6-c2058791ab1a

To pick just one example, infinite scrolling can be seen as a modern equivalent of cigarettes— a product that made people billionaires, and that consumers obviously want but are not free to stop using because of hyper-sophisticated dark patterns.

Is Elon a trillionaire because he created a trillion dollars of value from thin air, or is it because he created an information asymmetry flywheel that lets him allocate capital more efficiently than other actors?

It’s genuinely unclear to me whether the universe in which we incentivize this kind of scale is better than the universe in which we do not (because the counterfactual universe has massive externalities too). But this is obviously not just a matter of compounding value creation and becoming a billionaire fair and square in ten years.

What OP said is definitely true on the micro level-- not "even/might/some aspects", but the whole thing. It's true that in any given organization there are fewer senior roles because of hierarchical nature, it's true that as you progress up the ladder the demands change and increase, and it's true that many people fail or choose not to adapt.

The macro argument seems right as well. If you measure it longitudinally the numbers don't stay constant. It's 1 in 4 today, maybe it was 1 in 10 fifteen years ago. Anecdotally there is definitely something strange going on with the labor market that's new, and that you can't explain by micro realities alone.

The second equilibrium seems more likely-- the capitalist class grants the public a bare minimum to keep us from forcing political action. In the AI world "the minimum" is probably a much better standard of living than we have now, as the marginal cost of many products and services approaches zero. So we end up living much better material lives, but are still not free. Maybe this is stable, or maybe the ruling class loses dominance over time. At that point, who knows.