threethirtytwo

Why does this happen? This can't be drift because then it would go both ways but it's predominantly moving in one direction.

The mechanistic interpretability literature does not imbue a human with these abilities. That literature is primitive. Nowhere near the point of editing model weights and creating models from scratch or even being able to figure out that a model is playing an atari game.

I think people are rather ignorant about how much we don't know. You throw some big words out there like "mechanistic interpretability" as if this is an actual field where we understand what's going on. The reality is we don't know what's going on. We have to use reinforcement training to do anything close to what the agent did here.

It's over. People need to stop being delusional. These things are intelligent beyond anything we've ever seen before. We are at a point in time in human history where there's a paradigm shift and half the world is so full of themselves and there own specialness as humans that they can't even admit it.

Here's a thought experiment:

Have the AGENT interpret and edit it's OWN weights WHILE it runs. Make it so that this is how an agent learns and improves.

This is proof. These things are not stochastic parrots. Agents are not trained on other agentic weights. The fact that it can do this is EMERGENT intelligence.

Time for everyone to wake up. There's no denying it now. Everyone who called it stochastic parroting before was completely and utterly wrong. It's now categorically clear. We are dealing with an actual intellectual entity.

This is not a good measure. You have to measure productivity. If the AI is better, it DESERVES to be paid more.

Bro the future is that AI will be optimized to a point where it is superior to humans when performing this surgery.

That is a different kind of scary. Because it’s justified. Why limit the ai when human lives are at stake?

What if that mouse was a human?

Why would we limit the autonomy and power of ai when human lives are at stake?

The general concept is this: Control through complexity. By making things so complex people don’t understand they’re being controlled or getting fucked up the ass.

This is modern society 101, and techno feudalism is just one aspect of it.

The way you said “face reality” implies a lot of delusion.

There’s a lot of nostalgia. Both for what the city used to be and what I potentially can (but most likely won’t) become. The delusion is so rampant that it’s really what causes people to vote you (and me) down. Like objectively speaking sf is a terrible city and many people agree. At best you can like the city but you cannot ignore the fact that there are some pretty horrible downsides for living here and that many many people hate the city.

I will say if you work in tech or are a liberal extremist or you’re gay then it may be worth it to live here but I wouldn’t call it wonderful. You can’t make the city better if you don’t admit the faults.

You know people are saying that it’s not open source without regard to intent. The intent is 100 percent open source.

The reason why it’s not done is because it takes a huge amount of resources to not only record the training process but to release it in a readable format.

What you’re asking for is almost like surveillance documentation for a model that’s likely put together in a way that’s not easily peaceable into clean step by step procedures.

It’s the same way psychology works on humans. Black box observational stuff.

You can’t generalize anything here because the psychology of the model can be affected by system prompt, reinforcement training, and training data. What you observe here can be wildly different if you change any of the underlying things I mentioned.

It’s like watching an ant colony.

These are really subjective tests because the LLMs can have adjusted behavior based on the system prompt.

If you have the LLM operate on default behavior it likely won’t work because most systems if picked at random don’t work… even for humans.

The systems you see around us come from thousands of years of cultural evolution. If you simulate the same thing with LLMs where you have it write to its own context on what works and what doesn’t work you will eventually get something working.

The problem is a bit different. The world is not at war yet we spend billions on weapons. It's more to show off. It's gaming the system.

Why just SRE's it's deskilling everyone in this industry.

This is a drop in the bucket. Not worth complaining about. Look at Anduril. That company is using AI to make weapons to “slaughter” people.

Quotes around slaughter because their weapons are so useless no one is using them. So they’re using ai to make machines that don’t work.

The consequences can be made equivalent. Reinforcement training.

I think there’s a chance for anti ai terrorism along with Amish style anti ai behavior.

Ai is stripping people away from their identity. There’s tons and tons of hatred for it… even here on HN.

Exactly. The LLM is correct. But no one has the balls nor are they unbiased enough to agree.

Sure you can do that. But what’s the practical point if only your agent is reading it? Your agent doesn’t care about linting.

Not useful at all. It uses up context. You need all the context you can get for the LLM read the code later for you later.

>if you're going to be upset at anyone it should not be the reporters.

Everyone is just upset by AI. So they want to blame the side that was involved with AI. It's like how cops automatically assume black people are criminals.

This is cool. But of course we have to think of the other use case: "Hack the planet."

Teach him how to use ai.

Nobody is talking about how far we'll go or what the end game will be. That is NOT the topic of the conversation!

I'm saying right now in your personal workflow you are giving up many useful features of AI when you only allow it to implement your design as opposed to letting it design for you.

> I take this as a stronger signal that too many people with shitty opinions have a microphone these days

Look in the mirror.

> Until it doesn’t, and then you’re too stupid to understand why not. Was it that your prompt wasn’t good enough? Does the model not have enough context? Is it even possible for your request to be satisfied?

You’re unable to comprehend the topic of the conversation.

We’re talking about querying the model with prompts and asking for solutions beyond your intelligence. You’re talking about flawed prompts.

> You answered this yourself

You’re rude af. Not understanding something is not intrinsically bad.

In fact you use things you don’t understand on a daily fucking basis. Your iPhone, your car, your computer. All of these things have aspects of it beyond your intellectual capability to fully grasp or comprehend and yet you use it. Sometimes these people who make these things produce faulty products and you ask for a refund and are more careful in the future about what you trust, sometimes the product is exemplary.

Relying on an LLM to do something for you is much of the same.

Get it? Probably not. What are you going to do? Call me stupid again because you have nothing substantial to say? lol.

Edit: you edited your response which is uncharitable. But whatever I’ll respond to your new statements below.

> What’s the limit to this? “Claude build me a nuclear fusion reactor.” Surely we can agree that there’s a limit or we’d already be living in some post-scarcity utopia. So what’s the limit?

I agree. But this is not the topic of conversation. I’m talking about querying LLM to do something beyond your ability. This is something you can do and also stay well within the obvious limit you’re implying here.

> I’m suggesting a limit: human stupidity is limitless while compute is finite. We can use the word “ignorance” if the word “stupid” is too inflammatory.

You don’t need to suggest a limit. It’s obvious ai can’t build a nuclear reactor. Nobody needs to be told this.

A clever quote doesn’t prove shit. Ok so human stupidity is limitless. Computer is also finite. I agree.

How the fuck does the above two facts which I agree with have anything to do with me asking the LLM to make something I have no knowledge about? “Hey LLM write an emulator for this Zelda n64 game.”

The prompt is well within the “limit” of what an LLM can do. Compute is still limited. And human stupidity is still boundless. Nothing has changed. Why? Because someone is ignorant and unaware they are prancing off topic onto orthogonal concepts.

What does human stupidity have to do with this? If you're too stupid to ask it a good prompt then you're probably too stupid to design something for it to do.

I'm saying ask the AI something beyond your intellectual capabilities. Or ask it something beyond your intrinsic knowledge base. It can produce a working machine that does the job, but you won't be able to understand it.

You can still verify the machine works by using the machine... but you won't necessarily understand it. Similar to how you own a smartphone... you know it works, but you will never fully understand everything about it.

The issue is the LLM can often design thing superior to anything you can do. Now this isn't always the case but it is often the case. By limiting yourself to code only YOU design you are cutting off a major portion of of AI capabilities that is highly leverage-able.

The pace of AI is literally changing so fast that we can't keep up. Your strategy here was probably true around 3 to 4 months ago. Right now it's no longer true. Not many people understand navier stokes and that's what AI can offer, it can offer some superior problem solving methodology with the level of complexity of navier stokes that you have no hope of understanding without years of study and education to get yourself up that level.

This is not a trivial question with a trivial answer.

Unfortunately your answer is outdated. I think if trendlines continue, the answer is trending violently towards putting more and more trust towards AI.

It's a cheap shot when a woman questions a mans masculinity/courage. It's deliberate and done with malice. It's like someone asking you why are you being a little bitch? Honest question.

Read the last paragraph before posting something redundant.