nightski

From what I saw in the paper, no they did not. They tested that the specific change introduced increased performance on a single task (the one mentioned in the prompt). They did not study the impact of that change on overall model performance across a variety of benchmarks. But maybe I missed something.

It's not that they are "no better", it is that they have been specifically instructed through RL to follow the user's direction for better or worse.

So this is why we are seeing all these CVE's and vulnerability disclosures? Because of the careful curation? Feels to me it is more like ignorance is bliss.

I'm not sure that is strictly true. Speed of iteration is increased by interpreted languages in the small, but when working on larger systems or on teams where collaboration is required they can have the opposite effect.

I also find that thinking through a design first helps me achieve a result faster. So I prefer languages that encourage you to think first, type later. Interpreted languages encourage you to type first, think later.

If I had to guess it's that very few people use it for actual security guarantees and most use it just to be able to fill out the questionnaire or compliance box saying "we do this".

I don't think anyone was arguing the legality of it. At the end of the day though this is why open models will win.

Oh that is hilarious. I had assumed this was the M6 which they are skipping Pro/Max on. It's just an M5...

I feel like that might be due to the fact that it is not a Pro/Max model and that they are skipping on to M7 next year.

Are you saying it's hard to edit a docker compose file? Being one service or 5 doesn't really make things more complex. I often point servers at my own postgres container instead of spinning up an extra one (so just remove that from the compose for example). That's kind of the entire point of compose, it's so flexible.

In my experience I can heavily influence this based on the prompt.

But why? On reason is that implementation was a bottleneck. That made it expensive to change a system once it was entirely implemented. It was better to get the feedback sooner so you didn't waste time implementing the wrong thing.

The calculus on a lot of these old adages is changing.

Why do those working at frontier labs deserve charity?

Yeah I have no problems letting family and friends use cloud services. I keep the self hosted stuff for myself.

This isn't about cost. If these projects, like bitwarden, were run at cost the price would be negligible. Instead you see software companies with like 80% margins (or even higher). Not to mention that if the price was reduced to near cost then you'd get more users which would even further decrease the cost.

I'm confused, so you are self hosting it for other people?

What natural forces in the cloud services market do you see pulling it in that direction?

AI needs to be open weights at a minimum, but ideally open source. I'm not terribly worried as it is strictly inevitable. But having these companies steal work from humanity and use it to train their models while attempting a hostile takeover of the economy is morally bankrupt. I lose a lot of respect for anyone going to work for these labs.

It took me 10 minutes to figure that out, and I have been in tech for 30 years :) I'm probably just dense but it's not a good sign.

This sounds like that bitter old engineer that is unable to learn and adapt as things change, the kind that gives all of us more senior folks a bad stigma. Times are changing, adapt and grow.

I understand that. But I am specifically questioning if it was used to actually commit a war crime which is what the school attack was identified as by the U.N.

Since it's in quotes is that what was actually reported in the article? Or are you stating a hypothetical? Because if the school strike was actually AI led that is a big deal.

For me personally the snark isn't because their SRE team is incompotent. It's because software that tries to be everything to everyone is inherently terrible. It's not fun to use for the users, and so many compromises need to be made on the technology side to make that happen that it ends up just being crap all around. This includes Salesforce, SAP, Dynamics, any platforms like that which scale many industries.

Flexibility and abstraction come at a high cost. It doesn't really matter though, world domination at all costs is the name of the game.

I've played quite a few hours of Blizzard games throughout the years and as much as I loved the starcraft universe RTS is just not approachable for me. So I can vouch as one person who is really excited to get a taste of the universe outside the RTS genre.

That said, whether the game itself will be good is still TBD.

Maybe Enron eh?

Seeing as it adds a second PSU, I'm guessing the workstation one.

No, I'd have to guess that new model releases have either new from scratch or continued pre-training. This is not the same as continual learning. Starting a new pre-training session is a dramatically different affair and involves utilizing the entire source data set in some fashion. Not just continued training on new data. At least from my understanding.

Bolting on new data to an existing model (fine-tuning) is precisely what gets you catastrophic forgetting.

I disagree about your last statement. That only covers the initial creation/generation phase. It ignores the engineering aspect of maintaining that design through the entire lifecycle of the product. How do you evolve that table design?

Sure an LLM can update every table in your project every time you make a change, just as a human could. It also can make mistakes and burn significantly more tokens doing so. Or, it could update one component with a one line change and be done. Easy to verify, less change for mistakes.

Engineering is a discipline and does not change dramatically with the advent of outsourcing it to agents.

Alright let's assume your premise is true, that transformers can learn from interaction with the world by updating their weights - then why isn't this done?

Because backprop fundamentally wants the entire data set in every pass. It doesn't behave well and is destructive when you update after the pre-training phase. RLHF/LORA are attempts to work around that and effective at what they do, but it is not learning in the sense you are talking about and also do not fully address the catastrophic forgetting problem. This architecture as is - is not compatible with continual learning.

So you enjoy reviewing your entire code base and relying on an extremely detailed regression suite just to make simple changes? I don't think AI changes good engineering at all, it just changes who or what is doing it.

I was not saying that they are deterministic, rather that the distributions (aka weights) are fixed. A model as deployed today at anthropic/open ai/etc is not learning beyond the context as far as I know.

What prevents continuous fine-tuning from what I understand is catastrophic forgetting. You can do things like RLHF which are built to minimize the damage but that is more about bringing out capabilities of the base model than incorporating new knowledge (at least from my understanding, I am obviously not a researcher at a lab).