Was there at least performance gains to be measured?

AFAIK nobody was collecting analytics. The one team I was working on had put out a goal of "30% more efficient" using AI tools. Its about as subject as you can get. We never got around to what exactly that meant before everything got shut down.

Myself and several other devs were laughing about the whole thing. The company was so amped about what AI could do they never even bothered collecting any analytics that would affirm or deny any of this had a positive impact. Even some of my team members were talking about the placebo effect AI has had on a lot of C-Suite folks.

> Even some of my team members were talking about the placebo effect AI has had on a lot of C-Suite folks.

What do you mean placebo effect? They thought things were created with AI while they actually weren’t?

For a lot of C-suite there has been more of a fast-follow approach to AI. CEO hears 3rd hand what competitors are doing, tells his CTO to "do AI" and "hire a head of AI".

So a lot of motion (do AI) was created without a destination (product outcome). The motion was what was being measured (we are doing AI).

Headlines is what matters at the executive and board level. Start measuring things and they might not be able to spin a story to sell.

For a large part of society we have management in leadership positions.

Probably meaning C-Suite thought productivity was up because of AI, either because A) metrics showed high AI usage or more commits/LOC or B) we're mandating AI usage, why wouldn't it increase?

30% is my new favourite stat. It seems to come up a lot, like the old "this work will take 6 months" that I was equally suspicious of back in the day.

Bender: "I'm 30% [XYZ]"

http://theinfosphere.org/Bender_Bending_Rodriguez#Compositio...

This is all still going on at my company but they’ve been trying to create dashboards to measure us as good or bad with our output unsuccessfully but they keep shoving it through anyways. It’s demoralized a lot of people.

> It’s demoralized a lot of people

Isn't this a good thing? Doesn't it mean your company doesn't want employees to treat LLMs like a slot machines?

But they are, so it’s demoralizing to be punished for not hitting the jackpot

I saw incredible productivity gain call-outs from non-technical teams like "it saves me 15 minutes per day summarizing my email".

Trillions of CapEx for good Clippy/Siri, nice.

The only thing the people in charge care about is this: is their equity going up? Nothing else matters like results or revenue. Quaint concept those.

Maybe it was your company with the 500 million Claude bill.

Woah. I missed that one. Had to look it up. That’s one hell of a surprise bill.

Were there any managers fired for incompetence?

You like to believe that these companies know what they're doing, they do not, I mean don't even try to find the logic behind their actions. They just follow the shiny things and then when it doesn't work, react by screwing up their employees. Their decisions age like milk on the countertop. After I got laid off a few month ago, I can't imagine myself working another 9 to 5. Because above all, it's an incredibly stupid way to spend your fleeting time here.

Right Eventually you realize companies are just a bunch of people making it up as they go along

Institutions are neither permanent nor superintelligent

Probably a stupid question, but isn't giving developers $200 for a Max x20 subscription cheaper than using the API? I was discussing Claude's monthly bills with some friends recently. Most of us are in the $700-$1000 range, and I don't feel like I really spend that much. I have a Max x20 personal account, and I do a ton of work for my side projects through it.

Above a certain size you cannot get the quota-based models.

In my org, as our Team account (quota-based) was growing, to onboard more people we _had to_ switch to Enterprise (and thus API-based costs). The switch and now cost visibility really drives home the "did this code review worth $15?" kinds of questions...

We are just building the monitoring and I think there will be a crack-down sooner rather than later.

I'm more "code as craft" person, so I'm mainly unaffected (plus found my favourite local LLMs to be the missing rubber ducky with no associated cost), butI cannot say the same for everyone else around.

Most companies these days are incredibly disciplined on spending. That’s one reason why corporations have record profits in the last few years. The level of efficiency is so high.

That’s what makes the AI Wild West so fascinating to me. Companies just threw money at it out of fear of falling behind.

Now, I think that fear has subsided at many companies.

I always thought it was incredibly questionable that we were able to slurp down AI tooling with no questions asked. Meanwhile, if we want a piece of software for some other non-AI purpose, the prospect of spending $50/month per employee is met with deep scrutiny or outright denial by default.

All the tokenmaxxing stuff was just management signaling that every single employee better be using AI or else. Now that everyones got the message its back to business of course

I have been a major LLM skeptic and "late adopter." I had never actually used LLMs for anything at all (either in work or personal life) until about two months ago, although I had followed their development with interest and skepticism.

A couple months ago, I started using Claude for some tasks related to processing really messy data. Our company was making a big push for its use, and a co-worker gave me a fairly impressive demo of what he was able to do with this data in an hour or two.

Our company's Claude subscription defaulted to Opus 4.7 and we were encouraged to use it as much as we could.

I don't think I ever did more than 10 or 15 prompts in a day; I tend to do very detailed prompts and then spend 10-30 minutes playing around with the results and resulting code.

Today we get this:

> TL; DR: We're nearly at NNN Claude users and growing fast. Three quick asks: (1) default to Sonnet for everyday work; we have launched XXXXXXX to help with clarity, …

> …

> As we scale, let's be smart about how we work. Claude offers several models, and for everyday work — … — Sonnet is your go-to.