great_psy

I wonder if this might point to some efficiency loss coming from AI. Let me explain.

Say in 2015 they installed those smart refrigerators, they could go from 20 people monitoring temperatures to 1. That was some profits being made by those companies, and they got to enjoy those benefits until now.

With AI finding exploits for so many things people wouldn’t spend time looking, we might need to go back to dumb fridges out of security concerns.

Now extrapolate to many other industries, the economic gain from AI might be hampered.

Can we have a tldr of the implications of this for the uninitiated?

What grip do they have then ? Is Claude code and the suite of other interface that good compared to open source/ competitor offering ?

I wonder how much less useful it is if I use those models for open code or similar. What are you really learning about me, other than the fact that I am a technical person, which you could know by the fact that I signed up for open router to start with.

Interesting how it is all in mandarin. What I mean by this, is that it was prompted in mandarin to start with.

I wonder if the result would be better or worse if prompted exactly the same in English.

What would be a reasonable question, that’s similar difficulty as implementing a linked list ?

We have tried other questions as well, with similar results but perhaps our questions are outdated ?

I think to some extent is that the right candidates and employers are not meeting.

Working at a ~1k employee company, we have been trying to hire both junior and senior devs. Out of ~10 interviews I conducted for junior, only 2 was able to program a linked list, and even that was with hints. It was not much better for senior(more system design question instead).

I understand that leetcode questions are (and have been) outdated, but linked list for somebody fresh out of school seems reasonable.

And this overlooks all the people using ai to cheat on the interview.

I would expect in this market we would have better candidates. But for us it was a challenge to hire.

And let’s not even get started with all the ai generated resumes and candidates that don’t actually exist, or have the experience they say they do. Really a waste of my time to interview.

The recruitment process is not made for a world where anybody can just copy paste the job req and generate a resume that fits everything.

Shouldn’t we expect a convergence as models become more powerful ?

There is an optimal answer to any question. Something that maximizes utility and minimizes tokens.

As ai replaces more and more jobs, there should be plenty of humans available to teach. Even if the teachers are ai assisted themselves.

At least link the one made by anthropic. Can’t say if this is better or worse since I’m on mobile.

Could this be extended to lean into teaching quantum physics ?

Include seconds/ sub seconds hand in the watch, and people will realize the watch face time + time it takes to read will never equal the watch face time.

You can know the exact time, by looking at the analog watch face, or you can measure it (convert it) but it will not be the same anymore.

How do you measure quality at scale ? Is there another model that determines if it adheres to codebase standard ?

I think you’re missing how complex international operations and optimization are.

Each country has their own laws around what uber is and isn’t allowed to do. This needs to be formalized in code. For example you actually call a taxi, though the uber app, and the amount you pay is per mile, not a fixed fare decided ahead of time. To add to this complexity, some cities will have their own laws. What happens if you take an uber from town a to b, where each one has different laws ? A lawyer probably has an answer but the app needs to adhere to that. On top of that laws change all the time.

Optimization, well you can always optimize something. speed, costs, paths etc. In a way this never ends.

I think the part we interact with as consumers is a tiny sliver of the complexity those services have to build and operate.

You’re right about performance degradation, but good luck trying to sell that as a product.

You can drive this car, but the last mile of this trip will use as much gas as the first 20 miles.

I think it’s in anthropics interest to keep this fact hidden from CEOs who push for ai adoption.

My view is that ai makes it very easy to pump out a lot of code, and that makes it too time consuming to just merge it without understanding it 100%.

The person pushing the PR is getting promoted because their are delivering so mane features, but after some time the codebase is a mess.

Sure the same person might end up dealing with some of the cleaning up, but more likely those refactoring tasks end up getting spread across the team as the need arises.

People are incentivized to push out code, which under human-written coding standard usually meant a level of pro-efficiency, so they should be rewarded.

But with the new model of pushing out code with ai, a better metric of a good engineer that should be promoted would be lines deleted, or something like that. Much harder to measure, and hard to justify to management.

Problem is that it does not produce better or more work, it actually shifts the work to a different/future engineer. Today’s slop which gets engineer 1 a promotion, is engineer’s 2 problem next month when they are oncall and the codebase makes no sense.

Your horse riding analogy, is like riding a horse into battle without your weapon because it’s slowing you down. Sure you got through the enemy first by outmanoeuvring, but you missed the point all together. Maybe you got a shiny medal but all your mates are dead.

This might be silly, but … since the assistant models are so much smaller than the full models. What if we just use those smaller models?

Any idea how much worse they will be ? Or is the issue that their error will really diverge as you accept more of their tokens?

If you have a link to the study you read, please share it.

This is the first time I heard about estimating tokens for a task.

I feel like you’re on to something. Management will pick this up, and make it part of the sprint planning.

Engineers will pull out their hair wondering how you can do that.

That’s like estimating how many CPU cycles a task will take. How many instructions will your laptop use while you work on something.

Would it be possible to do some prompt injection so this string has the same effect but from a website ?

So if you add some special string to the docs, it stops Claude ?

I just suggest you use Claude to write the script for you. And then you run the script with cron. Really it’s not any more time, just takes a different view on what the goal is.

I see people highly trained engineers spend hundreds of thousand of tokens doing what can reliably be accomplished with 150 lines of python.

I think the push from management for us to use AI has made it so we don’t have to be efficient with our consumption, so now we write md files which we feed to Claude in a loop instead of python and bash scripts to do routine tasks.

Give me a python script that takes a string representing the output of a sha256 algorithm and a plain string and compares if the sha256 of plain strig matches the sha256 provided.

I think the article is making the point that it is a cultural problem about cost cutting and short term thinking.

LLM Memeory (in general, any implementation) is good in theory.

In practice, as it grows it gets just as messy as not having it.

In the example you have on front page you say “continue working on my project”, but you’re rarely working on just one project, you might want to have 5 or 10 in memory, each one made sense to have at the time.

So now you still have to say, “continue working on the sass project”, sure there’s some context around details, but you pay for it by filling up your llm context , and doing extra mcp calls

Is there any provided reason from anthropic why they changed the tokenizer ?

Is there a quality increase from this change or is it a money grab ?

What kind of things would you use this to simulate ?

Is this to make sure services are wired up with permission and communicate properly before you deploy or what is the use case ?

Also, why not have a dev environment where you can test things on the real thing (ideally with smaller size instances to save $)

Does this post feel AI generated to anyone else ?

But to actually answer the question: I’ve been putting research paper pdfs in notebook llm , and turning them into ~40 minute podcasts which I listen to on my walks. Yes it’s shallow learning, and it might have some hallucinations in there but I wouldn’t have read some of those otherwise.

Thanks for working on this!

Is there any way to get those running on iPhone ? I would love to have the ability for it to read articles to me like a podcast.