For ChatGPT and Gemini, yes.
But for Claude, they have a very deep & big one: Its the only model that gets production ready output on the first detailled prompt. Yesterday I used my tokens til noon, so I tried some output from Gemini & Co. I presented a working piece of code which is already in production:
1. It changed without noticing things like "Touple.First.Date.Created" and "Touple.Second.Date.Created" and it rendered the code unworking by chaning to "Touple.FirstDate" and "Touple.SecondDate"
2. There was a const list of 12 definitions for a given context, when telling to rewrite the function it just cut 6 of these 12 definitions, making the code not compiling - I asked why they were cut: "Sorry, I was just too lazy typing" ?? LOL
3. There is a list include holding some items "_allGlobalItems" - it changed the name in the function simply to "_items", code didnt compile
As said, a working version of a similar function was given upfront.
With Claude, I never have such issues.
It keeps trying to re-invent the wheel, does a bad job of it.
The physics sim was supposed to be a thin wrapper around existing libraries, but instead of that it tried to write all the simulation code itself as a "fallback" (but it was broken), and never actually installed the real simulators that already did this stuff despite being told to use them in the first place. The last few dozen(!) prompts from me have been pairs of ~["Find all cases where you've re-invented the wheel, add them to the planning document", "now do them"]. And it's still not finished removing the original nonsense, so far as I can tell.
One of the two Swift experiments is just a dice roller, it took about 10 rounds of non-compiling metal shaders (I don't know metal, which is why I didn't give up and do that by hand after 4) before I managed to get that to work, and when it did work it immediately broke it again on the next four rounds. It wrote its own chart instead of using Swift Charts, and did it badly. It tried to put all the hamburger menu options into a UIAlertController. Something blocks the UI for several seconds when you change the dice font. I didn't count how many attempts it took to correctly label the D4.
The other Swift experiment was a musical instrument app, that got me to the prototype stage, eventually, but in a way that still felt like a student's project rather than a junior's project.
Did you put in the original prompt the "wheels" you wanted it to use? It's a toss-up when you aren't very specific about what you want.
For the python physics sim, step 1 was to generate the plan, the prompt included "I want actual plasma physics, including high-density, high-field regimes, externally applied fields, etc., so consider which FOSS libraries would suit this.", and then it proceeded itself to choose some existing libraries, and I made sure those specific named FOSS libraries actually ended up in the plan.
My first clue this wasn't going to work was that even from step 1 it was pushing for writing all the simulation code and not actually using e.g. WarpX despite that it itself had suggested WarpX. In fact, even when WarpX was in the plan, it was "integrate" rather than "just use this from the get-go".
I may well throw the whole thing out and try again with Claude when this trial expires. Most of the runs have been comically non-physical, to the extent you don't even need a physics degree to notice, or even a physics GCSE.
Claude made far fewer mistakes in general, never gave me non-compiling code.
Definitely depends on your use.
That's not a moat though. Claude itself wasn't there 6 months ago and there's no reason to think Chinese open models won't be at this level in a year at most.
Maybe it is tech stack dependent (I have mostly used it with C#/.NET), but I have heard people say the same for C#. The only conclusion I have been able to draw from this, is that people have very different definitions of production ready, but I would really like to see some concrete evidence where Claude one-shots a larger/complex C# feature or the like (with or without detailed guidance).
What is so strange to me is that surely there is more C# out there than ESP-IDF code? I don't have a good explanation beyond saying that my codebase is extensively tested and used; I would know very quickly if it suddenly started shitting the bed in the way you explain.
We already have coding tuned models i.e. Codex. We should just have language / technology specific models with a focus on recent / modern usage.
Problem with something like Java is too old -- too many variants. Make a cut off like at least above Java 8 or 17.
The “just” part is a big assumption. It is far from easy, given that modern best practices are always underspecified. The effective model for coding must have reasoning signals to be much stronger than coding patterns, and that, I suspect, requires very different architecture.
I feel like this is an example of people having different standards of what “good” code is and hence the differing opinions of how good these tools are. I’m not an embedded developer but 600K LOC seems like a lot in that context, doesn’t it? Again I could be way off base here but that sounds like there must be a lot of spaghetti and copy-paste all over the codebase for it to end up that large.
Answering this question directly puts me in an awkward spot because I realized last fall that there was absolutely no way that I could talk about what I'm working on in a way that can be associated with my product because there's so much anti-AI activism right now. That sucks, because I'd like to be "loud and proud" but I have a family to feed. I strongly suspect that versions of my story are playing out for hundreds of entrepreneurs right now.
Here's what I can describe: it's an ESP32-P4 based consumer device with about 45 ESP-IDF components that all communicate over an event bus. There's a substantially modified LVGL front-end with a 3D rendering engine and SVG-like 2D animation in front of a driver for a customized variation of the ST7789. There is substantial custom code for both USB host and client functions across various modes of operation. There's custom drivers for several sensors and haptic feedback. There's a very elaborate menu UI system which is also backed by a BBS style terminal configuration system for power users. There's an assignable action system with about 40 actions that all have their own state machines and a lot of mutex locking. There's a very involved and feature-dense trigger scheduling system. There's a very flexible data stream routing matrix. There's a full suite of command line scripts for most functions. There's a self-hosted web app for configuration that also implements a screen share functionality via an HTML canvas object so that I can record videos of what's happening on the device with OBS without having to point a DSLR at it from a gantry.
Honestly, I could go on and on, but all of the people who think that 600kloc is a lot [sight unseen] are following YouTube tutorials and can eat me.
I responded to you because you asked politely. I hope it was an interesting reply.
same here :)
> one-shots a larger/complex C# feature
I can show you a timeseries data-renderer which was created with 1 initial very large prompt and then 3 following "change this and that" prompts. The file is around 5000 lines and everything works fine & exactly as specified.
Yep, this is another case of different standards for "production ready."
++1
Is these more related to the existing source code or is this a bad pattern thar you would never do regardless of the existing code?
One does often hear that where LLMs shine is with greenfield code generation but they all start to struggle working with pre-existing code. It could be that this wasn't a like for like comparison.
That said I do personally feel Claude to produce far better results than competitors.
In my experience working in a large codebase with a good set of standards that's not the case, I can supply examples already existing in the codebase for Claude to use as a guidance and it generates quite decent code.
I think it's because there's already a lot of decent code for it to slurp and derive from, good quality tests at the functional level (so regressions are caught quickly).
I do understand though that on codebases with a hodge podge of styles, varying quality of tests, etc. it probably doesn't work as well as in my experience but I'm quite impressed about how I can do the thinking, add relevant sections of the code to the context (including protocols, APIs, etc.), describe what I need to be done, and get a plan back that most times is correct or very close to correct, which I can then iterate over to fix gaps/mistakes it made, and get it implemented.
Of course, there are still tasks it fails and I don't like doing multiple iterations to correct course, for those I do them manually with the odd usage here and there to refactor bits and pieces.
Overall I believe if your codebase was already healthy you can have LLMs work quite well with pre-existing code.
Don't we all?
- literal Claude ads I see online
- my underperforming coworkers whose code I’ve had to cleanup and know first hand that no, it wasn’t flawless
This kind of sentiment is gaslighting CTOs everywhere though. Very annoying.
That's, just, like, your opinion, man.
OpenAI - since the beginning has been anything but open. If you spoke anything ill about OpenAI here until yesterday, you would be downvoted into oblivion because, let's face it, Sam has always been the poster child of this community.
So, basically, even after them publicly announcing they were evaluating licensing models where they wanted to take a % of your business for using their models [1], there was still 0 outrage, and anyone who pointed that out, always got shot back with "OpenAI CAN DO NO WRONG" in the comments always.
He makes one decision you all don't agree with and now it's cancel culture time?
And somehow, Anthropic is the hero in all this? Make no mistake - all the model providers are building detailed user models. Every bit of information you provide to it is of course being used to for detailed user targeting. This is no different than the "Apple GOOD, Google BAD!" tropes. There are no heroes in for-profit corporations. Everyone is operating a for-profit business model and optimizing for the same profits.
Stop with the NPC behavior. We are better than this.
[1] https://openai.com/index/a-business-that-scales-with-the-val...
"Licensing, IP-based agreements, and outcome-based pricing will share in the value created. That is how the internet evolved. Intelligence will follow the same path."
Why are you assuming these are real people and not NPCs?
The amount of money flowing around AI is staggering. To believe that the AI companies aren't flooding all the social media zones with propaganda is disingenuous.
Touché
You don't use "believe" with "disingenuous": it literally makes zero sense.
If people honestly believe that, they may be naive. Or they can be "disingenuous" if they're not being sincere. But if you just say what you believe, you're sincere (and maybe naive), and hence cannot possibly be disingenuous.
"Naive" is certainly a better choice.
> Cancels subscription
> Random guy on the internet tells you to be outraged
> Gets outraged
I'm not even a fan of OpenAI generally speaking, but, this is just silly cancelling them for no reason. If not them, some other lab would have done it. Or worse, DoW would've forced them to.
"When disagreeing, please reply to the argument instead of calling names. 'That is idiotic; 1 + 1 is 2, not 3' can be shortened to '1 + 1 is 2, not 3."
I think HN in particular as a crowd are very vulnerable to the halo effect and group think when it comes to Anthropic.
Even being generous they are only very minimally a "better actor" than OpenAI.
However, we are so enthralled by their product that we tend to let the view bleed over to their ethics.
Saying we want out tools used in line with the US constitution within the US on one particular point. Is hardly a high moral bar, it's self preservation.
All Anthropic have said is:
1. No mass domestic surveillance of Americans.
2. No fully autonomous lethal weapons yet.
My goodness that's what passes for a high moral standard? Really anything that doesn't hit those very carefully worded points is not "evil"?
However, I would think I'm not alone in that I'm generally wanting to do good while also wanting convenience, I know that really every bit of consumption I do is probably negative in some ways, and there is no real "apolitical" action anyone can take.
But can't I at least get annoyed and take my money somewhere else for the short amount of time another company is doing it better?
Yes, if openAI suddenly leaps forwards with codex and pounds anthropic into the dust, I'll likely switch back despite my moral grievances, but in a situation where I can get mildly motivated to jump over for something that - to me - seems like a better morality without much punishment to me, I'll do it.
there are some people (companies are run by people) that are so bad I boycott them. Most bad I treat like society cannot work without accepting them anyway.
edit: yes, nillistic - but sometimes you have to go there
You can see the significance of this is you look at German Nazi history. If more companies had stood up to the administration, the Nazi state would have been significantly harder to build.
In my opinion, what Anthropic did is not a small thing at all.
By contrast Anthropic wouldn't? Yet Anthropics stance is only two narrow restrictions. As I said are those two things the only evil things possible?
If not, why is it that people on HN think Anthropic would not allow evil usage?
My hypothesis is a halo effect. We are so enthralled by Claudes performance that some struggle to rationally assess what Anthropic has actually done.
Yes it's no small thing to say no to the Trump administration but that does not mean they haven't said Yes to otherwise facilitated other evils.
In fact to me the statements from Anthropic seem to make clear they are okay with many evils.
Really I think Anthropic should have a single restriction: to not assist with illegal or unconstitutional activities. If automated killings etc is illegal then it would be covered by that one rule.
I don't think Anthropic should be in the business of deciding what is "evil".
Everyone SHOULD continuously consider, decide, and live by moral judgements and codes they internalize, and use to make choices in life.
This aspect of life should NEVER be outsourced — of course, learn from and use codes others have developed and lived by — but ALWAYS consider deeply how it works in your situation and life.
(And no, I do NOT mean use situational ethics, I mean each considering, choosing, and internalizing the codes by which they live).
So, yes, Anthropic and anyone else building products absolutely should be deciding for themselves what they will build, for what purposes it is fit to use, and telling others about those purposes. For products like AI, this absolutely includes deciding what is "evil" and preventing such uses.
If the customer finds such restrictions are not what they want, they ARE FREE to not use the product.
This is easy imo. Two methods:
1. The law. It should not be legal for the US Govt to murder people at will. If it is legal, then of course they'll use tools to make it easier. Maybe AI, maybe Clippy. If they can't use AI then they'll fall back to using some other way of doing it like they've already been doing for several years.
2. Voting. For representatives that actually represent us and have our interest in mind rather than their own corrupt interests. And voting with our wallet against companies that do legal but morally bankrupt things.
Of course we're failing both of these hard right now. But imo the answer is not to give up and let corporations make the rules.
In other words, if it were legal for a normal citizen to murder anyone they wanted, of course they'll use Google Maps to help them do that. We don't put restrictions on how people can use Google Maps. Instead we've made murder illegal. We should be doing the same thing here.
And this AI software is not a mere static object like a hammer that can be handed off to a customer and what it is used for is their business, to build a house or bash a living skull.
This is a system that must be constantly maintained by it's builders.
Moreover, even if we use your standard, the law, it has already been decided in Anthropic's favor.
What you require is that Anthropic actively participate in activities that they consider abhorrent and/or unwise. SCOTUS has already ruled that a business cannot even be required to sell a cake to someone if it does not like the intended purpose (in that case, at a celebration at a gay wedding).
I support Anthropic here. They had a deal with the Govt and the Govt bullied them. That should not be allowed, and Anthropic is suing which makes sense to me. Anthropic should be allowed to set any terms of use for the product that they want, and gain or lose business based on those terms. That's fine.
I'm saying that the failure is actually upstream. It should not be possible for Anthropic's AI to be used to mass surveile or murder people, because those things should be illegal by law and the govt should not be allowed to do it and should not be doing it. Somehow it isn't this way though.
So now that we find ourselves in this failed state, we have to rely on Anthropic to be "the law": to identify what what's "evil" and disallow it. I'm saying that's out of scope for a tech company and they shouldn't be expected to do that. They should only be in the business of making good tech and then be free to let it be used by anyone for any purpose that that the law allows.
This also means that if it's illegal to share information on how to build a bomb without AI, then it should be illegal for Claude to share that information with AI. So Anthropic to does need to make sure they're not breaking the law themselves as well.
For sure, Anthropic should NOT have been forced to decide the ethics of deploying their tech
Nevertheless, they should always be considering the ethics of their own creations and actions, and it seems they are — as soon as they got bullied by a failing regime, they had the right answer: 'no, that is not ethical and we won't allow it with our products'.
The problem is that the law only very roughly captures what is right and just, so there are many things that are legal that are unethical, at the same time there are many things that are ethical but illegal. So, we can't entirely outsource our personal or corporate ethics to the law.
Nevertheless, it wasn't lawmakers, it was car makers who innovated to build-in airbags and seatbelts and lane assist and and and ... under the theory that though it's illegal, bad things are done anyway, and guardrails still matter.
Colloquialism: "belt and suspenders".
Many, like Volvo, go above and beyond the requirements to make their vehicles safer, and then having demonstrated these guardrails, some become law as well (even as other makers in the industry kick and scream about being forced to, and riders rebel against buckling up).
As we haven't solved this stand off for a century, we are unlikely to resolve it within the pace needed by expansion of AI. In this scenario, Anthropic is Volvo.
Although we shouldn't let that mean we misjudge what we are actually getting.
For marketing or personal stuff I do sometimes want images, but I don't really mind going somewhere else for that
The results are laughably bad.
Sure, it does get some of the tones and features, but any kind of actual real-world constraint is so far off, and the dimension indicators it includes are hilarious if they weren't so bad.
It's perfectly possible that 'truly evil purposes' were the goal all along. Slogans and ethics departments are mere speed bumps on the way to generational wealth.
And the weirdest thing that I noticed: instead of skimming the response to try finding what was relevant, I just straight up read it. Kind of felt like I got a slight amount of focus ability back.
Accuracy is something I can't really compare yet (all chatbots feel generally the same for non-pro level queries), but so far, I'm fairly satisfied.
On the contrary, it's great. It's fully capable of outputting a wall of text when required, so instead of feeling like I'm talking to something that has a minimum word count requirement, I get an appropriate sized response to the task at hand.
It works but not as well as I'd like -- the tone and word choice still ends up being really jarring to me (even after years of using ChatGPT). Maybe that's promptable too. Open to suggestions.
---
Respond in a natural conversational style. In terms of language, match my own tone and style.
Keep responses to half a page or so max. (Use context and your judgment. e.g. for example, initial response can be a page, and then specific follow up questions can be shorter, if the question is answered clearly)
Prefer minimal formatting. Don't use headings, lists etc. Bold and italics OK but keep it tasteful.
If you're starting a paragraph like so
Item name: description..
then it makes sense to bold item name for readability purposes.
I made one for Crush a while ago.
https://anduil.neocities.org/blog/?page=mcp
I'm not sure about the issues with reddit though? Do they block Claude's web fetch tool? I think Codex runs it thru some kind of cache proxy.
Sites like Reddit are blocking AI providers, they have to have some contract with them for access. OpenAI does seem to have that.
When sites are working in one chatbot and not another, there's a good chance that the latter is respecting the website rules. As an example with Reddit, you're probably blocked when using a VPN like Mullvad
You can store the page as markdown for future sessions, mash the data w other context, you name it.
The web Claude is incredibly limited both in capability and workflow integration. Doesn’t matter if you’re dealing with bids from arbor contractors or researching solutions for a DB problem.
Maybe I run an old PC adjacent to the scraper to manually visit the scraped pages without an adblocker, & buy something I need from an ad periodically (while a cohesive response is being generated in the meantime)
Ya sounds dumb, wishing for a middle ground that lets us be effective but also good netizens. Maybe that Cloudflare plan to charge the bots…
I never really used ChatGPT much though so maybe Claude is just relatively less egregious?
Wikipedia articles on demand are great, but not usually what I want.
Apparently this annoying "next step" behavior is driven by the system prompt, since the other day I was running Gemini 3 Thinking, and it was displaying it's thoughts which included a reminder to itself to check that it was maintaining a consistent persona, and to make sure that it had suggested a next step. I'd love to know the thought process of whoever at Google thought that this would make for a natural or useful conversation flow! Could you imagine trying to have a conversation with a human who insisted on doing this?!
In persona I think the AIs are that Claude is the engineer and Gemini is the sales-person and GPT is the eager and loud journeyman.
Sure the computer won't get mad, and that is all I do - just ignore what Gemini is suggesting and pretend it never said it - but it certainly makes me mad. The main reason I stick with Gemini is because of the generous free usage limits, but I know this annoying "next step" behavior (which was a relatively recent change) is going to push me back to Claude, even if I need to pay for it.
Over the last few iterations of Sonnet and Opus, anthropic has definitely trained me to ask it to explain something "in detail" (or even "in great detail") when I want as much nuance as possible.
It used to be the inverse - way too much detail when I didn't want it.
Of course, also OpenAI being ran by openly questionable people while Dario so far doesn't seem nowhere near as bad even if none of them are angels.
Moving back to doing this archaic thing called using my own brain to do my work. Shocking.
Though tbh I hardly feel Claude is innocent either. When their safety engineer/leader left, I didn't see any statements from the Anthropic team not one addressing the legitimate points of his for why he left. Instead we got an eager over-push in the media cycle of "Anthropic standing up to DOD! Here's why you can trust us!"
It's all sounds too similar to propaganda and astroturfing to me.
One day I'd like to create a server in my basement that just runs a few really really nice models, and then get some friends and CO workers to pay me $10 a month for unlimited access.
All with the understanding that if you hog the entire server I'm going to kick you off, and if you generate content that makes the feds knock on my door I'm turning over the server logs and your information. Don't be an idiot, and this can be a good thing between us friends.
It would be like running a private Minecraft server. Trust means people can usually just do what they want in an unlimited way, but "unlimited" doesn't necessarily mean you can start building an x86 processor out of redstone and lagging the whole server. And you can't make weird naked statues everywhere either.
Usually these things aren't issues among a small group. Usually the private server just means more privacy and less restriction.
I switched not because I thought Claude was better at doing the things I want. I switched because I have come to believe OpenAI are a bad actor and I do not want to support them in any way. I’m pretty sure they would allow AGI to be used for truly evil purposes, and the events of this week have only convinced me further.