I have had considerable trouble getting C99 code from any LLM that adheres to some basic safety conventions present in all my code. The biggest issue is generated functions have multiple return paths making it more difficult to visually spot cleanup errors.
My usual code has a single cleanup label that frees all resources and, if error is still set to true, cleans up the return value as well. Even when tired this convention makes it dead simple for me to spot errors.
So I've been writing most of my C projects by hand.
SQL otoh, LLMs are great.
Also in the same vein as SQL, I've seen Gemini is pretty good at generating formulae in Google Sheets; I used it to create an optimal player rota for a 24 hr online endurance race me and my friends were participating in using parameters like player_strength, the time slots they were fine with taking up, in order to min-max fatigue and consistency.
Pretty fun.
If the most critical part is provided by LLM how is least critical part written by you proof of LLM being inferior at writing code?
> If the most critical part is provided by LLM how is least critical part written by you proof of LLM being inferior at writing code?
That quote you quoted seems to say that he uses it to reformat his spec/design documents. Nothing about code.
He further goes on to say he cherry picks the best of the spec/design elements from that documents.
How did you get "It writes my code" from the paragraph you quoted?
Isn't that what i am doing here as well? https://www.reddit.com/r/ClaudeCode/s/Od8CsIkRVI
Approving plan, reviewing after LLM has done its job?
How does writing a few lines manually change anything?
I do not agree with his criticism of vibe coding at all. We don't even have access to any code he has written, so i am not knee on accepting his code superiority over what LLM creates.
On the other hand i am almost willing to bet, he writes worst code I do by just prompting LLM
My point being planning is the most critical part, not actual code writing part (that can be done by dumb model)
I'm still trying to figure out where he says the LLM wrote any code. You're still working on the assumption that he said the LLM wrote code, and he changed a few lines. To me it reads like, I rubber duck specs/plans with the LLM, and then I write the code).
It doesn't seem ambiguous to me
> On the other hand i am almost willing to bet, he writes worst code I do by just prompting LLM
I dunno; the LLM, by definition alone, is going to produce median quality code.
After all, this thread was started by me complaining that it's been very difficult for me to get any LLM, even SOTA ones, to produce C code in a convention that minimises resource errors (use-after-free, memleaks, etc).
It does it for one function. Then for the next one I have to remind it again that it must use single cleanup point.
Putting it in the prompt works sometimes, not all the time, but for it to work then all my prompts have a code example of how to structure C code to ensure a single cleanup point which returns the returned value (which could or could not be an error).
So, no, I've not been getting spectacular results using CC and similar tools for C. Maybe it works better for C++?
What I have been getting good results on is SQL (have to specify the variant, though).
I sometimes think that the LLM cannot tell human-readable code from unreadable messes; hence I get these almost comically shortened C snippets sometimes when (for example) a static lookup table, though longer, would be more readable.
I think the misunderstanding here is what metric we are using for "Quality". I am using "Is this the most readable way to solve the problem?"
Just this morning I tried to get it to write code to dispatch different serialisation functions based on a provider, it gave me a large-LoC monster of structs with vtables for dispatch[1], and some functions to perform the initialisation, then more functions for the dispatching properly.
I asked it to make it simpler; it give me a simpler, but very long, jump table using switch/case on the `type` field in the `struct`. Easier, but still it means that an unintended fallthrough, or a missing `case` is a bug I have to look for when visually inspecting the code for correctness.
I gave up and wrote this (some details changed), which is much easier for me to visually verify that no cases have been missed, no accidental fallthrough, etc:
typedef bool (provider_serialise_func_t) (const some_t *src_some, some_http_t **dst_rqst);
typedef bool (provider_deserialise_func_t) (const some_http_t *src_resp, some_t *dst_some);
struct some_provider_t {
enum some_provider_type_t type;
char *name;
provider_serialise_func_t *serialise_fptr;
provider_deserialise_func_t *deserialise_fptr;
};
// Serialisation/deserialisation functions, defined at the end of the file
static bool serialise_all(const some_t *some, some_http_t **http_request);
static bool deserialise_all(const some_http_t *http_response, some_t *some);
static bool serialise_prov1(const some_t *some, some_http_t **http_request);
static bool deserialise_prov1(const some_http_t *http_response, some_t *some);
static bool serialise_prov2(const some_t *some, some_http_t **http_request);
static bool deserialise_prov2(const some_http_t *http_response, some_t *some);
static bool serialise_prov3(const some_t *some, some_http_t **http_request);
static bool deserialise_prov3(const some_http_t *http_response, some_t *some);
static bool serialise_prov4(const some_t *some, some_http_t **http_request);
static bool deserialise_prov4(const some_http_t *http_response, some_t *some);
static bool serialise_prov5(const some_t *some, some_http_t **http_request);
static bool deserialise_prov5(const some_http_t *http_response, some_t *some);
static bool serialise_custom(const some_t *some, some_http_t **http_request);
static bool deserialise_custom(const some_http_t *http_response, some_t *some);
static const struct some_provider_t g_some_providers[] = {
// NOTE: Order is important, don't switch the order around
{ SOME_PROVIDER_ALL, "all", serialise_all, deserialise_all },
{ SOME_PROVIDER_PROV1, "prov1", serialise_prov1, deserialise_prov1 },
{ SOME_PROVIDER_PROV2, "prov2", serialise_prov2, deserialise_prov2 },
{ SOME_PROVIDER_PROV3, "prov3", serialise_prov3, deserialise_prov3 },
{ SOME_PROVIDER_PROV4, "prov4", serialise_prov4, deserialise_prov4 },
{ SOME_PROVIDER_PROV5, "prov5", serialise_prov5, deserialise_prov5 },
{ SOME_PROVIDER_CUSTOM, "custom", serialise_custom, deserialise_custom },
};
...
// Usage in an actual function
const struct some_provider_t *provider = &g_some_providers[src_some->provider];
And now there is not `switch` statement to verify (it's in the struct, one on a line, easier to read), there's no confusion via multiple indirect derefences to a vtable, etc.So if your metric for quality is "it works and has no bugs", then sure - you would feel that the LLM is providing high-quality code.
If your metric is "How easy is this to code-review, when it's midnight and I have been working all day long at my day job?" then things look a little different.
===========================
[1] Obviously it trained on OO designed C code, like the Linux kernel
6 months late and I know a clean up contractor who is making bank there with no hope of recovering it apparently.
They saw it as a way out from their cheap assery, lack of talent and lowest bidding outsourcer over the years and it’s a dead end. I wonder how many other projects are in this situation.
Here's my LLM coding method https://www.reddit.com/r/ClaudeCode/s/Od8CsIkRVI
Here's my project: https://aretecodex.pages.dev
I've many other projects which I've built with the same method, implementing 10-15 features a day on some
Ofc i cannot show all projects here but those who think LLM coding doesn't work, well....
Also not to shit on your project or anything, but I can whip that up in 10-15 minutes using something like Hugo/Astro, using just their bootstrapping commands.
I'd like to see if you have any examples of things that are hard to do being done by LLMs than just a SSG blog.
Also I never said "LLM coding doesn't work", read my comment. I just meant the cost of maintaining a pure vibe-coded system outweighs its' potential benefits.
You just looked at wiki part, not the tools section it seems: https://aretecodex.pages.dev/tools/
sure you may be able to clone but you lack domain knowledge to actually make these.
And here it clone claude code cli, https://www.reddit.com/r/ClaudeCode/comments/1ngmyqq/i_ran_c...
Show me your projects.
sure you may be able to clone but you lack domain knowledge to actually make these.
And here it clone claude code cli, https://www.reddit.com/r/ClaudeCode/comments/1ngmyqq/i_ran_c...
what do you want me to build? because these are what 90% softare project involve.
Again, none of what you've built is complicated in the least. Your domain knowledge may be commendable, sure, but that's got nothing to do with this discussion.
*> And here it clone claude code cli*
This is where I just gave up trying to understand your thought process, and after looking deeper into your Reddit and HN history you just seem to be a mindless shill for these products and your thought process instead of being able to critically analyze your approach.
Irrelevant to discussion other than you trying to stroke your own ego.
>Again, none of what you've built is complicated in the least.
Making simple things complicated isn't what i am trying to achieve.
>This is where I just gave up trying to understand your thought process, and after looking deeper into your Reddit and HN history
I don't have a single product i am shilling anywhere. I am only showing my workflow.
I doubt you've ever built anything but that's fine. Lot of people who cannot show a single project they've built but will criticize other product ad infinitum.
So far you've added nothing to discussion other than claiming to be a better programmer with no proof.
None of the programs i posted on my reddit are available for public use, they are demos.
Also really whacky implementations, things like every page load, sucking up a bunch of MD files, inserting them into an SQL database, and then querying that database to render the page, so every single page load is a total reconstruction of the database.
That’s just one example of 10 I could give right now.
Vibe coding might get you a janky zero to one, but the problem is that the price that you pay for that speed, is something so tangled, so messy, that the LLMs themselves cannot even get you to “two”
I actually like LLM coding and I find it helpful in many cases, but vibe coding (meaning where people don’t really understand the implementation and just hit accept) generates huge amounts of very low quality code, which isn’t just a matter of aesthetics, but has real product quality and security implications
I've seen the same thing happen when you outsource to third world countries. So which one is worst? LLM can be prompted to write bloated codebase, it can be prompted to write lean one. It depends on skill set of the guy driving LLM.
but I've yet to see actual vibe coded projects hitting the markets.
In a single day, I vibe coded an internal tool for my team to run unit, integration tests. It has a professional looking UI, a few nice to have features that usually never get built, and a server component that connects to our app APIs.In the past, it would have taken me at least 1-2 weeks to build this. It was actually never going to get built because of that. But vibe coding finally let us complete the project.
Whenever I see said projects, on Github or non-technical friends vibe coding their next new startups asking for a code review, all I can see is boilerplate hell, 100s of lines of redundancies, problems that could've been solved using 5-6 procedures being blown out to 4-5 directories with 20-30 procedures, basically over-engineered beyond belief.
The positive part of this is people who would've never thought of programming/writing code are beginning to do so but their confidence is highly misplaced because they simply don't understand what's going on to a certain degree and don't possess the necessary knowledge nor skillset to review the generated code. A successful compile or output on the web browser is not a very good metric for projects.
Great writeup that captures the current state of affairs.
----
Tangential, but hi Nommy :) glad to see you out there, remember competing against you back in school; those web dev comps used to be the shit. Think Exun 2013 was when we went head to head making that fake currency haha.