And I think he left out the most important part, was the answer actually right? The real value of any good dev at all is that he can provide reasonably accurate analysis with logic and examples. "Could have an error" is more like a compiler warning than the output of a good engineer.
Side note: "broke the benchmark script?" If you have an automated way to qualitatively evaluate the output of an LLM in a reasonably broad context like code reading, that's far bigger a story.
Wouldn't you have the AI annotate it?
Any tips on how should I get codebases and real world PRs? Are the ones on popular open source repos on GitHub sufficient? I worry that they don't really accurately reflect real world closed source experience because of the inherent selection bias.
Secondly, after getting all this, how do I evaluate which method gave better results? Should it be done by a human, or should I just plug an LLM to check?
Perhaps if they're some day augmented by formal methods, that might change.
I may accidentally have been inspired from your message when I wrote the following piece, yesterday: https://yoric.github.io/post/formal-ai/
Sigh.
Like cloud-scale, no-code scale, or NoSQL scale? You are confused, which shows that, maybe, you should not be using such tools with the experience that you don't have.
That is the dumbest statement I have heard this week. You should perhaps refrain from commenting, at-least until you gain the modicum of intelligence that you currently don’t have.
Don't need to switch to a different repo for quick test, just make it reproable on your current repo.
As opposed to what, yet another beginner React app? That’s what everyone seems to be testing with but none of the projects I’ve seen are reflective of a production codebase that’s years old and has been touched by a dozen developers.
Throw it at a complicated non-frontend mixed language repo like cxx-qt [1] or something, preferably where the training data doesn’t include the latest API.
That is the reason LLM's in their current shape are pretty useless to me for most tasks.
They happily mix different versions of popular frameworks, so I have to do so much manual work to fix it, I rather do all by myself then.
Pure (common) math problems, or other domains where the tech did not change so much, like bash scripts or regex are where I can use them. But my actual code? Not really. The LLM would need to be trained only on the API version I use and that is not a thing yet, as far as I am aware.
No need to be so hostile.
Let me spell it out for you. These results. Are. Not. Worthless.
Certainly what you said is correct on what he “should” do to get additional data, but your tonality of implying that the results are utter trash and falsely anthropomorphizing something is wrong.
Why is it wrong? Imagine Einstein got most things wrong in his life. Most things but he did discover special and general relativity. It’s just everything else was wrong. Relativity is still worth something. The results are still worthwhile.
We have an example of an LLM hallucinating. Then we have another example of additional contextual data causing the LLM to stop hallucinating. This is a data point leaving a clue about hallucinations and stopping hallucinations. It’s imperfect but a valuable clue.
My guess is that there’s a million causal factors that cause an LLM to hallucinate and he’s found one.
If he does what he did a multitude of times for different topics and different problems where contextual data stops an hallucination, with enough data and categorization of said data we may be able to output statistical data and have insight into what’s going on from a statistical perspective. This is just like how we analyze other things that produce fuzzy data like humans.
Oh no! Am I anthropomorphizing again?? Does that action make everything I said wrong? No, it doesn’t. Humans produce correct data when given context. It is reasonable to assume in many cases LLMs will do the same. I wrote this post because I agree with everything you said but not your tone which implies that what OP did is utterly trivial.
Their comment is "do it consistently, then I'll buy your explanation"
He didn’t literally say it but the comment implies it is worthless as does yours.
Humans dont “buy it” when they think something is worthless. The tonality is bent this way.
He could have said, “this is amazingly useful data but we need more” but of course it doesn’t read like this at all thanks to the first paragraph. Let’s not hallucinate it into something it’s not with wordplay. The comment is highly negative.
> Anything beyond that is your personal interpretation.
Not true. In the context of that post when something is implied that it only seems amazing it also implies that it is likely not amazing. That is a common human interpretation. I find if you’re not interpreting it that way your emotions are influencing your interpretation.
> Do you disagree that there is an excess of cherrypicked LLM examples getting anthropomorphized?
I disagree. I think everybody is doing the opposite and cherry-picking the instances where the LLM gets stuff wrong and saying LLMs are garbage because of that and ignoring the instances where it gets shit right and classifying that instance as regurgitation.
You’re not being rational here by bringing up anthropomorphization. We are mainly talking about correctness. If an aspect of LLM intelligence is similar to humans in generating correct data then anthropomorphizing it is the way to go. Whether it’s anthropomorphizing something is completely orthogonal to the problem. We are talking about correct results. If anthropomorphizing something gets us there… who cares? Human intelligence works and it’s even reasonable to believe LLMs think like humans because they are literally trained on human data.
The act of even mentioning anthropomorphization without mentioning correctness is irrational and illogical.
The comparison to how a senior dev would approach the assignment, as a metaphor explaining the mechanism, makes perfect sense to me.
> We are groking how to utilize them.
Indeed.
The fact that these tools have extremely weird and new to the world interfacial quirks is what the discussion is about…
Versus how no publicly-available AI can do it consistently (yet). Although it seems like a matter of time at this point, and then work as we know it changes dramatically.
After some time humans would gather some background info needed to be more productive and we need to find out how to copy that.
Humans who make lots of mistakes with confidence that they aren't mistakes usually get fired or steered into a position where they can do the least amount of damage.
It's not that AI needs more background info for this type of of thing. It needs the ability to iteratively check it's own work and make corrections. This is what humans do better.
Most other related issues of models these days are due to the tokenizer or poor choice of sampler settings which is a cheap shot on models.
LLMs can generlaly only do what they have data on, either in training, or instructions via prompting it seems.
Keeping instructions reliable, is increasing and testing, appears to benefit from LLMops tools like Agenta, etc.
It seems to me like LLMs are reasonably well suited for things that code can't do easily as well. You can find models on Hugging face that are great at categorizing and applying labels and categorization, instead of trying to get a generalized assistant model to do it.
I'm more and more looking at tools like OpenRouter to allow doing each step with the model that does it best, almost functionally where needed to increase stability.
For now, it seems to be one way to improve reliability dramatically, happy to learn about what others are finding too.
It seems like a pretty nascent area still where existing tooling in other areas of tech is still figuring itself out in the LLM space.
It's easy to get LLMs to do seemingly amazing things. It's incredibly hard to build something where it does this amazing thing consistently and accurately for all reasonable inputs.
> Analyzing authentication system files:
> - Core token validation logic
> - Session management
> - Related middleware
This hard coded string is doing some very heavy lifting. This isn't anything special until this string is also generated accurately and consistently for any reasonable PR.
OP if you are reading, the first thing you should do is get a variety of codebases with a variety of real world PRs and set up some evals. This isn't special until evals show it producing consistent results.