News for hackers. Stuff that matters.
Re:Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison by gavinray on Monday March 31, 2025 @09:47AM (#43535026)
Convert the GTK 3 and GTK 4 API documentation into a single `.txt` file each.

Upload one of your platform-specific C++ file's source, along with the doc `.txt` into your LLM of choice.

Either ask it for a conversion function-by-function, or separate it some other way logically such that the output doesn't get truncated.

Would be surprised if this didn't work, to be honest.

Re:Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison by pera on Monday March 31, 2025 @10:50AM (#43535694)
Do you really need to provide the docs? I would have imagined that those docs are included in their training sets. There is even a guide on how to migrate from GTK3 to GTK4, so this seems to be a low-hanging fruit job for an LLM iff they are okay for coding.
Re:Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison by iamjackg on Monday March 31, 2025 @10:57AM (#43535785)
You might not need to, but LLMs don't have perfect recall -- they're (variably) lossy by nature. Providing documentation is a pretty much universally accepted way to drastically improve their output.
Re:Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison by Workaccount2 on Monday March 31, 2025 @11:16AM (#43535999)
LLMs are not data archives. They are god awful at storing data, and even calling them a lossy compression tool is a stretch because it implies they are a compression tool for data.

LLM's will always benefit from in context learning because they don't have a huge archive of data to draw on (and even when they do, they are not the best at selecting data to incorporate).

Re:Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison by jchw on Monday March 31, 2025 @12:22PM (#43536778)
In my experience even feeding it the docs probably won't get it there, but it usually helps. It actually seems to work better if the document you're feeding it is also in the training data, but I'm not an expert.
Re:Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison by dagw on Monday March 31, 2025 @12:52PM (#43537121)
Feeding them the docs makes a huge difference in my experience. The docs might be somewhere in the training set, but telling the LLM explicitly "Use these docs before anything else" solves a lot of problems the the LLM mixing up different versions of a library or confusing two different libraries with a similar API.
Re:Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison by baq on Monday March 31, 2025 @01:12PM (#43537318)
It moves the model from 'sorta-kinda-maybe-know-something-about-this' to being grounded in the context itself. Huge difference for anything underrepresented (not only obscure packages and not-Python not-JS languages).
Re:Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison by vasergen on Monday March 31, 2025 @02:16PM (#43537986)
The training set is huge and model "forgets" some of the stuff, providing docs in context makes sense, plus docs could be up to date comparing to training set.
Re:Gemini 2.5 Pro vs. Claude 3.7 Sonnet: Coding Comparison by nurettin on Monday March 31, 2025 @11:56PM (#43542689)
Docs make them hallucinate a lot less. Unfortunately, all those docs will eat up the context window. Claude has "projects" for uploading them and Gemini2.5+ just has a very large window so maybe that's ok.