The idea is that you give the libraries and APIs as context with your prompt.

There's a fairly low ceiling for max context tokens no matter the size of the model. Your hobby/small codebase may work, but for large codebases, you will need to do RAG and currently it's not perfect at absorbing the codebase and being able to answer questions on it.

Thank you. But that doesn't work for me.

If you mean just the name of the version in the prompt? No way.

If you mean all the libary and my code in the contextwindow?

Way too small.

Maybe the LLM could issue queries to fetch parts of your codebase as it needs to look at them, using something like GDB or cscope.

Not _all_ the code. Just the relevant parts.

Play around with projects in Claude for an hour. You'll see.

Give it examples of the library being used in the way you need.

Here's an example transcript where I did that: https://gist.github.com/simonw/6a9f077bf8db616e44893a24ae1d3...

Thank you, I experimented in that direction as well.

But for my actual codebase, that is sadly not 100% clear code, it would require lots and lots of work, to give examples so it has enough of the right context, to work good enough.

While working I am jumping a lot between context and files. Where a LLM hopefully one day will be helpful, will be refactoring it all. But currently I would need to spend more time setting up context, than solving it myself.

With limited scope, like in your example - I do use LLMs regulary.

We are getting closer and closer to that. For a while llm assistants were not all that useful on larger projects because they had limited context. That context has increased a lot over the last 6 months. Some tools will even analysis your entire codebase and use that in responses.

It is frustrating that any smaller tool or api seem to stump llms currently but it seems like context is the main thing that is missing and that is increasing more and more.

I have not kept up, can you recommend something?

My review of 2024 is a good place to catch up on what's changed in the past 12 months: https://simonwillison.net/2024/Dec/31/llms-in-2024/

That post is the best summary I've seen of what happened in LLMs last year, but what's crazy is that it feels like you wrote it so long ago, and it's only been four weeks! So much has changed since then!

Do you mean mainly deepseek, or did I missed something big?

Mainly DeepSeek, but also the fallout: a trillion-dollar drop in US stock markets, the new vaporware Qwen that beats DeepSeek, the apparent discrediting of US export controls, OpenAI Operator, etc.

...Trump speaking about DeepSeek: https://www.youtube.com/watch?v=mKZ6UdVn-oQ

Oh, and apparently Kimi k1.5 and the supposedly-half-trillion-dollar "Stargate" announcement.

I am working on something even deeper. I have been working on a platform for personal data collection. Basically a server and an agent on your devices that records keystrokes, websites visited, active windows etc.

The idea is that I gather this data now and it may become useful in the future. Imagine getting a "helper AI" that still keeps your essence, opinions and behavior. That's what I'm hoping for with this.

I am not sure if this was sarcasm, but I believe big data was already yesterday?

Not sarcasm. This is more a reaction to big data. Here's an analogy: Imagine cloud providers like iCloud, Google Drive, OneDrive etc. As a reaction to those, Owncloud and Nextcloud emerged for personal (well, also business) use.

My idea with this is inspired by that. It's just for personal use and to address my own needs.

eh, a hint. i was digging around some thing in these veins long time ago - more like collecting one's notions, not exact low-level actions - but apart of it being impossible back then, i dropped it for this simple reason: if you build such thing, it will know about you much more than you know. And that, in somebody else's hands.. identity theft would seem like walk in the park.

For sure, thank you for that hint. One of the most important things to consider is that something like this can't be misused on someone else, e.g. as a surveillance tool.

I should have clarified, I'm only building this for myself and my own use, there are no plans to take it further than that. Basically, I am trying to learn while building something that satisfies my own needs.