Things we learned about LLMs in 2024

Re:Things we learned about LLMs in 2024 by simonw on Tuesday December 31, 2024 @07:35PM (#42562983)

I use my https://github.com/simonw/files-to-prompt tool like this:

  files-to-prompt . -e py -e md -c | pbcopy

Now I have all the Python and Markdown files from the current project on my clipboard, in Claude's recommended XML-like format (which I find works well with other models too).

Then I paste that into the Claude web interface or Google's AI Studio if it's too long for Claude and ask questions there.

Sometimes I'll pipe it straight into my own LLM CLI tool and ask questions that way:

  files-to-prompt . -e py -e md -c | \
    llm -m gemini-2.0-flash-exp 'which files handle JWT verification?'

I can later start a chat session on top of the accumulated context like this:

  llm chat -c

(The -c means "continue most recent conversation in the chat").

[ Parent | on HN ]

Re:Things we learned about LLMs in 2024 by layer8 on Tuesday December 31, 2024 @07:58PM (#42563089)

Thanks. Google AI Studio isn’t local, I think, is it? I’ll have to test this, but our project sizes and specification documents are likely to run into size limitations for local models (or for the clipboard at the very least ;)). And what I’d be most interested in are big-picture questions and global analyses.

[ Parent | on HN ]

Re:Things we learned about LLMs in 2024 by simonw on Tuesday December 31, 2024 @07:59PM (#42563096)

No, it's not. I've not seen any local models that can handle 1m+ tokens.

I haven't actually done many experiments with long context local models - I tend to hit the hosted API models for that kind of thing.

[ Parent | on HN ]