You have to learn how to prompt tools like Copilot. A few tricks I use a lot:

1. Write comments saying what you want the next lines of code to do

2. Write function definitions with clear function names and type annotations on the arguments - this can result in the full function body being provided if your definitions are clear enough

3. For repetitive code (like parameterized unit tests) provide a couple of examples and then use comments to hint at what it should write for you next based on those examples

4. Sometimes it's good to temporarily copy and paste a chunk of code in from elsewhere. For example, copy a CREATE TABLE SQL statement into a Python file when you are writing code that will interact with that table - or even an HTML page when you are writing out the HTML for a form associated with that table

For 4, this is where I think using the target language(s) LSP / intellesense would be useful. The AI tools should know that I'm referring to a specific tables/types/libraries through references/locality/imports and supply that automatically to the context for me.

Copilot does that in a really clever way already: any time it performs a completion it looks at other nearby files in the project and tries to find pieces of code that look semantically similar to the area you are editing, then includes those snippets in its prompt.

This works well as a general rule, but sometimes it doesn't fine the right snippets - which is why I occasionally help it out through copy and paste.

As described, that seems to miss out on files that are far away through imports, and if not similar to existing code, such as when writing unique code.

My main thoughts behind this are that

1. The LLMs tend to hallucinate library functions

2. I don't want to have to copy and paste a schema

Definitely not disagreeing that this stuff can be done better!

Yeah, needs a lot of plumbing for sure, and I suspect more complex (agent) systems over calling a single LLM

The workflow of having AI suggestions pop up on the rest of the line is really nice. You can ignore when you know what you are trying to write, but the really low-friction interaction of “write a very rigorous comment and then let the LLM autocomplete the implementation” is often enough to solve one-liners in languages where I’m not fluent in the standard lib, which means I don’t need to break flow to go read docs or whatever.

Seems small but I think it’s actually a major productivity win for polyglot programming (which is a lot of my current $dayjob).

I also like the convenience of “start a session with my current file automatically in the context”, again, lowers the friction substantially.

LLMs must be trained for full-in-middle completion to be useful in this scenario, but think “the next stage of autocomplete that uses the context around it” more than “writes entire functions”.

I’ve found it great when manipulating data between two formats, like a CSV export into a JSON config. Something that might be too short to write a script for but long enough to be tedious, you can now tab complete your way through it.

You can specify FIM objective models for completion. I find that Codestral from Mistral works pretty well.

That next stage is currently what I am working. I'm building out a code splitter using TreeSitter right now and already have experimental vector search in the language server.

Do you find yourself having to go to 3 or 4 different files to get everything chat gpt needs for its context to solve the problem? Tools like this can help with that use case.

I use RustRover/VS Code + Codeium or Zed + Supermaven and I have used Copilot before. To be honest it takes some time to get used to the flow. I have turned them off multiple times and finally the workflow has set well in my brain. Now I feel very productive with them.

I work full-time for my own product (very early stage) but I am happy to share my own journey of using AI code assistants. Please feel free to check the commits: https://github.com/brainless/dwata

Auto complete directly inline inside the editor is pretty magical feeling. Sometimes you'll pause for a moment and it will suggest the exact next 3-4 lines you were about to type. You press tab and move on.

Try Cody with gpt-4o and explicit prompting (option+k). It is a nice experience to have the llm at your finger tips instead of having to do round-trips to a different ui.

The models used for autocompletion in Github Copilot and other systems are usually not as strong but faster and cheaper.

You can still get decent results from the autocomplete models if you guide them with comments but I find explicit prompting less frustrating when I care about getting a good result.

I am having to make enhancements to a Ruby codebase where I barely know the language and the code base is completely foreign. So I tried using Cursor to provide AI autocompletion, suggests etc. It's definitely useful for this use case and so I can see the appeal for junior developers.

For my other code bases I find that the amount of time it saves is definitely nice but only barely worth paying another SaaS subscription for.

And so I'm happy to just wait for Jetbrains and Microsoft to roll this into the existing products for free.