Sentence embeddings + retrieving context makes sense.

By any chance, have you done any work with indexing code using embeddings? I'd like to do something similar there, but there's no obvious notion of "sentence", especially across languages.

Probably the closest analogue is just lines of code, but breaking lines on newlines might break an expression in the middle removing meaning from both halves.

I was planning on trying indexing overlapping groups of lines but haven't had time yet.