Re:Chunking 2M files a day for code search using syntax trees
by kevinlu1248 on Monday July 31, 2023 @08:47PM
(#36950937)
We used to use line numbers but it became problematic so we switched over to search-and-replace pairs, which works significantly better. The only potential problems are with setting up a fuzzy search system since sometimes the search doesn't match exactly with the code (missing comments, etc.). We're going to write about our core algo and diff managing system soon.
Looking forward to reading through your docs and repo later tonight to see how you’re addressing issues like this.