Thanks, I will check it out. Good luck with the project!

This looks really cool! I've on a search project as well, that looks like it has the same goal, and uses a similar approach and philosophy.

... until you want to search for something verbatim with quotes in it. Google doesn't seem to respect escaped quotes like "hello \"this\" is a test" which is quite surprising to me considering how they invent programming languages and all.

I imagine this is because the punctuation isn’t in their index at all, so it makes no difference how you format the query.

Is there an whole-Web search engine that isn’t a fulltext search engine, but rather works more like grepping a virtual text file that contains the entire Internet, such that one could match on punctuation and so forth?

I don't know ... would you be willing to insert a $1000 bill every time you wanted to make a full pass over such a corpus?

They could execute the query on the index and then filter before serving.

I'm sure a thousand engineers at Google can figure this out.

You think it's practical to run the query on the index, materialize the full result, re-index it, run the full query string, and rank? This suggests you do not understand the scale of the problem.

You don't need to re-match or re-index on the full set of results to display just the first page.

I fully understand the scale of the problem. I also think this is a very approachable problem for a thousand Google engineers who have done a lot harder things.