ajeet
I agree with the framing that scaling a language is about scaling to more engineers easily.
Having said that, you can write scalably in any language, and write unscalably elsewhere. This is a bigger factor on scalability overall by far, than the language itself.
For example, I can make unmaintainable macro magic in rust. I can avoid dynamic pitfalls by using pydantic effectively in python.
Anyway, my 2c is speed to delivery of features matters more than scalability as defined here. Again, that depends more on technical components that have been invested in, rather than the selected language.
The goal of my library is to enable quick data mining on wikipedia. Search is just one use case. As an example, you might want to build a content classifier to automatically categorize web pages into wikipedia categories (like politics, sports, etc). To go about doing this, you would need to parse wiki pages and extract features (like n-grams) for a particular category. The C library transforms plain wiki text to a parsed object, that you can use to extract what information you want. The only advantage is that it does this incredibly fast.
Ask HN: Interested in a Wikipedia parsing tool for data mining?
I recently developed a C library that parses Wikipedia, with the goal of being _fast_. For example, I was able to parse and extract text from the entire wikipedia dump (~35 GB uncompressed) in under an hour (on a 5yr old iMac).It is a work in progress. If there is sufficient interest, then I will clean up the code and put up some documentation.