however, there are increasingly such written in C++ (pisa) and rust (tantivy). They handily beat lucene in benchmark suites [1] - so it seems like lucene does suffer from a java penalty - despite getting even more developer attention than pisa and tantivy I would think.
Most of the libraries you mention don't come close to even implementing a tiny portion of Lucene. Classic case of apples and oranges. Also, good examples of the niche things I was talking about. Benchmarks like you mention are kind of self serving like that. They measure something but not everything and probably very selectively. They are faster at what exactly? Under what circumstances? Why? As soon as you answer those questions, what inherent limitations do the Lucene developers have replicating that?
A lot here boils down to how the underlying search engine implements tokenization, stemming, language analyzers, fuzzy matching, and a few other things that you'd need to build a search engine that doesn't suck. The benchmark conveniently does not specify any of that; presumably because it lacks many of those features or has extremely naive implementations of those things. That would be the kind of corner cutting I was talking about. What kind of relevance ranking is being used here? How good is it? Was that even evaluated or considered? Hint, Lucene gives you many options here. Search quality and performance are the big trade off here. Anything that trades off quality over performance is going to look good until you look at the quality.
Also things like the index size and document volume are not specified. Or the hardware this ran on. Or the JVM configuration, compiler flags, etc. It's like benchmarking a formula 1 car by how quick it is at parallel parking. Yes, a T-Ford is going to be faster at that maybe. But is that even meaningful to look at?
But for core text search and indexing - tantivy does indeed support tokenization, stemming, fuzzy matching, and so on. For the core use case of performing a text search on a large corpus of text a lot of the functionality one would need is there - as you can see if you look at the list of queries in the benchmark. From the list of queries there's little I would miss from our "enterprise" use of lucene today - speaking for myself.
One might argue "oh but if you add some obscure features that 5% of people use like a real enterprise solution then it will slow down by 2x" - but I doubt it.
To me, tantivy and the likes are a technology demonstrator - they show that lucene could be significantly faster if it wasn't in java.
Lucene is a Swiss army knife for building you something to get you the best results possible. Those aren't enterprise features if you actually care about what your users do with search results. For example because your sales are directly correlated to that. The trick with Lucene is to do as much as you can possibly get away with as opposed to doing as little as you can do to call it a day that seems to be the strategy here for proving X is faster than Y with this particular benchmark. It's faster because it does less useful work.
The proof of concept here would be building something that is as good and as fast. By not even trying to benchmark how good things are, you kind of make the point that you either don't know or don't care enough to even bother to measure it.
And since the article is called "why C is faster than Java" and the main argument is literally "here's this thing implemented in C that is fast", it's an excellent counterpoint to go "here's this other thing in Java that is fast".
A historical accident isn’t a counterpoint.
I am not a big Java fan. We are not talking a Haskell or Rust or similarly safe language here. However, Java is still far ahead of C in that regard.
For these use cases, you could swap out lucene for something faster or more “low level” in implementation without affecting the typical application using it much, as all interaction happens across some rest API.
If you replace a more or less safe implementation, with a buggy, memory unsafe C version of it, you are not going to make anyone happy about the performance improvement. Always assuming, that you put in "only" the same amount of time and number of people, not more, than the original implementation.
The amount of things you need to keep in mind with all the memory management and other details when writing C code can also make you blind to other issues like proper input validation and implementation of invariants of application settings. You have to put a lot of energy into making sure you are not leaking memory and avoiding memory safety issues, so you lack that energy when it comes to higher level issues. This is directly affecting developer productivity.
But the point at issue is that by using java you are trading off speed (potentially for other benefits).
By the way - tantivy at least is implemented in rust so it shouldn’t suffer from memory safety issues just as lucene shouldn’t. And the rust language will provide some guarantees that java doesn’t - like freedom from data races.
Actually, there is [1]. Clucene has been around for years, it is a re-implementation of Lucene in C++, and yet the only serious usage I've seen was it as a file indexing engine in KDE many years ago (although I might miss other projects). Later it was dropped from KDE.
I've had (indirect) relations with search/indexing stuff for years, and no one ever complained about Lucene's speed or had any problems with it.
The debate was always about idiomatic Java vs. idiomatic C.
What’s amazing to me is that I’ve had such discussions 15 years ago, 10 years ago, etc. I would have hoped that the Java community would have accepted the reality by now :-)
Still, there’s also an interesting new development happening in this area: the members of the Rust community have picked up the mantle of native vs. managed performance. By now they’ve spent so much social capital on “rewrite it in Rust”, that it’s not clear if this will be a positive for native dev advocacy, but the show will definitely be fun to watch.
I've rewritten a few things in rust. Seems pretty positive to me, because you can mix some of the best optimizations and data structures you'd write in C, with much better developer ergonomics.
A few years ago I wrote a rope library in C. This is a library for making very fast, arbitrary insert & delete operations in a large string. My C code was about as fast as I could make it at the time. But recently, I took a stab at porting it to Rust to see if I could improve things. Long story short, the rust version is another ~3x faster than the C version.
https://crates.io/crates/jumprope
(Vs C equivalent: https://github.com/josephg/librope )
The competition absolutely isn't fair. In rust, I managed to add another optimization that doesn't exist in the C code. I could add it in C, but it would have been really awkward to weave in. Possible, but awkward in an already very complex bit of C. In rust it was much easier because of the language's ergonomics. In C I'm using lots of complex memory management and I don't want to add complexity in case I add memory corruption bugs. In rust, well, the optimization was entirely safe code.
And as for other languages - I challenge anyone to even approach this level of performance in a non-native language. I'm processing ~30M text edit operations per second. A few years ago I tried something similar in JS and got about 100k edits per second. (300x slower)
But these sort of performance results probably won't scale for a broader group of programmers. I've seen rust code run slower than equivalent javascript code because the programmers, used to having a GC, just Box<>'ed everything. And all the heap allocations killed performance. If you naively port python line-by-line to rust, you can't expect to magically get 100x the performance.
Its like, if you give a top of the line Porsche to an expert driver, they can absolutely drive faster. But I'm not an expert driver, so I'll probably crash the darn thing. I'd take a simple toyota or something any day. I feel like rust is the porsche, and python is the toyota.
The same algorithm not going to run faster in Rust than tuned C, period. Just like C is never going to run faster than tuned Assembly. I wish we could move on to discussing something that matters.
u/josephg's point is the that implementing equally performant code in 'C' was cost prohibitive for them.
This is applying the principle behind "economics of compiler optimizations" writ large. Not a religious battle.
Weirdly enough, before I added that optimization my rust code was ~20% faster than the C code anyway. And I have no idea why - the programs were (as far as I could tell) identical. I was using the same compiler backend (llvm) and before alias analysis was turned on for rust. And -march=native in both cases.
Could just be a weird coincidence of the compiler’s inlining decisions - though I suspect not. I tried investigating it but I don’t understand x86_64 assembly enough to understand how the binaries differ.
From a very quick look, the sample code prints 'r' instead of 'str' which was confusing. Also, is 'str' leaked or freed with 'r'?
After writing a lot of rust code this year it’s sort of weird reading that, and seeing that aspect of the code not be explicit or obvious.
Eg. would a program with very diverse code paths improve that much from fitting data into L1 cache better? I am genuinely asking it. Because of course for things like pipewire handling audio streams it is very important and Java would not be a good fit. But for a web application I really don’t see C beating Java by much if by any (and the techempower benchmark shows native languages at top only when those are heavily optimized very specifically eg. depending on the exact format of HTTP requests)
In my experience, C#/.NET is in a really interesting position when it comes to these questions. It does support value types out of the box which can add another order of magnitude over Java implementations of the same code. One example of this is the LMAX Disruptor. This was originally written for Java and then ported to C#. Because C# supports value types, a special ValueDisruptor variant was developed specific to its port, which enables performance that would otherwise be impossible in the Java solution (which is constrained to using reference types in the buffer).
https://medium.com/@ocoanet/improving-net-disruptor-performa...
I've worked at two fintech companies, a prop shop and an investment bank and seen some APIs for other companies' matching engines and they've pretty much all been C/C++.
The article talks about hacks used to get around garbage collection in Java and increase performance, including memory mapped files and byte buffers. It also talks about the pain points of using these hacks and the costs they have to pay despite these hacks.
It also concludes that it is practical to build Git in a higher level language and JGit performs reasonably well, despite not getting quite C level performance or its memory utilization.
Frustratingly, most advice I get is non-actionable, and amounts to restating general principles like “be aware of cache”. Are there any books or online resources that can help me build a more principled approach to runtime performance?
Interestingly, I've found that studying other engineering disciplines outside of software engineering to be best. The best explanations and modeling frameworks for concurrency I learned were from network engineering books and a couple hardware design classes I took.
Re: principles around performance analysis and diagnostics, Brendan Gregg is a name you should look up. His book on systems performance is a tremendous resource in both principles and methodology. Even though it's not focused on coding specifically, the same principles apply.
It's a very powerful concept, and there really isn't all that much beyond it. That's the problem with pretty much all of the deep truths about computing, they're frankly pretty simple once you grok them.
But let's at least restate that well: https://www.youtube.com/watch?v=CSqbjfCCLrU https://www.youtube.com/watch?v=C6EWVBNCxsc
How do they cast raw memory pointers into Java types? In C, you just cast a void pointer to a pointer to intended type, and voila - that piece of memory is interpreted as if an object of that particular type resided there. Is anything of this sort possible in Java?
Most of these efforts are relatively niche compared to Lucene because they never quite catch up in terms of features, scale, etc. and because the programmers involved are messing around on the fringes of the problem space instead of coming up with algorithmic breakthroughs, which at this point is the quickest way to make things faster. Well, that and cutting some major corners and pretending it's all the same by running some silly benchmark.
The fallacy here is confusing language, idioms, memory models, frameworks, etc. and assuming it's all set in stone. It isn't. Just because you are using Java does not mean everything has to be garbage collected, for example. Lucene actually uses memory mapped files, byte buffers, etc. for a lot of things. So, it does not actually need to do a lot of garbage collecting. It uses the same kinds of solutions you'd pick when using C. And they perform ballpark as you'd expect them too meaning that unless you improve the algorithms, you are not going to be magically a lot faster. The same is true of hotspot, the JVM's runtime compiler, which is written in C and, surprise, uses a lot of the same kind of trickery used by the fine people working on e.g. LLVM. So, a lot of things people assume must surely be slower just aren't and a lot of the insurmountable bottlenecks that people assume must surely be there always actually have well known ways of being worked around.
Of course there is always room for more optimization. Lucene is well over two decades old now and they still regularly come up with major performance improvements. There's nothing magical about how they do that; just a lot of hard work that goes into it that would be somewhat challenging to improve on just by switching language and compilers.