This is what having consistent interfaces is for. Keep your interface consistent, not your implementation.
This is also a strong argument for not releasing code and advertising it on HN if it's not mature.
> The add-onlys were thrown together in a week or so many months ago, just to get those classes of data strutures out there, as writing the full versions of them will take some time. There is in fact for eaxmple a full, balanced binary tree, in the literature. That's on the list!
Which begs the question why you included immature implementations in your library.
> That's a bit unfair. The library is eight years old now. It's not a flash in the pan.
Which doesn't negate the criticism in any way.
> It's actually as portable as it is possible for a lock-free library to be. Only a fairly few platforms provide atomic instrinsics. As far as GCC provides support, they are all suppoted, and I believe GCC supports all of them.
There are working examples (ConcurrencyKit) of lock-free data structures that target more architectures and more compilers, so "It's actually as portable as it is possible for a lock-free library to be." is provably false.
I have to disagree. If the underlying library used has changed, then the code which uses the library has to be revalidated. You cannot assume it simply continues to work - to do so is to fully rely on the library provider getting it right, introducing no bugs, or anything unexpected. This is not viable for serious projects.
The and the ONLY way in which existing code does not have to be revalidated is if it is completely and wholly unchanged.
> Which begs the question why you included immature implementations in your library.
There's nothing immature about them, and I don't know why you are saying there is. They were written about a year ago and writing them means also writing their test suite, which they pass.
The choices you disagree with in the library seem to be being taken to justify the charge of "code immaturity". Those choices may be right, or they may be wrong, and so could potentially be mistakes, but this is an entirely different matter to maturity.
> There are working examples (ConcurrencyKit) of lock-free data > structures that target more architectures and more compilers, so > "It's actually as portable as it is possible for a lock-free > library to be." is provably false.
I was thinking of processor support, rather than compiler support. Processor support is the main part of the work - all of the code has to be written with it in mind. Compiler support is much easier; it's just a matter of porting their atomic instrinsics, which basically means writing about a dozen straightfoward macros.
> The and the ONLY way in which existing code does not have to be revalidated is if it is completely and wholly unchanged.
And when I integrate a new release, how do I verify that you didn't accidentally change one of the older numbered versions of your code? I'm going to have to test either way when I integrate a new version of your code, so I might as well get any performance improvements or whatever you introduced into the implementation. Versioning your interface does not solve the problem you claim it does.
> There's nothing immature about them, and I don't know why you are saying there is. They were written about a year ago and writing them means also writing their test suite, which they pass.
I am saying they're immature because you said they were immature. You said, "The add-onlys were thrown together in a week or so many months ago, just to get those classes of data strutures out there, as writing the full versions of them will take some time."
> I was thinking of processor support, rather than compiler support.
I was also thinking of both processor support--you're the only one who mentioned any compilers. Given this supports only 2 processors and ConcurrencyKit supports 5, your claim that this is "actually as portable as it is possible for a lock-free library to be" is still provably wrong.
They're only released once. No release is changed after its release. Once it's out, it's out. The archive is never touched again.
When a release is made, the source code is exported, archived and uploaded to the library web-site. To accidentally change that code, I would have to accidentally repeat that manual process. It's the same with github - I actually use SVN at home, and I make a new repos per release and upload the code, once, to that repos. Github is a release only, write-once source control system, for me.
> I'm going to have to test either way when I integrate a new version of your code
Well, now I'm confused. If that's so, what's the problem? you keep using the old code, it's not changed, either the API or the library binary, so you don't have to test that; and for any new code, you can use the new release version. To the extent the APIs are unchanged, moving fully from one release to another is a matter of issuing a global search-and-replace for the string "liblfdsNNN" to "liblfdsMMM".
> Given this supports only 2 processors and ConcurrencyKit > supports 5
The library supports on Linux almost every processor GCC offers atomic instrincs for - ARM32, IA64, POWERPC32/64, SPARC32/64, MIPS32/64, x64 and x86. I'm pretty sure the library does not support Alpha, however. Of those platforms, I only have access to actual hardware for ARM32, MIPS32 and x64, so only those platforms actually have the test code compiled and run and being seen to pass.
On Windows, support is provided for ARM32, x86, x64 and IA64. Again, of these, I only have access to x64.
1. Let's say I do things the way you suggest: I use different versions of your code simultaneously. 3 years in I have 5 versions of your code and I find a bug in my code in an area from the second version. Now I have to know 5 libraries to understand the code and I'm constantly getting tripped up by the subtle differences between versions 2 and 3.
2. So to fix this plan I finally persuade my boss to give me a week to address tech debt. With your search-and-replace plan, except I forget to do it in the library that Bob the contractor kept in a separate repo before we fired him. And then there's the separate problem of when I passed it into a function that took a function pointer and so the name is different and not find-and-replacable as being the same function. And then there's the place where Joe, who is a good programmer but kind of inexperienced, poor kid, he thought he'd simplify things by creating a macro that inserts the number, and this got past code review because I reviewed it right after a meeting with the client from hell. So now nothing is working after the find-replace, it takes us the entire week to figure out why, and we don't get to spend any of our tech debt time fixing the place where Bob the contractor decided it would be a good idea to "optimize" the graph generation functions.
For the record, I'm speaking from experience here. The way you wrote this code is an ideology I've worked with before and it's miserable to consume libraries written in this style. You can argue for the "benefits" all you want but I've eaten this dog food before and I'm not going to eat it again.
There's actually rather few alive platforms that don't either have ll/sc or cas. The original i386, armv5, sparcv8 (leon IIRC added cas though), pa-risc come to mind (that's the postgres platforms for which fallback atomics support is used). Once you have CAS it's easy to provide fallback implementation for other atomics. It's also not all that hard to provide a "fallback" implementation for atomics, which guarantees atomicity using some locking mechanism.
Intel in that regard are a pain, because they have a mandatory, built-in full memory barrier in their atomic operations. ARM does not, and I see the freelist on ARM running about 25% faster (relatively speaking) than Intel, because of it.
If you look at the first two bars (first is the new GCC atomic instrincs, second the old GCC sync intrinsics) in the one-core chart from these two gunplots, the first gnuplot being ARM32 and the second a Core i5,
http://liblfds.org/pages/images/liblfds710_freelist_push1_th...
http://liblfds.org/pages/images/liblfds710_freelist_push1_th...
You will see on Intel they're level, and on ARM, the atomic bar (the first bar) is about 25% higher.
The new GCC atomic instrincs only issue memory barrier when told to, whereas the old sync instrincs normally (e.g. on most platforms - the docs are a little nebulous) issue memory barriers.
The freelist doesn't need a memory barrier on pop, but on Intel, you get one anyway, and on ARM, with the sync instrincs, you get one anyway. The atomic instrinics also issue on Intel, because Intel forces it to happen, but they do not issue on ARM. I think this gives the 25% performance improvement.
I'm not totally convinced about that, cf: https://en.wikipedia.org/wiki/Release_early,_release_often
Yes and no. The site currently is arranged with a set of tabs on the left and a frame on the right. It had to be a frame because some of the web-based apps, such as the mediawiki used for documentation, generate output encapsulated in the "<HTML>" tag. As such, their output cannot be captured and placed into a div; it has to go into its own frame.
However, the documentation can be opened in a new tab (right click, open in new tab) and then where it has its own browser window, the URL is displayed as normal.
Of course, this is not the default behaviour - the user has to do something to get it, and to realise that it is possible. That is a problem. The only obvious solution is to have the documentation automatically open in a new window. That's a bit unpleasent though - new windows popping up unexpectedly is not good for the user experience.
> The API changes with every release. WTF? Really? Just... really? In order to use a newer version this library, you have to change your entire application for the updated function names.
No. All versions of the library can concurrently be included and linked against. The reason for this is so that any code which using an existing version of the library does not have to be modified at all - and so does not need to be revalidated - when a new version is published.
IME, once code is written, and given that it works, it tends quite strongly not to change. It works, so why spend time and effort merely to change it? however, if a new version of a library comes out, anything could have happened - so existing code has to be revalidated. It's extra work. This is avoided by supporting concurrent use of multiple library versions.
> And many of the lock-free structures are add-only. The binary tree is unbalanced. Which makes it not a very > good binary tree.
The add-onlys were thrown together in a week or so many months ago, just to get those classes of data strutures out there, as writing the full versions of them will take some time. There is in fact for eaxmple a full, balanced binary tree, in the literature. That's on the list!
Regarding being unbalanced, one way to address this is to hash the key before inserting a new node.
> This looks like it was written by somone who heard "lock free", and went "cool, I'll write a library!"
That's a bit unfair. The library is eight years old now. It's not a flash in the pan.
> The web site is bad, the APIs are bad, the functions aren't very useful, the library isn't very portable, > etc.
It's actually as portable as it is possible for a lock-free library to be. Only a fairly few platforms provide atomic instrinsics. As far as GCC provides support, they are all suppoted, and I believe GCC supports all of them.