https://news.ycombinator.com/newsguidelines.html
We detached this subthread from https://news.ycombinator.com/item?id=29403597.
The key thing to remember is that THERE IS NOTHING WRONG WITH THIS ACADEMIC PROCESS. I go to the Filesystems and Storage Technology (FAST) conference, where many of these BetrFS papers were published, to harvest ideas which I might use in my production systems, and of course, to see if any of the graduate students who have decided that the academic life is not for them, whether they might come to work for my company[1]. I personally find the FAST conference incredibly useful on both of these fronts, and I think the BetrFS papers are super useful if you approach them from the perspective of being a proving ground for ideas, not as a production file system.
So it's unfortunate that people seem to be judging BetrFS on whether they should "trust my data/workloads to such systems", and complaining that the prototype is based on the 3.11 kernel. That's largely irrelevant from the perspective of proving such ideas. Now, I'm going to be much more harshly critical when someone proposes a new file system for inclusion in the upstream kernel, and claiming that it is ready for prime time, and then when I run gce-xfstests on it, we see it crashing right and left[2][3]. But that's a very different situation. You will notice that no one is trying to suggest that BetrFS is being submitted upstream.
A good example of how this works is the iJournaling paper[4], where the ideas were used as the basis for ext4 fast commits[5]. We did not take their implementation, and indeed, we simplified their design for simplicity/robustness/deployment concerns. This is an example of academic research creating real value, and shows the process working as intended. It did NOT involve taking the prototype code from the jJournaling research effort and slamming it into ext4; we reimplemented the key ideas from that paper from scratch. And that's as it should be.
[1] Oligatory aside: if you are interested in working on file systems and storage in the Linux kernel; reach out to me --- we're hiring! My contact information should be very easily found if you do a Google search, since I'm the ext4 maintainer
[2] https://lore.kernel.org/r/[email protected]
[3] https://lore.kernel.org/all/[email protected]/
[4] https://www.usenix.org/conference/atc17/technical-sessions/p...
> Like many research projects, this one will also probably last as long as there is funding. Remember, the goal of PhD students is to publish papers, not develop and maintain software. Thus, without skin in the game, I couldn’t trust my data/workloads to such systems.
Sadly true. For-profit companies only care about $$$. Academia only cares about publishing to get funding.
Both options are not ideal for developing trusted and user-focused software in the long term. OpenSSL is a good example.
No-profits really struggle to get funding. Government grants are a mess.
The world really needs a new approach to R&D.
That's just not true. To do well in academia you have to be truly invested in your field. You can just about get by if you're only in it for the papers, but it's just like getting by in a job that you're only in for the money. At the end of the day, though, in a world where everyone is forced to be productive or be homeless, there are times when publishing becomes a necessity. This doesn't mean they only care about publishing, though.
I never said the opposite. For individuals it takes a huge lot of dedication.
But academia, as a whole entity, is being forced into the publish-or-die mindset.
I am not even sure it wants to be production ready but may be it is a playground for ideas.
Part of our challenge is that we are also exploring non-standard extensions to the VFS API - largely supported by kallsyms + copied code to avoid kernel modifications. This makes rolling forward more labor intensive, but we are working to pay down this technical debt over time, or possibly make a broader case for a VFS API change.
And, the way to get production-ready code is to write a kernel module, with hopes that others in the kernel community will pick it up. Linux certainly didn't start out mature, but you're probably using it now.
All your written-in-production, battle-hardened code with no effete book-larnin' algorithms aren't going to run very well without a functional electricity grid.
It sets the tone with dismissive snark (“find and dandy”), then implicitly asserts that the project is not interesting because it’s not production-ready.
Of course it’s obvious to everyone here that version 0.2 beta software is not production-ready, so obvious that comments to that effect are at best superfluous, at worst annoying.
But its production-readiness is clearly not the focus of the discussion, rather its novelty and potential is. That’s what makes it interesting and worth discussing here.
> looks at extremely practical paper, which rightly won a best paper award, probably for the very reason that it was extremely practical
> decides to have the thread descend into a dismissal of the value of best paper awards on the basis that they do not reward practicality
> That’s all fine and dandy, but [...] Winning best paper awards doesn’t say a thing about the implementation [...] the goal of PhD students is to publish papers [...] I couldn't trust [this]
and a comment like this
> This is a really impressive project. Obviously this is deeply academic, but since I am so impressed, I wonder what the plans are for this (or the same idea in a new fs) to reach the kind of commercial quality where I can use it in a production system.
If you were so aware of the general nature of academic research vs battle-tested implementations, then you would also know that filesystems are so incredibly complicated that the latter invariably takes on the order of 10+ years from a big team to create. When you forget this fact and say that a few-years-old implementation probably sucks because it's from academia, you're ignoring that NOBODY could have made it production-ready in that time, not even Microsoft or Apple or Oracle. Why would you criticise it on this basis? Choosing to do that was the biggest dismissal of the value of the work. Instead, you buried what was in effect a compliment (this would be useful for my production systems) under ten layers of insults.
The jJournaling ideas hit the upstream kernel in 2021 as ext4 fast commits, and no I don't consider it production ready yet. If the fast commits journal gets corrupted, it's possible that the file system will not be automatically recoverable, and may even lead to kernel crashes. I'd give it another year or so before it's completely ready for prime time.
But the other reason for the four year delay between 2017 and 2021 is because I had to find the business justification (and after that, the budget and head count) to invest the SWE time to actually implement the idea. A lot of people want new sexy file system features, but very few people are willing to PAY for them. So part of the job of an open source maintainer is not just to perform quality control and create a technical roadmap, but also to help the developers workin on that subsystem to make business cases to their respective employers to make a particular investment. The dirty little secret is that most people are pretty happy with the file systems as they currently exist; the bottleneck is often not the file system, and while certain file system features are _nice_, they very much aren't critical --- or at least not enough that people are willing to pay the SWE cost for them.
Is this an anecdote vs. data pun? :D
It's not necessarily going to be useful for production use. There are exceptions to this; for example, there are papers were the authors claim that a flash/SSD emulator is suitable for use by other academics to experiment with their FTL ideas, or to grab network traces from NFS traffic so they can be replayed to test file system performance using real world data. In those cases, the point of the paper is to share a tool that can be used by other researchers (as well as the team that created the tool in the first place), and in that case, the code had better d*mned well work. (But even then, there might be buffer overrun bugs in the SSD emulator; which is fine, since the FTL is intended to be provided by an academic researcher, and it is not expected to accept arbitrary inputs including from a malicious attacker.)
I don't know whether the papers in your case were meant to be documentation for code that was meant to be shared, or to explore a particular research idea, the code was only meant for that particular purpose. Even if it was for the former, there's an awful lot of bitrotted, unreliable "abandonware" on sourceforge or github that can be pretty skanky; that's a problem which is not restricted to academically published papers.
I assert that all this leads to people being paranoid about information of subjects well outside their expertise. Which is a really scary place to be. The answer seems non-obvious to me, but is likely nuanced.. and the public doesn't do well with propagating nuance in my experience.
I'm really interested in tooling to help disseminate information.
My career path was 25 years engineering, before migrating into a hybrid EE/PM role as sort of a natural progression from being "the engineer who knew how to run the project". Once I started learning the more formal approaches to PM, it uncovered an entire world of engineers who have an incredible hatred of any sort of planning of any kind, because all planning time is wasted and we should all just be doing.
The parent comment here feels the same way. Hatred towards research because it's all theoretical (I guess?). It seems clear as day that the best approach is a marriage between the two.
I think the origin of this might be on how I (or we) see those people. You're supposed to follow what the doctor tells you, what the scientists tell you. But in a way, since you're supposed to follow what they say, they have some kind of responsibility towards you. And when they say something wrong, it's way worse than when a regular person says something wrong. It's like when you're young and your teacher or your parents are wrong, it's very frustrating.
Your example about the sugar industry is also a great one. Try to understand a bit more about nutrition, and soon you'll hear all kind of conflicting advice and explanations from very different experts.
I know that personally I have to work on myself and accept that those people are humans, and make mistakes, just like me. But just like telling people to eat less and move more didn't solve the obesity epidemic, I'm not sure that this solution will scale to a large population.
Some people seem to confuse expertise for a claim of infallibility, and when some expert get something wrong, the reaction is to conclude that expert advice is worth no more than the guy on the teevee hawking vitamins and anti-expert bile.
It is a sort of Leveler belief wrapped in a search of an Oracle.
The same can be said of medicine where encompasses a very broad set of skills. Your doctor may have been an expert in sports medicine or brain surgery but it doesn't automatically make him competent and epidemiology. It also doesn't force him to pay attention to current developments in the news which is likely what informed your opinion. Personally I found it was completely obvious in January that we would be dealing with a crisis because I followed the situation and suspect strongly that your doctor friend did not.
There is also the issue of survivor-ship bias. We worry about many things and we will absolutely recall the times our worry was justified and forget when it we are mistaken. If Yellowstone ever blows there will be many people who knew it was just around the corner and this will be true if it blows now or in a century, whether or not we have any scientific basis for the thought process.
TLDR: A singular doctor of unknown specialty getting it wrong in January isn't a flaw in science. Science isn't expected to be very good at ensuring a single expert of only tangential expertise gives you the right answer whereas it is reasonable good groups sometimes slowly arriving at increasingly correct answers. If you want a more correct answer consider consulting or reading what several people of relevant expertise who are up to the minute on current information have to say.
There is no paranoia, an allegory is an ivory tower of studies about language from non native speakers with a PhD, and a native speaker who gets no recognition for using it daily but doesn’t have a fancy diploma or credentials so someone who speaks “proper” Spanish from Spain who has never been to Spain is more “credible” than the Mexican speaking “improper” Spanish daily.
Academia is to be ignored unless it’s relevant, Fritz Harber didn’t need the Nobel prize to have real world effects in nitrogen fixing to help farmers grow and sustain our population, Obama wasn’t more relevant because of his Nobel prize, and Perelman’s refusal of the Fields Medal doesn’t change his contributions.
That’s not the case with computer science, at least in systems subfields like filesystems, where theories can be implemented in isolation and shown to either work or not.
Proof of correctness doesn't rely on having a large cohort of test subjects undergoing an experimental trial of some sort, and then interpreting the results with statistical models, distributions, p-values, etc.
I don't know psychology in depth, but if there are similar kinds proofs without requiring statistical analysis of a large experimental cohort, then I don't think Taleb's criticisms are aimed at those either.
It's the fundamental problem of knowledge - can truth be known via logic and reason, or via empiricism and observation? The answer to both is, sometimes, but with caveats.
Peter Norvig also wrote a good take on all the ways studies using experimental cohorts can go wrong: https://norvig.com/experiment-design.html
I never assumed that PhD students can’t code. They can and they are pretty good at that. My point is that their incentives are in writing papers and running experiments that support claims in their papers, not produce reliable software. It might be reliable, but mostly it’s not. When we use tools build by PhD students, it’s usually when there are companies/startups built around it, and that is what I refer to as having skin in the game.
I’ve known people with PhDs in computer science (from a top tier school) that couldn’t code. Their research was all done in Matlab for simulations, modeling a biological process. It was a very specific set of skills required. And at the time, this person couldn’t have written a web front end to a database to save their lives.
Just because one is good at the theory behind CS doesn’t mean they understand software engineering. Similarly, because one is good at the theory doesn’t mean they can’t code.
They are two related, but different, skill sets.
and many graduates in freaking computer science can't read or write proofs, either
> They are two related, but different, skill sets.
exactly.
I think it would be valuable to enforce more crossover in our educational institutions, though. We should have clearer boundaries between computer science and software engineering, and then also require students (at every level) in each to do some study in the other.
Researchers should be in touch with the concerns and needs of ordinary programmers, and ordinary programmers should be capable of looking at the output of researchers to take good ideas and make them practicable and polished.
Sometimes the disconnect means research effort gets wasted and practical technology lingers on designs that serve it relatively poorly.
But of course software engineering and computer science are distinct and deep specializations.
Average programmer here. PhDs in computer science can't code.
Ok, it's an overgeneralization. And it's probably based on a flawed sample of job applicants that make it past HR screening to get to me. The base rate of applicants who can't code is disturbingly high, probably around 20%. (Not that high numerically, but given that they've passed pre-screening and have something impressive-sounding on their resume, it's too high.) The rate of applicants with a PhD in CS who can't code is way higher, probably around 60%.
Note that these tend to be fresh graduates. And it even makes sense -- most theses require just enough coding to prove a point or test a hypothesis. In fact, the people who like to code tend to get sucked into the coding and have trouble finishing the rest of their thesis work, which may start out interesting but soon gets way less fun than the coding part. Often such people bail out with an MS instead of a PhD.
(Source: personal experience, plus talking to people I've worked with, plus pulling stuff out of my butt.)
At the same time, many of the best coders I know have PhDs.
> Implementation and edge cases are the easy part, the hard part is design and algorithms.
Hahahaha. <Snarky comment suppressed, with difficulty.>
I agree that design and algorithms can be hard. (Though they usually aren't; the vast majority of things being built don't require a whole lot.) But the entire history of our field shows that even a working implementation Just Isn't Good Enough. Especially when what you're writing is exposed in a way that security vulnerabilities matter.
Though it's a bit of a false dichotomy. Handling the edge cases and the interaction with the rest of the system requires design, generally much more so than people give it credit for. Algorithms sometimes too, to avoid spending half your memory or code size on the 1% of edge cases.
Developing queueing theory doesn't make you a great coder for Kafka environments.
Working on file system design and new innovative data structures (a persistence and retrieval environment) has nothing to do with writing kernel drivers.
On the other hand, a lot of SE graduates dont want to code, because they think there should be code writing tools and frameworks and infrastructure and process.
They'll spend endless hours talking about those things and focusing on their own needs instead of the actual purpose of the code they're supposed to be developing.
Most production software (esp low level stuff like Kernel, filesystem) today is written and maintained by people having that work as jobs. I wish it was any other way. Also, what users expect from production software is way different than situation 30-40 years ago. An Operating sytem must work for different CPU, GPU. A bare-bones OS is basically a non-starter. I mean look at Haiku-OS or any of other operating system projects, for most part they have gone nowhere.
A filesystem is also fairly complicated piece and what we expect from a filesystem is different. Speed is good but that is not the only criteria and I am afraid it does take serious engineering effort (edge cases and all) to get it usable on today's hardware.
That doesn’t really apply here obviously. The BetrFS team has experienced members.