You don't want to use a data serialization library that is so poorly written that it likely corrupts stuff sooner or later, do you? Or a crypto library that documents almost nothing and requires you to review x86 assembler to find how parameters are actually interpreted cough?
Taking at least a cursory glance at the code is one part of vetting dependencies before using them. Other activities would be:
checking use of best practices ("Your parser written in K&R C is probably fine without fuzzing and any tests"),
availability of documentation ("source is there, no?"),
checking for known vulnerabilities and bugs (First open issue: "Library segfaults when parsing long lines of []{}"),
maintainership ("Last commit: 2004") and
portability ("(void*)a + (int)5.7").
Python's CPickle comes to mind. There's a race condition in there somewhere, but I've never been able to reproduce it with a small program. [1] Using the version of Pickle written in Python works.
This sort of thing is why I'm somewhat negative on the argument that "if Python is too slow, write the important parts in C". It's too easy to break Python's memory model in C code.
Edit: Ah that kind of bug can be ... tricky.
If it corrupts data and has terrible documentation it wouldn't be popular. It's usually very obvious from the community around the library if it's got major problems or not.
Length is an important consideration and most never consider it, partly why software is notorious for getting bloated over time. At some point it would be easier to rewrite code from scratch with less complexity and overhead.
On the other hand, McDonald's is pretty reliable and you know you're getting something palatable that is probably safe to eat (in the short term, anyway). You don't need to pick the best-in-class library for everything.
There are some rare cases where that makes sense (Keyword: RARE), but most times I've seen developers take that stance, it leads to longer development times, less consistent code, more maintenance work, and generally pretty shitty outcomes.
Popularity is ABSOLUTELY an indicator of things other than just popularity. However you need to understand why the thing is popular and vet that against your needs.
Finally: McDonalds IS good. I don't eat there often, but they provide a consistently satisfactory experience for their customers. There's a reason you can find McDonalds nearly everywhere on the planet.
It's popular to diss on companies that have become large iconic chains, but they're large iconic chains for a DAMN good reason. Plus, they're solving supply chain issues you aren't even considering, much like large popular libraries are handling use-cases and problems you don't even know about.
Can I make a better burger? Sure. Can I make a better burger for the same that McDonalds charges, as consistently as McDonalds does? Fuck no.
The same probably applies to your software.
McDonald's is consistent, predictable, cheap, generic, and ubiquitous. Sometimes those are good qualities. "Good enough" is usually good enough, after all.
Regionally, there are a half-dozen choices that I'll take over McD's: Less iconic world-wide, but as cheap or cheaper, just as consistent, and more pleasing to the palate. I think that's closer to what they were advocating: Not necessarily building it in-house, but also not taking popularity as a reliable indicator of code quality (beyond a bar of minimum acceptability).
>At some point it would be easier to rewrite code from scratch with less complexity and overhead.
I'd agree, and probably wouldn't have commented.
But I think being all of these things:
>consistent, predictable, cheap, generic, and ubiquitous.
Means you have a damn good product. No one is claiming you can't get something better, but come on, I'd love to have software that's consistent, predictable, cheap, generic, and ubiquitous.
If those words describe a popular library, and you decide instead to build your own thing in-house... you better have a really REALLY good reason for doing it.
Note that I never said "consistently good" or "predictably reliable". There are a lot of things in the world that are known to be low-quality, single-use crap that are still everywhere.
> If it hadn't been for the last sentence:[...]
I think it depends on their idea of where "at some point" lies.
A very good reason to write your own software is if the problem is novel enough to warrant it. I'm not advocating that there should be n+1 JSON serialization libraries.
It doesn't make it good, but it does mean that it's unlikely to make you ill.
The difference between code and a burger is that anyone can consume a burger but code is consumed almost exclusively by professionals. A better comparison would be power tools where you'll often find that if a brand is popular, it tends to make solid, long lasting products.
To rephrase: PHP is the most widely used server-side scripting language on the web. That does not make it good.
Like what? If you're looking for a JavaScript library for example, you can't really go wrong picking one that has many contributors, an active issue tracker, thousands of stars, lots of forks etc. Unless it's some core part of your stack, I don't see how it's practical to spend a lot of time hunting down super well implemented libraries that aren't likely to be supported in the future because nobody uses them.
> To rephrase: PHP is the most widely used server-side scripting language on the web. That does not make it good.
I'm not a huge fan of PHP but you can still write good software in it.
Libraries are relatively easy to swap out compared to say operating systems and programming languages. Personally I find many of the best libraries tend to be the most popular ones.
To beat the average, by definition you can't just do what everyone else is doing and expect outstanding results. If your problem is unique there won't be a library for it. If you find or create a hidden gem, it's your secret weapon, you are its user and responsible for it.
I don't know how helpful reading source is for determining whether or not to use a dependency, but I find it very helpful to have at least a small understanding of how my dependencies work:
1. I find it useful for debugging. If a coworker or I have used a library incorrectly, understanding how the library works can help us figure out why it isn't doing what we wanted. Also, my IDE lets me do step-through debugging through dependency code as well, which can be helpful.
2. It helps with determining the capabilities of a library. If I know how a library works, then I can make reasonable guesses about what features the library has (or could easily add). I find myself thinking, "I'd like to do X, and from what I know of how library foo works, that seems like something it should be able to do." So I can go investigate that.
3. It helps with knowing the performance characteristics of the library. Just yesterday, I needed to implement some stats logic in java. I found an Apache Commons class that does exactly what I need, but by looking at the source code, I found that it uses an O(n) algorithm, which is fine for the cases they are probably targeting. In this case, I need more performance than that, and it's not hard to write this as O(log n). So I wrote my own instead. If my IDE (Intellij) didn't give me super easy access to dependency source code, this would have been more painful.
When you review a dependency, you answer why it should not be written THIS way. (Or why it's junk and shouldn't be used.)
If you want to write secure code it's necessary. Otherwise you just end up with PHP web apps that have remotely exploitable memory corruption bugs because the developer didn't understand that he absolutely has to sanitize all user inputs to certain functions.
>If the code really is horrendous nobody is going to be using it
This just makes it pretty clear that you haven't ever looked.
Do you check the source code for Apache, MySQL, PHP etc.?
If I'm picking a library for a component where its flaws would be high impact (e.g. security) and there isn't a strong community behind that library I'm going to be very cautious though.
Has this ever been beneficial? I generally just check if a library is popular and maintained and that's always been enough for me. If the code really is horrendous nobody is going to be using it and if you always read the hundred of thousands of lines of code you depend on you'd never get anything done.