One cool feature of functional package managers such as Nix and Guix is that the dependency graphs are entirely transparent and can be inspected programmatically.

I wrote a script that lists the number of dependents for each package in Guix by traversing the package graphs:

https://gist.github.com/mbakke/f354272666fbef09c5229f7b85377...

Running it takes about 16 seconds on my laptop, and piping to 'grep -v bootstrap | sort -rn | tail -n 20' gives:

  16695 guile
  16692 ld-wrapper
  14686 pkg-config
  14512 perl
  14444 ncurses
  13135 readline
  13133 zlib
  13022 libffi
  12470 xz
  12448 libunistring
  12433 openssl
  12353 bash
  12050 libxml2
  12016 gettext-minimal
  11950 tar
  11942 bzip2
  11896 expat
  11857 tzdata
  11850 net-base
  11846 python-minimal

(GCC, glibc and binutils are missing for complicated reasons, but should be up there with Guile)

By changing (all-packages) on line 26 to:

  (fold-packages (lambda (package result)
                   (if (string-prefix? "python-" (package-name package))
                       (cons package result)
                       result))
                 '()))

We get the most popular Python projects:

  6672 python-setuptools-scm
  6667 python-nose
  6662 python-pyparsing
  6662 python-more-itertools
  6661 python-wcwidth
  6660 python-py
  6660 python-pluggy
  6660 python-atomicwrites
  6657 python-six
  6593 python-mock
  6587 python-appdirs
  6586 python-lxml
  6584 python-filelock
  6584 python-distlib
  6583 python-sortedcontainers
  6580 python-hypothesis
  6577 python-elementpath
  6576 python-xmlschema
  6575 python-pytest
  6303 python-libxml2

This approach misses "end user" packages such as browsers and QEMU, but provides some insight into high value targets.

For end user packages, Debians "popularity contest" can be useful:

https://popcon.debian.org/

analysing a source-only-based distro (one where the compiler is required as a dependency) should make the importance of the compiler(s) clearer.

source-only distros include macports, gentoo, and so on.

Guix and Nix _are_ source-based distros. They just happen to have a facility for transparently downloading "binary substitutes" for local builds.

The graphs I'm traversing are the build-time dependency graph of all packages. I could also traverse run-time dependency graphs, but then I'd need to actually build all the things, because those are not known up-front unlike Gentoo and Macports.

The reason the toolchain is missing is because it is an "implicit input" as opposed to a "normal" input, so it's invisible/transparent to the query I'm making.

The methodology is pretty silly. It rewards activity and popularity.

A lot of critical infrastructure software is not active and not often spoken about!

Case in point: The https://www.cip-project.org/faq project highlight the needs of very-long-term support for OS components that run on critical infrastructure such as power stations.

The https://www.cip-project.org/faq project is based on Debian. Very little of it is on github.

The activity metric is especially frustrating because it seems to me that the higher quality a project is, the less bugs will be filed, and so the less activity will result.

With one of my open source projects I've got to extreme lengths to test it under a huge number of compiler configurations, so it compiles on anything with no warnings. I get nearly zero bugs filed when people port it to esoteric platforms.

When randomly searching a while back, I found a blog post someone wrote about getting my project running on Arduino [1], something I had never tried. Turns out they just had to set some configuration flags and everything worked fine. They therefore filed zero bugs, and it resulted in zero activity for my project. They didn't even tell me about it!

I was extremely pleased that it worked of course, but disappointed that my project continues to look like it's dead. I imagine Google's criticality score for my project is near zero.

[1]: https://www.thingforward.io/techblog/2017-08-03-compiling-lu...

We have to start somewhere. It is understandable that this is not complete, so welcome your ideas to discover such projects. Please think of any metrics/ways to find such projects.

I strongly recommend you use the packages maintained in Linux distributions as a means for discovery. They're well-organized and maintained and easily accessible programmatically - you can even parse the package dependencies programmatically, as well as have full access to the original source code.

good advice. "apt-get install apt-rdepends" and it becomes possible to work out the reverse-dependencies of packages.

by counting the numbers it becomes pretty blindingly obvious what the critical dependencies are. as mentioned in another post above, bash and glibc6 are blindingly-obviously high on the list... yet the GNU Project receives an unbelievably low amount of funding despite their critical importance.

likewise, this particular bug in binutils ld, which centres around the incredibly short-sighted "4GB should be enough for anyone" removal of Dr Stallman's memory-resident algorithms in the late 90s, is having some very serious consequences:

https://sourceware.org/bugzilla/show_bug.cgi?id=22831

yet because there's no money not even from redhat nobody's looking at it.

likewise: PAM no longer has a proper maintainer, and hasn't had for... a decade?

these are projects that people are relying on yet completely forgetting they're a critical part of the infrastructure!

why? because, just as rhencke said above: they're not on github, they've not got "unnecessary changes" which are counted as "activity to be glorified and worshipped".

abharya: i heard on slashdot the intent to start from github, to exclusively focus on github. this will turn out to be a serious mistake.

There is no exclusive focus, we are just starting somewhere where we can see the various metrics. Plan is to expand to non-github projects and other places (like custom issue trackers), but this is not straightforward as it sounds. Ideas welcome!. https://github.com/ossf/criticality_score/issues/29

I would like to see a measure of criticality that takes the following into account:

* Critical projects may have very little activity/maintenance. For example, Bash 4.0 to Bash 5.0 was only 123 commits over 8 years. But, Bash is a absolutely a critical project (ask any org about how much work they had to do when affected by https://en.wikipedia.org/wiki/Shellshock_(software_bug) ).

* A measure of criticality should understand _as many of the various forms of dependence on software_ that may occur that it can. Dependencies can take many forms, such as:

a package manager resolving a dependency

a user purchasing a mobile phone with software pre-installed

a user visiting a website (react/jquery/etc)

etc.

* Criticality should understand if, how, and when dependencies are updated. For example, fixing a bug in Chrome and distributing that fix to 80% of users in 1 week is feasible. Fixing a bug in Bash and distributing that fix to 80% of users in 1 week is not so feasible.

filed https://github.com/ossf/criticality_score/issues/23

people. as a computer scientist you're probably thinking, "this can be solved by analysing a source code forge" or, "this can be solved by running an algorithm". it can't (or, more to the point: it can tell you quantities, but not quality or value).

i mentioned in another post: github "glorifies" the person and the changes that they make. "look at mee! look at mee! i'm making a commit! i'm wiping my backside now! aren't i great!" which gets you precisely zip in terms of actual strategic value.

changes measure change.

people will tell you - if you let them - by providing you with the information needed to make a qualitative assessment.

so.

provide a type of wiki/website that allows qualitative assessments to be made, on a per-library / per-project basis. then put the metrics (the "criticality value") onto that.

pre-seed that wiki/website with stuff from github if you feel so inclined but DO NOT limit the wiki/website to exclusively github. i repeat again: doing so would be a disastrous mistake.

one of the things that's important is not to include google's lawyers bias against the GPL license. i always wondered where the bias against the GPL came from, within google, and learned of the existence of the in-house legal team. it turns out that they have been advising google employees for some considerable time, "avoid the GPL, avoid the GPL".

unfortunately, as legal advice, those google employees (right the way to management) do not have the backbone to say, "err no actually, GPL-licensed code is the critically strategically important leveller that forces aberrant companies to collaborate rather than sponge off of underfunded projects".

Github's downloads don't tell you when FAANG added your project to their internal project tree, which is then deployed in hundreds of millions of devices.

...not to mention all the critical software running on power plants, industry, aircrafts, ships, cars, satellites, core Internet routing...

Did i forget military, e.g. weapons?

I don’t see any open source software in power generation. Vxworks on plcs and then osi pi and some horrible HMI software.

One exception is inductive automation ignition HMI which uses java

- Other metrics such as how many contributors and organizations are involved, how many user feature requests and bugs getting reported, those are all important project importance and not just "dependency count". some projects can be standalone, so as per your algo, those should be very low. - Downloads data is not available for most repos, please find a reliable metric to use. - Package repo dependencies works, but it does not work for C/C++. At some point, we will integrate github's dependency count info as well.

So basically the algorithm is designed to not find projects that are critical because they are deep in the foundations, depended on by nearly everyone but only worked on by a few people? (which is what "critical" would suggest at least to me) This seems to be mostly a "github marketing index"...

I seriously hope no actual decisions about resource allocation etc are made based on this.

We are working on this problem, it is not simple. Identifying dependency trees reliably across languages is not straightforward [only nice for package manager ones]. Follow https://github.com/ossf/criticality_score/issues/8

You didn’t identify certifi, urllib3, chardet or pytz in your top 10 critical Python dependencies. These are all highly download packages, mostly maintained by one person, which are totally critical to millions of other packages and the Python ecosystem as a whole.

A few of your top-10 I can agree with, but when you’re saying a home-automation package (“core”) is more critical than something like pytz then something has gone terribly wrong.

Filed https://github.com/ossf/criticality_score/issues/20, we will fix this, have an idea on the issue.

Maybe I mis-understood what's supposed to be being checked here. I thought the point was surfacing critical projects that are under supported.

So a highly dependent project that has plenty of support for its needs should not need to be brought to anyone's attention. I'm guessing python, llvm, clang, chromium, webkit, C#, visual studio code, off the top of my head are all well funded and supported open source projects.

I don't know what good examples of under supported but critical open source projects are. I guess I've read that OpenSSL was massively under supported but apparently the solution chosen was to throw it under the bus and promote libssl or boringssl or something like that (completely out of my expertise)