Re:Security scorecards for open source projects
by spydum on Monday November 09, 2020 @08:55PM
(#25042100)
I think you miss the point then. The things it's looking for are indicators security practices are even in place. If you wanted vuln scanning, many solutions for that today exist. This is almost like a reputation score. The higher it is, the more likely vulnerabilities will be addressed in the future (or outright prevented).
I like the idea, I think it will just be tough to work out the right heuristics.
Re:Security scorecards for open source projects
by prepend on Tuesday November 10, 2020 @07:36AM
(#25045264)
Having a quantitative, ordinal scale gives the wrong picture I think. Since 10/10 sounds good, but doesn’t actually mean something is safe.
Also the name is “scorecard” that again connotes that a high score is good. I don’t get a 10/10 on my assignment because I did my homework, said hello to the teacher each morning, and was polite to my classmates.
Those are all positive things and if I wanted to measure students they may be a part of the grade. But if I only graded on these factors that’s not terribly useful.
Process measures are good when it’s not possible to measure outcome, but I think in this situation more outcome measures could be factored in.
Re:Security scorecards for open source projects
by abharya on Tuesday November 10, 2020 @01:14PM
(#25049092)
These are just some initial metrics we picked for v1. Please suggest more by filing by a github issue here - https://github.com/ossf/scorecard/issues
Over time, we will add more real time metrics such as open vulnerabilities, etc
Re:Security scorecards for open source projects
by abharya on Monday November 09, 2020 @10:32PM
(#25042584)
Yes, we will automate score capturing over all projects in real-time in the future. https://github.com/ossf/scorecard/issues/12
The work to capture current vuln state is tracked in https://github.com/ossf/scorecard/issues/52
Also, this scorecard doesn’t look for CVEs or problems in particular versions. It seems like it’s much more important that there’s a valid vulnerability in version 1.04 that I’m using than the current version has code reviews for everything.
The reason I care is that I wish there was some stamp of approval on pypi packages that would make it easier for me to trust particular packages and releases.