I even have trouble with this premise, in addition to general agreement with a lot of the other comments in this tree arguing that SSN detection is a red herring.
I agree that displaying SSNs for human consumption has an agreed upon standard format. This doesn't imply that code (and therefore any tests or distributions of data) working with SSNs is handling them as XXX-XX-XXXX.
I've seen plenty of clients storing them without dashes in database tables. I saw one storing them as INTs and handling padding in display logic. I don't know if an SSN can start with a leading 0, but they guarded against that.
Especially given the storage and memory implications of a 32-bit integer vs a 9-11 character string, I see lots of reasons to work with SSNs in code as ints. Should we flag all ints as SSNs? Or maybe we can be "reasonable" and flag any integer in the range 100,000,000-999,999,999 (aka 100-00-0000 to 999-99-9999) as an SSN?
I can trivially generate a list containing all valid SSNs with a simple loop. Should I hesitate to publish code with loops over integers in the range above? That code could easily be used by a malicious hacker to generate the SSN list!
But you could easily disable ctrl+f and throw up your own search box with keypress capture. Not all browsers show the search box outside of the browser viewport, and even for those that do (such as Chrome), you could display a hovering modal inside the viewport, as most users won't remember the exact location of the search box.
Even if hackers could just encode the SSN numbers, it would at least mitigate the spreading of PII.
Edit : I don't care about the downvotes, I care about privacy. Enough of the argument "but wait, can't you imagine the cost?", well if you can't afford to protect people's privacy, don't do business at all.
Edit 2 : people are totally missing my points. The goal is to not display any plaintext SSN that would be scraped by bots. As I said, the hackers could just encode the SSNs, but then the numbers won't be readable by scrapers
Edit 3 : once a project is reviewed and verified, it would stop to trigger alerts. This is trivial, but the HN mentality is just disrespectful regarding people's data, until it's their own personal data that leaked
Why don't they also flag repositories with lots of names, those are PII. Too many addresses? That's a red flag.
Why would you even for a second entertain this nonsense? Non-technical people who think this:
https://github.com/Hack-with-Github/Awesome-Hacking
is a smoking gun to prove Github as a company is supporting hackers are the ones putting forth this idea, no one with any technical knowledge should come close to supporting this.
In particular, one excerpt from the complaint reads like output from a noisy scanner that has detected “exposed SSNs” in client-side code:
> [...] Social Security numbers are readily identifiable: they are nine digits in the XXX-XX-XXXX sequence.
GitHub is popular enough that less technical people also browse it, in particular young people looking to learn.
Overly litigious firms causing rising legal costs across industries? I don't care if it's hard to solve, the onus for fixing it is on the firms, they figure it out or face penalties. Perhaps in the interests of helping the disenfranchised we could institute something like what real estate has, where banks are required to sell a quota of mortgages in certain areas regardless of the financial viability. Lawyers could be forced to seek out clients they would ordinarily never entertain due to the risk of loss. If would stink for them, but what do I care? That's their problem.
Just the other day my mom texted my address to a friend that wanted to mail me something. I want to know why Verizon didn't stop that from happening!
Verizon should be censoring all addresses from text messages, but I suppose it could allow them after a manual review process.
HN is guilty here as well:
1600 Pennsylvania Ave NW, Washington, DC 20500
If you don't want your mom to send PII about you to other people without your consent, and it incurs damage, you can surely sue your own mom.
You could leave a comment in README: "To decrypt, just run 'l33t-xor-crypt --key bob --file hacked-data.txt'" and Github's scanner is none the wiser.
Meanwhile, it will be finding all the false positives already mentioned, especially test suites with fake data.
>>> import codecs
>>> codecs.encode('foobar', 'rot_13')
'sbbone'
Should Pastebin be allowed to exist? Should the Open Source developers behind Wordpress and Ghostery be liable for not scrubbing PII off of websites? The "cost" here isn't really money, it's a social cost. It's regular people's access to services that, on net, make their lives much better.
I'm not worried about Microsoft making less money, I'm worried about damaging one of the best software repository services online and making it less useful to ordinary developers.
In regards to your edit 2, Github is a collection of software repositories. Say you replace all social security numbers with <number-redacted>. If I'm a screen scraper, that doesn't block me -- I can just clone the repository. There is no way that Github can block this unless they delete or replace the numbers in the actual uploaded code, which would obviously be a bad idea.
In regards to your edit 3, are you planning on linking Github repositories to real-life identities? Probably not, since that would be a huge privacy problem, and you're trying to improve privacy.
So what happens when ownership transfers? Or when someone makes an innocuous repo and then later on pushes PII? Note that this is not an abstract problem, we've seen multiple malware attacks on packages, browser extensions, and phone apps that boiled down to, "it looked safe, and then somebody stole the credentials or just decided to push malicious code."
I don't understand how a review process would help here unless it was a review process on literally every commit.
Yes, because as far as I know, Pastebin doesn't process sent data to provide its service. They just store it, they could simply store it encrypted and say they can't access data. On the other hand, GitHub process repositories to provide further services. They already read the sent data.
> The "cost" here isn't really money, it's a social cost. It's regular people's access to services that, on net, make their lives much better.
What about the cost for people whose data leaked? Are you saying we should treat them as collateral damage for others to have an "accessible" service? That sounds irresponsible.
> Should the Open Source developers behind Wordpress and Ghostery be liable for not scrubbing PII off of websites? The "cost" here isn't really money, it's a social cost.
No, as long as they don't operate public instances of those softwares, they don't need to include all the tooling for detecting and handling PII. However, anyone who wishes to operate those softwares with a way to publicly sent data to it should implement privacy safeguards.
> ordinary developers.
In 2000s, we were the "ordinary developers". All the leaks happening today is because we didn't care enough about that aspect of software engineering. The new wave of developers should always have privacy in mind before writing software for businesses, and to change the mentality, changing a platform like GitHub would help spreading a new culture of security and privacy-oriented mindset.
> In regards to your edit 2, Github is a collection of software repositories. Say you replace all social security numbers with <number-redacted>. If I'm a screen scraper, that doesn't block me -- I can just clone the repository. There is no way that Github can block this unless they delete or replace the numbers in the actual uploaded code, which would obviously be a bad idea.
Github is an opinionated centralized collections of software repositories. It should be simpler : If your repository is private, then there should be no filter or review at all. If you repository is public or becomes public, then it should be draconian about what is posted and shared. No hate speech, no PII, etc. If there is a positive detection, then the repository should automatically turn private or be suspended until it's resolved.
> So what happens when ownership transfers?
Unsolved issue so far, I don't have proposition about it
> I don't understand how a review process would help here unless it was a review process on literally every commit.
Regarding publicly sharing information on famous platforms like GitHub, it should be mandatory IMHO. I would happily trade a few false positives for a better peace of mind.
Pastebin is literally already scanning pastes:
https://pastebin.com/doc_privacy_statement
> In addition, while we do not generally search for content in your pastes, we may scan our servers for certain tokens or security signatures.
Yes. That's what being in a society means, you have to put some thought into the collective good of allowing a service to exist. Living in a society means that sometimes you accept a greater personal risk in order to allow a large number of people to access services that better their lives. It's how cars work, for example.
You're drawing a distinction between private and public that seems completely arbitrary to me. If we really ought to be doing everything in our power to limit PII leaks, I don't see why operating a private service is any excuse.
Why do I care whether a message was meant to be public or private if it leaks my PII? I could see an argument for something like end-to-end encryption being exempt, because in that case the service provider literally can't scan the messages. But where my texts, or a private software repository, or self-hosted software are concerned, there's nothing technical or legal that prevents companies and software providers from running the exact same tests as they would on publicly facing content.
If we're going to hold public hosts accountable, why are we letting private hosts off the hook? And if it seems obvious that private hosts shouldn't be subject to those restrictions, then what's fundamentally different about a public host that means they should? You can't think about PII as a black-and-white issue, these are a set of tradeoffs that have to be run through a cost-benefit analysis.
> Github is an opinionated centralized collections of software repositories.
A side-effect of making moderation into an indicator of responsibility is that platforms will stop moderating. Platforms like Facebook and Twitter are bad at moderation, but we don't want them to turn off all of their moderation and become 8Chan. The "they already moderate some things" argument can have some really negative side-effects, because it punishes companies and increases their liability just for trying to be better.
Quick side-note, Pastebin does do some text processing for spam detection, particularly around links. But let's assume a service which did no moderation at all. It sounds good to say this service should be treated differently than Github. But what you end up with in that scenario is that everybody stops doing moderation.
If your goal is to make it harder for people to publicly post PII, then this is counterproductive. You should be trying to make it easier and less risky for companies like Github to moderate content.
> Unsolved issue so far, I don't have proposition about it.
Well... don't you think you should solve that before you make a policy? If I propose a car with square wheels, I can't just say, "don't worry, I'll figure out how to make them roll after I build it."
I think you can. Now when we have neural nets capable of distinguishing 100s of different dog breeds, we still have to trip on the most basic and structured type of entity extraction? No. A simple regex, combined with heuristics, and a linear model on top can reliably detect SSN's.
Just that there will be a few false positives (no matter if you automate this, or do this manually) does not mean it is a Herculean technical challenge to do this.
Then Cap 1 thought: If some rando can find this after a lot of damage has been done, why can't Github find these seconds after upload?
And, really, there is no technical excuse. It is perfectly possible to do this, and lots of big companies do this (or hire security companies to do this for them). Mention their name on some deep web hacking forum, a pastebin, or inside Github code, and somewhere an alarm goes off.
Github could (and should) warn if a user uploads loads of PII-like data. For the cost of running a search server and a few moderators. "Are you sure you want to upload your AWS credentials in a public repository?".
Github is somewhere halfway between moderated and a content platform. They already have a history of taking down repositories if they link to PII data (or infringe copyright, or damage U.S. national security): http://web.archive.org/web/20180619172528/https://github.com... so not acting on this specific repo with SSN numbers could be seen as a poor/shoddy job on their part. Github is certainly in the dominant position to mitigate spread of PII data, so they should have their stuff in order.
Given enough SSNs as training data, could someone make an ML model that churns our mostly valid SSNs?
Because that could be really really bad
Most SSNs are deterministically derived. Given a single valid SSN, you can often generate millions of valid preceding values.
for (let i = 0; i < 1000000000; i++) console.log(i);
Every single valid social security number, and several (e.g. 000-00-0000) that probably aren't. You can store the whole thing in about a gig.
I don't see how they can expect to enforce this with 100% accuracy.
SSNs do have a fixed format but other things could potentially follow the same format.
For example what if you had a library that lets you configure randomly generated codes in a XXX-XX-XXXX format and it just so happens one of the random codes matches a valid SSN pattern?
Is GitHub going to mess with your code? What if you have tests that matched on a hardcoded random number in a SSN-like format. If GitHub scrubs that then suddenly your CI tests might not pass. Also how would it deal with modifying your git history so others couldn't clone it with the potentially sensitive data?