Why isn't the US doing this?

Word on the grapevine is saying that Google is doing similar. One of the "perk" of being a well-known DNS resolver (8.8.8.8) is getting an early notification whenever a server goes "online" on the internet.

> is getting an early notification whenever a server goes "online" on the internet.

Please elaborate.

Someone types in your new server/domain, like "ijustmadethissite.com", or "newlocation.existingsite.com"

For their computer to resolve this domain name, it's going to call out to a DNS server, of which Google hosts a major one. It can be assumed that they log these names, and can then use that as a "notification" for a site coming up.

But what does that have to do with scanning webservers for vulnerabilities, do they do something with the "newly seen sites", and if so is it documented what they do for scanning?

Because if the vulnerability involves an HTTP request, then the Host header needs to have the domain name of the target website.

So you need: IP address and port for the TCP headers, and the domain name to go in the TCP packet content.

One example of a vulnerability would be having phpMyAdmin with a database password hardcoded and no login needed. Without the domain name it would still be impossible to access. (Of course, domain names shouldn't be considered secret so this would be a very insecure setup.)

True, they have a DNS resolver, but they also have Chrome. And the Certificate Transparency list. Google Analytics. And so on…

I'd never considered the value all those things have when it comes to finding out what to index. Clever, actually.

Way back in the early 2000s the FBI contacted a company I was working for to inform us that someone was hosting Disney movies on our servers. So something like this is at least sort of happening.

I don’t know if copy right protection is the same as penetration testing.

no, but takedowns for copyright protection is the same thing as censorship and goes against free speech. we're all just too blinded by capitalism to see it that way.

So nobody should pay for anything? I pirate tons of stuff and still pay for things that I think are worthy of my payment

I would be surprised if this was the result of active scanning. It's more likely the FBI received a report from someone, and just forwarded it along.

Disney: "Hey FBI, this server is pirating us, plz 2 takedown tyvm"

Maybe it has something to do with the Nord Stream pipeline, maybe it doesn't

I think it's illegal under the Computer Fraud and Abuse act. Also, what should the government do when it finds something? What if the site operators are unresponsive or cannot be contacted? There are a lot of practical problems.

> What if the site operators are unresponsive or cannot be contacted?

This seems like only a minor problem. If people are unresponsive, then oh well, they tried to tell you you're hacked. If the site owner cannot be determined, they can email your ISP. This seems to work well for "one of your customers is torrenting movies", and since every ISP is known by definition (thanks, IP addresses), it should be fairly straightforward to get that message to the actual customer. (Send it with the invoice; if the customer doesn't pay invoices, then it's easy to resolve the hacked site. You were shutting them off anyway.)

Everything's illegal under the CFAA. It's an old bad overreaching law that should be repealed. The government rarely prosecutes itself though, so that's no reason why. Unfortunately, the culture in the US is such that the populace would freak out if the government tried to do such a thing, never mind practical surmountable issues.

Does CFAA restrict government interactions?

If the site operators are unresponsive then that sucks, but it would still help secure those that are responsive.

I mean... They could at least ty to contact the operator.

Greetings Citizen,

We have detected a dangerous virus or service in your hosting environment. Conspiracy theorists and foreign state actors often use these types of methods to spread fake news and influence our elections. These are serious threats to our Democracy, but Fatherland Security is here to help you through this difficult time. Your local neighborhood Security Helper will be at your home in the next few minutes to assist you in removing the dangerous HTTP service. For your safety, please stay away from all doors and windows.

Sincerely,

Rob E. Friendly

The way I read the article, they're actually collecting vulnerability information. So they check a site with Version X running on it, and detect the vuln; then they later see Version Y, without the vuln, and update their vulnerability database.

Nothing in the article suggests that they contact site-owners (I haven't re-read the article, so might be wrong).

I'm not sure why you think it's a potential violation of CFAA to connect to a public server and probe it. There's no suggestion of unauthorized access; that would involve exploiting vulnerabilities they find, and that would be unauthorized access.

> I think it's illegal under the Computer Fraud and Abuse act.

Things that are illegal for individuals to do aren't necessarily illegal for governments to do. This is a reason why the government should be vigorously doing this, rather than leaving it to private citizens, who risk being charged under the Computer Fraud and Abuse Act.

-----

> Also, what should the government do when it finds something?

It should contact the site operator.

-----

> What if the site operators are unresponsive or cannot be contacted?

I would imagine that in the case that site operators couldn't be contacted, they wouldn't be contacted.

Something tells me that even with the somewhat stretched version of extraterritoriality that the US claims about laws like CFAA, they wouldn't try applying that to their closest intelligence/defence partner country operating largely domestically...

Too busy spying on citizens. And maybe they want to use vulns for their own gain.

The US does do this, it is offered as part of security hygiene.

https://www.cisa.gov/cyber-hygiene-services

Looks like it's offered only to "critical infrastructure organizations".

Correct

"Who can receive services? Federal, state, local, tribal and territorial governments, as well as public and private sector critical infrastructure organizations."

However, methinks US definition of critical infrastructure organizations, both public and private, will be quite broad.

Why not both? They will never tell you the unsavory things they're doing. At least, not without coercion.

In Germany the BND does this. You get an annoying email from them if they find UDP ports available for an amplification attack on your Linux server...

Not as annoying as getting DDoS'ed with amplification attacks because some people can't properly configure their servers... (Also I doubt the BND does this, as another commenter pointed out.)

That depends on whether the BND are testing that it could be used in an attack, or just seeing a port is open. Having UDP/11211 could mean you're running a vulnerable memcached service, but not necessarily so.

As others pointed out, its indeed the BSI not the BND. Sorry for the confusion.

Are you sure that it's not BSI?

You are correct, its the BSI.

Hah - one of my first ever network programming tasks was to do this at a UK hosting company. That and SMTP relays. Good that (some) governments are wise enough to try to keep this sort of thing in check.

I hope they aren't using a perl script triggered by a cronjob on a hand-rolled VM though..

Did that company happen to be fast?

Goodness, no. I imagine the um, fast, companies didn't bother.

It was a charming little outfit that has since sold out to iomart. Alas.

It's the BSI (https://en.wikipedia.org/wiki/Federal_Office_for_Information...) and I found the one warning I got years ago useful. ElasticSearch open default port I think.

I have some doubts. For example, if they are just outputting the scan results from some tool with a high false positive rate, how is that helpful? It's a waste of time and money for the government. Bug bounty programs have the same issue that probably most bugs found are trash results from a scanning tool.

On the other hand, a custom built tool that tries to find the most serious known vulnerabilities with a low false positive rate would probably be a good thing for the government to run.

I'd imagine part of the job of the people working there would be to limit the number of false positives.

Could be, but it is certainly not how it works at my org.

What scale does your org function at?

So if they use a bad tool, it would be bad, but if they use a good tool it would be good?

correct. fortunately, the sales person from the security vendor, the media, and the public officials are aware of this constraint.

There are no good tools. Just a bunch of shady vendors.

I think you misunderstand.

I'm reading that the UK government is spying on us, and their retrospective plausible excuse is that they are scanning web servers for, erm, vulnerabilities.

No, I don't think that the government is here to help. It allows itself only to maintain force, that it then uses to forcibly extract wealth from its herd, er, sorry citizens.

The downvoting tells us about the crowd, not about your comment.

It tells you that the crowd don't want to read unsubstantiated cynicaler-than-thou hot takes on HN.

Downvoting "It's raining because Soros and his globalist Jewish cabal control the weather" does not mean I disagree that it's raining but the edit always comes in [downvoters can't handle the TRUTH, stay classy HN] or similar.

e.g. how is scanning for vulnerabilities "spying on us"? How is scanning for vulerabilities "forcibly extracting wealth"? How is informing people of vulernabilities "not here to help"? It's a thinly disguised flamewar comment, not a comment on the topic.

>> e.g. how is scanning for vulnerabilities "spying on us"?

To play Devil's advocate: once you discover a vulnerability you always have two options: report it and have it fixed, or exploit it for your own gain. You charitably assume that government is somehow obligated to chose the former, while in reality in some cases it might choose the latter.

> To play Devil's advocate: once you discover a vulnerability you always have two options: report it and have it fixed, or exploit it for your own gain. You charitably assume that government is somehow obligated to chose the former, while in reality in some cases it might choose the latter.

This assumes that the government that wants to compromise a domestic host can't do it in a way that is a lot more deniable than porting scanning you from a gov owned IP range.

If the government wants to find and exploit a vulnerability they likely will find a way they don't need some loose cover story for it.

This is a fair point - in organisational terms it'd be better if NCSC was under a non-ministerial body, independent of political influence and control. Similar format to a university, maybe.

The bulk of UK government revenues are dispersed to the sick and poor, and to educate children. Iron fisted despots.

https://yougov.co.uk/topics/politics/articles-reports/2014/1...

Depends what they do when they find a vuln, there is incentive to not always reveal it.

Well, as tradition I maintain a watch on postmaster and webmaster at... so I'd hope for a friendly heads-up. Basically well done.

RFC 2142 role mailboxes are critical. It's pretty telling that the webhosts that ignore the RFC are the ones I constantly find problems at. Godaddy not only routinely hosts all kinds of evil, but they make reporters jump through hoops to let them know they need to clean up their mess. If you do try to email [email protected] they'll ignore your report and kick back an autoresponder message telling you to fill out a shitty web form instead. Nothing like making more work for people who already went out of their way to let you know you're causing problems for everyone else.

But, do they tell you about the vulnerabilities before they exploit them?

Maybe they put it like this to exempt themselves...