News for hackers. Stuff that matters.
Re:Google Safe Browsing can kill a startup by yuvalr1 on Saturday January 16, 2021 @03:51PM (#25805390)
This is an amazing story. It really demonstrates the way we pave our road to hell with good intentions...

We should really do something about this issue, where so few companies (arguably, a single one) hold so much power over the most fundamental technology of the era.

Re:Google Safe Browsing can kill a startup by teamspirit on Saturday January 16, 2021 @04:28PM (#25805675)
Here-here! I really wish there was more human involvement in a lot of these seemingly arbitrary AI-taken actions. Everything from app review to websites and more. This heavy reliance on automated systems has led us down this road. Shoot, keep it, just give us the option to guarantee human review - with of course transparency. We don't need anymore "some human looked at this and agreed, the decisions is final, goodbye."

I know it's easier said than done, especially when taking the scale of the requests into account, but the alternative has, does, and will continue to do serious harm to the many people and businesses caught in this wide, automated net.

Re:Google Safe Browsing can kill a startup by pryce on Saturday January 16, 2021 @04:43PM (#25805776)
It's interesting how closely the unfolding of this awful scenario has followed an entirely predictable path based on the shifting incentives: now hundreds of thousands of businesses face the same massive hazard of blocklisted without adequate human review, and with mediocre options to respond to it if it occurs.

Without a shift in incentives, its unlikely the outlook will improve. Unless the organisations affected (and those vulnerable) can organise and exert enough pressure for google to notice and adjust course, we're probably going to be stuck like this -or worse- for a long time.

Re:Google Safe Browsing can kill a startup by Retric on Saturday January 16, 2021 @07:24PM (#25806866)
Blacklisting a site incorrectly seems like a perfectly adequate reason for a defamation lawsuit. So, I think the real issue is with the legal system.
Re:Google Safe Browsing can kill a startup by craftinator on Saturday January 16, 2021 @06:05PM (#25806368)
> this awful scenario has followed an entirely predictable path

The interesting things about predictable paths is that at the start there are a LOT of them, then over time there becomes just one of them. I don't see that this path was any more predictable at the start than any other.

Re:Google Safe Browsing can kill a startup by judge2020 on Saturday January 16, 2021 @09:23PM (#25807477)
It feels like the need for automated systems is a result of the ever-increasing size of the world (there are now nearly 5 billion internet users[0]). For Apple, app review can take days, mainly because doing human review [consistently] well and constantly for 8 hours a day isn't easy[1], leading to staffing issues when bad reviewers get weeded out and only a small percentage of hires stick around. Outside of hiring 10,000 employees just to endlessly review phishing links for 40 hours a week, you need automation to triage these phishing sites and deal with the outcome later such as via on-demand review by a human (which worked in this case, but won't always work - humans still make mistakes). I'm not sure if there is a solution for this problem outside of just not having the safe browsing product if 'makes no errors' is a requirement.

0: https://en.wikipedia.org/wiki/Global_Internet_usage

1: https://www.businessinsider.com/heres-why-it-really-sucks-to...

Re:Google Safe Browsing can kill a startup by cthor on Sunday January 17, 2021 @12:29AM (#25808428)
There's no reason the number of humans dealing with these problems can't scale alongside the number of humans creating them.

But it's a lot cheaper to pay for a few really expensive programmers to make a just-good-enough AI than to pay for thousands of human moderators. So we end up with a stupid computer creating tonnes of human misery all for the sake of FAANG's already fat profit margins.

Re:Google Safe Browsing can kill a startup by hutzlibu on Sunday January 17, 2021 @06:49AM (#25809995)
"So we end up with a stupid computer creating tonnes of human misery all for the sake of FAANG's already fat profit margins."

I don't want to blame this entirely on the big companies, though. Also the people want and expect "free" things on the internet. This is how we ended up like this.

Re:Google Safe Browsing can kill a startup by unishark on Sunday January 17, 2021 @02:14AM (#25808858)
> There's no reason the number of humans dealing with these problems can't scale alongside the number of humans creating them.

I would think the attackers are using automation also, to spam attacks as in other areas of fraud. It can only be a battle of AI ultimately.

Re:Google Safe Browsing can kill a startup by Perseids on Sunday January 17, 2021 @04:12AM (#25809343)
Depends on which problem your tackling. With App reviews for example it is very easy to rate limit the 100 USD developer licenses. And also in cases like the one the medium article is facing businesses would gladly pay a hundred bucks to get real humans to produce competent answers/reviews/decisions. And if you dislike this solution because it creates a google tax (pay us or we'll block your site), make it not a service payment, but a security deposit which they'll only keep if you are fraudulent in some way.
Re:Google Safe Browsing can kill a startup by jsinai on Sunday January 17, 2021 @06:19AM (#25809849)
Is it just me but the way things are currently stacked, human insight is still the best line of defence? The OP and other anecdotes in the comments are examples why we’re not quite at “AI vs AI” yet
Re:Google Safe Browsing can kill a startup by TimTheTinker on Saturday January 16, 2021 @04:48PM (#25805810)
:s/Here-here/Hear hear/
Re:Google Safe Browsing can kill a startup by kubanczyk on Sunday January 17, 2021 @09:20AM (#25810804)
Hear hear!

Where where!?

Here here!

Re:Google Safe Browsing can kill a startup by alfiedotwtf on Saturday January 16, 2021 @04:40PM (#25805763)
> I really wish there was more human involvement in a lot of these seemingly arbitrary AI-taken actions.

Narrator: but it was only ever to get worse

Re:Google Safe Browsing can kill a startup by RileyJames on Saturday January 16, 2021 @05:16PM (#25806038)
Couldn’t agree more, the transparency is key. It enables faith in the system and outcome.

The counter argument to transparency will be that it provides too much information to those who aim to build phishing sites not blocked by the filter.

That said, we’ve experienced systems in which obfuscation wins out over transparency and it would be nice to tackle the challenges of transparency.

Re:Google Safe Browsing can kill a startup by ehsankia on Saturday January 16, 2021 @05:33PM (#25806169)
Are you implying that the list no longer has a good intention? I wouldn't be surprised if there are multiple orders of magnitude more phishing and hacked websites in 2021 than there was in 2004. Even with human checking, I doubt you'll even have 0% failure rate. Is the solution to just give up on blocking phishing sites?
Re:Google Safe Browsing can kill a startup by kovac on Saturday January 16, 2021 @10:26PM (#25807806)
The failure rate doesn't need to be 0%. If the solution is good, at least it'll be close to 0% which means that it'd be possible for the vendor to provide better support for the small number of mistakes so that they can be clearly explained to the affected party and rectified more quickly. If the failure rate is too high to make better support infeasible, then the current solution is not really a good one and we need to consider a revision.
Re:Google Safe Browsing can kill a startup by donmcronald on Sunday January 17, 2021 @06:55PM (#25816569)
> Are you implying that the list no longer has a good intention?

Most of the time I run into blocked sites they seem to be blocked because of copyright infringement, not phishing. The only phishing sites I've seen in the last year or so are custom tailored. For example, I had to deal with a compromised MS365 account last year where the bad actor spun up a custom phishing site using the logo, signature, etc. of the victim.

So IMHO the intentions are no longer pure plus the effect is diminished and being worked around.

Re:Google Safe Browsing can kill a startup by cft on Saturday January 16, 2021 @05:55PM (#25806304)
The solution is for the legitimate sites that are driven out of business by Google AI to sue Google for tortuous interference and libel.
Re:Google Safe Browsing can kill a startup by nearbuy on Saturday January 16, 2021 @10:19PM (#25807766)
This helps one group and hurts another. If Google is liable for blocking potential malware and phishing pages, they'll either stop blocking it, or adjust their algorithm to strongly err on the side of allowing phishing sites.

Businesses become safer, but more regular people will get phished.

Re:Google Safe Browsing can kill a startup by coldtea on Sunday January 17, 2021 @03:10AM (#25809094)
>or adjust their algorithm to strongly err on the side of allowing phishing sites.

It'a not the role of Google to disallow phishing sites (as a browser) just like it's not the role of the ISP.

Make it hookable so people can chose their own phising protection service.

Re:Google Safe Browsing can kill a startup by bonoboTP on Sunday January 17, 2021 @07:51AM (#25810282)
People wouldn't know or care which to pick. They would see the pop-up asking to select a phishing protection provider, would get confused and angry and think "where do I click to get past this pop-up, I want to go on Facebook and this stupid computer is nagging me with stuff again!"

Phishing protection is mostly needed for people who have no clear concept of phishing or technicalities. They just want to do things on the internet, like social media, they don't care about things behind the scenes, that's boring uncool nerd stuff.

Re:Google Safe Browsing can kill a startup by msh on Sunday January 17, 2021 @05:10AM (#25809565)
And then they will choose the same block list and sites will have the same problem.
Re:Google Safe Browsing can kill a startup by coldtea on Sunday January 17, 2021 @05:37AM (#25809666)
All? I doubt it. Not to mention they could offer control to override whatever you like.
Re:Google Safe Browsing can kill a startup by nearbuy on Monday January 18, 2021 @03:20AM (#25819476)
> they could offer control to override

Chrome lets you override and proceed to the site. The problem for the small business is that a large fraction of their customers see the scary red warning page.

Re:Google Safe Browsing can kill a startup by msh on Sunday January 17, 2021 @08:03AM (#25810348)
Well enough that it will still be a blocker.
Re:Google Safe Browsing can kill a startup by coldtea on Sunday January 17, 2021 @10:20PM (#25817745)
Well, that goes without saying. If you want a blocker, you want a blocker. So all the nigerian princes and the like should still be blocked.

You just don't want to give control over the blocking blacklist/whitelist to a single entity, even less so to a huge powerful one, possibly in a country other than your own (which e.g. forces their foreign policy dictums to your blacklist), and even less so the one that already makes your browser, that should be a totally neutral conduit.

Re:Google Safe Browsing can kill a startup by nearbuy on Monday January 18, 2021 @03:14AM (#25819437)
I don't think this solves the problem from the article, since small businesses will still have to deal with getting mistakenly blocked by whatever the popular blockers are. With 40,000 new phishing sites per week, it's not an easy task. If the blockers are free (I imagine they'd have to be to get widespread adoption), who's going to review the false positives? Volunteers?

But also, it would leave the people most vulnerable to phishing unprotected, namely those not tech-savvy enough to install a phishing protection service. Most internet users don't even have ad-blockers.

Re:Google Safe Browsing can kill a startup by fredophile on Sunday January 17, 2021 @11:34PM (#25818187)
The problem isn't the company that blocked it. The problem is the company that reported that there was a problem when there wasn't. In this case it sounds like Google is both companies.
Re:Google Safe Browsing can kill a startup by encom on Saturday January 16, 2021 @08:26PM (#25807191)
>Is the solution to just give up on blocking phishing sites?

IMHO yes. It's too much power for one company to wield. And especially a company with such questionable morals as Google. This cure is worse than the disease.

Re:Google Safe Browsing can kill a startup by fbi-director on Saturday January 16, 2021 @08:51PM (#25807317)
I thought you said, the curse is worse than the disease... which also would've made sense.
Re:Google Safe Browsing can kill a startup by hutzlibu on Sunday January 17, 2021 @06:53AM (#25810013)
" Is the solution to just give up on blocking phishing sites?"

But maybe not do it by default on browser-level.

But if you do, then there really needs to be ways to combat wrong decisions in a timely manner.

Re:Google Safe Browsing can kill a startup by ocdtrekkie on Saturday January 16, 2021 @07:46PM (#25807015)
The solution is simple: Liability. As soon as it becomes legally infeasible to let algorithms block people, it will stop happening.

Make it easy and affordable to submit legal complaints for tech misbehavior and make the penalties hurt.

Re:Google Safe Browsing can kill a startup by vkou on Saturday January 16, 2021 @09:02PM (#25807368)
Ah, so you suggest liability for the vendors of the software blocking websites, with, in practice [1], no liability for the operators of a compromised website, if it is phishing/malware?

This is a great approach, if your goal is to optimize for increasing the amount of dangerous crap on the web. But, eh, that's surely worth it, because the profitability of startups is more important then little things like the security of the average netizen...

[1] Even if you make the operators liable [2], in practice, you'll never be able to collect from most of them. Whereas the blacklist curators are a singular, convenient target...

[2] If you can demonstrate how the operators of compromised websites can be held liable for all the harm they cause, I will happily agree that we should do away with blacklists. Unfortunately, the technical and legislative solutions for this are much worse than the disease you are trying to treat.

Re:Google Safe Browsing can kill a startup by smsm42 on Saturday January 16, 2021 @11:40PM (#25808181)
Since phishing is not going to go anywhere with or without blacklists - for obvious reasons, e.g. lists can't cover everything, and you can't add sites to the list instantly - I am willing to tolerate a slight increase in fishing which is going to exist anyway in exchange for not having Google (or any other megacorp, or any other organization for that matter) as a gatekeeper of everybody's access to the internet. The potential for abuse of such power is much greater and much more dangerous than the danger from tiny increase of phishing.
Re:Google Safe Browsing can kill a startup by mewse on Sunday January 17, 2021 @12:40AM (#25808475)
> I am willing to tolerate a slight increase in fishing

According to Google's most recent transparency report[1], as of December 20th of last year they were blocking around 27,000 malware distribution sites and a little over 2,000,000 phishing sites.

In your view, would turning off those blacklists and allowing those >2,000,000 sites to become functional again count as a "slight" increase?

(edit: That's a real question, incidentally, not a disagreement or an attempt at a 'zing'; I have no knowledge in this area but went to look up the numbers, and am curious whether 2,000,000 is truly a vanishingly small amount, relative to everything else that's out there that's not already on the list)

[1]: https://transparencyreport.google.com/safe-browsing/overview

Re:Google Safe Browsing can kill a startup by smsm42 on Sunday January 17, 2021 @11:16PM (#25818056)
I'm not sure what is counted as "sites" - i.e. if Google closes foo.bar/baz123 and the same server gets assigned bar.foo/zab345 and continues to serve malware, is it 2 separate sites? Did Google really achieve this much by forcing the changing of the URL? Sure, bunch of people that got the phish link in the mail that was sent before switch but then shut down won't be phished, but I have no idea how much that changes the picture - I'm sure phishers are well aware that their domains are short-lived and already adapted for that, otherwise they'd be extinct. However, I'd be glad to read some field-validated data about how much closing those 2M sites, whatever is meant by "sites", actually helps against phishing.

I mean if we could trust Google (or anybody else of that kind) to have blacklist strictly limited to reasonable definition of malware and phishing, and knew that usage of such list if strictly voluntary under control of the user, it would be an acceptable, if decidedly imperfect, remedy. But we know we can't trust any of this, even if whoever works on this at Google right now are sincerely ironclad committed to never any mission creep and abuse happen, once the means exist, these people can always be replaced with others that would use it to fight "misinformation", or "incitement", or "blasphemy", or whatever it is in fashion to fight this week. There's no mechanism that ensures it won't be abused, and abuse is very easy once the system is deployed.

Moreover, we (as, people not in control of Google's decisions) have absolutely no means to prevent any abuse of this, since Google owns the whole setup and we have no voice in their decision making process. Given that, it seems to be prudent to make all effort to reject it while we still can. Otherwise next time you'd want to make a site questioning Google's decisions about the malware list, nobody would be able to read it because it'd be marked as a malware site.

Re:Google Safe Browsing can kill a startup by eXpl0it3r on Monday January 18, 2021 @08:26AM (#25821537)
You can also be certain that these numbers include all the false-positives. One of the Open Source pages I maintain got blocked as well, because too many AV reported one library package as malware.

There's no "report as false-positive" button at Google, so these reports likely have a lot of false positives in them...

Re:Google Safe Browsing can kill a startup by qwantim1 on Saturday January 16, 2021 @10:13PM (#25807742)
This was the case with railroads too, only a few controlled the biggest and most transforming and business-integral tech of 1800s.

Prior to that it was those that controlled the printing presses.

...

History continues to repeat itself.