News for hackers. Stuff that matters.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by noident on Wednesday November 06, 2019 @07:12PM (#21469063)
You can and should restrict the number of people with access to the data, but in a tech company there's always going to be a significant number of people with direct access to the raw data. Software engineers on an on-call rotation, full-time site reliability engineers, data analysts, maybe even some external contractors... maybe this isn't the case for Twitter, but many companies also have to put complete trust in their cloud provider or datacenter, which puts even more people in the loop.

Even if you follow best practices with access control, in the end you're always going to have a group of people who you need to trust with access to folks' personal data. Maybe the solution is better audit logging and even tighter access, but I'm not sure leaks of this nature are preventable.

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by nostrademons on Wednesday November 06, 2019 @07:28PM (#21469182)
"Software engineers on an on-call rotation...data analysts, maybe even some external contractors"

These groups usually get access through a bastion that anonymizes data and logs access. I remember that as a SWE at Google, I could run aggregated & anonymized statistics across query logs, but some info (eg. IPs, user logins) had been scrubbed before any of my code could get access to it, and for things that were more personal (eg. your GMail login) you could only get access to your own account.

There's nothing you can do about SREs who have root access on the box or the SWEs who need to implement & maintain the bastion servers, but that's presumably a more restricted, vetted, and trusted group.

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by heartbreak on Wednesday November 06, 2019 @08:58PM (#21469710)
> usually get access through a bastion that anonymizes data and logs access

I think this may be something that is unique to a certain size tech company that simply isn't the case at 99.99% of companies with user data in their possession.

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by CydeWeys on Wednesday November 06, 2019 @11:24PM (#21470368)
It should be the case at Twitter though.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by twitter_anon on Thursday November 07, 2019 @08:34AM (#21473000)
It hasn't been until recently. We are currently locking down all internal systems based on the minimum necessary accesses for various systems. We should have done this a lot sooner.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by CydeWeys on Friday November 08, 2019 @12:44PM (#21484730)
Good to hear it. Better late than never.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by pcwalton on Wednesday November 06, 2019 @08:23PM (#21469535)
Even Google's policies aren't—or at least weren't—foolproof. In 2010 there was a major incident involving a Google SRE named David Barksdale: https://www.telegraph.co.uk/technology/google/8003925/Google...
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by nostrademons on Wednesday November 06, 2019 @09:07PM (#21469744)
Yeah, I was there when it happened (and it was a fairly big deal internally). He was an SRE with pretty high-level access. There's relatively little you can do for this kind of threat - the nature of their jobs usually requires that they have root on the box. (Though a sibling comment suggests that there's even more rigorous procedures now.)
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by retrovm on Wednesday November 06, 2019 @09:28PM (#21469848)
The centralization of infrastructure has really changed the insider risk profile at that company. A decade ago it was common for a service to own their hardware and to have broad authority over those machines but not so much these days. Nobody wants to own their hardware, everything is uniform and it's easier to spot deviants that way.

Of course many of the changes around that time were in response to the breach by the Chinese government, not only in response to embarrassing privacy incidents. Later improvements came about because of things Snowden published.

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by x220 on Wednesday November 06, 2019 @10:17PM (#21470101)
Why would Google change because of the Snowden Leaks? They knew about it. They were handing over data to the NSA.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by noident on Wednesday November 06, 2019 @10:41PM (#21470201)
There was some spying by the NSA without their knowledge or consent. Network traffic between their internal networks wasn't encrypted, which enabled snooping.

https://www.washingtonpost.com/world/national-security/nsa-i...

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by bcrosby95 on Wednesday November 06, 2019 @10:42PM (#21470205)
IIRC they didn't know about the tap on their international cable and the leaks caused them to prioritize the encryption of traffic on their internal network. Not sure about anything else though.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by C1sc0cat on Thursday November 07, 2019 @07:41AM (#21472648)
Its the same for telcos, these days there is much stricter auditing and actual real security vetting - I know team leaders on some systems had to be PV (top secret in US terms)

I suspect that at some stage this might come for some FANG employees

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by remarkEon on Wednesday November 06, 2019 @07:35PM (#21469232)
Would it be smart for companies like google and twitter to publish how they vet people for these roles? That’s a seriously tremendous amount of power. I almost think it needs to be regulated like heath info. I get that that would make it more expensive to handle this data at all, and would serve as a significant barrier to entry for startups ... but data breach after data breach is pushing me in the regulation direction.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by retrovm on Wednesday November 06, 2019 @07:56PM (#21469383)
Pretty much nobody at Google has that kind of access. You can be an SRE of just about anything at Google and never access user data. The "break-glass" means of emergency access is ridiculously booby-trapped. A person wanting to do this thing has to 1) badge into a special room, at which time both production security and privacy incident teams are notified, 2) use a special hardware security device that is used for no other purpose than to activate a VPN box with a hard-line into the production network. By the way if a random Googler just rolls up to a datacenter without a reason to be there, that also triggers privacy incident response, even though physical access to production storage is virtually useless due to all the encryption.

I would say it is much more likely that Google will accidentally lose the organizational ability to become root-in-prod, than it is that a person has done this thing without being noticed.

In short, insider risk cannot be mitigated with hiring practices. You need robust technical measures against insider risk.

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by remarkEon on Wednesday November 06, 2019 @08:03PM (#21469419)
Thanks for proving my point, I guess? I’d love to see a formal write up from google about these procedures. It would go a long way to increasing confidence in how they handle this data.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by the-rc on Thursday November 07, 2019 @12:47AM (#21470764)
Even if you managed to get direct access to production through the special room's direct link, you'd still need a special kind of credentials to send RPCs to any service.

There was a video with some of the datacenter security measures, e.g. iris scanning (just to get yours in the DB required approvals from senior people). On the actual floor, to which very few have actual access, you need to badge both on your way in and out, individually. If you badge out without having badged in, the door won't open and an alarm will go off.

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by belltaco on Wednesday November 06, 2019 @10:20PM (#21470113)
When did that start? Here's an account of someone working at Google accessing info to stalk teens.

https://www.businessinsider.com/google-engineer-stalked-teen...

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by dredmorbius on Wednesday November 06, 2019 @11:47PM (#21470477)
That story dates from 2010.

Snowden's revelations (2013) were a major watershed. There'd been several measures taken since, based on what I heard on the outside, largely through discussions, mostly public, a few direct, with Google staff via G+.

Starting on, of all days, November 9th, 2016, I began regularly posting an image of Jewish shop windows shattered during Krystallnacht, asking whether Google were thinking of brownshirt-proofing their data. That generated responses including from G+'s architect (then in a role with user data safety & privicy), and the data security lead.

It wasn't until some time later that I realised I'd entirely accidentally picked the anniversary of the event for the post. Though the coincidence was useful.

My understanding was that numerous protections were in place by that time. I continue to have concerns.

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by ForHackernews on Thursday November 07, 2019 @05:07AM (#21471889)
> I began regularly posting an image of Jewish shop windows shattered during Krystallnacht

This is so incredibly cringey. You're actively building the panopticon and yet you think of yourselves as righteous warriors for justice.

I don't mean this to be a personal attack, but yours is such a revealing comment about the mindset of people inside these surveillance behemoths.

(See also: this "pledge" http://neveragain.tech/ to not build registries for targeting citizens...signed by a bunch of people who work at companies whose entire business is targeting citizens with ads)

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by mehrdadn on Wednesday November 06, 2019 @10:11PM (#21470062)
What I would love to see is somebody at Facebook comparing their barriers against accessing user data with these from Google.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by remarkEon on Wednesday November 06, 2019 @11:45PM (#21470468)
I’m actually inclined to think they have similar procedures, if only because we haven’t seen “whistle blower” stories in the news about folks reading texts and looking at other private user data. Maybe I’ve missed them, but because bashing Facebook is kind of a trend one would think there’d be an appetite for “I read illicit group texts for a year here’s what I saw AMA” stories.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by dannyw on Thursday November 07, 2019 @03:37AM (#21471497)
If this happens, every incentive (of every party that is aware) is to not whistleblow, as it is a criminal offence for the snooper, and a PR disaster for the company.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by iratei on Thursday November 07, 2019 @12:46AM (#21470760)
> Pretty much nobody at Google has that kind of access.

OK, Google.

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by justinclift on Thursday November 07, 2019 @12:14AM (#21470616)
> Pretty much nobody at Google has that kind of access.

Sounds like you'd be surprised at what storage, and backup engineers have access to.

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by retrovm on Thursday November 07, 2019 @04:19AM (#21471658)
Tremendous amounts of ciphertext?
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by justinclift on Friday November 08, 2019 @01:38AM (#21480383)
Because encryption keys are never backed up by the same system either?

/s

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by mcphage on Wednesday November 06, 2019 @09:10PM (#21469762)
> Would it be smart for companies like google and twitter to publish how they vet people for these roles?

I think much like anti-hacking and anti-fraud efforts, publishing information about how they vet candidates would just make it easier for attackers to figure out how to game the system.

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by ENOTTY on Wednesday November 06, 2019 @10:03PM (#21470027)
Ironically, one of the vetting regimes that has the most publicly available information about it is the US government's security clearance system. https://ogc.osd.mil/doha/isp.html
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by goatinaboat on Thursday November 07, 2019 @12:43AM (#21470752)
one of the vetting regimes that has the most publicly available information about it is the US government's

Why wouldn’t the information be public? It’s the same concept as crypto algos being published and peer-reviewed.

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by C1sc0cat on Thursday November 07, 2019 @07:43AM (#21472661)
The same way defences industries do I would imagine for SF86 etc
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by CydeWeys on Wednesday November 06, 2019 @07:15PM (#21469084)
"but in a tech company there's always going to be a significant number of people with direct access to the raw data"

This is absolutely not true. This number can and should be reduced to an absolute minimum number of people.

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by tristanstcyr on Wednesday November 06, 2019 @07:18PM (#21469103)
Sure, but it comes at a cost. Companies rarely push the limits of these kinds of policies because customers are not willing to pay for them.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by rapind on Wednesday November 06, 2019 @08:17PM (#21469501)
Making the penalties for breaches far more severe would be a good place to start though.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by CydeWeys on Thursday November 07, 2019 @01:21AM (#21470914)
Twitter is big enough to pay that cost. Easily.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by bmiller2 on Wednesday November 06, 2019 @07:18PM (#21469101)
The absolute minimum number of people may still be significant
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by verletx64 on Wednesday November 06, 2019 @07:26PM (#21469167)
Twitter has a lot of staff. Even 1% is a relatively large amount of people.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by ComputerGuru on Wednesday November 06, 2019 @10:40PM (#21470196)
(Just as an FYI, the last time I checked that number was staggeringly skewed towards sales/marketing/evangelism teams rather than coders. Not that your point is diminished.)
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by jacobsenscott on Thursday November 07, 2019 @01:18AM (#21470897)
Those people probably have more access to PII than staff engineers - that's who the data is for.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by ComputerGuru on Thursday November 07, 2019 @09:25AM (#21473413)
Hardly. They get the filtered data that SEs designed for them to get, probably on an account-by-account basis. It’s the raw data SEs have access to that we are mainly talking about here.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by CydeWeys on Thursday November 07, 2019 @01:22AM (#21470917)
It doesn't need to be 1% of staff, it just needs to be a few dozen people (distributed geographically across the globe) who have root access.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by RhysU on Wednesday November 06, 2019 @08:36PM (#21469605)
Moreover, anyone seeking privileged access will percolate into such roles.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by journalctl on Wednesday November 06, 2019 @07:23PM (#21469141)
Maybe because tech companies tend to be careless with data. It isn’t NP-complete to track and monitor access to prod data, or prevent access to it entirely. It’s just that people don’t want to do it.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by closeparen on Wednesday November 06, 2019 @07:29PM (#21469192)
It is pretty damn hard to reliably operate a service that you can’t introspect or debug in any way. The real world is more creative than you; things will happen in production that you didn’t think of in your synthetic test fixtures.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by journalctl on Wednesday November 06, 2019 @10:31PM (#21470158)
That’s still no excuse. Maybe Bill has to access some customer data to troubleshoot. Fine, log it. Maybe Susan has to look at sensitive logs to fix a bug. Fine, log it. These aren’t new or unsolvable problems.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by closeparen on Thursday November 07, 2019 @12:40AM (#21470741)
Who says there weren’t logs? They got caught.
Re:Former Twitter Employees Charged with Spying for Saudi Ar... by d1zzy on Wednesday November 06, 2019 @07:32PM (#21469213)
Please back this up and define the terms used.

As in, define "tech companies" (name specific companies that do this and why you think that can be generalized to the entire industry) and define "careless" (it's a relative term so please say if it's less or more careless of other examples of organizations that manage similarly large amounts of data but that aren't "tech companies").

Because to me the opposite seems true. It's the non-tech companies (if you equate that to FAANG) that manage large amounts of data that tend to have a lot more data leaks (internal or external) than the tech companies.

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by JohnFen on Wednesday November 06, 2019 @07:39PM (#21469257)
> in a tech company there's always going to be a significant number of people with direct access to the raw data.

Why?

I've worked for a few large tech companies that handled very sensitive customer data, and they didn't allow unsanitized access to it by a significant number of people. Typically (on the dev side, anyway), there was a small designated team (less than 10 people) who were the only ones who had such access. Any dev work that absolutely required access to that data -- which was very rare -- was performed by that team.

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by icedchai on Wednesday November 06, 2019 @10:42PM (#21470204)
It's not like this at smaller companies. It's anything goes.

I worked at one place that had the development network permanently VPN'd into prod. One day, a developer accidentally configured his local environment to connect to a production queue and database. It was like this for over a week.

A previous company didn't bother with the VPN. They had an AWS environment that predated VPC, so SSH and many other service ports were open to the office IP addresses. And several people's homes, for remote work.

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by JohnFen on Thursday November 07, 2019 @10:53AM (#21474246)
> It's not like this at smaller companies. It's anything goes.

It depends on the company (as with large ones, apparently). I currently work for a small company, and it is no less diligent about this stuff than the major companies I've worked for.

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by justinclift on Thursday November 07, 2019 @12:19AM (#21470644)
Those large tech companies probably had a team or teams of people whose job is to look after the backups for production servers.

Backups generally have "god mode" access (best description) as they need to backup and restore not just filesystem data, but the audit log data as well.

Most (corp) places I worked, the developers and SysAdmin's working on production servers gave little thought to the backup component apart from making sure the software is installs and runs. ;)

Re:Former Twitter Employees Charged with Spying for Saudi Ar... by ForHackernews on Thursday November 07, 2019 @05:01AM (#21471866)
You can significantly reduce these problems by decentralizing data and moving away from giant platforms that make such enticing targets for espionage.