This article fails to take into account that time and time again we've seen that 'anonyimized aggregate data' is never truly anonymous.

In the AOL anonimized data leak there were plenty of individuals identified:

http://www.nytimes.com/2006/08/09/technology/09aol.html

MIT researchers also showed that four anonymous purchases are enough metadata to identify 90% of individuals:

https://www.technologyreview.com/s/536501/data-sets-not-so-a...

And, a personal favorite of mine where researchers from Standford and Princeton are reporting at the World Wide Web Conference this April: "Researchers found that they could identify the person behind an 'anonymized' data set 70% of the time just by comparing their browsing data to [often public] social media activities"

https://www.techdirt.com/articles/20170123/08125136548/one-m...

It would not be hard to buy a zipcode worth of data and compare it to known facts about a person until you de-anonymized it.

Also, the Netflix prize data set was used to identify individuals from only a handful of movie ratings.

http://www.cs.cornell.edu/~shmat/shmat_oak08netflix.pdf

While I agree with the sentiment that companies have a poor track record of protecting personal data, I'd like to clarify that a "zip code of data" is not anonymized intentionally, which is what makes picking out individuals so easy.

There is a technique of intentionally anonymizing data [1] that I learned about because that Apple was talking it up in relation to storing health information. I'm only a layman, but my understanding is that it makes it much, much harder to do an analysis like you describe.

[1] The term is Differential Privacy. Here's a tutorial video (1 hour, 34 mins): https://youtu.be/ekIL65D0R3o

The zip code was intentionally not anonymized because it was seen as acceptable to release, and that is part of the issue of anonymized data.

US Zip codes serve around 7,500 per code on average (US Census 2010) which is different than common wealth postal codes like in Canada where they serve an average of 19 households (25 - 75 people).

But, zip data can be interchanged with plenty of other unique identifiers on the web. Maybe it is browser language setting, or version of java etc.

Think of it like an Excel spread sheet, if in column A you can have options "1" or "2" then in a list of 100 people there will be at least 50 who share the same data footprint. If you keep adding columns from B onward with the same logic eventually you'll have pretty unique strings.

Things like searching history or web history are even worse. Ever done a search for a pizza place near your address, or Google map directions from your home to another location? That identifies you pretty easily. So does connecting to your works website, and the school your kid attends. Web browsing data is nearly impossible to anonymize by its nature unless it was compiled to something like "XX% of users in Zip XXXXXX visited website.com"

As for differential privacy, it is a nice emerging theory, but there are challenges with it as there is a significant trade off right now in terms of data accuracy when applying differential privacy. It is primarily effective at casting doubt on if variable "A" about user "B" in a data set is true or not, but if you don't have a specific target or specific metric then enough of the data is true that it could still in theory be deanonymized, and since most of the anonymity is based on incorrect variables in a data-set, all it would take to reverse engineer it is a large enough data-set and a few known variables.

I hope people like Apple continue to champion the advancement of differential privacy though - it is a major step in the right direction. But, being able to buy browsing history, even in aggregate does not protect individuals.

>The zip code was intentionally not anonymized because it was seen as acceptable to release, and that is part of the issue of anonymized data.

Forgive me for asking, but you seem to have two definitions of "anonymized":

- anonymized - not anonymized, but claimed to be anonymized

I think this argument (which I agree with) would be more forceful if we could stop calling non-anonymized data "anonymized". "Depersonalized", perhaps?

So there is the version of "anonymized" that most companies currently use which is as you say 'depersonalized' - they remove your personal information and think it is safe, but data is still tied together in unique records. (i.e. your name is replaced with an ID number)

Then there is actually 'anonymized' data which would be the release of data in which you cannot in anyway identify a user. An example that comes to mind for me is the census releasing aggregate stats such as "14% of American's speak language X."

If the census instead had records of each American line by line, listing which language they speak and other associated factors about them then this data is likely to paint a unique picture of the individual even if their personal information like name and address were removed.

I think most data is very hard, if not impossible, to truly anonymize. Even if the search history that gets sold wasn't broken out into history per tuple/record, then you could still identify at least a few trends in it.

Does that make more sense? But yes, I agree that these companies are more attempting to 'de-personalize' data for the sake of research, but, that is far from anonymous and naming it as such is misleading to the public.

The key part from that TD post for me was:

[...]it requires a social media feed that includes a number of links to outside sites. However, they said that "given a history with 30 links originating from Twitter, we can deduce the corresponding Twitter profile more than 50 percent of the time."

I'm paranoid enough to stay away from big social media altogether, but I realize that is uncommon.

Wow only 30 links, that is both impressive and horrifying.

I have very few tweets < 200, but I think you can find 30 outbound links in my first 50 posts since most of my use for it was sharing interesting articles.

Can you imagine combining that with something else as simple as which Oauth apps someone has approved in Twitter? You'd reach near 100% accuracy in no time.

"and they every major network has robust safeguards in place to prevent you from working back to a single person’s web history."

hahaha. THat's quite a talking point. What would these "Robust safeguards" be? The history of pretty much every study on this, ever, is that it's pretty easy to deaggregate and deanonymize data.

But look i'm sure, verizon, at&t, etc, those great bastions of "doing it right", have done this right too!

20 bucks says if someone buys large amounts of the aggregated data, they can extract significant information that can be pinpointed to individual congresspeople.

The safeguards might just be that they won't sell you "large amounts of" the aggregated data unless you're clearly a profit-driven multinational corporation rather than a ideologically-driven individual. KYC and all that.

What about a profit-driven multinational corporation with political ties?

Say it is indeed possible to pinpoint individual congresspeople, as the grandparent suggested. If some of these congresspeople have a browsing history they'd rather not have made public, identifying them could leave them open to blackmail.

At some point the threat also just takes the form of "here's a list of random sites we totally made up and will tell people you visited". How well does blackmail like this even work? Even if somebody has every naughty site Trump has visited, where will you publish it and why are his supporters going to read it and believe you? Why will they not assume you made it all up?

Regardless of the source of the compromising information (browser history or other), one difference between factual and fabricated information is that you cannot deny the former without the risk of legal consequences, because denying it would be lying. Think perjury, for example (the Lewinsky scandal comes to mind).

And Trump would need to either ignore the claim made by the media, which could be seen as an admission of guilt...

Deny the claim, and then be later impeached for dishonesty.

Admit the claim, and then well, admit it.

IF some of the content is serious enough, it could surely be a foundation of a congressional hearing or perhaps even impeachment process? How can he respond to the questions there?

In any scenario... I can't see how he could "win" this? (Unless he is really innocent of anything questionable of course)

He was recorded admitting to groping women who weren't his wife. What on earth makes you think that his browsing history is going to bring him down?

He's not going to be impeached for dishonesty, the government is entirely run by his fellow Republicans.

He's not a Republican. He hijacked the party and beat the living shit out of every single one of them, and they fear him.

He is a Republican. He's the leader of the Republican party by way of winning the Republican nomination and the Presidency. The rest of the party may not like that, but the fact that Republican voters chose Trump over a more mainstream candidate demonstrates that mainstream Republican ideology is no longer relevant to the future of the party.

Which is why Trump probably won't be impeached, or even seriously censured - doing so would only cause an insurgency by Trump's supporters and it would weaken Republicans in the face of the Democrats. It would be suicidal.

I think we agree.

He's not the sort to make any friends. The GOP is just whistling sideways hoping the rest of America stops wondering what Trump is actually up to beyond the surface noise.

The NSA has it all already, and data is shared around via Five Eyes. The blackmail you speak of is likely already happening, just not at the multi-national corp. level, but higher.

I believe the correct term is lobbying not blackmail.

Because ideologically-driven individuals don't work for profit-driven multinational corporations? Because they're never hacked by ideologically-driven individuals? That kind of safeguard to me sounds like the worst of both worlds: Not meaningfully safeguarding anything, and lost business. A "clearly profit-driven multinational corporation" will (practically by definition) care about the latter.

That will be an interesting lawsuit to follow whether you are allowed to be discriminated against because of not being a profit-driven multinational corporation.

No, it won't and yes, you are, respectively and unfortunately.

People are missing the point on the bill, it isn't just horrible for privacy. Really this is a jab at net neutrality, the open selling of data by ISPs is a power grab away from the FCC that helped to make net neutrality a thing by labeling broadband/ISPs as common carriers.

Republicans (this was a party line vote) say it is unfair that Google and Facebook have your personal information and use it for ads but why can't ISPs have that and also sell it? One big major reason is people sign up to Google and Facebook for the purpose of sharing and agree to their ads in exchange for a service. Google built the most powerful search engine and Facebook built the social graph. Google/Facebook built value and they only use your info to target ads to you, they don't sell it because others would do the same. They sell ads and people use them because they have info on you, not necessarily to sell off to others.

If you ask me it is unfair for republicans to legally allow ISPs to do the same because we expect privacy from ISPs in ways we do not from Google and Facebook. You can choose not to use Google or Facebook but you cannot choose your ISP/broadband provider. In my opinion this is like letting someone view your mail, read it and then sell information about you.

It is also an unfair competitive advantage for ISPs above all because they can place ads on any website if they want or track you across all sites not just like Google/Facebook which are huge but only see a portion of what you do. ISPs built no value product like a search engine or social graph for this purpose, they should do that if they want access like Google and Facebook. It seems almost like the GOP are harming innovative companies and rewarding/catching up non-innovators. I bet broadband companies/ISPs won't even use the profits to improve broadband and rollout gigabit service for real. It is a rewarding of lazy semi-monopolies over innovative companies and products.

Republicans also control the FTC not the FCC so they want all control to fall to the FTC instead. It is both a power grab and a bending over of all their constituents.

Most of all, it is also another step in dismantling net neutrality as FCC protected that by categorizing the broadband/ISPs as a common carriers and they want to sap the FCCs power in that regard.

1. Google/Facebook try a lot of underhanded things to triangulate more information about you (for e.g. with sleazy tactics such as inference based on your friends' data even after you have left FB)

2. The very notion of a lookalike audience (and its unfortunate popularity amongst the growth hacker types) is an immediate giveaway that FB/Goog are never going to take this tracking down even a single notch.

3. Google/Facebook actively encourage you to be always signed in to their networks

4. Google Analytics has more or less ensured that they see a BIG portion of what you do. Besides, just imagine a casual web user jumping from site to site. How many times are they going to land on a site which doesn't use Google Analytics?

5. Android is the dominant phone OS

6. Let us not even get into the infuriatingly patronizing attempt by Facebook to become an ISP in countries like India. Why didn't FB try to pull that stunt in a more developed country? Food for thought!

This is a case where pretty much every company is in the wrong.

>> People are missing the point on the bill, it isn't just horrible for privacy.

However, the overall point you make is true.

>( (...) even after you have left FB)

even before you join FB or even if you never join.

I think nobody should be able to deal in private information, but that's some fine needle splitting there. Due to powerful network effects, I have more choices of ISP than I have choices for social media to keep in touch with my family. (Unless you want to teach all my aunts, uncles, cousins, and my daughter's various grandparents how to use whatever social network doesn't spy on you!)

And as for creating value, ISPs invest billions of dollars a year building physical infrastructure. It sure as hell wasn't Google that built fiber to my house.

Google Fiber may not have built fiber to your house but they definitely pushed competitively to get broadband/ISPs to do more. Cox in Arizona definitely upped their fiber rollout and bandwidth/transfer rates, mainly by using channels/docsis 3 etc because of competition from Google Fiber. I still have no fiber options to my house in a good area.

Overall broadband ISPs have failed at gigabit rollout for which they got tons of federal funds to do so. They don't put in enough in profits back into R&D or rollouts unless to an extremely wealthy area.

If you truly think broadband/ISPs care whether you have good internet or fiber then my guess is you live in one of the areas that had some competition. Otherwise they sit back on their semi-monopolies and stop innovation in broadband where they can to milk it. They could drop channels on digital tv and use more for internet and slowly have started to do that, but they are drawing it out big time. Their "big innovation" in the last decade was data caps and constantly messing with VPNs and other valid uses of broadband. They only innovate when they have a chance to further their monopoly or stranglehold on an area.

I'd say broadband/ISPs, once a bright shining innovative force in the 90s/early 2000s, have gotten lazy and are actively slowing down innovation now [1], and this lobbying to get a competitive advantage over Google/Facebook is a good example of non-innovation but using the system to slow innovators and innovation in broadband/internet services and products.

[1] https://arstechnica.com/information-technology/2013/12/why-c...

My point is that Google made a lot of noise, built Fiber to less than half a million customers, and is now throwing in the towel. Verizon, meanwhile, built fiber to more than 7 million customers, has already done one major technology transition (BPON to GPON) and is testing another (GPON to NGPON-2).

A 10x improvement is pretty great. I can't say Google is 10x more useful than it was ten years ago. And many other products by "innovative" companies (e.g. Facebook, MacOS, Windows), have gotten worse in that timeframe.

> Overall broadband ISPs have failed at gigabit rollout for which they got tons of federal funds to do so.

Factually untrue.

(The Ars article elides an important factor: upgrading speeds isn't just a matter of flipping a switch. You need to upgrade expensive head-end equipment, the lines from the headend to the core network, split nodes, etc. All of this is very expensive.)

I see you are on Verizon and telecoms team, the anti-innovation force run by the MBAs and bean counters over what is needed nationwide.

>> Overall broadband ISPs have failed at gigabit rollout for which they got tons of federal funds to do so. >Factually untrue.

Factually untrue that it is factually untrue.

To this day most people don't have access to fiber[4], that is a failure. Maybe you do but most do not.

In the late 90s there were $400 billion given to broadband providers and the job was by 2006 for everyone to have fiber running to their homes. They stole that cash [1].

Verizon is horrible at keeping it's promises on fiber [2][3][4]. Verizon has sucked from the time it was BellAtlantic, for a doctor/pharma web network right when the internet started going we used to run fused ISDN lines for 128-bit (two 64bit lines together, one for video one for data) and they didn't get one line right the first time and rarely did it meet bandwidth needs. Whenever there was a Bell Atlantic truck around everyone's network went down. They are still as sloppy and slacking big time in gigabit/fiber. But then again so is every broadband mini-monopoly.

I live in Chandler, AZ one of the first places to get cable internet with Dimension/Cox. They were innovators then, now they drag the rollout of gigabit and Cox's gigablast stopped when Google Fiber slowed down. They can't even get fiber to the place that was one of the first cable internet locations (primarily because of Intel being here).

Telcos have purposely failed on the fiber/gigabit rollout. I guess we will have to agree to disagree. Broadband/cable companies were innovative in the 80s and 90s and maybe early 00s, now they are just bean counting and sitting back with their mini-monopolies comfortably slowing innovation in the US.

[1] http://newnetworks.com/ShortSCANDALSummary.htm

[2] https://www.techdirt.com/articles/20131012/02124724852/decad...

[3] https://arstechnica.com/tech-policy/2017/03/nyc-sues-verizon...

[4] http://gizmodo.com/after-billions-in-subsidies-the-final-ver...

> In the late 90s there were $400 billion given to broadband providers and the job was by 2006 for everyone to have fiber running to their homes. They stole that cash [1].

That number is fictitious. I've addressed it at length here: https://news.ycombinator.com/item?id=7709556. The number is built out of two pieces:

1) Comparing the company's rate of return to that of utilities like power companies, and calling everything above that "excess profits." (By that analysis, taxpayers are paying Google and Facebook enormous subsidies.)

2) Calling accelerated depreciation of telecom network infrastructure a "tax break," while ignoring that accelerated depreciation made total sense in the 1990s as telcos upgraded their core networks with fiber.

> Verizon is horrible at keeping it's promises on fiber [2][3][4].

Everyone is horrible at keeping promises on fiber. See Google halting Fiber rollout. Beyond that, the question is: what's the harm? Did New York pay Verizon a bunch of money for building out broadband in New York City, only to have Verizon renege on its end of the bargain?

Here is the lawsuit New York City filed against Verizon relating to the deployment: https://consumermediallc.files.wordpress.com/2017/03/verizon.... Here is the underlying franchise agreement: https://www1.nyc.gov/assets/doitt/downloads/pdf/verizon_nyc_.... Read both, and point me to the big check New York wrote to Verizon to get FiOS service. Or even the big tax break. You won't find either. If those things existed, they'd be in there (because "and then they pocketed the money we gave them" makes for a way better lawsuit).

What you do find is that Verizon got to pay New York millions of dollars + 5% of gross revenues for the privilege of spending another billion dollars building the FiOS network out to New York City.

So on the moral outrage meter that ranks up there with Lenovo teasing the Thinkpad Classic but never releasing it. People aren't outraged because they pre-ordered and Lenovo kept their money. They're outraged because they want it and they can't get it.

> They are still as sloppy and slacking big time in gigabit/fiber.

They've deployed fiber to 15x as many subscribers as Google, and are rolling out near-gigabit (provisioned at 1024 mbps but marketed as 750 mbps) throughout the FiOS service area through 2017.

You keep going back to comparing them to Google who only entered the market really to add some competitive pressure. It worked, nearly every company re-doubled their fiber/broadband efforts. It took a non broadband company to only hint of competition and it spawned the biggest movement up in download speeds and fiber rollouts than anytime before, yet you disregard that entire effort.

Again I think we'll have to agree to disagree.

We get it, you love Verizon, I hope you are invested with this much love for a broadband company that will take you at every turn they can. But have fun if you ever need a line installed from them or make a deal with them to do so across a city.

Do you have fiber from Verizon? If not maybe take a step back.

Fiber has been largely danced around by broadband/ISPs and even Verizon is far, far from coverage that people expected by 2017.[1]

In most cases they were offered monopolies over areas in agreement with them wiring it and offering good service to all, an immense benefit for monthly customer subscriptions. They have failed so far, it is 2017 and barely anyone in the US has fiber or gigabit (<10%). We are letting the broadband/ISPs drag us down, we should be leading the world in broadband/gigabit/fiber.

Verizon doesn't actually want to run fiber.

The simple fact is that Verizon has been trying its damnest to get out of the wired business altogether. Back when Ivan Seidenberg was in charge, he made a giant bet on fiber, which is why Verizon became such a national leader in broadband with FiOS -- a service that people really seem to love. However, Wall Street has always hated it, because it's capital intensive, and Wall St. recognizes that without any real competition in the broadband space, Verizon can avoid investing in such infrastructure upgrades, and just swim in larger profits while America's broadband infrastructure suffers and falls further and further behind other countries. Once Seidenberg left, the beancounters quickly took over and looked for ways to stop all that investment. Why invest in the future if there are no competitors to push you to do so? [2]

Verizon has tricks.

A decade ago, we wrote about how Verizon had made an agreement in Pennsylvania in 1994 that it would wire up the state with fiber optic cables to every home in exchange for tax breaks equalling $2.1 billion. In exchange for such a massive tax break, Verizon promised that all homes and businesses would have access to 45Mbps symmetrical fiber by 2015. By 2004, the deal was that 50% of all homes were supposed to have that. In reality, 0% did, and some people started asking for their money back. That never happened, and it appeared that Verizon learned a valuable lesson: it can flat out lie to governments, promise 100% fiber coverage in exchange for subsidies, then not deliver, and no one will do a damn thing about it. [2]

I am with the people that want fiber across the US, beancounters and investors of Verizon want the opposite. I guess you are in the latter category?

[1] http://fiberforall.org/fios-map/

[2] https://www.techdirt.com/articles/20131012/02124724852/decad...

You can't "agree to disagree" about facts. You're throwing out assertions that are simply false (e.g. that companies agreed to deploy fiber in return for being granted monopolies). Doing that has been illegal under federal law for 20 years. Did you read the Verizon franchise I linked you? Did you notice how it said it was "non-exclusive?"

> Do you have fiber from Verizon? If not maybe take a step back.

My parents have had FiOS for more than a decade, and I've had it for several years in four different houses and apartments. I always get 5-10% over the advertised speed (even during peak times) and my ping times are similar to the Cogent business internet we have at work.

> They have failed so far, it is 2017 and barely anyone in the US has fiber or gigabit.

About 11% of all U.S. broadband connections are fiber: http://www.oecd.org/sti/broadband/oecdbroadbandportal.htm. 25% of the country can get it: http://broadbandnow.com/Fiber (FiOS's uptake rate is less than 40%). We're doing better than most of the other big OECD countries. Canada is at 8%, France at 6.5%, Germany, Italy, and Austria are under 3%. Spain, South Korea, and Japan are the only big OECD countries doing better than us.

Moreover, the overall statistic ignores the fact that broadband is largely a state level issue, and deployment varies a lot by state. E.g. Maryland, where I live, has about the same population and population density as Switzerland, and has similar fiber availability (over 60%).

> A decade ago, we wrote about how Verizon had made an agreement in Pennsylvania in 1994 that it would wire up the state with fiber optic cables to every home in exchange for tax breaks equalling $2.1 billion.

There was never any "tax breaks": https://news.ycombinator.com/item?id=8401102. (Read the thread--the guy who came up with the $400 billion and $2.1 billion numbers explains how the number is based on calculating "excess profits" compared to regulated rates).

> My parents have had FiOS for more than a decade, and I've had it for several years in four different houses and apartments. I always get 5-10% over the advertised speed (even during peak times) and my ping times are similar to the Cogent business internet we have at work.

Consider yourself lucky then. You might view it differently if you are in an area that is yet to have fiber. Here in AZ, Tempe and Scottsdale have it but nowhere else. Cox put gigablast/fiber in only in response to Google Fiber and has subsequently stopped after Google slowed, so maybe we have different experiences. It is very frustrating. You are in a very small state that is near D.C. which is one of the heavy focuses of fiber (New York, DC, CA, WA, etc). If you move to an area where it isn't as prevalent, you might not be so pro-broadband/ISP providers.

> You can't "agree to disagree" about facts. You're throwing out assertions that are simply false (e.g. that companies agreed to deploy fiber in return for being granted monopolies).

I should have said semi-monopolies like I did initially. In many cases this is true simply by being the one to run the lines and discouraging competition. In many, many cases we only have one provider for most of the US. Providers are also heavily against municipal broadband and other competitive services, only Google Fiber really scared them into increasing rollouts.

I can agree to disagree, many of the things I have posted back up my main point that fiber rollout is slow, there have been lies and incomplete rollouts and it is holding us back in the US. According to you broadband/ISPs are doing exactly what they can and pushing the limits of capabilities and innovating like they did in the 90s. That may be the case in Maryland in your area, I have a differing opinion and frankly I have never met someone so pro-broadband/ISP/Verizon unless they were investors or worked there. I do hope broadband/ISP providers return to their innovative glory of the 90s and early 00s. They are failing in that regard today.

It also gives ISPs the cover, funding and motivation to archive customers' browsing data en masse. The value of these data will be huge for both domestic and foreign intelligence.

This got lobbyied in European Law about 10 years ago in the mandatory data retention directive ( Directive 2006/24/EC [1]). It got challenged in court for being unconstitutional up to the high instances and got invalidated by the Court of Justice of the European Union for violating fundamental rights in 2014.

Some countries have decided to face fines and sanctions to keep data retention in their national law as long as they can but most have promptly killed their local implementation of data retention.

[1]: http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:320... and https://en.wikipedia.org/wiki/Data_Retention_Directive

> most [EU countries] have promptly killed their local implementation of data retention

Source?

Some countries have even made such data retention a legal requirement. For example in the UK:

http://www.bbc.co.uk/news/uk-politics-34715872

Funny thing is that law is actually illegal: https://www.theguardian.com/technology/2014/jun/24/british-g...

At least, until brexit...

> you cannot choose your ISP/broadband provider

Wait, what? Never visited the US but I assumed you had the choice of providers, just as most of the rest of the world where internet is available.

I see the reason for this to allow ISPs to have infrastructure to collect data setup, so when some organization needs* access to it, it's already there and easily accessible.

* for national security or whatever

> Never visited the US but I assumed you had the choice of providers, just as most of the rest of the world where internet is available.

Unless you live in a major metro area - where you might, if you're lucky, have two choices of cable provider - there's a reasonable chance you only have a choice between cable, ADSL over some neglected wiring to an exchange a few miles away, and satellite. So if you depend on having fast internet speeds, you're stuck with the one cable provider.

Also - it's not particularly uncommon that an apartment block is wired up for a specific provider and unable to get service from any other provider due to a monopoly contract the building owner signed. Then you also have that happening over entire areas in places, just with local Government signing the contract.

This is crazy. I never realised this. Here in the UK I switch ISPs regularly to take advantage of new customer discounts, cashback incentives etc. I'd never dream of sticking with an ISP longer than I contractually had to.

I'm in the UK too, I just have American friends... and it's a crying shame that they had the opportunity to have the same system as us - when the Bell breakup happened - but instead split up the company geographically.

That's not true. AT&T was broken up around the same time British Telecom was privatized, both in the early 1980s. The difference was that British Telecom had been a government organ (part of the Post Office until 1981), while AT&T had always been a private corporation. When BT was privitized, the government had enormous latitude in imposing conditions on investors purchasing the assets. The U.S. government was not in a similar position. An antitrust violation doesn't give the government the right to just do whatever it wants with the company's assets.

The best course of action on the part of the U.S. likely would've been to permit the AT&T monopoly to continue. AT&T's regulatory treatment at the time permitted it to set prices to ensure a reasonable return on its capital investment. That had the effect of creating an enormous incentive on AT&T's part to invest a lot in its network (the more capital investment, the more profits). As a result, the U.S. telephone system was extremely high quality for its time.

Your second paragraph flies directly in the face of everything that is right about capitalism. As a simple example, long distance call rates dropped by 40-50% after the breakup with no difference in quality.

Capitalism will give you what the market wants, not what a snob would consider "good." E.g. popular music and movies suck, but that's because that's what the market wants. Phone call quality has gone to shit compared to the POTS network of yore, but again the market doesn't care.

Keeping AT&T would've given people what folks on HN want: a wired network that was expensive built gold-plated.

The US tends to have local monopolies; in order to change ISP you need to move house, or maybe state.

This is because the national telco monopoly was broken up early into horizontal companies, rather than the "local loop unbundling" system in the UK (and I think other European countries).

LLU is a little strange because the lines are still owned by one company, so there's no choice of who does the maintenance and installation work (it's always BT OpenReach in the UK). But you do get a choice of what happens to your packets after they reach the DSLAM.

LLU is a regulation to break existing monopolies and allow some degree of competition to happen because an unhealthy internet impedes the country economy.

I'm in utah...maybe it's just different here, but we have Utopia which owns the conduit, and leases out to different ISP's who own the networking equipment. I'm with X-mission who's CEO is a big privacy advocate, and supporter of EFF. But there's google fiber in places, there's x-mission and about 5-6 other utopia-leasing ISP's locally you can choose - and that's just for fiber, you also have Comcast, Qwest(DSL), and CenturyLink, and a few others.

Even in rural Cedar City down south you have about 5 ISP choices. Though the highest speed looks to be TDS at 300MBps

In mass you are lucky if you get 2 options. My building finally got another company to connect after 12+ years of Comcast only. In suburbs around boston you get a mix of Comcast+RCN, Comcast only, Comcast+Verizon, Verizon only.

Ive usually seen 120 MBps/$80 as the max but in my aforementioned building Comcast sent us a friendly email saying we were being upgraded to 200 MBps at the same rate, the day before our building announced the second ISP available at 100 MBps/$69. That smells to me a lot like they always had the capacity, but never turned it on until they finally had competition.

Note: It looks like Comcast has started offering a 2000 MBps connection for 299 a month. Maybe it was just a coincidence that they were adding new capacity at the same time

Bay area here. Its in my lease that I need to use Comcast for any wired internet connection.

I'm so incredibly surprised this hasn't been sued out of existence for anti-competitive behavior. Not that people in the Bay Area ever have more than one choice to begin with for internet service (AT&T and Comcast seem to split the area up amongst themselves).

No. In the US almost always whoever owns the wire that leads to your house also provides your IP transit service. Since there is almost always only one wire, it follows that there is also only one ISP.

My understanding is that there is a cartel of operators in the US. If ISP 1 goes to area A, then ISP 2 will not go to area A and go to area B instead. ISP 1 will then not go to area B.

This is an effective way to maximize profit at little to no cost and is not uncommon in many markets around the world.

They've also convinced your average, market-loving American that there are options and if you don't like your current ISP, you can just switch. I'll switch from Comcast to Wave! Oh wait, Wave just resells Comcast. Ok, I'll switch to AT&T! Sorry, the nearest switch box is more than a mile from your house...your speeds are going to be ~ 400K/s.

Yay, free market.

You can choose your ISP provider, I'm lucky enough to be with X-mission in utah, the CEO runs a TOR exit node, has given MANY anti-NSA speeches, and is a BIG advocate for privacy and has guaranteed he won't be selling any of our information. And they have 1GIG fiber, so pretty much everything I could possibly want.

That's fantastic for you, but not everyone has the luxury of choice. I'm in PA, and the only option we have here is Comcast. Pretty shitty, being forced to choose the lesser of one evil.

It wasn't exactly a party line vote, with 5 more Republicans voting against it would have failed.

Senate was party line which was introduced first by Jeff Flake R-AZ, House had a handful flip but for the most part party line.

This controversy seems overblown to me. It would have been nice if these rules had gone into effect as scheduled, but let's remember that they never actually did. So nothing has been lost - ISP's have gained no new abilities, despite how the extremely biased media coverage I have seen of this paints the situation. They have been able to sell aggregated targeting data forever, and will be able to continue doing so.

This article is actually somewhat encouraging, in that it displays the limits of the laws as they exist today and will apparently remain. You cannot go buy the browsing histories of specific individuals - enemies, employees, etc. Many of the ridiculous "sky is falling" leaps of fact and logic that have been portrayed in the media to get views (ironically so that they can make money from displaying targeted advertising on articles decrying it) have been disproved by this article.

You make it sound like the FCC privacy rules were bringing something totally new to the table. That's not really correct. They were more bringing back something that had recently been lost.

Until mid 2016 ISP privacy abuses could be dealt with through the FTC, basically the same as privacy abuses at most other internet businesses are dealt with.

In mid 2016 a court ruled that the FTC did not have authority to deal with these issues for common carriers, which ISPs had been since being reclassified as part of the 2015 net neutrality rules. The result was a significant loosening of privacy regulation for ISPs.

The FCC rules would have undid that loosening.

Thanks for explaining this. I didn't know the full chain of events nor the FTC's involvement and how it affected the 2016 FCC rules. This makes it much more clear.

> You cannot go buy the browsing histories of specific individuals - enemies, employees, etc.

You could buy pretty detailed documentation, including internet usage history, from IRC bots on the shadier shades of the doxxing market 15 years ago.

Not everybody's would be readily available or as detailed but if you could afford it you go for on-demand doxxing, the more detailed and the more high profile target the more expensive. Among the data extraction methods used, one was gaining remote access to target's computer, collect the data collected, curate it and sell it.

So yes it was possible to buy browsing history of specific individuals 15 years ago, so I would be surprised if it wasn't still possible today. Don't ask me where to look, I haven't gravitated under these shades since 15 years ago.

Of course you can buy a non-identifying information perfectly legal. And then guess what's the definition of non-identifying information? Think again. If ISP thinks it sells you that, nothing stops you to look at that yourself closely - and may be, just may be you'll uncover something which ISP missed. After all, scrubbing data to be perfectly non-identifying could be, you know, expensive. And doesn't exactly fit ISPs business model. So sharing even Congress-related history could be an interesting thing.

You're right. My guess is most "anonymized" data is lazy--stripping out obvious identifying information. There are quite a few cases where you can pinpoint users from correlating to publicly available data. CGPGrey posted a video today showing a layman's example with Social Security Numbers[1].

One cool thing I heard Apple talk about[2] was injecting noise or subsampling to help mask individuals. Although, I don't see that mentioned in their public page on privacy[3] or their whitepaper[4].

[1] https://youtu.be/Erp8IAUouus

[2] https://www.wired.com/2016/06/apples-differential-privacy-co...

[3] http://www.apple.com/privacy/approach-to-privacy/

[4] http://images.apple.com/business/docs/iOS_Security_Guide.pdf

Buy aggregate browsing history for users in DC and nearby area codes. Look for browsing of .gov websites that are effectively government intranet tools. Correlate with access to RNC sites, Infowars, FoxNews, etc. While I'm sure you would both miss people and have some false positives, you'll likely also identify many members of congress and their staff.

Google searches have the query in the URL... probably most "important" people google themselves occasionally.

Why are important people searching google without https?

Important doesn't equal or imply smart nor being good at what you do.

They aren't. Google upgrades to HTTPS automatically, and the old method of overriding the DNS to an unsecured Google server stopped working years ago. Same with Bing, Yahoo, DDG, etc. No SSL, no search in today's world.

HTTPS will not protect the URL of the request, just the contents though.

This is wrong. The domain name is revealed through SNI but the path of the URL is sent as part of the encrypted HTTP request.

HTTPS protects URL parameters. That stuff doesn't go to the DNS request.

Prohibiting the sale of individually identifiable information is a very weak protection. It's alarming how quickly multiple dimensions of "anonymized" data can be used to zero in on any target person, and it won't take long for amoral opportunists to start selling de-anonymized data to the highest bidder. See also k-anonymity.

Combine the data with publicly available information about politicians and other public figures, feed it to a machine learning algorithm and I bet you could get very high accuracy results very, very quickly.

Look no further than the news in Congress last week. Devin Nunes' claim is that he saw conversations where names were masked, but it was obvious who they were about.

When doing this procedurally, it's amazing how few data points are needed to pinpoint a specific person.

One should check out the talk of 33c3, where a group of german journalists bought this type of data and deanonymized some users.

https://media.ccc.de/v/33c3-8034-build_your_own_nsa

tl;dw Even worse, they found stuff like links to a police detective's application for a search warrant in the "anonymized" data.

Oh, and of course they found and deanonymized some German politicians and their staff.

So while it might not be possible to deanonymize and "expose" all of Congress, if you really tried and got some money to actually buy the data, you will be able to at least deanonymize some of your targets, especially if the data originates from a man-in-the-middle like an ISP and not just some random ad network/tracker.

So I was a call-in on NPR yesterday(http://www.wbur.org/onpoint/2017/03/29/internet-privacy-cong...) that discussed the ISP privacy issue. I brought up the cried fund and all of the guests immediately jumped to the defense of the data seeking companies saying how these companies were too reputable to "allow" such a thing to happen. I'm finding all of this love for these previously unknown companies very strange.

Your link is incomplete. I think you meant to link to http://www.wbur.org/onpoint/2017/03/29/internet-privacy-cong...

I'm still trying to figure out what, "the cried fund" is exactly, but I just DL'd the 22MB mp3 and will listen. Direct mp3 link for anyone else:

https://dts.podtrac.com/redirect.mp3/traffic.megaphone.fm/BU...

I don't see this as a particular obstacle. Yes, I can't buy Paul Ryan's data individually.

But I can buy anonymized browsing and demographic data for the downtown DC zip codes, and de-anonymization is not particularly hard when you have so much information to work with. Zip code, gender, and birth date together are enough to get you to 87%.

> You can’t buy Congress’s web history

They probably don't understand or know that, though, so why not give them a scare?

Right. Only us smart HNers living in SV know anything about tech.

That's a stupid straw man.

> In the meantime, the two biggest campaigns have collectively raised nearly $140,000 for the purchase of web histories that will never go up for sale. It’s anyone’s guess where the money will end up.

Well, NSA staff did LOVEINT. Maybe some ISP staff will want to profit.

Edit: But I wonder how it'd be possible without self-pwning.

Didn't Netflix pull their 2nd competition because people figured out how to de-anonymize all the anonymized data? Can you not figure out who someone is by repetitive filtering?

> that's not how any of this works (theverge.com)

While the publicity pieces may be poorly stated, (in particular one should never use the word "individual" in any context) the desired teachable moment is based on buying data or access to a segment of at least 500 individuals based on factors like travel habits, income and demographics. (In some proposals that segment could be as high as 5-10K as I see mention of congressional aids and the like.)

That desire could be quite easy to achieve and quite hard to block without upsetting the intended market for ISP data since being able to buy affects on segments down to such granularity is exactly how the market works.

If the data can be used to target ads at individuals, then it clearly isn't anonymous in any meaningful sense of the word.

For those that say this is impossible, take a look at these slides from Dstillery, a major demand side advertising company: https://www.matroid.com/scaledml/2017/claudia.pdf Pay particular attention to slide #23, and also the following few slides, which shows the architecture of their data processing stack. They are deanonymizing most of what people do on the web, and they undoubtedly have a file on most all members of Congress. Part of the input to this model is the "anonymized" location data that mobile phone companies sell. If you already know where a person lives and works though, or simply have a few retail transactions with timestamps, you can deanonymize these location traces.

>you want to get really clever, the Wiretap Act also makes it illegal to divulge the contents of electronic communications without the parties’ consent, which arguably includes browsing history.

So wouldn't it be illegal for sites to sell visitor information (email lists)?

Also, people were talking about buying Congress's search history, which of course ISPs don't even have - only Google does and their ability to sell it was not in any way affected by this rule that covered only ISPs.

Those people are confused, at least for searchinternethistory.com the plan is to buy internet history and make it public and searchable.

Then again, I'm pretty sure google does not know about what people search on bing or yahoo, and it seems to me that what you search on google appears in the URL so your ISP would know.

> what you search on google appears in the URL so your ISP would know.

Well no, Google searches go over HTTPS, so everything but the host is encrypted.

Thank you for fixing my shameful mistake, don't know what I was thinking there. my bad!

Too late to edit so I'm replying to myself because I made a mistake here. Google search has enabled https following the snowden revelations so the ISP is only able to tell one has been to google search website at a specific time but not what one has searched.

It was my understanding that there would be some deaggregating and deanonymizing to be done on the data.

Hasn't this been shown repeatedly that it is not only feasible but also quite easy ? It seems to me that the verge missed the point here.

Lotta speculation in the news for something that could be answered by looking at the data. Since this isn't a new program, has anyone seen it? (please no speculation)

This raises an interesting question: who is the ISP for congress? My guess is the government itself, but anyone have anything more specific on that?

Using the history of congress editing wikipedia pages I found the answer -- they are their own ISP:

http://whois.arin.net/rest/net/NET-143-228-0-0-1/pft

http://whois.arin.net/rest/net/NET-143-231-0-0-1/pft

https://en.wikipedia.org/wiki/Wikipedia:Congressional_staffe...

https://www.whoismyisp.org/ip/143.228.255.255

This led me to an idea: why not buy online ads targeted by IP address at the IP ranges of congress? It would be extremely targeted since it would only be seen by computers on the hill.

We detached this flagged subthread from https://news.ycombinator.com/item?id=13993120. This account has been posting a lot of unsubstantive comments, so please stop and re-read the guidelines:

https://news.ycombinator.com/newswelcome.html

https://news.ycombinator.com/newsguidelines.html

I really don't think you can apply any sort of free market principles to ISPs.

They're almost all regional monopolies or duopolies -- with franchise agreements and, in many cases, state regulations/laws that make it hard for anyone else (including municipalities) to enter that market.

Unless you're someone big like Google or Verizon then you're not going to be able to enter the last mile ISP market and I don't see much incentive for companies even attempting to do that outside of highly populated urban areas.

>I really don't think you can apply any sort of free market principles to ISPs.

Shouldn't we then work towards that. Instead of applying more regulations.

Why prefer that to having a good set of regulations?

Well, one good argument would be that there's a party which'll tear down any reasonable set of regulations every eight years. You could argue that the solution is to convince people that electing this party is a terrible idea, but... that doesn't look to be happening any time soon.

Fix the market first, then deregulate.

> Discrimating between companies leads to communism.

Pardon me for being blunt, but that is complete nonsense. If that was the case, the US would have become a communist state a long time ago.

> Good opportunity for isps to gain market share by offering privacy protection.

By that reasoning, one might allow car manufactures to leave out seat belts, so they could "gain market share" by offering to install seat belts as an expensive upgrade.

You mean people will get the opportunity to pay their ISP not to sell them out? Sounds pretty consumer friendly to me.

Any way we see this new development, this weakening of privacy is a big win for telecoms and its sustained and vast efforts at lobbying Congress. Bluntly, Congress sold out the American people...anywhere else (I believe) and the ability for entire industries to buy votes would be called corruption, especially when elected representatives remove consumer protections (whoever gave them that mandate?!) in favour of (a paying) industry.

[0]- The 265 members of Congress who sold you out to ISPs, and how much it cost to buy them (http://www.theverge.com/2017/3/29/15100620/congress-fcc-isp-...)

You know, it is possible that these members didn't vote because of some quid pro quo relationship but because of ideological beliefs that aligned with voting this way. Maximizing liberty and 'leveling the playing field' (WRT FANG) seem like beliefs one could reasonably hold. The end doesn't justify the means to all people, and reaching decisions one doesn't like is sometimes the outcome of remaining true to one's principles.

No ISPs will get opportunity to differentiate themselves.

I don't know where you live, but where I live there is no choice for which ISP you use. This is true for many, if not the majority of people in the US. I am unable to use the ISP I want to use because it is blocked in my building by a building-exclusive contract with Comcast. So no ISPs have any chance for an opportunity.

Can you lobby your building administration with the help of your fellow residents? I would get a hold of the exclusivity agreement it and comb over it to look for loopholes, and also determine when it expires.

I've tried. They just stonewall me by claiming that the building is only wired for Comcast and refuse to communicate with me further on the subject.

edit: It's an apartment building not a condo. In my previous condo building it would have been easier to discuss this issue.

Those are called VPNs.