Maybe I am just being paranoid...

However my concern is more to do with thinking as an outsider. I have toyed with idea of a company that requires establishment of trust among users. Although it may seem as simple as do no evil on the surface, it is an extremely hard undertaking. The thing with Cloudflare is that its niche is its own problem, "Man in the Middle", even if I were to place trust in the privacy policy, there are other things I need to be worried about. What if there is a break in and a hacker places a sniffer to sniff the communication between Cloudflare and the client sites? Sure the client followed the best practices such as, one way hash, strong crypto (eg bcrypt), 1000+ iterations, database encryption, https communication, etc but all of it is pointless or redundant now because the browser ends up sending the password in plaintext. Now imagine if instead of an hacker its a govt agency and that too with a gag order. See my concern?

CloudFlare gets to see the cleartext of all traffic they serve as they MITM HTTPS connections.

CloudFlare's (non-enterprise) prices simply aren't even in the required order of magnitude.

Now: whether or not metadata, request bodies, etc. are logged, and to what scale, is another story/discussion of possibility.

At some small, targeted scale, it's safe to say that total duplication (certainly request bodies, etc.) is possible, if they were so interested.

Suppose the IP was behind a fat enough pipe, why not load balance behind it instead of DNS load-balancing in front of it (and additionally behind each as I presume now happens)? Also, if that IP was anycast then you could ignore the issue of client latency as well, assuming you have the necessary private network behind endpoints to manage state.

If you don't like/can't solve the problem at the level of IP anycast, when not leverage a third-party anycast DNS and just have a few fixed IP for specific geographic locales, again with fat enough pipes and load balancing behind them.

I guess what I'm saying is that there's no reason for an organization, a monolithic entity, to have more that a handful IP addresses at most.

Another thing they can do is use anycast to load balance across data centers. So, if a data center rather than a website is a target - the attackers will need to know which IPs to attack. They can start flooding the broadcasted IPs from a particular route. However, if this happens then hypothetically Cloudflare could just stop broadcasting the IPs at this particular data center, re-broadcast them at all the surrounding data centers, and basically spread out the attack load across multiple sites. If the attackers change the IPs that they target based on new routes, then Cloudflare can continue fast-fluxing the IPs every 5 minutes and mitigate the attack.

It's pretty cool use of BGP and anycast, but being able to change IPs of website and where they are broadcasted in real-time is core to Cloudflare's security.

Any research on the real entropy of (source,port) entropy on the Internet? The are also real issues like the distribution of (source, port) is hardly uniform, and is especially nasty when undergoing an attack, i.e. you want to manage latency based the both the distribution and authenticity of traffic.

This is a very interesting mathematical problem. I have to work on expressing it a bit better before I can hope of formulating a solution, but yes I can totally see now how leveraging BGP, anycast, and DNS TTL are all knobs to heuristically solve this problem, instead of a some crazy genius way of making use of router TCAM silicon.

In order to protect latency to other GET targets, you're going to have to start doing interesting things.

One future solution I can see is multipath-tcp the anomalous traffic, and closing the original connection. But at that point you have to refilter based on genuine vs malicious traffic, and then there's the encrypted state you have to share for the proper stream handover. Ooof... what a nightmare.

At least it's an interesting one. :)

https://www.cloudflare.com/ips

2. People attack IP addresses. Handy to be able to change the IP address of a web site.

3. Countries block sites based on IP addresses. Handy to be able to move sites around to prevent collateral damage.

I guess I'm asking this because of how woeful looking the "load-balancing" solutions are from the major cloud providers. I feel they way they're externally documented, and how their APIs are specified, hitting them with more than a 40Gbps fat-server's load of traffic will cause issues, regardless of how many hosts you have serving that load.

I'd appreciate some insight from those who handle such crazy amounts of traffic.

1) Anycast doesn't give you fine grain control. Once we announce our anycast routes, what traffic actually gets sent where is out of our control - it is based on the peering arrangements of our transit providers. If we need to balance traffic between our pops, we need finer grained control than a single anycast IP.

2) IP addresses get blocked for all sorts of reasons (looking at you China!) If all customers were on one IP address, as soon as China decides to block one customer, they are all blocked.

3) Anycast sometimes has weird behavior. For example, traffic might be sent to a datacenter that might be close in terms of peer links, but far in terms of physical distance and latency. Using DNS, we can route around these issues.

I am not sure what you mean about the "40gbps fat-server's load of traffic" causing issues. We handle many customers that push more than that.

I guess this is step 1 in the same effort from CloudFlare, before they add AWS and Azure. But their interface is over-simple, understandable considering the technical proficiency of their average customer.

CloudFlare is too one-size fits all, but from a business perspective it's totally understandable.

I know it's a pipe dream, but I wish we could defragment the IP space and clean up the BGP tables. It would at least make anycast more reliable without resorting to DNS tricks like edns-client-subnet.

As for IP blocking, if undesirable sites are behind the same IP as publically demanded ones, it could make blocking actions harder to get the populace to support. But worrying about authoritative regimes is not my concern. After all, why make a service accessible if you cannot monetize the user base sufficiently.

Yes, I'm a little jaded.

Feature, not a bug.

But the real issues are the ones that outline above.

It would fit in well with their silent yet never ending reach across the internet.

You mean this service? https://developers.google.com/speed/pagespeed/service

PageSpeed Service is in a limited field trial, and is not currently accepting new signups

If google still doesnt think it has a marketable product after 4 years.. maybe a cloudflare acquisition isn't too far fetched.

It sounds like they now have a peering agreement so Google can directly communicate with CloudFlare's network, resulting in 2x faster performance. It looks like that's the primary benefit (other than the regular benefits of CloudFlare).

>2x Web Performance Speed - CloudFlare uses advanced caching and the SPDY protocol to double web content transfer speeds, making web content transfer times significantly faster.

That should be speeds.

And that made me a little uneasy.

[1]: https://projectshield.withgoogle.com/en/

And that's what they have built using Google Cloud Platform: http://www.cloudways.com/en/managed-google-compute-engine.ph...

Not sure why you think most Google Cloud Services are in beta.

The Google Cloud products page [1] lists 17 main products. Two are in alpha (Container Engine, Deployment Manager), one is in beta (Pub/Sub).

The rest are fully supported. There are some beta features here and there...but saying "most" are in beta is certainly not correct.

[1] https://cloud.google.com/products/

Wait a second. I just clicked though the items on that page, and the following are listed as alpha:

  Container Engine
  Cloud Dataflow
  Cloud Deployment Manager

  HTTP/HTTPS Load Balancing
  Virtual Private Network
  Cloud Pub/Sub
  Cloud Monitoring
  Cloud Logging

In some cases, there are GA products with alpha/beta features & languages, so it may be difficult to figure out the best way to communicate at the top level (e.g., the examples pointed out for a given feature in a given language, or HTTP load balancing vs Network Load balancing).

But in cases where it's clearly a beta product, it should be clear.

What's the issue, here?

Autoscaler and Instance Groups on compute engine are beta as well.

    reddit.com.		22	IN	A	198.41.209.143
    reddit.com.		22	IN	A	198.41.208.141
    reddit.com.		22	IN	A	198.41.209.137
    reddit.com.		22	IN	A	198.41.208.139
    reddit.com.		22	IN	A	198.41.208.143
    reddit.com.		22	IN	A	198.41.208.142
    reddit.com.		22	IN	A	198.41.209.139
    reddit.com.		22	IN	A	198.41.209.141
    reddit.com.		22	IN	A	198.41.209.138
    reddit.com.		22	IN	A	198.41.209.140
    reddit.com.		22	IN	A	198.41.208.138
    reddit.com.		22	IN	A	198.41.208.137
    reddit.com.		22	IN	A	198.41.209.142
    reddit.com.		22	IN	A	198.41.208.140
    reddit.com.		22	IN	A	198.41.209.136

    news.ycombinator.com.   76  IN  CNAME   news.ycombinator.com.cdn.cloudflare.net.
    news.ycombinator.com.cdn.cloudflare.net. 124 IN A 198.41.191.47
    news.ycombinator.com.cdn.cloudflare.net. 124 IN A 198.41.190.47

Save for the scenario of expensive, relatively difficult-to-implement pieces of crypto hardware (and even then, a nation-state could probably defeat it), your traffic is likely vulnerable to determined aggressors.

It's one thing to possess such high-end, esoteric security technology, it's another thing entirely to implement it (and protection for other far more realistic attack vectors) at a CloudFlare-number (or other CDN) of global locations.

I guess the bit that really grates on me is the "just give us your private keys, and trust us!" approach, especially with someone who's then routing not insignificant percentages of the total web traffic through their infrastructure.

Snowden showed us the NSA can and does target "high volume" opportunities for mass surveillance - if you look at the PRISM slides and estimate what percentage of global email their "top ten" targets represents, how much would you bet against them already having a similar program in place backdooring Cloudflare (and Akamai and all other significant players in the SSL CDN market)?

It's probably a false hope, but I feel my own SSL cert on a VM on a more Lavabit scale "local colo" is - while no safer from a targeted NSA/GHCQ/DSD probe looking for _me_ specifically - still significantly less likely to get caught up in a firehose scale "collect all the things" program.

Although, it's probably just as likely a "red flag" that marks me as a "potential terrorist" at least as accurately as having a public PGP key or a secure messaging app… :-/