Best for the internet is encouraging IPv6. There aren't extre fees for that, so people with less money are equal.

Meanwhile, if Amazon is going to use all these in the medium-term future, that seems OK to me.

(3.3.3.3 and 3.2.1.0 would be more memorable.)

While 3.2.1.0 is a valid IP address, some users might experience problems with IP address ending in 0. It would not be a good idea to use 3.2.1.0 if you want best compatibility with all available hardware/software on the market.

Cloudflare's work with 1.1.1.1 has demonstrated not only that it's _possible_ to use a previously-unusable IP address, but that it's _valuable_ to do so because it compels reporting of – and repair of – broken hardware and software on the market.

The cynic in me thinks that inbound traffic on 1.1.1.1 is "valuable" mostly to help balance Cloudflare's inbound vs outbound data flows. This balance is important in deciding whether their peering is settlement free.

They're a CDN and send out a lot of bits, so every last bit of inbound traffic helps. Or am I totally off base on this?

Yes, you are off base. We don't care at all about the traffic ratio with 1.1.1.1.

I would be very curious to find out what the actual bandwidth looks like.

I vaguely recall a site that that shows approximate throughput for given properties but I can't remember the terms to google for to (re)find it. But 1.1.1.1 might not be listed - or the numbers might only represent valid traffic and be wildly inaccurate.

Traffic breakdowns would thus be very interesting to see (for whatever is shareable, at least - heh, you'd probably have enough fireside stories to drown multiple HNs with :) ). In the case of 1.1.1.1 I'm wondering about what percentage of traffic is coming from valid DNS clients (and I also wonder where the requests are coming from - although that's just trivia to me), versus eg nonsense from horribly misconfigured lightbulbs.

I'm 95 percent sure that every dns response is bigger than the request, so it might not help much. Although if they're getting a lot of garbage in it might be helpful.

It depends on what methodology is used to classify traffic as inbound/outbound. The large DNS response in your example counts as outbound packets, but it's part of an inbound TCP connection. I don't know how peering agreements work.

Interesting.

I would have thought that they would go with the simplest measure possible, bytes in vs bytes out.

Pardon the ignorance, but I'm curious: why is it important for them (or anybody) to receive inbound traffic, given they offer 1.1.1.1 for free? And also what does it mean "This balance is important in deciding whether their peering is settlement free."?

I'm not a network engineer, so take this with as much credibility as you'd give an explanation after a few beers:

Most network providers have peering agreements to handle reciprocal traffic flows. In other words, if you're Comcast, you send a shit-ton of traffic to, say, Verizon. But Verizon also sends a shit-ton of traffic to you, as well. Generally, companies will have peering agreements that express the price for which they will route traffic for other network providers, and generally if there's a lot of reciprocal traffic, the peering will be "settlement free" - in other words, neither party charges the other to route traffic. So, in the example above, you as Comcast would agree to route Verizon's traffic across your network for free as long as Verizon did the same.

Cloudflare is a CDN, which means they push a LOT of traffic out across a lot of networks, and it's likely they're not getting as much inbound traffic as they're pushing out. That makes it harder for them to negotiate settlement-free peering, since they're not providing as much reciprocal value to their partners. By owning 1.1.1.1, they can now claim any trafic sent to that IP as "inbound" to Cloudflare's networks for the sake of peering agreements. Since 1.1.1.1 gets a bunch of traffic from either misconfigured equipment or people doing silly tests, routing that traffic helps improve Cloudflare's ability to negotiate better peering agreements.

Which, honestly, is pretty clever, since most of that traffic is garbage.

Balanced inbound/outbound traffic only matters for transit networks. That is, networks which are neither the source nor destination of the traffic they carry.

Nobody expects a residential ISP or a CDN to have balanced flows at part of a settlement-free peering agreement.

Cloudflare as a CDN with a lot of peering connections likely helps reduce internal traffic for other carriers by virtue of having endpoint data sources close to the destinations of the data.

I wouldn't bet my life on it, but I've never experienced a problem with an IP address ending in 0, and I've been using them for over 10 years. Our primary VPN end-point used to end in .0 - so we had lots of opportunity to find chance for breakage. :-)

I have, actually. Most of our networks were /23s, and the syslog server was a .0 in the middle of one. An SMS sending appliance we bought would not accept IPs ending in .0. Fortunately, the check was client-side in javascript, and once overridden, everything worked.

IIRC, IPs ending in .0 or .255 were unreachable from some older Windows systems.

That's been my experience too.

Even in 2003, the network I was involved in was handing out .0s out of larger CIDR blocks in DHCP leases with no issues.

Anything that cares about the zeroth IP of a destination not on the local subnet is broken. Anything that caters to broken is worrying about the wrong things.

Catering to the broken is like 90% of every developer and data scientist's day.

I've spent the past two weeks just trying to fix some horribly wrong/invalid data that is absolutely positively guaranteed to always be correct due to certain standards and lots of money and one of the biggest best tech companies in the world stands behind it.

Guess what it's still broken they just refuse to admit it.

Jon Postel (who wrote an early specification) of TCP would like to have a word with you:

"be conservative in what you do, be liberal in what you accept from others"

I like the spirit of this, but the problem is that it only works if everyone operates at least somewhat in line with it. If you're liberal in what you accept, then people are going to start being liberal in what they send, and soon we'll have a proper mess on our hands.

And there's no takebacks -- once you start permitting something, you can't take it away later, or you'll break things. You're creating more legacy requirements that may cause you problems when you try to evolve in the future.

As a web developer, I can only dream of how clean web standards might be today if the browsers of yore were a bit more conservative.

Do you have an example? I could possibly imagine this being a problem on a local /24, in that after you apply the netmask the local part is all zero bits. But for a remote DNS server, I cannot fathom how this could be a problem.

r-really? What do people do with routers that use 192.168.1.0?

To potentially clarify here:

Many home routers issue IPs using the follow netblock:

Network: 192.168.1.0 Netmask: 255.255.255.0 (/24) Broadcast: 192.168.1.255

Gateway: 192.168.1.1 (the first valid IP in that block) or less often .254 (the last valid IP in that block)

So for the past thirty years, nearly _every_ network block issued by _any_ network admin used /24, to the point that some things simply refuse .0 and .255 as valid IP addresses.

The suggestion upstream is to set up the IP 3.2.1.0, within _any_ network block that can contain it legally. If you adhere to strict netmasking, that'd be (this is from memory, correct me if I'm wrong):

Network: 3.2.0.0 Netmask: 255.255.254.0 (/23) Broadcast: 3.2.1.255

Which would provide a contiguous block of valid IPs from 3.2.0.1 through 3.2.1.254, _including_ the two perfectly valid IPs 3.2.0.255 and 3.2.1.0.

Those valid IPs end in .255 and .0, which is completely legal since they're in the middle of a /23, and the default gateway wouldn't be anywhere near them.

A lot of human-operated networks will blacklist issuing .0, .1, .254, .255 on the theory that it's simpler to just prohibit them and lose 4 IPs per /24 within a block, than deal with what's truly possible for a /23 or wider block.

EDIT:

Public BGP requires /24 or wider, and /24 cannot contain .0 or .255 as a valid IP, so the next widest is /23, which can contain even-numbered .x.255 and .x+1.0. If you're doing this on an internal network that permits narrower subnets, the narrowest available would be a /30, based (irregularly) on either .1.254 or .1.255:

Network: 3.2.0.254 (or .255) Netmask: 255.255.255.252 (/30) Broadcast: 3.2.1.1 (or .2)

With 3.2.1.0 as the live IP and either 3.2.0.255 or 3.2.1.1 as the in-network gateway for that device.

> Network: 3.2.0.0 Netmask: 255.255.254.0 (/23) Broadcast: 3.2.1.255

In classful addressing it is a network 3.0.0.0 with a netmask 255.0.0.0 (/8). In a classless, it is whatever has been advertised to me.

> Public BGP requires /24 or wider, and /24 cannot contain .0 or .255 as a valid IP, so the next widest is /23, which can contain even-numbered .x.255 and .x+1.0. If you're doing this on an internal network that permits narrower subnets, the narrowest available would be a /30, based (irregularly) on either .1.254 or .1.255:

It does not. You can advertise whatever the hell you want. What matters is what the other side would accept. It used to be that no one would accept anything less specific than /24 in a class C address space, /16 in a class B address space and /8 in a class A address space because the internet ran on AGS and maybe AGS+ which just did not have enough memory to do real classless. That stopped around the time GlobalCrossing showed up (or maybe around the time Mr V did terrible things while running AS7007). But anyway, ten years ago /27s are definitely accepted routes by those that comprised over 80% of the gloabal internet by the single origin prefixes.

But /27s, or anything from /24 through /30 inclusive, won’t include a .0 or .255 host. Only /23s and smaller or /31 and /32 can have hosts with .0 or .255

Are there any publicly advertised, unsummarised /31s or /32s in the global BGP tables?

You are correct:

Only /31s could include .0 and .255 and those would end up on router point to point links.

> Are there any publicly advertised, unsummarised /31s or /32s in the global BGP tables?

I filter them out as martians, so I cannot answer this.

> Public BGP requires /24 or wider, and /24 cannot contain .0 or .255 as a valid IP

That requirement is about how you advertise your routes not how you subnetted the internal network. .0 and .255 of your advertised /24 can work fine as /32s on the server. Using the internet FW to NAT the network and broadcast addresses is another common trick to get more addresses, particularly when you only get an e.g. /29 from a carrier who is just routing that /29 to your specified next hop and 2 more addresses is a big bump.

You can also use a fake IP for the default gateway on a wholly different segment with local segment ARP, or an IPv6 default router with 4-wrapping, or set up a router that uses promiscuous mode to capture all packets it receives from your “default route eth0” host. These are all irregular solutions, though, so I didn’t want to go too deep into the confusing weeds.

It's probably less of an issue when you're on a local network with the device you're trying to connect to. That said, I think 192.168.0.1 (ending in a 1) is much more common.

A /23 with the router in the middle of it? Surely that can't be common. If someone actually chose that (to be annoying or security through obscurity or whatever) then they know what they're doing

DHCP giving out a client address of 192.168.1.0 in the middle of a lagre block sure, but a router?

Routers being the “bottom of the block” in a subnet is just convention. In fact I’d say someone with a router in the “middle” of a block probably knows exactly what they are doing.

And thus would be able to change it if they actually had a problem with ip stacks failing on valid .0 or .255 ips

I'd rather companies like Amazon or Microsoft has them, in a way that people can cheaply rent them along with compute power at competitive rates, than companies like "Prudential" or some giant bank sit on them.

I might be tempted to use 3.1.4.1, or 3.1.4.2, if you round. Or those could resolve to a Raspberry Pi AWS instance :)

I'd avoid the rounding by using double digits...

3.14.15.9

3.141.59.27

But instead I would want a webserver on there that just serves up digits of Pi.

Twist: it serves up digits of e.

Twist: it serves up the first 10 digits of pi followed by e[10:].

Now that's just evil. (Better do it from the 100th digit or so to make it less obvious.)

It opens a stream á la websocket and just pipes digits down.

That would have a different answer based on whether you rounded or truncated. It would be 3.141.59.26 if you truncated.

It would also be 3.141.59.26 if you round half towards even.

It probably wouldn't be better for the health of the internet.

At this point it seems like a desperate play by a company with deeply entrenched IPv4-only infrastructure (hi EC2) to eke out more time without major upgrades. Meanwhile IPv4 addresses remain scarce for small ISPs, and the (healthy, natural) push to IPv6 infrastructure continues apace everywhere else.

AWS is IPv4-only? Can you elaborate? It seems IPv6 is pretty well supported.

https://docs.aws.amazon.com/vpc/latest/userguide/get-started...

I wasn't implying it was IPv4 only, I was merely teasingly repeating rumors that a lot of low level infrastructure still has IPv4 assumptions that Amazon has had enough IPv4 addresses to date that they don't need fix those underlying assumptions in their data centers. It's mostly dumb rumor mongering, I suppose, as I'm not sure I have a citation for that off hand other than other HN jabs at Amazon/AWS. I'm sorry for that, it was an unnecessary jab/tease, that I will blame on pre-coffee brain.

They used to -- that's why IPv6 was so slow to arrive. But that's fixed now except possibly in some parts of "legacy EC2".

We have IPv4 and IPv6 public interfaces on our AWS services.

The people desperate to use IPv4 are not AWS themselves. AWS doesn't buy stuff just to have it. They buy it because customers are using it. AWS customers are moving stuff out of their data centres and don't know how to use serverless, load balancers, dynamic cloud capabilities (like spinning down instances when they aren't in use) and so on. Customers are doing lift-and-shift in vast volumes and they're doing it in the only way they know how.

This. A lot of those lifted apps will be legacy services built in 2000-2009 that must run 24/7 and only know about ipv4.

Why specifically -2009? What happened in 2010?

(I'm aware a lot of things happened in 2010, but (as someone interested in the industry but somewhat removed from it) I'm not aware of anything that specifically happened in 2010 that changed the landscape in a big way.)

AWS has heavy support for IPv4 because back-end stuff is all IPv4 and there isn't any push to change. IPv6 doesn't add any real value for servers that are all living in a private address space anyway, it adds an overhead to each packet, and learning off your addressing schemes is far easier with dotted quad.

IP isn’t supposed to have “private address space”. The only reason it does is because the address space is scarce. NAT does not make your network “private” or provide security, firewalls and good host software do that. NAT literally breaks IP.

RFC 1918 (Address Allocation for Private Internets) disagrees with you, and has done since at least 1996.

RFC 1918 describes space for entirely private networks, not “private extensions” to the public globally routable internet. There’s a subtle but important difference. The private addresses strictly aren’t routable on the public internet. NAT (which there’s also an RFC for) is what what allows packets to flow between private disjoint IP networks. IP is only scoped to work within a single network, whether it’s public or private or triangle.

That screws up when two private entities merge and discover that they're using the same RFC1918 block.

(or just want to interoperate, even)

End-to-end principle disagrees with you. As does my network. All routable IP space baby. Makes things a breeze.

IPv6 got a private address space, even if we shouldn't run out of IPv6 addresses anytime soon.

Then what I need is a DNS client on my PC which can use all these major DNS services, but at random to provide incomplete DNS info to any given one, so it'll maybe query any random two, confirm they respond the same to verify nobody's playing unfair.

DNSCrypt-Proxy[1] already supports the randomized DNS resolver need[2]. It also supports DNS-over-HTTPS, providing assurance that even your ISP won't be able to snoop on all of your DNS queries.

While it doesn't support live comparison of DNS results, it can log out entries per DNS resolver and you can post-process those logs to validate their responses against each other, considering your queries will over time hit different resolvers. Not perfect since there are legitimate reasons to return different responses over time, but it's something.

[1] https://github.com/jedisct1/dnscrypt-proxy [2] https://github.com/jedisct1/dnscrypt-proxy/wiki/Load-Balanci...

DNS servers often return different results to different clients for load balancing, so checking multiple resolvers would lead to many conflicts.

On Linux (and OSX?) you could run a local resolver configured to do round-robin? Comparing answers would probably be new functionality though.

> Then what I need is a DNS client on my PC which can use all these major DNS services, but at random to provide incomplete DNS info to any given one, so it'll maybe query any random two, confirm they respond the same to verify nobody's playing unfair.

What kind of tricks are you afraid of these DNS services could get up to?

Well, DNS is both a significant way one could profile what types of information a given connection is used for and could be used to profile you, so I want to give a DNS provider incomplete information where possible. And as DNS is also a common way to redirect or censor traffic, I feel like there'd be value in a system that would double check that everyone's sending me to the same place. If I get inconsistent results, I could dump both and query two other ones in the pool.

The problem I have there is that Akamai and other CDNs seem to return different results if I'm using my ISP's resolvers vs any others. Getting good local nodes is important in a bandwidth constrained (and far away) environment like Australia.

> Coming soon: Amazon launches free DNS service on 3.3.3.3 and 3.4.3.4?

Do they then cover Seattle in stickers and chalking with 3.3.3.3?

Wish it was 3.14.15.92

-ss

Nah, they'll assign that one to something python related. (Get it? pi-thon. I'll see myself out...)

It would get used up quickly and we'd be right back where we are.

Amazon rank up there just behind Google and Facebook in the list of companies i don't want harvesting any more data from me.

IP block allocation has nothing whatsoever to do with harvesting your data.

Gp was probably reacting to the dns idea.

Amazon is on the extreme end of the customer-privacy side. Wtf are you talking about? If you're going to put them in a bucket, they belong with Apple.

Amazon’s recommendation engine says differently.

Retail in general wants to know a lot about you.

PS: I don’t know about Amazon Echo, but it’s definitely on the creepy side of data privacy.

Why not 3.0.0.0 or 3.255.255.255 ?

3.0.0.0 is the network address, and cannot be assigned to a host. 3.255.255.255 is the broadcast address, and cannot be assigned to a host.

Both are valid addresses and can be assigned to hosts.

Network address is only really used for directly attached networks, non directly attached networks will route to any address in the block correctly.

Same for broadcast address, they're also relative to whatever block you're talking about at the time, so whilst 3.255.255.255 is the broadcast for 3.0.0.0/8 subnet it's just another "usable" address in the 3.0.0.0/5 subnet and when you send a packet then you, and probably your router, don't know what subnets in use on the other side :) (unless it's directly attached)

There's no reason why you can't assign 3.0.0.0 to a host, it'll work fine. Just make sure you don't put anything else there by convention.

3.255.255.255 would be the default broadcast address, but not only can you use a different address (in fact, any address you want for broadcast, you just need to configure it), but this is also not a real /8... it's 3.0.0.0/15 according to ARIN.

https://whois.arin.net/rest/net/NET-3-0-0-0-2/pft?s=3.0.0.0

3.255.255.255 is default broadcast for 3.128.0.0/9.

They delegated 3.0.0.0/15 to Singapore, out of the AWS 3.0.0.0/9 block, which is adjacent to their 3.128.0.0/9 block... which gives them all of 3.0.0.0/8...

Broadcast addresses have no meaning outside of a broadcast domain / subnet. Nobody would use a full /9 as a subnet. It would be split into much smaller blocks...

I think your statement answers pretty well an observation made earlier: Why is IPv6 adoption so bad? I see little incentive for anyone as long as IPv4 is such a profitable business with IP addresses being traded as an asset. IPv6 and equality to the masses? Would be nice but economy took adifferent turn

I don't think this analysis is correct. Imagine Chrome started charging small amounts of bitcoin for each HTTP request but Firefox still allowed free access. I don't think you'd say "I'm now incentivized to use Chrome because I have to have an artificial scarce resource."

ISPs can basically get all the IPv6 resources they need, but IPv4 addresses are becoming scarce and costly. Amazon just spent a lot of money to get more IPv4 addresses: that's cost, not profit.

If Amazon owned all the addresses and they were making great profits as a monopoly seller, this would indeed be an incentive not to move to IPv6. Instead, it's really just driving up people's costs.

Adoption is slow because the extra costs of IPv4 addresses are still smaller than the costs of really getting every piece of infrastructure and software working correctly with IPv6. We're not that far away, but there's a bit of a chicken-and-egg problem until we're close enough that people can start to turn off IPv4 and effectively force stragglers to adopt.

That's fully backwards. The people with the incentive to delay moving to IPV6 are the ones selling IPs. Those buying - ISPs - are also the ones that have the incentive to move to IPV6 as in an IPV6 world, they wouldn't have the spend that money. And ISPs are the ones best positioned for further IPV6.

That IPV6 adoption is slow is precisy because buying ranges of IPV4 addresses is still cheap enough that people are doing it.

ISP are really the bottleneck for IPv6 adoption. Pretty much all the infrastructures of the internet (network, datacentres, clouds) support it now.

When an equivalent solution becomes cheaper that's a recipe for disruption, not the other way around.