Cloudflare's work with 1.1.1.1 has demonstrated not only that it's _possible_ to use a previously-unusable IP address, but that it's _valuable_ to do so because it compels reporting of – and repair of – broken hardware and software on the market.

The cynic in me thinks that inbound traffic on 1.1.1.1 is "valuable" mostly to help balance Cloudflare's inbound vs outbound data flows. This balance is important in deciding whether their peering is settlement free.

They're a CDN and send out a lot of bits, so every last bit of inbound traffic helps. Or am I totally off base on this?

Yes, you are off base. We don't care at all about the traffic ratio with 1.1.1.1.

I would be very curious to find out what the actual bandwidth looks like.

I vaguely recall a site that that shows approximate throughput for given properties but I can't remember the terms to google for to (re)find it. But 1.1.1.1 might not be listed - or the numbers might only represent valid traffic and be wildly inaccurate.

Traffic breakdowns would thus be very interesting to see (for whatever is shareable, at least - heh, you'd probably have enough fireside stories to drown multiple HNs with :) ). In the case of 1.1.1.1 I'm wondering about what percentage of traffic is coming from valid DNS clients (and I also wonder where the requests are coming from - although that's just trivia to me), versus eg nonsense from horribly misconfigured lightbulbs.

I'm 95 percent sure that every dns response is bigger than the request, so it might not help much. Although if they're getting a lot of garbage in it might be helpful.

It depends on what methodology is used to classify traffic as inbound/outbound. The large DNS response in your example counts as outbound packets, but it's part of an inbound TCP connection. I don't know how peering agreements work.

Interesting.

I would have thought that they would go with the simplest measure possible, bytes in vs bytes out.

Pardon the ignorance, but I'm curious: why is it important for them (or anybody) to receive inbound traffic, given they offer 1.1.1.1 for free? And also what does it mean "This balance is important in deciding whether their peering is settlement free."?

I'm not a network engineer, so take this with as much credibility as you'd give an explanation after a few beers:

Most network providers have peering agreements to handle reciprocal traffic flows. In other words, if you're Comcast, you send a shit-ton of traffic to, say, Verizon. But Verizon also sends a shit-ton of traffic to you, as well. Generally, companies will have peering agreements that express the price for which they will route traffic for other network providers, and generally if there's a lot of reciprocal traffic, the peering will be "settlement free" - in other words, neither party charges the other to route traffic. So, in the example above, you as Comcast would agree to route Verizon's traffic across your network for free as long as Verizon did the same.

Cloudflare is a CDN, which means they push a LOT of traffic out across a lot of networks, and it's likely they're not getting as much inbound traffic as they're pushing out. That makes it harder for them to negotiate settlement-free peering, since they're not providing as much reciprocal value to their partners. By owning 1.1.1.1, they can now claim any trafic sent to that IP as "inbound" to Cloudflare's networks for the sake of peering agreements. Since 1.1.1.1 gets a bunch of traffic from either misconfigured equipment or people doing silly tests, routing that traffic helps improve Cloudflare's ability to negotiate better peering agreements.

Which, honestly, is pretty clever, since most of that traffic is garbage.

Balanced inbound/outbound traffic only matters for transit networks. That is, networks which are neither the source nor destination of the traffic they carry.

Nobody expects a residential ISP or a CDN to have balanced flows at part of a settlement-free peering agreement.

Cloudflare as a CDN with a lot of peering connections likely helps reduce internal traffic for other carriers by virtue of having endpoint data sources close to the destinations of the data.

I wouldn't bet my life on it, but I've never experienced a problem with an IP address ending in 0, and I've been using them for over 10 years. Our primary VPN end-point used to end in .0 - so we had lots of opportunity to find chance for breakage. :-)

I have, actually. Most of our networks were /23s, and the syslog server was a .0 in the middle of one. An SMS sending appliance we bought would not accept IPs ending in .0. Fortunately, the check was client-side in javascript, and once overridden, everything worked.

IIRC, IPs ending in .0 or .255 were unreachable from some older Windows systems.

That's been my experience too.

Even in 2003, the network I was involved in was handing out .0s out of larger CIDR blocks in DHCP leases with no issues.

Anything that cares about the zeroth IP of a destination not on the local subnet is broken. Anything that caters to broken is worrying about the wrong things.

Catering to the broken is like 90% of every developer and data scientist's day.

I've spent the past two weeks just trying to fix some horribly wrong/invalid data that is absolutely positively guaranteed to always be correct due to certain standards and lots of money and one of the biggest best tech companies in the world stands behind it.

Guess what it's still broken they just refuse to admit it.

Jon Postel (who wrote an early specification) of TCP would like to have a word with you:

"be conservative in what you do, be liberal in what you accept from others"

I like the spirit of this, but the problem is that it only works if everyone operates at least somewhat in line with it. If you're liberal in what you accept, then people are going to start being liberal in what they send, and soon we'll have a proper mess on our hands.

And there's no takebacks -- once you start permitting something, you can't take it away later, or you'll break things. You're creating more legacy requirements that may cause you problems when you try to evolve in the future.

As a web developer, I can only dream of how clean web standards might be today if the browsers of yore were a bit more conservative.

Do you have an example? I could possibly imagine this being a problem on a local /24, in that after you apply the netmask the local part is all zero bits. But for a remote DNS server, I cannot fathom how this could be a problem.

r-really? What do people do with routers that use 192.168.1.0?

To potentially clarify here:

Many home routers issue IPs using the follow netblock:

Network: 192.168.1.0 Netmask: 255.255.255.0 (/24) Broadcast: 192.168.1.255

Gateway: 192.168.1.1 (the first valid IP in that block) or less often .254 (the last valid IP in that block)

So for the past thirty years, nearly _every_ network block issued by _any_ network admin used /24, to the point that some things simply refuse .0 and .255 as valid IP addresses.

The suggestion upstream is to set up the IP 3.2.1.0, within _any_ network block that can contain it legally. If you adhere to strict netmasking, that'd be (this is from memory, correct me if I'm wrong):

Network: 3.2.0.0 Netmask: 255.255.254.0 (/23) Broadcast: 3.2.1.255

Which would provide a contiguous block of valid IPs from 3.2.0.1 through 3.2.1.254, _including_ the two perfectly valid IPs 3.2.0.255 and 3.2.1.0.

Those valid IPs end in .255 and .0, which is completely legal since they're in the middle of a /23, and the default gateway wouldn't be anywhere near them.

A lot of human-operated networks will blacklist issuing .0, .1, .254, .255 on the theory that it's simpler to just prohibit them and lose 4 IPs per /24 within a block, than deal with what's truly possible for a /23 or wider block.

EDIT:

Public BGP requires /24 or wider, and /24 cannot contain .0 or .255 as a valid IP, so the next widest is /23, which can contain even-numbered .x.255 and .x+1.0. If you're doing this on an internal network that permits narrower subnets, the narrowest available would be a /30, based (irregularly) on either .1.254 or .1.255:

Network: 3.2.0.254 (or .255) Netmask: 255.255.255.252 (/30) Broadcast: 3.2.1.1 (or .2)

With 3.2.1.0 as the live IP and either 3.2.0.255 or 3.2.1.1 as the in-network gateway for that device.

> Network: 3.2.0.0 Netmask: 255.255.254.0 (/23) Broadcast: 3.2.1.255

In classful addressing it is a network 3.0.0.0 with a netmask 255.0.0.0 (/8). In a classless, it is whatever has been advertised to me.

> Public BGP requires /24 or wider, and /24 cannot contain .0 or .255 as a valid IP, so the next widest is /23, which can contain even-numbered .x.255 and .x+1.0. If you're doing this on an internal network that permits narrower subnets, the narrowest available would be a /30, based (irregularly) on either .1.254 or .1.255:

It does not. You can advertise whatever the hell you want. What matters is what the other side would accept. It used to be that no one would accept anything less specific than /24 in a class C address space, /16 in a class B address space and /8 in a class A address space because the internet ran on AGS and maybe AGS+ which just did not have enough memory to do real classless. That stopped around the time GlobalCrossing showed up (or maybe around the time Mr V did terrible things while running AS7007). But anyway, ten years ago /27s are definitely accepted routes by those that comprised over 80% of the gloabal internet by the single origin prefixes.

But /27s, or anything from /24 through /30 inclusive, won’t include a .0 or .255 host. Only /23s and smaller or /31 and /32 can have hosts with .0 or .255

Are there any publicly advertised, unsummarised /31s or /32s in the global BGP tables?

You are correct:

Only /31s could include .0 and .255 and those would end up on router point to point links.

> Are there any publicly advertised, unsummarised /31s or /32s in the global BGP tables?

I filter them out as martians, so I cannot answer this.

> Public BGP requires /24 or wider, and /24 cannot contain .0 or .255 as a valid IP

That requirement is about how you advertise your routes not how you subnetted the internal network. .0 and .255 of your advertised /24 can work fine as /32s on the server. Using the internet FW to NAT the network and broadcast addresses is another common trick to get more addresses, particularly when you only get an e.g. /29 from a carrier who is just routing that /29 to your specified next hop and 2 more addresses is a big bump.

You can also use a fake IP for the default gateway on a wholly different segment with local segment ARP, or an IPv6 default router with 4-wrapping, or set up a router that uses promiscuous mode to capture all packets it receives from your “default route eth0” host. These are all irregular solutions, though, so I didn’t want to go too deep into the confusing weeds.

It's probably less of an issue when you're on a local network with the device you're trying to connect to. That said, I think 192.168.0.1 (ending in a 1) is much more common.

A /23 with the router in the middle of it? Surely that can't be common. If someone actually chose that (to be annoying or security through obscurity or whatever) then they know what they're doing

DHCP giving out a client address of 192.168.1.0 in the middle of a lagre block sure, but a router?

Routers being the “bottom of the block” in a subnet is just convention. In fact I’d say someone with a router in the “middle” of a block probably knows exactly what they are doing.

And thus would be able to change it if they actually had a problem with ip stacks failing on valid .0 or .255 ips