Don't manually parse those things. If your database provides a way to parse those (like PostgreSQL does), make use of it. Use what your programming language provides, and if it really doesn't provide the functionality to parse IPv6 addresses, use a library to do so.

For example, in Rust you can write this:

    use std::error::Error;
    use std::net::{AddrParseError, SocketAddr};
    
    fn parse_address(addr: &str) -> Result<SocketAddr, AddrParseError> {
        addr.parse()
            .or_else(|_| Ok(SocketAddr::new(addr.parse()?, 10443)))
    }
    
    fn main() -> Result<(), Box<dyn Error>> {
        let addresses = [
            "[2001:0db8:f00f::0553:1211:0088]:10444",
            "2001:0db8:f00f::0553:1211:0088",
        ];
        for address in &addresses {
            println!("{:?}", parse_address(address)?);
        }
        Ok(())
    }

This isn't specific to IPv6 by the way. It also applies to other standards like CSV (although I suppose with CSV it does vary, I saw so many broken CSV files that sometimes a custom implementation is the best way to go to parse those).

The most stupid thing about CSV is it's entirely pointless, ASCII has codepoints for file, group, record and unit separators.

Which you can't type on a keyboard, sensibly print on a terminal, or express with a symbolic escape (like '\n') in any language i know.

If history had gone differently, the separators would be useful for separating things in files, but it didn't.

I am sure I am not alone at chuckling at the idea that one cannot type FS, GS, RS, and US on a keyboard, clearly borne of GUI conventions and thinking. Some of us have been typing control characters on a keyboard for quite a number of years. This is what the [Control] key is for. (-:

FS enjoys quite significant use nowadays on terminal keyboards in the worlds of Unix and Linux, it being the usual special character mapped to SIGQUIT. GS and RS are similarly fairly mainstream commands in VIM/NeoVIM.

> I am sure I am not alone at chuckling at the idea [...]

You might want to hold back on your condescending laughter for a bit. Neither vim nor nano allow me to insert a control character by pressing Ctrl+key, so how am I supposed enter the FS character when editing a text file? Besides, there are other operating systems besides Unix and Linux (cough Windows cough).

Actually, VIM does support this. Type Ctrl+V in insert mode followed by whatever you want to insert (e.g. Ctrl+\ for the ASCII FS character). VIM will insert the literal value of the control code into the buffer.

Thanks! I had tried Ctrl+V followed by the character from the caret notation (e.g., "Ctrl+V \", instead of "Ctrl+V Ctrl+\"), and that hadn't worked.

The condescension here is supplied by you. Alas, it is based upon inaccuracy.

VIM/NeoVIM cannot stop you from entering control characters by pressing [Control] and a key. It is a function of one's terminal or terminal emulator that (aside from employing some fairly unconventional keyboard maps) does not get turned off.

You are confusing entering text with typing control characters on a keyboard. One types these control characters with VIM/NeoVIM all of the time. One of them, after all, is ESC, and the others are fairly common as I said. That the control characters are commands, even in insert mode, is a different matter. One such command is the SYN control character, typed on the keyboard of course with the [Control] key as [Control]-V, whose use demonstrates that in VIM/NeoVIM one most definitely can insert control characters as text.

The fact that you brought up nano is also somewhat risible. That has a large banner across the bottom of the screen telling you what control characters the editor responds to, which one uses the [Control] key to type on the keyboard. It's staring you right in the face. (FS is even one of the ones on the main menu.)

I'll let you discover the equally simple key chord that you can use with nano, which likewise does let you insert control characters as text with the [Control] key.

As for Windows, you might want to fire up a console and actually try pressing some [Control]-key chords. For best results, run a console program that follows much the same convention. In TCC, for example, Alt-255 is the literal-next-character convention. But even without such a program, SUB as an end of input indicator, typed on the keyboard as [Control]-Z, is a fairly easy demonstration; and an application-level convention that goes back a long way.

* https://jpsoft.com/help/editing.htm

> The condescension here is supplied by you.

Please edit that sort of thing out of your comments here. The GP shouldn't have included that provocation, but there are a lot of provocations on the internet, and for good discussion here we all need to build a certain immunity. A perfectly good thread about control characters needn't be marred by ill will!

https://news.ycombinator.com/newsguidelines.html

It's not even a particularly good thread about control characters :(.

> Which you can't type on a keyboard

Your keyboard doesn’t have the following keys?

ctl-\, ctl-], ctl-^, and ctl-_

Re: “symbolic” escapes, sure, but there’s always:

\x1c, \x1d, \x1e, and \x1f

Not everything can be a single character, there’s only 26 of those in the English language.

My system seems to terminate the program I'm typing into when I press Control-\. Also, with the default German kez^Hymap, I can't seem to type Control-^.

Try Ctrl+V first, if it's not been bound to paste. No idea how a german layout affects things though.

You can type them! E.g., ctrl-^ is record separator.

  $ echo ^\^]^^^_|od -c
  0000000 034 035 036 037  \n
  0000005

So, ^\ is FS, ^] is GS, ^^ is RS, and ^_ is US.

It's probably worth remembering that these combinations are unintuitive and not readily printed on keyboards. Additionally, all text editors can render a CSV correctly; I doubt the same can be said for Unix control characters.

I suspect fewer than 1% of computer users has these memorized. For that matter, I suspect fewer than 1% of software developers or sysadmins have these memorized.

In contrast, the comma, tab, and return keys are easily located on any keyboard, and CSV delimiters are all so common as to be readily printable by any text editor.

Meanwhile, unix escape sequences don't solve the quoting / escape problem of representing strings containing the delimiters anyway.

Good luck remembering what characters those are and viewing them in most terminals without a hex dump.

Remember that CSV is also for human consumption.

Come up with a convention and stick to it. Bracketed network/host, trailing port. Shoot anyone who does it any other way. Ignore case in network/host...and yes you need to be able to do this. Counting on some other system to do the right thing is silly.

> Shoot anyone who does it any other way.

> Counting on some other system to do the right thing is silly.

Do you see the contradiction in these statements?

Why yes, thank you. But without being morbidly explicit joining these phrases without context could put someone in jail...and that is what you did there. My point was to pick a convention and enforce it and (secondly) that using some agency ( I believe postgresql, various APIs, were mentioned throughout) is silly imho. People are going to use strings and parse them. Just make it the same format everywhere.

Well, I mean, yes, those are all mistakes people are going to make, no doubt about that. But somehow the solution is missing?

The core mistake here is using string manipulation at all. The only correct way to handle string data is to parse it, and then either operate on the parsed data structure, or re-serialize into a canonical format and use that for comparisons and stuff.

And in the best case, you don't try to build your own parser, but use a well-tested one that's already there. For the particular use case of parsing IP addresses, it's probably best to use getaddrinfo() with AI_NUMERICHOST, and for re-serialization getnameinfo() with the same flag. If those don't understand the address, you most likely won't be able to connect anyway. And they will handle stuff like link-local addresses correctly, at least as long as you are on the host that actually has the respective interface.

For databases, you use column types intended for storing IP addresses, so the database will do the parsing and canonicalization for you.

And when you actually have to build a parser yourself, read the damn spec for the format instead of going by what you think the format is, because most likely it's not that.

And mind you that most of those problems are not really IPv6-specific. There are also many ways to write an IPv4 address that your average parser will understand. Most of those are generally frowned upon, so they don't occur often, but if you want to reliably compare IPv4 addresses, you actually need to do the same as for IPv6.

One reason people keep using string manipulation is that in C getaddrinfo gives a addrinfo, which is quite a complex struct. It contains a struct sockaddr, which is actually a sockaddr_in, which contains both an address and a port. So now you have to carefully keep track of which of your addrinfos and sockaddrs have the port in them and which are just the address.

It won't solve most of the problems, but if you're using PostgreSQL its native inet datatype (which supports 4 or 6) instead of a text or binary string can save a world of pain and there are alternatives in some other RDBMSs:

https://www.postgresql.org/docs/current/datatype-net-types.h... https://www.postgresql.org/docs/current/functions-net.html

First class network and ip types that properly support contains and exclusion operators make a lot of things less error-prone and potentially much faster.

Yeah, most of these problems can be solved by declaring a new type of data, defining how a string address is parsed into that data type, defining rules for comparison, and using that type whenever a host is input into the system. These are problems that happen when you don't use the type system of a language or the language doesn't allow you to enforce specific types as input.

I doubt we're ever going to get standardization on a given ip data type even within most languages, much less across them, but there's a tiny bit of hope that the various databases might converge on something vaguely portable. It's taken twenty years, but I'm actually seeing it with some other types. We just need to make sure MySQL doesn't release something where internally all ip addresses (4 & 6) are stored as 47 EBCDIC characters and silently corrupts your dumps if you're using a really-no-I-mean-it UTF-8 locale. (Not entirely fair to the MySQL devs, but a lot of database people have been burned by their approach and Oracle is not making this better.)

Any language with networking in the standard library should have a data type for IP addresses. There's no good reason for API bindings for database libs to not use them, and no good reason for you to not use them even if you're not using a database.

Agreed in theory, but I've seen plenty of hand-rolled solutions even so and I think the dbs are at least slightly more likely to get standardization via ORM devs. One of the most painful I've run across was a system that was allocating ips for short, but highly variable lifespan VMs and trying to store them in MySQL database manipulated with both Ruby and Python. Even with language support there was no real defense against coworkers trying to brute-force complicated network checks on assignments when sub-second responses were needed. Unless your work is only ever in a small corner of a large system a single language and standard library having a native type won't always protect you from your code getting handed ip addresses from a foreign source in various text formats. The original post explains why this is a potential nightmare in detail.

Don't try to parse things without a standard. Even CSV and e-mail addresses are more complicated than they seem.

Also, it's pretty silly that we still use these unintuitive conventions from 40 years ago for modern systems. Is 192.168.2.8:10443 an address? A phone number and extension? Is it TCP, UDP? IPv4, IPv6? An HTTPS service, or just something resembling its decimal notation assigned service number? Are there multiple services proxied behind this one address? Can I route between them? When I request a URI, does the application know what I really want/expect? What about a timeout for my request? What about authentication/authorization? Consistency requirements? Idempotence? Security guarantees?

Operating systems don't even take <host>:<port> arguments for network syscalls, that's just a convention we sort of came up with and later stuck to. But as a URL it's pretty crap. I suggest we replace them with modern URLs that can embed tiered information such as session IDs, service types, routes, security requirements, operational parameters, etc. Most people may only need https://google.com/, but sometimes we may also want to request webv2+uquic+v6:/SC,TLSv1.3/userid[s:84742049]@google.com(r:NA)/ . I know that's ugly as sin, but hopefully people wouldn't need to specify all of that all of the time (service name/version, transport, address, strong consistency, TLS 1.3, userid, session id, host/namespace, North American region).

A workable solution would have been to promote the "[address]:port" representation for both IPv4 and IPv6 after IPv6 was introduced. Then we'd have a universal way to represent UDP and TCP endpoints.

It's kind of mental that my browser will accept https://1.1 today but not https://[1.0.0.1]

This is the problem that multiaddr tries to solve. Unfortunately, it's still very much underspecified and incomplete (eg. it does not support domain names).

https://github.com/multiformats/multiaddr

> it does not support domain names

I'm confused, what do you mean? /dns4 and /dns6 appear to be ways to use domain names.

Oh indeed, it got updated since I last checked.

Why bother with text when ASN.1 solves all these problems too?

But then you have to solve ASN.1's problems.

Some of such, circa 2000: https://tools.ietf.org/html/draft-yu-asn1-pitfalls-00

Holy mother of cows! The specs are available! https://www.itu.int/en/ITU-T/asn1/Pages/introduction.aspx

Did you forget the /s tag on the end of that post?

This type of feature-itis -webv2+uquic+v6:/SC,TLSv1.3/userid[s:84742049]@google.com(r:NA)/- is exactly the reason google is going to hell.

It’s certainly been said but IPv6 to my eyes is awash in second system syndrome, which has largely slowed its adoption.

The complexity of handling addresses is plainly a failure of the design. Using colons for separators when they’re already being used for ports served no purpose but to confuse. Having more than a single valid form of an address again only serves to confuse.

If there’s anything to learn from the UNIX principals it’s there is great power in making things easily manipulated as strings. The design of IPv6 makes this impossible.

None of the standard calls use colons for separating ports - I can’t recall exactly whether it’s even a standard URI thing or just something web browsers did.

Further, even UNIX paths containing ‘..’ and other things need normalisation, and with symlinks you can’t necessarily just compare two paths to determine whether they point to the same file.

The problem, as plenty of other people have said in these comments, is that you just need to parse the address and compare the structure. There are standard library calls for this, and most databases have types that can do this.

> None of the standard calls use colons for separating ports - I can’t recall exactly whether it’s even a standard URI thing or just something web browsers did.

https://tools.ietf.org/html/rfc3986#section-3.2.3

Specifically part of the URI RFC.

Probably should have picked something wildly unrelated as a separator instead of colons - but the horse has already left the barn, so now we just get to play cleanup.

You're right that there's already a decent amount of support for this in various languages and persistence mechanisms, but I've spent enough time in my career playing with truly bleeding edge barely-an-alpha platforms to know that sometimes you do just get stuck writing your own parser and comparators. It's good to know in advance what sort of potholes you will be trying to navigate :)

Edit: I just discovered the URI RFC is dated 2005, with IPV6 being dated 1998. ( https://www.ietf.org/rfc/rfc2460.txt )

Still, it struck me as odd that we had been using host:port for so long. The URL conceptual predecessor to URI RFC also mentions using host:port in section 3.1.1 - which came out in 1994.

Regardless of which RFC made a hash of things though, all 3 are widely used specifications, so we're stuck in this boat now!

There are many programs beyond browsers that use colon for ports.

When ipv6 was devised, : was already widely used to define ports by all sorts of applications.

Let’s be honest, the creation of ipv6 was badly handled.

> Having more than a single valid form of an address again only serves to confuse.

IPv4 address also have multiple valid forms. Besides the most common form, a.b.c.d where a, b, c, and d are decimal numbers in [0, 255] representing four 8-bit parts of a 32-bit address, there is a.b.c form, where a and b are decimal numbers in [0-255] and c is a decimal number in [0-65535], representing two 8-bit parts and one 16-bit part of a 32-bit address. Then there is a.b form where a is a decimal number in [0-255] and b is a decimal number in in [0-16777216], representing an 8-bit part and a 24-bit part of a 32-bit address. There is also a form, where a is a 32-bit decimal number, representing a 32-bit address.

Any of the aforementioned decimal numbers may instead be given in octal by prefixing with 0, or hex by prefixing with 0x or 0X.

I agree that some of IPv6’s features have caused it to be adopted really, really slowly, but I also think that those features are still better than anything else we could have come up with.

> Using colons for separators when they’re already being used for ports served no purpose but to confuse.

Dots and colons are the only separators I can think of that are available on common keyboards, and don’t look ugly. We can’t re-use dots because then there’d be addresses that are _both_ IPv4 and IPv6. And I’m serious about the ugly part! Try writing an address out using commas or semicolons or asterisks or something, it just looks weird.

> Having more than a single valid form of an address again only serves to confuse.

Yes, as the article demonstrates, it certainly introduces edge cases that aren’t present in IPv4 — but the alternative is having to type out all the zeroes. Would you want to type out ’0000:0000:0000:0000:0000:0000:0000:0001’ each time?

> If there’s anything to learn from the UNIX principals it’s there is great power in making things easily manipulated as strings.

I think this is a testament to how wrong that statement is. How many scripts have broken because a file path has a space character in it? How many content injection attacks have there been, caused by code that blindly concatenates strings together? IPv6 addresses are just another example.

It’s a shame, but IPv6 is the best we’re going to get.

> We can’t re-use dots because then there’d be addresses that are _both_ IPv4 and IPv6.

How so? 192:168:1:1 is not a valid IPv6 address, so it'd be easy to differentiate with dots

If you were to interpret it as ::192:168:1:1 then it'd be a valid IPv6 address, but that interpretation would be incorrect and even so would be in the "IPv4-Compatible IPv6 address" range that was intended to map directly to the IPv4 address as a transitional function anyways.

Not really sure why that idea got dropped, it seems to me like it makes a lot of sense to do any v4 mapping in the ::0/96 range.

> 192:168:1:1 is not a valid IPv6 address

Ack, you’re right. For some reason, I was thinking it would get parsed as “192:168::1:1”.

That mapping does make sense.

192:168:1::1 is understandable if you were thinking of "1- or 2-dot" IPv4 addresses (i.e., 192.168.1 => 192.168.0.1, 192.1 => 192.0.0.1), but 192:168::1:1 doesn't really make sense.

> Having more than a single valid form of an address again only serves to confuse.

That's not fair. Features like this obviously have a purpose. (It makes a whole lot of IPv6 addresses a whole lot shorter to type!). You may not like it or use it or think that the complexity tradeoff was worth it, and that's a reasonable position to take, but it's not fair to say that the other perspective serves no purpose.

Re: colons for separators

I think we should just define a .ip6 pseudo-tld and then you can write e.g. ff::12:34 as ff.x.12.34.ip6 (with .x. being the equivalent of ::). This should be enough for any practical purpose of users specifying free-form addresses, where the address can be ipv4, ipv6 or a host name. For configuration files and the like where you're only writing IPs, the colon form would be acceptable as well. The good part about this is that it also won't require us updating any address validation regexes (of which there must be many, despite all the advice against it).

Not just ipv6.

  $ ping 127.1
  PING 127.1 (127.0.0.1): 56 data bytes
  ...
  $ ping $(((127 << 24) + 1))
  PING 2130706433 (127.0.0.1): 56 data bytes
  ...

I've primarily seen the second form in the early days of the web when spammers were presumably trying to bypass mail or web filters that were scanning for blacklisted IPs.

Edit: fix formatting

Hex works too (tested on OS X):

    ping 0x7f000001
    PING 0x7f000001 (127.0.0.1): 56 data bytes

or, as a URL (tested in OS X Chrome, and the URL formatter of HN also shows it, with Chrome hover showing 127.0.0.1): https://0x7f000001/

The problem was (and remained) programmers thinking addresses are strings.

Right. I was waiting for the author to get to this point, and much to my surprise they never did.

The takeaway shouldn’t be “strings are hard”. Strings are hard, but that’s not the problem here. The problem is using inappropriate data types for the task at hand, and the takeaway should be that representation matters.

Representing an IP endpoint as a string only really makes sense at the human-computer interface. In general, the first thing a program should do with such a string is convert it into a more suitable data type. And as noted elsewhere, more often than not there will already be a library function to do so.

Strings are especially pernicious because they are ubiquitous, in that just about anything can be represented as a string, but the operations one can do on a string representation of an object do not generally correspond with the operations one would want to do on the object itself. This disparity is the source of many bugs, which the article exemplifies (though falls short of directly addressing).

Similarly problematic are assumptions that converting an object to a string and a string to an object (i.e. `parse` and `format` functions) are bijective. This is not generally the case. (Wise practitioners might choose such a mapping, though, when the opportunity presents.)

I actually prefer them as integers personally. 4 bytes for IPv4, 16 bytes for IPv6, is more space efficient than any string representation usually.

16 bytes isn't always enough to store an IPv6 address. Depending on the requirements, you may need to store a fully scoped IPv6 address, such as fe80::1%eth0.

Edit to note: I store the fully scoped IPv6 address in a database as part of my CMDB mechanism.

Just curious, but what use do you have for storing link-local addresses?

This. The proper approach is to normalize/deduplicate on ip objects that understand ipv6 domain logic. Then most issues just go away.

They are strings, sometimes, and that is what this article is about. Another non-obvious fix to this issue is to start using DNS. (Well obvious to some)

I'm really stumped as to why they didn't eliminate ports when designing IPv6. Having the last bits in the address take the role of ports makes sense given the hierarchical nature of IPv6 addresses. The first bits specify the network (my home), next bits the device, and last bits the service/endpoint on that device.

It would also make parsing much easier given that they chose : as separator in the addresses.

Ports operate a different layer in the OSI model - the transport layer, not the network layer (which IP operates on). Ports are TCP and UDP concepts (e.g., a different application can bind to TCP port 443 than UDP port 443, as they're essentially different ports). Some protocols don't have ports at all, e.g., ICMP, which instead has a "type".

Sure, but so what?

Imagine you could only bind services to their standard ports. What's the difference between a service running on my computer having a unique ip address, or my machine having a unique ip address and only running that service?

Compatibility. Code written for layer 4 should be compatible even if layer 3 is changed. In practice much code had to be changed anyways, but the amount of violence applied to existing code was limited by not getting rid of ports. It's not just that either, but much else, including cognitive load.

OK sure but TCP/IP was not designed to follow the OSI model.

Another gotcha is that ipv4 and ipv6 has different firewall tables. So if you for example have blocked all traffic besides port 80, you also need to do it on ipv6 !

That does depend on the firewall you use. Using the "pf" firewall you can have nice clean rules which covers both.

Simple example:

    tcp_inbound = "{ ssh, domain, http, https }"
    tcp_outbound = "{ domain, http, https, imaps, smtps }"

    udp_services = "{ domain, ntp }"

    block all
    pass proto udp to any port $udp_services keep state
    pass out proto tcp to any port $tcp_outbound keep state
    pass in proto tcp to any port $tcp_inbound keep state

pfft, who is still using iptables? :)

FYI, nftables has the 'inet' table which processes both IPv4 and IPv6 packets, and 'ip' and 'ip6' which processes each respectively.

iptables' are easy to configure and available almost everywhere.

easy to configure?? iptables is the most infamously difficult firewall to configure.

Ubuntu created the ufw "uncomplicated firewall" wrapper for a reason :D

I think it's similar to CSS... I however have a sense that IPv6 is much different so we might get better and easier tools for future IP versions once IPv4 is gone, I however don't expect that to happen during my lifetime :/

And your an authority on this? I find PF much more difficult to deal with and I like having two clean interfaces to v4 and v6 in iptables....rather than some overeager tool set that thinks it knows what I mean when I create a rule and the tool then (willy-nilly) applies it across both stacks.

I’m curious to know what you find so difficult about iptables?

It has always seemed reasonably straightforward to me, although I will concede that not being able to combine tcp & udp into the same rule is a bit annoying (though also rarely needed).

That should be obvious as sunlight though, if you know anything at all about IPv6 networking.

For someone who is just doing port blocking it's less obvious. The port numbers are the same on v4 and v6.

All of this flailing around would have been avoided by either thinking carefully about canonical representations, or reading https://tools.ietf.org/html/rfc5952

fwiw, rfc-5952, specifically, sec:6 outlines recommended string representations. also, it is generally much nicer to use 'sock_storage' as an underlying representation of AF_{UNIX/INET,INET6}, endpoints rather than building abstractions to tide over differences of 'socaddr_*'

Maybe this is a stupid question at this point but if using colons introduces this degree of ambiguity why not stick with simple periods? Or any other separator for that matter.

Even putting aside the ambiguity of using ':' as a address:port delimiter, there are differences between platforms in parsing just IPv6 addresses

e.g. on Linux/glibc 2.28

    [cling]$ #include <arpa/inet.h>
    [cling]$ ::in6_addr dst;
    [cling]$ dst
    (::in6_addr &) @0x7ff318aff010
    [cling]$ inet_pton (AF_INET6, "1234:1234:1234:1234:1234::1234:8.8.8.8", &dst)
    (int) 0
    [cling]$ inet_pton (AF_INET6, "1234:1234:1234:1234:1234:1234:8.8.8.8", &dst)
    (int) 1

Here '::' in this 'full' address is (correctly) rejected.

However, although I haven't verified this on FreeBSD (perhaps someone can?), there's a comment in the libc source suggesting that this will be accepted there

https://github.com/freebsd/freebsd/blob/1d6e4247415d264485ee...

Parsing sucks.

Obviously the point of the article is to not manually parse ip addresses.

Well actually, python can do that just as well as all the other languages.

    >>> import ipaddress
    >>> ipaddress.ip_address('2001:db8::')
    IPv6Address('2001:db8::')

Actually, in Java you can just use InetAddress.getByName.

man 3 getaddrinfo

Why don't you rewrite that in rust?

    use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
    let localhost_v6 = IpAddr::V6(Ipv6Addr::new(0, 0, 0, 0, 0, 0, 0, 1));
    assert_eq!("::1".parse(), Ok(localhost_v6));

The fundamental problem in the example:

    leader_host = bigdata.example.org:10443

is ":10443" is not part of the hostname. The field is called "leader_host"; if a port is needed, it should use it's own field instead of trying to overload the host field.

    leader_host = bigdata.example.org
    leader_port = 10443

(and as as others have already mentioned, don't write your own parser when they already exist in your stdlib/etc)

IPv6 was over-engineered. We need IPv5 which just fixes the address space, then have a rethink about v7 ... without all the nonsense.

Why not split the parameters?

Address and port as separate values makes more sense to me.

Indeed, several mechanisms for passing this stuff to programs do exactly that. Notice that they are two distinct command-line arguments to these programs, for examples:

* http://jdebp.eu./Softwares/nosh/guide/tcp-socket-listen.html

* https://cr.yp.to/ucspi-tcp/tcpserver.html

One of the principles underlying this design was, famously, "Don't parse". Don't have one string with a separator character. Have two strings that are already separate.

* http://cr.yp.to/qmail/guarantee.html

Mostly because of convention that the single identifier be a single string from the user. You might be able to get away with it e.g. in your home grown app but try to land that change in Chrome and you'll get pushback for not being standards compliant.

The explicit way to define a v6 URL was first defined here https://www.ietf.org/rfc/rfc2732.txt and is to use []'s around the address portion.

Canonicalize everything, then compare the canonicalized results. We've got this ;)

':' is also a perfectly valid character in a domain name, so the naive splitting already does not work in the IPv4 ecosystem.

No, a colon is not legal in a domain name or host name. See RFC 1123. Only alphanumerics and the hyphen character are allowed in DNS labels, which are separated by full-stop characters. Non-hostnames in DNS may also allow the underscore character as part of a DNS label.

RFC 1123 specifies the preferred format, not what is mandatory. This is clarified in RFC 2181:

"The DNS itself places only one restriction on the particular labels that can be used to identify resource records. That one restriction relates to the length of the label and the full name. The length of any one label is limited to between 1 and 63 octets. A full domain name is limited to 255 octets (including the separators). The zero length full name is defined as representing the root of the DNS tree, and is typically written and displayed as ".". Those restrictions aside, any binary string whatever can be used as the label of any resource record. Similarly, any binary string can serve as the value of any record that includes a domain name as some or all of its value (SOA, NS, MX, PTR, CNAME, and any others that may be added). Implementations of the DNS protocols must not place any restrictions on the labels that can be used."

You are correct that it can not be used in a hostname and I am not sure how it should be encoded when used in a URI, but for a domain name it is perfectly valid.

Indeed, I tested this on my local DNS server running Knot and I can create a domain name with colons in it and it also resolves using dig.

I can’t register “foo:bar.com” on Hover, GoDaddy, or anywhere else.

There’s a difference between how DNS software is supposed to treat label data as opaque octet strings for forward-compatibility reasons, versus the standards for valid host and domain names enforced by the rest of the Internet standards.

I've taken to using python split(':')[:-1] and like constructs

See also rsplit(':', 1). But instead of either of those, you should really be using library functions that already understand the format in question instead of making up your own rules.

  >>> u = urllib.parse.ParseResult('', '[2001:0db8:f00f::0553:1211:0088]:10443', '', '', '', '')
  >>> u.hostname
  '2001:0db8:f00f::0553:1211:0088'
  >>> ipaddress.ip_address(u.hostname)
  IPv6Address('2001:db8:f00f::553:1211:88')
  >>> u.port
  10443

Urllib? There has to be a better choice than that. I've been using inet_ntop and pton to canonically convert bare IPs in to sane forms.

What's the problem with urllib for this purpose?

It'll fail you with port vs. no port.

1234::5678:9876 is ambiguous. Is 9876 a port or part of the address?

The common explicit format is [1234::5678]:9876/

If you don't know if it has a port or not in advance, how can you know the last :thing is not part of the IPv6 address deterministically? If you luck into a four digit port, and the address has :: maximal compression you're SOL.

I have exclusively done this :-1 knowing I had a port.

Not all sources (tcpdump) use your canonical form.

You should read TFA.

I did. It basically said to change your input stream to avoid the problem. Not all data sources can be changed. If you don't have canonically parsable inputs, then assumptions don't work and you can't predict if an add::d:ress is with or without the :port syntax, a priori unless you know the input stream.

So between your input and your parsing library you should put a bit of code to clean up the incoming data. Don’t create a library that processes incorrect data because at some point someone will absolutely try to use it to parse different input data and it will cause problems.

Don't do ad-hoc string parsing. This case is going to explode on ipv4-mapped addresses with dot notation for the last 32bits.

https://tools.ietf.org/html/rfc4038#section-4.2

Yes. It's a weak-as expedient. I've managed to avoid mapped in the last few years.

IPv6 is a clusterfuck of "hey lest add x!" and "lets forget reasonable previous experience!".

Call me a luddite but I'm not looking forward to IPv6.

How about IPv5; add another byte.

We can keep the colons 192:168:1:1:1.