News for hackers. Stuff that matters.
Re:Poll on macOS 10.12 is broken by kazinator on Tuesday October 11, 2016 @10:36AM (#12684732)
On Mac OS, you must initialize Mac-OS-specific members of struct sockddr_in to zero, or it won't bind a local listening socket. You can't fill in just the address family, address and port.

E.g.

http://www.kylheku.com/cgit/txr/commit/?id=8bc6ba624272e9df4...

Re:Poll on macOS 10.12 is broken by raimue on Tuesday October 11, 2016 @11:12AM (#12685113)
It is generally a good idea not to pass unitialized struct members. Since the BSD socket API is meant to be extensible, new struct members could be added at any time. Enforcing this rule leads to more robust and portable code.
Re:Poll on macOS 10.12 is broken by kazinator on Tuesday October 11, 2016 @12:55PM (#12686155)
Look at what's actually in there:

  /* Apple's socket address. */

  struct sockaddr_in {
    __uint8_t      sin_len;
    sa_family_t    sin_family;
    in_port_t      sin_port;
    struct in_addr sin_addr;
    char           sin_zero[8];
  }
Why is sin_len that necessary? The socket functions have a length parameter, including bind:

  int bind(int sockfd, const struct sockaddr *addr,
           socklen_t addrlen);
If this redundant length is necessary, why is it okay to lie and set it to zero? The length of the address sure isn't zero.

If sin_zero must be zero, why doesn't the bind function make it so?

Don't add a member to the end of a struct just so that padding has a name, and then make me initialize it.

POSIX defines a type called struct sockaddr_storage which is at least as large as the largest sockaddr_* type. There is no need to pad the various sockaddr-s up to that size; the sockaddr_storage can be used to declare storage large enough to receive any address.

Re:Poll on macOS 10.12 is broken by raimue on Tuesday October 11, 2016 @01:40PM (#12686595)
Depending on your architecture, access to data of different size needs different alignment. You may never have heard of it, as it mostly does not exist on x86 except for special instruction set extensions such as SSE. However, by using structs of different size, you could violate these alignment requirements on ARM, SPARC, MIPS, etc.

For example, struct sockaddr only contains 1-byte members. Therefore it does not have special requirements on alignment. struct sockaddr_in contains at least one member of 4 bytes (struct in_addr), therefore it also has a alignment requirement of 4 bytes to avoid splitting the 4 byte member between memory words.

In order to support casting pointers to any of these struct types, implementations pad all of these structs to make them all the same size and therefore having sane alignment requirements.

You will not read anything about these kind of problems in POSIX, since these are limititations implied by the hardware architecture.

Re:Poll on macOS 10.12 is broken by kazinator on Tuesday October 11, 2016 @06:02PM (#12688882)
Yes, you will read about these kinds of problems in POSIX, where it defines sockaddr_storage:

http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys...

Quote: "The sockaddr_storage structure solves the problem of declaring storage for automatic variables which is both large enough and aligned enough for storing the socket address data structure of any family."

You're never going to define the storage for an object of `struct sockaddr`; that's just the "abstract base" so to speak: pointers are cast to that type when calling the API. The API switches to the right department in the code, which converts the pointer back to the right concrete type again, like "struct sockaddr_in". So all that is needed is the property that the "struct sockaddr " type can store and recover any other "struct sockaddr_whatever " pointer.

You're also never going to be doing completely silly things like converting a "struct sockaddr_in " to "struct sockaddr_ax25 " or whatever.

Re:Poll on macOS 10.12 is broken by kazinator on Tuesday October 11, 2016 @11:51AM (#12685521)
That is simply nonsense. The members are not documented in POSIX or anywhere else.

The API is extensible in that we can introduce new address types, and overlay new structures. You can implement a new network "foo", then in the API add AF_FOO address family enumeration and a struct sockaddr_foo which exhibits whatever members you want (which are documented and must be initialized).

The AF_INET family has only sin_family, sin_addr and sin_port. If we init these, we are good to go.

If you want a struct similar to struct sin_addr, but with more fields that are user-visible and must be initialized, you must introduce a new derived type AF_INET_SPECIAL along with struct sockaddr_in_special.

> Enforcing this rule leads to more robust and portable code.

Enforcing this rule ensures that correct code that works everywhere breaks on (i.e. does not cleanly port to) Mac OS.

Re:Poll on macOS 10.12 is broken by raimue on Tuesday October 11, 2016 @01:00PM (#12686210)
FreeBSD also has the same .sin_len member that macOS uses. If you still do not believe me that this is allowed, here is the relevant section in POSIX.1-2008:

http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/net...

> The <netinet/in.h> header shall define the sockaddr_in structure that includes at least the following members: [...]

The "at least" is the keyword. Additional members in the struct are allowed by POSIX. You have to expect and handle them according to the standard.

Re:Poll on macOS 10.12 is broken by kazinator on Tuesday October 11, 2016 @07:00PM (#12689217)
I don't dispute that certain structures can have additional members. A good example is "struct tm" from ISO C's <time.h>

Is there a blanket requirement somewhere in POSIX which says that whenever a structure may have members in addition to the required ones, these members must be initialized (and specifically to zero) when the application creates such a structure and passes it to a library function?

(If so, why are they repeating that requirement for struct sockaddr_in6?)

In any case, does the issue reproduce on FreeBSD? If I google for this bind problem, it only appears to be reported against Mac OS X, and no other Unix, historic or current.

Re:Poll on macOS 10.12 is broken by kazinator on Wednesday October 12, 2016 @12:34PM (#12694319)
Thinking I might be going "crazy", I raised this in comp.std.c. Long time regular J. K. whose opinions I can practically take as fact concurs with me.

https://groups.google.com/forum/#!topic/comp.std.c/zl15FkixB...

If a standard structure may have additional members, those do not have to be initialized by the program, unless the requirement is explicit somewhere. For example, we do not have to initialize members of "struct tm" other than the standard ones when calling asctime; there is no wording which absolves asctime of behaving as required if those members are not initialized.

The same reasoning applies to the POSIX bind function and the members of struct sockaddr_in. (But not struct sockaddr_in6, for which initialization requirements are specified).

Re:Poll on macOS 10.12 is broken by LukeShu on Tuesday October 11, 2016 @01:17PM (#12686371)
POSIX leaves it ambiguous whether you need to clear it or not.

About sockaddr_in, POSIX-2001, 2004 edition[1] has to say:

    10004     The <netinet/in.h> header shall define the sockaddr_in structure that includes at least the
    10005     following members:

    10006       sa_family_t     sin_family    AF_INET.
    10007       in_port_t       sin_port      Port number.
    10008       struct in_addr  sin_addr      IP address.
So implementations are free to add whatever extra members they feel like.

However, about IPv6 it says:

    10016     The <netinet/in.h> header shall define the sockaddr_in6 structure that includes at least the
    10017     following members:

    10018       sa_family_t      sin6_family     AF_INET6.
    10019       in_port_t        sin6_port       Port number.
    10020       uint32_t         sin6_flowinfo   IPv6 traffic class and flow information.
    10021       struct in6_addr  sin6_addr       IPv6 address.
    10022       uint32_t         sin6_scope_id   Set of interfaces for a scope.

    10023     The sin6_port and sin6_addr members shall be in network byte order.

    10024     The sockaddr_in6 structure shall be set to zero by an application prior to using it, since
    10025     implementations are free to have additional, implementation-defined fields in sockaddr_in6.
So raimue's point is explicitly true for sockaddr_in6; while it is left ambiguous whether the (allowed) implementation-defined members of sockaddr_in must be initialized.

[1]: The current UNIX standard is UNIX V7, which corresponds to SUSv4, which corresponds to the POSIX-2008. However, macOS is certified UNIX under the previous version, UNIX 03, which corresponds to SUSv3, which corresponds to POSIX-2001.

Re:Poll on macOS 10.12 is broken by kazinator on Tuesday October 11, 2016 @01:51PM (#12686743)
For IPv6 that makes a modicum of sense, because IPv6 has additional fields in the address not directly related to identifying the endpoint, and these have been historically in churn.
Re:Poll on macOS 10.12 is broken by kazinator on Wednesday October 12, 2016 @12:35PM (#12694333)
There is no ambiguity: there is no requirement in POSIX that the additional members of struct sockaddr_in be initialized by the application. The requirements are given for sockaddr_in6 only.