News for hackers. Stuff that matters.
Re:Stop Memsetting Structures by wbl on Saturday April 27, 2019 @04:07PM (#19767355)
Please explain how you send a struct over the network without UB and leaking the padding.
Re:Stop Memsetting Structures by amboar on Saturday April 27, 2019 @04:50PM (#19767550)
Use a packed struct (e.g. __attribute__((packed))). You may take a performance hit due to lack of alignment, but that's the judgement call
Re:Stop Memsetting Structures by saagarjha on Saturday April 27, 2019 @11:01PM (#19769050)
__attribute__((packed)) is not part of the standard, and therefore doesn't quite answer the question. I believe the standards-compliant way of doing this is that "you can't".
Re:Stop Memsetting Structures by klingonopera on Saturday April 27, 2019 @10:28PM (#19768911)
Would this work

  struct S {char data, char pad0, char pad1, char pad2};
  int main() {S dataStruct; printf("%s", dataStruct.data); return 0;}
and give me a 4-byte struct or would the compiler optimize it all away and leave me with a 1-byte struct?
Re:Stop Memsetting Structures by saagarjha on Saturday April 27, 2019 @11:03PM (#19769057)
Since you don't actually access the size of the structure, I see no reason why it matters. Also, FWIW, your code has undefined behavior because you call printf with the wrong type.
Re:Stop Memsetting Structures by klingonopera on Sunday April 28, 2019 @10:52AM (#19771385)
Yes on the UB, I'm passing a "char" into a "char *", sorry about that. I don't pass lone char data that often.

Well, I chose this example specifically to ask whether the resulting memory structure, completely independent of calling a sizeof(), would still be as if I were to call a sizeof() (which then I believe would be 4 bytes large) if not accessing the total size of the struct or any of the padding members.

It makes a difference in the memory footprint, which can become important, if you're programming a device with just 64-bytes total RAM.

Re:Stop Memsetting Structures by Gibbon1 on Saturday April 27, 2019 @06:57PM (#19768150)
> performance hit due to lack of alignment

This isn't true of modern processors.

Re:Stop Memsetting Structures by jzwinck on Saturday April 27, 2019 @08:00PM (#19768465)
Sure you get free unaligned access for scalars on x86, but unaligned arrays are still trouble if you use SSE (which basically everyone does).
Re:Stop Memsetting Structures by anyfoo on Saturday April 27, 2019 @04:13PM (#19767383)
By knowing its exact size with sizeof(), and using that and a cast to uint8_t* to both bzero() it out before use and memcpy()ing it when sending it. Unless the C standard says that the compiler may change the padding bytes of the existing storage?
Re:Stop Memsetting Structures by hedora on Saturday April 27, 2019 @04:36PM (#19767479)
To be clear, it is something like this, which is not bad at all to write:

  struct st s;
  bzero(&s, sizeof(s));
  // ...
  write(fd, &s, sizeof(s));
I’ve dealt with many systems that have tons of unnecessary serialization logic, and found them slower and buggier than just letting the C spec define the data format.

Having said that, it is possible to get this sort of thing right with C++ and templates (thanks to inlining).

Higher-level languages without compile time code generation (like Java and Python) usually make serialization a train wreck in comparison with “send raw structs” or “make the [C++] compiler generate and typecheck the machine code that does the serialization”

Re:Stop Memsetting Structures by kabdib on Saturday April 27, 2019 @04:45PM (#19767524)
There's enough wiggle room in the standard that two compilers would be happy making two compliant but still different physical layouts for the same struct declaration. I'm not even worrying about cross-platform issues or bitfields yet.

You can get away with directly writing a struct for simple structs, but eventually it will bite you.

Re:Stop Memsetting Structures by hedora on Saturday April 27, 2019 @06:11PM (#19767941)
Packed structs are reasonably standard (even gcc and visual studio can be made to pack in the same way using the same syntax these days).

The remaining issue is endianness, but you can configure ARM to match x86 at boot so shrug.

Also, if I tell any reasonably experienced developer my protocol sends a packed struct with these fields:

  uint8_t version
  uint16_t reserved
  uint8_t op
  int32_t status
  uint32_t data_length
  // void data[]
They’ll be able to write a correct deserializer in pretty much any language, and can probably write it in ~ 10 lines of code.

I agree that packed structs can be tricky, but they’re much better than the vast majority of serialization libraries I’ve dealt with (including all of the open source ones, and I’ve used all the usual suspects).

Re:Stop Memsetting Structures by blattimwind on Saturday April 27, 2019 @09:21PM (#19768707)
> The remaining issue is endianness, but you can configure ARM to match x86 at boot so shrug.

ARM in BE mode is pretty rare / nonexistent. Small MIPS chips still float around in mostly low-end networking gear, though, and those do tend to run in BE.

Re:Stop Memsetting Structures by Gibbon1 on Saturday April 27, 2019 @07:21PM (#19768252)
My experience with packed structs is there really aren't any gotcha's. Either works or not.

The big annoyance is network people all use big endian probably just to spite everyone else.

Re:Stop Memsetting Structures by shaklee3 on Saturday April 27, 2019 @07:40PM (#19768359)
Why would it be to spite everyone? Big endian is network order by definition. It's hard-coded into every asic in switches and routers for decades. If you don't use big endian your packets won't even make it to where you want. Of course, your own payload can do whatever you want as long as you know with 100% certainty nobody else will ever want to talk to your application and get confused.
Re:Stop Memsetting Structures by Gibbon1 on Saturday April 27, 2019 @09:13PM (#19768689)
Consider the lorawan spec which was written in the last 5 years. The stack is designed to be implemented in software on small 32 bit microcontrollers which are _all_ little endian. And it's big endians.

The people that wrote that spec knew that and went with big endian anyways.

Also I work with people that do network hardware. They can't care less about little or big endian. Makes no difference to them.

Re:Stop Memsetting Structures by saagarjha on Saturday April 27, 2019 @11:04PM (#19769063)
Someone needs to pick one or the other: you can't have it both ways.
Re:Stop Memsetting Structures by Gibbon1 on Sunday April 28, 2019 @01:20AM (#19769497)
Yeah they are picking the side everyone else abandoned 25 years ago.
Re:Stop Memsetting Structures by tom_mellior on Sunday April 28, 2019 @08:37AM (#19770806)
In the "..." part you presumably meant to include writes to members of s, and those writes are allowed to change padding bytes and leak information.
Re:Stop Memsetting Structures by _kst_ on Saturday April 27, 2019 @06:16PM (#19767966)
A small quibble: bzero() is not defined by the C standard. memset() is.
Re:Stop Memsetting Structures by unwind on Sunday April 28, 2019 @12:55AM (#19769424)
Obligatory pet peeve note: sizeof is a unary operator whose argument is written in parentheses if it is the name of a type. It is not a function.
Re:Stop Memsetting Structures by tom_mellior on Sunday April 28, 2019 @08:38AM (#19770814)
All answers to this must be wrong as it's simply not possible to do it. You need to take endianness of each field into account, and you cannot do that by treating the struct as a single blob.