News for hackers. Stuff that matters.
Re:A bug story: data alignment in C on x86 (2016) by valarauca1 on Tuesday September 04, 2018 @03:48PM (#17912215)
code sample + name (and already dealing with SIMD alignment issues) told me what the problem is.

Pointer casts are ALWAYS bad in C. Try to avoid them. Reply and down vote you old hat C programmers who will defend this. Your old practices are actively harmful to people learning the language.

Re:A bug story: data alignment in C on x86 (2016) by userbinator on Tuesday September 04, 2018 @10:25PM (#17914618)
Your old practices are actively harmful to people learning the language.

If anything it's the insanity of standards-lawyering antagonistic compiler-writers which is "actively harmful".

No one wants things like integer overflow checks being silently removed, yet there's an apparent surplus of people whose only reaction is to worship The Holy Standard and laugh at you for trying to use the language in the straightforward portable-assembler way that it was originally designed for.

Re:A bug story: data alignment in C on x86 (2016) by saagarjha on Wednesday September 05, 2018 @02:29AM (#17915501)
> trying to use the language in the straightforward portable-assembler way that it was originally designed for

Just because it was designed that way doesn't mean that's the way it works now. There are reasons why pointer casts are left undefined (as opposed to many other sources of undefined behavior, to which I heartily agree should have been left implementation-defined).

Re:A bug story: data alignment in C on x86 (2016) by saagarjha on Tuesday September 04, 2018 @08:39PM (#17914159)
> Pointer casts are ALWAYS bad in C.

Nope. There's a reason that void * exists. How would you write a "generic" function like qsort without a pointer cast?

Re:A bug story: data alignment in C on x86 (2016) by caf on Tuesday September 04, 2018 @10:02PM (#17914547)
You don't actually need a cast to write a qsort comparison function, because void * is implicitly converted to and from other pointer types in C. Eg:

  int compare_xyzzy(const void *a, const void *b)
  {
      const struct xyzzy *sa = a;
      const struct xyzzy *sb = b;

      return (sa->i > sb->i) - (sa->i < sb->i);
  }
On the other hand, pointer casts are needed if you want to write a function like strchr() (where you can pass a const-qualified pointer, but want to return an unqualified pointer for the case where an unqualified pointer was passed).
Re:A bug story: data alignment in C on x86 (2016) by saagarjha on Wednesday September 05, 2018 @02:25AM (#17915485)
D'oh! You're right, of course.
Re:A bug story: data alignment in C on x86 (2016) by v_lisivka on Wednesday September 05, 2018 @06:25PM (#17922398)
If you want to write "generic" function in C, use #define to generate static function for your specific type. But it much much better to use C++ or Rust instead.

Example: https://svnweb.freebsd.org/base/head/sys/sys/tree.h?revision...

Re:A bug story: data alignment in C on x86 (2016) by rightbyte on Tuesday September 04, 2018 @05:42PM (#17913064)
How do you prefer to use e.g. malloc?
Re:A bug story: data alignment in C on x86 (2016) by unwind on Wednesday September 05, 2018 @06:09AM (#17916543)
Please see https://stackoverflow.com/questions/605845/do-i-cast-the-res... for some thoughts on this.

In short (as already stated, of course), you don't need to cast void * in C to/from other object pointers. Adding a cast to the return value of malloc() is a bad idea (I'm biased).

Re:A bug story: data alignment in C on x86 (2016) by icedchai on Tuesday September 04, 2018 @05:50PM (#17913120)
You don't need to cast there... malloc returns a void*.
Re:A bug story: data alignment in C on x86 (2016) by AnimalMuppet on Tuesday September 04, 2018 @08:47PM (#17914205)
Which you are going to use how? Don't you have to cast it to something to be able to use it?
Re:A bug story: data alignment in C on x86 (2016) by deathanatos on Tuesday September 04, 2018 @09:14PM (#17914334)
No, you don't. The following,

    int *i = malloc(sizeof(int));
is valid C, and will compile just fine. I think it's considered good form, too, as the cast doesn't really add anything not already present, and might even drift out of date with the surrounding code. (And just gets in the way of legibility.)

IIRC, it is not valid C++. But malloc()/void * is somewhat rarer to find there, as you have better options.

Re:A bug story: data alignment in C on x86 (2016) by shakna on Tuesday September 04, 2018 @09:21PM (#17914379)
That's an implicit cast from void * to int * .

    void *malloc(size_t size);
Re:A bug story: data alignment in C on x86 (2016) by tedunangst on Tuesday September 04, 2018 @10:04PM (#17914555)
In standard parlance, that is a conversion. Only (this) is a cast.
Re:A bug story: data alignment in C on x86 (2016) by caf on Tuesday September 04, 2018 @10:04PM (#17914552)
There's no such thing as an "implicit cast". A cast is an explicit conversion (a cast expression), and the assignment expression discussed here has an implicit conversion.
Re:A bug story: data alignment in C on x86 (2016) by valarauca1 on Tuesday September 04, 2018 @07:05PM (#17913638)
you mean `mmap`?

Also yes I'm radical enough to suggest the 50+ year old C standard library API is bad. Which isn't really a radical position >.>

Re:A bug story: data alignment in C on x86 (2016) by deathanatos on Tuesday September 04, 2018 @09:20PM (#17914374)
mmap and malloc do very different things. You didn't really clarify why you thought malloc() was bad, or why you think someone should call mmap(), so I'm mostly left to guess.

mmap, for example, is free to require that the requested range be a multiple of a page; trying to use it as if it were malloc() would, on such systems, cause any allocations less than a page to require a whole page. And since this is a rather common case, you'd be using a ton more memory.

malloc(), however, can subdivide a large allocation from the OS into small allocations for the process; it is my understanding that most modern malloc()s will do this, in some form. (It is also my understanding that some malloc() implementations will use mmap() for allocating memory from the OS…)

Re:A bug story: data alignment in C on x86 (2016) by valarauca1 on Wednesday September 05, 2018 @03:56PM (#17921360)
`malloc` is part of the standard library. the underlying implementation is environmental specific, or implementation specific depending on the host application.

Generally speaking `mmap` is the system call that

* Allocates memory

* Moves allocations around

`malloc` is the common entry point into a library that attempts to ensure you invoke the `mmap` system call as little as possible.

   trying to use it as if it were malloc() would, on such systems,
   cause any allocations less than a page to require a whole page.
   And since this is a rather common case, you'd be using a ton more
   memory.
This is a misconception [1]. Malloc is governed by hardware limitations. If you ask for 50bytes, it'll give you a full 4KiB page, because hardware can only segment memory in 4KiB chunks. Actually some allocators will attempt to _use_ those non-allocated regions of the remaining ~4046 bytes for other allocations in order to be faster (less systemcalls), this is why sometimes writing past buffer ends can give you garbage data instead of segfaults.

[1] https://stackoverflow.com/questions/45458732/how-are-memory-...

Re:A bug story: data alignment in C on x86 (2016) by deathanatos on Wednesday September 05, 2018 @09:07PM (#17923194)
The link you post is about mmap; the section you quote is about malloc. The link is correct — about mmap; but that limitation does not apply to malloc. The link precisely backs up the part you're quoting, so saying "This is a misconception" makes no sense.

> If you ask for 50bytes, it'll give you a full 4KiB page, because hardware can only segment memory in 4KiB chunks.

No, not really. (But you seem to realize this, as you contradict this point in the next quote I'll pull.) A malloc implementation is free to divvy up a page in order to service several allocations. Just because the pointer it returns to you is somewhere within a page that must, almost by definition, be a full page is irrelevant to the point of this discussion (that malloc() is not needed b/c mmap exists, and is somehow superior). The point is that the allocation consumes, in the grand scale of things, an amount on par with the requested size. (There is some overhead, of course, but not nearly the same are the mmap-must-return-a-full-page overhead.)

> Actually some allocators will attempt to _use_ those non-allocated regions of the remaining ~4046 bytes for other allocations in order to be faster (less systemcalls), this is why sometimes writing past buffer ends can give you garbage data instead of segfaults.

That was part of the point I was making.

You seem to have misunderstood my post. malloc() is free to divvy up a page (potentially handed to it by mmap) into several allocations. mmap() cannot. The point here is that replacing every malloc() call with a call to mmap() results in the loss of that ability to divvy up pages, requiring each allocation to be a full page; this would significantly bloat the memory consumed by an application, as small allocations would not share pages. Hence why those two calls are not equivalent, which is what the original post:

> you mean `mmap`?

> Also yes I'm radical enough to suggest the 50+ year old C standard library API is bad.

Very much seemed to imply. Though, again, I asked for clarification on your point, to make sure I'm understanding it, and you again failed to provide it. So, again,

> You didn't really clarify why you thought malloc() was bad, or why you think someone should call mmap() [in lieu of malloc()], so I'm mostly left to guess.

Re:A bug story: data alignment in C on x86 (2016) by valarauca1 on Thursday September 06, 2018 @12:54PM (#17927614)

    A malloc implementation is free to divvy up a page in
    order to service several allocations
But like the hardware won't enforce that. So yeah you can malloc 50bytes, but the hardware is just going to assume there is a 4096 based page behind it. Because what memory can/cannot be written/read too is controlled by mmap -> linux -> hardware MMU.

So when ever `malloc` gives you 50bytes without allocating an entire page it's just lying to you and giving you a pointer offset from another allocation. This isn't strictly being enforced by anything. There is zero contract that enforces what memory you can/cannot use.

Re:A bug story: data alignment in C on x86 (2016) by deathanatos on Friday September 07, 2018 @03:19PM (#17936933)
> There is zero contract that enforces what memory you can/cannot use.

There is, though; malloc's specification forms a contract. You are not allowed to access outside the bounds of the returned pointer. That some hardware allows it sometimes does not mean that there is "zero contract".

The function represents an interface, and an interface is a set of rules and behavior both the user & consumer of that interface expect. If you stray from that, it's UB, and the machine can do whatever at that point.

In this case, that's what matters. We don't know what hardware malloc or mmap will run on, and it isn't impossible that someone could implement a malloc() implementation that did enforce those bounds. valgrind, for example, is very close to this: it already knows when you access memory outside of the bounds of a malloc'd region (as that's its purpose: to report such accesses), and it would be a trivial modification to it to kill the program on an illegal access.

Re:A bug story: data alignment in C on x86 (2016) by shakna on Tuesday September 04, 2018 @09:17PM (#17914352)
Is your irritation aimed at the implementations of malloc in the standard compilers? Or all malloc implementations? (e.g. Hoard [0]).

[0] https://github.com/emeryberger/Hoard

Re:A bug story: data alignment in C on x86 (2016) by valarauca1 on Wednesday September 05, 2018 @03:46PM (#17921277)
my irritation is C's primitive type system. And programmers willingness to conflate their standard library with system level operations.