News for hackers. Stuff that matters.
Re:C puzzles by hatsunearu on Thursday November 10, 2016 @11:16AM (#12921655)
That IA-64 one is really puzzling, anyone know what the heck is happening?
Re:C puzzles by mikeash on Thursday November 10, 2016 @11:21AM (#12921707)
There's no #include <stdlib.h> first, so you get an implicit prototype for malloc. With an implicit prototype, the function is assumed to return int. The cast then converts the returned int to int*. This works on 32-bit where int is the size of a pointer, but on 64-bit with 32-bit ints, the top half of the pointer gets chopped off and you end up with a nonsense value.

This is why it's considered bad form to cast the result of malloc in C. Of course, modern compilers will warn about implicit prototypes, and as of C99 it's no longer legal.

Re:C puzzles by innocenat on Thursday November 10, 2016 @01:07PM (#12922677)
Oh so he meant to say Intel 64/amd64. IA-64 is Intel Itanium. I just skipped it because I know nothing about IA-64 at all.
Re:C puzzles by mikeash on Thursday November 10, 2016 @01:28PM (#12922863)
He might have meant IA-64. The problem is likely to appear on any 64-bit architecture where int is smaller than a pointer, which is most (or all?) of them. Depending on the compiler and such, big discussion below if you're interested in boring details.
Re:C puzzles by ndesaulniers on Thursday November 10, 2016 @08:31PM (#12926579)
inserts big discussion for those interested

https://nickdesaulniers.github.io/blog/2016/05/30/data-model...

Re:C puzzles by hatsunearu on Thursday November 10, 2016 @04:53PM (#12924777)
Worked on my x64 machine and my IRC's. I googled it (just google parts of the introducing text) and apparently it's related to IA-64 architecture specifics.
Re:C puzzles by asamy45 on Thursday November 10, 2016 @02:36PM (#12923483)
Actually he clearly misunderstands the difference, IA-32 refers to x86, which he then says IA-64, which refers to Itanium, which is sort of a bad comparison then.
Re:C puzzles by asamy45 on Thursday November 10, 2016 @12:14PM (#12922183)
There is no cast that happens behind the scene, look at the generated assembly.
Re:C puzzles by mikeash on Thursday November 10, 2016 @12:18PM (#12922215)
Do you mean that the cast in the code gets compiled out?
Re:C puzzles by asamy45 on Thursday November 10, 2016 @12:26PM (#12922305)
No. The int to int * truncation does not happen, the cast does not matter. What matters is the `int *p' which is 64-bit on 64-bit systems, so the compiler will just move the returned value in RAX into wherever p is, so no truncation happens, even without the cast. Look at the generated assembly, you'll see what I mean.
Re:C puzzles by mikeash on Thursday November 10, 2016 @12:48PM (#12922533)
Clang is too clever and gives malloc the correct implicit declaration even when you don't have an explicit prototype:

    test.c:10:12: warning: implicitly declaring library function 'malloc' with type 'void *(unsigned long)'
      [-Wimplicit-function-declaration]
I think that having a prototype mismatch with the actual declared function is undefined behavior, so this is a legal way to resolve it, but not every compiler will. Older compilers tended to treat malloc as just another function and wouldn't do this.

I was able to replicate the older behavior by wrapping malloc in my own function. In one file:

    int main()
    {
        int* p;
        p = (int*)wrapped_malloc(sizeof(int));
        *p = 10;
        return 0;
    }
    
And in another file:

    void *wrapped_malloc(int size) {
        return malloc(size);
    }
Compiling them into one program results in a crash. The relevant bit of assembly is:

    0000000100000f4f	movslq	%eax, %rdi
So it is indeed only extracting the lower 32 bits of the returned pointer.

The page looks pretty old (the IA-64 reference sure is dated, anyway) so I'd guess that it was referring to an older compiler that didn't have a special case for malloc like this.

Re:C puzzles by asamy45 on Thursday November 10, 2016 @01:05PM (#12922671)
Right, C compilers will automatically known it's part of libc. You didn't need to do the separate file thing, you could just have the wrapper and it will trigger the movslq, or a cltq.
Re:C puzzles by mikeash on Thursday November 10, 2016 @01:08PM (#12922696)
With the wrapper in the same file, clang threw an error because the definition conflicted with the implied prototype.
Re:C puzzles by asamy45 on Thursday November 10, 2016 @01:25PM (#12922838)
Doesn't happen for me. Both GCC and Clang compile it fine.
Re:C puzzles by mikeash on Thursday November 10, 2016 @01:26PM (#12922845)
Strange. Here's my erroring code:

    int main()
    {
        int* p;
        p = (int*)wrapped_malloc(sizeof(int));
        *p = 10;
        return 0;
    }
    
    void *wrapped_malloc(int size) {
        return malloc(size);
    }
And the actual error:

    test.c:9:11: error: conflicting types for 'wrapped_malloc'
        void *wrapped_malloc(int size) {
              ^
    test.c:4:19: note: previous implicit declaration is here
            p = (int*)wrapped_malloc(sizeof(int));
Are you doing anything differently?
Re:C puzzles by abbeyj on Thursday November 10, 2016 @01:52PM (#12923089)
Try putting wrapped_malloc above main. Or at least declaring it above main.
Re:C puzzles by mikeash on Thursday November 10, 2016 @02:05PM (#12923204)
If it's above main then there's no longer an implicit prototype and you're no longer replicating the problem.
Re:C puzzles by asamy45 on Thursday November 10, 2016 @02:25PM (#12923387)
No, I am not sure why this happens.
Re:C puzzles by abbeyj on Thursday November 10, 2016 @01:50PM (#12923070)
The question asks about IA-64 (Itanium) and IA-32. You're talking about the RAX register which is x86-64.

If you call malloc without a prototype in scope, bad things can happen. Just because it happens to work out OK with the platform and compiler that you tested doesn't mean that it will work everywhere or that it will keep working in the future.

Re:C puzzles by asamy45 on Thursday November 10, 2016 @02:25PM (#12923383)
His scope of talk sounds as if he is referring to x86-64. Many people mistake the IA-64 for x86-64, that's why I assumed he'd be talking about it. Not to mention that IA-32 refers to x86, which he clearly seems to misunderstand.
Re:C puzzles by to3m on Thursday November 10, 2016 @04:39PM (#12924656)
VC++ pops a CDQE in there. For ``p=(int *)malloc(1000);'', with no prototype for malloc, it generates this:

        mov     ecx, 1000
        call    malloc
        cdqe
        mov     QWORD PTR p$[rsp], rax
I'll have to check if it's still true today, but Windows/the VC++ CRT certainly used not have no qualms about handing out pointers to memory below the 4GByte mark. So if you have a problem like this, it can go undetected for quite some time...

(Don't know about Linux. 64-bit OS X binaries usually start with a 4GByte section at 0, so the bottom 4GBytes simply isn't available.)

Re:C puzzles by gowrikumar on Thursday November 10, 2016 @04:32PM (#12924598)
Yup! that's correct. Later versions of compilers have become intelligent with standard library function and I doubt if this reproes on the current compilers - for standard library functions.
Re:C puzzles by mikeash on Friday November 11, 2016 @09:33AM (#12930907)
It definitely doesn't happen in whatever recent versions of clang and gcc I tested with. I had to wrap malloc in a function that the compiler didn't have a special case for in order to make reproduce the crash.
Re:C puzzles by mfukar on Thursday November 10, 2016 @01:42PM (#12922996)
Close. Sign extension. But this is a C puzzle, so I'd get away with "undefined behaviour".
Re:C puzzles by mikeash on Thursday November 10, 2016 @02:06PM (#12923208)
Where does sign extension occur here?
Re:C puzzles by asamy45 on Thursday November 10, 2016 @02:46PM (#12923571)
The movslq instruction will sign extend the upper 32 bits.
Re:C puzzles by mikeash on Thursday November 10, 2016 @03:07PM (#12923800)
Right you are! Thanks. I tried to check, but didn't realize my test case had produced a zero bit in that position so didn't expose sign extension.
Re:C puzzles by jandrese on Thursday November 10, 2016 @01:36PM (#12922939)
It didn't work (as in crash) for me:

  % uname -a
  FreeBSD 10.1-RELEASE FreeBSD 10.1-RELEASE #0 r274401: Tue Nov 11 21:02:49 UTC 2014     [email protected]:/usr/obj/usr/src/sys/GENERIC  amd64
  % gcc -o foo foo.c
  foo.c: In function 'main':
  foo.c:4:17: warning: incompatible implicit declaration of built-in function 'malloc' [enabled by default]
       p = (int*)malloc(sizeof(int));                 ^
  % ./foo
  %
This is with GCC 4.8.5
Re:C puzzles by mnay on Friday November 11, 2016 @12:04AM (#12928060)
The code in question violates the newer C standards (C99, C11). For instance, it doesn't even compile using "gcc -std=c99 -pedantic-errors".

In the older C standard (C89), the code has undefined behaviour because implicit declaration of the function malloc is not compatible with its definition in the standard library. It is a futile task to try to define the undefined.

Re:C puzzles by asamy45 on Thursday November 10, 2016 @12:12PM (#12922159)
Has nothing to do with includes, it has no error. The C compiler will automatically know its imported. He probably thinks that `int *` is 32-bit regardless.
Re:C puzzles by exprF on Thursday November 10, 2016 @12:29PM (#12922335)
It has everything to do with includes. Just think how implicit declarations are defined.
Re:C puzzles by asamy45 on Thursday November 10, 2016 @12:32PM (#12922370)
Just think of the generated assembly.
Re:C puzzles by bnmfsd on Thursday November 10, 2016 @11:30AM (#12921813)
hint: missing include