News for hackers. Stuff that matters.
Re:Show HN: Progressbar - A C library for displaying command... by _RPM on Saturday August 29, 2015 @12:33AM (#10139073)
Consider checking the return value of malloc (e.g: https://github.com/doches/progressbar/blob/master/lib/progre...) You're setting users/yourself up for a disaster if you're assuming it will never fail.

The standard [0] states that the size of `char` will be 1 byte. So there is no point in doing the sizeof operator on char. (e.g: https://github.com/doches/progressbar/blob/master/lib/progre...)

6.5.3.4 - Paragraph 3

> The sizeof operator... (page 80)

[0] http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf

Re:Show HN: Progressbar - A C library for displaying command... by pjtr on Saturday August 29, 2015 @04:11AM (#10139414)
Just in case anyone else finds this fascinating: Correct, sizeof(char) is always 1 byte. A byte contains CHAR_BIT bits (page 37). The implementation-defined value of CHAR_BIT is at least 8 (page 21). There are some implementations [1] that chose other values like 16 or 32. So while sizeof(char) is always 1 byte it's not necessarily always 1 octet.

[1] http://bytes.com/topic/c/answers/223565-implementations-char...

Re:Show HN: Progressbar - A C library for displaying command... by danieltillett on Saturday August 29, 2015 @12:45AM (#10139097)
How often is knowing malloc has failed useful? You going to segfault either way.

I always use sizeof with char just to stay consistent - if you don’t you will get caught out one day when you change something and forget to add in the needed new sizeof.

Re:Show HN: Progressbar - A C library for displaying command... by _RPM on Saturday August 29, 2015 @12:50AM (#10139111)
It's _very_ important to know if it failed if you don't want to introduce undefined behavior. With undefined behavior, _anything_ can happen, (e.g. arbitrary code execution). But let me make it clear: malloc failing doesn't mean that undefined behavior will happen. However, pretending like malloc didn't fail when it really did will most likely get you undefined behavior. You're not going to segfault either way if you handle things correctly. Consider a web server running out of memory to allocate a request structure, do you think it is a good idea to just let it SIGSEV?
Re:Show HN: Progressbar - A C library for displaying command... by aivarsk on Saturday August 29, 2015 @02:53AM (#10139321)
I have some questions:

Have you experienced malloc failing a lot? How do you test to ensure error handling code paths work as expected?

Have you read 'notes' section of Linux malloc manpage? It says:

By default, Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no guarantee that the memory really is available. In case it turns out that the system is out of memory, one or more processes will be killed by the OOM killer.

Re:Show HN: Progressbar - A C library for displaying command... by dalke on Saturday August 29, 2015 @04:22AM (#10139428)
I had malloc fail recently, on a machine with low rlimits. Here's an example on FreeBSD 8.2:

    #include <stdlib.h>
    #include <stdio.h>
    #include <sys/resource.h>
    
    const int SIZE = 64*1024*1024;
    
    int main() {
      char *buffer = NULL;
      struct rlimit r;
    
      /* Limit all memory to 1024 bytes */
      r.rlim_cur = 1024;
      r.rlim_max = 1024;
    
      if (setrlimit(RLIMIT_AS, &r) == -1 ) {
        perror("setrlimit(RLIMIT_AS) error\n");
        exit(EXIT_FAILURE);
      }
    
      if ( (buffer = malloc( sizeof(long) * SIZE) ) == NULL) {
        perror("malloc() failed\n");
        exit(EXIT_FAILURE);
      }
      free(buffer);
      printf("Malloc worked\n");
      return 0;
    }
Here's the output:

    %cc fail.c
    %./a.out
    malloc() failed
    : Cannot allocate memory
If malloc failure testing is important to you, then you'll need some way to replace your use of the system malloc/free with special versions of your own. (Eg, through function pointers, or LD_PRELOAD.) Your own versions can be crafted to insert faults in just the right spots.

It's tedious work.

Re:Show HN: Progressbar - A C library for displaying command... by danieltillett on Saturday August 29, 2015 @03:17AM (#10139350)
I have never knowingly experienced malloc failing in any of my code which is why I asked if it is something to worry about in practice. Once you run out of memory things start dying or the system locks up hard.

Edit. It is impossible for malloc to fail (at least on linux) [1]. Seems rather pointless checking for something that can’t happen.

1. https://scvalex.net/posts/6/

Re:Show HN: Progressbar - A C library for displaying command... by sovande on Saturday August 29, 2015 @04:28AM (#10139438)
On virtual memory systems, it is a safe bet that malloc will "never" fail to return a value. It is just good form to check the return value of system calls. If you program for embedded systems (e.g. Arduino) and bring over the bad habit of never checking malloc, you are going to be in for a surprise. I'm not sure, but I believe on iOS, malloc could fail on the device.
Re:Show HN: Progressbar - A C library for displaying command... by dalke on Saturday August 29, 2015 @04:27AM (#10139437)
That link shows that malloc failed, just not due to exceeding available RAM. At the end it says "malloc finally failed because of address space exhaustion."
Re:Show HN: Progressbar - A C library for displaying command... by danieltillett on Saturday August 29, 2015 @05:01AM (#10139475)
If the memory was actually being used it would have crashed long before that. The test was kind of artificial.
Re:Show HN: Progressbar - A C library for displaying command... by dalke on Saturday August 29, 2015 @05:38AM (#10139515)
Absolutely. I was pointing out that "It is impossible for malloc to fail" is, according to the linked page, an incorrect summary.
Re:Show HN: Progressbar - A C library for displaying command... by danieltillett on Saturday August 29, 2015 @06:55AM (#10139623)
It is still pointless checking if malloc fails on linux :)

Actually the real issue here is that you should be aware that on some systems malloc might fail and not segfault.

Re:Show HN: Progressbar - A C library for displaying command... by danieltillett on Saturday August 29, 2015 @01:26AM (#10139172)
If you have run out of memory there is not much you can really do except crash. Even logging is a problem. Better to just crash and deal with it.

I was under the impression that compiler optimisation often removed any NULL checks anyway [1].

1. http://blog.llvm.org/2011/05/what-every-c-programmer-should-...

Re:Show HN: Progressbar - A C library for displaying command... by AndyKelley on Saturday August 29, 2015 @01:37AM (#10139189)
> I was under the impression that compiler optimisation often removed any NULL checks anyway

Goodness.

No, you are misunderstanding the example. The optimizer did not introduce a bug by removing a NULL check. The code had a bug because it dereferenced the pointer before checking for NULL.

Re:Show HN: Progressbar - A C library for displaying command... by danieltillett on Saturday August 29, 2015 @01:59AM (#10139234)
Yes, but they were making the point that thinking you are checking for a null pointer may not always hold especially once inlining happens.

Are there any modern platforms where dereferncing a NULL pointer will not cause a crash?

Edit. Answering my own question it looks like ARM is one [1].

1. https://www.securecoding.cert.org/confluence/display/c/EXP34...

Re:Show HN: Progressbar - A C library for displaying command... by AndyKelley on Saturday August 29, 2015 @02:31AM (#10139286)
> thinking you are checking for a null pointer may not always hold especially once inlining happens

Incorrect. The example code did not check for a null pointer. It dereferenced a pointer, which implies that the pointer will never be null, and then it pointlessly checked for null on something that could not have been null. The optimizer in fact made the code more clear and easier to read, while maintaining correctness.

Re:Show HN: Progressbar - A C library for displaying command... by danieltillett on Saturday August 29, 2015 @02:52AM (#10139318)
Quoting from the reference:

"While this is intentionally a simple and contrived example, this sort of thing happens all the time with inlining: inlining a function often exposes a number of secondary optimization opportunities. This means that if the optimizer decides to inline a function, a variety of local optimizations can kick in, which change the behavior of the code. This is both perfectly valid according to the standard, and important for performance in practice.”

In practice derefencing a NULL pointer on any x86 platform will always cause a crash (defined undefined behaviour I guess), but you can’t assume this on other platforms (ARM seems to be the big one).

I have to say before today I had assumed dereferncing a NULL pointer would always cause a segfault. It is always good to learn something new.

Re:Show HN: Progressbar - A C library for displaying command... by sherjilozair on Saturday August 29, 2015 @02:28AM (#10139276)
You can't be further from the truth. A vast magntitude of bugs are due to the system running as if nothing has happened operating on garbage data, and giving you garbage results. If your system is robust to garbage data, say neural network training, good luck finding the bug then.
Re:Show HN: Progressbar - A C library for displaying command... by danieltillett on Saturday August 29, 2015 @03:00AM (#10139333)
But what do you do in practice if you have run out of memory?
Re:Show HN: Progressbar - A C library for displaying command... by davvid on Saturday August 29, 2015 @03:51AM (#10139394)
One approach is to free memory, retry, and hope for the best.

e.g. https://github.com/git/git/blob/master/wrapper.c#L63

Re:Show HN: Progressbar - A C library for displaying command... by dalke on Saturday August 29, 2015 @04:33AM (#10139444)
Another is to pre-allocate enough reserve memory to be able to save and exit cleanly. When malloc fails, switch to the reserve memory and either shut down nicely, or post a dire warning to the user that the end is neigh and to "SAVE AND EXIT IMMEDIATELY".
Re:Show HN: Progressbar - A C library for displaying command... by fra on Saturday August 29, 2015 @01:54AM (#10139222)
No no no. You can't predict where that library is going to be used. Check the return value of malloc, communicate the failures to the caller and let them handle it in an application-appropriate way. They may want to crash, and they may not.
Re:Show HN: Progressbar - A C library for displaying command... by AndyKelley on Saturday August 29, 2015 @01:30AM (#10139179)
You're not going to segfault either way, especially if you're a library. If you're a library then when the API user creates your context, if possible, you should preallocate all the memory you're going to use, and if that fails, report the error. If preallocation is not possible then, as a library, it is your duty to report memory allocation errors just as you would any other error, ideally as a return code.
Re:Show HN: Progressbar - A C library for displaying command... by thinkpad20 on Saturday August 29, 2015 @12:38AM (#10139085)
There's still a point, in terms of readability and consistency. Certainly there doesn't seem to be a downside aside from a few extra characters.
Re:Show HN: Progressbar - A C library for displaying command... by _RPM on Saturday August 29, 2015 @12:43AM (#10139092)
Redundant != consistent. But I agree with you, there isn't really a downside. I'm being pedantic here. -pedantic
Re:Show HN: Progressbar - A C library for displaying command... by spash on Saturday August 29, 2015 @01:18AM (#10139157)
> I'm being pedantic here.

Please be. Seemingly random bits of (honestly - actually very useful) information like yours are why I'm reading HN in the first place instead of, say, random Youtube comments.

Someone may argue about this being overly pedantic, or about such comment being more suitable for sites like Stack Overflow, or about the whole thing being completely clear from the standards documentation itself... etc. But that's really not the point of it when this isn't the kind of thing you would be even intentionally looking at, or for, in the first place.

Every time I open a HN thread, I can be sure that I will find at least a good few nice and memorable comments that seemingly don't add anything revolutionary to the topic at hand, but instead will make you pause, slap your forehead and exclaim "holy moly, that's in essence actually perfectly obvious, yet somehow I never even thought about it from this point of view before!" Which in turn suddenly makes you realize something completely unrelated in your own world, yet affected or plagued by a similar type of issue, that would otherwise end up completely unnoticed if not for your comment.

In short, one man's pedantic can still be other man's sudden epiphany.

Re:Show HN: Progressbar - A C library for displaying command... by Doches on Saturday August 29, 2015 @10:13PM (#10142041)
This sort of pedantry is exactly why I submitted this library to Show HN in the first place -- I'm not a C programmer by any particular training other than self-taught, and I've never worked at such a low level in industry. Knowing that C programmers consider something like sizeof(char) to be distractingly weird (where I would just consider it consistent/defensive) is good to know!