Using zero for errors means you can express error checking directly with C expressions that appear boolean. It's not as safe or rigorous, but is more readable.

Using nonzero for errors means you can do the same thing. This is common in things like this:

    if(somefunc()) goto fail;

Additionally, if the return is a fail/pass status, there is only one success (0), and you can use all the nonzero values for error codes.

Yes, you can, but that doesn't read very well to my eyes, given most function names. if(open()), if(read()), if (CreateWindowExW()), all suggest success to me.

   if (!open(...)) goto fail;

   if (!read(...)) goto fail;

   if (!close(...)) goto fail;

reads a lot nicer to me. Although, the reality is that you need to assign to an error variable, which makes it not as nice and clean.

No. CreateFile returns INVALID_HANDLE_VALUE which is -1. Compare this to CreateSemaphore which also returns a HANDLE, but returns NULL[1] on failure.

Returning int(0) as a failure is often done in C if the return value is a boolean since C lacks a true boolean type.

[1] NULL should never be used in actual code because it is ambiguous.

Those are functions that return more possible success values than failure values. The three common conventions seem to be:

Nonzero: error code, zero: success

Nonzero: one of many possible success values, zero: error (e.g. malloc())

Negative: error code, positive: one of many possible success values

And each has its tradeoffs.