News for hackers. Stuff that matters.
Re:Building the Linux kernel with clang by bitcharmer on Thursday September 21, 2017 @08:54AM (#15302820)
This is a great step forward. Compiling with clang will allow kernel developers to run code checks and analysis unavailable to gcc.

This will definitely have a positive effect on code quality.

Not even mentioning it should be a little bit easier to configure IDEs like CLion for working with Linux source tree.

Re:Building the Linux kernel with clang by taeric on Thursday September 21, 2017 @10:00AM (#15303446)
Is there a list of checks and analysis that you can't do from a gcc pipeline?
Re:Building the Linux kernel with clang by melkiaur on Thursday September 21, 2017 @11:56AM (#15304732)
the static analyzer, along with all the other checks available from clang-tidy: https://clang.llvm.org/extra/clang-tidy/checks/list.html
Re:Building the Linux kernel with clang by taeric on Thursday September 21, 2017 @07:13PM (#15308105)
I'm surprised there is a static analyzer available for clang that isn't somewhat equivalently offered in gcc toolchains. I guess I'm assuming folks would be using something like Coverity for major static analysis.

Specifically, this sounds like it is just preferring the other toolchain. Are there literally things that don't have equivalents?

Re:Building the Linux kernel with clang by kevindqc on Thursday September 21, 2017 @10:16AM (#15303618)
Maybe he's talking about the sanitizers? eg: https://clang.llvm.org/docs/AddressSanitizer.html
Re:Building the Linux kernel with clang by pksadiq on Thursday September 21, 2017 @10:32AM (#15303783)
Those sanitizers are present in gcc too[0].

[0] https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.h...

Re:Building the Linux kernel with clang by ndesaulniers on Thursday September 21, 2017 @12:05PM (#15304841)
Present != Equal, feature wise. Not by a long shot. At the same conf, one of my friends who develops the sanitizers said it's easier to develop for LLVM, so that's where Dev work is done.
Re:Building the Linux kernel with clang by pksadiq on Thursday September 21, 2017 @12:19PM (#15304992)
> Present != Equal, feature wise.

It is developed by google[0] afaik. The link says it is present in LLVM and gcc. I don't know what makes the difference (may be License?). On the contrary I have felt gcc better for my work. I have been following a clang bug[1] for long and have seen no fix so far.

[0] https://github.com/google/sanitizers/wiki/AddressSanitizer

[1] https://bugs.llvm.org/show_bug.cgi?id=23910

Re:Building the Linux kernel with clang by pebers on Thursday September 21, 2017 @01:12PM (#15305502)
Clang has better support for plugins and calling it as a library; that sort of thing was resisted by GCC. Google has a bunch of large-scale tooling based on the Clang frontend which AFAIK wouldn't be possible with GCC.
Re:Building the Linux kernel with clang by jcelerier on Thursday September 21, 2017 @02:39PM (#15306253)
sure sure, but today (and even two years ago) you can add -fsanitize={address,undefined,etc} on both clang and gcc. So this does not answer the question.
Re:Building the Linux kernel with clang by kevindqc on Thursday September 21, 2017 @10:58AM (#15304068)
ah! thanks
Re:Building the Linux kernel with clang by albinofrenchy on Thursday September 21, 2017 @11:16AM (#15304261)
Sanitizers are run-time tools; I'd be surprised if any of them worked out of the box with the kernel.

Address sanitizer was "ported" to the kernel a while ago; not sure if UBSAN is or will be in the future.

Re:Building the Linux kernel with clang by ndesaulniers on Thursday September 21, 2017 @12:06PM (#15304849)
There are kernel versions. KASAN, KUBSAN, KMSAN.
Re:Building the Linux kernel with clang by sanxiyn on Thursday September 21, 2017 @10:35AM (#15303819)
All sanitizers are also available in GCC, so no.
Re:Building the Linux kernel with clang by bitcharmer on Thursday September 21, 2017 @12:02PM (#15304803)
Are you sure? Last time I checked the ones llvm had and gcc did not were:

Dynamic alloca overflow detection

Symbol size changing for global variables

Adaptive global redzone sizes

Invalid pointer pairs detection

Support for no_sanitize attribute

Support for dead stripping of globals on Linux

I might be out of date on some or even all, so please correct me where I'm wrong. Thank you in advance.

Re:Building the Linux kernel with clang by ndesaulniers on Thursday September 21, 2017 @12:06PM (#15304853)
Wrong. Not all kernel versions.
Re:Building the Linux kernel with clang by Sir_Cmpwn on Thursday September 21, 2017 @11:20AM (#15304316)
I know this isn't the opinion of many kernel developers, but I think a lot of the problems encountered are better solved by removing proprietary GNU C extensions from the kernel code. I would love to start sending patches but I expect they'd just be NACKed.
Re:Building the Linux kernel with clang by taeric on Thursday September 21, 2017 @11:59AM (#15304755)
Sounds like this is a prime example where evidence would help. Just thinking "a lot of the problems..." doesn't do much to show why that is the case. Especially without even knowing which extensions you are specifically referring to.

That is, my guess is these extensions are the equivalent of Chesterton's fence. They were introduced for a reason. If you have evidence that that reason is no longer apropros, present the evidence. But don't just complain about the fence.

Re:Building the Linux kernel with clang by Sir_Cmpwn on Thursday September 21, 2017 @12:10PM (#15304901)
I mean, the problems faced by compiling the kernel with Clang (or any other standards-implementing C compiler). The problems are pretty easy to find, just look at the relevant commits in Clang and Linux. The main problem Clang faced with the kernel was lots of use of GNU C extensions everywhere.

I think it's self-evident that the use of proprietary, nonstandard language extensions is a bad idea. It leads to multi-year efforts getting other compilers to support "features" that aren't in the spec (and as a result, are typically underspeced by the compilers that implement them and require reverse engineering to fully understand and implement correctly). Linux isn't written in C, it's written in some other language that's deceptively similar to C. It should be enough to write a compiler that accurately implements the C standard to compile the kernel with.

Re:Building the Linux kernel with clang by taeric on Thursday September 21, 2017 @12:18PM (#15304977)
But, I could just as easily say it is self-evident that the use of proprietary, nonstandard languages extensions was required to get the linux kernel written.

Don't get me wrong, I'm highly amenable to this argument, but it is not an evidence based one.

Now, there could be some easy evidence that would help. For example, evidence that kernels which are built using clang build just as easily using gcc. Or icc. Or ___. That would help. Right off, I do not know if that is the case.

Re:Building the Linux kernel with clang by Sir_Cmpwn on Thursday September 21, 2017 @12:23PM (#15305025)
I mean, I said it was self-evident but then provided evidence as well. Here is the evidence:

- Multi-year effort by Clang, a well regarded standards-compliant C compiler, to support the kernel

- Proprietary extensions do not conform to the same strict guidelines for specifications as the C standard and may require reverse engineering to implement correctly

- C is defined by the C specification and Linux is not written in C by this metric

- Therefore, C compilers cannot compile the kernel

Correct me if I'm wrong but I cannot extract similar facts from your comment.

Re:Building the Linux kernel with clang by taeric on Thursday September 21, 2017 @12:37PM (#15305169)
But my counter evidence was that these extensions were required to actually build the kernel. Which is also somewhat self-evident.

So my question to you is if you have evidence that those extensions are not required to actually build the kernel? I get that they are the "sticking" point to moving to clang. But are they also required for the kernel to actually do its job?

Re:Building the Linux kernel with clang by Sir_Cmpwn on Thursday September 21, 2017 @02:55PM (#15306357)
I see what you mean. Yes, there are counter examples of kernels that can be compiled without extensions. With my rough knowldge of the Linux kernel in particular, I imagine there are a few places where the use of extensions makes for much better code than without, and for those places the change is of debatable utility. However, extensions are the rule rather than the exception for the kernel. The bulk of the kernel could be written without them.
Re:Building the Linux kernel with clang by halfteatree on Thursday September 21, 2017 @12:44PM (#15305244)
FreeBSD can be built with gcc, although it's built with clang by default: https://wiki.freebsd.org/BuildingFreeBSDWithClang

The patches to make it work for icc is simple enough that this is almost a non-issue: https://lists.freebsd.org/pipermail/freebsd-arch/2003-Novemb... (albeit old)

Re:Building the Linux kernel with clang by pjmlp on Thursday September 21, 2017 @12:38PM (#15305190)
Unix managed to be written without those extensions.
Re:Building the Linux kernel with clang by taeric on Thursday September 21, 2017 @12:43PM (#15305238)
Which one? And did it successfully compile with multiple compilers?
Re:Building the Linux kernel with clang by pjmlp on Thursday September 21, 2017 @01:54PM (#15305884)
> Which one?

UNIX, the genesis of C and all that derived from AT&T original code.

> And did it successfully compile with multiple compilers?

Any that was K&R C compatibile.

GCC was a toy compiler not used for any serious UNIX development until Sun decided to start selling the SDK as part of a SunOS developers edition, quickly followed by other UNIX vendors.

So by definition, until GCC became an usefull project, there were no UNIX kernels written with GCC C extensions.

Re:Building the Linux kernel with clang by taeric on Thursday September 21, 2017 @05:28PM (#15307516)
So, I would have understood if you said the original AT&T UNIX. I've always heard the term used to cover the entire family. And, I honestly had no clue whether or not the likes of SunOS/HP-UX/etc were compiled with multiple compilers. My understanding was each typically came with their own compiler.
Re:Building the Linux kernel with clang by kchoudhu on Thursday September 21, 2017 @12:19PM (#15304994)
An inability to compile on nothing but GCC is a feature for a lot of zealots out there.
Re:Building the Linux kernel with clang by Koshkin on Thursday September 21, 2017 @11:08AM (#15304192)
I wonder what Torvalds' opinion on Clang is. He had harsh words to say about a recent version of gcc, as I remember.
Re:Building the Linux kernel with clang by paulddraper on Thursday September 21, 2017 @01:17PM (#15305547)
IIRC, his complaints are of the nasal demon variety.

He hates compiler writers being tricky.

Re:Building the Linux kernel with clang by d33 on Thursday September 21, 2017 @02:39PM (#15306254)
Context:

In the C community, undefined behavior may be humorously referred to as "nasal demons", after a comp.std.c post that explained undefined behavior as allowing the compiler to do anything it chooses, even "to make demons fly out of your nose".

https://en.wikipedia.org/wiki/Undefined_behavior

Re:Building the Linux kernel with clang by MichaelMoser123 on Thursday September 21, 2017 @09:55AM (#15303385)
I found that clang compiled code was requiring more memory on the stack than the same code compiling with gcc. Was it some obscure switch that did that? no idea. However with the Linux kernel you have some tight limitations with regard to stack size. Could the author run his clang compiled kernel? The article does not tell.
Re:Building the Linux kernel with clang by benmmurphy on Thursday September 21, 2017 @09:05AM (#15302904)
how do you even have VLAs where the array is the non-terminal field? do you embed the size of the array in the type definition or in the structure?
Re:Building the Linux kernel with clang by KenoFischer on Thursday September 21, 2017 @09:38AM (#15303211)
It's about whether you can put stack allocated VLAs into temporary structs:

   keno@anubis:~$ cat test2.c
   void bar(int *);
   int foo(int n) {
       struct {
           int non_terminal_val[n];
           int some_other_member;
       } val;
       bar(val.non_terminal_val);
       return val.some_other_member;
    }
    keno@anubis:~$ gcc -c test2.c
    keno@anubis:~$ clang test2.c
    test2.c:4:13: error: fields must have a constant size: 
    'variable length array in
          structure' extension will never be supported
            int non_terminal_val[n];
                ^
    1 error generated.
Re:Building the Linux kernel with clang by gjem97 on Thursday September 21, 2017 @11:10AM (#15304202)
"will never be supported" I wonder how many times you have to be asked before you put that in the error message.
Re:Building the Linux kernel with clang by scatters on Thursday September 21, 2017 @09:35AM (#15303167)
You're thinking of flexible or zero-length array members (`int[]` as the terminal field, or mostly equivalently, where supported, `int[0]`). VLAs in a struct only really make sense for a local type i.e. a struct defined and used within a function:

    void foo (int n) {
        struct S { int x[n]; };
    }
https://gcc.gnu.org/onlinedocs/gcc/Variable-Length.html https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html
Re:Building the Linux kernel with clang by benmmurphy on Thursday September 21, 2017 @09:44AM (#15303278)
thanks. that makes sense.

    #include <stdio.h>
    void foo (int n) {
      struct S { int x[n]; int y; };

      printf("offset: %ld\n", __builtin_offsetof(struct S, y));
    }

    int main() {
      foo(2);
      foo(16);
    }

    offset: 8
    offset: 64
Re:Building the Linux kernel with clang by wildmusings on Thursday September 21, 2017 @09:34AM (#15303155)
It has to be in the definition since it is a VLA, not a flexible array member. Accessing the VLAs or members after/between them will generate an offset based on whatever values were used to define the length of the VLAs, so the definition needs to be in scope. Seems like the use case is for struct types defined within a function and used only within that same function execution, or perhaps also accessed through another execution of that same function that results in the exact same definition. In the latter case, you would need to cast the memory to void* for storage between executions, and then back to the regenerated type, since you can only reference types within the scope that they are defined.

Weird, but I can see the potential utility when you're optimizing for locality in heap allocations, since the allocator will give you a single contiguous memory block with all of the VLAs, while making several allocations for multiple VLAs could very well return disjoint memory ranges. And then there's the aesthetic utility of behaving like objects in higher-level languages, where having multiple variable-length arrays within an object is standard.

See https://stackoverflow.com/questions/14629504/variable-length... and linked pages.

Re:Building the Linux kernel with clang by wahern on Thursday September 21, 2017 @03:41PM (#15306694)
This would be a really useful feature in C if it was made more general, even if the scope of the definition remained limited. But with GCC 7

  struct { int arr[n]; } s = { 0 };
gives

  error: variable-sized object may not be initialized
And

  (struct { int arr[n]; }){ 0 }
gives

  error: compound literal has variable size
GCC still doesn't even permit a simpler compound literal like (int[n]){ 0 }. I understand that compound literals and VLAs are distinct, and that the above limitations are, strictly speaking, consistent with the standard constructs. But I would think that once you have variable-length struct members that it wouldn't be too difficult to unify the semantics of VLAs and compound literals. If GCC can't be bothered to improve language consistency, then that's pretty strong evidence that VLA struct members are a dead-end and I can't fault clang for avoiding it.

But if GCC did improve consistency, it might significantly help push C toward safer array semantics. The last hurdle would be definition scope and working out how declarations worked. Presumably the ABI would require fat pointers that embed VLA annotations. Getting there would be a long, winding road, but this is how it would happen. It's complex, but the cost+benefit ratio would, I think, be better for both compiler writer and programmer compared to the complexity of C++.

Re:Building the Linux kernel with clang by unwind on Thursday September 21, 2017 @09:34AM (#15303153)
It's weird. As far as I can tell, GCC's documentation[1] clearly states:

Flexible array members may only appear as the last member of a struct that is otherwise non-empty.

What is the article referring to, then?

1: https://gcc.gnu.org/onlinedocs/gcc-7.2.0/gcc/Zero-Length.htm...

Re:Building the Linux kernel with clang by jwilk on Thursday September 21, 2017 @03:10PM (#15306458)
flexible array member ≠ variable-length array

See: https://news.ycombinator.com/item?id=15303211

Re:Building the Linux kernel with clang by ezoe on Thursday September 21, 2017 @02:51PM (#15306329)
The Linux kernel use the variable length array in the structure which is a GCC extension and not part of the C standard.

Clang don't want to support that GCC extension because it's too ugly.

Re:Building the Linux kernel with clang by swills on Thursday September 21, 2017 @10:57AM (#15304055)
Given the number of operating system kernels in the world, it might be helpful to change the title to say "the Linux kernel" instead of "the kernel". I understand TFA is on lwn.net, which has "Linux" in it's title, so it's clear there, but here it becomes a little vague.
Re:Building the Linux kernel with clang by sctb on Thursday September 21, 2017 @02:06PM (#15305979)
OK! We've added “Linux”.
Re:Building the Linux kernel with clang by swills on Thursday September 21, 2017 @09:12PM (#15308689)
Thanks!
Re:Building the Linux kernel with clang by howard941 on Thursday September 21, 2017 @11:09AM (#15304201)
This. FreeBSD's been building on CLANG for amd64 and i386 since before 10.0 (now on 11).

See, for ex., https://wiki.freebsd.org/BuildingFreeBSDWithClang https://unix.stackexchange.com/questions/49906/why-is-freebs...

Re:Building the Linux kernel with clang by notaplumber on Thursday September 21, 2017 @12:49PM (#15305288)
OpenBSD is also now building the kernel/userland and most of the ports tree with Clang for i386/amd64 and arm64. 6.2 will ship with it as the default system compiler.

> Kroah-Hartman said that he wished there was a competitor to the Linux kernel.

Linux people refuse to look outside their vacuum chamber.

Re:Building the Linux kernel with clang by ndesaulniers on Thursday September 21, 2017 @01:09PM (#15305463)
That was not what he said. Article is wrong. He started he wants more competition amongst compiler vendors for the benefit of the kernel.
Re:Building the Linux kernel with clang by kasabali on Sunday September 24, 2017 @01:20PM (#15325571)
> Sorry, but you might have missed my follow-on comment which was, "I wish we had a viable competitor to the Linux kernel as well". Jake correctly quoted me here.

https://lwn.net/Articles/734405/

Re:Building the Linux kernel with clang by jwilk on Thursday September 21, 2017 @03:13PM (#15306487)
Actually, L in LWN no longer stands for Linux. From their FAQ <https://lwn.net/op/FAQ.lwn >:

What does LWN stand for, anyway?

LWN, initially, was "Linux Weekly News." That name has been deemphasized over time as we have moved beyond just the weekly coverage, and as we have looked at the free software community as a whole. We have yet to come up with a better meaning for LWN, however.

Re:Building the Linux kernel with clang by jacobush on Thursday September 21, 2017 @03:16PM (#15306508)
What about Little Wicked Newsletter?
Re:Building the Linux kernel with clang by cup-of-tea on Thursday September 21, 2017 @02:07PM (#15305996)
Oh, come on... Don't tell me you even for a second thought it might have been another kernel.
Re:Building the Linux kernel with clang by Koshkin on Thursday September 21, 2017 @11:05AM (#15304147)
Perhaps, that was in reference to https://www.kernel.org/.
Re:Building the Linux kernel with clang by mac01021 on Thursday September 21, 2017 @12:06PM (#15304860)
I, for one, assumed they meant linux.
Re:Building the Linux kernel with clang by paulddraper on Thursday September 21, 2017 @01:15PM (#15305527)
You may have background knowledge.

It's about a kernel for which compiling with Clang is (1) interesting and (2) novel. If you know Linux, you'll recognize that's Linux.

Re:Building the Linux kernel with clang by wiz21c on Thursday September 21, 2017 @10:00AM (#15303453)
Clang builds the kernel, EFF leaves W3C... What's next ? Amazon rewrites libc ?
Re:Building the Linux kernel with clang by pjmlp on Thursday September 21, 2017 @12:39PM (#15305201)
Every C compiler has a libc.
Re:Building the Linux kernel with clang by wiz21c on Friday September 22, 2017 @02:39AM (#15309877)
I didn't know...

https://libcxx.llvm.org/

But according to what I see on that page, my interrogation remains : once the kernel is built without gcc, then GNU's power will fade away.

GPL and HTML have, in their own ways, enforced that some important code remains open (and sharable and modifiable). By weakening them, the openness will be weakened too.

Of course it won't be 1000 years of darkness, but well, it makes me a bit sad :-)

Re:Building the Linux kernel with clang by pjmlp on Friday September 22, 2017 @04:04AM (#15310214)
Why do you think Apple, Google, Sony, Nintendo and many other embedded OEMs have moved to clang?

It wasn't only due to the better plugin support.

Re:Building the Linux kernel with clang by ndesaulniers on Thursday September 21, 2017 @12:09PM (#15304886)
1000 years of darkness!!!