News for hackers. Stuff that matters.
Re:Clang vs GCC benchmark on UnrealEngine3 by tenfingers on Tuesday April 22, 2014 @06:44AM (#7626840)
gcc at -O0 -g has always been decent enough for me, and I never saw any significant difference with clang (either in the past when gcc -O0 used to be /slighly/ slower, or now).

Things change with -Ox, but then again, who really cares? Generating optimized binaries is not something I do routinely, except when tuning for performance.

When I'm generating a binary for release, I couldn't care less if the project actually took 2x times to compile. In fact, to use feedback-directed optimization (which on some projects may make a marked difference, usually larger then the generated binary at O3 between gcc/clang), you actually have to build it twice, and _very_ few people actually bother.

clang used to be much faster at building, but then again the binary was never equally on-par performance wise. As clang is getting better at optimizing, the compilation times are getting more and more similar.

This is again, all hardly surprising for me.

I _have_ worked on projects using C++ where build times where incredibly slow. They all shared the common attribute of #include <megaheader_with_all_class_definitions> and "using namespace library".

For other projects, as long as rebuilding an object takes 2 seconds or 10, does it really make a difference? In those cases, linking actually takes disproportionately longer, to the point that compilation times are insignificant.

It's a genuine question.

Re:Clang vs GCC benchmark on UnrealEngine3 by cLeEOGPw on Tuesday April 22, 2014 @07:04AM (#7626899)
2x release build compile time might not be a huge deal to you personally, but over thousands of builds that are happening every day, a 2x decrease in time would mean a lot of saved processing power and electricity. Speed is always important in tools like compiler, so there is never a legitimate excuse for it to be slower than it could be.
Re:Clang vs GCC benchmark on UnrealEngine3 by tenfingers on Tuesday April 22, 2014 @08:35AM (#7627231)
Cannot deny that. And it's undoubtedly helpful also for continuous integration. Poor performance is never a good excuse.

But my perception reading these articles is that the tone is different. There's seems to be a "huge problem with compilation speed in c++", that's also "a detrimental factor to using c++ itself" that I _never_ really experienced, except, like I said, in badly-organized projects.

It always felt to me, by programming, that writing and debugging were _always_ the problem, never compilation itself. By reading these articles, it feels like I'm either too slow, or there is some magic that I would need to be doing to use the compiler 20x more than what I'm doing now.

Re:Clang vs GCC benchmark on UnrealEngine3 by rwbt on Tuesday April 22, 2014 @08:49AM (#7627287)
Usually, when compiling projects that use Boost libraries which are sometimes very template heavy, the compile times are very long in my experience. Especially, something like the Boost Spirit.
Re:Clang vs GCC benchmark on UnrealEngine3 by Narishma on Tuesday April 22, 2014 @08:34AM (#7627228)
Your projects aren't video games, are they? It's not very feasible to debug games at -O0, unless they're very small. And I don't how if profile guided optimization is feasible either.
Re:Clang vs GCC benchmark on UnrealEngine3 by endianswap on Tuesday April 22, 2014 @11:19AM (#7628192)
I've worked on several AAA games and have never had a problem running/testing/debugging with optimizations off. There's a framerate hit, of course, but I've never had a problem debugging because of it. Trying to debug with optimizations on is usually a bigger time sink than just debugging at 20fps. When I'm dealing with gameplay code and not engine code I'll normally just build with all optimizations on except -O0 the gameplay modules so I get full framerate and perfect debugging of the code I'm interested in. But usually that's not necessary (except it gives me faster load/restart times.)
Re:Clang vs GCC benchmark on UnrealEngine3 by tenfingers on Tuesday April 22, 2014 @08:39AM (#7627247)
I've worked in large projects where the build of the entire project (c++ of course) took around 75 minutes, and performance was important (real time visualization of volumetric data).

It happened to me _very_ rarely to change a global header that would cause a massive rebuild. Most of the time I would rebuild just the current object I was working on, taking anywhere from 10 to 60 seconds.

Yet, linking took massively longer (around 4 minutes as I recall).

Re:Clang vs GCC benchmark on UnrealEngine3 by bananas on Tuesday April 22, 2014 @08:00AM (#7627108)
Either way the flat compile time is not important to me even on projects of this size. Most of it is probably going to be linking the .o files on dev builds which is considerably quicker than a full compile cycle.

For CI builds ~ 10 mins is nothing (we're currently hitting 3 hours and managing fine with a rats nest of csc and cl compilers being invoked galore).

What really brings the shit to the table with clang is simply this:

Compiler error messages are beyond awesome as is checker.

That is all. Productivity is way up for me. I do my own clang builds of stuff because it's a lot more helpful as a compiler.

Re:Clang vs GCC benchmark on UnrealEngine3 by davidgerard on Tuesday April 22, 2014 @06:24AM (#7626780)
So how did the resulting binaries compare for performance? Compilation happens once, execution many times.
Re:Clang vs GCC benchmark on UnrealEngine3 by shoo on Tuesday April 22, 2014 @06:40AM (#7626825)
except during iterative development, say, where maybe you execute once for test between each edit
Re:Clang vs GCC benchmark on UnrealEngine3 by thesz on Tuesday April 22, 2014 @06:46AM (#7626847)
You build Unreal Engine from scratch everytime after each edit?

I think not.

The difference will be between 1s and 2s, given overall difference in speed. I suspect that your time to think about problems will two orders of magnitude more.

Mine is - minutes between builds.

Re:Clang vs GCC benchmark on UnrealEngine3 by rockdoe on Tuesday April 22, 2014 @06:21AM (#7626770)
So GCC takes longer to compile but produces a better optimized binary? That makes the comparison pretty useless as you can probably just reduce the GCC optimization level.
Re:Clang vs GCC benchmark on UnrealEngine3 by matthewmacleod on Tuesday April 22, 2014 @06:49AM (#7626854)
It produces a smaller binary, but that doesn't really give us any information on whether it's a better one.

AFAIK Clang and GCC are now roughly neck-and-neck in terms of performance though (at least as of December) - http://www.phoronix.com/scan.php?page=article&item=llvm34_gc...

Re:Clang vs GCC benchmark on UnrealEngine3 by cottonseed on Tuesday April 22, 2014 @06:54AM (#7626874)
GCC produces a slightly smaller binary when debug information is stripped, but LLVM produces a massively smaller binary (half as big) when debug information is included. If you're building debug, why would you care how big the stripped binary is?
Re:Clang vs GCC benchmark on UnrealEngine3 by userbinator on Tuesday April 22, 2014 @07:29AM (#7626987)
This invites the question of whether GCC is outputting more detailed debugging information than LLVM (which is a good thing), or if it's just being inefficient at it.
Re:Clang vs GCC benchmark on UnrealEngine3 by cottonseed on Tuesday April 22, 2014 @07:52AM (#7627074)
The g++/gdb experience for C++ has always been pretty bad. Anyone know how lldb compares?
Re:Clang vs GCC benchmark on UnrealEngine3 by dmm on Tuesday April 22, 2014 @08:11AM (#7627144)
In what ways is it bad? It works pretty damn good here.
Re:Clang vs GCC benchmark on UnrealEngine3 by cottonseed on Tuesday April 22, 2014 @08:49AM (#7627289)
Generally, failing to parse expressions at the level of abstraction of the source code being debugged: I find it fails on user defined operators, can't resolve overloaded functions, etc.

Even being able to reliably evaluate C++ expressions is far from what I want: I want to be able to execute arbitrary C++ statements (including loops), use templated objects that weren't instantiated at compile time, etc. I feel like this might be close to possible using, say, the LLVM JIT, although I haven't seen anything that implies anyone is trying to build it.

edit: for example (stock gcc/gdb on Ubuntu 12.04):

    #include <iostream>
    int main() { std::cout << "foo\n"; }
in gdb, break at main:

    (gdb) p std::cout << "foo\n"
    Cannot resolve function operator<< to any overloaded instance
edit edit: lldb doesn't fare much better (OSX 10.9):

    (lldb) p std::cout << "foo\n"
    error: no member named 'cout' in namespace 'std'
    error: 1 errors parsing expression
Re:Clang vs GCC benchmark on UnrealEngine3 by emaste on Tuesday April 22, 2014 @09:18AM (#7627437)
LLDB does JIT your expression via Clang, and then either interprets it locally or executes it in the target, and you can execute arbitrary commands - for example:

  (lldb) expr for (int i=1; i<5; i++) { printf("%d\n", i * i); }
  1
  4
  9
  16
It does still fail if the appropriate constructors, templated object instantiations, etc. aren't in the target, as you point out.
Re:Clang vs GCC benchmark on UnrealEngine3 by cottonseed on Tuesday April 22, 2014 @09:44AM (#7627564)
Oh, very nice. That's clearly the right architecture.

It still fails on a lot of things for me (including your above example and the std::cout example, although variants work), which I'm guessing (hoping?) is just a lack of maturity. Looking forward to see how it progresses.

Re:Clang vs GCC benchmark on UnrealEngine3 by celebril on Tuesday April 22, 2014 @08:37AM (#7627241)
It's bad because it's not as hip and cool as LLVM's offering.
Re:Clang vs GCC benchmark on UnrealEngine3 by mitchty on Tuesday April 22, 2014 @09:18AM (#7627438)
No gdb is bad for c++ debugging because it doesn't include parsing support for c++.

At least I'm not aware of being able to type in c/c++ in gdb on the fly like you can with lldb contexts.

So what exactly is "hip and cool" about having your debugger actually use the same parsing engine as the compiler infrastructure? If you have a technical point make it, otherwise stop being the discussion equivalent of acid.

Re:Clang vs GCC benchmark on UnrealEngine3 by x3ro on Tuesday April 22, 2014 @07:31AM (#7627000)
In the comments there is a run with -O0 where GCC still takes almost 50% more time than clang. Is it okay to assume that unoptimizied binaries would be quite similar in terms of performance?
Re:Clang vs GCC benchmark on UnrealEngine3 by JupiterMoon on Tuesday April 22, 2014 @12:45PM (#7628724)
According to this: http://gcc.gnu.org/ml/gcc/2014-04/msg00195.html these benchmarks are already out of date...
Re:Clang vs GCC benchmark on UnrealEngine3 by cpeterso on Tuesday April 22, 2014 @12:15PM (#7628539)
Building Firefox for OS X using clang on my MacBook Pro takes about 8 minutes. Cross-compiling Firefox for Android using gcc 4.7 on the same MacBook Pro takes about 45 minutes. :(
Re:Clang vs GCC benchmark on UnrealEngine3 by azakai on Tuesday April 22, 2014 @12:54PM (#7628787)
But that's not apples-to-apples. How about building for OS X using gcc?
Re:Clang vs GCC benchmark on UnrealEngine3 by cpeterso on Wednesday April 23, 2014 @12:17AM (#7632067)
Would you expect building for OS X using gcc to be much faster than building for Android? The KLOCs compiled for OS X and Android should be about the same, though the Android build is emitting ARM code and compiling some Java files (but that only takes 1–2 minutes).
Re:Clang vs GCC benchmark on UnrealEngine3 by azakai on Wednesday April 23, 2014 @12:34PM (#7635082)
Very possible the ARM and x86/x86-64 backends are very different in compilation speed, for example.
Re:Clang vs GCC benchmark on UnrealEngine3 by Steltek on Tuesday April 22, 2014 @08:16AM (#7627166)
If gcc is producing a final binary that's twice the size of clang's, wouldn't that point to IO being a big factor in compile time?

What is this test really trying to determine? How the performance of the core of each compiler impacts productivity? Shouldn't the test parameters be adjusted to real world conditions, as seen by the test creator, then?

Like most benchmarks, you could fabricate scenarios that favor one or the other all day long. Some closer to reality, some completely synthetic.

Re:Clang vs GCC benchmark on UnrealEngine3 by masklinn on Tuesday April 22, 2014 @08:23AM (#7627185)
> If gcc is producing a final binary that's twice the size of clang's, wouldn't that point to IO being a big factor in compile time?

250MB of output adding 5mn of compile time would mean 800KiB/s. That's about what a 5400RPM HDD gets on random writes. Possible, but means GCC has an utterly terrible I/O architecture and flushes to disk after more or less every write call.

> What is this test really trying to determine?

Nothing? It's just trying to infirm or confirm the initial off-hand remark: https://twitter.com/icculus/status/448281959538888704

> Like most benchmarks, you could fabricate scenarios that favor one or the other all day long. Some closer to reality, some completely synthetic.

I'm not sure how you can get closer to reality than "I spend my days compiling Goat Simulator with Clang and GCC, let's compile Goat Simulator with Clang and GCC".

Re:Clang vs GCC benchmark on UnrealEngine3 by azakai on Tuesday April 22, 2014 @10:50AM (#7627975)
> If gcc is producing a final binary that's twice the size of clang's, wouldn't that point to IO being a big factor in compile time?

Very possibly IO, yes.

And it is twice as big for a debug build, whereas a non-debug build is smaller in gcc - 69 MB vs 74 MB in clang.

Perhaps more importantly, not all debug builds are equal, so the comparison is not apples-to-apples without saying whether the gcc debug builds contain more useful information. gcc also has various levels of debug info, perhaps a different one would provide less info for a smaller binary that would be more of a parallel to clang.

Re:Clang vs GCC benchmark on UnrealEngine3 by mschuster91 on Tuesday April 22, 2014 @06:14AM (#7626750)
Seriously, a 70MB binary?

Is this all program code or does this include resources?

Re:Clang vs GCC benchmark on UnrealEngine3 by Cthulhu_ on Tuesday April 22, 2014 @06:44AM (#7626841)
Well, judging by the title, it's the Unreal 3 engine; I'd say that would end up being a pretty big binary.
Re:Clang vs GCC benchmark on UnrealEngine3 by mschuster91 on Tuesday April 22, 2014 @07:00AM (#7626891)
The problem is performance: a small binary might end up having lots of its code in L1 (16KByte) / L2 (512 KByte) cache (dec2013 values for i7, taken from https://www.scss.tcd.ie/Jeremy.Jones/CS3021/5%20caches.pdf)... but a 70mb binary should produce a lot of cache misses.

I wonder what performance penalties arise from that.

Re:Clang vs GCC benchmark on UnrealEngine3 by w0utert on Tuesday April 22, 2014 @07:17AM (#7626946)
I assume the 70MB binary statically links in all kinds of assets that blow up the file size, static arrays of pre-calculated data etc. There's no way it compiles to 70MB of assembly.

The code size would still be massively bigger than the CPU code cache, but the working set that actually matters for performance does not have to have any relation to the size of the total code itself, as long as time-critical sections are small and localized.

Re:Clang vs GCC benchmark on UnrealEngine3 by sharpneli on Tuesday April 22, 2014 @07:32AM (#7627002)
Considering the massive size increase due to debug symbols I'd guess that a sizable part of the 70MB indeed is assembly, at least 20MB and probably even more.

C++ templates, while being absolutely lovely to write and use, tend to do that. Every single different type you instantiate the template into produces a separate compiled copy of the whole template. Write a sorting function and use it for 5 types -> you get the same sort in assembler 5 times with differences that relate to the types they use, alongside with 5 massive symbol names (C++ template symbols tend to be really large).

Re:Clang vs GCC benchmark on UnrealEngine3 by exDM69 on Tuesday April 22, 2014 @08:19AM (#7627170)
> Write a sorting function and use it for 5 types -> you get the same sort in assembler 5 times with differences that relate to the types they use, alongside with 5 massive symbol names (C++ template symbols tend to be really large).

To be fair, this is also why the performance of C++ std::sort beats C qsort hands down. Calling the comparator function with a function pointer is expensive compared to doing an integer comparison. Additionally, the boundary between translation units (ie. object files) inhibits a lot of optimization that could otherwise take place (and does in the C++ case, when the function calls are inlined).

There are also techniques to avoid C++ template bloat, such as type erasure. Although this makes the template code only a thin shim that provides type safety.

Re:Clang vs GCC benchmark on UnrealEngine3 by sanxiyn on Tuesday April 22, 2014 @09:18AM (#7627433)
If functions really are same in assembly (e.g. type sizes are same) LLVM's mergefunc pass actually can merge them, leaving only a single copy.

http://llvm.org/docs/Passes.html

Re:Clang vs GCC benchmark on UnrealEngine3 by Negitivefrags on Tuesday April 22, 2014 @08:39AM (#7627254)
GCC seems to produce fairly fat binaries. Compared to what I'm used to seeing with MSVC at least.

The code for our linux game server compiles to an 11MB stripped binary with 608 megs of debug data.

The code for the windows game client contains the majority of that same code, but in addition also contains the entire renderer, UI, and a pile of other client side garbage and comes in at only 7MB with 86MB of debug data.

The 70MB for UE which would presumably be a much larger code base seems reasonable to me in that context.