News for hackers. Stuff that matters.
Re:Increasing the D Compiler Speed by Over 75% by kevingadd on Thursday July 25, 2013 @02:41PM (#6104136)
The idea of never free()ing and then taking advantage of that with a dumb allocator to get better performance is pretty clever. I wish I could do that with my compiler; sadly I can't let it leak since I invoke it from unit/functional tests so the test runner would run out of memory and explode :( For DMD tests do you just eat the cost of a process setup and compiler startup for every test run?
Re:Increasing the D Compiler Speed by Over 75% by acqq on Thursday July 25, 2013 @03:13PM (#6104347)
I do the related small allocations from the pools, and the pools are destroyed at every compilation end. If the pool element is for example 32K and the average element is e.g. 32 bytes that's 1K times less "real" mallocs. Having separate pools for the members of different "structures" also makes for better CPU cache usage.

Edit: as aaronblohowiak writes, this method is commonly known as a http://en.wikipedia.org/wiki/Region-based_memory_management

Re:Increasing the D Compiler Speed by Over 75% by WalterBright on Thursday July 25, 2013 @02:49PM (#6104200)
The compiler is a batch tool, so restarting the process for every run is normal usage.
Re:Increasing the D Compiler Speed by Over 75% by to3m on Thursday July 25, 2013 @04:19PM (#6104699)
I've taken to not bothering to free anything in batch tools if it's not super-obvious what to do. Objects with a simple life time (including memory owned by a stack-local object such as a std::vector, etc.) get freed; everything else just leaks. You'd think this would cause masses of problems, but it doesn't. It's easy to imagine data that would be too large, but it seems to be rarer in practice than you might think.

If the alternative is something like a handle-based or smart pointer system, then you'll reap the benefits in terms of ease of debugging on a daily basis.

Re:Increasing the D Compiler Speed by Over 75% by aaronblohowiak on Thursday July 25, 2013 @03:18PM (#6104380)
A simple region allocator could do that and then you just pay for one free() at the end of the unit test.
Re:Increasing the D Compiler Speed by Over 75% by p9idf on Thursday July 25, 2013 @10:25PM (#6106240)
> The idea of never free()ing and then taking advantage of that with a dumb allocator to get better performance is pretty clever.

Ken Thompson's C compiler does this.

http://plan9.bell-labs.com/sources/plan9/sys/src/cmd/cc/comp...

http://plan9.bell-labs.com/sources/plan9/sys/src/cmd/cc/lex.... (near the end)

http://plan9.bell-labs.com/sources/plan9/sys/src/cmd/cc/macb... (near the end)

Re:Increasing the D Compiler Speed by Over 75% by gngeal on Thursday July 25, 2013 @05:35PM (#6105156)
The idea of never free()ing and then taking advantage of that with a dumb allocator to get better performance is pretty clever.

Doesn't Erlang do something like this with processes?

What about doing scratch heaps for data with limited lifetime? Especially if you have a way of somehow reasonably predicting how much heap you're likely going to need (perhaps a heuristics obtained with a bit of machine learning?). Allocate a bit more to give yourself some headroom, use a dumb allocator, and if you run out of space, allocate an emergency area. Obviously, in most cases you won't have to do that, though. (If you're obsessed about speed, you can allocate just blindly and use a memory fault handler to detect that you've run out of space. :-) CLISP does that and it seems to work. Look at GNU libsigsegv.)

Re:Increasing the D Compiler Speed by Over 75% by willvarfar on Thursday July 25, 2013 @03:10PM (#6104335)
In the per test tear down you could reset the pointer to the start to reuse the same memory repeatedly?