News for hackers. Stuff that matters.
Re:Doom 3 source released by afhof on Tuesday November 22, 2011 @06:12PM (#3267884)
I was under the impression that most large projects end up having to write a custom memory allocator, but after a few random probes into the source I didn't see anything that stood out. Did I just miss it, or did they use the built in one?
Re:Doom 3 source released by barrybe on Tuesday November 22, 2011 @07:11PM (#3268077)
There's one here:

https://github.com/TTimo/doom3.gpl/blob/master/neo/idlib/Hea...

Surprisingly there's not a lot of platform-specific cruft in that code. The Xbox version of Doom3 must have been a separate code branch.

Re:Doom 3 source released by justinhj on Tuesday November 22, 2011 @08:42PM (#3268364)
the xbox version was done by another activision studio called vicarious visions. no xbox or playstation code can be released due to developer license agreements
Re:Doom 3 source released by mohaps on Wednesday November 23, 2011 @01:17AM (#3269030)
idlib/Heap.h and idlib/Heap.cpp implement a replacement malloc/free.

From the header comments:

/* ===============================================================================

Memory Management

This is a replacement for the compiler heap code (i.e. "C" malloc() and free() calls). On average 2.5-3.0 times faster than MSVC malloc()/free(). Worst case performance is 1.65 times faster and best case > 70 times.

=============================================================================== */

Re:Doom 3 source released by gurkendoktor on Tuesday November 22, 2011 @07:13PM (#3268082)
Quake 2 didn't have its own memory allocator either. Maybe the system allocator(s) got better with the years.

http://fabiensanglard.net/quake2/index.php

Re:Doom 3 source released by klodolph on Tuesday November 22, 2011 @09:19PM (#3268467)
I don't know which Quake 2 you are looking at. Do a code search for Z_TagMalloc and Hunk_Alloc.
Re:Doom 3 source released by gurkendoktor on Wednesday November 23, 2011 @07:04AM (#3269765)
To quote the review (which I've read more than the source):

  The Zone Memory Allocator(common.c: Z_Malloc, Z_Free,
  Z_TagMalloc , Z_FreeTags) is still here in Quake2 but
  it is pretty much useless:
  Tagging is never used and allocation/deallocation is
  build on top of malloc and free (I have no idea why id 
  software decided to trust the C Standard Library at this
  point).
  The overflow detector (using the Z_MAGIC contant) is
  never used either.
So, unless I totally misunderstand things, it is half C stdlib, half Hunk allocator?
Re:Doom 3 source released by ars on Tuesday November 22, 2011 @07:08PM (#3268067)
Why do you have that impression?

That makes very little sense to do - every single project? Don't you think someone would have released a library to implement the "new and improved allocator" if everyone needs to use it?

Re:Doom 3 source released by anaisbetts on Tuesday November 22, 2011 @07:36PM (#3268145)
Most game projects do end up having to write a custom allocator, due to very specific memory budgets, deterministic malloc/free perf requirements, and issues with fragmentation.
Re:Doom 3 source released by vilya on Tuesday November 22, 2011 @07:24PM (#3268117)
That has been borne out by my experience so far too.

There have been several successful efforts at writing better performing general-purpose allocators:

    * TCMalloc (http://goog-perftools.sourceforge.net/doc/tcmalloc.html)
    * jemalloc (http://www.canonware.com/jemalloc/)
There's a great Facebook Engineering blog post about jemalloc which describes why a fast allocator is needed & why the standard malloc is not always sufficient: https://www.facebook.com/notes/facebook-engineering/scalable...
Re:Doom 3 source released by forrestthewoods on Tuesday November 22, 2011 @08:06PM (#3268237)
I've worked with several projects where a TCMalloc based allocator (threaded) was used to great success. A simple switch to DLMalloc (non-threaded) can also provide a significant performance boost.

Moderate usage of STL can do nasty, nasty things to memory performance. In one case a game I worked on ran at 5-10fps and a simple, fast change to DLMalloc instantly took perf to 60fps.

Re:Doom 3 source released by caf on Tuesday November 22, 2011 @09:12PM (#3268438)
It's worth noting that at least on some platforms the system malloc is dlmalloc (presuming that DL is Doug Lea).
Re:Doom 3 source released by iso8859-1 on Wednesday November 23, 2011 @07:01PM (#3272369)
Which platforms?
Re:Doom 3 source released by caf on Thursday November 24, 2011 @03:11AM (#3273613)
glibc's malloc has long been based on Doug Lea's malloc.
Re:Doom 3 source released by vetler on Wednesday November 23, 2011 @02:15AM (#3269153)
+1 for TCMalloc.

We used TCMalloc at Opera Community[0], and they still might be using it there for all I know ... it greatly improved performance on our MySQL servers!

[0]: http://my.opera.com/

Re:Doom 3 source released by Poyeyo on Thursday November 24, 2011 @12:11PM (#3274832)
Aren't you using Percona build ?
Re:Doom 3 source released by vetler on Friday November 25, 2011 @03:10AM (#3276554)
Not when I worked there, but it's a few years ago. Don't know what they use now.
Re:Doom 3 source released by danssig on Wednesday November 23, 2011 @07:59AM (#3269860)
No, the point of a custom allocator is writing one that works with your specific requirements. A general allocator will never be more efficient than a custom one designed for the specific task. It becomes just an issue of is the cost worth what the optimization buys you, and games are an area where you need every drop of speed you can get.