Since we're chiming in on possible solutions, why not do the most general thing first? Check memory usage of the system and process right before segfault, check environment variables that govern malloc behavior (i.e. MALLOC_OPTIONS, OS X-specific ones), check if using (s)brk() system call succeeds right before that, check ulimits.

Then you can either litter break points and start stepping or load the core file in gdb, and start looking closer.