Re:Gigablast Search Engine, Now Open Source (C/C++)
by runarb on Sunday August 04, 2013 @08:28AM
(#6155429)
Just found a little gem myself. I am working on another open source search engine[0], and needed a way to make bad behaving document filters timeout.
Unfortunately the document filter in questioning dose spawn child processes, so the normal way of using fork() and a monitoring process was not working. However using ulimit like this should work: https://github.com/gigablast/open-source-search-engine/blob/... . Hadn’t thought about spanning a new shell and let it have control like that :)
Re:Gigablast Search Engine, Now Open Source (C/C++)
by conductor on Sunday August 04, 2013 @08:36AM
(#6155448)
There is possible buffer overflow right there (if the HOME directory is long enough). Why don't people use snprintf?
Re:Gigablast Search Engine, Now Open Source (C/C++)
by runarb on Sunday August 04, 2013 @10:59AM
(#6155829)
>Why don't people use snprintf?
Old habits perhaps? When I look back at it I remember that my first books on C were full of problematic sprintf and strcpy use. It may then easy to continue using what you first learned, even when you know better. It basically the "Baby duck syndrome"[0] for C functions.
0: http://en.wikipedia.org/wiki/Imprinting_(psychology)#Baby_du...
EDIT: Gotta say, this has some very useful pieces of code. I'm working on a niche-specific crawler and am battling the url stripping/cleanup part of it. This is very useful: https://github.com/gigablast/open-source-search-engine/blob/...