News for hackers. Stuff that matters.
Re:Show HN: Logparser – Alternative to GoAccess Written in P... by jerf on Thursday September 23, 2021 @10:18AM (#28629503)
I am skeptical of those benchmarks. This is written in Python, and, looking at the core loop, yes, it really is Python, not Python wrapped around C or some other acceleration technology. For pure Python to come out appearing to get four times the through put of a C program is pretty dubious. That would have to be one crappy C program. GoAccess looks like it ought to be far enough along that somebody has at least taken a bit of a crack at optimization, but, perhaps not. C ought to be able to smoke pure Python at this task. (Possibly, you know, unsafely, where a crafted referrer may get to arbitrary code execution or something, but still it ought to be way faster.)
Re:Show HN: Logparser – Alternative to GoAccess Written in P... by messe on Thursday September 23, 2021 @11:17AM (#28630404)
> This is written in Python, and, looking at the core loop, yes, it really is Python, not Python wrapped around C or some other acceleration technology

It seems to use a library clfparser to parse apache common log format logs; internally that uses Python's regex engine which is written in C.

6000 line/s seems incredibly slow to me for a C program parsing a log file. I'm seeing a lot or strstr's, strlen's, strdup's, and strchr's in GoAccess's parse.c, all of which are O(n) per line and, while fine in isolation, could be causing GoAccess to do quite a bit more work per line than just using an optimized regex engine.

Re:Show HN: Logparser – Alternative to GoAccess Written in P... by brundolf on Thursday September 23, 2021 @02:44PM (#28633380)
I wonder what percentage of real-world C programs are exponentially slower than they could be because of the str functions
Re:Show HN: Logparser – Alternative to GoAccess Written in P... by jerf on Thursday September 23, 2021 @11:34AM (#28630703)
Thank you. That does sound like something that could resolve my skepticism into concrete facts.