Re:Show HN: Logparser – Alternative to GoAccess Written in P...
by messe on Thursday September 23, 2021 @11:17AM
(#28630404)
> This is written in Python, and, looking at the core loop, yes, it really is Python, not Python wrapped around C or some other acceleration technology
It seems to use a library clfparser to parse apache common log format logs; internally that uses Python's regex engine which is written in C.
6000 line/s seems incredibly slow to me for a C program parsing a log file. I'm seeing a lot or strstr's, strlen's, strdup's, and strchr's in GoAccess's parse.c, all of which are O(n) per line and, while fine in isolation, could be causing GoAccess to do quite a bit more work per line than just using an optimized regex engine.
It seems to use a library clfparser to parse apache common log format logs; internally that uses Python's regex engine which is written in C.
6000 line/s seems incredibly slow to me for a C program parsing a log file. I'm seeing a lot or strstr's, strlen's, strdup's, and strchr's in GoAccess's parse.c, all of which are O(n) per line and, while fine in isolation, could be causing GoAccess to do quite a bit more work per line than just using an optimized regex engine.