Re:Pattern Matching Without Regex – Introducing the Rosie Pa...
by eutropia on Wednesday November 14, 2018 @11:43AM
(#18451206)
Is there anyway to make use of grok on the command line like the post does with rosie patterns? I rather like the idea of `| grep 'net.ipv4'`
Re:Pattern Matching Without Regex – Introducing the Rosie Pa...
by busterarm on Wednesday November 14, 2018 @11:56AM
(#18451334)
I wish. I think the closest that you can get is to run Logstash locally with the stdout output plugin. You'll have to do some work with the logstash config to set up your grok input filters each time, but it should give you what you want otherwise.
Re:Pattern Matching Without Regex – Introducing the Rosie Pa...
by aequitas on Wednesday November 14, 2018 @12:15PM
(#18451529)
Grok is implemented in other languages too. For example https://github.com/garyelephant/pygrok/ It makes developing and unit testing Logstash Grok patterns a lot quicker compared to spinning up a full Logstash instance for every line of code changed.
For example this grok pattern (taken from [1] )
%{TIMESTAMP_ISO8601:timestamp} \[%{IPV4:ip};%{WORD:environment}\] %{LOGLEVEL:log_level} %{GREEDYDATA:message}
refers to a pattern called TIMESTAMP_ISO8601 and calls it "timestamp" in the resulting output data structure.
In logstash, TIMESTAMP_ISO8601 is predefined in a patterns file, such as [2], which is made of up of a mix of regex and other patterns like YEAR, MONTHNUM, etc.
TIMESTAMP_ISO8601 %{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE}?
MONTHNUM is a regex (optional 0 followed by 1-9; or 10 through 12):
MONTHNUM (?:0?[1-9]|1[0-2])
I'm not sure what all of Rosie's base patterns are. This appears to be a valid regex though, from the example: [:alpha:]+ (regex character class and "+" meaning 1 or more). It's C instead of java which is useful in more/different places.
[1] https://www.elastic.co/blog/do-you-grok-grok
[2] https://github.com/logstash-plugins/logstash-patterns-core/b...