News for hackers. Stuff that matters.
Re:Pattern Matching Without Regex – Introducing the Rosie Pa... by drivers99 on Wednesday November 14, 2018 @11:35AM (#18451142)
Kind of looks like grok. Grok lets you name patterns, then build up larger patterns by those names, and then also name the groups that it matches to those sub-patterns so you can refer to them in the data. It's built on top of regex, as each pattern can be defined by a mix of other patterns and/or regex.

For example this grok pattern (taken from [1] )

%{TIMESTAMP_ISO8601:timestamp} \[%{IPV4:ip};%{WORD:environment}\] %{LOGLEVEL:log_level} %{GREEDYDATA:message}

refers to a pattern called TIMESTAMP_ISO8601 and calls it "timestamp" in the resulting output data structure.

In logstash, TIMESTAMP_ISO8601 is predefined in a patterns file, such as [2], which is made of up of a mix of regex and other patterns like YEAR, MONTHNUM, etc.

TIMESTAMP_ISO8601 %{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE}?

MONTHNUM is a regex (optional 0 followed by 1-9; or 10 through 12):

MONTHNUM (?:0?[1-9]|1[0-2])

I'm not sure what all of Rosie's base patterns are. This appears to be a valid regex though, from the example: [:alpha:]+ (regex character class and "+" meaning 1 or more). It's C instead of java which is useful in more/different places.

[1] https://www.elastic.co/blog/do-you-grok-grok

[2] https://github.com/logstash-plugins/logstash-patterns-core/b...

Re:Pattern Matching Without Regex – Introducing the Rosie Pa... by eutropia on Wednesday November 14, 2018 @11:43AM (#18451206)
Is there anyway to make use of grok on the command line like the post does with rosie patterns? I rather like the idea of `| grep 'net.ipv4'`
Re:Pattern Matching Without Regex – Introducing the Rosie Pa... by busterarm on Wednesday November 14, 2018 @11:56AM (#18451334)
I wish. I think the closest that you can get is to run Logstash locally with the stdout output plugin. You'll have to do some work with the logstash config to set up your grok input filters each time, but it should give you what you want otherwise.
Re:Pattern Matching Without Regex – Introducing the Rosie Pa... by aequitas on Wednesday November 14, 2018 @12:15PM (#18451529)
Grok is implemented in other languages too. For example https://github.com/garyelephant/pygrok/ It makes developing and unit testing Logstash Grok patterns a lot quicker compared to spinning up a full Logstash instance for every line of code changed.
Re:Pattern Matching Without Regex – Introducing the Rosie Pa... by Twirrim on Wednesday November 14, 2018 @12:00PM (#18451380)
for things like that, I write bash aliases. If you'll excuse the useless use of cat:

    $ cat /etc/hosts | netgrep
    127.0.0.1
    255.255.255.255
    ..
    ..