News for hackers. Stuff that matters.
Re:Learn Regex the Easy Way by maddyboo on Thursday August 10, 2017 @04:30AM (#14977600)
My biggest issue with regular expressions is remembering the exact syntax of each of the different common regex engines. Javascript, Perl, `grep`, `grep -E`, vim, `awk`, `sed`, etc...

Each one seems to have slightly different syntax, require different characters to be escaped, has different defaults (is global search enabled by default? Multiline? What about case sensitivity?), some don't support certain lookarounds, how does grouping work, and so on.

Re:Learn Regex the Easy Way by dang on Thursday August 10, 2017 @01:30PM (#14982007)
Completely agree. Emacs too.

It's a real pain, especially when you want a quick one-off regex. In that case the learning curve changes the economics of what the right tool is for the job. Often I'll just end up writing a program using a tool I already know, even though I'm aware that it's a less efficient choice—at least its inefficiency is predictable. Of course if you do that enough times then you've spent more than the original learning curve would have cost! I have done this in painfully many contexts. Regexes are an obvious case, probably because they're so obviously doing the same thing, just differently enough to waste your time.

Some people actually like searching for and paging through documentation to learn how, e.g., regex format X does character escaping. And they tend to remember such things, too. I don't and don't.

Re:Learn Regex the Easy Way by hashhar on Thursday August 10, 2017 @04:03PM (#14983586)
Relevant XCKD: http://www.xkcd.com/1205

This is a very big issue and one I recently faced in the workplace. Needed a script to parse some log files to generate CSV reports for the business users. I knew jq (https://stedolan.github.io/jq/) and hence was able to write less than 10 lines of jq to do it combined with some preprocessing using sed.

I then realised that NOBODY on the team knew jq apart from me and I had to rewrite it in Python which took me 4 days to do correctly and handle everything that jq did for me.

Re:Learn Regex the Easy Way by sglane on Thursday August 10, 2017 @09:27AM (#14979389)
There are three types of regex as far as I know: basic (aka GNU), extended and PERL. Grep uses GNU as the name implies. egrep or grep -E uses extended. PERL is used elsewhere like JavaScript. Typically you'll see pcre which is the library for Perl Compatible Regular Expressions.
Re:Learn Regex the Easy Way by jwilk on Thursday August 10, 2017 @10:24AM (#14979976)
The name "grep" is unrelated to GNU. According to The Jargon File, the etymology is:

from the qed/ed editor idiom g/re/p, where re stands for a regular expression, to Globally search for the Regular Expression and Print the lines containing matches to it

In fact, grep predates the GNU project by almost a decade.

Also, it's the first time I hear about "GNU" regexps.

Re:Learn Regex the Easy Way by sglane on Thursday August 10, 2017 @11:01AM (#14980382)
Thanks, just learned something new. grep predates GNU by about a decade it seems.
Re:Learn Regex the Easy Way by Hello71 on Thursday August 10, 2017 @12:08PM (#14981069)
a more accurate term would be "BRE with GNU extensions", e.g. \(\)
Re:Learn Regex the Easy Way by jwilk on Thursday August 10, 2017 @12:57PM (#14981683)
\(\) are part of BRE syntax. They are not GNU extensions.

Source: http://pubs.opengroup.org/onlinepubs/7908799/xbd/re.html#tag...

Re:Learn Regex the Easy Way by Hello71 on Thursday August 10, 2017 @02:03PM (#14982383)
er, I meant \{\}.
Re:Learn Regex the Easy Way by jwilk on Thursday August 10, 2017 @02:12PM (#14982482)
\{n\}, \{n,\} and \{n,m\} are all in POSIX.

\{,m\} is a (pretty obscure) GNU extensions.

Re:Learn Regex the Easy Way by Hello71 on Thursday August 10, 2017 @05:35PM (#14984541)
hm, I could've sworn it wasn't. http://www.regular-expressions.info/gnu.html says that \?, \+, and \| are GNU extensions though.
Re:Learn Regex the Easy Way by 1wd on Thursday August 10, 2017 @10:01AM (#14979723)
There are many differencese between "compatible" regular expressions, for (many) examples read the end of section "Important Notes About Lookbehind" on [1] or compare 24 dialects on [2] for example.

[1] http://www.regular-expressions.info/lookaround.html [2] http://www.regular-expressions.info/refadv.html

Re:Learn Regex the Easy Way by macintux on Thursday August 10, 2017 @11:07AM (#14980446)
Erlang requires doubling up on escape backslashes, which generally causes me grief even after I flail about and finally remember it. Subtleties abound.
Re:Learn Regex the Easy Way by pjtr on Thursday August 10, 2017 @12:34PM (#14981393)
Another strange one is the (old) Visual Studio syntax with its :i (actually often quite helpful) and $1 instead of \1 (even in the new syntax).

https://msdn.microsoft.com/en-us/library/2k3te2cs(v=vs.110)....

Re:Learn Regex the Easy Way by tzs on Thursday August 10, 2017 @12:14PM (#14981158)
I wonder if there is a tool that lets you enter a regular expression from one engine and get back the equivalent regular expressions for other engines?

Some of the look arounds might be too powerful to do in some engines so it would not always work, but it would still be quite useful if it just handled the differences in escaping, specifying modes like case insensitivity or multiline matching, and referencing match groups in replacements.

Re:Learn Regex the Easy Way by ken on Thursday August 10, 2017 @01:43PM (#14982163)
Not exactly, but that's a(n internal) feature of my Strukt[1]. It's how the optimizer is able to take any of {exact string, glob, ICU regex} as user input, and convert them into efficient queries for various data sources.

For example, if you do ListFiles[folder=~/Downloads] -> FilterString[field=filename, regex=\.pdf$], it will parse the regex, verify that it only uses features which are available in Spotlight's query syntax (roughly, a glob with slightly funny syntax), and rewrite that adjacent pair of operations into a single API call behind the scenes:

    kMDItemFSName LIKE[c] "*.pdf"
I can report, having written parsers for a few common regex dialects, that there are a ton of obscure regex features, with different semantics everywhere. 100% conversion is almost never possible.

A lot of the work in deciding if a translation is possible is in identifying if something is good enough, e.g., SQLite can't do case-insensitive search (in general), but if your regex happens to be /([0-9]+)/ then case-sensitive search will work just fine. Fortunately, for Strukt, if a conversion is impossible, I can just run the operations as written: it's much slower, but still correct.

I've thought about breaking this part of Strukt off into a tool just for regex editing, but that always seemed rather esoteric. Would there be any use or demand for that, do you think?

[1]: https://freerobotcollective.com

Re:Learn Regex the Easy Way by LyndsySimon on Thursday August 10, 2017 @12:22PM (#14981235)
That's an interesting idea - interesting enough that it would be fun to at least throw together a little proof-of-concept...
Re:Learn Regex the Easy Way by masklinn on Thursday August 10, 2017 @06:34AM (#14978299)
Also replacement (sed, emacs, idea, …), is it $1, is it \1, is it something else, does it support named groups?
Re:Learn Regex the Easy Way by crispyambulance on Thursday August 10, 2017 @12:05PM (#14981034)

    >  remembering the exact syntax of each of the different common regex engines
I have the same problem. There's no way around those little differences other than to suffer them. It simply isn't feasible to remember all the nuances of the different flavors.

I just know one really well and rely on getting it wrong for the others: guess, test, revise, repeat.

Re:Learn Regex the Easy Way by Fileformat on Thursday August 10, 2017 @01:44PM (#14982183)
Try http://www.regexplanet.com/ for testing. It supports a bunch of different regex engines & switching between them.
Re:Learn Regex the Easy Way by blackkettle on Thursday August 10, 2017 @03:47PM (#14983425)
My go to resource for this issue is:

* http://www.regular-expressions.info/tools.html

nooks and crannies of pretty much every major flavor and variant, prepared as an easy reference.

Re:Learn Regex the Easy Way by JepZ on Thursday August 10, 2017 @10:54AM (#14980297)
I wanted to write that exact same first sentence (what a weird experience). :D :D

Another thing is escaping: when you write sed regexs in a bash script you have to use slightly different escaping sequences sometimes...

Re:Learn Regex the Easy Way by davidreiss on Thursday August 10, 2017 @09:18AM (#14979305)
That's one of my pet peeves too. Another pet peeve is that I only touch regex once every few months. When you have a regex set to parse a domain of data you move on to other things. After a while you forget the regex syntax and when you have to parse another set of text or have to debug the older regex, I end up having to relearn a bit of it. But I guess it comes with the territory.
Re:Learn Regex the Easy Way by ridiculous_fish on Thursday August 10, 2017 @12:03PM (#14981022)
Python has a nice trick for commenting regexs with re.VERBOSE. I use it for all non-trivial regexs.

https://docs.python.org/2/library/re.html#re.VERBOSE