News for hackers. Stuff that matters.
Re:Regex Puzzle by hokkos on Thursday July 06, 2017 @10:21AM (#14710594)
I've worked on the project where some XSD files defined fields with regex restrictions, also some rules over fields added other stricter regexps or negative regexps depending on some context in a format called Schematron. I had to generate XML files conforming to those XSD, so I used some tools around Z3 solver and Microsoft.Automata to generate those strings conforming to multiple regexps. It would convert the regexps to finite automaton and intersecting them, walking it from the starting state to a final one over a charset.

Links :

https://www.microsoft.com/en-us/research/publication/symboli...

https://www.microsoft.com/en-us/download/details.aspx?id=523...

It now seems to be Open Source (MIT):

https://github.com/AutomataDotNet/Automata

Re:Regex Puzzle by eru on Thursday July 06, 2017 @03:56PM (#14713381)
There's also redgrep (https://github.com/google/redgrep) that supports intersection and complements of regular expressions.

I am toying the idea of writing a little game where player A thinks of a regular expression, and player B tries to guess. If B guesses right, they win. If B guesses wrong, A has to provide a false positive and a false negative (if they exist), and B gets to guess again.

Can you think of ways to automate the roles of A and/or B?

Re:Regex Puzzle by long on Thursday July 06, 2017 @06:23PM (#14714439)
In computer science academia, this kind of game is called grammar induction (of which inferring regular expressions is a special case).

A classic algorithm for inferring regular expressions was given by Angluin: https://people.eecs.berkeley.edu/~dawnsong/teaching/s10/pape...

(This isn't quite the same setup as you're thinking of but there are a ton of variations on the basic idea)

Re:Regex Puzzle by eru on Friday July 07, 2017 @11:24AM (#14719058)
Thanks. I had figured out that grammar induction was the right word to look for a while ago. (But took me a bit to find it.) I know the paper you linked to, but yes, it's not quite the right setup.
Re:Regex Puzzle by long on Friday July 07, 2017 @03:43PM (#14721447)
There's a conference on grammar induction called ICGI; might wanna browse through the proceedings to see if there's anything closer.
Re:Regex Puzzle by eru on Friday July 07, 2017 @03:57PM (#14721551)
Thanks!

I'm basically interested in the equivalent of the "guessing game" for regular expressions. (See eg https://stackoverflow.com/questions/5440688/the-guess-the-nu... for a rational number solution.)

With a fixed guesser, that would encode all regular expressions / finite automata as sequences of binary digits. (But in a interestingly different way from just serializing the table for a DFA, or writing down the regular expression in ASCII characters.)

Re:Regex Puzzle by long on Friday July 07, 2017 @04:49PM (#14721965)
So I do AI research on something pretty related to the guessing game -- I'll shoot you an email.
Re:Regex Puzzle by hokkos on Monday July 10, 2017 @01:51PM (#14738163)
I think AutomataDotNet can do all that :

Automation of regular expression generation, it seems easy : use RE fragments and aggregate them, or walk the type hierarchy of the RE AST and generate them randomly.

B needs to guess A's RE so we need to generate examples of strings belonging to it to gives hints : this is exactly the use case of AutomataDotNet.

Also if B guess a RE that is equivalent to A's RE it seems unfair to not attribute a win, so we need to tell if 2 RE belong to the same equivalence class. AutomataDotNet does have a AreEquivalent method.

You can automate the generation of false positive and a false negative with the method Minus to creates an automaton that accepts A-B or B-A and generate an example.

Re:Regex Puzzle by eru on Monday July 10, 2017 @04:18PM (#14739276)
Thanks, I'll have a look.

You are right about the equivalence classes: for that you want to talk about the corresponding DFA (which have a unique normal form in the shape of the minimum DFA).

I am not sure about the rest of what you are saying: in general even just minimizing regular expressions is EXP-SPACE complete, if I remember right.

Yes, generation of false negative and false positive ain't so hard---theory agrees with you. But automating the guesser is, as far as I know.

Re:Regex Puzzle by hokkos on Tuesday July 11, 2017 @09:04AM (#14743791)
I was talking about generating hints to give an opportunity to the RE guesser to find the secret RE. You seem to talk about automating the RE generation based on the pattern of the hints, yeah I didn't thought about that, the other answer seem to talk about it.
Re:Regex Puzzle by jfries on Thursday July 06, 2017 @10:26AM (#14710639)
How did you solve backreferences with that approach?
Re:Regex Puzzle by eru on Thursday July 06, 2017 @10:55AM (#14710865)
You don't. Back reference leave the space of mathematical regular expressions, that all this nice theory works for.
Re:Regex Puzzle by hokkos on Thursday July 06, 2017 @11:53AM (#14711294)
I had the luck that XSD regex doesn't support backreference.