A Regular Expression Matcher (2007)

https://www.cs.princeton.edu/courses/archive/spr09/cos333/beautiful.html

by ville • 6 years ago

109 22 6 years ago

cs.princeton.edu

adriantam• 6 years ago

This is chapter 1 of "Beautiful Code" (http://shop.oreilly.com/product/9780596510046.do)

drivers99• 6 years ago

I thought I had seen it before. Verified you can actually see the whole chapter in a nicer (book vs web) layout in Amazon's "Look inside" feature since it's at the beginning of the book.

https://www.amazon.com/Beautiful-Code-Leading-Programmers-Pr...

hstaab• 6 years ago

I saw this in the repo of single file language implementations that was posted yesterday. Glad to see it on the front page now.

Here’s the repo if anyone is interested in checking out the others:

https://github.com/marcpaq/b1fipl

leoh2• 6 years ago

This would be a very good technical interview question — i.e. "implement this simple regex specification."

adambyrtek• 6 years ago

I used to use this exact problem years ago when interviewing candidates at Google. At that time I thought it was a great question, but now I'm not that sure anymore if this approach to interviewing is the best way to assess candidates.

leoh2• 6 years ago

Super interesting. Could you say more? Perhaps it depends a bit on the role?

elteto• 6 years ago

We software types must be masochists. It took Rob Pike an hour in the comfort of his private office to write this piece of code. How is this _at all_ like a typical software interview?

dmos62• 6 years ago

You're right. At the same time, could it be interesting to see how someone approaches (not solves) an intimidating problem?

dekhn3• 6 years ago

There's a common question, "convert an RE into an NFA" (https://en.wikipedia.org/wiki/Thompson%27s_construction)

abecedarius2• 6 years ago

Have you seen that used for interviewing? Yes, some people can do it in a 1-hour interview, but wouldn't you rather ask something that can't be done from memory?

glangdale2• 6 years ago

I have seen this one go by, which, from the perspective of someone who has been doing regex implementation for 14 years, more or less, is fairly amusing. It seems unfair, as someone aware of potential pitfalls of backtracking would presumably feel obligated to build a automata-based solution; someone aware also of the potential pitfalls of DFAs (failure to determinize quickly on some nasty patterns) might also feel obligated to build a Thompson or Glushkov NFA solution as well.

It's quite possible you might turn someone competent into a gibbering wreck (they are asking me to implement RE2 or Hyperscan in an hour!) :-)

raverbashing• 6 years ago

Seeing code like this I can understand why the pioneers though C code could be pretty.

But now all I see there is something more akin to a demonstration of ice carving with chainsaws than something that should be used in a production system.

dang• 6 years ago

A thread from 2016: https://news.ycombinator.com/item?id=12199836

2013: https://news.ycombinator.com/item?id=5672875

2011: https://news.ycombinator.com/item?id=2723366

Maybe others?

p.s. these links are just for curious readers; reposts are ok after a year or so—see https://news.ycombinator.com/newsfaq.html.

glangdale2• 6 years ago

These posts are great for history, but regular expression implementation has moved on considerably from early Thompson implementations whether backtracking or NFAs. There is a considerable body of literature about regex implementation, including many quite convincing implementations and alternate formulations, some of which weren't even done by people at, or from, Bell Labs!

We seem to have something of an Eternal September of regex implementation knowledge (abetted by Russ Cox's amazingly selective bibliography in his posts introducing RE2).

jstimpfle2• 6 years ago

Skimming over it, the implementation looks inefficient:

    int matchstar(int c, char *regexp, char *text)
    {
        do {    /* a * matches zero or more instances */
            if (matchhere(regexp, text))
                return 1;
        } while (*text != '\0' && (*text++ == c || c == '.'));
        return 0;
    }

That can be used to match short strings, but not to grep through a filesystem. A good regex matcher has runtime O(N*M) where N is the length of the regex (typically very short) and M is the length of the scanned text.

ufo• 6 years ago

A surprising number of regex tools used in the wild employ a backtracking algorithm like this one. Some popular features such as lookaheada, lookbehinds, and backreferences can't always be matched using a finite automaton.

user2994cb• 6 years ago

An FSA matcher will be O(M) - one state transition per input character.

jstimpfle2• 6 years ago

If you're looking at a specific regex (~FSA), then N is constant... If you have a function that runs an arbitrary regex, then N will vary.

A Regular Expression Matcher (2007)

Related Stories