GNU Bison 3.3 released

http://savannah.gnu.org/forum/forum.php?forum_id=9357

by jrepinc • 7 years ago

73 43 7 years ago

savannah.gnu.org

I always kind of wonder what people use lex and yacc for. 20 years ago I made something to pretty print random C structs, but I have no idea what people do with these tools. Historically, these have been billed as "compiler compilers", and while that's certainly a use case, it never struck me as a very popular use case.

dntf0llow• 7 years ago

I mean, they're basically used whenever you need either a lexer or a parser, or both. So yeah, there is quite a number of possible use cases.

yanowitz• 7 years ago

They are used to build a number of tools you probably rely on or use regularly, including ruby, golang, bash, MySQL and postgresql.

CalChris• 7 years ago

golang uses a handwritten recursive decent parser and handwritten lexer.

https://github.com/golang/gofrontend/blob/master/go/parse.cc https://github.com/golang/gofrontend/blob/master/go/lex.cc

I can't think of a single major tool that uses ANTLR or bison. I say that being a huge ANTLR fan. Parser/lexer generators are really easy to use for DSLs but for real languages and tools, they're too heavyweight and constraining.

gnulinux5• 7 years ago

You use it when you need a parser or a lexer. Even GCC used to use it (they switched to a recursive descent parser a few years ago). Not sure what's surprising here. Maybe you're not familiar parsers?

userbinator• 7 years ago

In the C family of languages all the notable compilers either never used a parser generator and instead simple recursive descent, or moved from using one. It's easier to write, debug, and extend an RD (or table-driver precedence-climbing) parser than work with the (large amounts of) generated code tools like Bison will create.

x12xc• 7 years ago

In general, I don't think so. It's one thing if you parse a well understood grammar like C. But many people who use recursive descent from the start for a new language or DSL just don't bother about the grammar, and when someone takes a look it is suddenly LL(4) and relies on whatever the hand-written parser does.

derefr• 7 years ago

The main advantage, to me, is that the input of Yacc is a (mostly) self-contained, (mostly) declarative representation of the grammar to be parsed. This format is portable between projects.

You can, for example, take Postgres’s SQL syntax parsing “engine” out of Postgres and use it in your own project, because it’s not actually an “engine”, but rather just a Yacc grammar.

wwright2• 7 years ago

How often are yacc/lex grammars actually portable? Truly portable too — usable in different threading contexts, for example.

You say they are declarative. My experience is admittedly very limited, but I would call them anything but declarative. There are declarative elements, but the actual payload of your code is very, very often imperative. (A lot of this is shaped by the API, imo).

I compare this to parser combinators (like parsec) and PEGs (like luapeg and rust-peg). I find these other options are more naturally side-effect free, both internally and externally.

coliveira2• 7 years ago

Yacc is a very important part of UNIX because it allows one to create a parser for a language with limited knowledge of parsing. Of course, if a language is very complicated, it is better to create a special purpose parser, but with yacc one can easily create parsers for most simple DSLs.

chrisseaton2• 7 years ago

Yacc is a very important part of UNIX because it allows one to create a parser for a language with limited knowledge of parsing.

This isn't true. You basically need to know the entire algorithm Yacc uses to use it. What does someone with limited knowledge of parsing do when Yacc tells them this?

    5: reduce/reduce conflict (reduce 3, reduce 4) on c
    state 5
            B : x y .  (3)
            E : x y .  (4)

losvedir• 7 years ago

yecc, the erlang port of yacc, is used by the Elixir parser.

tptacek2• 7 years ago

https://github.com/search?q=yylex&type=Code

UncleEntity• 7 years ago

Open Shading Language (from Sony) uses it to parse the, umm, shading language.

Pixie (the Renderman renderer) uses two or three bison grammars.

I personally like SQLite's lemon parser generator for my grammar poking arounding when I'm not messing with my C++ Earley parser implementation -- which I can never seem to get working properly with empty rules since the papers on the subject are less than clear.

But I'm probably a bit of a strange one in that I'll spend days getting a random grammar working then drop it for some shiny new thing to play with.

cryptonector• 7 years ago

Hand-coding {object code, codecs, parsers, ...} is a disaster. That's why compilers exist. It's also why code generators exist. That's what lex/yacc (flex/bison) are: code generators specialized for generating parsers. They are not compiler-compilers, but tools used in the construction of compilers (and other things that aren't compilers too).

memling• 7 years ago

I would guess that a lot of your cell data are processed by code generators that use the lex/yacc toolchain. While perhaps they're not commonly used in implementations, they have a large number of deployments.

ethagnawl• 7 years ago

I'd assumed VimScript was written using Flex/Bison; turns out it's written directly in C: https://github.com/vim/vim/blob/master/src/eval.c

mruts• 7 years ago

I think a lot of people just use parser combinators which I personally find a lot easier to use and much more powerful. Code as data, data aa code, yadda yadda.

beefhash• 7 years ago

lex and yacc also used by the ircd-hybrid and ircu families of IRCds to parse the configuration files. OpenBSD also uses yacc for configuration files for pf, doas, httpd, etc.

jeffdavis• 7 years ago

SQL parsers, like postgres and mysql.

stabbles• 7 years ago

"There are also many smaller improvements, including a fix for a bug which is at least 31 years old."

benj111• 7 years ago

Its weird that web pages can succumb to link rot so quickly, apps succumb to bit rot almost as quickly, but a bug will live on.

In a thousand years time will archeologists study us through the bugs left behind in Linux 1300.05 and windows (30)95? Do you think there will still be jobs for Cobol programmers?

perlgeek• 7 years ago

but a bug will live on.

Sometimes, yes. But it has also happened often to me that an old bug I was trying to fix was simply gone, either because the error mode has gone away, or the feature that the bug related to has been removed.

gumby• 7 years ago

It will be the "junk DNA" of the source code -- because of workarounds and baked in assumptions it will be impossible to eradicate.

indyz• 7 years ago

Something similar is actually a plot point in Vernor Vinge's "A Fire Upon The Deep".

jimktrains2• 7 years ago

I wish it were easier to have more target languages. (And that there were more target languages.)

I know the docs say that additions are welcome, but I'm still in my infancy of working out how to :)

Congrats though! I love it when these tried-and-true tools continue to perform and improve!

GNU Bison 3.3 released

Related Stories