The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

Scanner generator for win32

Hi, I'm looking for a free scanner generator which would be usable on win32. The native windows version of flex seems to be very old, and it doesn't support a lot of things that later flex versions do. I tried to compile the latest flex source with cygwin, but that binary says "flex: fatal internal error, exec failed" when reading the input file.

So, any ideas if there is some usable and free scanner generator?

Monday, November 27, 2006
Antlr is a good one that does lexical analysis and parsing. I 've used it for C# and Java, not sure how good the generated code for C/C++ is.
Jason Moore Send private email
Monday, November 27, 2006
Oh, I forgot to add. I need to generate C++ code. :)

Monday, November 27, 2006
Try here:

bison 2.1, flex 2.5, both pretty recent I think? If you're already familiar with both you should be good to go.

(I vaguely recall having to add a unistd.h somewhere on the include path due to an erroneous compiled-in skeleton, I just ignored the issue as it is easy enough to work around.)
Monday, November 27, 2006
Tom, latest native win32 version is 2.5.4a and latest source code version is 2.5.33. I'm not going to check it from their cvs now, but I think there is more than 2 years between these versions.

2.5.6 added switch to disable the unistd.h annoyance btw. So what I'd really like to know is if anyone has got a new version of flex to run on win32. Even cygwin compiled would be ok if it didn't crash with internal error.

Monday, November 27, 2006
Have you looked at boost::spirit? Not a generator, but very cool. I haven't used it for anything too complex, but I've used it for parsing HTTP requests and a semi-complex RPN calculator engine, and it's worked great for me.
Tuesday, November 28, 2006
The latest snapshot of MSYS has flex 2.5.33. You can get it from (at the bottom of the page)
I haven't tried it myself though.
Vesselin Atanasov Send private email
Tuesday, November 28, 2006
I tried the MSYS snapshot. There was not any version of Flex included. I'll be looking at Spirit. Looks interesting.

Wednesday, November 29, 2006
This is strange, since I have just tried downloading the MSYS flex snapshot and there is definitely a flex.exe binary there which claims to be version 2.5.33.
Vesselin Atanasov Send private email
Wednesday, November 29, 2006
Actually here is a direct link from the sourceforge download page:
Vesselin Atanasov Send private email
Wednesday, November 29, 2006
Ah, ok. I tried the MSYS .exe installer. I'll give that a try. Thanks.

Wednesday, November 29, 2006
Lemon is free, suspected to work, and used in the SQLite produict whifh is moderately well known and used, which is always a good sign.

From the web page:

Differences With YACC and BISON
Programmers who have previously used the yacc or bison parser generator will notice several important differences between yacc and/or bison and Lemon.

* In yacc and bison, the parser calls the tokenizer. In Lemon, the tokenizer calls the parser.

* Lemon uses no global variables. Yacc and bison use global variables to pass information between the tokenizer and parser.

* Lemon allows multiple parsers to be running simultaneously. Yacc and bison do not.

If you need C++ code, then a trivial wrapper around the generated C code will probably do fine - it's designed to produce legal C++ code, as well as legal C code.

Sunday, December 03, 2006
Lemon is just a parser generator. Since the OP was talking about FLEX, I'm assuming the desire was for a tokenizer generator.

Lemon is, however, pretty cool if you do need a C parser.
Chris Tavares Send private email
Monday, December 04, 2006

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
Powered by FogBugz