Using flex/bison as replacement for lex/yacc (was Re: LEX's crash!)

[td@almeria.inria.fr (Thierry Despeyroux)]

In article <45738@sophia.inria.fr>, Jean-Michel.Hufflen@loria.fr (Jean-Michel Hufflen) writes:
|> 
|>    Please, does someone know how to enlarge the analysis of right contexts in
|> LEX?  For one of my formalisms, my command "ctmake" crashes with the following
|> message:
|> 
|> 	sed -f FISC.tokens FISC.tks > lexFISC.l
|> 	lex  lexFISC.l
|> 	"lexFISC.l", line 204: (Error) Too many right contexts
|> 	1137/100000 nodes(%e), 2189/100000 positions(%p), 204/100000 (%n), 3701 transitions
|> 	, 106/100000 packed char classes(%k), 519/100000 packed transitions(%a), 0/100000 output slots(%o)
|> 	*** Error code 1
|> 	make: Fatal error: Command failed for target `lexFISC.c'
|> 

I dont know if there is a lex solution to your problem. However for all of you
who get problems using lex and yacc (too many rules for yacc, parser tree to big for lex,
etc...) it is possible (if you have them somewhere) to use flex and bison (some GNU
products). Yacc and bison are compatible. Lex and Flex are almost compatible.
It should be possible to replace only yacc or lex by resp. bison or flex.

To use bison, just put the following in your Buildfile:
------------
YACC=bison -y
(The -y option tells bison to be yacc compatible for file managament)

The file L.c (where L is the name of your formalism) that is generated by the Metal
compiler must be replaced by:

#define _filbuf _flushfilbuf
#include <stdio.h>
extern FILE  *yyin, *yyout;
extern char *yytext, *yyprevious, *yysptr, *yysbuf;

#include "yaccL.c"
main()
{
  if ( yyin == NULL ) {
    yyin = stdin;
  }
  if ( yyout == NULL ) {
    yyout = stdout;
  }
  do {
    BEGIN waitdata;
    yyparse();
    fputs("-7\n", yyout);
    if (yyin != stdin) {
      fclose(yyin);
      yyin = stdin;
      /* The following is needed in case of a syntax error */
/*      yyprevious = '\n'; /* Starting a line */
/*      yysptr = yysbuf;   /* Empties the lex buffer */
      YY_NEW_FILE;
    }
  } while ((yy_current_buffer != NULL) &&
	   ((yy_current_buffer->yy_eof_status) != EOF_DONE));
}

To use flex, put in your Buildfile
-----------
LEX=flex_patched

where flex_patched is a file containing:

#!/bin/sh

PATH=/usr/ucb:/usr/bin/gnu:/usr/bin:/bin

flex -I $*

ed - lex.yy.c <<!
/#define YY_NULL/a
#undef YY_INPUT
#define YY_INPUT(buf,result,max_size) \
{ fflush(yyout); \
	if ( (result = read( fileno(yyin), (char *) buf, max_size )) < 0 ) \
	    YY_FATAL_ERROR( "read() in flex scanner failed" ); \
	    }
.
w
q
!



Lex and flex are almost compatible. The only difference (see the reference manual
for flex) is in the semantics of ^ (begin of line), $ (end of line),
and the use of abbreviations {X}.
Flex must be used with the -I option (incremental mode). There are some options
to play with the size of the automaton and its efficiency. The default seems ok.

The L.tokens file must begin with the two following lines:
\?PARSEFILE?s/\\n/$/
\?<entrypoint>"?s/"/\\n?"/

%p %e etc... are comments (not used) by flex.




Using flex and bison seems to be a good solution and is easy. But I cant tell you if we
will offer better support for using them in the next versions of Centaur.

Thierry

-- 
Send contributions and compliments to centaur@sophia.inria.fr. Registration
and administrative matters should be sent to centaur-request@sophia.inria.fr.
+---------------------------------------------------------------------------+
|     Thierry Despeyroux      | email: Thierry.Despeyroux@sophia.inria.fr   |
| I.N.R.I.A. Sophia-Antipolis | phone: +33 93 65 77 07 fax: +33 93 65 77 66 |
+---------------------------------------------------------------------------+