A Guide to Lex & Yacc

5. More Lex

5.1 Strings

Quoted strings frequently appear in programming languages. Here is one way to match a string in lex:

char *yylval; #include <string.h>

%} %% \"[^"\n]*["\n] {

yylval = strdup(yytext+1); if (yylval[yyleng-2] != '"')

warning("improperly terminated string"); else

yylval[yyleng-2] = 0; printf("found '%s'\n", yylval); }

The above example ensures that strings don’t cross line boundaries, and removes enclosing quotes. If we wish to add escape sequences, such as \n or \", start states simplify matters:

char buf[100];

char *s;

%x STRING

<STRING>\\n <STRING>\\t <STRING>\\\' <STRING>\"

{ BEGIN STRING; s = buf; }

{ *s++ = '\n'; }

{ *s++ = '\t'; }

{ *s++ = '\"'; }

{

*s = 0;

BEGIN 0;

printf("found '%s'\n", buf); }

{ printf("invalid string"); exit(1); } { *s++ = *yytext; }

<STRING>\n <STRING>.

Exclusive start state STRING is defined in the definition section. When the scanner detects a quote, the BEGIN macro shifts lex into the STRING state. Lex stays in the STRING state, recognizing only patterns that begin with <STRING>, until another BEGIN



5. More Lex

5.1 Strings Quoted strings frequently appear in programming languages. Here is one way to match a string in lex:

%{	*char yylval; #include <string.h>**
	*char yylval; #include <string.h>**
*%} %% \"[^"\n]["\n] { yylval = strdup(yytext+1); if (yylval[yyleng-2] != '"') warning("improperly terminated string"); else yylval[yyleng-2] = 0; printf("found '%s'\n", yylval); } The above example ensures that strings don’t cross line boundaries, and removes enclosing quotes. If we wish to add escape sequences, such as \n or \", start states simplify matters: %{ char buf[100]; char s;* %} %x STRING %%

\" <STRING>\\n <STRING>\\t <STRING>\\\' <STRING>\"		{ BEGIN STRING; s = buf; } *{ s++ = '\n'; } { s++ = '\t'; }* *{ s++ = '\"'; }** { s = 0; BEGIN 0;* printf("found '%s'\n", buf); } *{ printf("invalid string"); exit(1); } { s++ = yytext; }*
<STRING>\n <STRING>.

Exclusive start state STRING is defined in the definition section. When the scanner detects a quote, the BEGIN macro shifts lex into the STRING state. Lex stays in the STRING state, recognizing only patterns that begin with <STRING>, until another BEGIN

32