Lexical analyzer generator lex

V.Anusuya,AP(SG)/CSE
1/21/2020 V.Anusuya,AP(SG)/CSE 1

LEX
 LEX is a tool that allows one to specify a Lexical
Analyzer by specifying RE to describe patterns for
tokens.
 Input Notation-Lex language(Specification)
 Lex Compiler-Transforms Input patterns into a
Transition diagram and generates code in a file called
lex.yy.c

Lexical Analyzer Generator - Lex
Lexical Compiler
Lex Source program
lex.l
lex.yy.c
C
compiler
lex.yy.c a.out
a.outInput stream Sequence
of tokens

Structure of Lex programs
Lex program has the following form:
declarations
%%
translation rules
%%
auxiliary functions
Pattern {Action}

 The declarations section includes declarations of
variables, manifest constants (identifiers declared to
stand for a constant, e.g., the name of a token), and
regular definitions.
 The translation rules each have the form
 Pattern { Action }
 pattern is a regular expression
 Action-Fragment of code written in C.
 Third Section-holds whatever additional functions are used in the
actions.
 Alternatively, these functions can be compiled separately and loaded
with the lexical analyser.

 The lexical analyzer returns a single value, the token
name, to the parser, but uses the shared, integer
variable yylval to pass additional information about
the lexeme found, if needed.

Sample Lex Program:
digit [0-9]
letter [a-zA-Z]
%%
{letter}({letter}|{digit})* printf(“id: %sn”, yytext);
n printf(“new linen”);
%%
main() {
yylex();
}

 Lex Predefined Variables
 yytext -- a string containing the lexeme
 yyleng -- the length of the lexeme
 yyin -- the input stream pointer
 yyout -- the output stream pointer

 Lex Library Routines
 yylex() - The default main() contains a call of yylex()
 yymore() - return the next token
 yyless(n) - retain the first n characters in yytext
 yywarp()-is called whenever Lex reaches an end-of-
file.By default yywarp() always returns 1

Program: LEX Specification
%%
{letter}({letter}|{digit})* printf("id: %sn", yytext);
n printf("new linen");
%%
int main(int argc,char **argv)
{
if (argc > 1)
{
FILE *file;
file = fopen(argv[1],"r");
if(!file)
{
printf("could not open %s n",argv[1]);
exit(0);
}
yyin = file;
}

yylex();
fclose(yyin);
return 0;
}
int yywrap()
{
return 1;
}

Input File
Sam.c
void main()
{
int a=5,b;
char c[]="GOOD";
b=a+10;
printf("%d",b);
}

Output
id: c
[]="id: GOOD
";new line
id: b
=id: a
+10;new line
id: printf
("%id: d
",id: b
);new line
}new line
new line

 Conflict Resolution in Lex
 There are two rules that Lex uses to decide on the
proper lexeme to select, when several prefixes of the
input match one or more patterns:
 1. Always prefer a longer prefix to a shorter prefix.
 2. If the longest possible prefix matches two or more
patterns, prefer the pattern listed first in the Lex
program.

The Lookahead Operator
 Lex automatically reads one character ahead of the last
character that forms the selected lexeme, and then
retracts the input so only the lexeme itself is consumed
from the input.
 However, sometimes, we want a certain pattern to be
matched to the input only when it is followed by a certain
other characters. If so, we may use the slash in a pattern
to indicate the end of the part of the pattern that
matches the lexeme.
 What follows / is additional pattern that must be
matched before we can decide that the token in question
was seen, but what matches this second pattern is not
part of the lexeme.

%{
/* program to recognize a c program */
int COMMENT=0;
%}
identifier [a-zA-Z][a-zA-Z0-9]*
%%
#.* { printf("n%s is a PREPROCESSOR
DIRECTIVE",yytext);}
int | float | char |double |while |for |do |if |break |
continue |void | switch |case |long |struct |const |
typedef | return |else | goto {printf("nt%s is a
KEYWORD",yytext);}
"/*" {COMMENT = 1;}
"*/" {COMMENT = 0;}

{identifier}(
{if(!COMMENT)printf("nnFUNCTIONnt%s",yytext);
}
"{" {if(!COMMENT) printf("n BLOCK BEGINS");}
"}" {if(!COMMENT) printf("n BLOCK ENDS");}
{identifier}([[0-9]*])? {if(!COMMENT) printf("n %s
IDENTIFIER",yytext);}
"a"|"n"|"b"|"t"|"t"|"b"|"a" printf("n%stis
Escape sequences",yytext);
""%d""|"%s"|"%c"|"%f"|"%e" printf("n%stis a format
specifier",yytext);
".*" {if(!COMMENT) printf("nt%s is a
STRING",yytext);}

[0-9]+ {if(!COMMENT) printf("nt%s is a
NUMBER",yytext);}
"=" {if(!COMMENT)printf("nt%s is an ASSIGNMENT
OPERATOR",yytext);}
"+"|"-"|"*"|"/"|"%" {printf("n %s is an arithmetic
operator",yytext);}
"<="|">="|"<"|"=="|">" {if(!COMMENT) printf("nt%s is
a RELATIONAL OPERATOR",yytext);}
%%

int main(int argc,char **argv)
{
if (argc > 1)
{
FILE *file;
file = fopen(argv[1],"r");
if(!file)
{
printf("could not open %s n",argv[1]);
exit(0);
}

yyin = file;
}
yylex();
fclose(yyin);
return 0;
}
int yywrap()
{
return 1;
}

Lexical analyzer generator lex

More Related Content

What's hot (20)

Similar to Lexical analyzer generator lex (20)

More from Anusuya123 (16)

Recently uploaded (20)

Lexical analyzer generator lex