Lex Tool
Download as PPTX, PDF1 like1,694 views
This document discusses language processing development tools (LPDTs) that focus on generating the analysis phase of language processors. It specifically describes Lexical Analyzers and the LEX tool. LEX accepts an input specification with regular expressions that define the tokens/terminals of a language. It then generates a lexical analyzer that tokenizes the input stream. The analyzer identifies lexical units, classifies them, and builds descriptor tokens containing a class code and number. It also performs semantic actions like entering identifiers and numbers into symbol tables.
![LEX Program
%{
letter [A-Za-z]
digit [0-9]
}%
This component defines the symbol used in
specifying the string of L. It defines the symbol
‘letter’ to stand for any upper/lower case letter.](https://image.slidesharecdn.com/lex-170901152236/85/Lex-Tool-6-320.jpg)
Lex Tool
- 1. Sardar VallabhBhai Patel Institute Of Technology Name: RAJAN SHAH Topic: LEX TOOL
- 2. Language Processor Development Tools The analysis phase of language processor has a std form irrespective of its purpose, the source text is subjected to lexical, syntax, semantic analysis and the results of analysis are represented in an IR (Intermediate Representation). LPDT’s focus on generation of the analysis phase of language processors.
- 3. LEXICAL ANALYZER Lexical analysis identifies the lexical units in the source statement. It then classifies them into different lexical classes, e.g. id’s, constants etc. and enter them into different tables. Lexical analysis builds a descriptor, called a TOKEN, for each lexical unit. A Token contains two fields: class code, and number in class. Token is depicted as : Code#no, e.g. Id#2.
- 4. LEX Lexical analyzers tokenize input streams Tokens are the terminals of a language English: words, punctuation marks, … Programming language: Identifiers, operators, keywords, … Regular expressions define terminals/tokens
- 5. Continue... LEX accepts an i/p specification which consists of following two part: 1. Specification of strings representing the lexical units in L, e.g. Id’s and constants. 2. Specification of semantic actions aimed at building an IR. The semantic actions makes new entries in the tables and build tokens for the lexical units.
- 6. LEX Program %{ letter [A-Za-z] digit [0-9] }% This component defines the symbol used in specifying the string of L. It defines the symbol ‘letter’ to stand for any upper/lower case letter.
- 7. Continue... %% begin {return (BEGIN); } end {return (END); } “:=” {return (ASGOP); } {letter} ({letter} ¦ {digit}) * {yylval=enter_id(); return (ID); } {digit}+ {yylval=enter_num(); return (num); } %%
- 8. Continue... enter_id() { /* enters the id in the symbol table and returns the entry number. */ } enter_num() { /* enters the number in the constants table and returns entry number */ }
- 9. Overview of LEX Lexical Analyzer lex.yy.c lex.yy.c a.out Input tokens Lex C compiler a.out
- 10. LEX with YACC
- 11. THANKY OU