Language for specifying lexical Analyzer
- 1. The Language for Specifying Lexical Analyzer Myself Archana R Assistant Professor In Department Of Computer Science SACWC. I am here because I love to give presentations.
- 2. Compiler design principles provide an in-depth view of translation and optimization process. Compiler design covers basic translation mechanism and error detection & recovery. It includes lexical, syntax, and semantic analysis as front end, and code generation and optimization as back-end. COMPILER DESIGN
- 3. The Language for Specifying Lexical Analyzer
- 4. A LEX source program is a specification of a lexical analyzer, consisting of a set of regular expressions together with an action for each regular expression. The action is a piece of code which is to be executed whenever a token specified by the corresponding regular expression is recognized. The output of LEX is a lexical analyzer program constructed from the LEX source specification. LEX
- 5. Unlike most programming languages, a source program for LEX does not supply all the details of the intended computation. Rather, LEX itself supplies with its output a program that simulates a finite automaton. This program takes a transition table as data. The transition table is that portion of LEX's output that stems directly from LEX's input. where the lexical analyzer L is the transition table plus the program to simulate an arbitrary finite automaton expressed as a transition table. Only L is to be included in the compiler being built. (cont)
- 6. Input Text Sequence of tokens Lexical Analyzer L LEX Source Lexical Analyzer L LEX Compiler THE ROLE OF LEX
- 7. A LEX source program consists of two parts, a sequence of auxiliary definition followed by a sequence of translation rules. AUXILIARY DEFINITIONS: The auxiliary definitions are statements of the form: D1=R1 D2=R2 ….. Dn=Rn Where each Di is a distinct name, and each Ri is a regular expression whose symbol are chosen from Σ ∪ {D1, D2, Di-1}, i.e., characters or previously defined names. The Di's are shorthand names for regular expressions. Σ Is our input symbol alphabet.
- 8. We can define the class of identifiers for a typical programming language with the sequence of auxiliary definitions. Letter = A|B|…|Z Digit = 0|1|…|9 Identifier = Letter (Letter|Digit)* Example:
- 9. The translation rules of a LEX program are statements of the form:- P1 {A1} P2 {A2} …… Pm {Am} Where each Pi is a regular expression called a pattern, over the alphabet consisting of Σ and the auxiliary definition names. The patterns describe the form of the tokens. Each Ai is a program fragment describing what action the lexical analyzer should take when token Pi is found. The Ai's are written in a conventional programming language, rather than any particular language, we use pseudo language. To create the lexical analyzer L, each of the Ai's must be compiled into machine code. Translation Rules
- 10. The lexical analyzer L created by LEX behaves in the following manner: L read its input, one characters at a time, until it has found the longest prefix of the input which matches one of the regular expressions, Pi. Once L has found that prefix, L removes it from the input and places it in a buffer called TOKEN. (Actually, TOKEN may be a pair of pointers to the beginning and end of the matched string in the input buffer itself.). L then executes the action Ai.
- 11. It is possible, that non of the regular expressions denoting the tokens matches any prefix of the input. In that case, an error has occurred, and L transfers control to some error handling routine. It is also possible that two or more patterns match the same longest prefix of the remaining input. If that is . . . the case, L will break the tie in favor of that token which came first in the list of translation rules.
- 13. BEGIN {return 1} END {return 2} IF {return 3} THEN {return 4} ELSE {return 5} letter(letter|digit)* {LEX VAL:= INSTALL( ); return 6} digit+ {LEX VAL:= INSTALL( ); return 7} < {LEX VAL := 1; return 8} <= {LEX VAL := 2; return 8} = {LEX VAL := 3; return 8} < > {LEX VAL := 4; return 8} > {LEX VAL := 5; return 8} >= {LEX VAL := 6; return 8} TRANSLATION RULES :
- 14. Suppose the lexical analyzer resulting from the above rules is given input BEGIN followed by blank. Both the first and sixth pattern matches BEGIN, and no pattern matches a longer string. Since the pattern for keyword BEGIN precedes the pattern for identifier in the above list, the conflict is resolved in favor of the keyword. For another example, suppose <= are the first two characters read. While pattern < matches the first character, it is not the longest pattern matching a prefix of the input. Thus LEX's strategy that the longest prefix matching a pattern is selected makes it easy to resolve the conflict between < and <=, by choosing <= as the next tokens.
- 15. Lexical analyzer method is used by programs like compilers which can use the parsed data from a programmer’s code to create a compiled binary executable code It is used by web browsers to format and display a web page with the help of parsed data from JavaScript, HTML, CSS A separate lexical analyzer helps you to construct a specialized and potentially more efficient processor for the task Advantages of Lexical analysis