Compiler lecture 05
Download as PPTX, PDF0 likes138 views
This document discusses the key concepts of a lexical analyzer in compiler construction. It defines important terms like alphabets, strings, languages, and special symbols. It explains that a lexical analyzer scans source code character by character, using pointers to identify tokens when it encounters whitespace, operators, or other special symbols based on the longest match rule. The analyzer uses input buffering, with begin and forward pointers to track the scanned input and identify lexemes.
Compiler lecture 05
- 1. Compiler Construction By Noor Wali Khan UOCH Continue From Previous Lecture
- 2. Lexical Analyzer • Specifications of Tokens • Let us understand how the language theory undertakes the following terms: • Alphabets • Any finite set of symbols {0,1} is a set of binary alphabets, {0,1,2,3,4,5,6,7,8,9,A,B,C,D,E,F} is a set of Hexadecimal alphabets, {a-z, A-Z} is a set of English language alphabets. • Strings • Any finite sequence of alphabets is called a string. Length of the string is the total number of occurrence of alphabets, e.g., the length of the string Noor is 4 and is denoted by |Noor| = 4. • A string having no alphabets, i.e. a string of zero length is known as an empty string and is denoted by ε (epsilon).
- 3. Lexical Analyzer • Special Symbols • A typical high-level language contains the following symbols:
- 4. Lexical Analyzer • Language • A language is considered as a finite set of strings over some finite set of alphabets. • Computer languages are considered as finite sets, and mathematically set operations can be performed on them. • Finite languages can be described by means of regular expressions. • Longest Match Rule • When the lexical analyzer read the source-code, it scans the code letter by letter; and when it encounters a whitespace, operator symbol, or special symbols, it decides that a word is completed.
- 5. Lexical Analyzer • Input Buffering • The lexical analyzer scans the input from left to right one character at a time. It uses two pointers begin ptr(bp) and forward to keep track of the pointer of the input scanned.
- 6. Lexical Analyzer • Input Buffering • Initially both the pointers point to the first character of the input string as shown below:
- 7. Lexical Analyzer • The forward ptr moves ahead to search for end of lexeme. • As soon as the blank space is encountered, it indicates end of lexeme. • In above example as soon as ptr (fp) encounters a blank space the lexeme “int” is identified. • The fp will be moved ahead at white space, when fp encounters white space, it ignore and moves ahead. then both the begin ptr(bp) and forward ptr(fp) are set at next token.