"Lexical Analysis for GATE and CS Exams"

COMPILER DESIGN S.KRISHNAVENI,
LEXICAL ANALYSIS ASSISTANT PROFESSOR
INFORMATION TECHNOLOGY

Lexical Analysis Notes for GATE
• Lexical analysis is an important topic, belonging to the
Computer Science family. And, when you are
preparing for competitive examinations like GATE, you
need to have complete information about this topic.
• What is Compiler?
• What is Lexical Analysis?
• Terminologies
• The architecture of the Lexical analyzer
• Functions of lexical analyzer
• Types of Tokens

• Lexical analysis is the starting phase of the compiler.
• It gathers modified source code that is written in the
form of sentences from the language preprocessor.
• The lexical analyzer is responsible for breaking these
syntaxes into a series of tokens, by removing
whitespace in the source code.
• If the lexical analyzer gets any invalid token, it
generates an error. The stream of character is read by
it and it seeks the legal tokens, and then the data is
passed to the syntax analyzer, when it is asked for.
What is Lexical Analysis?

• Token: It is a sequence of characters that
represents a unit of information in the source
code.
• Pattern: The description used by the token is
known as a pattern.
• Lexeme: A sequence of characters in the source
code, as per the matching pattern of a token, is
known as lexeme. It is also called the instance
of a token.
Terminologies

• To read the input character in the source code and produce
a token is the most important task of a lexical analyzer.
• The lexical analyzer goes through with the entire source
code and identifies each token one by one.
• The scanner is responsible to produce tokens when it is
requested by the parser.
• The lexical analyzer avoids the whitespace and comments
while creating these tokens.
• If any error occurs, the analyzer correlates these errors with
the source file and line number.
The Architecture of Lexical Analyzer

• The lexical analyzer is responsible for removing the
white spaces and comments from the source program.
• It corresponds to the error messages with the source
program.
• It helps to identify the tokens.
• The input characters are read by the lexical analyzer
from the source code.
Roles and Responsibility of Lexical Analyzer

Roles and Responsibility of Lexical Analyzer

• Lexical analysis helps the
browsers to format and
display a web page with
the help of parsed data.
• It is responsible to
create a compiled binary
executable code.
• It helps to create a more
efficient and specialised
processor for the task.
• It requires additional
runtime overhead to
generate the lexer table and
construct the tokens.
• It requires much effort to
debug and develop the lexer
and its token description.
• Much significant time is
required to read the source
code and partition it into
tokens.
Advantages and Disadvantages
of Lexical Analyzer

Advantages and Disadvantages
of Lexical Analyzer
• It will identify lexemes in the given input.
• It will produce tokens as output.
• It will ignore/delete comments & white spaces
• It will produce lexical error if any
• It makes an entry into symbol table for each
identifiers.
• It should scan 1st
character of source code till
last character.
Functions of Lexical Analyzer

1. Which of the following is not a token?
(a) Identifier
(b)Keyword
(c) Operator
(d)None of these
Lexical Analyzer

2. Which of the following is recognized by Lexical
analysis?
(a) Token
(b)Comment
(c) White Spaces
(d)All of these
Lexical Analyzer

3.Which of the following is Lexical error?
(a) a1
(b)1a
(c) a
(d)1
Lexical Analyzer

4. Which of the following statement is lexically
valid?
(a) INT x;
(b)int x;
(c) x int;
(d)All of these
Lexical Analyzer

5. Find number of tokens in the following code.
int x==y;
(a) 6
(b)5
(c) 4
(d)3
Lexical Analyzer

6. Find the number of tokens in the following
code
x+=y++***z
(a) 11
(b)10
(c) 9
(d)8
Lexical Analyzer

code.
int/*comment*/ x;
(a) 4
(b)3
(c) 2
(d)1
Lexical Analyzer

code.
in/*comment*/t x;
(a) 4
(b)3
(c) 2
(d)1
Lexical Analyzer

9. Which of the following is not a single token?
(a) **
(b)++
(c) <=
(d)All of these
Lexical Analyzer

9. Which of the following is a single token?
(a) &&
(b)++
(c) +=
(d)All of these
Lexical Analyzer

10. Which of the following is not a single token?
(a) ==
(b)+=
(c) =+
(d)All of these
Lexical Analyzer

11. Which of the following can be used to
recognize the tokens in lexical analysis?
(a) Finite Automata
(b)Push down Automata
(c) Turing Machine
(d)All of these
Lexical Analyzer

12. Which of the following phase uses symbol
table in compiler?
(a) Lexical Analysis
(b)Syntax Analysis
(c) Semantic Analysis
(d)All of these.
Lexical Analyzer

"Lexical Analysis for GATE and CS Exams"

More Related Content

Similar to "Lexical Analysis for GATE and CS Exams" (20)

Recently uploaded (20)

"Lexical Analysis for GATE and CS Exams"