Syntax analysis
18 likes10,131 views
The document discusses syntax analysis and parsing. It defines a syntax analyzer as creating the syntactic structure of a source program in the form of a parse tree. A syntax analyzer, also called a parser, checks if a program satisfies the rules of a context-free grammar and produces the parse tree if it does, or error messages otherwise. It describes top-down and bottom-up parsing methods and how parsers use grammars to analyze syntax.
Syntax analysis
- 2. Syntax Analyzer Syntax Analyzer creates the syntactic structure of the given source program. This syntactic structure - parse tree. Syntax Analyzer is also known as parser. The syntax analyzer (parser) checks whether a given source program satisfies the rules implied by a context-free grammar or not. If it satisfies, the parser creates the parse tree of that program. Otherwise the parser gives the error messages.
- 3. INTRODUCTION Every programming language has precise rules that prescribe the syntactic structure of well-formed programs. Program is made up of functions, a function out of declarations and statements, a statement out of expressions The syntax of programming language constructs can be specified by context- free grammars A context-free grammar gives a precise syntactic specification of a programming language. the design of the grammar is an initial phase of the design of a compiler. a grammar can be directly converted into a parser by some tools.
- 4. Parser • Parser works on a stream of tokens. • The smallest item is a token. Lexical Analyzer Parser source program token get next token parse tree
- 5. Parsing Parsing is the process of determining whether a string of tokens can be generated by a grammar. Parsing methods The top-down Bottom-up methods. Top-down parsing, construction starts at the root and proceeds to the leaves. Bottom-up parsing, construction starts at the leaves and proceeds towards the root. Top-down parsers are easy to build by hand. Bottom-up parsing, Can handle a larger class of grammars. They are not as easy to build, but tools for generating them directly from a grammar are available. Both top-down and bottom-up parsers scan the input from left to right (one symbol at a time).
- 6. Top- Down Parsing Done by starting with the root, labeled with the starting nonterminal stmt, and repeatedly performing the following two steps. At node N, labeled with nonterminal A, select one of the productions for A and construct children at N for the symbols in the production body. Find the next node at which a subtree is to be constructed, typically the leftmost unexpanded nonterminal of the tree. The current terminal being scanned in the input is frequently referred to as the lookahead symbol.
- 10. Top-Down Parsing Top-Down Parsing is an attempt to find a left-most derivation for an input string Example: S cAd Find a derivation for A ab | a for w cad S S Backtrack S / | / | / | c A d c A d c A d / | a b a
- 11. Predictive Parsing Recursive-descent parsing is a top-down method of syntax analysis in which a set of recursive procedures is used to process the input. Simple form of recursive descent – Predictive Parsing
- 12. Syntax Error Handling Goals in error handling Report the presence of errors clearly and accurately. Recover from each error quickly enough to detect subsequent errors. Add minimal overhead to the processing of correct programs.
- 13. Error-Recovery Strategies The simplest approach is for the parser to quit with an informative error message when it detects the first error. Panic-mode recovery Phrase-level recovery Error-productions Global-correction.
- 14. Panic-Mode Recovery The parser discards input symbols one at a time until one of a designated set of synchronizing tokens is found. The synchronizing tokens are usually delimiters, such as ; or }. Skips a considerable amount of input without checking for additional errors It has the advantage of simplicity, and is guaranteed not to go into an infinite loop.
- 15. Phrase-Level Recovery Perform local correction on the remaining input; It may replace a prefix of the remaining input by some string that allows the parser to continue. A typical local correction is to replace a comma by a semicolon. Delete an extraneous semicolon. Insert a missing semicolon. Disadvantage in coping with situations in which the actual error has occurred before the point of detection.
- 16. Error Productions Expand the grammar for the language at hand with productions that generate the erroneous constructs. The parser can then generate appropriate error diagnostics about the erroneous construct that has been recognized in the input.
- 17. Global Correction Compiler to make as few changes as possible in processing an incorrect input string. Given an incorrect input string x and grammar G, algorithms will find a parse tree for a related string y, such that the number of insertions, deletions, and changes of tokens required to transform x into y is as small as possible. Not implemented.
- 18. Syntax Definition A grammar describes the hierarchical structure of programming language constructs. Eg: if ( expression ) statement else statement An if-else statement is the concatenation of the keyword if, an opening parenthesis, an expression, a closing parenthesis, a statement, the keyword else, and another statement. Stmt -> if ( expr ) stmt else stmt Rule is called a production. In a production, lexical elements if and the parentheses are called terminals. Variables like expr and stmt are called nonterminals.
- 19. A Context Free Grammar A context-free grammar has four components: A set of terminal symbols, sometimes referred to as "tokens.“ A set of nonterminals, sometimes called "syntactic variables." A set of productions, where each production consists of a nonterminal,called the head or left side of the production, an arrow, and a sequence of terminals and/or nonterminals , called the body or right side of the production A designation of one of the nonterminals as the start symbol.
- 20. A Context Free Grammar The terminal symbols are
- 21. Notational Conventions These symbols are terminals: Lowercase letters early in the alphabet, such as a, b, c. Operator symbols such as +, *, and so on. Punctuation symbols such as parentheses, comma, and so on. The digits 0, 1, . . . , 9. Boldface strings such as id or if, each of which represents a single terminal symbol.
- 22. Notational Conventions These symbols are nonterminals: Uppercase letters early in the alphabet, such as A, B, C. The letter s, which, when it appears, is usually the start symbol. Lowercase, italic names such as expr or stmt. Uppercase letters may be used t o represent nonterminals for the constructs. For example, nonterminals for expressions, terms, and factors are often represented by E, T, and F, respectively.
- 23. Notational Conventions Uppercase letters late in the alphabet, such as X, Y, Z, represent grammar symbols; that is, either nonterminals or terminals. Lowercase letters late in the alphabet , chiefly u, v, ... ,z, represent (possibly empty) strings of terminals. Lowercase greek letters,α, β, γ for example, represent (possibly empty) strings of grammar symbols. A set of productions a -> α 1 , a -> α2, ... , a -> α k with a common head A (call them a-productions) , may be written A -> α 1 I α 2 I . , . I α k · call α1 , α2 , ... ,αk the alternatives for A. Unless stated otherwise, the head of the first production is the start symbol
- 25. Derivations E E+E : E+E derives from E E E+E id+E id+id A sequence of replacements of non-terminal symbols is called a derivation of id+id from E. A if there is a production rule A in our grammar and and are arbitrary strings of terminal and non-terminal symbols 1 2 ... n (n derives from 1 or 1 derives n ) : derives in one step : derives in zero or more steps : derives in one or more steps * +
- 26. CFG - Terminology L(G) is the language of G (the language generated by G) which is a set of sentences. A sentence of L(G) is a string of terminal symbols of G. If S is the start symbol of G then is a sentence of L(G) iff S where is a string of terminals of G If G is a context-free grammar, L(G) is a context-free language. Two grammars are equivalent if they produce the same language. S - If contains non-terminals, it is called as a sentential form of G. - If does not contain non-terminals, it is called as a sentence of G. * *
- 27. Derivation Example E -E -(E) -(E+E) -(id+E) -(id+id) OR E -E -(E) -(E+E) -(E+id) -(id+id) At each derivation step, we can choose any of the non-terminal in the sentential form of G for the replacement. If we always choose the left-most non-terminal in each derivation step, this derivation is called as left-most derivation. If we always choose the right-most non-terminal in each derivation step, this derivation is called as right-most derivation.
- 28. Left-Most and Right-Most Derivations Left-Most Derivation E -E -(E) -(E+E) -(id+E) -(id+id) Right-Most Derivation E -E -(E) -(E+E) -(E+id) -(id+id) We will see that the top-down parsers try to find the left-most derivation of the given source program. We will see that the bottom-up parsers try to find the right-most derivation of the given source program in the reverse order. lmlmlmlmlm rmrmrmrmrm
- 29. Parse Trees and Derivations A parse tree is a graphical representation of a derivation that filters out the order in which productions are applied to replace nonterminals. The interior node is labeled with the nonterminal A in the head of the production; The children of the node are labeled, from left to right, by the symbols in the body of the production The leaves of a parse tree are labeled by nonterminals or terminals Read from left to right, constitute a sentential form, called the yield or frontier of the tree. There is a many-to-one relationship between derivations and parse trees.
- 30. Ambiguity a grammar that produces more than one parse tree for some sentence is said to be ambiguous
- 32. Writing a Grammar Grammars are capable of describing most, of the syntax of programming languages . Grammar should be unambiguous. Left-recursion elimination and left factoring - are useful for rewriting grammars . From the resulting grammar we can create top down parsers without backtracking. Such parsers are called predictive parsers or recursive-descent parser
- 33. Eliminating Ambiguity ambiguous grammar can be rewritten to eliminate the ambiguity. stmt -> if expr then stmt |if expr then stmt else stmt |other is ambiguous since the string if E1 then if E2 then S1 else S2 has the two parse trees
- 34. Two parse trees for an ambiguous sentence
- 35. Eliminating Ambiguity The general rule is, "Match each else with the closest unmatched then."
- 36. Left Recursion A grammar is left recursive if it has a non-terminal A such that there is a derivation. A A for some string Top-down parsing techniques cannot handle left-recursive grammars. The left-recursion may appear in a single step of the derivation (immediate left- recursion), or may appear in more than one step of the derivation. *
- 37. Immediate Left-Recursion A A | where does not start with A eliminate immediate left recursion A A’ A’ A’ | A A 1 | ... | A m | 1 | ... | n where 1 ... n do not start with A eliminate immediate left recursion A 1 A’ | ... | n A’ A’ 1 A’ | ... | m A’ | an equivalent grammar In general,
- 38. Left-Recursion -- Problem • A grammar cannot be immediately left-recursive, but it still can be left-recursive. S Aa | b A Sc | d S Aa Sca A Sc Aac causes to a left-recursion
- 39. Eliminate Left-Recursion -- Algorithm - Arrange non-terminals in some order: A1 ... An - for i from 1 to n do { - for j from 1 to i-1 do { replace each production Ai Aj by Ai 1 | ... | k where Aj 1 | ... | k } - eliminate immediate left-recursions among Ai productions }
- 40. Eliminate Left-Recursion S Aa | b A Ac | Sd | f - Order of non-terminals: S, A - A Ac | Aad | bd | f - Eliminate the immediate left-recursion in A A bdA’ | fA’ A’ cA’ | adA’ | So, the resulting equivalent grammar which is not left-recursive is: S Aa | b A bdA’ | fA’ A’ cA’ | adA’ |
- 41. Eliminate Left-Recursion – Example2 S Aa | b A Ac | Sd | f - Order of non-terminals: A, S - Eliminate the immediate left-recursion in A A SdA’ | fA’ A’ cA’ | - Replace S Aa with S SdA’a | fA’a - Eliminate the immediate left-recursion in S S fA’aS’ | bS’ S’ dA’aS’ | So, the resulting equivalent grammar which is not left-recursive is: S fA’aS’ | bS’ S’ dA’aS’ | A SdA’ | fA’ A’ cA’ |
- 42. Left-Recursive Grammars III Here is an example of a (directly) left-recursive grammar: E E + T | T T T * F | F F ( E ) | id This grammar can be re-written as the following non left- recursive grammar: E T E’ E’ + TE’ | є T F T’ T’ * F T’ | є F (E) | id
- 43. Left Factoring Left factoring is a grammar transformation that is useful for producing a grammar suitable for predictive, or top-down, parsing. Stmt -> if expr then stmt else stmt |if expr then stmt A ->α 1 | α 2 So it should be left factored as
- 45. Left-Factoring -- Algorithm For each non-terminal A with two or more alternatives (production rules) with a common non-empty prefix A 1 | ... | n | 1 | ... | m convert it into A A’ | 1 | ... | m A’ 1 | ... | n
- 46. Left-Factoring – Example1 A abB | aB | cdg | cdeB | cdfB A aA’ | cdg | cdeB | cdfB A’ bB | B A aA’ | cdA’’ A’ bB | B A’’ g | eB | fB
- 47. Left-Factoring – Example2 A ad | a | ab | abc | b A aA’ | b A’ d | | b | bc A aA’ | b A’ d | | bA’’ A’’ | c
- 48. Top-Down Parsing The parse tree is created top to bottom. Top-down parser Recursive-descent parsing Backtracking is needed It is a general parsing technique, but not widely used. Not efficient Predictive parsing No backtracking Efficient Needs a special form of grammars - (LL(1) grammars). Recursive predictive parsing is a special form of recursive descent parsing without backtracking. Non-recursive (table driven) predictive parser is also known as LL(1) parser.
- 49. Recursive Predictive Parsing Each non-terminal corresponds to a procedure. Ex: A aBb proc A { - match the current token with a, and move to the next token; - call ‘B’; - match the current token with b, and move to the next token; }
- 50. Recursive Predictive Parsing (cont.) A aBb | bAB proc A { case of the current token { ‘a’: - match the current token with a, and move to the next token; - call ‘B’; - match the current token with b, and move to the next token; ‘b’: - match the current token with b, and move to the next token; - call ‘A’; - call ‘B’; } }
- 51. Top-down parse for id + id * id
- 53. FIRST and FOLLOW FIRST and FOLLOW allow us to choose which production toapply, based on the next input symbol. FIRST(α), where α is any string of grammar symbols, to be the set of terminals that begin strings derived from α. If α => ε, then ε is also in FIRST(α) . A => cY, so c is in FIRST(A) FOLLOW(A) is the set of the terminals which occur immediately after (follow) the non-terminal A in the strings derived from the starting symbol. a terminal a is in FOLLOW(A) if S Aa $ is in FOLLOW(A) if S A * *
- 54. FIRST 1. If X is a terminal, then FIRST(X) = {X}. 2. If X is a nonterminal and X -> YI Y2 ... Yk is a production for some k>=1, then place a in FIRST(X) if for some i, a is in FIRST(Yi), and ε is in all of FIRST(YI), ... ,FIRST(Yi-I); that is , YI Y2 ... Yi-1 => ε. If ε is in FIRST (Yj) for all j = 1, 2,... ,k, then add ε to FIRST (X). For example, everything in FIRST(Y1) is surely in FIRST(X) . If Yi does not derive ε then we add nothing more to FIRST (X) , but if Y1 => ε, then we add FIRST(Y2), and so on. 3. 3. If X => ε is a production, then add ε to FIRST (X). *
- 55. FOLLOW
- 57. LL ( 1 ) Grammars L: scanning the input from left to right L: producing a leftmost derivation 1 : one input symbol of lookahead at each step A grammar G is LL(1) if and only if whenever A -> α | β are two distinct productions of G, the following conditions hold:
- 58. Construction of a predictive parsing table.