Context free langauges

1
CS311 Computational Structures
Context-free Languages:
Grammars and Automata
Andrew Black
Andrew Tolmach
Lecture 8

Chomsky hierarchy
In 1957, Noam Chomsky published Syntactic
Structures, an landmark book that deﬁned the so-
called Chomsky hierarchy of languages
2
original name language generated productions:
Type-3 Grammars Regular
Type-2 Grammars Contex-free
Type-1 Grammars Context-sensitive
Type-0 Grammars Recursively-enumerable no restriction
∩
∩
∩
A → γ
αAβ → αγβ
A, B: variables, a, b terminals, α, β sequences of terminals and variables
A → α and A → αB

Regular languages
• Closed under ∪∩∗·and ⎯
• Recognizable by ﬁnite-state automata
• Denoted by Regular Expressions
• Generated by Regular Grammars
3

Context-free Grammars
• More general productions than regular
grammars
S → w

where w is any string of terminals

and non-terminals
4

grammars
S → w


and non-terminals
• What languages do these grammars
generate?
S →(A)
A → ε | aA | ASA
4

grammars
S → w


and non-terminals
• What languages do these grammars
generate?
S →(A)
A → ε | aA | ASA
4
S → ε | aSb

Context-free languages more
general than regular languages
5

• {anbn | n ≥ 0} is not regular
5

‣ but it is context-free
5

• Why are they called “context-free”?
5

‣ Context-sensitive grammars allow more than one
symbol on the lhs of productions
5

‣ Context-sensitive grammars allow more than one
symbol on the lhs of productions
° xAy → x(S)y can only be applied to the non-terminal A
when it is in the context of x and y
5

Context-free grammars are widely
used for programming languages
• From the definition of Algol-60:

procedure_identifier::= identifier.

actual_parameter::= string_literal | expression | array_identifier | switch_identifier | procedure_identifier.

letter_string::= letter | letter_string letter.

parameter_delimiter::= "," | ")" letter_string ":" "(".

actual_parameter_list::= actual_parameter | actual_parameter_list parameter_delimiter actual_parameter.

actual_parameter_part::= empty | "(" actual_parameter_list} ")".

function_designator::= procedure_identifier actual_parameter_part.
• We say: “most programming languages are
context-free”
‣ This isnʼt strictly true
‣ … but we pretend that it is!
6

Example

adding_operator::= "+" | "−" .

multiplying_operator::= "×" | "/" | "÷" .

primary::= unsigned_number | variable | function_designator | "(" arithmetic_expression ")".

factor::= primary | factor | factor power primary.

term::= factor | term multiplying_operator factor.

simple_arithmetic_expression::= term | adding_operator term |
simple_arithmetic_expression adding_operator term.

if_clause::= if Boolean_expression then.

arithmetic_expression::= simple_arithmetic_expression |
if_clause simple_arithmetic_expression else arithmetic_expression.
if a < 0 then U+V else if a * b < 17 then U/V else if k <> y then V/U else 0
7

Example derivation in a Grammar
• Grammar: start symbol is A
A → aAa
A → B
B → bB
B → ε
• Sample Derivation:
A aAa aaAaa aaaAaaa aaaBaaa
aaabBaaa aaabbBaaa aaabbaaa
• Language?
8

9
A
Aa
a
a
b
b
ε
a
a
a
A
A
B
B
B
A
A
a a a b b ε a a a
A
A
B
B
B
Derivations in Tree Form

Arithmetic expressions in a programming
language
• Derive: a + a × a
10
see more easily what this language is if you think of a as a left parenthesis
and b as a right parenthesis ")". =ewed in this way, L(G3)is the language of
strings of properly nested parentheses.
EXAMPLE 2.4 ........................................................................................................................
Consider grammar G4 = (V,S ,R, (EXPR)).
V is {(EXPR),(TERM),(FACTOR)}and E is {a,+, x, (, I}. The rules are
(EXPR)+ (EXPR)+(TERM)1 (TERM)
(TERM)+ (TERM)x (FACTOR)1 (FACTOR)
(FACTOR)+ ( (EXPR)) 1 a
The two strings a+axa and (a+a)xa can be generated with grammar
The parse trees are shown in the following figure.

11
FIGURE 2.5
Parse trees for the strings a+axa and (a+a)xa

11
FIGURE 2.5
Parse trees for the strings a+axa and (a+a)xa
Notice how the grammar gives the meaning a + (a×a)

Grammars in real computing
• CFGʼs are universally used to describe the
syntax of programming languages
• Perfectly suited to describing recursive syntax of
expressions and statements
• Tools like compilers must parse programs; parsers
can be generated automatically from CFGʼs
• Real languages usually have a few non-CF bits
• CFGʼs are also used in XML DTDʼs
12

Formal definition of CFG
• A Context-free grammar is a 4-tuple
(V, Σ, R, S) where
1. V is a finite set called the variables (non-
terminals)
2. Σ is a finite set (disjoint from V) called the
terminals,
3. R is a finite set of rules, where each rule maps
a variable to a string s ∈ (V ∪ Σ)*
4. S ∈ V is the start symbol
13

Deﬁnition of Derivation
• Let u, v and w be strings in (V ∪ Σ)*, and let
A →w be a rule in R,
• then uAv uwv (read: uAv yields uwv)
• We say that u v (read: u derives v) if
u = v or there is a sequence u1, u2, … uk,
k ≥ 0, s.t. u u1 u2 … uk v
• The language of the grammar is

{w ∈ Σ* | S w }
14
*
*

Derivations Parse Trees
• Each derivation can be viewed as a
parse tree with variables at the internal
nodes and terminals at the leaves
• Start symbol is at the root
• Each yield step uAv uwv where w=w1w2...wn
corresponds to a node labeled A with children
w1,w2,...,wn.
• The ﬁnal result in Σ* can be seen by reading
the leaves left-to-right
15

Simple CFG examples
• Find grammars for:
• L = {w ∈ {a,b}* | w begins and ends with the
same symbol}
• L = {w ∈ {a,b}* | w contains an odd number of
aʼs}
• L [(ε + 1)(01)*(ε + 0)]
• Draw example derivations
16

All regular languages have
context free grammars
• Proof:
• Regular language is accepted by an NFA.
• We can generate a regular grammar from the
NFA (Lecture 6, Hein Alg. 11.11)
• Any regular grammar is also a CFG. (Immediate
from the deﬁnition of the grammars).
17

Example
• S→aQ1 S→bR1
• Q1 →aQ1 Q1 →bQ2
• Q2 →aQ1 Q2 →bQ2
• R1 →aR2 R1 →bR1
• R2 →aR2 R2 →bR1
• Q1 →ε R1 →ε
• Resulting grammar may be quite
different from one we designed by hand.
18

Some CFGʼs generate
non-regular languages
• Find grammars for the following
languages
• L = {anbman | a,b ≥ 0}
• L = {w ∈ {a,b}* | w contains equal numbers of
aʼs and bʼs}
• L = {wwR | w ∈ {a,b}*}
• Draw example derivations
19

Ambiguity
• A grammar in which the same string can
be given more than one parse tree is
ambiguous.
• Example: another grammar for
arithmetic expressions
‹EXPR› →
‹EXPR› + ‹EXPR› |

‹EXPR› × ‹EXPR› |

( ‹EXPR› )

| a
• Derive: a + a × a
20

• This grammar is ambiguous: there are
two different parse trees for a + a × a
• Ambiguity is a bad thing if weʼre
interested in the structure of the parse
• Ambiguity doesnʼt matter if weʼre interested
only in deﬁning a language.
21
GURE 2.6
e two parse trees for the string a+axain grammar G5
This grammar doesn't capture the usual precedence relations and s

Leftmost Derivations
• In general, in any step of a derivation, there
might be several variables that can be reduced
by rules of the grammar.
• In a leftmost derivation, we choose to always
reduce the leftmost variable.
• Example: given grammar S → aSb | SS | ε
• A left-most derivation:
S aSb aSSb aaSbSb aabSb aabb
• A non-left-most derivation:
S aSb aSSb aSb aaSbb aabb
22

Ambiguity via
left-most derivations
• Every parse tree corresponds to a
unique left-most derivation
• So if a grammar has more than one left-
most derivation for some string, the
grammar is ambiguous
• Note: merely having two derivations (not
necessarily left-most) for one string is not
enough to show ambiguity
23

Ambiguity
E E ⨉ E E + E ⨉ E
a + E ⨉ E a + a ⨉ E
a + a ⨉ a
E E + E a + E
a + E ⨉ E a + a ⨉ E
a + a ⨉ a
24
GURE 2.6
e two parse trees for the string a+axain grammar G5
This grammar doesn't capture the usual precedence relations and so
up the + before the x or vice versa. In contrast grammar Ga, gen

Context-free languages
• Closed under ∪, ∗ and ·, and under
∩ with a regular language
‣ How do we prove these properties?
• Not closed under intersection,
complement or difference
• Recognizable by pushdown automata
‣ A pushdown automaton is a generalization of a
ﬁnite-state automaton
25

Pushdown Automata
• Why canʼt a FSA
recognize anbn?
‣ “storage” is finite
• How can we fix the
problem?
‣ add unbounded storage
• Whatʼs the simplest kind
of unbounded storage
‣ a pushdown stack
26
finite
state
control
input
accept
reject
stack

History
• PDAs independently invented by
Oettinger [1961] and Schutzenberger
[1963]
• Equivalence between PDAs and CFG
known to Chomsky in 1961; ﬁrst published
by Evey [1963].
27

Executing a PDA
• The behavior of a PDA at any point depends on
⟨state,stack,unread input⟩
• PDA begins in start state with stated symbol on the
stack
• On each step it optionally reads an input character,
pops a stack symbol, and non-deterministically
chooses a new state and optionally pushes one or
more stack symbols
• PDA accepts if it is in a ﬁnal state and there is no
more input.
28

Example PDA
29
he PDA Ati that recognizes {on1" n >01

Example Execution: 0011
30
Stack
Begin in initial state
with empty stack

31
Stack
$

32
Stack
$
0

33
Stack
$
0
0

34
Stack
$
0

35
Stack
$

36
Stack
Accept!

PDAʼs can keep count!
• This PDA can recognize {0n1n | n ≥ 0} by
‣ First, pushing a symbol on the stack for each 0
‣ Then, popping a symbol off the stack for each 1
‣ Accepting iff the stack is empty when the end of input
is reached (and not before)
• The size of the stack is unbounded.
‣ That is, no matter how big the stack grows, it is
always possible to push another symbol on it.
‣ So PDAʼs can use the stack to count arbitrarily high
37

Pushdown Automata (PDA)
• A pushdown automaton    is deﬁned as a
7-tuple:                                     , where:
‣ Q is a set of states,             is the start state
‣ Σ is the input alphabet,
‣   is the stack alphabet,            is the initial stack symbol
‣                                                  is the transition function
‣             is a set of ﬁnal states, and
‣                      , the set X augmented with
38
M
Q
Γ
q0 ∈ Q
F ⊆ Q
εXε = X ∪ {ε}
Z0 ∈ Γ
M = (Q, Σ, Γ, δ, q0, Z0, F)
δ : (Q × Σε × Γε) → P{Q × Γ∗
}

Executing a PDA
• The conﬁguration (or PDA + Hopcroftʼs
instantaneous description, ID) of a PDA
consists of
1. The PDA,
2. The current state of the PDA,
3. the remaining input, and
4. The contents of its stack.
39

• We deﬁned
‣ The transitions                are applicable iff
°     is the current state,
°             , or the next character on the input tape is    , and
°             , or the top of the stack is
‣ If you select a transition             , then
° The new state is
° if              ,      is popped off of the stack, and
° the (possibly empty) sequence of symbols     is pushed
onto the stack
Transitions
40
δ(q, a, γ)
q
a
q
γ
γ
(q , ω)
ω
γ = ε
γ = ε
a = ε
δ : (Q × Aε × Γε) → P{Q × Γ∗
}

PDAs are non-deterministic
For two reasons:
• The transition function δ answers a set
• We have the choice of taking an ε-
transition, or reading an input symbol
41

Trivial Example
Hein Example 12.4
42
Context-Free Languages and Pushdown Automata
Start
pop
!, X
0 f
push(X)
!, Y
s
nop
!, Y !
As the example shows, algorithm (12.5) doesn’t always give th
s. For example, a simpler PDA to accept {"} by final state ca
follows:
Start
• δ(s, ε, Y) = {(0, XY)}
• δ(0, ε, X) = {(0, ε)}
• δ(0, ε, Y) = {(f, Y)}

push(X) ε
ε
ε

Accepting a String
• There are two ways in which a PDA can
accept a string:
1. ﬁnal state is in the set F (implies F non-empty)
2. if F is empty (there are no ﬁnal states), then the
PDA accepts when the stack is empty
• The previous example accepts by …
43

Acceptance by Final State
44
A run of PDA M = (Q, A, Γ, δ, q0, γ0, F) is a sequence
such that:
(q0, γ0)
a0
→ (q1, s1)
a1
→ · · ·
an−1
→ (qn, sn)
for all i ∈ [0 .. n − 1]. (qi+1, γi+1) ∈ δ(qi, ai, γi) and
with q0, . . . , qn ∈ Q, s1, . . . , sn ∈ Γ∗
, and a0, . . . , an−1 ∈ A
si = γiti and si+1 = γi+1ti for some ti ∈ Γ∗
, and
w = a0a1a2 . . . an−1 is the input.
The run accepts if qn ∈ F.w
The language of is given byM, L(M)
L(M) = {w ∈ A∗
| w is accepted by some run of M}

Acceptance by Empty Stack
• There is an alternative deﬁnition of
acceptance:
‣ If the set of ﬁnal states is empty, then
‣ the PDA accepts an input string if it can read all of
the characters of the string and end up in a state
in which the stack is empty.
45

Acceptance by Empty Stack
46
such that:
(q0, γ0)
a0
→ (q1, s1)
a1
→ · · ·
an−1
→ (qn, sn)
for all i ∈ [0 .. n − 1]. (qi+1, γi+1) ∈ δ(qi, ai, γi) and
with q0, . . . , qn ∈ Q, s1, . . . , sn ∈ Γ∗
, and a0, . . . , an−1 ∈ A
si = γiti and si+1 = γi+1ti for some ti ∈ Γ∗
, and
w = a0a1a2 . . . an−1 is the input.
The run accepts ifw
The language of is given byM, L(M)
L(M) = {w ∈ A∗
| w is accepted by some run of M}
A run of PDA M = (Q, A, Γ, δ, q0, γ0, ∅) is a sequence
sn = ε.

It doesnʼt matter!
• PDAs that accept by empty stack and
PDAs that accept by ﬁnal state have
equivalent power
• What does this mean?
‣ That given either one, we can build the other
‣ Consequently, the class of languages that they
accept are the same
47

Example
(Hein 12.2)
means that (in state 0), with input a and
with X on the top of the stack, we can transition
into state 0, and put an additional X onto the
stack.
48
push(X)
ε

Example
(Hein 12.2)
means that (in state 0), with input a and
with X on the top of the stack, we can transition
into state 0, and put an additional X onto the
stack.
48
Empty stack
acceptance
push(X)
ε

49
M = ({0,1}, {a, b},
{X}, δ, 0, X, ∅)
Example
(Hein 12.2)
push(X)
ε

‣ δ(0, a, X) = {(0, XX)}
‣ δ(0, ε, X) = {(1, ε)}
‣ δ(1, b, X) = {(1, ε)}

49
M = ({0,1}, {a, b},
{X}, δ, 0, X, ∅)
Example
(Hein 12.2)
push(X)
ε

Transformation to a Final State PDA
50
ε

50
S
ε

50
SStart
ε

50
SStart
_____
push(Y)
ε

50
FSStart
_____
push(Y)
ε

50
FS
ε, Y
push(X)Start
_____
push(Y)
ε

50
FS
   ε, Y
  push(X)Start
ε, Y
  nop
ε, Y
  nop_____
push(Y)
ε

‣ N = ({0,1, S, F}, {a, b}, {X,Y}, δ´, S, Y, {F})
50
FS
   ε, Y
  push(X)Start
ε, Y
  nop
ε, Y
  nop_____
push(Y)
ε

‣ N = ({0,1, S, F}, {a, b}, {X,Y}, δ´, S, Y, {F})
‣ δ´=δ+δ´(S, ε, Y) = {(0, XY)} +
     δ´(0, ε, Y) = {(F, ε)} + δ´(1, ε, Y) = {(F, ε)}
50
FS
   ε, Y
  push(X)Start
ε, Y
  nop
ε, Y
  nop_____
push(Y)
ε

Transformation to an EmptyStack PDA
51
Start
pop
!, X
0 f
push(X)
!, Y
s
nop
!, Y !
follows:
Start

push(Y)
ε
ε
ε

51
Start
pop
!, X
0 f
push(X)
!, Y
s
nop
!, Y !
follows:
Start
S

push(Y)
ε
ε
ε

51
Start
pop
!, X
0 f
push(X)
!, Y
s
nop
!, Y !
follows:
Start
S
_____
push($)

push(Y)
ε
ε
ε

51
Start
pop
!, X
0 f
push(X)
!, Y
s
nop
!, Y !
follows:
Start
S
_____
push($)
ε
ε
ε

51
Start
pop
!, X
0 f
push(X)
!, Y
s
nop
!, Y !
follows:
Start
S
_____
push($) ε, $
push(Y)
ε
ε
ε

51
Start
pop
!, X
0 f
push(X)
!, Y
s
nop
!, Y !
follows:
Start
S e
_____
push($) ε, $
push(Y)
ε
ε
ε

51
Start
pop
!, X
0 f
push(X)
!, Y
s
nop
!, Y !
follows:
Start
S e
_____
push($)    ε, $
  push(Y)
ε, ε

ε
ε
ε

51
Start
pop
!, X
0 f
push(X)
!, Y
s
nop
!, Y !
follows:
Start
S e
_____
push($)    ε, $
  push(Y)
ε, ε

ε, Y
  pop
ε
ε
ε

51
Start
pop
!, X
0 f
push(X)
!, Y
s
nop
!, Y !
follows:
Start
S e
_____
push($)    ε, $
  push(Y)
ε, ε

ε, Y
  pop
ε, X
  pop
ε
ε
ε

51
Start
pop
!, X
0 f
push(X)
!, Y
s
nop
!, Y !
follows:
Start
S e
_____
push($)    ε, $
  push(Y)
ε, ε

ε, $
  pop
ε, Y
  pop
ε, X
  pop
ε
ε
ε

Other Variations
‣ Hein says: “the execution of a PDA always begins
with one symbol on the stack”
‣ Other authors say “the stack is initially empty”
‣ Does it matter?
52

Other Variations
‣ Does it matter?
52
• Can you always convert an initially empty PDA to
one with an initial symbol on the stack?

Other Variations
‣ Does it matter?
52
• Can you always convert an initially empty PDA to
one with an initial symbol on the stack?
• Can you always convert a PDA with an initial
symbol to one whose stack is initially empty?

Stack Operations
• Hopcroft et al. says:
‣ on each transition, to look at the top of the stack you must pop it, but
‣ you can push on as many stack symbols as you like, including the
one that you just popped.
• Other authors say
‣ you can push zero or one symbols onto the stack
• Hein says:
‣ on each transition, you can push, pop or nop the stack, but you
canʼt do more than one thing
• Does it matter?
53

More Examples
L1= {anbn: n≥0}
54
_____
push($)
a,ε
push(#)
b,#
pop
ε,$
nop
ε, ε
nop

Palindromes of even length
L2= {wwR⎮ w ∈ {a, b}*}
55
_____
push($)
a,ε
push(a)
ε,$
nop
ε, ε
nop
b,ε
push(b)
b,b
pop
a,a
pop

L2= {wwR⎮ w ∈ {a, b}*}
55
_____
push($)
a,ε
push(a)
ε,$
nop
ε, ε
nop
b,ε
push(b)
b,b
pop
a,a
pop
• This machine is non-deterministic. (Why?)

L2= {wwR⎮ w ∈ {a, b}*}
55
_____
push($)
a,ε
push(a)
ε,$
nop
ε, ε
nop
b,ε
push(b)
b,b
pop
a,a
pop
• This machine is non-deterministic. (Why?)
• Can you design a deterministic PDA for L2?

Balanced Parenthesis
56
_____
push($)
ε,$
nop
(,ε
push(#)
),#
pop
L3= {w ∈ {(, )}* : w is a balanced string of parenthesis}

Equal numbers of as and bs
57
_____
push($)
ε,$
nop
b,ε
push(b)
b,a
pop
a,ε
push(a)
a,b
pop

Context free langauges

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Context free langauges (20)

Recently uploaded (20)

Context free langauges