![25
The Diagonalization Language
■Ld = { wi | wi L(M
∉ i) }
■The language of all strings whose corresponding
machine does not accept itself (i.e., its own code)
1 2 3 4 …
1 0 1 0 1 …
2 1 1 0 0 …
3 0 1 0 1 …
4 1 0 0 1 …
i
j
…
. .
.
diagona
• Table: T[i,j] = 1, if Mi accepts wj
= 0, otherwise.
(input word
w)
(TMs
)
• Make a new language
called
Ld = {wi | T[i,i] = 0}](https://image.slidesharecdn.com/undecidability-250322034336-03fe07b9/85/Undecidability-of-Turing-Machine-in-theory-of-Computation-25-320.jpg)
![26
Ld is not RE (i.e., has no TM)
■ Proof (by contradiction):
■ Let M be the TM for Ld
■ ==> M has to be equal to some Mk s.t.
L(Mk) = Ld
■ ==> Will wk belong to L(Mk) or not?
1. If wk L(M
∈ k) ==> T[k,k]=1 ==> wk L
∉ d
2. If wk L(M
∉ k) ==> T[k,k]=0 ==> wk L
∈ d
■ A contradiction either way!!](https://image.slidesharecdn.com/undecidability-250322034336-03fe07b9/85/Undecidability-of-Turing-Machine-in-theory-of-Computation-26-320.jpg)
Undecidability of Turing Machine in theory of Computation
- 2. 2 Decidability vs. Undecidability ■ There are two types of TMs (based on halting): (Recursive) TMs that always halt, no matter accepting or non- accepting ≡ DECIDABLE PROBLEMS (Recursively enumerable) TMs that are guaranteed to halt only on acceptance. If non-accepting, it may or may not halt (i.e., could loop forever). ■ Undecidability: ■ Undecidable problems are those that are not recursive
- 3. 3 Recursive, RE, Undecidable languages Regul ar (DFA) Context- free (PDA) Context sensitiv e Recursiv e Recursively Enumerable (RE) Non-RE Languages (all other languages for which no TMs can be built) LBA TMs that always halt TMs that may or may not halt No TMs exist “Undecidable” problems “Decidable” problems
- 4. 4 Recursive Languages & Recursively Enumerable (RE) languages ■Any TM for a Recursive language is going to look like this: ■Any TM for a Recursively Enumerable (RE) language is going to look like this: M w “accept” “reject” M w “accept”
- 5. 5 Closure Properties of: - the Recursive language class, and - the Recursively Enumerable language class
- 6. 6 Recursive Languages are closed under complementation ■If L is Recursive, L is also Recursive M w “accept ” “reject ” “reject ” “accept” w M
- 7. 7 Are Recursively Enumerable Languages closed under complementation? (NO) ■If L is RE, L need not be RE M w “accept ” “reject ” “accept” w M ? ?
- 8. Recursive Langs are closed under Union ■ Let Mu = TM for L1 U L2 ■ Mu construction: 1. Make 2-tapes and copy input w on both tapes 2. Simulate M1 on tape 1 3. Simulate M2 on tape 2 4. If either M1 or M2 accepts, then Mu accepts 5. Otherwise, Mu rejects. 8 w M1 M2 accept reject accept reject OR Mu
- 9. Recursive Langs are closed under Intersection ■ Let Mn = TM for L1 ∩ L2 ■ Mn construction: 1. Make 2-tapes and copy input w on both tapes 2. Simulate M1 on tape 1 3. Simulate M2 on tape 2 4. If M1 AND M2 accepts, then Mn accepts 5. Otherwise, Mn rejects. 9 w M1 M2 accept reject accept reject Mn AND AND
- 10. 10 Other Closure Property Results ■Recursive languages are also closed under: ■Concatenation ■Kleene closure (star operator) ■Homomorphism, and inverse homomorphism ■RE languages are closed under: ■Union, intersection, concatenation, Kleene closure ■RE languages are not closed under: ■complementation
- 11. 11 “Languages” vs. “Problems” A “language” is a set of strings Any “problem” can be expressed as a set of all strings that are of the form: ■“<input, output>” ==> Every problem also corresponds to a language!! Think of the language for a “problem” == a verifier for the problem e.g., Problem (a+b) ≡ Language of strings of the form { “a#b, a+b” }
- 12. 12 The Halting Problem An example of a recursive enumerable problem that is also undecidable
- 14. 14 What is the Halting Problem? Definition of the “halting problem”: ■Does a givenTuring Machine M halt on a given input w? Machine M Input w
- 15. 15 The Universal Turing Machine ■ Given: TM M & its input w ■ Aim: Build another TM called “H”, that will output: ■ “accept” if M accepts w, and ■ “reject” otherwise ■ An algorithm for H: ■ Simulate M on w ■ H(<M,w>) = accept, if M accepts w reject, if M does does not accept w A Turing Machine simulator Question: If M does not halt on w, what will happen to H? Implies: H is in RE
- 16. 16 A Claim ■Claim: No H that is always guaranteed to halt, can exist! ■Proof: (Alan Turing, 1936) ■By contradiction, let us assume H exists H <M,w> “accept ” “reject ”
- 17. 17 HP Proof (step 1) ■ Let us construct a new TM D using H as a subroutine: ■ On input <M>: 1. Run H on input <M, <M> >; //(i.e., run M on M itself) 2. Output the opposite of what H outputs; H <M > “accept ” “reject ” “reject ” “accept” D <M, “<M>” > Therefore, if H exists D also should exist. 🡺 But can such a D exist? (if not, then H also cannot exist)
- 18. 18 HP Proof (step 2) ■The notion of inputing “<M>” to M itself ■A program can be input to itself (e.g., a compiler is a program that takes any program as input) accept, if M does not accept <M> reject, if M accepts <M> D (<M>) = accept, if D does not accept <D> reject, if D accepts <D> D (<D>) = Now, what happens if D is input to itself? A contradiction!!! ==> Neither D nor H can exist.
- 19. 19 Of Paradoxes & Strange Loops A fun book for further reading: “Godel, Escher, Bach: An Eternal Golden Braid” by Douglas Hofstadter (Pulitzer winner, 1980) E.g., Barber’s paradox, Achilles & the Tortoise (Zeno’s paradox) MC Escher’s paintings
- 20. 20 The Diagonalization Language Example of a language that is not recursive enumerable (i.e, no TMs exist)
- 22. 22 A Language about TMs & acceptance ■ Let L be the language of all strings <M,w> s.t.: 1. M is a TM (coded in binary) with input alphabet also binary 2. w is a binary string 3. M accepts input w.
- 23. 23 Enumerating all binary strings ■Let w be a binary string ■Then 1w ≡ i, where i is some integer ■E.g., If w=ε, then i=1; ■ If w=0, then i=2; ■ If w=1, then i=3; so on… ■If 1w≡ i, then call w as the ith word or ith binary string, denoted by wi. ■ ==> A canonical ordering of all binary strings: ■{ε, 0, 1, 00, 01, 10, 11, 000, 100, 101, 110, …..} ■{w1, w2, w3, w4, …. wi, … }
- 24. 24 Any TM M can also be binary- coded ■M = { Q, {0,1}, Γ, δ, q0,B,F } ■Map all states, tape symbols and transitions to integers (==>binary strings) ■δ(qi,Xj) = (qk,Xl,Dm) will be represented as: ■==> 0i 1 0j 1 0k 1 0l 1 0m ■Result: Each TM can be written down as a long binary string ■==> Canonical ordering of TMs: ■{M1, M2, M3, M4, …. Mi, … }
- 25. 25 The Diagonalization Language ■Ld = { wi | wi L(M ∉ i) } ■The language of all strings whose corresponding machine does not accept itself (i.e., its own code) 1 2 3 4 … 1 0 1 0 1 … 2 1 1 0 0 … 3 0 1 0 1 … 4 1 0 0 1 … i j … . . . diagona • Table: T[i,j] = 1, if Mi accepts wj = 0, otherwise. (input word w) (TMs ) • Make a new language called Ld = {wi | T[i,i] = 0}
- 26. 26 Ld is not RE (i.e., has no TM) ■ Proof (by contradiction): ■ Let M be the TM for Ld ■ ==> M has to be equal to some Mk s.t. L(Mk) = Ld ■ ==> Will wk belong to L(Mk) or not? 1. If wk L(M ∈ k) ==> T[k,k]=1 ==> wk L ∉ d 2. If wk L(M ∉ k) ==> T[k,k]=0 ==> wk L ∈ d ■ A contradiction either way!!
- 27. 27 Why should there be languages that do not have TMs? We thought TMs can solve everything!!
- 28. 28 Non-RE languages Regul ar (DFA) Context- free (PDA) Context sensitiv e Recursiv e Recursively Enumerable (RE) Non-RE Languages How come there are languages here? (e.g., diagonalization language)
- 29. 29 One Explanation There are more languages than TMs ■By pigeon hole principle: ■==> some languages cannot have TMs ■But how do we show this? ■Need a way to “count & compare” two infinite sets (languages and TMs)
- 30. 30 How to count elements in a set? Let A be a set: ■If A is finite ==> counting is trivial ■If A is infinite ==> how do we count? ■And, how do we compare two infinite sets by their size?
- 31. 31 Cantor’s definition of set “size” for infinite sets (1873 A.D.) Let N = {1,2,3,…} (all natural numbers) Let E = {2,4,6,…} (all even numbers) Q) Which is bigger? ■ A) Both sets are of the same size ■ “Countably infinite” ■ Proof: Show by one-to-one, onto set correspondence from N ==> E n 1 2 3 . . f(n ) 2 4 6 . i.e, for every element in N, there is a unique element in E, and vice versa.
- 32. 32 Example #2 ■Let Q be the set of all rational numbers ■Q = { m/n | for all m,n N } ∈ ■Claim: Q is also countably infinite; => |Q|=|N| 1/1 1/2 1/3 1/4 1/5 2/1 2/2 2/3 2/4 2/5 3/1 3/2 3/3 3/4 3/5 4/1 4/2 4/3 4/4 4/5 5/1 5/2 …. …. …. …. ….
- 33. 33 Uncountable sets Example: ■Let R be the set of all real numbers ■Claim: R is uncountable n 1 2 3 4 . . . f(n ) 3 . 1 4 1 5 9 … 5 . 5 5 5 5 5 … 0 . 1 2 3 4 5 … 0 . 5 1 4 3 0 … E.g. x = 0 . 2 6 4 4 … Build x s.t. x cannot possibly occur in the table Really, really big sets! (even bigger than countably infinite sets)
- 34. 34 Therefore, some languages cannot have TMs… ■The set of all TMs is countably infinite ■The set of all Languages is uncountable ■==> There should be some languages without TMs ( by PHP)
- 35. 41 Summary ■ Problems vs. languages ■ Decidability ■ Recursive ■ Undecidability ■ Recursively Enumerable ■ Not RE ■ Examples of languages ■ The diagonalization technique ■ Reducability