3. A Turing Machine is represented by a 7-tuple T
= (Q, Σ, Γ, , q0, qaccept, qreject):
Q is a finite set of states
Γ is the tape alphabet, a superset of Σ; Γ
q0 Q is the start state
Σ is the input alphabet, where Σ
: Q Γ → Q Γ {L, R} is the transition
func
qaccept Q is the accept state
qreject Q is the reject state, and qreject qaccept
5. A TM recognizes a language iff it accepts all
and only those strings in the language.
A TM decides a language L iff it accepts all
strings in L and rejects all strings not in L.
A language L is called Turing-recognizable
or recursively enumerable
iff some TM recognizes L.
A language L is called decidable or recursive
iff some TM decides L.
6. A language is called Turing-recognizable or
recursively enumerable (r.e.) if some TM
recognizes it.
A language is called decidable or recursive
if some TM decides it.
recursive
languages
r.e.
languages
recursive
languages
r.e.
languages
7. Theorem: If A and A are r.e. then A is decidable.
Given:
a Turing Machine TMA that recognizes A and
a Turing Machine TMR that recognizes A,
we can build a new machine that decides A.
How can we prove this?
• Run TMA and TMR in parallel.
(Or more precisely, interleave them.)
• One of them will eventually recognize the
input string.
• If TMA recognizes it, then accept.
• If TMR recognizes it, then reject.
8. A TM that decides { 0 | n ≥ 0 }
High-Level Idea.
•Repeatedly divide the number of zeros in half
until it becomes an odd number.
•If we are left with a single zero, then accept.
•Otherwise, reject.
2n
We want to accept iff:
•the input string consists entirely of zeros, and
•the number of zeros is a power of 2.
9. A TM that decides { 0 | n ≥ 0 }
1. Sweep from left to right, cross out every other 0.
(Divides number in half.)
2. If in step 1, the tape had only one 0, accept.
3. Else if the tape had an odd number of 0’s, reject.
4. Move the head back to the first input symbol.
5. Go to step 1.
PSEUDOCODE:
2n
10. C = {ai
bj
ck
| k = i×j, and i, j, k ≥ 1}
Example
aaabbbbcccccccccccc
3 3×4 = 12
4
11. C = {ai
bj
ck
| k = i×j, and i, j, k ≥ 1}
High-Level Idea.
For each occurrence of a: {
For each occurrence of b: {
Delete an occurrence of c.
}
}
12. C = {ai
bj
ck
| k = i×j, and i, j, k ≥ 1}
1. If the input doesn’t match a*b*c*, reject.
2. Move the head back to the leftmost symbol.
3. Cross off an a, scan to the right until b.
Sweep between b’s and c’s, crossing out one of
each until all b’s are out. If too few c’s, reject.
4. Uncross all the b’s.
If there’s another a left, then repeat stage 3.
If all a’s are crossed out,
Check if all c’s are crossed off.
If yes, then accept, else reject.
PSEUDOCODE:
13. C = {ai
bj
ck
| k = i×j, and i, j, k ≥ 1}
aabbbcccccc
xabbbcccccc
xayyyzzzccc
xabbbzzzccc
xxyyyzzzzzz
14. TURING-MACHINE VARIANTS
Turing machines can be extended in various ways,
but so long as a new TM only reads and writes a
finite number of symbols in each step,
an old TM can still simulate it!
Example: Turing machines with multiple tapes.
Input comes in on one tape, and
other tapes are used for scratch work.
16. Theorem: Every Multitape Turing Machine can be
transformed into a single tape Turing Machine
FINITE
STATE
CONTROL
0 0
1
FINITE
STATE
CONTROL 0 0
1 # # #
. . .
17. Theorem: Every Multitape Turing Machine can be
transformed into a single tape Turing Machine
FINITE
STATE
CONTROL
0 0
1
FINITE
STATE
CONTROL 0 0
1 # # #
. . .
18. THE CHURCH-TURING THESIS
Anything that can be computed
by algorithm (in our intuitive
sense of the term “algorithm”)
can be computed by a Turing
Machine.
19. We can encode a TM as a string of 0s and 1s
0n
10m
10k
10s
10t
10r
10u
1…
n states
m tape symbols
(first k are input
symbols)
start
state
accept
state
reject
state
blank
symbol
( (p, a), (q, b, L) ) = 0p
10a
10q
10b
10
( (p, a), (q, b, R) ) = 0p
10a
10q
10b
11
20. Similarly, we can encode DFAs, NFAs,
CFGs, etc. into strings of 0s and 1s
ADFA = { (B, w) | B is a DFA that accepts string w }
ANFA = { (B, w) | B is an NFA that accepts string w }
ACFG = { (G, w) | G is a CFG that generates string w }
So we can define the following languages:
21. ADFA = { (B, w) | B is a DFA that accepts string w }
Theorem: ADFA is decidable
Proof Idea: Simulate B on w
ACFG = { (G, w) | G is a CFG that generates string w }
Theorem: ACFG is decidable
Proof Idea: Transform G into Chomsky Normal
Form. Try all derivations of length 2|w|-1
ANFA = { (B, w) | B is an NFA that accepts string w }
Theorem: ANFA is decidable
23. w L ?
accept reject
TM
yes no
w Σ*
L is decidable
(recursive)
w L ?
accept reject or no output
TM
yes no
w Σ*
L is semi-decidable
(recursively enumerable,
Turing-recognizable)
Theorem: L is decidable if both L and L
are recursively enumerable
24. There are languages over {0,1}
that are not decidable.
If we believe the Church-Turing Thesis, this is
major: it means there are things that formal
computational models inherently cannot do.
We can prove this using a counting argument.
We will show there is no function from the set
of all Turing Machines onto the set of all
languages over {0,1}. (Works for any Σ.)
Then we will prove something stronger:
There are semi-decidable (r.e.) languages that
are NOT decidable.
26. Let L be any set and 2L
be the power set of L
Theorem: There is no map from L onto 2L
Proof: Assume, for a contradiction, that
there is an onto map f : L 2L
Let S = { x L | x f(x) }
We constructed S so that, for every elem x in L,
the set S differs from f(x):
S ≠ f(x) because x S iff x f(x)
Cantor’s Theorem
27. Theorem: There is no onto function from the
positive integers to the real numbers in (0, 1)
1
2
3
4
5
:
0.28347279…
0.88388384…
0.77635284…
0.11111111…
0.12345678…
:
Proof: Suppose f is such a function:
[ nth
digit of r ] =
2
8
6
1
5
1 if [ nth
digit of f(n) ] 1
0 otherwise
f(n) r for all n ( Here, r = 0.11101... )
30. Not all languages over {0,1} are decidable, in fact:
not all languages over {0,1} are semi-decidable
{Turing Machines}
{Strings of 0s and 1s} {Sets of strings
of 0s and 1s}
{Languages over {0,1}}
Set L Powerset of L: 2L
{decidable languages over {0,1}}
{semi-decidable langs over {0,1}}
31. ATM = { (M, w) | M is a TM that accepts string w }
THE ACCEPTANCE PROBLEM
Theorem: ATM is semi-decidable (r.e.)
but NOT decidable
ATM is r.e. :
Define a TM U as follows:
On input (M, w), U runs M on w. If M ever
accepts, accept. If M ever rejects, reject.
Therefore,
U accepts (M,w) M accepts w (M,w) ATM
Therefore, U recognizes ATM
U is a universal TM
32. ATM = { (M,w) | M is a TM that accepts string w }
ATM is undecidable: (proof by contradiction)
Assume machine H decides ATM
H( (M,w) ) =
Accept if M accepts w
Reject if M does not accept w
Construct a new TM D as follows: on input M,
run H on (M,M) and output the opposite of H
D( M ) =
Reject if M accepts M
Accept if M does not accept M
D
D D
D D
Contradiction!
33. Theorem: ATM is r.e. but NOT decidable
Theorem: ATM is not even r.e.!
The Halting Problem is Not Decidable
34. We have shown:
Given any presumed machine H for ATM,
we can effectively construct a TM D such that
(D,D) ATM but H fails to tell us that.
In other words,
For any machine H that recognizes ATM
we can effectively give an instance
where H fails to decide ATM
In other words,
Given any good candidate for deciding the
Halting Problem, we can effectively construct
an instance where the machine fails.
35. HALTTM = { (M,w) | M is a TM that halts on string w }
Theorem: HALTTM is undecidable
THE HALTING PROBLEM
Proof: Assume, for a contradiction, that TM H
decides HALTTM
We use H to construct a TM D that decides ATM
On input (M,w), D runs H on (M,w)
If H rejects then reject
If H accepts, run M on w until it halts:
Accept if M accepts and
Reject if M rejects
36. In many cases, one can show that a
language L is undecidable by showing
that if it is decidable, then so is ATM
We reduce deciding ATM to deciding
the language in question
ATM ≤ L
We just showed: ATM ≤ HaltTM
Is HaltTM ≤ ATM ?
#2:No homework this week! (Valentine’s day gift?)
We’ve seen problems that Finite Automata and Push-Down Automata can’t solve.
Now we’ll talk about problems that even Turing machines can’t solve.
#4:We can completely describe a Turing machine at any step of its computation using strings called configurations.
These are strings over the tape alphabet and the states (treating the states as symbols).
For example, this string means that the Turing machine has
these 1’s and 0’s on its tape,
the state is q7, and
its tape head is reading the 0 to the right of it.
Configurations are like “snapshots” of the TM at a given point.
With configurations, we can give a formal definition of TM accepting a string. (Read Sipser.)
On input w, the “start” configuration is ‘q0 w’
From any particular configuration, there is a “next” configuration we can get to by applying the transition function.
And if we reach a configuration that has an accept state in it, then the machine accepts.
#5:Now, since a TM can run forever on some inputs,
we have two different ways for a TM to “compute” a language.
#7:So what does this theorem mean?
(Click)
How might we prove this?
(Don’t go on until it’s proved.)
#8:How can we get a TM that decides this language?
As with the last lecture, we’ll want to use precise pseudocode whenever we can, because it’s just so much easier to understand.
But for every line of pseudocode you write, you need to think about how you’d convert that into a formal specification.
(Every execution of stage 1 halves the number of 0’s.
If we get to “only one 0” and we never got an odd number of 0’s greater than one,
then we must have had a power of two.)
#9:How can we get a TM that decides this language?
As with the last lecture, we’ll want to use precise pseudocode whenever we can, because it’s just so much easier to understand.
But for every line of pseudocode you write, you need to think about how you’d convert that into a formal specification.
(Every execution of stage 1 halves the number of 0’s.
If we get to “only one 0” and we never got an odd number of 0’s greater than one,
then we must have had a power of two.)
#10:This is a state diagram corresponding to the pseudocode of the previous slide.
Let’s think about how the two correspond.
We have an extra symbol “x” that is used to cross out symbols.
We start by writing the first 0 over with a blank. If there was just a single 0, we accept.
(This corresponds to stage 2.)
In states q3 and q4, we cross out every other 0, until we see a blank. (Stage 1)
If we see a blank in state q4, then the number of 0’s left was odd, so we reject. (Stage 3)
If we see a blank in state q3, then the number was even, so now in q2 we move left until we hit blank
(Stage 4).
We move back to q1.
#11:Let’s look at what this TM does by looking at its configurations on an input with four zeroes.
….
….
And the process will continue from q1 at this point.
#12:(If not much time left, skip this.)
We can have a TM that does some basic arithmetic.
If we run out of c’s in stage 3, we’ll reject in that case too.
#13:(If not much time left, skip this.)
We can have a TM that does some basic arithmetic.
If we run out of c’s in stage 3, we’ll reject in that case too.
#14:(If not much time left, skip this.)
We can have a TM that does some basic arithmetic.
If we run out of c’s in stage 3, we’ll reject in that case too.
#16:One amazing aspect of Turing machines is that you can define all sorts of variations on them, give them more complicated ways to store information, faster ways to access information, whatever, but so long as your new TM only reads and writes a finite number of symbols in each step, it doesn’t matter: an old TM can simulate it!
Here’s an example.
We can define Turing machines with multiple tapes.
We can imagine the input coming in (on one tape), and using the other tapes for scratch work.
This can easily be stuck into the formalism, by redefining the transition function: now it takes k tape symbols, and outputs k tape symbols, and k tape directions.
#17:One amazing aspect of Turing machines is that you can define all sorts of variations on them, give them more complicated ways to store information, faster ways to access information, whatever, but so long as your new TM only reads and writes a finite number of symbols in each step, it doesn’t matter: an old TM can simulate it!
Here’s an example.
We can define Turing machines with multiple tapes.
We can imagine the input coming in (on one tape), and using the other tapes for scratch work.
This can easily be stuck into the formalism, by redefining the transition function: now it takes k tape symbols, and outputs k tape symbols, and k tape directions.
#18:We can simulate these multiple tapes with one tape, in the following way.
We write down the contents of all these tapes in order, and put a # between two tapes.
To keep track of where all the heads of the multitape machine are,
we double the size of the alphabet to include “marked” symbols,
and a mark above a symbol indicates the head position.
#19:When some tape head moves, we move the mark.
When the multi-tape machine makes one of its tape contents “longer” (by writing over some blanks),
the simulation of it moves all the tape content beyond over by one.
Like, suppose the multitape machine adds a 0 to the end of the first tape.
Then everything from the # on is shifted over to the right, and 0 is written.
#20:The facts that:
(1) we can change the definition of a TM so much, without changing what can be decided by them,
AND
(2) everything we can think of that can be computed with an intuitive idea of an algorithm can be done with TMs,
led mathematicians to state the Church-Turing thesis.
If some language can be computed,
in our intuitive sense of what it means to “compute”,
then we can build a Turing machine that decides it.
#21:An important observation that is used time and time again in computability theory and complexity theory,
is that we can encode TMs.
This means that we can have TMs compute on the code of other TMs.
…
Now if we think of the states and tape symbols as integers, then we can encode transitions as follows…
#24:No homework this week! (Valentine’s day gift?)
We’ve seen problems that Finite Automata and Push-Down Automata can’t solve.
Now we’ll talk about problems that even Turing machines can’t solve.
#26:Recall: What does “onto” mean?
Here a function is onto if for every language L over {0,1}, there is some TM that the function maps to L.
In other words, no matter how you map TMs to languages, some language will always be “left out”.
The proof will carry over for any finite alphabet.
#27:Pictorially, this is what we’ll show: if we take any function that maps TMs to languages, there will be some languages (in the yellow) that won’t get mapped to.
#29:f is supposed to map every natural number to a real number, and somehow cover all the reals in the process.
Now we construct a real number r such that there is no positive integer n with f(n) = r.
This process is called “diagonalization” which you may have heard of…
#30:Interestingly enough, while there is no bijection from the positive integers to the real line,
there is a bijection from Z+ to Z+ x Z+.
Consider all possible pairs of positive integers. We can map 1 to (1,1), 2 to (1,2), 3 to (2,1), and so forth.
In this way we can “stretch” Z+ x Z+ so that it fits into Z+.
#32:Consider the set of all decidable languages. This is a subset of the set of all semi-decidable languages.
Every semi-decidable language has associated with it a Turing machine.
Recall we can encode every TM with a binary string, so the set of Turing machines is contained in the set of all strings of 0’s and 1’s. Let this set be L. (Sigma*)
…
Now consider the set of all languages over {0,1}.
This is the set of all sets of strings of 0s and 1s, which is 2^L.
…
Since the Turing machines are contained in L, we know they are properly contained in 2^L.
#33:Recall a few lectures ago we were looking at the acceptance problems for DFAs, NFAs, and CFGs.
…
Now it’s not totally obvious that there is a Turing machine that can simulate any other Turing machine given as code, but it turns out that you can do it. Read Sipser for more details.
#34:We’re going to show that this D leads to an absurdity, that it just can’t exist.
But the only assumption we made was that there is an H that decides A_{TM}.
Therefore THERE IS NO H THAT DECIDES A_{TM}!
#35:Now consider the machine D.
Suppose it gets its own code as input. What does it output?
Suppose D outputs accept. That means H(D,D) rejected. But that means that D rejects on the code of D!
OK, what if D outputs reject. Then H(D,D) accepted. But that means D accepts on the code of D!
Either way is a contradiction!
#36:How can we show that A_{TM} is r.e.?
Remember, we just need a TM that accepts the strings in the language, it can run forever on other inputs…
Now why is the complement of A_{TM} not r.e.?
#39:H rejects: We know that if M doesn’t halt, it won’t accept!
H accepts: Then we can run M itself.
#40:Diagram. Suppose we can solve the Halting problem with a machine H.
We’ll treat it as a black box.
We’ll make a machine D that takes input (M,w).
If it says that M will halt, reject. If it says M halts, then we run M on w and output its answer.
This way, we decide if M accepts w!
#41:That is, by using a decider for L as a black box,
we can code up another algorithm that decides A_{TM}.
So in a sense, A_{TM} is “easier” than L, since an algorithm deciding L would give us an algorithm deciding A_{TM}.
Now does the inequality go in reverse?
Can we say that if you have a black box B for solving A_{TM}, you can solve the halting problem? Sure!
Given (M,w), make a new machine M’ that simulates M on w. If the simulation ever halts, M’ accepts. Now we call B(M’,w). This says accept if and only if M halts on w.
![Theorem: There is no onto function from the
positive integers to the real numbers in (0, 1)
1
2
3
4
5
:
0.28347279…
0.88388384…
0.77635284…
0.11111111…
0.12345678…
:
Proof: Suppose f is such a function:
[ nth
digit of r ] =
2
8
6
1
5
1 if [ nth
digit of f(n) ] 1
0 otherwise
f(n) r for all n ( Here, r = 0.11101... )](https://image.slidesharecdn.com/lecture9x-250310051743-ee1bd0c9/85/Turing-machine-power-point-presentations-27-320.jpg)
