The axiomatic power of Kolmogorov complexity

.
.
.
How useful are random axioms?
Laurent Bienvenu (CNRS & University of Paris 7)
Andrei Romashchenko (University of Montpellier 2)
Alexander Shen (University of Montpellier 2)
Antoine Taveneaux (University of Paris 7)
Stijn Vermeeren (University of Leeds)
Séminaire Logique
October
28, 2012

.
.
Peano arithmetic
In this talk, we work in the axiomatic system of Peano arithmetic, PA, over the
language (0, 1, +, ×). Its standard model N is the set N of natural numbers,
with standard addition and multiplication.
1. Random axioms 3/31

.
.
Peano arithmetic
In this talk, we work in the axiomatic system of Peano arithmetic, PA, over the
language (0, 1, +, ×). Its standard model N is the set N of natural numbers,
with standard addition and multiplication.
As it is well known, Gödel’s theorem applies to this theory: there are sentences
that are true (in the standard model) but not provable.
PA ⊢ φ
⇒
⇍
N |= φ

.
.
Random axioms
Now imagine that a formula φ has the following properties:
1. PA ⊬ φ(¯n) for any n ≤ N

.
.
Random axioms
2. At least 99.9% of integers n ≤ N satisfy φ

.
.
Random axioms
3. Item 2. is provable inside PA

.
.
Random axioms
Then, if we pick an integer r ≤ N at random and consider the theory
PA + φ(¯r)
with error probability at most 0.1% our new theory is coherent (assuming PA is)
and true.

.
.
Random axioms
Then, if we pick an integer r ≤ N at random and consider the theory
PA + φ(¯r)
with error probability at most 0.1% our new theory is coherent (assuming PA is)
and true.
Now if
PA + φ(¯r) ⊢ ψ
we have (in some sense) “evidence” that ψ is likely to be true.

.
.
The canonical example
The canonical example:
C(x) ≥ L − 20 for binary strings x of length ≤ L
where C denotes Kolmogorov complexity (to which we will return shortly).

.
.
..
Theorem (Chaitin)
The above statement is true for a fraction ≥ 1 − 10−6
of strings x (and this we
can prove!), but is not provable in PA for any x (assuming L is large enough).
..
Theorem (Chaitin)
.
.
Theorem (Chaitin)

.
.
..
Theorem (Chaitin)
..
Theorem (Chaitin)
.
.
Theorem (Chaitin)
On this particular example, we could choose a binary string r of length L at
random (say by ﬂipping a coin), add to PA the new axiom C(r) ≥ L − 20 and try
to prove new statements.

.
.
Probabilistic proofs (1)
This yields the natural idea of probabilistic proof.

.
.
This yields the natural idea of probabilistic proof.
Fix an error threshold δ in advance (margin of error that you ﬁnd acceptable)
which will be treated as a initial capital. Start with the theory T0 = PA.

.
.
At step n, we have two options:

.
.
• We can use the statements of our theory Tn to prove (in the normal way) a
new fact ψ and take Tn+1 = Tn ∪ {ψ}

.
.
• We can use the statements of our theory Tn to prove (in the normal way) a
new fact ψ and take Tn+1 = Tn ∪ {ψ}
• Or, if we have in Tn a statement of type
#{n ∈ A | φ(¯n)} ≥ (1 − ε) · |A|
for some ﬁnite set A, we can choose some r ∈ A at random, pay ε from
our capital, and take Tn+1 = Tn ∪ {φ(¯r)} (unless our capital becomes
negative in which case this step is not allowed)

.
.
Such a proof scheme (or strategy) is probabilistic: if we run it twice we might
prove completely different statements!

.
.
Still, if a ﬁxed statement ψ could be proven with high probability with this
scheme, our intuition tells us that ψ is very likely to be true. This raises two
questions:

.
.
questions:
1. Is this intuition correct? i.e., does this proof scheme make sense?

.
.
questions:
1. Is this intuition correct? i.e., does this proof scheme make sense?
2. Is it useful? i.e., can we prove more things than we classically could?

.
.
The answer: Yes, it makes sense.

.
.
..
Theorem
If a proof strategy with initial capital δ proves ψ with probability > δ, then ψ is
true.
..
Theorem
true.
.
.
Theorem
true.

.
.
..
Theorem
true.
..
Theorem
true.
.
.
Theorem
true.
No, it is not (really) useful.
..
Theorem
If a proof strategy with initial capital δ proves ψ with probability > δ, then in fact
ψ is already provable in PA.
..
Theorem
..
Theorem

.
.
Probabilistic proofs are still useful in the following sense: in the determinization
argument (transforming a probabilistic proof into a deterministic one), there is an
exponential blow-up, i.e., the deterministic proof is in general exponentially
longer than the probabilistic one.

.
.
Probabilistic proofs are still useful in the following sense: in the determinization
argument (transforming a probabilistic proof into a deterministic one), there is an
exponential blow-up, i.e., the deterministic proof is in general exponentially
longer than the probabilistic one.
In fact, this cannot be avoided:
..
Theorem
Unless NP=PSPACE, there are statements with polynomial-size probabilistic
proofs and only exponential-size deterministic ones.
..
Theorem
..
Theorem

.
2. The axiomatic power
of Kolmogorov complexity

.
.
Kolmogorov complexity (1)
The Kolmogorov complexity C(x) of a ﬁnite binary string x is the length of the
shortest program (written in binary) that generates x. In a sense, it measures
the amount of information contained in the string x.
of Kolmogorov complexity 12/31

.
.
For any given string x, we have
0 ≤ C(x) ≤ |x| + O(1)
C(x) ≈ 0 meaning that x is simple (= far from random), while C(x) ≈ |x| means
that x is quite random.

.
.
For any given string x, we have
0 ≤ C(x) ≤ |x| + O(1)
C(x) ≈ 0 meaning that x is simple (= far from random), while C(x) ≈ |x| means
that x is quite random.
Most strings are quite random: there is only a fraction ≤ 2−c
of strings of a given
length such that C(x) ≤ |x| − c.

.
.
Variation 1: K(x) is the preﬁx-free Kolmogorov complexity, i.e. the Kolmogorov
complexity one gets if the set of valid programs is preﬁx-free.

.
.
Variation 1: K(x) is the prefix-free Kolmogorov complexity, i.e. the Kolmogorov
complexity one gets if the set of valid programs is prefix-free.
Variation 2: C(x | y) and K(x | y) denote the conditional and prefix-free
conditional Kolmogorov complexity, i.e., the length of the shortest program that
produces x given y as input.

.
.
Kolmogorov complexity is not a computable function. It is however
upper-semicomputable: one can computably enumerate pairs (x, n) such that
C(x) ≤ n.

.
.
Kolmogorov complexity is not a computable function. It is however
upper-semicomputable: one can computably enumerate pairs (x, n) such that
C(x) ≤ n.
To summarize:
• If C(x) ≤ n1 we will ﬁnd out eventually
• If C(x) ≥ n2 we might never know for sure

.
.
Translation into the language of logic:
..
Theorem (Chaitin)
• If C(x) ≤ n1, then PA ⊢ “C(x) ≤ n1”
• No matter what the actual value of C(x) is, PA ⊬ “C(x) ≥ n2” (provided n2 is
large enough).
..
Theorem (Chaitin)
• If C(x) ≤ n1, then PA ⊢ “C(x) ≤ n1”
large enough).
.
.
Theorem (Chaitin)
• If C(x) ≤ n1, then PA ⊢ “C(x) ≤ n1”
large enough).

.
.
Translation into the language of logic:
..
Theorem (Chaitin)
• If C(x) ≤ n1, then PA ⊢ “C(x) ≤ n1”
large enough).
..
Theorem (Chaitin)
• If C(x) ≤ n1, then PA ⊢ “C(x) ≤ n1”
large enough).
.
.
Theorem (Chaitin)
• If C(x) ≤ n1, then PA ⊢ “C(x) ≤ n1”
large enough).
Proof: Berry’s paradox → otherwise, given n, look for an xn such that
PA ⊢ “C(xn) ≥ n”. Since n is enough to describe xn, C(xn) ≤ O(log n), a
contradiction.

.
.
Gödel’s theorem
In the previous argument, PA can be replaced by any stronger, recursive and
coherent theory. Thus, we have obtained a proof of Gödel’s ﬁrst incompleteness
theorem using a Kolmogorov complexity interpretation of Berry’s paradox.

.
.
Gödel’s theorem
In the previous argument, PA can be replaced by any stronger, recursive and
coherent theory. Thus, we have obtained a proof of Gödel’s ﬁrst incompleteness
theorem using a Kolmogorov complexity interpretation of Berry’s paradox.
Note: one can also get a very elegant proof of Gödel’s second
incompleteness theorem via a Kolmogorov complexity interpretation of the
surprise examination paradox (Kritchman and Raz).

.
.
Adding axioms about C (1)
Since PA does not prove them, a natural question:
How useful is the set C of true axioms of type “C(x) ≥ n”?

.
.
..
Theorem
The statements provable from PA + C are exactly those of Π1-Th(N), i.e., the
statements φ that only contain ∀ as unbounded quantiﬁers and are true in N.
..
Theorem
.
.
Theorem

.
.
..
Theorem
..
Theorem
.
.
Theorem
Proof. This is a direct translation into logic of the fact that if one can compute C
then one can compute the halting problem ∅′.

.
.
Interestingly, among the axioms “C(x) ≥ n”, some are more useful than others:
..
Theorem
There is a constant c and sequence of strings (xm) such that |xm| = m, C(xm) ≥
m − c, and adding to PA all axioms “C(xm) ≥ m − c” proves all statements of
Π1-Th(N).
..
Theorem
Π1-Th(N).
.
.
Theorem
Π1-Th(N).

.
.
Interestingly, among the axioms “C(x) ≥ n”, some are more useful than others:
..
Theorem
Π1-Th(N).
..
Theorem
Π1-Th(N).
.
.
Theorem
Π1-Th(N).
On the other hand...
..
Theorem
If φ is a statement of Π1-Th(N) which is not provable in PA, it is possible to
add, for each length m, 99% of the true statements of type “C(y) ≥ m −c”, with
|y| = m, and still be unable to prove φ.
..
Theorem
..
Theorem

.
.
This raises the question:

.
.
..
Open question
Characterize the “useful” xn, i.e., those such that the collection “C(xn) ≥ n − c”
is true and proves all of Π1-Th(N).
..
Open question
.
.
Open question

.
.
..
Open question
..
Open question
.
.
Open question
One last important point: axioms need to be precise to be useful.

.
.
..
Open question
..
Open question
.
.
Open question
One last important point: axioms need to be precise to be useful.
..
Theorem
There is no sequence (xn) such that C(xn) ≥ n − c and such that adding the
collection of axioms “C(xn) ≥ n/2 − c” proves all of Π1-Th(N).
..
Theorem
..
Theorem

.
.
Another example: consider for a given k the set CCk of true axioms of type
“C(x | y) ≥ k”

.
.
“C(x | y) ≥ k”
..
Theorem
For any k large enough, the statements provable from PA+CCk are exactly those
of Π1-Th(N).
..
Theorem
of Π1-Th(N).
.
.
Theorem
of Π1-Th(N).

.
.
“C(x | y) ≥ k”
..
Theorem
of Π1-Th(N).
..
Theorem
of Π1-Th(N).
.
.
Theorem
of Π1-Th(N).
Proof. We show that being able to decide the truth of “C(x | y) ≥ k” allows us
to decide the halting problem, and translate the proof into logic.

.
.
More interesting: the translation into logic does not necessarily go through!

.
.
..
Theorem (BBFFGLMST)
If one is given for each x a pair of values n1 and n2 such that C(x) = n1 or
C(x) = n2, then one can compute ∅′.
..
Theorem (BBFFGLMST)
.
.
Theorem (BBFFGLMST)

.
.
..
Theorem (BBFFGLMST)
..
Theorem (BBFFGLMST)
.
.
Theorem (BBFFGLMST)
But...
..
Theorem
Let φ be any statement not provable in PA. Then it is possible to add for each x
a true axiom of type “C(x) = n1 ∨ C(x) = n2” and still have a theory too weak
to prove φ.
..
Theorem
to prove φ.
..
Theorem
to prove φ.

.
3. Inﬁnite binary sequences

.
.
Random sequences
Kolmogorov complexity measures the amount randomness for finite binary
sequences. For infinite binary sequences, it even allows us to provide an exact
definition of randomness.
3. Infinite binary sequences 23/31

.
.
Random sequences
Kolmogorov complexity measures the amount randomness for finite binary
sequences. For infinite binary sequences, it even allows us to provide an exact
definition of randomness.
A sequence X = x1x2x3 . . . is said to be (Martin-Löf) random with constant c if
K(x1x2 ... xn) ≥ n − c

.
.
Randomness of a sequence, axiomatized (1)
Some random sequences do compute ∅′, such as Chaitin’s Omega number.

.
.
Could it then be that expressing the randomness of some sequences would
prove all of Π1-Th(N)?

.
.
Could it then be that expressing the randomness of some sequences would
prove all of Π1-Th(N)?
Interestingly, it depends on the way we express the randomness of X.

.
.
Given a sequence X random with constant c, the obvious way is to take the set
MLRc(X) of all axioms of type
K(x1x2 ... xn) ≥ n − c

.
.
Given a sequence X random with constant c, the obvious way is to take the set
MLRc(X) of all axioms of type
K(x1x2 ... xn) ≥ n − c
In that case, we cannot prove all of Π1-Th(N):
..
Theorem
• If MLRc(X) is consistent, adding it to PA is never sufﬁcient to prove
Π1-Th(N).
• Moreover, MLRc(X) is asymptotically useless: if φ is not provable in PA,
then PA + MLRc(X) does not prove φ for any c large enough.
..
Theorem
Π1-Th(N).
..
Theorem
Π1-Th(N).

.
.
A very rough sketch of the proof. Suppose for the sake of contradiction that
MLRc(X) is true and proves Π1-Th(N).

.
.
We use an interesting back-and-forth between logic and computability.

.
.
• Note that X can compute the set of axioms MLRc(X), hence X can
compute Π1-Th(N), thus X ≥T ∅′.

.
.
• Now, consider the set of complete coherent extensions of PA, coded as
inﬁnite binary sequences.

.
.
• Now, consider the set of complete coherent extensions of PA, coded as
inﬁnite binary sequences.
• This forms a Π0
1 class P, so by the basis theorem for randomness, there is
A ∈ P such that X is random relative to A.

.
.

.
.
• Let T be the logical theory coded by A. Argue that X being A-random
implies that X is seen as random in the theory T .

.
.
• Argue that therefore, in the theory T , the collection of axioms MLRc(X) is
(almost) true.

.
.
(almost) true.
• Thus the theory T proves Π1-Th(N).

.
.
(almost) true.
• Thus A ≥T ∅′.

.
.
(almost) true.
• Thus A ≥T ∅′.
• Thus X is random relative to ∅′, a contradiction with X ≥T ∅′.

.
.
An alternative way: take the set MLR′
c(X) of all axioms of type
(∀N)(∃σ ≻ x1...xn) |σ| = N and K(σ) ≥ N − c

.
.
An alternative way: take the set MLR′
c(X) of all axioms of type
(∀N)(∃σ ≻ x1...xn) |σ| = N and K(σ) ≥ N − c
In this case, MLR′
c(X) may be powerful.
..
Theorem
• There exists a consistent set of axioms MLR′
c(X) such that PA+MLR′
c(X)
proves all of Π1-Th(N) [take Chaitin’s Omega].
• Still, MLR′
c(X) is asymptotically useless as before.
..
Theorem
c(X)
• Still, MLR′
..
Theorem
c(X)
• Still, MLR′

.
.
Non-random sequencess can help too!
We finish on this intriguing fact: while MLRc(X) is never very useful for a
random X, there is a non-random sequence Z whose complexity gives useful
information:
..
Theorem
There is a sequence Z = z1z2... and constant c such that K(z1 ... zn) ≥ n − c
for infinitely many n, and such that if we take these infinitely many statements as
axioms, we can prove all of Π1-Th(N).
..
Theorem
axioms, we can prove all of Π1-Th(N)...
Theorem
axioms, we can prove all of Π1-Th(N).

.
.
Open questions

.
.
Open questions
..
Open question
..
Open question
.
.
Open question

.
.
Open questions
..
Open question
..
Open question
.
.
Open question
..
Open question
What axioms about Kolmogorov should we look at if we want to say something
about Πn-Th(N)?
..
Open question
about Πn-Th(N)? ..
Open question
about Πn-Th(N)?

.
.Thank You !

The axiomatic power of Kolmogorov complexity

More Related Content

What's hot (20)

Similar to The axiomatic power of Kolmogorov complexity (20)

Recently uploaded (20)

The axiomatic power of Kolmogorov complexity