Newsgroups: comp.ai.philosophy,comp.theory
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!howland.reston.ans.net!torn!watserv2.uwaterloo.ca!undergrad.math.uwaterloo.ca!math.uwaterloo.ca!tromp
From: tromp@math.uwaterloo.ca (John Tromp)
Subject: Re: Algorithmic Information: Computable?
Message-ID: <D0Hyyq.Cp6@undergrad.math.uwaterloo.ca>
Sender: news@undergrad.math.uwaterloo.ca (news spool owner)
Nntp-Posting-Host: math.uwaterloo.ca
Organization: University of Waterloo
References: <1994Dec2.143356.8747@oracorp.com> <3bq5oq$g06@news1.shell> <jqbD0D7ou.56I@netcom.com> <3c5bra$pc7@news1.shell>
Distribution: inet
Date: Thu, 8 Dec 1994 14:59:14 GMT
Lines: 64
Xref: glinda.oz.cs.cmu.edu comp.ai.philosophy:23390 comp.theory:11302

In article <3c5bra$pc7@news1.shell>, hfinney@shell.portal.com (Hal) writes:
> [crossposted to comp.theory from comp.ai.philosophy]
> 
>There has been some debate on comp.ai.philosophy about the nature of
>algorithmic complexity (Kolmogorov complexity) and whether it is
>numerically computable for a given string.  Is it possible as suggested

[deletia]

>>Jim Balter <jqb@netcom.com> replied:
>>I don't want to comment on the rest of the AC issues, but I do take exception
>>to this.  The halting problem says that no Turing Machine can determine
>>whether every Turing Machine halts.  But that doesn't apply here.  We have a
          ^^^^^

Clearly, not every TM halts:-) A more precise statement is:
No TM can decide whether any TM given to it as input will halt.

>>program and a finite data set.  We can enumerate all shorter programs.  We can
>>analyze each of those shorter programs working on that finite data set.  The
>>HP does not say that we cannot determine, for each such case, whether it
>>terminates.  In fact, I'm fairly certain we always can.  (I'm way too rusty
>>and too lazy to prove such a thing, but if I'm wrong the HP is a much stronger
>>statement than necessary.)

Sure. What you're saying is that for some specific objects, we can compute
their Algorithmic Complexity by showing a program that computes the object,
and by proving that each smaller program either doesn't halt, or computes
something other than the desired object.
The point is, though, that there is no procedure that can do this for all
objects. In fact, a stronger statement can be made:

There is no unbounded partial recursive function that never exceeds the
Algorithmic Complexity.

Although its proof rests in essence on the Halting problem, a more
elegant argument is possible. Namely, suppose that such a function f exists.
So, for any x on which f is defined, we have that f(x) is a lower bound on
the Algorithmic Complexity of x. And for any n, there exists an x, such
that f is defined on x and f(x) >= n.

Then a small TM, T, can start computing f on more and more inputs, until
it finds an x such that f(x) > size(T), and output this x.
Now we have a contradiction in that object x is computed by a machine of
size size(T), yet by assumption, f(x) > size(T) is a lower bound on its
Algorithmic Complexity.

The procedure by which to compute "f on more and more inputs" must of course
avoid the halting problem, so it works by what is known as dovetailing:
compute step 1 on input 0, then step 1 on input 1, then step 2 on input 0,
then step 1 on input 2, etc, dealing with many computations in parallel.
It may seem like cheating that I assumed TM T to "know" its own size,
but this a standard trick in recursion theory, and in fact,
size(T) may be replaced by any bigger number.

Hope this clarifies things.

regards,

%!PS			     %  -John Tromp (tromp@math.uwaterloo.ca)
42 42 scale 7 9 translate .07 setlinewidth .5 setgray/c{arc clip fill
setgray}def 1 0 0 42 1 0 c 0 1 1{0 3 3 90 270 arc 0 0 6 0 -3 3 90 270
arcn 270 90 c -2 2 4{-6 moveto 0 12 rlineto}for -5 2 5{-3 exch moveto
9 0 rlineto}for stroke 0 0 3 1 1 0 c 180 rotate initclip}for showpage
