From newshub.ccs.yorku.ca!ists!helios.physics.utoronto.ca!news-server.csri.toronto.edu!rpi!usc!cs.utexas.edu!swrinde!gatech!rutgers!rochester!cantaloupe.srv.cs.cmu.edu!kck Wed Dec 18 16:02:05 EST 1991
Article 2180 of comp.ai.philosophy:
Path: newshub.ccs.yorku.ca!ists!helios.physics.utoronto.ca!news-server.csri.toronto.edu!rpi!usc!cs.utexas.edu!swrinde!gatech!rutgers!rochester!cantaloupe.srv.cs.cmu.edu!kck
>From: kck+@cs.cmu.edu (Karl Kluge)
Newsgroups: comp.ai.philosophy
Subject: Godel's theorem and AI criticism
Message-ID: <1991Dec17.043358.111702@cs.cmu.edu>
Date: 17 Dec 91 04:33:58 GMT
Organization: School of Computer Science, Carnegie Mellon
Lines: 80
Nntp-Posting-Host: g.gp.cs.cmu.edu
Originator: kck@G.GP.CS.CMU.EDU

***************************************************************************
"...The lunatic, on the other hand doesn't concern himself with logic; he
works by short circuits. For him, everything proves everything else. The
lunatic is all idee fixe, and whatever he come across confirms his lunacy.
You can tell him by the liberties he takes with common sense, by his flashes
of inspiration, and by the fact that sooner or later he brings up the
Templars."

"Invariably?"

"There are lunatics who don't bring up the Templars, but those who do are
the most insidious. At first they seem normal, then all of a sudden..."
				(from FOUCAULT'S PENDULUM by Umberto Eco)
***************************************************************************

In the case of AI criticism, substitute "Godel's theorem" for "the
Templars."

What are the implications of Godel's theorem for AI? To give an example of
one, suppose we adopt (provisionally, and with the understanding that it is
subject to other criticisms) Gold's definition of learnability.

In this definition, we supply a sequence of examples (positive and negative)
to an algorithm to determine the generating function for the positive
examples.  The algorithm determines it's best guess of a generating function
by a sieve algorithm, in which generating functions are listed from shortest
function description to longer ones. After each element in the sequence
generating functions which don't fit the element given and its
classification are crossed out, and the shortest generating function
remaining is output as the current best guess.  Gold suggested defining the
learnability of a concept given some familiy of possible generating
functions in terms of the convergence of the sieve algorithm given above.

One consequence of Godel's theorem is that there are families of generating
functions such that a given sequence has a legal generating function, but
that generating function cannot be found in finite time using bounded space.

(Sketch proof -- if you allow unbounded space, you can just keep track of
the position in the enumeration of the current shortest generating function
and all the examples to date, and step through to the next element that
covers the past and current examples. If all you keep is the current
shortest correct-to-date guess F1(x) and misclassified current example x1,
then you have to find the shortest F2(x) such that for all x, if x != x1
then F1(x) == F2(x), and if x == x1, F1(x) != F2(x). There will be classes
of generating functions such that such an F2 exists, but there is no finite
proof that F2 meets the criteria. If the above is not correct, I'm sure
someone will let me know -- possibly in unflattering terms that question my
ancestry and habits. The Eco quote shouldn't be taken to heart.)

Should this bother us? Not really. 

To gave an analogous example, look at a very simple problem from computer
vision. Given a line drawing of polyhedral objects built so that either two
or three surfaces come together at a vertex (the so-called "trihedral blocks
world"), you can ask if there is a legal labelling of the lines between
vertices in the drawing as either convex, concave, or occluding edges in the
scene. If there is no legal labelling, then the line drawing isn't a
possible scene from the trihedral blocks world. 

Papdimitrou and Kirousis proved that determining the labelability of a line
drawing is NP complete. In fact, if you add the constraints on surface
normal orientation provided by the geometry of the drawing, the problem is
still NPC. That came as a bit of a suprise, as very simple constraint
propagation algorithms do very well at determining labelability quickly. A
little reflection suggests a simple explanation -- scenes of real polyhedra
generally don't encode 3-CNF SAT clauses.

Godel's theorem, like NP completeness, talks about worst cases. There is no
good reason (as far as I can see) to believe that useful concepts would have
generating functions which were Godel sentences.  For that matter, there is
no evidence (and can be no evidence) that given a set of axioms that
includes the axioms of arithmatic over the integers, humans can recognize
true sentences which have no finite proof from the axioms.  It is therefore
far from clear to me what the supposed implications of Godel's theorem are
for computational models of human cognition.

Karl Kluge (kck@cs.cmu.edu)

Disclaimer: How can these opinions be those of CMU when my advisor thinks I'm
using this machine to write my thesis?


