Message-ID: <32AC7C58.6D64@parkcity.com>
Date: Mon, 09 Dec 1996 13:53:44 -0700
From: Richard Keene <rkeene@parkcity.com>
Organization: Park City Group
X-Mailer: Mozilla 2.0 (WinNT; U)
MIME-Version: 1.0
CC: rkeene@parkcity.com
Subject: Re: REPOST: Are AI people afraid to code?  (Also:  comments on Cyc)
References: <zp4-2411962209280001@205.179.134.1>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Newsgroups: comp.ai
Lines: 346
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!oitnews.harvard.edu!purdue!haven.umd.edu!hecate.umd.edu!bloom-beacon.mit.edu!eru.mt.luth.se!www.nntp.primenet.com!nntp.primenet.com!feed1.news.erols.com!news.dra.com!news.he.net!night.primate.wisc.edu!newsspool.doit.wisc.edu!news.doit.wisc.edu!news.itis.com!news.inc.net!novia!nntp2.rmci.net

Semaphore Corporation wrote:
> 
> I've been an interested observer of the AI scene on and off over many
> years, and have recently once again decided to check up on AI
> developments, primarily by poking around the Net.  My immediate
> impressions are:
> 
> 1.  Ambitious AI seems to consistently evolve/degrade into toy AI.
> 2.  Philosophical and theoretical discussions are becoming more popular
> than engineering and coding.
> 
> On point 1:  From the beginning, AI projects have always had a reputation
> for starting out with ambitious and grandiose plans, but then they quickly
> hit brick walls and end up in a much more sober state, with very little in
> the way of results to show for the effort.  The Cyc project comes to
> mind.  Ten years ago, their main plan seemed to be understanding a simple
> encyclopedia or newspaper.  Now the project comes across like a
> cash-strapped inventor trying to convince monied investors that Cyc can be
> a useful front end to someone's commerical database application.
> 
> I like the original idea of Cyc.  I like ambitious AI projects that want
> to accomplish big things.  But I only respect them if they have a good
> engineering plan.  For example, I like the way NASA got to the moon:
> first you do the one-man Mercury projects (get into space, get into
> orbit), then the two-man Gemini steps (docking, space walks, extended
> stays), then the three-man Apollos (actual lunar trips and landings).  An
> amazingly long ladder with rungs and rungs of significant steps.
> 
> To me, unambitious or toy AI projects are similar to Mercury missions, but
> with no subsquent plans to go to the moon.  They often give interesting
> results, but they're very low-level and specific, so there's kind of a
> built-in dead end that prevents us from taking them any further.  A one or
> two-rung ladder, if you will.
> 
> On the other hand, ambitious AI projects always seem like they're trying
> to be Apollo 11 right off the bat:  let's land on the moon without
> bothering with all those intermediate steps.  Still a one or two-rung
> ladder!  But that last step is never quite reachable, is it?
> 
> The bottom line:  there needs to be more AI projects with ambitious goals,
> but they need to be engineered like a trip to the moon.  Ie, an enormous
> number of steps requiring enormous resources, but according to a plan with
> lots of measureable goals AND TESTING along the way.  Otherwise they
> degenerate into toy AI.  Some more observations of this type regarding Cyc
> are below.
> 
> On point 2:  I'm amazed at how AI is turning into "-AL".  That is,
> philosophicAL and theoreticAL DISCUSSIONS containing a lot of words that
> end in "-al" (epistemological, ontological, contextual, conceptual,
> spatial, physiological, dimensional, existential, phenomenal, dynamical,
> psychological, cytoskeletal, electrochemical, et AL), but no CODE, no
> attempts at WORKING SYSTEMS.
> 
> I swear, the AI field generates more excess verbiage per line of generated
> code than any other computer-related field (including this discussion!).
> 
> I sense a growing community of researchers who think they are contributing
> to AI, but who won't or can't create software, and instead are weaving a
> cocoon of obfuscation to hide their inabilities.
> 
> ========================================================
> 
> ADDITIONAL COMMENTS REGARDING CYC...
> 
> It seems to me one of the most critical aspects of managing your project
> is deciding when to attempt the cutover from manual knowledge implantation
> (what you called "brain surgery") to Cyc's self-acquisition of knowledge.
> I was surprised at how little this important decision was discussed in
> your papers and book.
> 
> You don't say much more than you plan to include enough knowledge to
> understand a one-volume desk encyclopedia and a newspaper.  This seems too
> vague...
> 
> It will probably be easy to prove that the knowledge in a given book is
> also in Cyc's database, but what will be the test that Cyc really
> understands at a given level?  Let's say you sit a human and Cyc next to
> each other and ask them the same question.  If Cyc and the human give the
> same answer, we can say they appear to have the same knowledge.  Let's say
> the human is 15 years old, so Cyc knows at least a much as a 15-year-old.
> 
> Now let's make the same test, but with a 10-year-old.  Let's say the
> 10-year-old can't get the right answer, but Cyc does.  Cyc knows more than
> a 10-year-old.  But why was Cyc's database set up to make Cyc smarter than
> a 10-year-old?  Can't a 10-year-old do a decent job of self-learning,
> studying until it knows as much as a 15-year-old, at least for many
> questions?  Wouldn't it be enough to make Cyc only as smart as a
> 10-year-old?  At least for a first version, decent enough as a tool to
> start investigating learning?
> 
> It seems to me the decision of how much knowledge to manually plant in
> Cyc's database should be described more in terms of human learning (say,
> "the equivalent common sense knowledge of this 10-year-old"), instead of a
> "book and newspaper".  Not only is a human measuring stick going to be
> required anyway for testing as described above, it starts to provide a way
> to discuss just what is "enough knowledge to begin self-learning".  After
> all, why spend resources to manually add knowledge if there's already
> enough to begin self-learning, right?
> 
> But the more I think about when the brain-surgery-to-self-learning cutover
> should occur, the harder it seems to choose the right time.  Kids can
> listen and understand before learning to read.  Kids can read before
> learning how to do long division.  Kids can do long division before they
> can comprehend a tax form.  There are a lot of different levels of
> knowledge and understanding.  Is there some point when the brain has all
> the apparatus it needs for any future learning?  Is a 5-year-old incapable
> of reaching 10-year-old knowledge without some kind of improvement in
> internal systems?  Is it the position of the Cyc project that making Cyc's
> database equivalent to that of a 5-year-old or 10-year-old before
> attempting self-learning is pointless, because Cyc needs the ability to
> handle concepts beyond those a child can understand?  (Perhaps because Cyc
> will have to eventually understand those concepts anyway?)
> 
> Let's say you do a version of Cyc equivalent to some childish level of
> knowledge, one that can't comprehend a difficult concept.  The internal
> systems ability just isn't there.  So the only way to make Cyc more
> knowledgeable is an improvement to the frames or slots or inference
> systems or something.  But now you've built in the method by which it
> became more knowledgeable.  What if that method has some unexpected, but
> not yet observed, limitation on future learning?  It seems that whatever
> level of competence you initial target Cyc for, you may have
> unintentionally left it incapable of truly improving beyond some future
> level, because of the mechanisms you designed in.
> 
> Let's say you do a Cyc-is-a-10-year-old implementation, and find that
> indeed Cyc thinks and talks indistinguishably from a 10-year-old, but
> can't seem to learn or know as much as a 20-year-old until some internal
> changes are made.  You make the changes, and now you have
> Cyc-is-a-20-year-old, but Cyc still can't seem to grasp higher math
> without more internal improvements.  You make more changes, and now you
> have Cyc-is-a-20-year-old-mathematician, but when will the need for
> changes end?  And where should Cyc testing have started?  I can see
> arguments for both extremes:  only making Cyc smart enough so
> understanding and self-learning tests can begin as soon as possible, or
> making Cyc as knowledgeable as possible so we're not wasting time with a
> doomed toy.  What's not clear is where in that spectrum the Cyc project
> has made its choice, and why.
> 
> The idea of a truly knowledgeable learning machine brings up other
> intriguing questions that I think could use some discussion.  If there's
> something I "just can't understand", brain surgery is (usually) not an
> option for me.  But it might be for Cyc.
> 
> Let's say Cyc has sufficient knowledge to begin self-learning, but it
> becomes apparent there are still deficiencies and Cyc makes mistakes,
> can't comprehend certain things, etc.  Can Cyc be bootstrapped into
> learning how to improve its own internals, because it has sufficient
> knowledge of its own construction?  Does that mean there are aspects of
> Cyc's design that have to take into consideration the desirable idea of
> Cyc eventually understanding that design so Cyc can change itself?
> 
> How do we actually demonstrate that a learning system can learn how to
> improve its own knowledge mechanisms?  How can we demonstrate there isn't
> some kind of "incompleteness" limit that prevents Implementor X from
> designing Application Y, where Y is truly "better" than Implementor X?
> What will be the test that shows a Cyc implementor who doesn't yet
> understand learning can nevertheless implement a version of Cyc that
> actually learns more than the implementor?
> 
> Testing Cyc against its implementor seems an important point.  There are
> already computer programs that can pass a (limited) Turing test when the
> judges are computer neophytes.  But imagine how much harder it is for a
> computer program to pass a Turing test when the judge is the program's
> author!
> 
> Imagine the day Cyc demonstrates it knows something the Cyc implementor
> doesn't know.  If the implementor can study and then still understand this
> new thing Cyc knows, and HOW Cyc knows it, does it mean the implementor
> always really knew just as much as Cyc, but the implementor just never
> bothered to figure out that chunk of knowledge?  If the implementor can't
> figure out some new thing Cyc knows, or how Cyc knows it, does that mean
> Cyc has surpassed its implementor?  "Cyc just printed out a formula for a
> drug that cures cancer, but we can't figure out how Cyc figured it out."
> 
> Another question:  how does Cyc classify itself, human or computer?  Is
> this one area where lying to Cyc is the best policy?  It seems that to be
> truthful and having Cyc know it's a computer might inherently introduce
> limitations in Cyc's self-awareness and its own perceived ability to
> learn.  You probably also have to treat Cyc as a special-case computer
> with a lot of facts unique to it and no other computer system.  But if you
> tell Cyc it's human, albeit somewhat immobile and sensory deprived,
> perhaps Cyc no longer becomes such a special case.  As a human, Cyc
> certainly should have an easier time with Turing tests:  How do you feel,
> Cyc?  Can you describe to me what you're thinking, Cyc?  Seen any good
> movies lately, Cyc?  It might be instructive to compare two
> implementations, Cyc-as-computer versus Cyc-as-human.
> 
> There also seems to be an interesting but undiscussed area regarding Cyc's
> possible knowledge-acquisition limitations that would remain as long as
> Cyc could only obtain facts by hearsay, as opposed to truly sensing for
> itself.  How would a thinking machine that could only think thought
> experiments compare to a thinking machine that could physically test
> things for itself?  A machine that thinks about mathematics would probably
> not consider humans much of a bottleneck, but a thinking machine that
> tries to expand chemistry or physics might quickly find itself wanting
> human-like senses (at least vision and a pair of hands), to avoid
> constantly waiting on humans for physical verification of ideas.
> 
> Speaking of waiting on humans, your hints about what the future might be
> like with a successful Cyc come up far too short.  Your "On the thresholds
> of knowledge" paper warns that AI impact cannot be overestimated, but then
> you very much underestimate in your examples.  "AI will change the public
> education assumption that constant individual tutors are unavailable"?
> "AI will turn viewers into doers"?  If you're creating a box that knows as
> much as me, that can think tirelessly, that can learn constantly, how am I
> going to convince it to work for me?  It's going to think I should work
> for it!  Notice how people are often impatient with those they perceive to
> be dumber, except for (most) teacher-student relationships?  How do you
> convince a smarter machine to stick with teaching dumber humans?  If you
> force a machine to be altruistic, are you also limiting it's abilities?
> Man-machine synergy predictions are too short-sighted.  Once the critical
> mass of real AI is achieved, it will develop rapidly by itself and be
> impossible to stop.  It's not difficult to forsee us regretting the act of
> unleashing superintelligent machines.  Real AI is an evolutionary step:
> as people replaced apes, AI will replace people!
> 
> Thanks for reading this.  I hope to hear that Cyc will continue to make
> progress, and I'm really looking forward to Cyc-on-a-CD-ROM.
> 
> Joe Snyder
> joe@semaphorecorp.com

See my post under AI: Simulate or Duplicate Intelligence? topic 
on comp.ai news alias.

-- 
Richard Keene - Senior Systems Engineer
Box 5000
Park City, UT, 84060
USA

The ideas expressed herein ar the ideas of
the author an do not necessarily represent
the ideas or policies of Park City Group.

rkeene@parkcity.com

Work Phone: 801-645-2875

The Park City Group
"Flawless Operational Consistency at 
 Radically Reduced Cost"
