From newshub.ccs.yorku.ca!ists!helios.physics.utoronto.ca!news-server.csri.toronto.edu!rutgers!cs.utexas.edu!wupost!darwin.sura.net!gatech!bloom-beacon!eru!kth.se!sunic!seunet!kullmar!pkmab!ske Tue Feb 11 15:26:02 EST 1992
Article 3613 of comp.ai.philosophy:
Xref: newshub.ccs.yorku.ca comp.ai.philosophy:3613 sci.philosophy.tech:2106
Path: newshub.ccs.yorku.ca!ists!helios.physics.utoronto.ca!news-server.csri.toronto.edu!rutgers!cs.utexas.edu!wupost!darwin.sura.net!gatech!bloom-beacon!eru!kth.se!sunic!seunet!kullmar!pkmab!ske
>From: ske@pkmab.se (Kristoffer Eriksson)
Newsgroups: comp.ai.philosophy,sci.philosophy.tech
Subject: Re: Table-lookup Chinese speaker
Summary: Different ways to use a table
Message-ID: <6555@pkmab.se>
Date: 8 Feb 92 11:29:39 GMT
References: <1992Jan27.023623.8118@husc3.harvard.edu> <6522@pkmab.se> <1992Jan28.103015.8159@husc3.harvard.edu>
Organization: Peridot Konsult i Mellansverige AB, Oerebro, Sweden
Lines: 324

In article <1992Jan28.103015.8159@husc3.harvard.edu> zeleny@zariski.harvard.edu (Mikhail Zeleny) writes:
>In article <6522@pkmab.se> ske@pkmab.se (Kristoffer Eriksson) writes:
>
>>In article <1992Jan27.023623.8118@husc3.harvard.edu> (Mikhail Zeleny) writes:
>
>MZ
>>> Compare two programs generating the same
>>>kind of output for the same kinds of input (e.g. two sort algorithms).  Why
>>>would the intensional difference in program stricture be relevant to our
>>>issue?
>
>KE:
>>This is fascinating! I suppose my TV-set could be considered equivalent to
>>an enormous table of pre-recorded pictures and sounds, that were stored in
>>the apparatus at the time of fabrication, and now are being poured out in
>>endless streams whenever I power it up. It is absolutely fascinating to
>>think about the foresight of the designers, having been able to supply this
>>marvelous device with all the pictures that it would ever need to give the
>>illusion of reporting of the current happenings all over the world, and
>>showing what is absolutely indistinguishable from live programs, and so on;
>>all this with my remote control as the only input to select the appropriate
>>pictures out of that homungus table. Or is my control a dummy, and has my
>>input also been foreseen? I _knew_ those TV transmitters all over the
>>country were just fakes.
>>
>>I'm just a little worried about the day when my TV runs out of pictures.
>>Surely such a day must come?
>
>Wake up, and pay attention: compare two programs generating the same kind
>of output for the same kinds of input.  You know what input is, don't you?

Where is your sense of humour? Your discussions provoced the above fantasy
on my part, regardless of whether it meets your exact point or not. I still
find it a fascinating thought. Or do you find any logical impossibility in it,
considered on its own?

However, regarding the relevance to your actual discussion, table-driven
AI, I've found it rather unclear exactly what kind of table-based system
the participants in the discussion have in mind, at least throughout the
part of the discussion I've been able to follow. Each participant seems to
have his own idea of what table-based system to discuss. There are quite
differing possibilities. I'll work through all that I can come up with
(there are six).

The simplest possibility is a straight lookup-table, where the current
input (a letter, word, message, ...?) is used as an address into the table,
where the output corresponding to the current input is retrieved, and
nothing more. This is too simple to allow any complicated behaviour, since
it will present the same output each time the same input is repeated, and
will never learn anything new, and can't even remember the context of the
latest input when the next comes along. The table size is determined only
by the number of possible inputs, though. (1)

The next possibility is to extend the current input with a timestamp,
before looking up the current output in the table, and nothing more. This
solution allows the output to never repeat itself in any particular manner,
however it doesn't support any desired repetitions either. Everything that
is going to change with time, has to be known already at table construction
time. It will still not learn anything from experience. The required table
size increases with the desired run time, without bound. This matches my TV
example above, without antenna input, that I found amusing in its
limitedness. (2)

Next, as was recently suggested, one can take all past input together with
the current input, as the adress into the table. Now we start getting some-
where. This is quite powerful, as it definitely allows the system to modify
its behaviour based on past experience to any extent desired. However, it
still requires every possible combination of events, in any sequence, to
be worked out at table construction time, and the table quickly takes on an
absolutely unmanagable size. At the same time, the table will probably
contain vast amounts of redundant data. The table size explodes with the
desired run time, without bound. (3)

These are the most lookup-table-oriented solutions that I can think of.
Still nothing that matches the characteristics of a human: bounded brain
size despite unbounded run time (at least not bounded by the brain running
out of addresses), and ability to learn.

To facilitate learning and adaption to the environment, without having to
work out everything in detail before-hand, we can start introducing some
internal state to the system. But then the system will not consist solely
of a table, but rather of a table plus storage for current state information
coupled some kind of feed-back loop.

First, we can amend the input with the current state information before
looking up the current output, together with new current state information
for the next cycle. This makes the lookup-table into a finite state automata.
Very powerful. Indeed, so powerful that it is dubious to regard it is an
alternative to ordinary computers, which I suppose must have been the
purpose of limiting the discussion to tables. This solution still requires
awkwardly large tables, at least if they are going to have much internal
state and perform complicated operations. The table size depends on the
amount of state information, and will probably contain a lot of redundancy,
but will not run out after any amount of time (unless there is some terminate
state included). The system will of course sooner or later enter the same
state more than once, but presumably that state will be appropriate for the
situation that causes it, every time. After all, there is a limit to the
resolution with which it can perceive the environment. The number of states
might be extended by variable-size external storage, though. (4)

Finally, we can add a large memory, addressible in small chunks, operated
on by a traditional processor and a program. Or alternatively, perhaps a
neural net. This will probably use the least table space for the program,
with the least redundancy, and give the most space to store current state
in without exploding the table size. Like the previous solution, it can run
arbitrarily long on a bounded size program, and will still have enough
storage to almost never enter the same state more than once. If you include
external storage too into view (for instance pencil and paper), then the
state need never repeat. Here the designer will have to program methods of
adapting to new information, rather than specific outputs. But now it is
certainly no "simple lookup-table". (5)

Or, as you seem to have thoughts of, we can make the table rewriteable, and
let the current input look up the current output plus an arbitrary rewrite
of the table. Then the table itself will in principle duplicate the state
information storage of the other solutions. I think this solution is as
powerful as the previous one, except that it may be hard to program if
everything really is viewed only as program, and never as data; and you'd
have to define how each table entry is to describe arbitrarily complex
changes that should be applied to the table. Essentially, though, this is
the same idea that underlies the development of modern computers: store
both program and data in the same, rewriteable memory. So this isn't either
a mere lookup-table. (6)

An interesting observation is that these kind of systems (4-6) can practice
controlled loss of less important memories, in order to continually adapt
to new situations that consume state information, in contrast to the earlier
clean table systems (1-3) that never forget anything, and consequently
eventually fail.

Now, of course my TV story neglects one input that real television has, but
it illustrates the limit of one of the "simple" table approaches, an approach
that at least one or another of the participants in this discussion, seem to
have had in mind for the AI case, although you yourself perhaps have noted
the distinction.

However, no matter which model each one of you respectively have had in mind,
I think the discussion would be well served if everyone tried to be a little
more specific about what kinds of table systems they reason about. I don't
think every-one is talking about the same kind of table.

If you, for instance, want to claim something about very simple and weak table
systems in contrast to modern computers, then you should keep away from the
more advanced table systems, even if they at first seem like small extensions.
Or if you need to use more advanced table systems to obtain reasonably capable
systems, then you can as well speak about fully equipped computers.

>In fact, intensional difference may indeed be important, i.e. when

If I should say something about this too, the subject that I happend to
cite, then I will say that as long as two systems produce identical
output, under all circumstances, then, as far as I can see, they should
be regarded as functionally equivalent (except for secondary characteristics,
like speed, that in practice can be important too), whether they are simple
lookup tables or intricate computer programs.

However, note the provision "under all circumstances". That means, that
they have to produce the same output for all possible sequences of inputs,
whether realised or not. For instance, a simple lookup table, that will
eventuelly run out of table contents, is not equivalent to a more advanced
system that will behave the same up till that point, but then carry on,
possibly for ever. And a random number generator connected to a typewriter,
that in one test happens to reproduce the output of more deterministic
system, is not equivalent to that system (unless you take them out of
service after that). Determining this may require a comparison of their
internal structure, since comparing all possible input sequences may take
forever.

Until you can exactly compare the internal structures of the systems (which
may never become possible), you may be confined to using Turing test like
tests.

---
In <1992Jan21.182524.7880@husc3.harvard.edu> you wrote:
> Thus the notion of a predefined table of meaningful conversations is
>inherently unrealizable, being that the infinitely variable context can't
>*in principle* be fixed in advance.

Here you appear to talk about the simpler table solutions with non-rewrite-
able tables and no state. In a few other places you've had to mention tables
with continuous updates, too, though.

The part beyond the comma, seems doubtful, though. The context may be
infinitely variable, but after quantization onto the inputs of the system,
there is only a limited number of states that the system can perceive the
context as being in. The quantization makes it possible to fixed in advance,
as regards to what the system can perceive of it. Obviously, some information
will get lost, but isn't that always so, and does it necessarily entail that
anything essential is lost? However, if it is one of the time-limited
solutions, then there will of course be a limit to how far the system can
go, perhaps ending in catastrofic failure at some point in time.

---
Jeff Dalton in <1992Jan29.023448.11610@aisb.ed.ac.uk> wrote in reply to Zeleny:
>I still don't see what's wrong with a static table.  How is it
>different from someone who's been asleep (while the table was
>built, say) and is now sitting in a room empty except for a 
>teletype?  What constant update is required?

I can't tell exactly which version of static table you had in mind, but
probably one with no extra frills. If you mean a table with lookup of the
current input alone, or with a timestamp, then I hope you see now that they
have a hard time of remembering what was said a moment ago, and are of
limited time duration, all in contrast to someone who's been asleep. If you
meen a table which takes all past input as input, then it is more or less
impossible to program in practice (unless you simulate it with one of the
other solutions, which really is cheating), and still time limited.

>>I am afraid you misunderstood me.  My point was that, due to the contextual
>>factors, the "conversation state" can't be determined on the basis of words
>>alone.
>
>Why do they have to be addressed after the creation of the table?

Why the conversation state should not be possible to determine on the basis
of words alone, if the conversation is conducted with words alone, I don't
quite see, although there may be limits to how far a meaningful conversation
can be conducted that way without independant contact with the world, and
there will abviously be certain kinds of questions that the artificial system
can not comment on.

The way to address the conversation state is, anyway, either to program every
possibilty in advance and use all past input as the address (an unrealistic
approach, and time-limited), or to change the internal state or program along
the way.

If it is desired that the system should continuously be updated about what
goes on in the surrounding world, then the way to address that should of
course be to add extra input channels for that kind of information, and
count them to the overall input of the system, whether text or something
else. An excerpt from another article suggest that this (data about the
surrounding world) is, in fact, what Zeleny is driving at:

Mikhail Zeleny in <1992Jan22.103309.7900@husc3.harvard.edu> wrote:
>Humans exist within a culture, making constant reference to their
>surroundings.  <---

And background cultural and real-world knowledge, I expect to be part of
the original table that the system starts out with.

---
Jeff Dalton in <1992Jan28.233247.9657@aisb.ed.ac.uk> wrote in reply to Zeleny:
>>This isn't what I had in mind.  Compare two programs generating the same
>>kind of output for the same kinds of input (e.g. two sort algorithms).  Why
>>would the intensional difference in program stricture be relevant to our
>>issue?
>
>It's easy to change my example so that there's I/O or arguments and
>results (which, from your sort example, I take it is sufficient).

>...On the other hand, if we suppose that some programs might produce
>consciousness, it seems to me that it could matter how they work.
>The table lookup machine might be an example of a program that
>passes the Turing Test but wouldn't be conscious.

Compare this to my approach of comparing the output for all possible
sequences of input. With this approach, it doesn't matter whether you
can change internal states into output. Nevertheless, most top-level
construction choices (like the alternatives I've explained), will probably
affect the output for one or another possible input. For instance a
time-limited table solution will not compare equivalently to an unlimited
solution. (By the way, I am not talking about limited versions of
"implementation", or algorithmic step-by-step equivalence now.)

---
Pontus Gagge in <1992Jan31.025035.15035@ida.liu.se> wrote in reply to Zeleny:
>>Not true.  Temporal considerations undermine the very idea of a static
>>table, requiring constant update thereof through some input mechanism.
>
>I fail to see the deadly effect of temporal considerations.

Here we probably see a classical example of each participants of the discussion
refering to a different kind of table based system. I'll say no more.

> I assume you are not merely considering ie questions of "What time is it",
>as this can be remedied by the timer ticks someone (sorry, lost the source)
>proposed incorporating into the input stream:

Yes, that's about the most that timer ticks (with nothing additional) solves
reasonably well. As I have explained, other temporal problems can be solved
too, like "What was it we were talking about?", with more elaborate additions
(as you seem to have anticipated), but the better the solution, the less
static the table. You'll have to work out for yourselfs which one of you wins
this particular point.

>>Read it here: it's [it=table construction] impossible IN THEORY.

I have no idea how Zeleny arrived at this conclusion.

>>Simple: it has native intelligence, insofar, and inasmuch, as it can be
>>ascribed free will by an omniscient observer possessed of a perfect insight
>>into its construction and functioning.
>
>"Simple"? Oh well, no practical test may exist. However, this is
>saying, in effect, that the problem is undecidable

I've complained about this too, but am still waiting for an answer.

>I would greatly welcome a discussion on this; as I do not really myself
>have a good answer to the problem of Turing-cheating. If there is a
>realizable cheat, the Turing Test would lose much, if not most of
>its appeal. Would a suspicious tester always detect the cheats?

I think it should come as no real surprise that the Turing test is
susceptical to cheating with limited data, in the same way as any empirical
experiment is susceptical to the risk that future repetitions may possibly
show a completely different result than earlier experiments did. We haven't
junked science because of that yet, though.

For another kind of answer, look at my principle about producing the same
output for every _possible_ sequence of inputs. However that principle in
itself does not go so far as to give any independent definition of conscious-
ness or intelligence, and on a detailed level, one can't expect all
conscious brains and other devices to produce exactly the same output for
all inputs; it only identifies the functional equivalence between individual
devices, not entire classes of devices (ill-defines classes, at that). We
all have our individual brains, after all. So that question is still open
for discussion. Personally, I would bet on some degree of self-reflection
(including memory) and power of abstraction as essential factors.

-- 
Kristoffer Eriksson, Peridot Konsult AB, Hagagatan 6, S-703 40 Oerebro, Sweden
Phone: +46 19-13 03 60  !  e-mail: ske@pkmab.se
Fax:   +46 19-11 51 03  !  or ...!{uunet,mcsun}!mail.swip.net!kullmar!pkmab!ske


