HELP CHARS_TO_ITEMS                          Jocelyn Paine November 1992


Module chars_to_items exports a routine of the same name for converting
a list of characters into a list of items. It is useful for itemising
sentences read by a bug. You can load it by doing
    lib chars_to_items;
If you are using Eden, this will have been done for you.


Introduction
------------

Novices may not know what ``itemising'' means. It's the process of
converting characters into numbers, names, and other basic Pop-11 units,
and is often called tokenisation or lexical analysis by
compiler-writers. Thus:
    Characters: 'I saw 2 dogs'
    items:      [% "I" "saw" 2 "dogs" %]
It is useful here, because (in most cases), Pop-11 items correspond to
words in English. Hence itemising is a quick way to convert a continuous
stream of characters into a list whose elements you can treat as English
words.


Exported procedures
-------------------   

PUBLIC chars_to_items( chars ):

chars is a list or vector of characters, or a string. The result is the
corresponding list of Pop-11 items.


Example
-------

Here is a bug which responds to the queries
    where is the food?
    what are you holding?


    define think();
        lvars sent;

        chars_to_items( sentence() ) -> sent;

        if sent matches [ where is the food ? ] then
            say( [ the food is ^smell() ] )
        elseif sent matches [ what are you holding ? ] then
            if inventory() = ` ` then
                say ( [ i am holding nothing ] )
            else
                say( [ i am holding % consword(inventory(),1) % ] )
            endif
        endif;

        if inventory() = `+` then
            "use" -> action
        elseif smell() = "here" then
            "grab" -> action
        else
            smell() -> action;
        endif;

    enddefine;
