We have modeled TOAST's behavior as a composition of various bindings and interleavings with a basic policy for a schematic environment. In the case of TOAST, the basic policy is simple enough to be implemented by table-lookup. The hard part is implementing the bindings and interleavings given realistic limitations on short-term memory and perceptual bandwidth.
One approach would be to assume a relatively complete representation of the world. Each egg would be represented by a logical constant and its state would be represented by a set of propositions involving that constant. A binding would be implemented as a frame structure or a set of variables that point at the logical constants. The problem is that this approach presupposes the underlying perceptual and motor systems maintain a correspondence between logical constants and eggs in the world. When one of the eggs changes, the visual system has to know to be looking at it and to update the assertions about the egg in the model.
This is not an assumption to be taken lightly. The capacity of the human perceptual system to keep track of objects in the world is extremely limited. Ballard et al. [6]found that their experimental subjects adopt strategies that minimized the amount of world state they needed to track internally, preferring to rescan the environment when information is needed rather than memorize it in advance. The environment could even be modified during saccadic eye movements without the subjects noticing.
An alternative is to treat the limitations of the body, its locality in space, and its limited attentional and motor resources as a resource for implementing bindings directly. A person can visually focus on one object, stand in one place, and grasp at most a few objects at any one time. The orientation of the body's parts relative to the environment can be used to encode the selection of objects being operated on at the moment. In other words, it can implement a binding. Actions of the body, gaze shifts, and movements to new places can be used to shift that binding.
Another alternative is to use the states and relationships of objects in the world to keep track of bindings. An egg is being cooked if it is in the frying pan. A fork is available for use if it is in the drawer, but not if it is in the sink waiting to be washed.
In this section, we will model the use of the body and conventions to implement bindings and interleavings. To simplify the presentation and to be concrete, we will focus on materials, particularly eggs.