Exploring Tekkotsu Programming on Mobile Robots

Exploring Tekkotsu Programming on Mobile Robots:

Visual Routines / Dual-Coding Representations

"Visual routines", as defined by Shimon Ullman (1984), are a set of elementary operations thought to underlie early stages of processing in the mammalian visual system. Many other writers have adopted the term, often using it to refer to low-level primitives in machine vision. It is in this sense that we use it in Tekkotsu.

Tekkotsu visual routines provide operations such as connected components labeling, region intersection or merge, neighbor-sum, flood-fill, and bounded distance computation. Their design has been influenced by the work of James Mahoney on visual routines when he was at Xerox PARC (see Forbus et al. paper below.)

The set of primitives is easily extensible, The basic data structure is the "sketch", which holds a 2D array of pixels of a specified type. Sketch objects are actually smart pointers. Internal functions handle all storage allocation and reference counting chores, and the usual arithmetic and logical operators are overloaded to operate on sketches in an intuitive way.

Tekkotsu actually takes a "dual coding" approach to representation. The term comes from the work of the psychologist Alan Paivio on mental representations. "Sketches" provide an iconic code, while "shapes" provide a symbolic one. Paivio argued that people use both kinds of representations, and when necessary, convert between one and the other. In Tekkotsu, sketches exist in a "sketch space" and shapes in an associated "shape space". Extraction-operators can extract shapes (such as lines or ellipses) from sketches, while rendering operators produce a sketch from a shape.

Dual-Coding Haiku:
Sketch space and shape space:
One has pixels, one has lines.
Back and forth we go.
-- Thomas Tuttle

There are several sketch space/shape space pairs built in to Tekkotsu. The most basic is camera space, which corresponds directly to camera coordinates. Local space is based on egocentric (body-based) coordinates, and world space is based on allocentric (world) coordinates. A tool called the SketchGUI, which can be launched from the ControllerGUI, allows you to remotely examine any of these spaces and see the sketches and shapes that populate them.

Where to Find the Documentation

The dual-coding vision component of Tekkotsu lives in a namespace called DualCoding, which has separate online documentation from the main Tekkotsu system. From the Tekkotsu home page, click on "Reference" to reach the online documentation. The, under "Library Sub-Documentation", click on DualCoding.

Visual Routines Table of Contents

The following topics are covered in the Visual Routines chapter of this tutorial:

VisualRoutinesBehavior and the SketchGUI tool
Sketch primitives
Shape primitives
Shape predicates and functors
Mixing sketch and shape operations
The Map Builder
The Lookout

References

Forbus, K. D., Mahoney, J. V., and Dill, K. (2001) How qualitative spatial reasoning can improve strategy game AIs. This is a longer, unpublished version of a paper that appears in shorter form in IEEE Intelligent Systems 17(4):25-30 (2002).
Halelamien, N. S. (2004) Visual Routines for Spatial Cognition on a Mobile Robot. Senior honors thesis, Computer Science Department, Carnegie Mellon University.
Paivio, A, (1986) Mental Representations: A Dual-Coding Approach. New York: Oxford University Press.
Ullman, S. (1984) Visual routines. Cognition 18:97-159.
Ullman, S. (1996) High-Level Vision. See chapter 9, "Visual cognition and visual routines". Cambridge, MA: MIT Press.

Prev: Tekkotsu Simulator

Up: Contents

Next: VisualRoutinesStateNode

Last modified: Fri Mar 5 05:59:00 EST 2010