15-494 Cognitive Robotics
Spring 2011
Course Links:  Main Lectures Labs Homeworks

Tekkotsu Links:   Wiki Tutorial Resources Bugs CVS

Cognitive Robotics: Lab 7

Part I: Using SIFT for Object Recognition

Review the SIFT demo video we looked at previously.

Study the SiftDemoTrain and SiftDemoTest programs in /usr/local/Tekkotsu/Behaviors/Demos/Vision.

Using SiftDemoTrain, teach your robot an object such as a soda can, cellphone, etc. Choose something with lots of edges so SIFT can find good features. Put your object in front of a blank background to avoid spurious features. Collect several different images with slight changes in rotation or distance.

Now use the SiftTest demo to recognize your object in a cluttered environment.

Hand in a screen shot showing the recognized object and the detected features.

Part II: Simple Gestalt Perception

Take 5-6 blue game pieces and lay them out in a circle. Place 1-2 pink game pieces inside the circle, and some outside it. Your task is to decide which pink pieces are in the circle. Use the camera space for this task.

To solve this task, start by extracting ellipses, as you've done before. (Use the MapBuilder for this.) Then form the convex hull of the blue ellipses, using PolygonData::convexHull. Render the resulting Shape<PolygonData> to produce a sketch of the region enclosed by the blue pieces. Check each pink ellipse to see if it falls within the region. Mark the ones that do, and produce a result sketch indicating which ellipses are marked.

Note: you may prefer to take some pictures and then work in the simulator rather than on the robot. That's fine. Also, you can choose any two colors of easter eggs for this problem; you don't have to use blue and pink.

Part III: Non-Convex Boundaries

In the children's game of "tag", a certain region is designated as "home base", and any player standing within it is safe and cannot be tagged. In the image below, where players are represented by green ellipses and the boundary of home base by pieces of pink tape, only one of the two depicted players is safe.

Question 1: What result would convex hull produce if you use it to determine whether a player is safe?

Question 2: What principle(s) of gestalt perception does the convex hull algorithm violate when used on a non-convex apparent boundary? Explain your reasoning.

To correctly determine inside/outside relationships in this case you must construct the boundary of "home base" some other way, without using convex hull. There are various ways you could do this. The best way is to use the line extractor to find pink line segments, and then consider each of the two line endpoints. However, the line extractor may not work well on short, distant lines, so you could use blobs to represent all the little pieces of tape that aren't perceived as lines, and take the centroid of each blob using getCentroid(). Use the minBlobAreas field of your MapBuilderRequest to eliminate any noise. Given this set of line endpoints and blob centroids, put the points in appropriate order to traverse the circumference of the convcave region. (Hint: start with one point, select the point closest to it, select the point closest to that one, etc., until all points have been selected. But once you've selected one endpoint of a line, your next choice must be the other endpoint.) You can then feed a vector of points (vertices) to the PolygonData constructor to make a polygon. Use visops::fillInterior() on the rendered polygon to solve the inside/outside problem.

Finish this problem for homework if you don't complete it during the regular lab period. Hand in your code, your answers to Questions 1 and 2, and a snapshot of the sketch showing your polygon and results by Friday, March 18.

Dave Touretzky and Ethan Tira-Thompson