Sorting Cards using SIFT
Danielle Millett
Problem Description and Introduction
The objective is for the aibo to sort playing cards according to whether they are a face card or not. Playing cards are taped to styrofoam blocks for easy manipulation and are scatter throughout the play pen. The aibo needs to find a card, then move close enough to it to be able to take a picture of the card. The card is then identified as either a face card or rank card using SIFT. Once identified the aibo pushes it to the appropriate goal location, marked by either a yellow or orange cone. Then repeats the process until all cards have been sorted.
Approach
There are several different task involved in the project. They are broken up as follows:
Finding A Card
 |
The cards are attatched to styrofoam blocks using pink tape. This allows the aibo to identify them by searching for pink blobs. The aibo searches around the play pen for pink blobs and chooses the first one above a certain size threashold. Once the card is found, the aibo continuously updates so that it is always pointing its head toward the card while it advances toward the card. Once the robot has come within a certain distance of the card, it stops and sets itself in a position to identify the card. This is done by sitting up on its hind legs, then searching again for the pink on the card so that it can point its head in the direction of the card so it has a good view of the card. |
| AIBO viewing card |
Card Identification using SIFT
Once it has a good view of the card it takes a snap shot of the card and sends it to a simulator running on the computer. This allows it to be put through SIFT to identify the type of card. SIFT stands for Scale-Invariant Feature Transform and is an object recognition algorithm. It works by identifying the features of training images which are of various scales and orientations. These features are then stored in a library that is then used for identifying new images. When identifying a new image, it finds clusters of features in the new image that agree with the features present in the training images. Below are examples of the features detected my sift for a King of Clubs and Three of Spades.
|
 |
| Card as viewed from AIBO |
 |
 |
 |
 |
The advantages of SIFT are that it is independent of scale, translation, rotation, and occlusion, which allows it to identify and object in a variety of different situations. For this project I trained sift on images of kings & queens of clubs & spades for face cards and two & threes of clubs & spades for rank cards. When an image is given to SIFT it compares the features in the images to the database of cards and then returns if there is a face or a rank card in the image. This information is sent back to the AIBO and used to identify the card.
Card Manipulation
 |
Once the card has been identified, the aibo must move the card to the appropriate goal. The goal for a face card is marked by and orange cone, and the goal for a rank card is marked by a yellow cone. To manipulate the card, the aibo squeezes the card with its front legs, and walks using its back legs (sliding along with its front legs) walkFront.mov The aibo searches for the appropriate goal, and then continuously updates to look at the goal as it pushes the card towards it. Once the aibo is close enough to the goal it opens its arms to set the card down and back away. After this the aibo turns around to look for another card to identify. |
| AIBO pushing card |
Results
Pushes card to Cone and Turn AroundFull run of pushing to Cone and searching for next Card
finalProj.h - Code run on AIBO: world navigation
finalProjSIFT.h - Code run on Computer: SIFT identification
Areas for Improvement
I ran into difficultly sending the image from the AIBO to the simulator on the computer to be able to run SIFT on the image. Because of this, the current version of the project isn't able to actually send the image to the computer, although the identification of the cards works fine on its own. Further work on the project would be to get the data transfer working. Also, the project could also be applied to other objects that are of similar size, or to sort playing cards in a different manner.