Card Picture

Vegas Vaibo

Black Jack Dealer Extraordinaire  

Leigh Ann Sudol, Dylan Hower, and Je Woong Heo

Cognitive Robotics Final Project Spring 2007



Project Description:

Vegas Vaibo is a combination of a variety of principles learned in the Cognitive Robotics class.  He deals with vision, kinematics, learning, and involved much thought and work into optimizing his environment for the problem posed (dealing blackjack)

Click the links below to find out more about each aspect of the problem.

Sony Aibo | The Game of BlackJack | Tekkotsu | Vision | Motion | Intelligence | Group Comments/Reflection | Future Work | Media Gallery
| | Source Code

Media Gallery

Vaibo Acquiring a card | Vaibo dealing to the player | Vaibo dealing to the opponent

Face Card:

Vaibo recognizes a face card due to the amount of yellow blobs he finds.  Because in BlackJack the suit or actual value of the face card does not matter, Vaibo only needs to recognize that it is a face card, and not what the exact card is.

Diamond 6:

In Cam Space with Red cards Vaibo identifies the pink blobs larger than a specific size in order to determine what
is the value of the card he is looking at.

5 of Spades:

Here is a look at Vaibo's identification of blobs to count the value of the card.  Because the small spade under the closest 5 is smaller than the threshold, it is ignored when determining the value of the card.


The Sony Aibo

The Sony Aibo ERS-7 is a Robotic Pet that can be programmed to perform any series of actions.  It contains a variety of motors and sensors that can be controlled through the software.

More information on the Aibo can be found at:

Return to Top

The Game of BlackJack

The game of BlackJack is a game of chance where there is a statistically correct strategy for playing.  The Aibo plays the dealer role against a single opponent.

The dealer deals a single card to each player (the aibo and himself) and then one more to the player, and a face down card to himself.  The player then has the option to hit or stay repeatedly until he either goes over 21 or until he is satisfied with his hand.  The dealer then reveals his card and hits repeatedly or stays.  The player closest to 21 without going over wins.

More information on the game can be found at:
BlackJack Info

Return to Top


Tekkotsu is an open source development framework for the Sony Aibo.  Tekkotsu provides a framework for interacting with the robot on a very high level.  You can find out more information about Tekkotsu through the link below.

Tekkotsu Homepage

Return to Top


Vegas Vaibo uses vision to do a few tasks throughout the game.  

Card Recognition - Before he deals to himself or players, he sees the card and counts how many blobs are there in the card. To avoid seeing noises as blobs, he sets the center point of the card and count the blobs around the center with certain area only.  Aibo recognizes face cards due to the other colors present on a face card other than red or black.  

Player Hit/Stay - The player communicates a hit or stay command to the Aibo using either a orange or yellow easter egg.  An orange egg indicates a hit command and a yellow egg indicates a stay command.

References used in the card recognition:
IEEE Paper on Computer Vision and BlackJack

Return to Top


Vegas Viabo uses several motion sequences in order to complete his tasks.

DealPlayer - The DealPlayer motion sequence starts with Aibo having a card in his mouth and moves him away from the Card Feeder, across to where his opponent would be seated, drops the card and then returns him to the dealer's home.

DealDealer - The DealDealer motion sequence starts with Aibo having a card in his mouth and moves him away from the Card Feeder to his card pile to his left.  He drops the card and then returns to home.

GrabCard - The GrabCard motion sequence starts from alignment for recognition.  It then opens the jaw, raises the head, moves forward to the card, and then closes the jaw to acquire the card.

Return to Top


Vegas Vaibo has two states that require him to make decisions.

Win/Loss - when the player goes over 21, or after the Aibo has dealt himself the win/loss state of the robot decides whether the player or dealer has won.  Because card recognition is done at the time of dealing, the Aibo simply accesses the numeric value stored in his memory to decide the winner.  No vision is required for this operation.

Dealer Logic - This is the cognitive part of the Aibo.  When initially started he has a 50% chance of hit or stay for each numeric value between 2 and 20.  As he plays he updates percentages for each value so that his choices become more intellegent over time.  We used a file written to the memory stick in order to record values and maintain changes over time.  For the demo we are presetting the weights for the aibo as they would look after approximately 100 games.

Return to Top

Group Comments/Reflections

Throughout the process of creating the dealer the group encountered and solved a variety of problems.  We endeavored to modify the environment of the Aibo rather than change the rules of the game, and with the exception of dealing all cards face up for recognition purposes we have stayed true to the rules of blackjack, including the sequence of cards to be dealt - dealing first the player, then the dealer and then the player again.  We modified one small aspect in that the dealer's second card is not dealt until after the player chooses to hit or stay.  Statistically, since the player cannot see the second dealer's card until after they have hit or stayed there should be no statistical difference on the outcome of the game.

Challenge #1 Card Manipulation
- Since cards are flat and would be extremely difficult for the Aibo to push on his own or to pick up we explored many different options in terms of manipulating the cards.  The following are some of the options we explored on the way to our final solution.

* A card shuttle - The card would sit inside of a "boat" shaped shuttle that could be pushed by the aibo.  At this time we were considering methods of card retrieval and discarded this as not feasable in that the aibo could not easily retrieve the card.

* A card pusher - The card pusher was a construct that was built as a enclosure for the Aibo that he could manipulate easily by hooking his chin over part of the pusher.  It allowed him to move cards forward, backwards, left and right simply by moving himself and maintaining the chin posture.  We discarded this as an overcomplication of the problem when we decided not to retrieve the cards.

* A card feeder - The card feeder is our final solution.  It is designed to be the optimal height for Aibo to be able to raise his chin and pick up a card in his mouth.  The cards are displayed at an angle so that Aibo can also do card recognition before he picks up the card.  In order to solve ambient lighting problems we created a light bar above and centered on the card.  The light bar increased the lighting to make card recognition more accurate, while reducing glare with the distance away from the card.

Challenge #2 Card Recognition and Lighting - We constantly ran into difficulties with the ambient lighting and decided we needed a more consistant form of light.  This became a light bar that was placed on the card pusher directly above the card. This caused too much of a glare on the card so when we moved the lightbar to the card feeder we raised it above the card and re-orientated the bar so that the lights run down the middle of the card.

Overall we feel that this project was an excellent choice as a final project.  It combines almost every topic covered in the course including vision, motion, recognition, navigation, and cognition.  

Return to Top

Future Work

Future projects that continue this work could include:
Recognition of multiple opponents
Collection of cards after a hand
Recognition of number of chips involved in a bet
Distrobution/collection of chips after a hand

Return to Top


Source Code











Return to Top