Exploring Tekkotsu Programming on Mobile Robots:

Troubleshooting and Debugging

Up: Contents

Common Troubleshooting Problems

The AIBO will not boot

When the AIBO boots Tekkotsu successfully, it emits a short growl and the LEDs begin to pulse, indicating that the robot is in Emergency Stop mode. If it fails to do this, or if it crashes during the boot process:
  1. Check that the battery is properly charged, or try booting the robot while it is attached to its charger. A battery that is almost empty will cause the robot to crash on boot.

  2. Check that the wireless switch on the belly of the ERS-7 is in the "on" position. Tekkotsu cannot boot unless the wireless is enabled.

  3. Check that you have a properly-formatted memory stick with the Tekkotsu software installed. If you have doubts, try a different stick, or try the same stick in a different robot.

ControllerGUI cannot connect

  1. Check that you do not already have a ControllerGUI running; perhaps you iconified it and then forgot about it, or perhaps you killed the window but the program is still partially alive. Use this command to check for ControllerGUI instances:
    ps -A | grep java
    Use the kill command to kill any stragglers.

  2. Check whether a ControllerGUI was left running on some other computer, and is reestablishing a connection to your robot each time you reboot.

  3. Check whether your robot has the correct IP address. If you're using DHCP to assign the robot a static IP, a transient wireless glitch during boot could cause it to pick up the default IP address, which is of form 169.254.yyy.zzz, instead of the correct one. If this happens, reboot the dog, and if it recurs, check your wireless network and router.

    To check the IP address, take the robot out of Emergency Stop mode. Then, simultaneously press and hold the head and chin buttons for two seconds, and the robot will speak its IP address.

  4. Check that your wireless parameters are configured correctly. Are you relying on a DHCP server, or are you trying to set a static IP address? Look in ~/project/ms/open-r/system/conf/wlandflt.txt for the current settings.

The AIBO crashes

  1. The first step in diagnosing a crash is to telnet to the AIBO's console (port 59000) to look for any error messages printed out before the crash occurs. But note: due to the way console output is buffered, you may not see the last few messages that were sent. Console output is therefore not definitive for diagnosis, but it's often helpful.

  2. Overloading the motors will cause the AIBO to shutdown, which looks like a crash. The error message on the console will say "OVirtualRobotComm : JAM. SHUTDOWN...". This can happen if you try to get the robot into a posture it's not strong enough to reach, e.g., if you try to get it to slide its legs together while its full weight is on them. Standing upright on all four legs can also be a difficult posture to maintain for a long time, since if a leg slips or twitches a little, the robot is not strong enough to force it back to the desired position. The robot's normal walking posture, which places more of the weight on the forearms, is a more stable position.

  3. Another common cause of crashes is a behavior's DoStart function not calling the parent class's DoStart. BehaviorBase::DoStart() increments a reference counter that keeps a running behavior from being deleted, so failing to call the parent DoStart can lead to memory corruption.

  4. A memory leak will eventually cause a crash when memory allocation fails. You can check memory usage by going to Root Control > Status Reports

  5. When the AIBO crashes, it writes a core dump to the memory stick. You can use the crashDebug script to examine this core dump and display a pretty good guess of which function was executing when the crash occurred. To do this, put the memory stick in the reader, and cd to the project directory from which the stick was generated, typically ~/project. You must be in this directory when you run crashDebug. To run the script, type
    It will quickly print out a crash analysis showing where the crash occurred. Then it will try to disassemble parts of the code; this can take a very long time. Just quit out of the program (by typing control-C).

    serr and port 10002 simulator; gdb "catch throw" hard-loop crashes: wait 5 mins

The simulator crashes

First, see the discussion of "The AIBO crashes" above. If the simulator crashes, there is another tool you can use to quickly display a backtrace: addr2line. When the crash occurs, the simulator will print out a stack trace like this:
Stack trace: 0x807e349 0xffffe420 0x8123175 0x812307c 0x8176dc8 0x8177543 0x4b1021d5
To translate this into readable information, cd to the directory containing the simulator executable file, say, tekkotsu-ERS7, use the addr2line command, and paste in executable name and the hex codes from the stack trace, like this:
% addr2line -e tekkotsu-ERS7  0x807e349 0xffffe420 0x8123175 0x812307c 0x8176dc8 0x8177543 0x4b1021d5

The addr2line command is currently available for Linux (and cygwin?) systems only; there is not yet an equivalent command for the iMac.

The trace_lookup command does the same kind of stack tracing using AIBO executable code instead of the simulator executable. It is available on all architectures, and may be used as an alternative to crashDebug if the AIBO prints a stack trace on its console before crashing.

The AIBO's ears are twitching

This means the battery is getting low. The faster the twitching, the lower the battery level. When the ears are twitching rapidly, the AIBO will crash soon. To continue running, place the robot on the charger.

Memory stick problems

  1. Make sure you're using the correct type of memory stick. Only Sony pink sticks created specifically for AIBO programming will work. You cannot reuse other color memory sticks containing proprietary Sony software.

  2. If you get errors when trying to write the memory stick, make sure that the little sliding write-lock tab is in the "unlocked" position (no red showing).

  3. Sometimes memory sticks become corrupted and the AIBO can no longer boot from that stick. If this occurs, cd to your ~/project directory and do a "make newstick" to reformat the stick.

    On Linux systems you can do /sbin/fsck.msdos to check for corruption.

There is no segmented camera image; SegCam is black

This will occur if the system failed to find the color threshold file or color names file, perhaps due to a typo in the tekkotsu.cfg file. Look back in the console log (port 59000) and you should see a message of form "ERROR: Could not load threshold file" or "ERROR: Could not load colors file".

The camera image doesn't look right

  1. If the image is too bright or too dark, adjust the camera gain, shutter speed, or lighting model.

  2. On the ERS-7 you will notice the corners of the camera image have a blue tint. There is no simple fix for this; someone will have to develop color correction code.

  3. Re-do the color segmentation.

The compiler is complaining about Tekkotsu source files

This is probably caused by a missing close delimeter in one of your own source files messing up the parsing of some subsequent Tekkotsu file being loaded by a #include.

After "cvs update", user code won't compile or won't link

  1. The Makefile may have changed. Be sure you have the latest Makefile in your project directory by copying it from the Tekkotsu/project directory.

  2. Other files normally copied into your project directory may have been updated, such as StartupBehavior_SetpuVision.cc. Be sure that you have the latest versions of these files in your project directory by copying them from the Tekkotsu/project directory.

  3. Changes may have been made to the Environment.conf file in your project directory. Make sure the pathnames and target platform selected in this file are correct.

My state machine isn't working

Use the Tekkotsu event logger to monitor the behavior of your state machine. From the ControllerGUI go to Root Control > Status Reports > Event Logger, and select statemachineEGID and stateTransitionEGID events. If you're using SignalTrans you should also select stateSignalEGID events. Then run your state machine and monitor the console output to see what it's doing.

Up: Contents

Last modified: Mon Jan 26 22:25:23 EST 2009