Sensing Projects

The Master Plan

Here are the slides I showed.

Depth Camera

Currently the Bluetechnix Sentis M100 depth camera is our best option. It can operate at very close range (up to a few cm) in full sunlight. When it becomes available, the Sentis P510 adds more illumination (helps deal with sunlight) and a color visible light camera.

Here is a movie of simulated grasping.

Here is the state of things.

There is an M100 depth camera in the DRC lab for you to work with.
Manipulation: The camera out of the box works well for things within 20cm even in strong sunlight, so vision for manipulation visual servoing is ready to go.
Locomotion does not work well in strong sunlight yet. I am working on this.
Aaron is going to find quick and dirty ways to mount the camera on the robot for using it for manipulation and locomtion.

How to program the camera? The camera has an ethernet interface. There is a TCP/IP control interface, and visual data is sent as UDP packets. To talk to the depth camera, you need to set an ethernet port to:

address 192.168.0.1
mask: 255.255.255.0
gateway 192.168.0.1
no DNS servers

All the info and code I developed for the camera is on the machine drc-dev-2 (130.215.73.140) in /home/cga/m100. You can log in using the standard atlas account and password. Here is info on what is there:

m100/manual      all the manuals. some are just html files.

m100/application    Bluetechnix example code
m100/application/BltTofVisualizer_V3.0.0  Windows demo code I found very
                                          useful to run the camera and see
                                          what it was doing.
m100/application/GraphicalStreamAnalyserLite  Windows demo code. Does
                                              something.
m100/application/Sentis-ToF-M100-Visualizer   Windows demo code. Does
                                              something
m100/application/Sentis-ToF-M100-Visualizer_V2.0.0_source_20130814
             Windows source
m100/application/sentis_tof_m100_api_lib_20140115  API and example code
                         that can be compiled for both Linux and Windows
m100/application/argos3d_p100_ros_pkg-master   ROS package for closely
             related P100 sensor, Check on web for more recent source.

m100/bin  compiled versions of
             m100/application/sentis_tof_m100_api_lib_20140115 for linux.

m100/firmware    presumably stuff related to firmware. I ignored it.

m100/test1/       my code. I wrote my own UDP multicast packet receiver.
                  I used BltTofVisualizer_V3.0.0 on a windows machine to
                  set the camera up, and then used a Linux machine to
                  actually collect data.
m100/test1/test1.c    My basic test program. Grabs frames.
m100/test1/collect-average.c   Averages depth images
m100/test1/make-movie.c        How I made the exciting movies.
              Takes 2000 frames and writes them to ./tmp
m100/test1/make-png.c make-png2.c sh-make-png.c utilities.c: utility
                                                             programs
m100/test1/data data for the movies
              Each directory has a .dat file for each frame.
              The depth and intensity data is originally 16 bit unsigned
              integers (depth in mm and intensity in unknown units).
              My .dat format just has everything as floats. First the
              entire depth image, and then the entire intensity image.
              The m*.png files are the frames of the movies (both
              intensity and depth in one image). The d*.png are just the
              depth images, and the i*.png are just the intensity images.
              Note that you have to make arbitrary scaling choices to
              generate these images. You can edit the program
              make-png2.c to change the choices and regenerate the
              .png files. (see m100/doit).

If you let the camera run continuously, it heats up and may overheat. Don't just let it run continuously at full blast. You can set the frame rate to 1Hz or just unplug it when not in use.

Here are things to work on:

Better handling of sunlight. IR filtering. Polarization filtering. More powerful illumination. Combination of images at multiple exposures. Task oriented adaptive gain control
Use of free core on the camera processor to do vision processing
New camera firmware that suits our needs
Use of multiple depth cameras
Combination of multiple RGB and depth cameras
Put the camera on the robot, and start using it for manipulation tasks.

Range Sensor Arrays

Use new digital Sharp sensor GP2Y0E02B with I²C bus. The arrangement of sensors in the array might look like this:

...

Programming Guide

Related Product GP2Y0E03 with both analog out and I²C bus.

My Arduino code to read the analog out of the GP2Y0E03.

Arduino note: you need to set the baud rate of the terminal window to match the baud rate set by Seril.begin(xxx) in your program.

Arduino Wire Library is used to drive the I²C bus.

Here are things to work on:

Get I²C working with these sensors. My code to do this on an Arduino Due, which has the desired 3.3V logic levels. Connections are:

Arduino 3.3V - Pin 1 (red wire) on GP2Y0E02B.
Arduino GND - Pin 2 (black wire) on GP2Y0E02B.
Arduino pin 20 SDA - Pin 3 (yellow wire) on GP2Y0E02B.
Arduino pin 21 SCL - Pin 4 (green wire) on GP2Y0E02B.

Figure out how to program 16 different I²C addresses. I failed to do it with this code and manually applying 3.3V to the Vpp trace exposed on the back of the GP2Y0E02B.
Test how well the array works, especially in sunlight.
Put the array on the robot, and start using it for manipulation tasks.

Distributed IMUs

We would like to put IMUs on every link connected by a SPI network (Arduino SPI Library).

We are currently using the MPU-6500 from Invensense.

USB Evaluation Board: Invensense documentation and software. Read notes.txt to get descriptions of the software. The USB evaluation board use UDL1-1.zip which works on only some versions of Windows.

Small Evaluation Board: Made by embeddedmasters.com. See the data sheet and schematic available from that page. Note that you have to remove jumpers to make SPI work.

Here are things to work on:

Get I²C working with these sensors. Seems to me that only two devices can be on the same I²C bus, since there is only one address bit (AD0) that can be set to change the address. This is a fatal problem for the I²C interface. This code works for the MPU6500 to read raw data using I2C.
Get SPI working with these sensors. Note that you have to remove jumpers to make SPI work on the embeddedmasters.com board.
Figure out whether I²C or SPI is the way to go. If SPI can be made to work, we will go with that. 20 MHz interface speed, and many more devices on the same bus.
Put the sensors on the robot.

Slip and Force Sensing Using Optical Mice

We can put an optical mouse in the sole of the foot pointed upwards to measure sideways force (the amount of soft material deformation) and slip (the amount the ground moves under the foot), as is shown in this picture:

Currently we are using an Avago ADNS 9800 chip which is sold in this circuit board with this code. We are going to need to set up an SPI network to handle multiple devices (Arduino SPI Library). I used an Arduino Due board with the following pinout:

VI - 3.3V pin on Due [power]
AG - GND pin on Due [analog ground]
DG - GND pin on Due [digital ground]
MOT - pin 8 on Due [interrupt line from chip] (but I never got interrupts to work)
SS - pin 10 on Due [chip select to chip]
MO - ICSP pin 4 on Due [SPI MOSI: Master Out/Slave In]
MI - ICSP pin 1 on Due [SPI MISO: Master In/Slave Out]
SC - ICSP pin 3 on Due [SPI clock]

Where are the ICSP pins on an Arduino?

My Arduino code for the Due to read the mouse. I couldn't get the ADNS9800test-serial code to work since interrupts didn't work. [UPDATE: New code as of Sep 26] [UPDATE: The MOTION line is now working (goes low when data is available), so interrupts should now work. We will probably be polling in actual use, since the computer will be dedicated to this task.]

Arduino note: you need to include My Arduino code in the sketch above for the firmward variables to be defined.

Arduino note: you need to set the baud rate of the terminal window to match the baud rate set by Seril.begin(xxx) in your program.

I would like to be able to use the Cypress Semiconductor chip which is programmable (more info), and we could customize the optical sensor to do exactly what we want

The sensor in the steelseries mouse seems to be a CYONS1002-LBXC. The closest data sheet I could find is the CYONS1001x data sheet. This does not seem to be the programmable version we were looking for, but it is a potential alternative to the Avago device. Here is my code to talk to it.

We could also develop our own custom sensor.

Here are things to work on:

Get N sensors connected to one computer.
Get the sensors on a robot foot.
Evaluate how well they work.
Iterate the design.

Networking

We need to get all this sensor information onto an ethernet wire, ultimately. We can network everything with ethernet (see Arduino ethernet shield) or we can create a USB network with the Arduino Due and its two USB ports.

The version 1.0 network plan: There is an Arduino EtherDue (faster clock, two i2c ports, on board ethernet) on each major link. These are connected by a gigabit switch in the torso. The IMUs are connected by SPI. The range finders are connected by I2C (up to 32 GP2Y0E02B can be connected given two I2C ports). The foot with the optical mice stuff will be a special case. This networking plan has 3-5 ethernet cables crossing major joints (hip, shoulder, knee, elbow). An alternative is to have each Arduino have two network ports, and have a linear chain of Arduinos connected by short ethernet cables. The problems with this approach are 1) we have to make a dual ethernet shield, and 2) if the chain fails anywhere, we lose everything down stream. The V1 network plan is more robust: if an ethernet connection fails, we lose sensing on a single link.

ROS interface

We need to create a ROS interface for all this information.

Cheap Color Cameras

We could develop cheap tsk oriented color camera sensing using Arduinos or Rasberry PIs to process camera data and put it on the ethernet.

Building a new foot

Another way to get Fx, Fy, and Mz on the feet is to build our own foot with our own force sensors. I am considering this option.

WPI Prof. C. Onal is working on Hall effect based sensors. This might be workable with just instrumenting the current Vibram sole.

www.optoforce.com is proposing us using some version of their six axis force torque sensor in a foot shape. Problems are: 1) too compiant? and 2) we don't know if the sensors degrade because of hydraulic oil (they are testing this).

Tecgihan makes a low profile 3 axis force sensor, the USL08-H6-2kN. We would use four for each foot (expensive). Needs big electronics box, and we don't know how it reacts to hydraulic oil.

PCB Piezotronics makes a low profile 3 axis force sensor. They said: To submerge a sensor, we offer the "W" (Water Resistant Cable) option. With this option a sensor can be submerged for a few days in shallow depth. You would then need to clean it and dry it out before submerging again. The only Series 260A that we have made with this option is W260A01/010G10 [Triax ICP force sensor, 1k lb (Fz), 500 lb (Fx,y), 2.5 mV/lb (z), 10 mV/lb (x, y), with attached & sealed water-resistant 10-ft cable].

Being water resistant is not the same thing as being resistant to hydraulic oil.