Interesting Roads Encountered During NHAA
During the trip, we encountered many different road types, despite the fact we
were on the same interstate, I-70, for most of the trip. Some of the roads,
like the two below were like you would expect from an interstate highway, nice
pavement and good lane markings.
RALPH drove at night as well as during the day, using images that looked like
the one on the left below. RALPH was also able to drive in situations where
the road had been freshly paved but not painted yet. In this situation, the
only good feature was the boundary between the road and the off road region.
We encountered rain in Kansas, which caused pretty severe specular reflections
off the pavement, obscuring the lane markings, as can be seen in these two
images. The tracks from previous vehicles were sufficient to allow RALPH to
continue steering accurately.
Some of the most difficult segments of the trip were when we passed through
cities. Here the traffic got heavy and the lane markings were often
non-existent or very hard to see. Sometimes during these segments RALPH locked
onto the vehicle ahead, and steered to follow it. For example, RALPH was able
to followed several trucks around almost the entire Denver beltway.
West of the Rocky Mountains, there were some stretches of really poor roads.
Often the lane markers were nearly invisible due to wear. Several times we
also encountered long stretches of construction where the road was basically
very fine, packed gravel. Needless to say there were no lane markings during
these stretches. But RALPH was able to key off the variations in appearance of
the packed gravel and the loose gravel around it and continue driving
The freeways in California were very interesting. Instead of having painted
lane markings to delineate lanes, they had reflectors that were nearly
invisible during the day (see left image below). But RALPH was able to key off
the discoloration from the oil spot down the center of the lane. In fact one
of RALPH's longest uninterrupted segments (56 miles) was on pavement like this
as we approached San Diego. We also drove on the I-15 HOV lane into San Diego,
and like the rest of the California freeways, it had reflectors marking the
lane boundaries. However it also had a strong boundary between the cement road
surface and the asphalt shoulder, making driving pretty easy for RALPH.
By clicking on the image, it will be downloaded, without the RALPH overlays, in
tiff format and an external viewer will start (if you have your browser set up
correctly.) You're welcome to try your algorithms on these images to see how
Questions and Answers about the Road Images
- Were you able to see what happened when Ralph drove into a sunset?
Yes. We experienced this effect most dramtically coming out of the mountains,
near Grand Junction CO., at about 7:00 pm. Although the camera mount is
equipped with a makeshift sun visor which shields some of the glare, the system
is still more sensative in these conditions. Basically, the camera iris has
problems adjusting properly. Glare off the road isn't too much of a problem
though, because the parts of the road which give you the most glare are
generally consistent - like the oil spot down the middle.
- Did the pictures of urban driving show Ralph locking onto the truck
ahead, or was it finding the lane edges? I couldn't tell from the red
- These pictures showed RALPH locking onto the truck, doing what we call
vision-based platooning. You can see this by looking at the lowest feature
template, which is the third one below the preprocessed image in the lower left
corner of the truck images. This shows the dark "feature" near the middle of
the template, caused by the rear of the truck. (Which is just inside the red
trapezoid in the high res image.) This is similar to what you see when there is
an oil spot down the middle of the road. (The reason we know that RALPH is
using the truck and not the lines is that if the truck changes lanes, RALPH
will follow it - is this a feature or a bug? )
- Did Ralph lock onto the trucks by happenstance, or did you train it to
- Ralph locks onto or adapts to whatever features are present. There is no
training in the sense of ALVINN. If we pull behind a truck slowly, RALPH can
adapt to following it. Opposite goes for if we slow down.
- In all of the pictures, how does the red trapezoid relate to what Ralph
is keying on?
- The area inside the red trapezoid is resampled and preprocessed into the
"birds-eye" image in the lower left corner. Basically, we use geometry to
project the pixels so that the image looks as if it was taken by a camera
looking straight down onto the road.
- Does Ralph lock onto trucks only when it can't find other, more expected features, like road edges, lane markers, oil slicks?
- RALPH locks on to truck if they represent a consistent feature of the
"road." This is not necessarily the best thing to do if there are clear lane
markings, but for city driving, where markings are poor or confusing, this
behavior is beneficial.
- Is there a danger of Ralph following a truck onto an exit ramp?
- Yes, but I wouldn't call it a "danger." We do have the ability to supress
certain parts of the image to bias RALPH to take or ignore exit ramps in normal
conditions, but haven't tested this extensively following trucks. An
interesting concept that we will probably investigate is moving the red
trapezoid forward and backward in response to things like trucks which are in
the viewing field. By bringing the viewing field in, you may be able to image
just the road and not the truck, or by moving it out, you may be able lock onto
another vehicle (cars work fine too) to get you through a tough environment. We
already move the trapezoid in and out based on speed, we just need an obstacle
detection sensor, like radar to test the other ideas.
firstname.lastname@example.org and email@example.com