% Master File: final.tex
\documentclass {article}

\usepackage[dvips]{graphicx}
\usepackage{mymath}
\usepackage{fancyheadings}
\usepackage{subfigure}
\usepackage{mypsfig}
\usepackage{url}
\usepackage{alg}

\RequirePackage{ifthen}
\RequirePackage{times}      % Loads the Times-Roman Fonts
\RequirePackage{pifont}     % Loads the Postscript symbol fonts

\setlength{\textwidth}{6.75in}  %{6.5in}
\setlength{\topmargin}{0.25in}   %{0.5in}
\setlength{\oddsidemargin}{-0.1250in}
\setlength{\textheight}{9.0in}  %{8.5in}

%
% on UNIX
\comment{
\psfigurepath{
  .
}

\graphicspath {
  {./Figs/}
}
}

\begin{document}

\title{ 
{Robot Lab Graduate Project Final Report}
}

\author{
Lisa Anthony
}
\date{
due December 17, 2001
}

%\maketitle


\noindent {\bf Lisa Anthony \\ 
  Robot Building Lab \\ 
  Graduate Project: Final Report \\ 
  Due December 17, 2001}

\vspace*{0.15in}


 
\section{Task}
\label{sec:task}

I propose to investigate the accuracy of the sonar sensors we have
been using in class; many groups received readings which did not make
sense, given certain assumptions that were made about the operation of
these sensors.  To attain a more reliable sonar sensor would be both
interesting and practical for future use of these sensors.

\subsection{Confounding Variables}

In the graphs of the elementary in-class experiments conducted with
the sonar sensors, several problems or inconsistencies were apparent
(see \emph{plan.ps} and \emph{design.ps} for details), implying that
the situations in which either \emph{sonar\_closeup()} and
\emph{sonar\_sample()} are most useful and accurate are not fully understood.
I performed many experiments using both sonar sampling methods, trying
to control for and isolate the following possible confounding
variables:

\begin{itemize}
\item     Distance of obstacle(s) from sonar/robot 
\item     Timing interval between subsequent sonar readings 
\item     Effect of using the two methods together instead of just one 
\item     Effects of the servo motor motion on sonar accuracy
\item     Variability among individual sonars
\item     Blanking time intervals used in each method 
\end{itemize}


\section{Hardware}
\label{sec:hw}

The only hardware used for this project was a sonar sensor mounted on
a servo motor attached to a HandyBoard, in similar configurations to
that used in class.  No drive train or other sensors were needed for
the experiments; although they were attached to the robot used, they
were simply idle.  The experiments were conducted in the same manner
used in class: the sonar/servo/HandyBoard combination took readings of
the world in a 180-degree sweep and sent them across a serial line to
a PC.


\section{Software}
\label{sec:sw}

I was able to set up one control program which used easily changed
global variables to set up each experiment I needed to run.  The code
(in three files, \emph{main.c}, \emph{readings.c}, \emph{dump.c}) set
up how the sonar would take the readings, obtained these readings and
stored them.  At the end of each experimental run, the results were
transmitted to a PC via TeraTerm Pro; I imported them into Microsoft
Excel to graph them.

The end result of this project was intended to be a ``filter''
function to add to the sonar library that would mask from its users
the procedure used to obtain the most accurate sonar reading possible.
However, it turned out that this function was not really necessary;
our hypothesis had been that there were situations in which one method was
much more effective than the other, but in reality, I was able to
achieve equal performance from each method after my alterations to the
\emph{sonar.c} file (discussed in Section~\ref{sec:exp}).

\subsection{Interactive-C Libraries}

Here is a list of the methods used from the default IC libraries:

\begin{itemize}
\item \emph{sonar.c}---for sonar control
	\begin{itemize}
	\item \emph{sonar\_init()}
	\item \emph{sonar\_closeup()}
	\item \emph{sonar\_sample()}
	\end{itemize}
\item \emph{servo.c}---for servo motor control
	\begin{itemize}
	\item \emph{servo\_on()}
	\item \emph{servo\_deg()}
	\end{itemize}
\item \emph{serialio.c}---for serial communication to a PC
	\begin{itemize}
	\item \emph{disable\_pcode()}
	\item \emph{serial\_putchar()}
	\end{itemize}
\item \emph{printdec.c}---for printing decimal numbers via serial communication to a PC
	\begin{itemize}
	\item \emph{printdec()}
	\end{itemize}
\end{itemize}


\subsection{Control Program Modules}

\begin{itemize}

\item {\bf Serial Communication Module:} In previous labs, my team had built
methods to use in taking collected sensor data and dumping it to a
serial line, which I also used in this project.

\item {\bf Experimental Control Modules:} The software template for running
sonar-based experiments and collecting data from them had already been
implemented by our team in class, and was the skeleton to which I
added the code to run the experiments I needed.

\begin{itemize}
\item \emph{main.c} - in charge of setting up current experiment by calling
\emph{readings()} with appropriate args; then dumps data to serial line
using \emph{dump()}

\item \emph{readings.c} - takes \emph{SONAR\_SAMPLES} (defined in
\emph{main.c}) number of readings using both \emph{sonar\_closeup()}
and \emph{sonar\_sample()} methods based on command line args

\item \emph{dump.c} - dumps arrays of readings to the serial line; one reading per
line, readings from \emph{sonar\_closeup()} followed by readings from
\emph{sonar\_sample()}; a line with 3 asterisks marks the data separation

\end{itemize}

\end{itemize}

\subsection{Alterations to \emph{sonar.c}}

Via experimental analysis, it was determined that the default blanking
time used in \emph{sonar\_closeup()} in \emph{sonar.c} was too short
and was causing the method to often return funky data.  I altered the
default blanking time from 0.5ms to 0.7ms based on experimental
discovery, and the method's performance immediately drastically
improved.  This is explained in more detail in
Section~\ref{sec:analys}.


\section{Experimental Results}
\label{sec:exp}

Each experiment set and its important characteristics, along with a
brief description of the results, is enumerated below.  In the
discussions and tables below, \emph{swait} is the amount of time to
wait between immediately subsequent sonar readings, \emph{vwait} is
the amount of time to wait between a sonar reading and servo movement,
\emph{iwait} is the amount of time between readings which are to be
averaged, and \emph{cwait} is the amount of time each method
``blanks'' before allowing a return pulse to be recognized.  Fields
marked with a ``*'' are discussed in detail in the Results section of
each experiment set section.


\subsection{Experiment Set 1: Determining Appropriate Amount of Time to Wait Between Sonar Readings}

\begin{figure}[h]
  \begin{center}
    \leavevmode {\psfig{figure=./exp1.eps,width=6.00in}}
  \end{center}
\caption{Experiment Set 1.}
\label{table:exp1}
\end{figure}

\paragraph*{Results}
The \emph{swait} values I tested were as follows: 1.0s, 0.5s, 0.25s,
0.2s, 0.15s, 0.1s, 0.05s, 0.025s, 0.02s, and 0s.  Preliminary
experiments were very promising; both methods were returning identical
values for the same positions (unlike in class experiments, where
there was often a parallel gap between the values return by each
method).  The \emph{sonar\_sample()} method returned good values for
every \emph{swait} used; however, the \emph{sonar\_closeup()} method's
values began spiking too low at about \emph{swait}=0.2s.  See
\emph{swait.xls} for graphs and data.


\subsection{Experiment Set 2: Determining Appropriate Amount of Time to Wait Between a Sonar Reading and Servo Movement}

\begin{figure}[h]
  \begin{center} \leavevmode {\psfig{figure=./exp2.eps,width=6.00in}}
    \end{center}
\caption{Experiment Set 2.}
\label{table:exp2}
\end{figure}

\paragraph*{Results}
Because the last \emph{swait} value which worked for both methods in
Experiment Set 1 was 0.25s, I used that value as the default lowest
one in most subsequent experiments, including this set.  The
\emph{vwait} values I tested were as follows: 0.5s, 0.25s, 0.10s,
0.05s, and 0s.  The \emph{sonar\_sample()} method
returned good values for every \emph{vwait} used; however, the
\emph{sonar\_closeup()} method's values began spiking too low at about
\emph{vwait}=0.10s.  See \emph{vwait.xls} for graphs and data.


\subsection{Experiment Set 3: Averaging Several Readings at One Position to Correct for Possible Random Error}


\begin{figure}[h]
  \begin{center} \leavevmode {\psfig{figure=./exp3.eps,width=6.00in}}
    \end{center}
\caption{Experiment Set 3.}
\label{table:exp3}
\end{figure}

\paragraph*{Results}
At this point, we had a hypothesis that perhaps the low-spike values
being returned by \emph{sonar\_closeup()} might be random errors which
could be corrected by taking several readings at the same spot and
averaging them together to cancel out any random erroneous readings.
I decided to run averages of 3 readings at each spot in the sweep.
The \emph{iwait} values I tested were as follows: 0.05s for both
methods, 0.05s for \emph{sonar\_sample()} and 0.10s for
\emph{sonar\_closeup()}, and 0.10s for \emph{sonar\_sample()} and
0.20s for \emph{sonar\_closeup()}.  The reason I started using
different intervals for each method was that \emph{sonar\_sample()}
was returning good values for any interval, but
\emph{sonar\_closeup()} was acting up even worse than before.  I
thought increasing the \emph{iwait} might help alleviate this problem,
but it did not.  See \emph{averaging.xls} for graphs and data.


\subsection{Experiment Set 4: Running \emph{sonar\_sample()} Independently}

\begin{figure}[h]
  \begin{center}
    \leavevmode {\psfig{figure=./exp4.eps,width=6.00in}}
  \end{center}
\caption{Experiment Set 4.}
\label{table:exp4}
\end{figure}

\paragraph*{Results}
I had started to worry that my results might be affected by the fact
that I was always running \emph{sonar\_sample()} before
\emph{sonar\_closeup()} during my sweeps; I ran this set of
experiments to try to isolate out any effect one method might have on
the other.  I ran a normal experiment with just the
\emph{sonar\_sample()} method, using different waiting times before
moving the servo to the next spot.  The \emph{vwait} values I tested
were as follows: 0s, 0.25s, 0.10s, 0.05s, 0.025s, 0.01s, 0.005s,
0.004s, 0.002s, and 0.001s.  The results showed that the
\emph{sonar\_sample()} method performed very well with no spiking at
all down to a \emph{vwait}=0.025s, and very little spiking down to
\emph{vwait}=0.001s.  See \emph{sample.xls} for graphs and data.


\subsection{Experiment Set 5: Running \emph{sonar\_closeup()} Independently}

\begin{figure}[h]
  \begin{center}
    \leavevmode {\psfig{figure=./exp5.eps,width=6.00in}}
  \end{center}
\caption{Experiment Set 5.}
\label{table:exp5}
\end{figure}

\paragraph*{Results}
I ran this set of experiments to try to isolate out any effect one
method might have on the other; I ran a normal experiment with
just the \emph{sonar\_closeup()} method (see above), using different
waiting times before moving the servo to the next spot.  The
\emph{vwait} values I tested were as follows: 0.25s, 0.5s, 1.0s, and
5.0s.  These were the most disheartening results, because even while
waiting 5 seconds between servo movement and sonar reading \emph{and}
taking averages with an \emph{iwait}=5.0s, the values returned were
basically complete junk.  See \emph{closeup.xls} for graphs and data.


\subsection{Experiment Set 6: Isolating the Servo's Influence (no sweeping)}

\begin{figure}[h]
  \begin{center} \leavevmode {\psfig{figure=./exp6.eps,width=6.00in}}
    \end{center}
\caption{Experiment Set 6.}
\label{table:exp6}
\end{figure}

\paragraph*{Results}
Due to the very bad results the experiments in Set 5 yielded, it was
clear that there were still too many variables that could be affecting
the performance of \emph{sonar\_closeup()}.  I ran this set of
experiments to try to isolate out any remaining effect the servo might
be having; I ran several experiments without moving the servo at all.
The obstacles in these experiments were at 90 degrees in front of the
sonar, 45 inches away.  The sampling routine used in each experiment
was different as well: I used \emph{sonar\_closeup()} followed by
\emph{sonar\_sample()} for one, vice versa for another, and then ran
all \emph{sonar\_closeup()} readings followed by all
\emph{sonar\_sample()} readings, and then vice versa.  While most of
these experiments just took one reading at each position, I also ran a
final experiment that averaged 3 readings for each position.  The
\emph{swait} values I tested were as follows: 1.0s and 0.25s.
The interchanging routines yielded the worst results, with many
low spikes in the \emph{sonar\_closeup()} method's values.  However,
none of the runs had really good results for the
\emph{sonar\_closeup()} method, so based on that, it seemed safe to
assume that the servo movements were not the culprit in the bad
results for Experiment Sets 4 and 5.  See \emph{nosweep.xls} for graphs
and data.


\subsection{Experiment Set 7: Adjusting \emph{sonar\_closeup()}'s Blanking Timing}

\begin{figure}[h]
  \begin{center} \leavevmode {\psfig{figure=./exp7.eps,width=6.00in}}
    \end{center}
\caption{Experiment Set 7.}
\label{table:exp7}
\end{figure}

\paragraph*{Results}
After Experiment Set 6, it began to be apparent that whatever was
causing \emph{sonar\_closeup()}'s performance to be so poor almost all
the time, must be isolated within the code itself, as we had tested
for every confounding variable we could think of.  Closer inspection
of the \emph{sonar\_closeup()} method's implementation revealed that,
while the \emph{sonar\_sample()} method used blanking for 2.38ms, and
the closeup() method was supposed to not use blanking, it actually did
blank for 0.5ms.  For this set of experiments, then, I adjusted the
\emph{sonar.c} code to increase the amount of blanking time.  I ran
two experiments without sweeping or averaging to re-create experiments
from Experiment Set 6, with \emph{cwait}=2.38ms, to ensure that the
results emulated those of \emph{sonar\_sample()}.  I then ran several
experiments with varying values of \emph{cwait}, during which I did
use sweeping and averaging.  That made these experiments similar to
the ones run in Experiment Set 3, so that I could compare the results
and be sure that my changes to the blanking interval were having an
effect.  The \emph{cwait} values I tested were as follows: 2.38ms,
2.00ms, 1.00ms, 0.75ms, 0.70ms, 0.65ms, and 0.625ms.  I gradually
decreased the blanking interval until the results started to look
goofy again.  The lowest value for \emph{cwait} that still yielded
good results was 0.70ms.  Based on this, I altered the \emph{sonar.c}
library to use this value within the \emph{sonar\_closeup()} method
permanently.  See \emph{blanking.xls} for graphs and data.


\subsection{Experiment Set 8: Different World Arrangements}

\begin{figure}[h]
  \begin{center} \leavevmode {\psfig{figure=./exp8.eps,width=6.00in}}
    \end{center}
\caption{Experiment Set 8.}
\label{table:exp8}
\end{figure}

\paragraph*{Results}
Satisfied with the results from the blanking experiments, the last set
I ran were designed to survey the performance of the
\emph{sonar\_sample()} method and the new \emph{sonar\_closeup()}
method in different world configurations.  I tested the methods with
obstacles at 12 inches, 5 inches, 4.75 inches, 4 inches and 1.75
inches.  The blanking interval of 0.70ms now being used in
\emph{sonar\_closeup()} should have allowed the minimum distance at
which an obstacle could be detected accurately to be 4.75 inches,
which was backed up in the experimental results.  I then did cursory
experiments with other obstacle shapes: an obstacle angled at +45
degrees (away), at -45 degrees (toward), a convex-cornered obstacle,
and a concave-cornered obstacle.  These experiments were merely
cursory due to time constraints; further investigation needs to be
done with these variables to be able to make intelligent assumptions
about the behavior of each sampling method under these circumstances.
See \emph{obstacles.xls} for graphs and data.


\subsection{Experimental Conditions}

As stated in the project plan, it is necessary to keep careful records
of the world map used in these experiments, for reproducibility's sake
as well as to ensure that the obtained data can be later analyzed in
the context of the experimental setting.  See Figure~\ref{fig:map} for
a diagram of the world used in most experiments (except for Experiment
Set 8).

\begin{figure}[h]
  \begin{center} \leavevmode {\psfig{figure=./world2.eps,width=4.00in}}
    \end{center}
\caption{Layout of the World Map Used in Most Experiment Sets.}
\label{fig:map}
\end{figure}


\section{Analysis of Results}
\label{sec:analys}

This project's progress has determined, through extensive empirical
study, that the \emph{sonar\_sample()} method is really very accurate
under most circumstances (regarding timing specifically) for obstacles
above 12 inches away. The \emph{sonar\_closeup()} method, on the other
hand, in its default form is rather useless. I altered the default
blanking period used in that method from 0.5ms to 0.7ms and saw
immense improvements. I recommend using the altered
\emph{sonar\_closeup()} especially for obstacles under 12 inches (but
according to my results, above 4.75 inches); \emph{sonar\_sample()}
now is not any more useful, but can be used to support or average
values obtained with
\emph{sonar\_closeup()}.


\section{Open Issues and Possible Future Improvements}
\label{sec:issues}


\paragraph*{Changes in Variables Studied}
In my original project plan, the confounding variables I had intended
to study were level of battery power, movement of the robot itself
using its DC motors, interference caused by taking sonar readings too
close in time together, movement of the sonar atop the servo motor,
and orientation or distance of the obstacle(s).  Time constraints as
well as intermediate results which suggested other avenues of
exploration (i.e., the blanking time interval) might yield more
interesting results causing me to abandon these other issues; in
future, perhaps another graduate student's final project will be able
to address them in more detail, if the sonar sensors continue to give
people trouble.


\paragraph*{Obstacle Orientations and Materials}
Because I was only able to run cursory experiments with other obstacle
shapes, further investigation needs to be done with these variables to
be able to make intelligent assumptions about the behavior of each
sampling method under varying world conditions.  It is probably also
worthwhile to study the sound absorption properties of any future
obstacles which may be used in the class; we generally used the
toolboxes, which are heavily sound-reflective, but other materials
will have different degrees of reflection.  This could cause the
results of sonar sampling to be completely different.


\paragraph*{Delicacy of the Sonar Sensors}
I ran most of the experiments with one particular sonar sensor, but
did run a few of them with other sonars (2 others).  The results of
these other experiments were convincing that variability among sonars
is not as significant a confounding factor as was originally
considered possible.  However, a major problem with using other sonar
sensors is that the sensors are so delicate that they must be handled
with extreme care.  The wires must not be pulled or stretched, and the
sonar must be mounted in such a way on the servo motor that there is
enough slack in the wire for the entire 180-degree sweep.  Luckily, my
main sonar for these experiments held up very well; however, I
``broke'' the other two I tried using after the first run.  It should
be emphasized to students that it's really not difficult to break
these sensors and that any rough handling could skew your results
without the cause being obvious.



\end{document}
 

