\section*{Conclusions and Future Work}

The Archimedes project was begun in order to bring
bring state-of-the-art computing to state-of-the-art engineering.
Our main goal was to design and build a 
general-purpose tool that would allow researchers in engineering and
science to model the complexities of real spatial domains encountered
in finite element computations.

The resulting system, Archimedes, 
is an automated system for solving partial differential equations on
irregular spatial domains using finite element methods.
Archimedes performs automated 
mesh generation, domain decomposition, 
placement and routing of subproblems, and code generation for 
distributed memory parallel machines.
While the mesh generation and domain decomposition stages of the process
are machine-independent, 
the characteristics of the target parallel machine are taken into account
in the placement, routing and code generation stages of the process.
We feel that Archimedes has demonstrated the feasibility of such
general-purpose automated tools for the parallel solution of complex
engineering problems.

Since its original implementation for the iWarp and CM-5 systems, we
have retargeted Archimedes to the Intel Paragon, as well as individual
workstations to ease the debugging process.  Other architectures are
planned.  It generally requires only a week or
two of effort to retarget the code generator.

Archimedes has duplicated the original 3D seismic ground
motion study that motivated its development, and is currently being used on a
production basis to model ground motion in
2D and 3D basins~\cite{bielak95}.  Ultimately,
Archimedes will be used to model the entire Los Angeles basin.


\iffalse
\section*{Experience and Conclusions}

The events that led to the development of Archimedes are informative.
In late 1991, civil engineers at Carnegie Mellon began exploring the
use of parallel computers for modeling earthquake-induced ground
motion.  They manually implemented an explicit solver on the Thinking
Machines CM-2 using an unstructured mesh.   The code ran slowly because
the processors spent almost all of their time communicating with each
other over the relatively slow global communication router.  So they
rewrote the code, this time creating a block-structured mesh and
painstakingly partitioning it over the processors to try to minimize
the use of the relatively slow router.  The fast nearest-neighbor
communication system was used for the regular communication patterns
within each block, while the global communication system was invoked
to ``stitch'' the blocks together.  The resulting code exhibited an
order-of-magnitude improvement in Mflop/s, due to the substantial
reduction in global communication.  Unfortunately, because the
gradation of the block-structured mesh was insufficiently flexible,
smaller elements than desired were generated in the rock surrounding
the valley.  These elements were critical with respect to the stability
condition.  Thus, the solver was forced to take many more iterations
than the physics required.  Switching to an implicit solver was
problematic, for it entailed a complete rewrite of the code.  Parallel
computers, rather than a tool, were becoming an impediment to doing
science!

Frustrated by the painstaking level of detail required to create
reasonably efficient programs, the engineers sought out computer
scientists at Carnegie Mellon for assistance.  During these
discussions, we realized that by exploiting exploiting prior work in
mesh generation, VLSI placement, and routing, and our own ongoing work
in graph partitioning and parallelizing compilers, it would be
possible to fully automate the irregular finite element modeling
process.  Thus was born the collaboration that produced Archimedes.

Since its original implementation for the iWarp and CM-5 systems, we
have retargeted Archimedes to the Intel Paragon and networks of
workstations.  As other parallel systems become available we will
retarget Archimedes to these as well.  It generally requires a week or
two of effort to retarget the system.

We have used Archimedes to duplicate the original 3D seismic ground
motion study, and the engineers are currently using Archimedes on a
production basis to model earthquake-induced ground motion in
moderate-sized 2D and 3D basins~\cite{bielak95}.  Ultimately,
Archimedes will be used to model the entire Los Angeles basin.

Our collaboration continues to grow in scope.
Problems that remain include automatic
generation of quality three-dimensional meshes, parallelization of the mesh
generation and partitioning algorithms, runtime methods for
meshes that change dynamically, and techniques for debugging very large
numerical simulations.
\fi
