Northridge earthquake aftershock 3D volume animations

[sfmap] The following AVS animations were produced by Greg Foss of the Pittsburgh Supercomputing Center Visualization Group from ground motion data supplied by the Quake project. The animations show different views of the first 19 seconds of an aftershock of the 1994 Northridge quake.

How was the data produced?

David O'Hallaron

The data for the animations was produced by the Quake group at Carnegie Mellon using an unstructured finite element simulation generated with the Archimedes tool chain. The simulation models 40 seconds of an aftershock from the Jan 17, 1994 Northridge quake in the San Fernando Valley of Southern California. The San Fernando model was provided by Magistrale and Day at San Diego State University and covers a volume of the earth roughly 50km x 50km x 10km in size. The corresponding unstructured mesh consists of 13,422,563 nodes and 76,778,630 linear tetrahedral elements. The mesh provides a frequency resolution of about 1 Hz and a spatial resolution of about 20 meters.

The simulation runs for 6.5 hours on 256 PEs of a Cray T3D. It performs 16,666 timesteps, with one 40M x 40M sparse matrix-vector product per time step. It performs about 80 trillion (80 x 10^12) floating-point operations in 6.5 hours for a sustained computational rate (including communication and file i/o) of about 3.5 GFLOPS/sec.

The simulation also transfers about 800 GBytes during the 575 seconds that comprise the communication phases for a burst rate (i.e. the sustained rate during the communication phases) of about 1.4 GBytes/sec.

I/O consists of reading a 3 GByte binary input file produced by Archimedes and writing a 14-15 GByte output file. The 15 GByte file was produced by a simulation of an event with a duration of ~20s, a time step of 0.0024s, while outputting one float per node (a single displacement) for all ~13.4x10^6 nodes every 30 timesteps. Therefore:

15 GB ~= 8,333 timesteps x (1 output step / 30 timesteps) x (13,422,563 nodes / output step) x (4 bytes / node)

The 14 GByte file was produced by simulating an event with a duration of ~40s, a time step of 0.0024s, while outputting, roughly, six floats per node (3 displacements and 3 velocities) for 1,064,711 surface nodes every 30 timesteps. Therefore:

14GB ~= 16,666 timesteps x (1 output step / 30 timesteps) x (1,064,711 surface nodes / output step) x (24 bytes / node)

Performance breakdown on 256 PE's of the Cray T3D

computation 16679 secs (71%)
communication 575 secs (2%)
input/output 5995 secs (25%)
total 23250 secs (388 mins)(6.5 hours)

Quake Project
Last modified: Wed Jun 8 11:41:23 EDT 2005