Downtime and Outages: Real Incidents & Causes

Surveys

• Causes of Failure in Web Applications Soila M. Pertet and Priya Narasimhan, PDL Technical Report PDL-CMU-05-109, Carnegie Mellon University, December 2005

Upgrades Gone Awry

• Failed Windows XP upgrade downs 60,000 UK Goverment PCs, November 2004
• Vonage voicemail vanishes during site upgrade, July 2005
• MCI upgrade glitch disrupts ATM service, August 1999
• Software upgrade glitch grounds Los Angeles airport, October 2000
• EDS: IT upgrade caused software glitch at UK agency, December 2004
• The truth: What caused the AT&T frame relay network outage, February 2001
• Upgrade glitch downs AT&T Wireless' CRM system, November 2001
• eBay says system upgrade tied to recent outages, October 2000

Other Incidents

• Netcraft News and Surveys
• Points of Reference: e-Commerce Failures Online Brokerage Industry Report, 1999
• Electric Blackout of August 2003 in Northeast US and Canada
  • Reports from the North Americal Electric Reliability Council
  • Reports from the US Department of Energy
• Slashdot stories on bugs, viruses and downtime
• Ariane 5 Flight 501 Failure
  Inquiry Board Report, Prof. J. L. Lions, July 1996
• Therac-25 radiation incidents
  • Investigation of the Therac-25 accidents, Leveson & Turner, July 1993
  • Therac-25 case materials, computingcases.org
• Navy Smart Ship USS Yorktown
  • Software glitches leave Navy Smart Ship dead in the water
    Government Computer News, July 13, 1998.
  • The Smart Ship is not the answer
    U.S. Naval Institute Proceedings, June 1998
• Human error called culprit in 3 rocket launch failures
  Florida Today Space Online, June 16, 1999
• Schwab Outage: IT Wake-Up Call
  InternetWeek, February 26, 1999
• Outages at eBay:
  • eBay Retrenches
    InternetWeek, June 17, 1999
  • eBay Servers Go Down -- Again
    InternetWeek, June 30, 1999
  • eBay Crashes Again
    InternetWeek, August 9, 1999
  • eBay goes back online after prolonged outages
    CNET News.com, January 4, 2001
• Outages at E-Trade:
  • E-Trade Suffers OutageInternetWeek, February 3, 1999
  • Another Outage at E-TradeInternetWeek, Feburary 4, 1999
  • E-Trade Explains Cause of Last Week's Outage InternetWeek, February 10, 1999
  • E-Trade CIO Discusses the OutagesInternetWeek, February 10, 1999
• Stock Market Outages Highlight Software Availability Issues The Payne Report, July/August 2001