We propose a metric to determine whether one version of a software system is more secure than another with respect to the system's attack surface. Rather than count bugs at the code level or count vulnerability reports at system level, we measure a system's attackability, i.e., how likely the system will be successfully attacked. We define the attack surface of a system in terms of the system's attackability along three abstract dimensions: method, data, and channel. Intuitively, the larger the attack surface, the more likely the system will be attacked, and hence the more insecure it is. We demonstrate the use of the attack surface metric by measuring and comparing the attack surface of two versions of a hypothetical IMAP server.