We propose a metric to determine whether one version of a system is relatively more secure than another with respect to the system's attack surface. Intuitively, the more exposed the attack surface, the more likely the system could be successfully attacked, and hence the more insecure it is. We define an attack surface in terms of the system's actions that are externally visible to its users and the system's resources that each action accesses or modifies. To apply our metric in practice, rather than consider all possible system resources, we narrow our focus on a "relevant" subset of resource types, which we call attack classes; these reflect the types of system resources that are more likely to be targets of attack. We assign payoffs to attack classes to represent likelihoods of attack; resources in an attack class with a high payoff value are more likely to be targets or enablers of an attack than resources in an attack class with a low payoff value. We outline a method to identify attack classes and to measure a system's attack surface. We demonstrate and validate our method by measuring the relative attack surface of four different versions of the Linux operating system.