This paper considers the problem of inferring local contact geometry from passive tactile information. The term ``passive'' means that the information results from motions of the object that are not necessarily under the control of the robot. To juxtapose, generally the term ``active sensing'' means that a robot actively explores an object, for instance with visual and/or tactile sensors that move around the object. This paper is not concerned with such active exploration. Instead, our goal is to determine how local shape geometry may be inferred from purely passive information. Shape recovery based on passive information will be useful both for dealing with unexpected events, as in the slippery grasping example above, and for more elaborate manipulation strategies, such as active exploration.