In this paper we present a non-intrusive model-based gaze-tracking system. The system estimates the 3D pose of a user's head by tracking as few as six facial feature points. The system locates a human face using a statistical color-model without any mark on the face. The system then finds and tracks the facial features, such as eyes, nostrils and lip-corners. A full perspective model is employed to map these feature points onto the 3D head pose. Several techniques have been developed to track the features points and recover from failure. The system has achieved a frame rate of 15+ frames per second using an HP 9000 workstation with a frame-grabber and a canon VC-C1 camera. Other performance of the system has also been measured through various experiments. The application of the system has been demonstrated by a gaze-driven panorama image viewer. The potential applications of the system include multimodal interface, virtual reality and tele-conferencing.