System for classifying an individual's gaze direction

Abstract

A system is provided to classify the gaze direction of an individual obseng a number of surrounding objects. The system utilizes a qualitative approach in which frequently occurring head poses of the individual are automatically identified and labelled according to their association with the surrounding objects. In conjunction with processing of eye pose, this enables the classification of gaze direction. In one embodiment, each observed head pose of the individual is automatically associated with a bin in a 'pose-space histogram'. This histogram records the frequency of different head poses over an extended period of time. Given observations of a car driver, for example, the pose-space histogram develops peaks over time corresponding to the frequently viewed directions of toward the dashboard, toward the mirrors, toward the side window, and straight-ahead. Each peak is labelled using a qualitative description of the environment around the individual, such as the approximate relative directions of dashboard, mirrors, side window, and straight-ahead in the car example. The labelled histogram is then used to classify the head pose of the individual in all subsequent images. This head pose processing is augmented with eye pose processing, enabling the system to rapidly classify gaze direction without accurate a priori information about the calibration of the camera utilized to view the individual, without accurate a priori 3D measurements of the geometry of the environment around the individual, and without any need to compute accurate 3D metric measurements of the individual's location, head pose or eye direction at run-time.