Presented are patents for aiding display control activities, such as:
(1) Automatic aiding of human cognition functions with computerized displays;
(2) Device and method for estimating a mental decision;
(3) Eye-tracker control of heads-up displays.
NOTE: The patents listed above are fee expired, now public domain, the technology presented as of interest for incorporation into ongoing efforts.
Automatic aiding of human cognition functions with computerized displays (more)
The invention is an automatic aider of human cognitive functions designed for the manual operation of computerized displays. The invention estimates the cognitive decision made by the human operator in response to a displayed stimulus, from the single-event, evoked cerebral potential and the corresponding visual process. The invention parameterizes the cerebral potential by computing: (1) the coefficients of an autoregressive filter model of the electroencephalogram, and (2) the attenuations of a network of parallel attenuators for the cerebral sources evoked by the display stimulus. The input to the filter is the summed outputs of the attenuators. These cerebral sources follow from the set of basic waveforms for the principal components analysis of the event average evoked potential differences derived in calibration for the set of display stimuli. An expert system computes the reliability of the estimation from the cognitive state which is determined from the power spectrum for the autoregressive coefficients. Further, a fuzzy-logic processor tracks the visual responses to the stimulus and classifies the attention state. These values of the source attenuations, the cognitive reliability, the attention state, and the response indexes, are the inputs to an artificial neural network which is used as a decision classifier. In turn, the outputs of the classifier are the probabilities of the occurrences of the set of possible decisions which can be made in response to the displayed stimulus. The invention determines the decision aiding to be provided to the human operator, with an expert system, from the disparities among the estimations and the decisions that are expected from task scripts for the displayed stimulus.
PATENT US5649061
Device and method for estimating a mental decision from eye fixation and single event evoked cerebral potential, and a method for controlling computerized machinery from eye gaze alone (more).
The invention estimates the mental decision to select a visual cue from the viewer’s eye fixation and corresponding single event evoked cerebral potential. The invention consists of an eye-gaze tracker, an electronic bio-signal processor and a digital computer. The eye-tracker determines the instantaneous viewing direction from oculometric measurements and a head position and orientation sensor. The electronic processor continually estimates the cerebral electroencephalogram potential from scalp surface measurements following corrections for electrooculogram, electromyogram and electrocardiogram artifacts. The digital computer analyzes the viewing direction data for a fixation and then extracts the corresponding single event evoked cerebral potential. The gaze fixation properties: duration, start and end pupil sizes, end state (saccade or blink) and gaze fixation count, and the parametric representation of the evoked potential are all inputs to an artificial neural network that has as output an estimate of the selection interest in the gaze point of regard. The network is trained off-line prior to application to represent the mental decisions of the viewer. The invention may be used to control computerized machinery from a video display by ocular gaze point of regard alone, by determining which visual cue the viewer is looking at and then using the estimation of the task-related selection as a selector switch.
PATENT US5689619
Eye-tracker control of heads-up displays (more)
The invention consists of an eye-gaze tracker, a display driver, an adaptive display interface controller, and a digital computer. A user of the invention can operate a heads-up display (or a head-mounted display) with an eye-gaze tracker, leaving his hands free to perform a manual control task. The user selects a display element while from an array of choices on the heads-up display by gazing at an element (i.e., icon) while activating a confirming mechanism (say, by switch, when gaze is within icon capture window). Both tasks share a common visual working area with overlaid visual images, and task interference is reduced since eye-movements and visual attention are dedicated to the same display surface. The adaptive display interface automatically aids the user in display operations during high manual task loading. The adaptive interface is driven by an electronic Expert System on human visual processes programmed as a Fuzzy Logic controller. The controller generates display cueing aids under computer control, in response to erratic eye movement patterns that occur during cognitive conflict. The cueing aids help the user to operate the display with a sequence of discrete eye gazes and confirming actions which only momentarily interrupt the manual control task. As cueing aids, the invention provides a timing logic for element enhancement and displays markers for task feedback. While the aids would be disruptive at low cognitive loadings, they provide necessary timing and memory prompts at high loadings. The element enhancement timing logic allows the user to momentarily look at the display and then return attention to the tracking task before executing confirmation. The display cues show the user where he is looking on the display relative to the element to be selected.