A method of detecting and tracking irises and eyelids in video

We locate the eye corners, eyelids, and irises in every frame of an image sequence, and analyze the movements of the irises and eyelids to determine changes in gaze direction and blinking, respectively. Using simple models for the motions of the head and eyes, we determine the head-independent motions of the irises and eyelids by stabilizing for the head motion. The head-independent motions of the irises can be used to determine behaviors like saccades and smooth pursuit. Tracking the upper eyelid and using the distance between its apex and the center of the iris, we detect instances of eye closure during blinking. In experiments on two short image sequences, in one of which the subject was wearing glasses, we successfully located the irises in every frame in which the eyes were fully or partially open, and successfully located the eyelids 80% of the time. When motion information in the form of normal flow was used, the irises were successfully tracked in every frame in which the eyes were fully or partially open, and the eyelids were successfully located and tracked 90% of the time.     

Integrating perceptual and cognitive modeling for adaptive and intelligent human-computer interaction

This paper describes technology and tools for intelligent human-computer interaction (IHCI) in which human cognitive, perceptual, motor and affective factors are modeled and used to adapt the H-C interface. IHCI emphasizes that human behavior encompasses both apparent human behavior and the hidden mental state behind behavioral performance. IHCI expands on the interpretation of human activities, known as W4 (what, where, when, who). While W4 only addresses the apparent perceptual aspect of human behavior the W5+ technology for IHCI described in this paper addresses also the why and how questions, whose solution requires recognizing specific cognitive states. IHCI integrates parsing and interpretation of nonverbal information with a computational cognitive model of the user which, in turn, feeds into processes that adapt the interface to enhance operator performance and provide for rational decision-making. The technology proposed is based on a general four-stage interactive framework, which moves from parsing the raw sensory-motor input, to interpreting the user's motions and emotions, to building an understanding of the user's current cognitive state. It then diagnoses various problems in the situation and adapts the interface appropriately. The interactive component of the system improves processing at each stage. Examples of perceptual, behavioral, and cognitive tools are described throughout the paper Adaptive and intelligent HCI are important for novel applications of computing, including ubiquitous and human-centered computing.     

Shooting a smooth video with a shaky camera

Abstract. The image sequence in a video taken by a moving camera may suffer from irregular perturbations because of irregularities in the motion of the person or vehicle carrying the camera. We show how to use information in the image sequence to correct the effects of these irregularities so that the sequence is smoothed, i.e., is approximately the same as the sequence that would have been obtained if the motion of the camera had been smooth. Our method is based on the fact that the irregular motion is almost entirely rotational, and that the rotational image motion can be detected and corrected if a distant object, such as the horizon, is visible. Received: 14 February 2001 / Accepted: 11 February 2002 Correspondence to: A. Rosenfeld     

Reconstruction and visualization of planetary nebulae

From our terrestrially confined viewpoint, the actual three-dimensional shape of distant astronomical objects is, in general, very challenging to determine. For one class of astronomical objects, however, spatial structure can be recovered from conventional 2D images alone. So-called planetary nebulae (PNe) exhibit pronounced symmetry characteristics that come about due to fundamental physical processes. Making use of this symmetry constraint, we present a technique to automatically recover the axisymmetric structure of many planetary nebulae from photographs. With GPU-based volume rendering driving a nonlinear optimization, we estimate the nebula's local emission density as a function of its radial and axial coordinates and we recover the orientation of the nebula relative to Earth. The optimization refines the nebula model and its orientation by minimizing the differences between the rendered image and the original astronomical image. The resulting model allows creating realistic 3D visualizations of these nebulae, for example, for planetarium shows and other educational purposes. In addition, the recovered spatial distribution of the emissive gas can help astrophysicists gain deeper insight into the formation processes of planetary nebulae.     

Lunar Astronomical Observatories: Design Studies

The best location in the inner solar system for the grand observatories of the 21st century may be the Moon. A multidisciplinary team including university students and faculty in engineering, astronomy, physics, and geology, and engineers from industry is investigating the Moon as a site for astronomical observatories and is doing conceptual and preliminary designs for these future observatories. Studies encompass lunar facilities for radio astronomy and astronomy at optical, ultraviolet, and infrared wavelengths of the electromagnetic spectrum. Although there are significant engineering challenges in design and construction on the Moon, the rewards for astronomy can be great, such as detection and study of Earth‐like planets orbiting nearby stars, and the task for engineers promises to stimulate advances in analysis and design, materials and structures, automation and robotics, foundations, and controls. Fabricating structures in the reduced‐gravity environment of the Moon will be easier than in the zero‐gravity environment of Earth orbit, as Apollo and space‐shuttle missions have revealed. Construction of observatories on the Moon can be adapted from techniques developed on the Earth, with the advantage that the Moon's weaker gravitational pull makes it possible to build larger devices than are practical on Earth.     

The Applicability of Green's Theorem to Computation of Rate of Approach

The rate of approach (ROA) of a moving observer toward a scene point, as estimated at a given instant, is proportional to the component of the observer's instantaneous velocity in the direction of the point. In this paper we analyze the applicability of Green's theorem to ROA estimation. We derive a formula which relates three quantities: the average value of the ROA for a surface patch in the scene; a surface integral that depends on the surface slant of the patch; and the contour integral of the normal motion field around the image of the boundary of the patch. We analyze how much larger the ROA on the surface patch can be than the value of the contour integral, for given assumptions about the variability of the distance to points on the surface patch. We illustrate our analysis quantitatively using synthetic data, and we also validate it qualitatively on real image sequences.     

Estimating the heading direction using normal flow

If an observer is moving rigidly with bounded rotation then normal flow measurements (i.e., the spatiotemporal derivatives of the image intensity function) give rise to a constraint on the oberver's translation. This novel constraint gives rise to a robust, qualitative solution to the problem of recovering the observer's heading direction, by providing an area where the Focus of Expansion lies. If the rotation of the observer is large then the solution area is large too, while small rotation causes the solution area to be small, thus giving rise to a robust solution. In the paper the relationship between the solution area and the rotation and translation vectors is studied and experimental results using synthetic and real calibrated image sequences are presented. This work demonstrates that the algorithm developed in (Horn and Weldon 1987) for the case of pure translation, if appropriately modified, results in a robust algorithm that works in the case of general rigid motion with bounded rotation. Subsequently, it has the potential to replace expensive accelerometers, inertial systems and inaccurate odometers in practical navigational systems for the problem of kinetic stabilization, which is a prerequisite for any other navigational ability.     

Far infrared properties of PLZT ceramics

Far infrared and mid infrared reflectivity measurements have been made of lead titanate zirconate (PLZT) ceramics at room temperature in the frequency range 40–1500 cm-1. Kramers-Krönig analysis and a fitting procedure based on a four parameter model of the spectrum, determined the infrared active transverse and longitudinal optical modes and their damping factors. Group theory analysis has been done and the number of experimentally observed infrared active modes was compared with the theoretical prediction. Three ionic oscillators for a cubic perovskite Pm3m structure should exist, but four ionic oscillators for the PLZT polycrystalline sample were observed. This excluded the possibility that the sample could have a cubic lattice and confirmed that the tetragonal structure was present at room temperature. X-ray analysis of the PLZT sample was performed and the results were compared with infrared spectroscopy work. It was concluded that infrared optical reflectivity measurements, as a non-destructive method, can give more reliable results at the boundary cases when it is very difficult to distinguish if a material has either a cubic or tetragonal lattice structure.     

Function from motion

In order for a robot to operate autonomously in its environment, it must be able to perceive its environment and take actions based on these perceptions. Recognizing the functionalities of objects is an important component of this ability. In this paper, we look into a new area of functionality recognition: determining the function of an object from its motion. Given a sequence of images of a known object performing some function, we attempt to determine what that function is. We show that the motion of an object, when combined with information about the object and its normal uses, provides us with strong constraints on possible functions that the object might be performing