Using stereokinetic effect to convey depth: computationally efficient depth-from-motion displays.

Recent developments in microelectronics have encouraged the use of 3D data bases to create compelling volumetric renderings of graphical objects. However, even with the computational capabilities of current-generation graphical systems, real-time displays of such objects are difficult, particularly when dynamic spatial transformations are involved. In this paper we discuss a type of visual stimulus (the stereokinetic effect display) that is computationally far less complex than a true three-dimensional transformation but yields an equally compelling depth impression, often perceptually indiscriminable from the true spatial transformation. Several possible applications for this technique are discussed (e.g., animating contour maps and air traffic control displays so as to evoke accurate depth percepts).     

An evaluation of a "time tunnel" display format for the presentation of temporal information

The time tunnel display design technique combines the benefits of configural displays (salient visual properties corresponding to critical domain semantics) with the benefits of temporal information (i.e., the value of variables and properties over time). In Experiment 1 a baseline configural display and a time tunnel display were evaluated using real-time measures of system control, fault detection, and state estimation in a simulated process control task. The results provided little evidence in support of the time tunnel format. In Experiment 2 access to the temporal context was limited: Participants performed the detection and estimation tasks with static "snapshots" of system states that had been generated in Experiment 1. The overall pattern of results indicates that the time tunnel display was more effective for state estimation tasks than was the baseline configural display and or a trend display. Issues in the design of temporal displays are discussed, including representational formats and the choice of temporal time frames. Issues in the evaluation of temporal displays are also discussed, including the role of temporal information and the critical nature of participants' access to this information. Actual or potential applications of this research include design techniques for improving graphical displays and methodological insights to guide future evaluations.     

Effects of curved computer displays on visual performance,visual fatigue,and emotional image quality

This study investigated the effects of the curvature of curved computer displays on the visual performance, fatigue, and emotional image quality experienced by users during the performance of visual tasks. The characteristics of curved displays were compared and analyzed for different curvatures to investigate whether the curved displays showed improvements in terms of convenience of use when compared with flat displays. The visual performance results for the evaluated displays can be classified into three groups according to their response times, as (2000R = 2500R) > (3000R) > (3800R = 4500R = Flat). The visual fatigue results for average pupil size during a search task was 3.468 mm for flat and 3.355 mm for 2000R. This means that a 2000R curved display was found to be easier to use and thus generated less visual fatigue than a flat display when performing the same visual search task. Further, the results indicated that visual performance improved and visual fatigue decreased significantly as curvature increased. In this study, a curvature limit of 1700R was derived, when considering the image distortion of a 34‐in 21:9 curved display, and there was found to be no difference in the emotional image quality at curvatures greater than 2000R.     

A geometric comparison of algorithms for fusion control instereoscopic HTDs

This paper concerns stereoscopic virtual reality displays in which the head is tracked and the display is stationary, attached to a desk, tabletop or wall. These are called stereoscopic HTDs (head-tracked displays). Stereoscopic displays render two perspective views of a scene, each of which is seen by one eye of the user. Ideally, the user's natural visual system combines the stereo image pair into a single, 3D perceived image. Unfortunately, users often have difficulty fusing the stereo image pair. Researchers use a number of software techniques to reduce fusion problems. This paper geometrically examines and compares a number of these techniques and reaches the following conclusions: In interactive stereoscopic applications, the combination of view placement, scale, and either false eye separation or α-false eye separation can provide fusion control that is geometrically similar to image shifting and image scaling. However, in stereo HTDs, image shifting and image scaling also generate additional geometric artifacts that are not generated by the other methods. We anecdotally link some of these artifacts to exceeding the perceptual limitations of human vision. While formal perceptual studies are still needed, geometric analysis suggests that image shifting and image scaling may be less appropriate than the other methods for interactive, stereo HTDs     

The Visual Computing of Projector‐Camera Systems

This article focuses on real‐time image correction techniques that enable projector‐camera systems to display images onto screens that are not optimized for projections, such as geometrically complex, coloured and textured surfaces. It reviews hardware‐accelerated methods like pixel‐precise geometric warping, radiometric compensation, multi‐focal projection and the correction of general light modulation effects. Online and offline calibration as well as invisible coding methods are explained. Novel attempts in super‐resolution, high‐dynamic range and high‐speed projection are discussed. These techniques open a variety of new applications for projection displays. Some of them will also be presented in this report.     

Adaptive Image‐Space Sampling for Gaze‐Contingent Real‐time Rendering

With ever‐increasing display resolution for wide field‐of‐view displays—such as head‐mounted displays or 8k projectors—shading has become the major computational cost in rasterization. To reduce computational effort, we propose an algorithm that only shades visible features of the image while cost‐effectively interpolating the remaining features without affecting perceived quality. In contrast to previous approaches we do not only simulate acuity falloff but also introduce a sampling scheme that incorporates multiple aspects of the human visual system: acuity, eye motion, contrast (stemming from geometry, material or lighting properties), and brightness adaptation. Our sampling scheme is incorporated into a deferred shading pipeline to shade the image's perceptually relevant fragments while a pull‐push algorithm interpolates the radiance for the rest of the image. Our approach does not impose any restrictions on the performed shading. We conduct a number of psycho‐visual experiments to validate scene‐ and task‐independence of our approach. The number of fragments that need to be shaded is reduced by 50 % to 80 %. Our algorithm scales favorably with increasing resolution and field‐of‐view, rendering it well‐suited for head‐mounted displays and wide‐field‐of‐view projection.     

Near‐Eye Display and Tracking Technologies for Virtual and Augmented Reality

Virtual and augmented reality (VR/AR) are expected to revolutionise entertainment, healthcare, communication and the manufacturing industries among many others. Near‐eye displays are an enabling vessel for VR/AR applications, which have to tackle many challenges related to ergonomics, comfort, visual quality and natural interaction. These challenges are related to the core elements of these near‐eye display hardware and tracking technologies. In this state‐of‐the‐art report, we investigate the background theory of perception and vision as well as the latest advancements in display engineering and tracking technologies. We begin our discussion by describing the basics of light and image formation. Later, we recount principles of visual perception by relating to the human visual system. We provide two structured overviews on state‐of‐the‐art near‐eye display and tracking technologies involved in such near‐eye displays. We conclude by outlining unresolved research questions to inspire the next generation of researchers.     

A selective attention-based method for visual pattern recognitionwith application to handwritten digit recognition and face recognition

Parallel pattern recognition requires great computational resources; it is NP-complete. From an engineering point of view it is desirable to achieve good performance with limited resources. For this purpose, we develop a serial model for visual pattern recognition based on the primate selective attention mechanism. The idea in selective attention is that not all parts of an image give us information. If we can attend only to the relevant parts, we can recognize the image more quickly and using less resources. We simulate the primitive, bottom-up attentive level of the human visual system with a saliency scheme and the more complex, top-down, temporally sequential associative level with observable Markov models. In between, there is a neural network that analyses image parts and generates posterior probabilities as observations to the Markov model. We test our model first on a handwritten numeral recognition problem and then apply it to a more complex face recognition problem. Our results indicate the promise of this approach in complicated vision applications     

Tele-immersive environment with tiled display wall for intuitive remote collaborative work

In remote collaborative work via WAN, sharing and recognizing various high-quality visual contents such as photography, visualization image, and real video streaming is extremely important. Supporting a high-quality display of these contents on a large-scale display system is necessary to support intellectual remote collaborative work. However, these contents are currently magnified in low resolution on a general projector used as large-sized display equipment, and these equipments are difficult to display the contents with sufficient quality. In this paper, we have focused on the tiled display wall technology which configure a wide-area screen with two or more LCD and tried to display realistic high-resolution visual contents to solve these issues. We have constructed a tele-immersive environment with a tiled display wall, and have studied the availability for intuitive remote collaborative work by implementing various collaborative applications for the effective display of high-resolution contents as well as developing interaction techniques for enormous visual contents in this environment. As a result, we have showed that the practical use of a tiled display wall is useful in the construction of intellectual remote collaborative environment.     

Visually controlled graphics

Interactive graphics systems that are driven by visual input are discussed. The underlying computer vision techniques and a theoretical formulation that addresses issues of accuracy, computational efficiency, and compensation for display latency are presented. Experimental results quantitatively compare the accuracy of the visual technique with traditional sensing. An extension to the basic technique to include structure recovery is discussed     

Configural display design techniques considered at multiple levels of evaluation

Two studies were conducted to examine issues in the design and evaluation of configural displays. Four design techniques (bar graphs/extenders, scale markers/ scale grids, color coding/color layering/color separation, and annotation with digital values) were applied, alone and in combination, to a baseline configural display, forming 10 displays. Two qualitatively different evaluations assessed performance for (A) low-level data probes (quantitative estimates of individual variables) and (B) system control and fault detection tasks. Three of the four design techniques improved performance significantly for low-level data probes (color coding was the exception). A display with digital values only (i.e., no analog configural display) produced the poorest performance for control/fault detection tasks. When both levels of evaluation are considered, a composite display (configural display with all four techniques applied) was clearly the most effective. Overall, the findings obtained in the two experiments provide very limited evidence for the generalization of results between evaluations. The two levels of evaluation, the display manipulations, and the patterns of results are considered in terms of a cognitive systems engineering evaluation framework. General implications for the evaluation of displays and interfaces are discussed. Actual or potential applications include design techniques to improve graphical displays and methodological insights to focus and improve evaluation efforts.     

Visual quality of high‐contrast projection screens. Part I: Visibility of screen‐based artifacts and noise

Abstract— The investigation of visual‐quality issues for high‐contrast projection screens such as those used for avionics projection displays and other applications requiring a high level of ambient light rejection will be described. Visibility of artifacts and noise generated by the optical and structural properties of a variety of high‐contrast projection screens was characterized by both empirical, subjective image quality evaluations and a suite of objective screen‐quality metrics. At least two of these metrics were found to provide very good correlation with the subjective assessment. These metrics are applicable to any display system where spatial‐noise artifacts are an issue, such as with direct‐view AMLCDs with anti‐glare treatments.     

Task-based scanpath assessment of multi-sensor video fusion in complex scenarios

The combining of visible light and infrared visual representations occurs naturally in some creatures, including the rattlesnake. This process, and the wide-spread use of multi-spectral multi-sensor systems, has influenced research into image fusion methods. Recent advances in image fusion techniques have necessitated the creation of novel ways of assessing fused images, which have previously focused on the use of subjective quality ratings combined with computational metric assessment. Previous work has shown the need to apply a task to the assessment process; the current work continues this approach by extending the novel use of scanpath analysis. In our experiments, participants were shown two video sequences, one in high luminance (HL) and one in low luminance (LL), both featuring a group of people walking around a clearing of trees. Each participant was shown visible and infrared (IR) inputs alone; and side-by-side (SBS); in an average (AVE) fused; a discrete wavelet transform (DWT) fused; and a dual-tree complex wavelet transform (DT-CWT) fused displays. Participants were asked to track one individual in each video sequence, as well as responding by key press when other individuals carried out secondary actions. Results showed the SBS display to lead to much poorer accuracy than the other displays, while reaction times in carrying out the secondary task favoured AVE in the HL sequence and DWT in the LL sequence. Results are discussed in relation to previous findings regarding item saliency and task demands, and the potential for comparative experiments evaluating human performance when viewing fused sequences against naturally occurring fusion processes such as the rattlesnake is highlighted.     

An empirical comparison of alternative methodologies for the evaluation of configural displays

Two different methodologies (visual, memory) were used to evaluate alternative versions of the same configural display. One version (composite display) had several graphical design techniques applied, whereas the other version (baseline display) did not. Two types of information probes (high-level property, low-level data) were administered. When the displays were visible during completion of the probes (visual methodology), the display manipulation had the largest impact on performance (composite display associated with better performance); when the displays were not visible (memory methodology) the probe manipulation had the largest impact on performance (high-level probes associated with better performance). These results are interpreted in light of the mutually interacting constraints introduced by factors in display design, task requirements, and the participants' cognitive and perceptual capabilities/limitations. Implications for both the design and the evaluation of displays and interfaces in general are discussed. Actual or potential applications of this research include design techniques for improving the quality of graphic displays and methodological insights for interpreting previous research and guiding future experimentation.     

Visual display characterization using temporally modulated patterns

Abstract— A novel method to characterize CRT displays through visual assessment and a pair of visual experiments that were conducted to validate this method will be discussed. The temporal integration method discussed in this paper has many advantages over previously described methods, which have employed spatially modulated patterns to characterize displays. Included among the advantages are improvements in usability and enhanced robustness to single errors in human judgment. The observer study demonstrates that the method provides a robust estimate of CRT display gamma for both novice and experienced observers. This method may be applied to provide improved electronic‐display matching in e‐commerce applications, including online catalogs and online photofinishing.     

Introduction to building projection-based tiled display systems

This tutorial introduces the concepts and technologies needed to build projector-based display systems. Tiled displays offer scalability, high resolution, and large formats for various applications. Tiled displays are an emerging technology for constructing semi-immersive visualization environments capable of presenting high-resolution images from scientific simulation. The largest impact may well arise from using large-format tiled displays as one of possibly multiple displays in building information or active spaces that surround the user with diverse ways of interacting with data and multimedia information flows. These environments may prove the ultimate successor to the desktop metaphor for information technology work. Several fundamental technological problems must be addressed to make tiled displays practical. These include: the choice of screen materials and support structures; choice of projectors, projector supports, and optional fine positioners; techniques for integrating image tiles into a seamless whole; interface devices for interaction with applications; display generators and interfaces; and the display software environment     

Assisting people with visual impairments in aiming at a target on a large wall-mounted display

Large interactive displays have become ubiquitous in our everyday lives, but these displays are designed for the needs of sighted people. In this paper, we specifically address assisting people with visual impairments to aim at a target on a large wall-mounted display. We introduce a novel haptic device, which explores the use of vibrotactile feedback in blind user search strategies on a large wall-mounted display. Using mid-air gestures aided by vibrotactile feedback, we compared three target-aiming techniques: Random (baseline) and two novel techniques – Cruciform and Radial. The results of our two experiments show that visually impaired participants can find a target significantly faster with the Cruciform and Radial techniques than with the Random technique. In addition, they can retrieve information on a large display about twice as fast by augmenting speech feedback with haptic feedback in using the Radial technique. Although a large number of studies have been done on assistive interfaces for people who have visual impairments, very few studies have been done on large vertical display applications for them. In a broader sense, this work will be a stepping-stone for further research on interactive large public display technologies for users who are visually impaired.     

Display modulation by Fourier transform: A preferred method

Abstract— A preferred method for determining the grating modulation of a rear‐projection display using a grating image and Fourier analysis is prescribed. This method is insensitive to spatial image noise and is in better correspondence with the response of the human visual system than is the standard technique. This method is not limited to rear‐projection displays and can be applied to any display technology.     

Distributed real-time rendering for ultrahigh-resolution multiscreen 3D display

Large-scale autostereoscopic three-dimensional (3D) displays can give audiences a truly immersive feeling with strong visual impact. However, the traditional autostereoscopic 3D display systems are limited by the display hardware, making it difficult to directly achieve large-scale 3D displays with high resolution. Multiscreen splicing with laser backlights can be used for large-scale and ultrahigh-resolution 3D display, but it normally results in subscreen image asynchronization, view zone error, or obvious edge overlapping. To solve the problems mentioned above, a distributed real-time rendering system for ultrahigh-resolution multiscreen 3D display is proposed. Fifteen 3D LCD display devices are driven through a host, cooperating with laser backlights, a lenticular lens array (LLA), and a directional diffuser to display high resolution, high frame rate, large size, and wide-viewing angle 3D images. The resolution of the whole display system can reach 23,040 × 21,600. The rendering system provides a large-scale and real-time 3D scene image with an ultrahigh-definition resolution at a speed of 40 frames per second and high quality.     

Perception‐driven Accelerated Rendering

Advances in computer graphics enable us to create digital images of astonishing complexity and realism. However, processing resources are still a limiting factor. Hence, many costly but desirable aspects of realism are often not accounted for, including global illumination, accurate depth of field and motion blur, spectral effects, etc. especially in real‐time rendering. At the same time, there is a strong trend towards more pixels per display due to larger displays, higher pixel densities or larger fields of view. Further observable trends in current display technology include more bits per pixel (high dynamic range, wider color gamut/fidelity), increasing refresh rates (better motion depiction), and an increasing number of displayed views per pixel (stereo, multi‐view, all the way to holographic or lightfield displays). These developments cause significant unsolved technical challenges due to aspects such as limited compute power and bandwidth. Fortunately, the human visual system has certain limitations, which mean that providing the highest possible visual quality is not always necessary. In this report, we present the key research and models that exploit the limitations of perception to tackle visual quality and workload alike. Moreover, we present the open problems and promising future research targeting the question of how we can minimize the effort to compute and display only the necessary pixels while still offering a user full visual experience.