Simulation of color breakup based on measured display temporal responses

Based on the measured temporal luminance behavior of each field, the perceived color breakup (CBU) artifact for moving block on a field sequential color display is simulated and predicted. The prediction accuracy is evaluated by a perceptual experiment on a 240 Hz liquid crystal display. The results show a perfect matching between the simulation and the subjects' actual visual experience. The simulation also works well when motion compensation is applied. It is validated by another perceptual experiment that a perfect or a near‐perfect compensation degree can suppress CBU significantly in field sequential color system. CBU of nature image was also simulated and reproduced for both stable and pulse backlight by using this model, which could be rather valuable for the further development of CBU metrics.     

A new field‐sequential‐color LCD without moving‐object color break‐up

Abstract— A new display method for field‐sequential‐color liquid‐crystal displays (FS‐LCDs) that reduces the negative effects of color break‐up associated with moving objects has been developed. The method is called Adjustment of Color Element on the Eyes (ACE), and it relies on the position on the eyes of RGB color sub‐images. It was confirmed that color break‐up also does not occur for peripheral objects when using ACE.     

Multi‐color LC/BL algorithm in field‐sequential‐color LCD for color‐breakup suppression

Abstract— The proposed liquid‐crystal and backlight (LC/BL) algorithm presents the dynamic field‐sequential‐color (D‐FSC) algorithm to reduce the color‐breakup (CBU) effect without greatly increasing the subframe rate. The D‐FSC algorithm can intelligently select one adequate color sequence from multiple color sequences according to the image data. In other words, the scope of CBU suppression of the proposed LC/BL algorithm is more extensive than other conventional FSCs. Simulation results show that the CBU suppression can be improved substantially by the proposed evaluation equation.     

Color‐breakup suppression and low‐power consumption by using the Stencil‐FSC method in field‐sequential LCDs

Abstract— Field‐sequential color (FSC) is a potential technique for low‐power liquid‐crystal displays (LCDs). However, it still experiences a serious visual artifact, color break‐up (CBU), which degrades image quality. Consequently, the “Stencil Field‐Sequential‐Color (Stencil‐FSC)” method, which applies local color‐backlight‐dimming technology at a 240‐Hz field rate to FSC‐LCDs, is proposed. Using the Stencil‐FSC method not only suppressed CBU efficiently but also enhanced the image contrast ratio by using low average power consumption. After backlight signal optimization, the Stencil‐FSC method was demonstrated on a 32‐in. FSC‐LCD and effectively suppressed the CBU, which resulted in more than a 27,000:1 dynamic contrast ratio and less than 40‐W average power consumption.     

Influence of 3‐D cross‐talk on qualified viewing spaces in two‐ and multi‐view autostereoscopic displays

Abstract— To estimate the qualified viewing spaces for two‐ and multi‐view autostereoscopic displays, the relationship between image quality (image comfort, annoying ghost image, depth perception) and various pairings between 3‐D cross‐talk in the left and right views are studied subjectively using a two‐view autostereoscopic display and test charts for the left and right views with ghost images due to artificial 3‐D cross‐talk. The artificial 3‐D cross‐talk was tuned to simulate the view in the intermediate zone of the viewing spaces. It was shown that the stereoscopic images on a two‐view autostereoscopic display cause discomfort when they are observed by the eye in the intermediate zone between the viewing spaces. This is because the ghost image due to large 3‐D cross‐talk in the intermediate zone elicits different depth perception from the depth induced by the original images for the left and right views, so the observer's depth perception is confused. Image comfort is also shown to be better for multi‐views, especially the width of the viewing space, which is narrower than the interpupillary distance, where the parallax of the cross‐talking image is small.     

Modeling motion‐induced color artifacts from the temporal step response

Abstract— LCD motion blur is a well‐known phenomenon, and a lot of research is attributed to characterize and improve it. Until recently, most studies were focused on explaining the effects visible in black‐and‐white patterns, and hence color effects were ignored. However, when a colored pattern is moving over a colored background, an additional motion‐induced artifact becomes visible, which is referred to as chromatic aberration. To describe this phenomenon, our model to characterize the appearance of moving achromatic patterns is extended in such a way that it now calculates the apparent image from the temporal step response of the individual primary colors. The results of a perception experiment indicate that there is a good correspondence between the apparent image predicted with the model and the actual image perceived during motion.     

Foveated Real‐Time Ray Tracing for Head‐Mounted Displays

Head‐mounted displays with dense pixel arrays used for virtual reality applications require high frame rates and low latency rendering. This forms a challenging use case for any rendering approach. In addition to its ability of generating realistic images, ray tracing offers a number of distinct advantages, but has been held back mainly by its performance. In this paper, we present an approach that significantly improves image generation performance of ray tracing. This is done by combining foveated rendering based on eye tracking with reprojection rendering using previous frames in order to drastically reduce the number of new image samples per frame. To reproject samples a coarse geometry is reconstructed from a G‐Buffer. Possible errors introduced by this reprojection as well as parts that are critical to the perception are scheduled for resampling. Additionally, a coarse color buffer is used to provide an initial image, refined smoothly by more samples were needed. Evaluations and user tests show that our method achieves real‐time frame rates, while visual differences compared to fully rendered images are hardly perceivable. As a result, we can ray trace non‐trivial static scenes for the Oculus DK2 HMD at 1182 × 1464 per eye within the the VSync limits without perceived visual differences.     

Image content adaptive color breakup index for field sequential color displays using a dominant visual saliency method

An index that can predict the perceptual visibility of color breakup for varying image content is valuable in field sequential color displays, whereas the current indices are usually for fixed patterns. To solve this problem, an image database containing 25 diverse reference images and 125 test cases with various color breakup visibility was first established. Next, visual experiments using a 240‐Hz liquid crystal display were performed to acquire the subjective color breakup scores of the test cases. A theorem based on visual saliency theory was proposed that the color breakup perception is mainly determined by the image regions with visual saliency values higher than a certain threshold, called the dominant visual saliency regions. A computational model based on this theorem was developed to obtain objective color breakup scores of the test cases from retinal images with and without color breakup. An analysis of the objective and subjective results revealed a Pearson linear correlation coefficient as high as 0.82, which matches the top‐level image quality assessment algorithms. Finally, the proposed color breakup index was used to benchmark against several mainstream field sequential color algorithms to determine their performances in color breakup suppression.     

A field‐sequential‐color display with a local‐primary‐desaturation backlight scheme

Abstract— Field‐sequential‐color technology eliminates the need for color filters in liquid‐crystal displays (LCDs) and results in significant power savings and higher resolution. But the LCD suffers from color breakup, which degrades image quality and limits practical applications. By controlling the backlight temporally and spatially, a so‐called local‐primary‐desaturation (LPD) backlight scheme was developed and implemented in a 180‐Hz optically compensated bend (OCB) mode LCD equipped with a backlight consisting of a matrix of light‐emitting diodes (LEDs). It restores image quality by suppressing color breakup and saves power because it has no color filter and uses local dimming. A perceptual experiment was implemented for verification, and the results showed that a field‐sequential‐color display with a local‐primary‐desaturation backlight reduced the color breakup from very annoying to not annoying and even invisible.     

Visual remapping by vector subtraction: analysis of multiplicative gain field models

Saccadic eye movements remain spatially accurate even when the target becomes invisible and the initial eye position is perturbed. The brain accomplishes this in part by remapping the remembered target location in retinal coordinates. The computation that underlies this visual remapping is approximated by vector subtraction: the original saccade vector is updated by subtracting the vector corresponding to the intervening eye movement. The neural mechanism by which vector subtraction is implemented is not fully understood. Here, we investigate vector subtraction within a framework in which eye position and retinal target position signals interact multiplicatively (gain field). When the eyes move, they induce a spatial modulation of the firing rates across a retinotopic map of neurons. The updated saccade metric can be read from the shift of the peak of the population activity across the map. This model uses a quasi-linear (half-rectified) dependence on the eye position and requires the slope of the eye position input to be negatively proportional to the preferred retinal position of each neuron. We derive analytically this constraint and study its range of validity. We discuss how this mechanism relates to experimental results reported in the frontal eye fields of macaque monkeys.     

Video 2‐D—to—3‐D conversion based on hybrid depth cueing

Abstract— With the maturation of three‐dimensional (3‐D) technologies, display systems can provide higher visual quality to enrich the viewer experience. However, the depth information required for 3‐D displays is not available in conventional 2‐D recorded contents. Therefore, the conversion of existing 2‐D video to 3‐D video becomes an important issue for emerging 3‐D applications. This paper presents a system which automatically converts 2‐D videos to 3‐D format. The proposed system combines three major depth cues: the depth from motion, the scene depth from geometrical perspective, and the fine‐granularity depth from the relative position. The proposed system uses a block‐based method incorporating a joint bilateral filter to efficiently generate visually comfortable depth maps and to diminish the blocky artifacts. By means of the generated depth map, 2‐D videos can be readily converted into 3‐D format. Moreover, for conventional 2‐D displays, a 2‐D image/video depth perception enhancement application is also presented. With the depth‐aware adjustment of color saturation, contrast, and edge, the stereo effect of the 2‐D content can be enhanced. A user study on subjective quality shows that the proposed method has promising results on depth quality and visual comfort.     

Density‐enhanced perceptual mosaic on GPU

Image mosaic effects are wildly applied in print media, domestic decoration, and many image beautification applications. However, the current image mosaic methods are mostly based on fixed‐size image tiles, simple color adjustment, and irregular image segmentation, which are inaccurate and very time‐consuming. In this paper, we present a graphics processing unit‐accelerated perceptual mosaic using density tiles replacement and brightness lighting optimization, keeping original image structure details and providing more expressive visual effects. Automatic density replacement map segmentation and color‐based region tiles replacement are performed to facilitate the mosaic. Delicate brightness optimization and perceptual color correction are further applied to enhance expressive lighting effects. We also consider the salience perception of images and similarity correlation among neighboring tiles for our perceptual mosaic. The experimental results have shown the efficiency and high‐quality performance of our density‐enhanced perceptual mosaic on graphics processing unit. Copyright © 2016 John Wiley & Sons, Ltd.     

Adapting softcopy color reproduction to ambient illumination

Abstract— The perceived colors of an image seen on a self‐luminous display are affected by ambient illumination. The ambient light reflected from the display faceplate is mixed with the image‐forming light emitted by the display. In addition to this direct physical effect of viewing flare, ambient illumination causes perceptual changes by affecting the adaptation state of the viewer's visual system. This paper first discusses these effects and how they can be compensated, outlining a display system able to adjust its output based on prevailing lighting conditions. The emphasis is on compensating for the perceptual effects of viewing conditions by means of color‐appearance modeling. The effects of varying the degree of chromatic adaptation parameter D and the surround compensation parameters c and N_c of the CIECAM97s color‐appearance model were studied in psychophysical experiments. In these memory‐based paired comparison experiments, the observers judged the appearance of images shown on an LCD under three different ambient‐illumination conditions. The dependence of the optimal parameter values on the level of ambient illumination was evident. The results of the final experiment, using a category scaling technique, showed the benefit of using the color‐appearance model with the optimized parameters in compensating for the perceptual changes caused by varying ambient illumination.     

Eye‐Centered Color Adaptation in Global Illumination

Color adaptation is a well known ability of the human visual system (HVS). Colors are perceived as constant even though the illuminant color changes. Indeed, the perceived color of a diffuse white sheet of paper is still white even though it is illuminated by a single orange tungsten light, whereas it is orange from a physical point of view. Unfortunately global illumination algorithms only focus on the physics aspects of light transport. The ouput of a global illuminantion engine is an image which has to undergo chromatic adaptation to recover the color as perceived by the HVS. In this paper, we propose a new color adaptation method well suited to global illumination. This method estimates the adaptation color by averaging the irradiance color arriving at the eye. Unlike other existing methods, our approach is not limited to the view frustrum, as it considers the illumination from all the scene. Experiments have shown that our method outperforms the state of the art methods.     

Passive‐matrix‐driven field‐sequential‐color displays

Abstract— Passive‐matrix‐driven field‐sequential‐color (FSC) displays were successfully fabricated. It makes use of a new multiplex driving scheme that does not depend on voltage averaging. Instead, a transient response of the liquid crystal is employed. An addressing and response time of less than 70 μsec and 2.0 msec, respectively, are used. Scanning time compensation is also introduced to improve the brightness uniformity of the display.     

Mechanism of color breakup in field-sequential-color projectors

Abstract— To investigate color breakup in field-sequential-color projectors, we measured the threshold of color fusion frequency relative to retinal velocity of the moving edge on the color flicker field. The characteristic changed as the velocity approached saccade. The result suggests that the mechanism of color breakup differs according to the velocity of the image on the retina.     

Evaluation of motion performance on scanning‐backlight LCDs

Abstract— The scanning‐backlight technique to improve the motion performance of LCDs is introduced. This technique, however, has some drawbacks such as double edges and color aberration, which may become visible in moving patterns. A method combining accurate measurements of temporal luminance transitions with the simulation of human‐eye tracking and spatiotemporal integration is used to model the motion‐induced profile of an edge moving on a scanning‐backlight LCD‐TV panel that exhibits the two drawbacks mentioned above. The model results are validated with a perception experiment including different refresh rates, and a high correspondence is found between the simulated apparent edge and the one that is perceived during actual motion. Apart from the motion‐induced edge blur, the perception of a moving line or square‐wave grating can also be predicted by the same method starting from the temporal impulse and frame‐sequential response curves, respectively. Motion‐induced image degradation is evaluated for both a scanning‐ and continuous‐backlight mode based on three different characteristics: edge blur, line spreading, and modulation depth of square‐wave grating. The results indicate that the scanning‐backlight mode results in better motion performance.     

Review Paper: The Helmholtz‐Kohlrausch effect

Abstract— The Helmholtz‐Kohlrausch (H‐K) effect is the influence of color purity on the perceived brightness of a color object (or source). In addition to a review of the effect, a survey of color and brightness‐perception studies from 1825 to the present (including our own studies on disabling glare) is presented. Disabling glare is the blinding experience which results from a bright source in our field of view. There has been a great deal of work on the H‐K effect, and this paper is our personal view of the subject. The H‐K effect has lead to new color‐appearance models which help to describe the brightness/luminance discrepancies, and we will only touch on this subject. It will be shown how the H‐K effect affects many of the displays, mobile devices, and phones. Additionally, how the H‐K effect is involved with our perception at mesopic light levels encountered during night driving will be discussed.     

A wide view auto‐stereoscopic 3D display with an eye‐tracking system for enhanced horizontal viewing position and viewing distance

This paper describes the development of auto‐stereoscopic three‐dimensional (3D) display with an eye‐tracking system for not only the X‐axis (right–left) and Y‐axis (up–down) plane directions but also the Z‐axis (forward–backward) direction. In the past, the eye‐tracking 3D system for the XY‐axes plane directions that we had developed had a narrow 3D viewing space in the Z‐axis direction because of occurrence of 3D crosstalk variation on screen. The 3D crosstalk variation on screen was occurred when the viewer's eye position moved back and forth along the Z‐axis direction. The reason was that the liquid crystal (LC) barrier pitch was fixed and the LC barrier was able to control the only barrier aperture position. To solve this problem, we developed the LC barrier that is able to control the barrier pitch as well as the barrier aperture position in real time, corresponding to the viewer's eye position. As a result, the 3D viewing space has achieved to expand up to 320–1016 mm from the display surface in the Z‐axis direction and within a range of ±267 mm in the X‐axis direction. In terms of the Y‐axis direction, the viewing space is not necessary to be considered, because of a stripe‐shaped parallax barrier.     

Viewpoint image generation for head tracking 3D display using multi‐camera and approximate depth information

A simple and high image quality method for viewpoint image synthesis from multi‐camera images for a stereoscopic 3D display using head tracking is proposed. In this method, slices of images for depth layers are made using approximate depth information, the slices are linearly blended corresponding to the distance between the viewpoint and cameras at each layer, and the layers are overlaid from the perspective of viewpoint. Because the linear blending automatically compensates for depth error because of the visual effects of depth‐fused 3D (DFD), the resulting image is natural to observer's perception. Smooth motion parallax of wide depth range objects induced by viewpoint movement for left‐and‐right and front‐and‐back directions is achieved using multi‐camera images and approximate depth information. Because the calculation algorithm is very simple, it is suitable for real time 3D display applications.