Smoothness perception

Despite the complexity of the Human Visual System (HVS), research over the last few decades has highlighted a number of its limitations. These limitations can be exploited in computer graphics to significantly reduce computational cost and thus required rendering time, without a viewer perceiving any difference in resultant image quality. Furthermore, cross-modal interaction between different modalities, such as the influence of audio on visual perception, has also been shown as significant both in psychology and computer graphics. In this paper we investigate the effect of beat rate on temporal visual perception, i.e. frame rate perception. For the visual quality and perception evaluation, a series of psychophysical experiments was conducted and the data analysed. The results indicate that beat rates in some cases do affect temporal visual perception and that certain beat rates can be used in order to reduce the amount of rendering required to achieve a perceptual high quality. This is another step towards a comprehensive understanding of auditory-visual cross-modal interaction and could be potentially used in high-fidelity interactive multi-sensory virtual environments.     

高质量的三维纹理硬件体绘制

与光线投射法相比,传统的3D纹理体绘制算法通常难以产生高质量的图像。为了增强渲染图像的真实感与质量,在基于GPU（Graphics Processing Unit）的三维纹理体绘制过程中以交互的速率实现了体阴影效果,并考虑现实图像合成中的可视化感知,提出将基于GPU的高动态范围色调映射技术应用到体绘制得到的结果图片中。最后对一些体数据集进行绘制,实验表明这些技术较好地解决了传统纹理绘制方法的缺点,提高了图像的质量。     

HVS模型及其在静止图象压缩质量评价中的应用

图象质量尺度是最优化图象压缩算法参数和提高图象质量的重要依据 .建立在人类视觉模型 (HVS)基础之上的感知质量尺度作为主客观联系的桥梁 ,能有效地反映出人对图象失真在视觉上的感知 .近年来 ,有许多研究者借助人类视觉系统研究中的最新成果 ,深入分析了与图象质量密切相关的视觉感知特性 ,提出了大量效果不错的静止图象压缩感知质量尺度 ,并对视觉感知特性在图象质量尺度中的应用方法进行了较全面的综述 ,揭示了影响其图象质量预测准确性、鲁棒性的主要因素 ,给出了该领域的最新研究成果和未来发展方向 .     

Analysis by synthesis in computational vision with application to remote sensing

The central problem in vision is to determine scene properties from image properties. This is difficult because the problem, formally posed, is underconstrained. Methods that infer scene properties from images make assumptions about how the world determines what we see. In remote sensing, some of these assumptions can be dealt with explicitly. Available scene knowledge, in the form of a digital terrain model and a ground cover map, is used to synthesize an image for a given date and time. The synthesis process assumes that the surface is a perfectly diffuse or "lambertian" reflector. A scene radiance equation is described based on simple models of direct solar irradiance, diffuse sky irradiance, and atmospheric path radiance. Parameters of the model are estimated from the real image. A statistical comparison of the real image and the synthetic image is used to judge how well the model represents the mapping from scene to image.
The methods presented for image synthesis are similar to those used in computer graphics. The motivation, however, is different. In graphics, the goal is to produce an effective rendering of the scene for a human viewer. Here, the goal is to predict properties of real images. In vision, one must deal with a confounding of effects due to surface shape, surface material, illumination, shadows, and atmosphere. These effects often detract from, rather than enhance, the determination of invariant scene characteristics.     

Steering Image Generation with Wavelet Based Perceptual Metric

It is often the case that images generated by image synthesis algorithms are judged by visual examination. The user resorts to an iterative refinement process of inspection and rendering until a satisfactory image is obtained. In this paper we propose quantitative metrics to compare images that arise from an image synthesis algorithm. The intent is to be able to guide the refinement process inherent in image synthesis. The Mean-Square-Error (MSE) has been traditionally employed to guide this process. However, it is not a viable metric for image synthesis control. We propose the use of a wavelet based perceptual metric which incorporates the frequency response of the Human Visual System. A useful aspect of the wavelet based metric is its ability to selectively measure the changes to structures of different sizes and scales in specific locations. Also, by resorting to the use of wavelets of various degrees of regularity, one can seek different levels of smoothness in an image. It is rare that such level of control can be obtained from a metric other than a wavelet based metric. We show the usefulness of our metric by examining its effectiveness in providing insights for common operations of an image synthesis algorithm (e.g., blurring). We also provide some examples of its use in rendering algorithms frequently used in graphics.     

Photorealistic rendering of mixed reality scenes

Photo‐realistic rendering of virtual objects into real scenes is one of the most important research problems in computer graphics. Methods for capture and rendering of mixed reality scenes are driven by a large number of applications, ranging from augmented reality to visual effects and product visualization. Recent developments in computer graphics, computer vision, and imaging technology have enabled a wide range of new mixed reality techniques including methods for advanced image based lighting, capturing spatially varying lighting conditions, and algorithms for seamlessly rendering virtual objects directly into photographs without explicit measurements of the scene lighting. This report gives an overview of the state‐of‐the‐art in this field, and presents a categorization and comparison of current methods. Our in‐depth survey provides a tool for understanding the advantages and disadvantages of each method, and gives an overview of which technique is best suited to a specific problem.     

Verification of Physically Based Rendering Algorithms

Within computer graphics, the field of predictive rendering is concerned with those methods of image synthesis that yield results that do not only look real, but are also radiometrically correct renditions of nature, i.e. which are accurate predictions of what a real scene would look like under given lighting conditions. In order to guarantee the correctness of the results obtained by such techniques, three stages of such a rendering system have to be verified with particular care: the light reflection models, the light transport simulation and the perceptually based calculations used at display time. In this report, we will concentrate on the state of the art with respect to the second step in this chain. Various approaches for experimental verification of the implementation of a physically based rendering system have been proposed so far. However, the problem of proving that the results are correct is not fully solved yet, and no standardized methodology is available. We give an overview of existing literature, discuss the strengths and weaknesses of the described methods and illustrate the unsolved problems. We also briefly discuss the related issue of image quality metrics.     

Selective rendering for efficient ray traced stereoscopic images

Depth-related visual effects are a key feature of many virtual environments. In stereo-based systems, the depth effect can be produced by delivering frames of disparate image pairs, while in monocular environments, the viewer has to extract this depth information from a single image by examining details such as perspective and shadows. This paper investigates via a number of psychophysical experiments, whether we can reduce computational effort and still achieve perceptually high-quality rendering for stereo imagery. We examined selectively rendering the image pairs by exploiting the fusing capability and depth perception underlying human stereo vision. In ray-tracing-based global illumination systems, a higher image resolution introduces more computation to the rendering process since many more rays need to be traced. We first investigated whether we could utilise the human binocular fusing ability and significantly reduce the resolution of one of the image pairs and yet retain a high perceptual quality under stereo viewing condition. Secondly, we evaluated subjects’ performance on a specific visual task that required accurate depth perception. We found that subjects required far fewer rendered depth cues in the stereo viewing environment to perform the task well. Avoiding rendering these detailed cues saved significant computational time. In fact it was possible to achieve a better task performance in the stereo viewing condition at a combined rendering time for the image pairs less than that required for the single monocular image. The outcome of this study suggests that we can produce more efficient stereo images for depth-related visual tasks by selective rendering and exploiting inherent features of human stereo vision.     

Realistic or Abstract Imagery: The Future of Computer Graphics?

The big idea in computer graphics, what makes CG different from other ways of making images, is that CG represents images symbolically. The artist or designer creates a symbolic representation of the image, and the computer converts that representation to physical media. Because computational processes are so flexible, we have the freedom to invent any abstract representation that suits our needs. Somewhat surprisingly, most of computer graphics research has focused on the science and technology needed to make photorealistic images representing the physical world. In this talk, I will argue that we should shift our focus to developing techniques for manipulating abstract image representations. Historically, abstract imagery is more recent and more innovative than realistic imagery. Functionally, abstract image representations are often more informative and more expressive than realistic ones. More fundamentally, abstract image models better depict our mental models of the world, and are hence more useful to most people that use computer graphics in their work. In addition to motivating this line of research, I will outline some potentially promising research directions.     

航天器飞行场景的真实感生成

太空场景及航天器动态飞行场景的真实感生成是计算机图形学研究者面临的新课题．该文提出了一种航天器飞行场景生成的新方法．该方法首先基于天文学的依巴谷星表,采用了基于图像的建模绘制及层次细节简化等技术加速绘制,准确地构造并绘制出不同时刻、不同观察点的地球外星空背景随时间的动态变化;再根据天体物理学原理确定出航天器的飞行轨迹,由此合成产生了航天器的太空飞行场景．然后根据量子光学及等离子体辐射原理对航天器再入大气层的过程进行了建模,确定出其再入轨迹、航天器尾焰和尾迹的颜色和几何形态．最后逼真地绘制出航天器返回时穿越大气不同阶段所产生的特异尾迹形状和发光效果．     

Multi‐Modal Perception for Selective Rendering

A major challenge in generating high‐fidelity virtual environments (VEs) is to be able to provide realism at interactive rates. The high‐fidelity simulation of light and sound is still unachievable in real time as such physical accuracy is very computationally demanding. Only recently has visual perception been used in high‐fidelity rendering to improve performance by a series of novel exploitations; to render parts of the scene that are not currently being attended to by the viewer at a much lower quality without the difference being perceived. This paper investigates the effect spatialized directional sound has on the visual attention of a user towards rendered images. These perceptual artefacts are utilized in selective rendering pipelines via the use of multi‐modal maps. The multi‐modal maps are tested through psychophysical experiments to examine their applicability to selective rendering algorithms, with a series of fixed cost rendering functions, and are found to perform significantly better than only using image saliency maps that are naively applied to multi‐modal VEs.     

Elements of early vision for computer graphics

Visually based techniques in computer graphics have blossomed. Important advances in perceptually driven rendering, realistic image display, high-fidelity visualization, and appearance-preserving geometric simplification have all been realized by applying knowledge of the limitations and capabilities of human visual processing. Much of this work is grounded in the physiology and psychophysics of early vision, which focuses on how visual mechanisms transduce and code the patterns of light arriving at the eye. The article surveys some of the fundamental findings in the study of early vision including basic visual anatomy and physiology, optical properties of the eye, light sensitivity and visual adaptation, and spatial vision     

State of the Art on Neural Rendering

Efficient rendering of photo-realistic virtual worlds is a long standing effort of computer graphics. Modern graphics techniques have succeeded in synthesizing photo-realistic images from hand-crafted scene representations. However, the automatic generation of shape, materials, lighting, and other aspects of scenes remains a challenging problem that, if solved, would make photo-realistic computer graphics more widely accessible. Concurrently, progress in computer vision and machine learning have given rise to a new approach to image synthesis and editing, namely deep generative models. Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e.g., by the integration of differentiable rendering into network training. With a plethora of applications in computer graphics and vision, neural rendering is poised to become a new area in the graphics community, yet no survey of this emerging field exists. This state-of-the-art report summarizes the recent trends and applications of neural rendering. We focus on approaches that combine classic computer graphics techniques with deep generative models to obtain controllable and photorealistic outputs. Starting with an overview of the underlying computer graphics and machine learning concepts, we discuss critical aspects of neural rendering approaches. Specifically, our emphasis is on the type of control, i.e., how the control is provided, which parts of the pipeline are learned, explicit vs. implicit control, generalization, and stochastic vs. deterministic synthesis. The second half of this state-of-the-art report is focused on the many important use cases for the described algorithms such as novel view synthesis, semantic photo manipulation, facial and body reenactment, relighting, free-viewpoint video, and the creation of photo-realistic avatars for virtual and augmented reality telepresence. Finally, we conclude with a discussion of the social implications of such technology and investigate open research problems.     

HVS：一种基于实景图象的虚拟现实系统

大多数的虚拟现实比试在于计算机图形技术,先由多边形构造虚拟场景的三维几何模型,再由计算机根据用户的观察点和观察方向实时绘制出用户所看到的虚拟场景,ＨＶＳ 另一种思路,由计算机自动拉接、变形与组织许多幅度散的实景图象或连续视频,生成虚拟场景,这种虚拟场人有照片质量的视觉效果,被我们称为虚拟实景空间,虚拟实时空间能为用户提供、后退、仰视、俯视、３６０度环境、近看、远看等漫游能力,运行它不需要高性能的图     

基于生成感知差异的无参考图像质量评价模型

人眼视觉系统中的视觉感知差异是图像质量评价过程中的重要组成部分,通过感知失真图像与原始图像之间的视觉差异,可对图像的失真程度进行判断,然而在无参考图像质量评价中无法获取原始未失真的图像,且缺乏对失真图像的视觉感知差异。通过对深度学习中的生成对抗网络进行分析,提出一种基于生成视觉感知差异的无参考图像质量评价模型。利用生成对抗网络产生与失真图像相对应的视觉感知差异图像,并将其与失真图像输入质量评价网络以进一步学习图像的失真信息,从而达到评估图像质量的目的。在TID2008和TID2013数据库上的实验结果表明,与CNN、SOM、CORNIA等模型相比,该模型能够使失真图像质量预测准确度提升1个百分点以上,且对不同种类失真也表现出良好的预测性能。     

Colour image compression based on the measure of just noticeable colour difference

To the human vision, there exists in colour images a certain amount of perceptual redundancy since the human visual system (HVS) has limited sensitivity in discriminating colour signals of small differences. By measuring the perceptual redundancy inherent in colour images and shaping the coding distortion into the perceptual redundancy, colour images are expected to be represented more efficiently. Approaches to perceptually optimise the efficiency of image coders in compressing colour images with the perceptual redundancy estimated by a colour visual model are presented. The model estimates the perceptual redundancy for each colour pixel as a visibility threshold of colour difference in any colour space and in a spatial or frequency domain. Two existing image coders are modified to take advantage of the perceptual redundancy and simulated to inspect if their coding efficiency is improved. In the spatial domain, the JPEG-LS coder in the near-lossless compression mode is modified to make coding errors part of the perceptual redundancy in compressing colour images in the RGB space. In the wavelet domain, the JPEG2000 coder is refined by minimising the perceptible distortion involved in the rate control of the compressed image in the YC/sub b/C/sub r/space. Simulation results show that, in both cases, the performance of the perceptually tuned coder is superior to that of the un-tuned coder in terms of the bit rate required for achieving the same visual quality.     

Image analysis and computer vision: 1990

This paper presents a bibliography of over 1600 references related to computer vision and image analysis, arranged by subject matter. The topics covered include architectures; computational techniques; feature detection, segmentation, and image analysis; matching, stereo, and time-varying imagery; shape and pattern; color and texture; and three-dimensional scene analysis. A few references are also given on related topics, such as computational geometry, computer graphics, image input/output and coding, image processing, optical processing, visual perception, neural nets, pattern recognition, and artificial intelligence.     

基于OpenGL的真实感图形绘制技术及应用

科学计算可视化, 计算机动画和虚拟现实是现代计算机图形学的三个热点, 而这三个热点的核心都是三维真实感图形的绘制。通过VC + + 环境下的编程实践, 系统地总结了基于OpenGL 技术实现真实感图形绘制( 包括反走样、消隐以及光照模型等) 的基本步骤和方法, 并结合具体实例, 介绍了真实感图形绘制技术在地矿工程三维实体构模中的应用。