Visualizing features and tracking their evolution

We describe basic algorithms to extract coherent amorphous regions (features or objects) from 2 and 3D scalar and vector fields and then track them in a series of consecutive time steps. We use a combination of techniques from computer vision, image processing, computer graphics, and computational geometry and apply them to data sets from computational fluid dynamics. We demonstrate how these techniques can reduce visual clutter and provide the first step to quantifying observable phenomena. These results can be generalized to other disciplines with continuous time-dependent scalar (and vector) fields     

A Survey of Simple Geometric Primitives Detection Methods for Captured 3D Data

The amount of captured 3D data is continuously increasing, with the democratization of consumer depth cameras, the development of modern multi‐view stereo capture setups and the rise of single‐view 3D capture based on machine learning. The analysis and representation of this ever growing volume of 3D data, often corrupted with acquisition noise and reconstruction artefacts, is a serious challenge at the frontier between computer graphics and computer vision. To that end, segmentation and optimization are crucial analysis components of the shape abstraction process, which can themselves be greatly simplified when performed on lightened geometric formats. In this survey, we review the algorithms which extract simple geometric primitives from raw dense 3D data. After giving an introduction to these techniques, from the acquisition modality to the underlying theoretical concepts, we propose an application‐oriented characterization, designed to help select an appropriate method based on one's application needs and compare recent approaches. We conclude by giving hints for how to evaluate these methods and a set of research challenges to be explored.     

State of the Art on 3D Reconstruction with RGB‐D Cameras

The advent of affordable consumer grade RGB‐D cameras has brought about a profound advancement of visual scene reconstruction methods. Both computer graphics and computer vision researchers spend significant effort to develop entirely new algorithms to capture comprehensive shape models of static and dynamic scenes with RGB‐D cameras. This led to significant advances of the state of the art along several dimensions. Some methods achieve very high reconstruction detail, despite limited sensor resolution. Others even achieve real‐time performance, yet possibly at lower quality. New concepts were developed to capture scenes at larger spatial and temporal extent. Other recent algorithms flank shape reconstruction with concurrent material and lighting estimation, even in general scenes and unconstrained conditions. In this state‐of‐the‐art report, we analyze these recent developments in RGB‐D scene reconstruction in detail and review essential related work. We explain, compare, and critically analyze the common underlying algorithmic concepts that enabled these recent advancements. Furthermore, we show how algorithms are designed to best exploit the benefits of RGB‐D data while suppressing their often non‐trivial data distortions. In addition, this report identifies and discusses important open research questions and suggests relevant directions for future work.     

Internet visual media processing: a survey with graphics and vision applications

In recent years, the computer graphics and computer vision communities have devoted significant attention to research based on Internet visual media resources. The huge number of images and videos continually being uploaded by millions of people have stimulated a variety of visual media creation and editing applications, while also posing serious challenges of retrieval, organization, and utilization. This article surveys recent research as regards processing of large collections of images and video, including work on analysis, manipulation, and synthesis. It discusses the problems involved, and suggests possible future directions in this emerging research area.     

树的建模技术研究综述与展望

树的建模技术是计算机图形学近年来研究的热点问题之一.侧重从计算机图形学的研究角度,对树的建模技术分别从基于规则的树建模技术、基于草图的树建模技术、基于图像的树建模技术3个方面进行了总结和综述,介绍了近年来提出的典型的树建模方法及最新研究进展,对其中涉及的关键技术进行了总结分析,给出了这些技术的基本思想、局限性和使用范围,并加以分析比较,最后对树的建模技术的未来研究方向给出展望.     

State of the Art on Neural Rendering

Efficient rendering of photo-realistic virtual worlds is a long standing effort of computer graphics. Modern graphics techniques have succeeded in synthesizing photo-realistic images from hand-crafted scene representations. However, the automatic generation of shape, materials, lighting, and other aspects of scenes remains a challenging problem that, if solved, would make photo-realistic computer graphics more widely accessible. Concurrently, progress in computer vision and machine learning have given rise to a new approach to image synthesis and editing, namely deep generative models. Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e.g., by the integration of differentiable rendering into network training. With a plethora of applications in computer graphics and vision, neural rendering is poised to become a new area in the graphics community, yet no survey of this emerging field exists. This state-of-the-art report summarizes the recent trends and applications of neural rendering. We focus on approaches that combine classic computer graphics techniques with deep generative models to obtain controllable and photorealistic outputs. Starting with an overview of the underlying computer graphics and machine learning concepts, we discuss critical aspects of neural rendering approaches. Specifically, our emphasis is on the type of control, i.e., how the control is provided, which parts of the pipeline are learned, explicit vs. implicit control, generalization, and stochastic vs. deterministic synthesis. The second half of this state-of-the-art report is focused on the many important use cases for the described algorithms such as novel view synthesis, semantic photo manipulation, facial and body reenactment, relighting, free-viewpoint video, and the creation of photo-realistic avatars for virtual and augmented reality telepresence. Finally, we conclude with a discussion of the social implications of such technology and investigate open research problems.     

三维视觉前沿进展

在自动驾驶、机器人、数字城市以及虚拟/混合现实等应用的驱动下,三维视觉得到了广泛的关注。三维视觉研究主要围绕深度图像获取、视觉定位与制图、三维建模及三维理解等任务而展开。本文围绕上述三维视觉任务,对国内外研究进展进行了综合评述和对比分析。首先,针对深度图像获取任务,从非端到端立体匹配、端到端立体匹配及无监督立体匹配3个方面对立体匹配研究进展进行了回顾,从深度回归网络和深度补全网络两个方面对单目深度估计研究进展进行了回顾。其次,针对视觉定位与制图任务,从端到端视觉定位和非端到端视觉定位两个方面对大场景下的视觉定位研究进展进行了回顾,并从视觉同步定位与地图构建和融合其他传感器的同步定位与地图构建两个方面对同步定位与地图构建的研究进展进行了回顾。再次,针对三维建模任务,从深度三维表征学习、深度三维生成模型、结构化表征学习与生成模型以及基于深度学习的三维重建等4个方面对三维几何建模研究进展进行了回顾,并从多视RGB重建、单深度相机和多深度相机方法以及单视图RGB方法等3个方面对人体动态建模研究进展进行了回顾。最后,针对三维理解任务,从点云语义分割和点云实例分割两个方面对点云语义理解研究进展进行了回顾。在此基础上,给出了三维视觉研究的未来发展趋势,旨在为相关研究者提供参考。     

深度相机在计算机视觉与图形学上的应用研究(英文)

越来越多的应用如几何重建、碰撞检测、混合现实、手势识别等,都依赖于对三维场景准确且快速的分析。通过基于图像的分析或者激光扫描技术来获取场景的深度图,其代价高昂且十分耗时。作为距离测量中一种可替代的设备,深度相机拥有传统的三维测量系统所不具备的一些优点,如较低的价格以及较高的拍摄速度等。最近出现了一些小巧低廉的深度相机设备,这将给计算机视觉、计算机图形学、人机交互等领域带来一系列革命性的变化,吸引了众多研究者的关注。对深度相机最新发展情况进行了介绍,并报告了深度相机在计算机视觉、计算机图形学中的应用现状。     

State of the Art on Monocular 3D Face Reconstruction,Tracking, and Applications

The computer graphics and vision communities have dedicated long standing efforts in building computerized tools for reconstructing, tracking, and analyzing human faces based on visual input. Over the past years rapid progress has been made, which led to novel and powerful algorithms that obtain impressive results even in the very challenging case of reconstruction from a single RGB or RGB‐D camera. The range of applications is vast and steadily growing as these technologies are further improving in speed, accuracy, and ease of use. Motivated by this rapid progress, this state‐of‐the‐art report summarizes recent trends in monocular facial performance capture and discusses its applications, which range from performance‐based animation to real‐time facial reenactment. We focus our discussion on methods where the central task is to recover and track a three dimensional model of the human face using optimization‐based reconstruction algorithms. We provide an in‐depth overview of the underlying concepts of real‐world image formation, and we discuss common assumptions and simplifications that make these algorithms practical. In addition, we extensively cover the priors that are used to better constrain the under‐constrained monocular reconstruction problem, and discuss the optimization techniques that are employed to recover dense, photo‐geometric 3D face models from monocular 2D data. Finally, we discuss a variety of use cases for the reviewed algorithms in the context of motion capture, facial animation, as well as image and video editing.     

几何引导的主动式三维感知与交互

随着三维感知设备的发展和大规模三维数据的出现,基于三维重建与理解的视觉感知技术得到了大量关注。与此同时,智能图形逐渐改变了传统图形系统在交互中的被动角色,朝着任务引导的、感知驱动的智能体对真实或虚拟环境的主动交互发展。可以说,计算机图形学正在突破“信息表达”这一传统范畴,逐步拓展迈入“信息感知”领域;图形学的交互技术也由传统的人机交互,逐渐延伸和发展出面向智能任务的主动三维交互。其中,数据驱动三维几何分析与建模的理论和方法,特别是在线重建与分析技术,对三维感知和三维交互形成了重要支撑。本文从图形学和视觉融合的视角,结合研究案例,介绍了主动式三维感知与交互,讨论了“主动式”的特点、优势和挑战,并试图探讨这一方向的开放问题与发展趋势。     

A survey of Optimal Transport for Computer Graphics and Computer Vision

Optimal transport is a long-standing theory that has been studied in depth from both theoretical and numerical point of views. Starting from the 50s this theory has also found a lot of applications in operational research. Over the last 30 years it has spread to computer vision and computer graphics and is now becoming hard to ignore. Still, its mathematical complexity can make it difficult to comprehend, and as such, computer vision and computer graphics researchers may find it hard to follow recent developments in their field related to optimal transport. This survey first briefly introduces the theory of optimal transport in layman's terms as well as most common numerical techniques to solve it. More importantly, it presents applications of these numerical techniques to solve various computer graphics and vision related problems. This involves applications ranging from image processing, geometry processing, rendering, fluid simulation, to computational optics, and many more. It is aimed at computer graphics researchers desiring to follow optimal transport research in their field as well as optimal transport researchers willing to find applications for their numerical algorithms.     

基于图像建模技术研究综述与展望

基于图像建模技术是计算机图形学和计算机视觉领域共同关心的重要问题．文中侧重从计算机图形学的研究角度对基于图像建模技术进行了综述,介绍了近年来提出的典型的基于图像建模方法及其最新研究进展,给出了这些方法的基本原理并加以分析比较,最后对基于图像建模技术的未来研究给出了一些建议。     

A Survey of Specularity Removal Methods

The separation of reflection components is an important issue in computer graphics, computer vision and image processing. It provides useful information for the applications that need consistent object surface appearance, such as stereo reconstruction, visual recognition, tracking, objects re‐illumination and dichromatic editing. In this paper we will present a brief survey of recent advances in separation of reflection components, also known as specularity (highlights) removal. Several techniques that try to tackle the problem from different points of view have been proposed so far. In this survey, we will overview these methods and we will present a critical analysis of their benefits and drawbacks.     

医学3D计算机视觉：研究进展和挑战

医学影像的诊断是许多临床决策的基础,而医学影像的智能分析是医疗人工智能的重要组成部分。与此同时,随着越来越多3D空间传感器的兴起和普及,3D计算机视觉正变得越发重要。本文关注医学影像分析和3D计算机的交叉领域,即医学3D计算机视觉或医学3D视觉。本文将医学3D计算机视觉系统划分为任务、数据和表征3个层面,并结合最新文献呈现这3个层面的研究进展。在任务层面,介绍医学3D计算机视觉中的分类、分割、检测、配准和成像重建,以及这些任务在临床诊断和医学影像分析中的作用和特点。在数据层面,简要介绍了医学3D数据中最重要的数据模态：包括计算机断层成像（computed tomography,CT）、磁共振成像（magnetic resonance imaging,MRI）、正电子放射断层成像（positron emission tomography,PET）等,以及一些新兴研究提出的其他数据格式。在此基础上,整理了医学3D计算机视觉中重要的研究数据集,并标注其数据模态和主要视觉任务。在表征层面,介绍并讨论了2D网络、3D网络和混合网络在医学3D数据的表征学习上的优缺点。此外,针对医学影像中普遍存在的小数据问题,重点讨论了医学3D数据表征学习中的预训练问题。最后,总结了目前医学3D计算机视觉的研究现状,并指出目前尚待解决的研究挑战、问题和方向。     

3D目标检测进展综述

目标检测算法应用广泛,一直是计算机视觉领域备受关注的研究热点。近年来,随着深度学习的发展,3D图像的目标检测研究取得了巨大的突破。与2D目标检测相比,3D目标检测结合了深度信息,能够提供目标的位置、方向和大小等空间场景信息,在自动驾驶和机器人领域发展迅速。文中首先对基于深度学习的2D目标检测算法进行概述;其次根据图像、激光雷达、多传感器等不同数据采集方式,分析目前具有代表性和开创性的3D目标检测算法;结合自动驾驶的应用场景,对比分析不同3D目标检测算法的性能、优势和局限性;最后总结了3D目标检测的应用意义以及待解决的问题,并对3D目标检测的发展方向和新的挑战进行了讨论和展望。     

深度学习在目标视觉检测中的应用进展与展望

目标视觉检测是计算机视觉领域的一个重要问题,在视频监控、自主驾驶、人机交互等方面具有重要的研究意义和应用价值.近年来,深度学习在图像分类研究中取得了突破性进展,也带动着目标视觉检测取得突飞猛进的发展.本文综述了深度学习在目标视觉检测中的应用进展与展望.首先对目标视觉检测的基本流程进行总结,并介绍了目标视觉检测研究常用的公共数据集;然后重点介绍了目前发展迅猛的深度学习方法在目标视觉检测中的最新应用进展;最后讨论了深度学习方法应用于目标视觉检测时存在的困难和挑战,并对今后的发展趋势进行展望.     

Neurosymbolic Models for Computer Graphics

Procedural models (i.e. symbolic programs that output visual data) are a historically-popular method for representing graphics content: vegetation, buildings, textures, etc. They offer many advantages: interpretable design parameters, stochastic variations, high-quality outputs, compact representation, and more. But they also have some limitations, such as the difficulty of authoring a procedural model from scratch. More recently, AI-based methods, and especially neural networks, have become popular for creating graphic content. These techniques allow users to directly specify desired properties of the artifact they want to create (via examples, constraints, or objectives), while a search, optimization, or learning algorithm takes care of the details. However, this ease of use comes at a cost, as it's often hard to interpret or manipulate these representations. In this state-of-the-art report, we summarize research on neurosymbolic models in computer graphics: methods that combine the strengths of both AI and symbolic programs to represent, generate, and manipulate visual data. We survey recent work applying these techniques to represent 2D shapes, 3D shapes, and materials & textures. Along the way, we situate each prior work in a unified design space for neurosymbolic models, which helps reveal underexplored areas and opportunities for future research.     

Deep learning-based 3D point cloud classification: A systematic survey and outlook

In recent years, point cloud representation has become one of the research hotspots in the field of computer vision, and has been widely used in many fields, such as autonomous driving, virtual reality, robotics, etc. Although deep learning techniques have achieved great success in processing regular structured 2D grid image data, there are still great challenges in processing irregular, unstructured point cloud data. Point cloud classification is the basis of point cloud analysis, and many deep learning-based methods have been widely used in this task. Therefore, the purpose of this paper is to provide researchers in this field with the latest research progress and future trends. First, we introduce point cloud acquisition, characteristics, and challenges. Second, we review 3D data representations, storage formats, and commonly used datasets for point cloud classification. We then summarize deep learning-based methods for point cloud classification and complement recent research work. Next, we compare and analyze the performance of the main methods. Finally, we discuss some challenges and future directions for point cloud classification.     

机器视觉技术研究进展及展望

摘 要：机器视觉是建立在计算机视觉理论工程化基础上的一门学科,涉及到光学成像、 视觉信息处理、人工智能以及机电一体化等相关技术。随着我国制造业的转型升级与相关研究 的不断深入,机器视觉技术凭借其精度高、实时性强、自动化与智能化程度高等优点,成为了 提升机器人智能化的重要驱动力之一,并被广泛应用于工业生产、农业以及军事等各个领域。 在广泛查阅相关文献之后,针对近十多年来机器视觉相关技术的发展与应用进行分析与总结, 旨在为研究学者与工程应用人员提供参考。首先,总结了机器视觉技术的发展历程、国内外的 机器视觉发展现状;其次,重点分析了机器视觉系统的核心组成部件、常用视觉处理算法以及 当前主流的机器视觉工业软件;然后,介绍了机器视觉技术在产品瑕疵检测、智能视频监控分 析、自动驾驶与辅助驾驶与医疗影像诊断等 4 个典型领域的应用;最后分析了当前机器视觉技 术所面临的挑战,并对其未来的发展趋势进行了展望,为机器视觉技术的发展和应用推广发挥 积极作用。