Fast Rendering of Image Mosaics and ASCII Art

An image mosaic is an assembly of a large number of small images, usually called tiles, taken from a specific dictionary/codebook. When viewed as a whole, the appearance of a single large image emerges, i.e. each tile approximates a small block of pixels. ASCII art is a related (and older) graphic design technique for producing images from printable characters. Although automatic procedures for both of these visualization schemes have been studied in the past, some are computationally heavy and cannot offer real‐time and interactive performance. We propose an algorithm able to reproduce the quality of existing non‐photorealistic rendering techniques, in particular ASCII art and image mosaics, obtaining large performance speed‐ups. The basic idea is to partition the input image into a rectangular grid and use a decision tree to assign a tile from a pre‐determined codebook to each cell. Our implementation can process video streams from webcams in real time and it is suitable for modestly equipped devices. We evaluate our technique by generating the renderings of a variety of images and videos, with good results. The source code of our engine is publicly available.     

Computationally efficient image deblurring using low rank image approximation and its GPU implementation

This paper presents a computationally efficient technique for reduction of blur caused by handshakes in images captured by mobile devices. This technique uses a short-exposure or a low-exposure image that is captured at the same time a normal or auto-exposure image is captured. The short-exposure image is enhanced by utilizing low rank image approximation of the auto-exposure image without requiring any user specified parameters. Based on the three quantitative measures of image quality, it is shown that this technique outperforms similar techniques used for image deblurring while it also offers computational efficiency. A GPU implementation of this technique is also reported.     

基于GPU的快速三维医学图像刚性配准技术*

自动三维配准将多个图像数据映射到同一坐标系中,在医学影像分析中有广泛的应用。但现有主流三维刚性配准算法(如FLIRT)速度较慢,2563大小数据的刚性配准需要300 s左右,不能满足快速临床应用的需求。为此提出了一种基于CUDA(compute unified device architecture)架构的快速三维配准技术,利用GPU(gra-phic processing unit)并行计算实现配准中的坐标变换、线性插值和相似性测度计算。临床三维医学图像上的实验表明,该技术在保持配准精度的前提下将速度提     

High-quality cardiac image dynamic visualization with feature enhancement and virtual surgical tool inclusion

Traditional approaches for rendering segmented volumetric data sets usually deliver unsatisfactory results, such as insufficient frame rate, low image quality, and intermixing artifacts. In this paper, we introduce a novel “color encoding” technique, based on graphics processing unit (GPU) accelerated raycasting and post-color attenuated classification, to address this problem. The result is an algorithm that can generate artifact-free dynamic volumetric images in real time. Next, we present a pre-integrated volume shading algorithm to reduce graphics memory requirements and computational cost when compared to traditional shading methods. We also present a normal-adjustment technique to improve image quality at clipped planes. Furthermore, we propose a new algorithm for color and depth texture indexing that permits virtual solid objects, such as surgical tools, to be manipulated within the dynamically rendered volumetric cardiac images in real time. Finally, all these techniques are combined within an environment that permits real-time visualization, enhancement, and manipulation of dynamic cardiac data sets.     

Sailfish: A flexible multi-GPU implementation of the lattice Boltzmann method

We present Sailfish, an open source fluid simulation package implementing the lattice Boltzmann method (LBM) on modern Graphics Processing Units (GPUs) using CUDA/OpenCL. We take a novel approach to GPU code implementation and use run-time code generation techniques and a high level programming language (Python) to achieve state of the art performance, while allowing easy experimentation with different LBM models and tuning for various types of hardware. We discuss the general design principles of the code, scaling to multiple GPUs in a distributed environment, as well as the GPU implementation and optimization of many different LBM models, both single component (BGK, MRT, ELBM) and multicomponent (Shan–Chen, free energy). The paper also presents results of performance benchmarks spanning the last three NVIDIA GPU generations (Tesla, Fermi, Kepler), which we hope will be useful for researchers working with this type of hardware and similar codes.     

Semantic Photo Synthesis

Composite images are synthesized from existing photographs by artists who make concept art, e.g., storyboards for movies or architectural planning. Current techniques allow an artist to fabricate such an image by digitally splicing parts of stock photographs. While these images serve mainly to “quickly”convey how a scene should look, their production is laborious. We propose a technique that allows a person to design a new photograph with substantially less effort. This paper presents a method that generates a composite image when a user types in nouns, such as “boat”and “sand.”The artist can optionally design an intended image by specifying other constraints. Our algorithm formulates the constraints as queries to search an automatically annotated image database. The desired photograph, not a collage, is then synthesized using graph‐cut optimization, optionally allowing for further user interaction to edit or choose among alternative generated photos. An implementation of our approach, shown in the associated video, demonstrates our contributions of (1) a method for creating specific images with minimal human effort, and (2) a combined algorithm for automatically building an image library with semantic annotations from any photo collection.     

Enhanced GPU-Based Anti-Noise Hybrid Edge Detection Method

Today, there is a growing demand for computer vision and image processing in different areas and applications such as military surveillance, and biological and medical imaging. Edge detection is a vital image processing technique used as a pre-processing step in many computer vision algorithms. However, the presence of noise makes the edge detection task more challenging; therefore, an image restoration technique is needed to tackle this obstacle by presenting an adaptive solution. As the complexity of processing is rising due to recent high-definition technologies, the expanse of data attained by the image is increasing dramatically. Thus, increased processing power is needed to speed up the completion of certain tasks. In this paper,we present a parallel implementation of hybrid algorithm-comprised edge detection and image restoration along with other processes using Computed Unified Device Architecture (CUDA) platform, exploiting a Single Instruction Multiple Thread (SIMT) execution model on a Graphical Processing Unit (GPU). The performance of the proposed method is tested and evaluated using well-known images from various applications. We evaluated the computation time in both parallel implementation on the GPU, and sequential execution in the Central Processing Unit (CPU) natively and using Hyper-Threading (HT) implementations. The gained speedup for the naïve approach of the proposed edge detection using GPU under global memory direct access is up to 37 times faster, while the speedup of the native CPU implementation when using shared memory approach is up to 25 times and 1.5 times over HT implementation.     

High accuracy optical character recognition using neural networkswith centroid dithering

Optical character recognition (OCR) refers to a process whereby printed documents are transformed into ASCII files for the purpose of compact storage, editing, fast retrieval, and other file manipulations through the use of a computer. The recognition stage of an OCR process is made difficult by added noise, image distortion, and the various character typefaces, sizes, and fonts that a document may have. In this study a neural network approach is introduced to perform high accuracy recognition on multi-size and multi-font characters; a novel centroid-dithering training process with a low noise-sensitivity normalization procedure is used to achieve high accuracy results. The study consists of two parts. The first part focuses on single size and single font characters, and a two-layered neural network is trained to recognize the full set of 94 ASCII character images in 12-pt Courier font. The second part trades accuracy for additional font and size capability, and a larger two-layered neural network is trained to recognize the full set of 94 ASCII character images for all point sizes from 8 to 32 and for 12 commonly used fonts. The performance of these two networks is evaluated based on a database of more than one million character images from the testing data set     

GPU-accelerated level-set segmentation

The level-set method, a technique for the computation of evolving interfaces, is a solution commonly used to segment images and volumes in medical applications. GPUs have become a commodity hardware with hundreds of cores that can execute thousands of threads in parallel, and they are nowadays ideal platforms to execute computational intensive tasks, such as the 3D level-set-based segmentation, in real time. In this paper, we propose two GPU implementations of the level-set-based segmentation method called Fast Two-Cycle. Our proposals perform computations in independent domains called tiles and modify the structure of the original algorithm to better exploit the features of the GPU. The implementations were tested with real images of brain vessels and a synthetic MRI image of the brain. Results show that they execute faster than a CPU-sequential implementation of the same method, without any significant loss of the segmentation quality and without requiring distributed parallel computer infrastructures.     

Near real-time stereo matching using geodesic diffusion

Adaptive-weight algorithms currently represent the state of the art in local stereo matching. However, due to their computational requirements, these types of solutions are not suitable for real-time implementation. Here, we present a novel aggregation method inspired by the anisotropic diffusion technique used in image filtering. The proposed aggregation algorithm produces results similar to adaptive-weight solutions while reducing the computational requirements. Moreover, near real-time performance is demonstrated with a GPU implementation of the algorithm.     

GLSC: LSC superpixels at over 130 FPS

Superpixel has been successfully applied in various computer vision tasks, and many algorithms have been proposed to generate superpixel map. Recently, a superpixel algorithm called “superpixel segmentation using linear spectral clustering” (LSC) has been proposed, and it performs equally well or better than state-of-the art superpixel segmentation algorithms in terms of several commonly used evaluation metrics in superpixel segmentation. Although LSC is of linear complexity, its original implementation runs in few hundreds of milliseconds for images with resolution of 481 × 321 stated by the authors, which is a limitation for some real-time applications such as visual tracking which may needs, for instance, 30 FPS for standard image resolution (e.g., 480 × 320, 640 × 480, 1280 × 720 and 1920 × 1080). Instead of inventing new algorithms with lower complexity than LSC, we will explore LSC to modify its structure and make it suitable to be implemented by parallel technique. The modified LSC algorithm is implemented in CUDA and tested on several NVIDIA graphics processing unit. Our implementation of the proposed modified LSC algorithm achieves speedups of up to 80× from the original sequential implementation, and the quality, measured by two commonly used evaluation metrics, of our implementation keeps being similar to the original one. The source code will be made publicly available.     

A translucent display algorithm for G-octree represented grey-scale images

A new translucent display algorithm is proposed for grey-scale images represented by G-octrees. The algorithm generates a special triangle quadtree as the display image for the isometric projection of a G-octree, while traversing the G-octree from front to back, with respect to the viewpoint. A method of using the translucent effect to analyse the interiors of three-dimensional objects is also described, together with its implementation results.     

Core-line tracing for fuzzy image subsets

A method allowing core-line tracing for grey-scale images and, in particular, for fuzzy image subsets is proposed. The method is based on the Maximal Square Moving technique, which allows a data-structure-type representation of the medial axis of an image. An application of the method to an astronomical imaging problem is shown.     

More IMPATIENT: A gridding-accelerated Toeplitz-based strategy for non-Cartesian high-resolution 3D MRI on GPUs

Several recent methods have been proposed to obtain significant speed-ups in MRI image reconstruction by leveraging the computational power of GPUs. Previously, we implemented a GPU-based image reconstruction technique called the Illinois Massively Parallel Acquisition Toolkit for Image reconstruction with ENhanced Throughput in MRI (IMPATIENT MRI) for reconstructing data collected along arbitrary 3D trajectories. In this paper, we improve IMPATIENT by removing computational bottlenecks by using a gridding approach to accelerate the computation of various data structures needed by the previous routine. Further, we enhance the routine with capabilities for off-resonance correction and multi-sensor parallel imaging reconstruction. Through implementation of optimized gridding into our iterative reconstruction scheme, speed-ups of more than a factor of 200 are provided in the improved GPU implementation compared to the previous accelerated GPU code.     

Stylized Image Triangulation

The art of representing images with triangles is known as image triangulation, which purposefully uses abstraction and simplification to guide the viewer's attention. The manual creation of image triangulations is tedious and thus several tools have been developed in the past that assist in the placement of vertices by means of image feature detection and subsequent Delaunay triangulation. In this paper, we formulate the image triangulation process as an optimization problem. We provide an interactive system that optimizes the vertex locations of an image triangulation to reduce the root mean squared approximation error. Along the way, the triangulation is incrementally refined by splitting triangles until certain refinement criteria are met. Thereby, the calculation of the energy gradients is expensive and thus we propose an efficient rasterization‐based GPU implementation. To ensure that artists have control over details, the system offers a number of direct and indirect editing tools that split, collapse and re‐triangulate selected parts of the image. For final display, we provide a set of rendering styles, including constant colours, linear gradients, tonal art maps and textures. Finally, we demonstrate temporal coherence for animations and compare our method with existing image triangulation tools.     

An improved parallel contrast-aware halftoning

Digital image halftoning is a widely used technique. However, achieving high fidelity tone reproduction and structural preservation with low computational time cost remains a challenging problem. This paper presents a highly parallel algorithm to boost real-time application of serial structure-preserving error diffusion. The contrast-aware halftoning approach is one such technique with superior structure preservation, but it offers only a limited opportunity for graphics processing unit （GPU） accel- eration. Our method integrates contrast-aware halftoning into a new parallelizable error-diffusion halftoning framework. To eliminate visually disturbing artifacts resulting from parallelization, we propose a novel multiple quantization model and space-filling curve to maintain tone consistency, blue-noise property, and structure consistency. Our GPU implementation on a commodity personal computer achieves a real-time performance for a moderately sized image. We demonstrate the high quality and performance of the proposed approach with a variety of examples, and provide comparisons with state-of-the-art methods.     

基于CUDA的声辐射力弹性成像算法研究

声辐射力弹性成像是一种新的测量组织硬度的超声成像方法。不同于其他超声组织弹性成像方法,声辐射力弹性成像能够定量测量组织的弹性模量数值,并且具有对操作者经验依赖性低的特点。然而,由于成像算法数据处理量大,运算时间长,声辐射力弹性成像还无法进行准实时的二维成像。为了获得实时的二维声辐射力弹性图像,提出并实现了一种适合于在GPU上并行计算的声辐射力弹性成像算法。通过与运行在CPU上的原始声辐射力弹性成像算法进行对比,证明在GPU上实现的算法大幅度地提高了运算速度。在自制弹性仿体上,比较了基于GPU和CPU两种算法所成的二维弹性分布图像的质量,结果证明两者的图像质量没有明显差异。     

Optical Flow Rendering

This paper proposes a new approach to image-based rendering that generates an image viewed from an arbitrary camera position and orientation by rendering optical flows extracted from reference images. To derive valid optical flows, we develop an analysis technique that improves the quality of stereo matching. Without using any special equipments such as range cameras, this technique constructs reliable optical flows from a sequence of matching results between reference images. We also derive validity conditions of optical flows and show that the obtained flows satisfy those conditions. Since environment geometry is inferred from the optical flows, we are able to generate more accurate images with this additional geometric information. Our approach makes it possible to combine an image rendered from optical flows with an image generated by a conventional rendering technique through a simple Z-buffer algorithm.     

粉笔画艺术风格模拟

目的 对不同艺术风格的模拟和绘制是非真实感绘制技术的主要任务之一,目前非真实感绘制技术已对油画、水彩画、中国书法等国内外艺术风格进行了模拟,然而对粉笔化艺术风格的模拟方法并不多见。本文提出了一种基于滤波扩散和线积分卷积（LIC）的粉笔画艺术风格绘制技术。方法 首先输入2维目标图像,通过对目标图像二值化处理、边缘提取操作,获得连续、光滑的边缘信息,并采用滤波扩散技术对边缘图像进行扩散处理,模拟粉笔画中笔划的毛糙效果,同时通过采用图像增强方法增强了笔划的细节信息;其次,由于真实粉笔画在创作时,粉笔颜料黏附在图像局部区域,形成具有方向的笔刷纹理效果,算法通过在目标图像中添加白噪声,基于线积分卷积LIC产生具有方向的粉笔画笔刷纹理,并通过形态学膨胀处理获得粉笔画的笔划纹理,模拟出粉笔画中笔划的笔触特征。再次,真实的粉笔画艺术效果往往在黑板、木材等材质中创作,算法将产生的笔刷纹理图像、色彩信息以及边缘图像通过图层映射方法,映射到黑板材质等输入背景图像中,产生最终的粉笔画艺术效果图像。结果 通过对输入2维图像进行实验,模拟出具有粉笔画艺术效果的结果图像,突出了粉笔画的线条细节信息和笔划艺术特征。结论 提出了一种粉笔画艺术效果模拟算法,非真实感绘制领域的有效补充,算法简单有效,能模拟出真实的粉笔画艺术效果,增强了艺术表现力。     

Orders-of-magnitude performance increases in GPU-accelerated correlation of images from the International Space Station

We implement image correlation, a fundamental component of many real-time imaging and tracking systems, on a graphics processing unit (GPU) using NVIDIA’s CUDA platform. We use our code to analyze images of liquid-gas phase separation in a model colloid-polymer system, photographed in the absence of gravity aboard the International Space Station (ISS). Our GPU code is 4,000 times faster than simple MATLAB code performing the same calculation on a central processing unit (CPU), 130 times faster than simple C code, and 30 times faster than optimized C++ code using single-instruction, multiple-data (SIMD) extensions. The speed increases from these parallel algorithms enable us to analyze images downlinked from the ISS in a rapid fashion and send feedback to astronauts on orbit while the experiments are still being run.