Colorization in YCbCr color space and its application to JPEG images

This paper presents a colorization method in YCbCr color space, which is based on the maximum a posteriori estimation of a color image given a monochrome image as is our previous method in RGB color space. The presented method in YCbCr space is much simpler than that in RGB space and requires much less computation time, while both methods in YCbCr and RGB space produce color images with comparable PSNR values. The proposed colorization in YCbCr is applied to JPEG compressed color images aiming at better recovery of downsampled chrominance planes. Experimental results show that colorization in YCbCr is usually effective for quality improvement of JPEG color images.     

Efficient image colorization based on seed pixel selection

Colorization is a technique to automatically produce color components for monochrome images and videos based on a few input colors. Generally, image colorization is initialized from a number of seed pixels whose colors are specified by users, and then the colors are gradually prorogating to the monochrome surroundings under a given optimization constraint. So, the performance of colorization is highly dependent on the selection of seed pixels. However, little attention has been paid to the selection of seed pixels, and how to improve the effectiveness of manual input remains a challenging task. To address this, an improved colorization method using seed pixel selection is proposed to assist the users in determining which pixels are highly required to be colorized for a high-quality colorized image. Specifically, the gray-scale image is first divided into non-overlapped blocks, and then, for each block, two pixels that approximate the average luminance of block are selected as the seeds. After the seed pixels are colored by users, an optimization that minimizes the difference between the seeds and their adjacent pixels is employed to propagate the colors to the other pixels. The experimental results demonstrate that, for a given amount of inputs, the proposed method can achieve a higher PSNR than the conventional colorization methods.     

基于自然彩色图像的彩色化权值函数建模

针对现有的权值函数采用人为指定的核函数,难以准确地反映图像复杂处像素间的色度相似关系,提出了利用自然彩色图像特性构建权值函数的方法。根据自然图像的像素在RGB空间的分布特征,提取出色度与灰度的局部线性关系,并利用其作为先验知识结合最小二乘法推广至整幅图像,进而获得一种新的权值函数。该权值函数能将相近的两像素间的灰度差异、位置差异及周围像素的灰度分布统一到色度相似程度的计算中。实验结果表明,该权值函数有助于获得更优的彩色化结果,特别是在复杂边界处效果提升明显。     

局部自适应的灰度图像彩色化

目的 现有的灰度图像彩色化方法为了保证彩色化结果在颜色空间上的一致性,往往采用全局优化的算法,使得图像边界区域易产生过渡平滑现象。为此提出一种局部自适应的灰度图像彩色化方法,在迁移过程中考虑局部邻域像素信息,同时自动调节邻域像素权重,在颜色正确迁移的同时保证清晰的边界信息。方法 首先结合SVM（support vector machine）和ISLIC（improved simple linear iterative clustering）算法获取彩色图像和灰度图像分类结果图;然后在分类基础上,确定灰度图像高置信度像素点,并根据图像纹理特征,在彩色图像中寻找灰度图像的像素匹配点;最后利用自适应权重均值滤波实现高置信度匹配像素点的颜色迁移,并利用迁移结果对低置信度像素点进行颜色扩散,以完成灰度图像彩色化。结果 实验结果显示,本文方法获得的彩色化迁移结果评分均高于3.5分,特别是局部放大区域评价结果均接近或高于4.0分,高于其他现有彩色化方法评价分数。表明本文方法不仅能够保证颜色迁移的准确性和颜色空间的一致性,同时也能获取颜色区分度高的边界细节信息。与现有的典型灰度图像彩色化方法相比,彩色化结果图在颜色迁移的正确性和抑制边界区域颜色的过渡平滑上都有更优的表现。结论 本文算法为灰度图像彩色化过程中抑制颜色越界问题提供了新的指导方法,能有效地应用于遥感、黑白图像/视频处理、医学图像着色等领域。     

勾画式局部颜色迁移

图像之间的颜色传输有效地利用了图像的基本统计信息.提出一种基于图像的简单统计信息和梯度域信息的局部颜色迁移算法,其中提供了简单易用的操作界面,在迁移过程中能够保持源图像的颜色细节;将其推广到灰度图上色问题,可有效地提高上色速度.实验结果表明,该箅法较好地实现了源图像和目标图像的局部区域颜色对应,生成具有较好视觉效果的真实自然的新图像.     

基于MRF的复杂图像抠图

所谓复杂图像抠图就是从复杂图像中抠取出目标物体的一种图像处理算法。为了取得更好的抠图效果,提出了一种基于马尔可夫随机场的自然图像抠图方法。该方法首先手工把图像分成3个区域:前景区域、背景区域和未知区域;然后,再将未知区域用手工粗略地划分成几个相交的小区域;接着在每一个小区域内,以其中的未知区域的像素点为节点,定义抠图标号,同时在这些节点上面建立MRF抠图模型,并把这些标号赋给这些节点,这样抠图问题被定义为在这个MRF模型和它的Gibbs分布上MAP估计问题;继而再计算出每个小区域的掩像;最后把这些掩像合并,即得到输入图像最终的掩像。和其他算法相比,对复杂图像的抠图问题,该方法可以取得更好的抠图效果。     

基于深层聚合结构网络的灰度图像彩色化方法

当前灰度图像彩色化方法普遍存在边界晕染、细节丢失和着色效果枯燥等问题。针对以上问题,提出了一种基于改进的深层聚合结构网络的灰度图像彩色化方法。将深层聚合结构网络引入图像彩色化领域中,且在传统网络基础上加入长连接,在缓解网络梯度消失问题的同时提升其特征利用率,从而提升算法模型对图像边界和细节的处理能力。另外,模型融合生成对抗网络结构,搭建判别网络,动态评价图片彩色化质量,缓解着色枯燥的问题。实验证明,该方法相比于传统彩色化方法,减轻了着色时边界漏色问题,还原了更多的图像细节,图像颜色更为丰富。     

基于KNN图层区分的优化式着色算法

针对灰度图像彩色化技术应用于彩色图像二次着色时往往忽略掉原始图像所带的色彩信息的问题,提出了一种基于[KNN]图层区分的优化式着色算法。与现有的优化式着色方法相比,该方法一方面采用基于[KNN]的图像前背景区分算法获得图层区分的图像,生成新的权值函数;另一方面将图层区分结果引入优化式着色方法,并对图像着色。实验结果表明,算法能有效解决物体边界处发生颜色渗漏的问题,得到颜色分布精确的图像。在相同输入前提下,算法可以得到更好的着色结果。     

A Markov Random Field Model-Based Approach to Natural Image Matting

This paper proposes a Markov Random Field (MRF) model-based approach to natural image matting with complex scenes. After the trimap for matting is given manually, the unknown region is roughly segmented into several joint sub-regions. In each sub-region, we partition the colors of neighboring background or foreground pixels into several clusters in RGB color space and assign matting label to each unknown pixel. All the labels are modelled as an MRF and the matting problem is then formulated as a maximum a posteriori (MAP) estimation problem. Simulated annealing is used to find the optimal MAP estimation. The better results can be obtained under the same user-interactions when images are complex. Results of natural image matting experiments performed on complex images using this approach are shown and compared in this paper.     

基于最优聚类数和直方图匹配的图像彩色化方法

针对颜色转移彩色化算法中速度慢、效果不佳及人工干预性强等问题,提出了一种新型的彩色化算法。该算法首先解决了图像聚类中聚类数的选取问题,然后利用聚类算法分别对目标图像和源图像进行聚类分割,之后用直方图匹配算法使各目标图像块自动找到匹配的源图像块,将源图像块的颜色转移到目标图像块中,实现局部图像彩色化,最后合并各结果图像。实验结果表明,该算法比以前算法在彩色化的速度和质量上有较大改进,且自动化程度高。     

局部线性嵌入与模糊 C-均值聚类的红外图像彩色化算法

针对红外图像彩色化处理后清晰度低、色彩不够自然等问题,将改进后的局部线 性嵌入算法(LLE)算法引入到红外图像彩色化应用中,提出了一种 LLE 与模糊 C-均值聚类的红 外图像彩色化算法。首先通过扩大邻域范围和添加权重信息等方法改善了 LLE 算法敏感于稀疏 矩阵的缺陷,在红外和彩色模板像素矢量化空间中,利用最近邻搜索计算最佳匹配系数,经彩 色值计算将彩色模板中的颜色映射到红外图像特定区域,完成模板彩色与红外目标图像的颜色 传递。利用改进后的模糊 C-均值聚类对彩色化后的红外图像进行颜色聚类,在颜色聚类集上利 用直方图均衡化进行分段颜色均衡处理,最后将均衡化后的图像合成。将本算法与其他两种红 外彩色化算法进行仿真对比,实验结果表明,与其他两种算法相比,提出的红外图像彩色化算 法在仅仅利用目标红外图像和彩色模板下就能获得图像较为清晰、目标突出的彩色化结果。     

一种利用回归神经网络黑白图像着色算法研究

研究了利用神经网络对序列黑白(灰度)图像进行着色的问题。针对以往基于人工或者半自动化技术的黑白图像着色技术效率低下、视觉效果较差的缺陷,提出了一种利用三层神经网络、无须人工干预的图像自动着色算法。首先将灰度图像分割成小块,通过对小块提取灰度特征、空间特征等作为神经网络的输入,训练得到一个回归神经网络。在着色过程中,可以利用该神经网络将图像中各像素由灰度空间投影到一个经过压缩的色彩空间,从而实现了图像的自动着色过程。实验结果显示本方法能有效地将灰度图像着色,并且由于使用了一个压缩的色彩空间,使得计算效率和着色效果都得到了有效的提高,能很好地逼近原始的真实图像。     

基于图切分的交互式图像染色算法

将黑白图像颜色化是一个需要大量用户交互和时间的事情,传统的基于交互的做法主要有两种：先分割后着色和全局优化．前者速度快,但是往往因为分块太大而丧失细节;后者能保持颜色变化的连续性,但是求解速度慢．对此文中提出了一种图像颜色化的新方法：基于图切分技术的分割算法．图切分是一种基于全局能量优化的分割技术,因而可以保证大部分区域的颜色分布光滑,而只在灰度变化剧烈的边缘地区产生颜色跳变,并且算法具有很快的求解速度．在用户简单给定颜色种子基础上,基于相同灰度倾向于相同彩色的基本假设,首先计算图像每个像素周围的灰度分布并构造一个全局的能量函数．接着利用图切分（graph cut）的技术快速有效地求得最佳分割．随着用户的进一步交互,图切分可以很快地迭代计算．这样,通过简单的交互,用户可以很快地对一幅黑白图像彩色化,并获得自然的效果．     

Automatic Colorization with Improved Spatial Coherence and Boundary Localization

Grayscale image colorization is an important computer graphics problem with a variety of applications. Recent fully automatic colorization methods have made impressive progress by formulating image colorization as a pixel-wise prediction task and utilizing deep convolutional neural networks. Though tremendous improvements have been made, the result of automatic colorization is still far from perfect. Specifically, there still exist common pitfalls in maintaining color consistency in homogeneous regions as well as precisely distinguishing colors near region boundaries. To tackle these problems, we propose a novel fully automatic colorization pipeline which involves a boundary-guided CRF (conditional random field) and a CNN-based color transform as post-processing steps. In addition, as there usually exist multiple plausible colorization proposals for a single image, automatic evaluation for different colorization methods remains a challenging task. We further introduce two novel automatic evaluation schemes to efficiently assess colorization quality in terms of spatial coherence and localization. Comprehensive experiments demonstrate great quality improvement in results of our proposed colorization method under multiple evaluation metrics.     

Colorization of Line Drawings with Empty Pupils

Many studies have recently applied deep learning to the automatic colorization of line drawings. However, it is difficult to paint empty pupils using existing methods because the convolutional neural network are trained with pupils that have edges, which are generated from color images using image processing. Most actual line drawings have empty pupils that artists must paint in. In this paper, we propose a novel network model that transfers the pupil details in a reference color image to input line drawings with empty pupils. We also propose a method for accurately and automatically colorizing eyes. In this method, eye patches are extracted from a reference color image and automatically added to an input line drawing as color hints using our pupil position estimation network.     

Affective Image Colorization

Colorization of gray-scale images has attracted many attentions for a long time.An important role of image color is the conveyer of emotions(through color themes).The colorization with an undesired color theme is less useful,even it is semantically correct.However this has been rarely considered.Automatic colorization respecting both the semantics and the emotions is undoubtedly a challenge.In this paper,we propose a complete system for affective image colorization.We only need the user to assist object segmentation along with text labels and an affective word.First,the text labels along with other object characters are jointly used to filter the internet images to give each object a set of semantically correct reference images.Second,we select a set of color themes according to the affective word based on art theories.With these themes,a generic algorithm is used to select the best reference for each object,balancing various requirements.Finally,we propose a hybrid texture synthesis approach for colorization.To the best of our knowledge,it is the first system which is able to efficiently colorize a gray-scale image semantically by an emotionally controllable fashion.Our experiments show the effectiveness of our system,especially the benefit compared with the previous Markov random field(MRF) based method.     

基于前景语义信息的图像着色算法

图像可分为前景部分与背景部分,而前景往往是视觉中心。在图像着色任务上,由于前景的类别多且情况复杂,着色困难,以至于图像中的前景部分会存在着色暗淡和细节丢失等问题。针对这些问题,提出了基于前景语义信息的图像着色算法,以改善图像着色效果,达到图像整体颜色自然、内容颜色丰富的目的。首先利用前景子网提取前景部分的低级特征和高级特征;然后将这些特征融合到全景子网训练中,以排除背景颜色信息影响并强调前景颜色信息;最后用生成损失和像素级别的颜色损失来不断优化网络,指导生成高质量图像。实验结果表明,引入前景语义信息后,所提算法在峰值信噪比（PSNR）和感知相似度（LPIPS）上有所提升,可有效改善视觉中心区域着色中的色泽暗淡、细节丢失、对比度低等问题;相比其他算法,该算法在图像整体上取得了更自然的着色效果,在内容部分上取得了显著的改进。     

有限色彩空间下的线稿上色

目的 线稿上色是由线条构成的黑白线稿草图涂上颜色变为彩色图像的过程,在卡通动画制作和艺术绘画等领域中是非常关键的步骤。全自动线稿上色方法可以减轻绘制过程中烦琐耗时的手工上色的工作量,然而自动理解线稿中的稀疏线条并选取合适的颜色仍较为困难。方法 依据现实场景中特定绘画类型常有固定用色风格偏好这一先验,本文聚焦于有限色彩空间下的线稿自动上色,通过约束色彩空间,不仅可以降低语义理解的难度,还可以避免不合理的用色。具体地,本文提出一种两阶段线稿自动上色方法。在第1阶段,设计一个灰度图生成器,对输入的稀疏线稿补充线条和细节,以生成稠密像素的灰度图像。在第2阶段,首先设计色彩推理模块,从输入的颜色先验中推理得到适合该线稿的色彩子空间,再提出一种多尺度的渐进融合颜色信息的生成网络以逐步生成高质量的彩色图像。结果 实验在3个数据集上与4种线稿自动上色方法进行对比,在上色结果的客观质量对比中,所提方法取得了更高的峰值信噪比（peak signal to noise ratio,PSNR）和结构相似性（structural similarity index measure,SSIM）值以及更低的均方误差;在上色结果的色彩指标对比中,所提方法取得了最高的色彩丰富度分数;在主观评价和用户调查中,所提方法也取得了与人的主观审美感受更一致的结果。此外,消融实验结果也表明了本文所使用的模型结构及色彩空间限制有益于上色性能的提升。结论 实验结果表明,本文提出的有限色彩空间下的线稿自动上色方法可以有效地完成多类线稿的自动上色,并且可以简单地通过调整颜色先验以获得更多样的彩色图像。     

Application of the Extended Distance Transformation in digital image colorization

In this paper we present a novel colorization scheme that takes advantage of the modified morphological distance transform to propagate the color, scribbled by a user on the grayscale image. First, based on the scribbled image, the topological distance values are computed for each image pixel, describing its distance to the inserted color markers. These values are then complemented with the structural information and luminance changes derived from the original grayscale image. The distances are then used along with gradient based features to reproduce original image structures while propagating the new colors obtained during the additive color blending process. Extensive experiments performed on various kinds of natural images demonstrated the effectiveness of the proposed colorization method. They also showed that the main advantage of the presented algorithm is its computational speed and ability to produce visually pleasing colorization results promptly after providing the color information.     

基于遗传算法的图像彩色化方法

为解决传统的彩色化算法中效果不佳、速度慢等问题,提出一种基于遗传算法的图像彩色化方法。该方法先产生初始种群,将源图像块和目标图像块的亮度特征以及Tamura纹理特征构成适应度函数,初始种群经过选择、交叉和变异等操作逐代进化后,将源图像块的颜色转移到匹配的目标图像块中,得到最终的彩色化图像。实验结果表明,与其他彩色化方法相比,该方法能在缩短运行时间的同时,有效提高彩色化质量。