Synthesis of Human Skin Pigmentation Disorders

Changes in the human pigmentary system can lead to imbalances in the distribution of melanin in the skin resulting in artefacts known as pigmented lesions. Our work takes as departing point biological data regarding human skin, the pigmentary system and the melanocytes life cycle and presents a reaction–diffusion model for the simulation of the shape features of human‐pigmented lesions. The simulation of such disorders has many applications in dermatology, for instance, to assist dermatologists in diagnosis and training related to pigmentation disorders. Our study focuses, however, on applications related to computer graphics. Thus, we also present a method to seamless blend the results of our simulation model in images of healthy human skin. In this context, our model contributes to the generation of more realistic skin textures and therefore more realistic human models. In order to assess the quality of our results, we measured and compared the characteristics of the shape of real and synthesized pigmented lesions. We show that synthesized and real lesions have no statistically significant differences in their shape features. Visually, our results also compare favourably with images of real lesions, being virtually indistinguishable from real images.     

Toward a combined tool to assist dermatologists in melanoma detection from dermoscopic images of pigmented skin lesions

In this paper we propose a machine learning approach to classify melanocytic lesions as malignant or benign, using dermoscopic images. The lesion features used in the classification framework are inspired on border, texture, color and structures used in popular dermoscopy algorithms performed by clinicians by visual inspection. The main weakness of dermoscopy algorithms is the selection of a set of weights and thresholds, that appear not to be robust or independent of population. The use of machine learning techniques allows to overcome this issue. The proposed method is designed and tested on an image database composed of 655 images of melanocytic lesions: 544 benign lesions and 111 malignant melanoma. After an image pre-processing stage that includes hair removal filtering, each image is automatically segmented using well known image segmentation algorithms. Then, each lesion is characterized by a feature vector that contains shape, color and texture information, as well as local and global parameters. The detection of particular dermoscopic patterns associated with melanoma is also addressed, and its inclusion in the classification framework is discussed. The learning and classification stage is performed using AdaBoost with C4.5 decision trees. For the automatically segmented database, classification delivered a specificity of 77% for a sensitivity of 90%. The same classification procedure applied to images manually segmented by an experienced dermatologist yielded a specificity of 85% for a sensitivity of 90%.     

Neural Network Based Terrain Classification Using Wavelet Features

Terrain perception technology using passive sensors plays a key role in enhancing autonomous mobility for military unmanned ground vehicles in off-road environments. In this paper, an effective method for classifying terrain cover based on color and texture features of an image is presented. Discrete wavelet transform coefficients are used to extract those features. Furthermore, spatial coordinates, where a terrain class is located in the image, are also adopted as additional features. Considering real-time applications, we applied a neural network as classifier and it is trained using real off-road terrain images. Through comparison of the classification performance according to applied feature sets and color space changes, we can find that the feature vectors with spatial coordinates extracted using the Daub2 wavelet in the HSI color space have the best classification performance. Experiments show that using the wavelet features and spatial coordinates features improves the terrain cover classification performance. The proposed algorithm has a promising results and potential applications for autonomous navigation.     

Terrain analysis using radar shape-from-shading

This paper develops a maximum a posteriori (MAP) probability estimation framework for shape-from-shading (SFS) from synthetic aperture radar (SAR) images. The aim is to use this method to reconstruct surface topography from a single radar image of relatively complex terrain. Our MAP framework makes explicit how the recovery of local surface orientation depends on the whereabouts of terrain edge features and the available radar reflectance information. To apply the resulting process to real world radar data, we require probabilistic models for the appearance of terrain features and the relationship between the orientation of surface normals and the radar reflectance. We show that the SAR data can be modeled using a Rayleigh-Bessel distribution and use this distribution to develop a maximum likelihood algorithm for detecting and labeling terrain edge features. Moreover, we show how robust statistics can be used to estimate the characteristic parameters of this distribution. We also develop an empirical model for the SAR reflectance function. Using the reflectance model, we perform Lambertian correction so that a conventional SFS algorithm can be applied to the radar data. The initial surface normal direction is constrained to point in the direction of the nearest ridge or ravine feature. Each surface normal must fall within a conical envelope whose axis is in the direction of the radar illuminant. The extent of the envelope depends on the corrected radar reflectance and the variance of the radar signal statistics. We explore various ways of smoothing the field of surface normals using robust statistics. Finally, we show how to reconstruct the terrain surface from the smoothed field of surface normal vectors. The proposed algorithm is applied to various SAR data sets containing relatively complex terrain structure.     

Image Sequence Stabilization in Real Time

This paper describes a method ofstabilizingimage sequences obtained by a camera carried by a ground vehicle. The motion of the vehicle can usually be regarded as consisting of a desired smooth motion combined with an undesired non-smooth motion that includes impulsive or high-frequency components. The goal of the stabilization process is to correct the images so that they are approximately the same as the images that would have been obtained if the motion of the vehicle had been smooth.We analyse the smooth and non-smooth motions of a ground vehicle and show that only the rotational components of the non-smooth motion have significant perturbing effects on the images. We show how to identify image points at which rotational image flow is dominant, and how to use such points to estimate the vehicle's rotation. Finally, we describe an algorithm that fits smooth (ideally, piecewise constant) rotational motions to these estimates; the residual rotational motion can then be used to correct the images. We have obtained good results for several image sequences obtained from a camera carried by a ground vehicle moving across bumpy terrain.     

基于改进SIFT算法的山谷地形影像匹配

针对山谷地形的低空影像中地貌单一且特征不易提取的问题,本文对SIFT算法改进,融合Harris特征提取算法优势,得到一种可用于山谷地形下低空无人机影像特征提取与匹配的算法。算法首先利用Harris算法和SIFT算法分别提取特征点,对两种算法提取的特征点进行合并,然后运用SIFT算法对合并后的特征点进行描述,再利用特征点特征向量的欧氏距离进行粗匹配,最后利用RANSAC算法进行精匹配来提高匹配精度。为了验证该算法的有效性,选用一组山地影像数据进行实验并与SIFT算法进行比较,结果表明:算法有效地提升了山谷地形影像上特征点匹配精度。     

融合注意力机制和高效网络的糖尿病视网膜病变识别与分类

目的 糖尿病视网膜病变（diabetic retinopathy,DR）是一种病发率和致盲率都很高的糖尿病并发症。临床中,由于视网膜图像不同等级之间差异性小以及临床医生经验的不同,会出现误诊、漏诊等情况,目前基于人工DR的诊断分类性能差且耗时费力。基于此,本文提出一种融合注意力机制（attention mechanism）和高效率网络（high-efficiency network,EfficientNet）的DR影像自动分类识别方法,以此达到对病变类型的精确诊断。方法 针对实验中DR数据集存在的问题,进行剔除、去噪、扩增和归一化等处理;利用EfficientNet进行特征提取,采用迁移学习的策略用DR的数据集对EfficientNet进行学习与训练,提取深度特征。为了解决病变之间差异小的问题,防止网络对糖尿病视网膜图像的特征学习时出现错分等情况,在EfficientNet输出结果上加入注意力机制;根据网络提取的特征在深度分类器中进行分类,将视网膜图像按等级进行五分类。结果 本文方法的分类精度、敏感性、特异性和二次加权（kappa）值分别为97.2%、95.6%、98.7%和0.84,具有较好的分类性能及鲁棒性。结论 基于融合注意力机制的高效率网络（attention EfficientNet,A-EfficientNet）的DR分类算法有效地提高了DR筛查效率,解决了人工分类的手动提取特征的局限性,在临床上对医生诊断起到了辅助作用,能更有效地防治此类恶性眼疾造成严重视力损伤、甚至失明。     

ON IMAGE CLASSIFICATION: CITY IMAGES VS. LANDSCAPES

Grouping images into semantically meaningful categories using low-level visual features is a challenging and important problem in content-based image retrieval. Based on these groupings, effective indices can be built for an image database. In this paper, we show how a specific high-level classification problem (city images vs landscapes) can be solved from relatively simple low-level features geared for the particular classes. We have developed a procedure to qualitatively measure the saliency of a feature towards a classification problem based on the plot of the intra-class and inter-class distance distributions. We use this approach to determine the discriminative power of the following features: color histogram, color coherence vector, DCT coefficient, edge direction histogram, and edge direction coherence vector. We determine that the edge direction-based features have the most discriminative power for the classification problem of interest here. A weighted k-NN classifier is used for the classification which results in an accuracy of 93.9% when evaluated on an image database of 2716 images using the leave-one-out method. This approach has been extended to further classify 528 landscape images into forests, mountains, and sunset/sunrise classes. First, the input images are classified as sunset/sunrise images vs forest & mountain images (94.5% accuracy) and then the forest & mountain images are classified as forest images or mountain images (91.7% accuracy). We are currently identifying further semantic classes to assign to images as well as extracting low level features which are salient for these classes. Our final goal is to combine multiple 2-class classifiers into a single hierarchical classifier.     

基于多期相注意力融合网络的肝脏病灶CT影像分类研究

肝脏病灶是指肝脏疾病集中的部位或是综合病症、感染的主要部位。由于不同类型的多期相肝脏病灶计算机断层扫描（CT）影像存在异病同影或同病异影的情况,导致同一类型的CT影像结构变化较大,传统方法难以提取丰富的图像特征信息,肝脏病灶分类准确率有待提高。提出一种多期相注意力融合网络MAFNet,使用单期相分支表征单期相病灶图像特征,并在融合分支中采用中期融合的方式,融合单期相分支中提取出的特征映射,从而充分提取图像中不同层次的特征。利用多期相注意力模块提取单期相分支中肝脏病灶的加权特征,重新组织多期相肝脏病灶的特征映射,以保持不同单期相图像信息,表达3个期相影像的时序增强模式,得到更准确的分类结果。实验结果表明,基于该网络的分类方法能充分利用多期相肝脏CT影像的时序特征,有效捕捉同一患者不同期相的信息,肝脏病灶CT影像分类的平均准确率为90.99%。     

High-order graph matching kernel for early carcinoma EUS image classification

Endoscopic ultrasonography (EUS) is limited by variability in the examiner’s subjective interpretation to differentiate between normal, leiomyoma of esophagus and early esophageal carcinoma. By using information otherwise discarded by conventional EUS systems, quantitative spectral analysis of the raw pixels (picture elements) underlying EUS image enables lesions to be characterized more objectively. In this paper, we propose to represent texture features of early esophageal carcinoma in EUS images as a graph by expressing pixels as nodes and similarity between the gray-level or local features of the EUS image as edges. Then, similarity measurements such as a high-order graph matching kernel can be constructed so as to provide an objective quantification of the properties of the texture features of early esophageal carcinoma in EUS images. This is in terms of the topology and connectivity of the analyzed graphs. Because such properties are directly related to the structure of early esophageal carcinoma lesions in EUS images, they can be used as features for characterizing and classifying early esophageal carcinoma. Finally, we use a refined SVM model based on the new high-order graph matching kernel, resulting an optimal prediction of the types of esophageal lesions. A 10-fold cross validation strategy is employed to evaluate the classification performance. After multiple computer runs of the new kernel SVM model, the overall accuracy for the diagnosis between normal, leiomyoma of esophagus and early esophageal carcinoma was 93 %. Moreover, for the diagnosis of early esophageal carcinoma, the average accuracy, sensitivity, specificity, positive predictive value, and negative predictive value were 89.4 %, 94 %, 95 %, 89 %, and 97 % respectively. The area under all the three ROC curves were close to 1.     

Fast, Accurate and Consistent Modeling of Drainage and Surrounding Terrain

We propose an automated approach to modeling drainage channels—and, more generally, linear features that lie on the terrain—from multiple images. It produces models of the features and of the surrounding terrain that are accurate and consistent and requires only minimal human intervention.We take advantage of geometric constraints and photommetric knowledge. First, rivers flow downhill and lie at the bottom of valleys whose floors tend to be either V- or U-shaped. Second, the drainage pattern appears in gray-level images as a network of linear features that can be visually detected.Many approaches have explored individual facets of this problem. Ours unifies these elements in a common framework. We accurately model terrain and features as 3-dimensional objects from several information sources that may be in error and inconsistent with one another. This approach allows us to generate models that are faithful to sensor data, internally consistent and consistent with physical constraints. We have proposed generic models that have been applied to the specific task at hand. We show that the constraints can be expressed in a computationally effective way and, therefore, enforced while initializing the models and then fitting them to the data. Furthermore, these techniques are general enough to work on other features that are constrained by predictable forces.     

Automatic Annotation and Retrieval of Images

Although a variety of techniques have been developed for content-based image retrieval (CBIR), automatic image retrieval by semantics still remains a challenging problem. We propose a novel approach for semantics-based image annotation and retrieval. Our approach is based on the monotonic tree model. The branches of the monotonic tree of an image, termed as structural elements, are classified and clustered based on their low level features such as color, spatial location, coarseness, and shape. Each cluster corresponds to some semantic feature. The category keywords indicating the semantic features are automatically annotated to the images. Based on the semantic features extracted from images, high-level (semantics-based) querying and browsing of images can be achieved. We apply our scheme to analyze scenery features. Experiments show that semantic features, such as sky, building, trees, water wave, placid water, and ground, can be effectively retrieved and located in images.     

Bilinear sparse coding for invariant vision

Recent algorithms for sparse coding and independent component analysis (ICA) have demonstrated how localized features can be learned from natural images. However, these approaches do not take image transformations into account. We describe an unsupervised algorithm for learning both localized features and their transformations directly from images using a sparse bilinear generative model. We show that from an arbitrary set of natural images, the algorithm produces oriented basis filters that can simultaneously represent features in an image and their transformations. The learned generative model can be used to translate features to different locations, thereby reducing the need to learn the same feature at multiple locations, a limitation of previous approaches to sparse coding and ICA. Our results suggest that by explicitly modeling the interaction between local image features and their transformations, the sparse bilinear approach can provide a basis for achieving transformation-invariant vision.     

融合深度网络和浅层纹理特征的甲状腺结节癌变超声图像诊断

目的 在甲状腺结节图像中对甲状腺结节进行良恶性分析,对于甲状腺癌的早期诊断有着重要的意义。随着医疗影像学的发展,大部分的早期甲状腺结节可以在超声图像中准确地检测出来,但对于结节的性质仍然缺乏准确的判断。因此,为实现更为准确的早期甲状腺结节良恶性超声图像诊断,避免不必要的针刺或其他病理活检手术、减轻病患生理痛苦和心理压力及其医疗费用,提出一种基于深度网络和浅层纹理特征融合的甲状腺结节良恶性分类新算法。方法 本文提出的甲状腺结节分类算法由4步组成。首先对超声图像进行尺度配准、人工标记以及图像复原去除以增强图像质量。然后,对增强的图像进行数据扩展,并作为训练集对预训练过的GoogLeNet卷积神经网络进行迁移学习以提取图像中的深度特征。同时,提取图像的旋转不变性局部二值模式（LBP）特征作为图像的纹理特征。最后,将深度特征与图像的纹理特征相融合并输入至代价敏感随机森林分类器中对图像进行良恶性分类。结果 本文方法在标准的甲状腺结节癌变数据集上对甲状腺结节图像取得了正确率99.15%,敏感性99.73%,特异性95.85%以及ROC曲线下面积0.997 0的的好成绩,优于现有的甲状腺结节图像分类方法。结论 实验结果表明,图像的深度特征可以描述医疗超声图像中病灶的整体感官特征,而浅层次纹理特征则可以描述超声图像的边缘、灰度分布等特征,将二者统一的融合特征则可以更为全面地描述图像中病灶区域与非病灶区域之间的差异以及不同病灶性质之间的差异。因此,本文方法可以准确地对甲状腺结节进行分类从而避免不必要手术、减轻病患痛苦和压力。     

New Spiking Cortical Model for Invariant Texture Retrieval and Image Processing

Based on the studies of existing local-connected neural network models, in this brief, we present a new spiking cortical neural networks model and find that time matrix of the model can be recognized as a human subjective sense of stimulus intensity. The series of output pulse images of a proposed model represents the segment, edge, and texture features of the original image, and can be calculated based on several efficient measures and forms a sequence as the feature of the original image. We characterize texture images by the sequence for an invariant texture retrieval. The experimental results show that the retrieval scheme is effective in extracting the rotation and scale invariant features. The new model can also obtain good results when it is used in other image processing applications.      

结合倒置特征金字塔和U-Net的高光谱图像分类

目的 地物分类是对地观测研究领域的重要任务。高光谱图像具有丰富的地物光谱信息,可用于提升遥感图像地物分类的准确度。如何对高光谱图像进行有效的特征提取与表示是高光谱图像分类应用的关键问题。为此,本文提出了一种结合倒置特征金字塔和U-Net的高光谱图像分类方法。方法 对高光谱数据进行主成分分析（principal component analysis,PCA）降维,获取作为网络输入的重构图像数据,然后使用U-Net逐层提取高光谱重构图像的空间特征。与此同时,利用倒置的特征金字塔网络抽取相应层级的语义特征;通过特征融合,得到既有丰富的空间信息又有较强烈的语义响应的特征表示。提出的网络利用注意力机制在跳跃连接过程中实现对背景区域的特征响应抑制,最终实现了较高的地物分类精度。结果 分析了PCA降维方法和输入数据尺寸对分类性能的影响,并在Indian Pines、Pavia University、Salinas和Urban数据集上进行了对比实验。本文方法在4个数据集上分别取得了98.91%、99.85%、99.99%和87.43%的总体分类精度,与支持向量机（support vector machine,SVM）等相关算法相比,分类精度高出1%~15%。结论 本文提出一种结合倒置特征金字塔和U-Net的高光谱图像分类方法,可以应用于有限训练样本下的高光谱图像分类任务,并在多个数据集上取得了较高的分类精度。实验结果表明倒置特征金字塔结构与U-Net结合的算法能够高效地实现高光谱图像的特征提取与表示,从而获得更精细的分类结果。     

基于虹膜纹理深度特征和Fisher向量的人种分类

目的 虹膜是位于人眼表面黑色瞳孔和白色巩膜之间的圆环形区域,有着丰富的纹理信息。虹膜纹理具有高度的区分性和稳定性。人种分类是解决虹膜识别在大规模数据库上应用难题的主要方法之一。现有的虹膜图像人种分类方法主要采用手工设计的特征,而且针对亚洲人和非亚洲人的基本人种分类,无法很好地解决亚种族分类问题。为此提出一种基于虹膜纹理深度特征和Fisher向量的人种分类方法。方法 首先用CNN（convolutional neural network）对归一化后的虹膜纹理图像提取深度特征向量,作为底层特征;然后使用高斯混合模型提取Fisher向量作为最终的虹膜特征表达;最后用支持向量机分类得到最终结果。结果 本文方法在亚洲人和非亚洲人的数据集上采用non-person-disjoint的方式取得99.93%的准确率,采用person-disjoint的方式取得91.94%的准确率;在汉族人和藏族人的数据集上采用non-person-disjoint的方式取得99.69%的准确率,采用person-disjoint的方式取得82.25%的准确率。结论 本文通过数据驱动的方式从训练数据中学习到更适合人种分类的特征,可以很好地实现对基本人种以及亚种族人种的分类,提高了人种分类的精度。同时也首次证明了用虹膜图像进行亚种族分类的可行性,对人种分类理论进行了进一步地丰富和完善。     

非张量积小波的肝脏CT图像检索

提出了一种基于内容的图像检索（CBIR）方法,用于医学肝脏带病灶CT图像的计算机辅助诊断（CAD）。根据医学CT图像的模糊边界和区域特征不明显等特点,将肝部感兴趣区域用半自动方法分离出来,提取局部纹理共生矩阵特征和灰度特征,然后利用改进的非张量积小波滤波器组提取图像全局特征。实验结果表明,该方法可以提高病灶的检出率,对较难鉴别诊断肝血管瘤和肝癌这两种丰富供血肿瘤的CT图像问题,也有较好的效果。     

Image Interpolation by Pixel-Level Data-Dependent Triangulation

We present a novel image interpolation algorithm. The algorithm can be used in arbitrary resolution enhancement, arbitrary rotation and other applications of still images in continuous space. High‐resolution images are interpolated from the pixel‐level data‐dependent triangulation of lower‐resolution images. It is simpler than other methods and is adaptable to a variety of image manipulations. Experimental results show that the new “mesh image” algorithm is as fast as the bilinear interpolation method. We assess the interpolated images' quality visually and also by the MSE measure which shows our method generates results comparable in quality to slower established methods. We also implement our method in graphics card hardware using OpenGL which leads to real‐time high‐quality image reconstruction. These features give it the potential to be used in gaming and image‐processing applications.     

压缩感知下溃疡性结肠炎辅助诊断

目的 长期感染溃疡性结肠炎（ulcerative colitis,UC）的患者罹患结肠癌的风险显著提升,因此早期进行结肠镜检测十分必要,但内窥镜图像数量巨大且伴有噪声干扰,需要找到精确的图像特征,为医师提供计算机辅助诊断。为解决UC图像与正常肠道图像的分类问题,提出了一种基于压缩感知和空间金字塔池化结合的图像特征提取方法。方法 使用块递归最小二乘（block recursive least squares,BRLS）进行初始字典训练。提出基于先验知识进行观测矩阵与稀疏字典的交替优化算法,并利用压缩感知框架获得图像的稀疏表示,该框架改善了原来基于稀疏编码的图像分类方法无法精确表示图像的问题,然后结合最大空间金字塔池化方法提取压缩感知空间金字塔池化（compressed sensing spatial pyramid pooling,CSSPP）图像特征,由于压缩感知的引入,获得的图像特征比稀疏编码更加丰富和精确。最后使用线性核支持向量机（support vector machine,SVM）进行图像分类。结果 对Kvasir数据集中的2 000幅真实肠道图像的分类结果表明,该特征的准确率比特征袋（bag of features,BoF）、稀疏编码空间金字塔匹配（sparse coding spatial pyramid matching,SCSPM）和局部约束线性编码（locality-constrained linear coding,LLC）分别提升了12.35%、3.99%和2.27%。结论 本文提出的溃疡性结肠炎辅助诊断模型,综合了压缩感知和空间金字塔池化的优点,获得了较对比方法更加精确的识别感染图像检测结果。