Green learning: Introduction,examples and outlook

Rapid advances in artificial intelligence (AI) in the last decade have been largely built upon the wide applications of deep learning (DL). However, the high carbon footprint yielded by larger and larger DL networks has become a concern for sustainability. Furthermore, DL decision mechanism is somewhat obscure in that it can only be verified by test data. Green learning (GL) is being proposed as an alternative paradigm to address these concerns. GL is characterized by low carbon footprints, lightweight model, low computational complexity, and logical transparency. It offers energy-efficient solutions in cloud centers as well as mobile/edge devices. GL also provides a more transparent, logical decision-making process which is essential to gaining people’s trust. Several statistical tools such as unsupervised representation learning, supervised feature learning, and supervised decision learning, have been developed to achieve this goal in recent years. We have seen a few successful GL examples with performance comparable with state-of-the-art DL solutions. This paper introduces the key characteristics of GL, its demonstrated applications, and future outlook.     

Image blind restoration based on degradation representation network

Most deep learning (DL)-based image restoration methods have exploited excellent performance by learning a non-linear mapping function from low quality images to high quality images. However, two major problems restrict the development of the image restoration methods. First, most existing methods based on fixed degradation suffer from significant performance drop when facing the unknown degradation, because of the huge gap between the fixed degradation and the unknown degradation. Second, the unknown-degradation estimation may lead to restoration task failure due to uncertain estimation errors. To handle the unknown degradation in the real application, we introduce a degradation representation network for single image blind restoration (DRN). Different from the methods of estimating pixel space, we use an encoder network to learn abstract representations for estimating different degradation kernels in the representation space. Furthermore, a degradation perception module with flexible adaptability to different degradation kernels is used to restore more structural details. In our experiments, we compare our DRN with several state-of-the-art methods for two image restoration tasks, including image super-resolution (SR) and image denoising. Quantitative results show that our degradation representation network is accurate and efficient for single image restoration.     

SSCDenseNet:一种空-谱卷积稠密网络的高光谱图像分类算法

基于深度学习的高光谱遥感图像地物分类是目前研究的热点.但由于其参数规模大以及结构复杂,深度网络通常需要大量训练样本和较长训练时间,如何在小规模样本下建立深度学习监督分类模型是需要解决的关键问题.本文提出了一种小规模样本下高光谱图像分类的空-谱卷积稠密网络算法,称为SSCDenseNet,其包含三种新颖的架构策略：（1）空-谱分离卷积,即采取光谱维一维卷积和空间维二维卷积的分离卷积结构构成隐层单元,并通过多个隐层单元堆叠构造深度网络;（2）隐层单元中使用批归一化,减少数据协方差漂移及加速网络训练;（3）隐层单元间构建稠密连接,缓解梯度消失问题并实现特征复用.通过Indian Pines、Pavia University与Salinas数据集进行综合测评,表明该方法优于若干最新深度学习方法,特别在小规模样本下具有优异的分类性能.     

Cross-domain representation learning by domain-migration generative adversarial network for sketch based image retrieval

Sketch based image retrieval (SBIR), which uses free-hand sketches to search the images containing similar objects/scenes, is attracting more and more attentions as sketches could be got more easily with the development of touch devices. However, this task is difficult as the huge differences between sketches and images. In this paper, we propose a cross-domain representation learning framework to reduce these differences for SBIR. This framework aims to transfer sketches to images with the information learned both in the sketch domain and image domain by the proposed domain migration generative adversarial network (DMGAN). Furthermore, to reduce the representation gap between the generated images and natural images, a similarity learning network (SLN) is also proposed with the new designed loss function incorporating semantic information. Extensive experiments have been done from different aspects, including comparison with state-of-the-art methods. The results show that the proposed DMGAN and SLN really work for SBIR.     

A survey on deep learning for textual emotion analysis in social networks

Textual Emotion Analysis (TEA) aims to extract and analyze user emotional states in texts. Various Deep Learning (DL) methods have developed rapidly, and they have proven to be successful in many fields such as audio, image, and natural language processing. This trend has drawn increasing researchers away from traditional machine learning to DL for their scientific research. In this paper, we provide an overview of TEA based on DL methods. After introducing a background for emotion analysis that includes defining emotion, emotion classification methods, and application domains of emotion analysis, we summarize DL technology, and the word/sentence representation learning method. We then categorize existing TEA methods based on text structures and linguistic types: text-oriented monolingual methods, text conversations-oriented monolingual methods, text-oriented cross-linguistic methods, and emoji-oriented cross-linguistic methods. We close by discussing emotion analysis challenges and future research trends. We hope that our survey will assist readers in understanding the relationship between TEA and DL methods while also improving TEA development.     

Iterative transductive learning for automatic image segmentation and matting with RGB-D data

In this paper, we propose a fully automatic image segmentation and matting approach with RGB-Depth (RGB-D) data based on iterative transductive learning. The algorithm consists of two key elements: robust hard segmentation for trimap generation, and iterative transductive learning based image matting. The hard segmentation step is formulated as a Maximum A Posterior (MAP) estimation problem, where we iteratively perform depth refinement and bi-layer classification to achieve optimal results. For image matting, we propose a transductive learning algorithm that iteratively adjusts the weights between the objective function and the constraints, overcoming common issues such as over-smoothness in existing methods. In addition, we present a new way to form the Laplacian matrix in transductive learning by ranking similarities of neighboring pixels, which is essential to efficient and accurate matting. Extensive experimental results are reported to demonstrate the state-of-the-art performance of our method both subjectively and quantitatively.     

Kernel dictionary learning based discriminant analysis

Sparse representation based classification (SRC) has been successfully applied in many applications. But how to determine appropriate features that can best work with SRC remains an open question. Dictionary learning (DL) has played an import role in the success of sparse representation, while SRC treats the entire training set as a structured dictionary. In addition, as a linear algorithm, SRC cannot handle the data with highly nonlinear distribution. Motivated by these concerns, in this paper, we propose a novel feature learning method (termed kernel dictionary learning based discriminant analysis, KDL-DA). The proposed algorithm aims at learning a projection matrix and a kernel dictionary simultaneously such that in the reduced space the sparse representation of the data can be easily obtained, and the reconstruction residual can be further reduced. Thus, KDL-DA can achieve better performances in the projected space. Extensive experimental results show that our method outperforms many state-of-the-art methods.     

Context-aware joint dictionary learning for color image demosaicking

Most digital cameras are overlaid with color filter arrays (CFA) on their electronic sensors, and thus only one particular color value would be captured at every pixel location. When producing the output image, one needs to recover the full color image from such incomplete color samples, and this process is known as demosaicking. In this paper, we propose a novel context-constrained demosaicking algorithm via sparse-representation based joint dictionary learning. Given a single mosaicked image with incomplete color samples, we perform color and texture constrained image segmentation and learn a dictionary with different context categories. A joint sparse representation is employed on different image components for predicting the missing color information in the resulting high-resolution image. During the dictionary learning and sparse coding processes, we advocate a locality constraint in our algorithm, which allows us to locate most relevant image data and thus achieve improved demosaicking performance. Experimental results show that the proposed method outperforms several existing or state-of-the-art techniques in terms of both subjective and objective evaluations.     

Optimizing Deep Learning Parameters Using Genetic Algorithm for Object Recognition and Robot Grasping

The performance of deep learning (DL) networks has been increased by elaborating the network structures. However, the DL netowrks have many parameters, which have a lot of influence on the performance of the network. We propose a genetic algorithm (GA) based deep belief neural network (DBNN) method for robot object recognition and grasping purpose. This method optimizes the parameters of the DBNN method, such as the number of hidden units, the number of epochs, and the learning rates, which would reduce the error rate and the network training time of object recognition. After recognizing objects, the robot performs the pick-and-place operations. We build a database of six objects for experimental purpose. Experimental results demonstrate that our method outperforms on the optimized robot object recognition and grasping tasks.     

面向开放域问答的问题分类技术研究进展

开放域问答是当前自然语言处理和信息检索领域的研究热点,作为开放域问答系统的重要组成部分,问题分类可以缩小答案的搜索空间并决定答案的选择策略.近年来,基于机器学习的问题分类技术受到广泛的关注,相关研究表明问题分类的准确性直接影响问答系统的整体性能.本文从分类体系与数据集、特征提取、分类器设计、性能评测等层面,总结了问题分类技术近年的主要研究成果.重点分析了各种基于监督学习的问题分类方法的特点和不足,讨论了核方法、半监督学习、主动学习、迁移学习等在问题分类中的应用,同时对问题分类技术未来研究动向进行了展望.     

Hybrid gradient-domain image denoising

Presented is a new gradient-domain denoising method based on hybrid diffusion (thresholding) functions, combining signal gradient detection (SGD) and signal local directional variance (SLDV). In the process of denoising, the contribution of SGD and SLDV is adaptive to the contents of image. The test results presented here demonstrate that the proposed hybrid method is always on par or exceeding the current state-of-the-art gradient-domain image denoising algorithm which is named as gradient-based Wiener filter (GWF) based on SLDV and the classical Gaussian regularization anisotropic diffusion (GRAD) based on SGD, both visually and quantitatively. At the same time, the comparison compared to other reported results with related local spatial domain diffusion-based methods further verifies the good performance of the proposed method.     

Bi-directional LSTM based channel estimation in 5G massive MIMO OFDM systems over TDL-C model with Rayleigh fading distribution

In this work, a deep learning (DL)-based massive multiple-input multiple-output (mMIMO) orthogonal frequency division multiplexing (OFDM) system is investigated over the tapped delay line type C (TDL-C) model with a Rayleigh fading distribution at frequencies ranging from 0.5 to 100 GHz. The proposed bi-directional long short-term memory (Bi-LSTM) channel state information (CSI) estimator uses online learning during training and offline learning during the practical implementation phase. The design of the estimator takes into account situations in which prior knowledge of channel statistics is limited and targets excellent performance, even with limited pilot symbols (PS). Three separate loss functions (mean square logarithmic error [MSLE], Huber, and Kullback–Leibler Distance [KLD]) are assessed in three classification layers. The symbol error rate (SER) and outage probability performance of the proposed estimator are evaluated using a number of optimization techniques, such as stochastic gradient descent (SGD), momentum, and the adaptive gradient (AdaGrad) algorithm. The Bi-LSTM-based CSI estimator is trained considering a specific number of PS. It can be readily seen that by incorporating a cyclic prefix (CP), the system becomes more resilient to channel impairments, resulting in a lower SER. Simulations show that the SGD optimization approach and Huber loss function-trained Bi-LSTM-based CSI estimator have the lowest SER and very high estimation accuracy. By using deep neural networks (DNNs), the Bi-LSTM method for CSI estimation achieves a superior channel capacity (in bps/Hz) at 10 dB than long short-term memory (LSTM) and other conventional CSI estimators, such as minimum mean square error (MMSE) and least squares (LS). The simulation results validate the analytical results in the study.     

一种基于稀疏编码的多核学习图像分类方法

 本文提出一种基于稀疏编码的多核学习图像分类方法.传统稀疏编码方法对图像进行分类时,损失了空间信息,本文采用对图像进行空间金字塔多划分方式为特征加入空间信息限制.在利用非线性SVM方法进行图像分类时,空间金字塔的各层分别形成一个核矩阵,本文使用多核学习方法求解各个核矩阵的权重,通过核矩阵的线性组合来获取能够对整个分类集区分能力最强的核矩阵.实验结果表明了本文所提出图像分类方法的有效性和鲁棒性.对Scene Categories场景数据集可以达到83.10%的分类准确率,这是当前该数据集上能达到的最高分类准确率.     

基于增量式深度神经网络的图像分类系统

为解决图像分类任务中模型结构固化、产生巨大内存消耗、时间消耗的问题,提出一种增量式深度神经网络（IDNN）。输入样本通过聚类算法激活不同簇并被分别处理：如果新样本激活已有簇,则更新该簇参数;否则为新簇开辟分支,并训练独立特征集。在Caltech-101、ORL Face、ETH-80数据库的验证结果表明,该系统能自动调整网络结构,适用于轮廓、纹理、视角等不同环境的增量式学习,例如在Caltech-101库分类任务中准确率超出VGGNet 5.08%、AlexNet 3.44%。     

PixelHop: A successive subspace learning (SSL) method for object recognition

A new machine learning methodology, called successive subspace learning (SSL), is introduced in this work. SSL contains four key ingredients: (1) successive near-to-far neighborhood expansion; (2) unsupervised dimension reduction via subspace approximation; (3) supervised dimension reduction via label-assisted regression (LAG); and (4) feature concatenation and decision making. An image-based object classification method, called PixelHop, is proposed to illustrate the SSL design. It is shown by experimental results that the PixelHop method outperforms the classic CNN model of similar model complexity in three benchmarking datasets (MNIST, Fashion MNIST and CIFAR-10). Although SSL and deep learning (DL) have some high-level concept in common, they are fundamentally different in model formulation, the training process and training complexity. Extensive discussion on the comparison of SSL and DL is made to provide further insights into the potential of SSL.     

基于身份感知模型的素描人脸识别方法

针对素描图像和光学图像之间存在较大的模态差异这一问题,提出了一种基于身份感知模型的素描人脸识别方法,实现跨模态图像生成和素描人脸识别。该方法应用新的感知损失来监督图像生成网络,生成更好的跨模态图像,减少模态差异带来的识别精度损失,并通过三元组损失来正则化类内和类间距离,增强识别模型的性能,用联合训练策略提升素描人脸识别能力。在UoM-SGFSv2、e-PRIP等素描人脸数据集上的实验结果表明,该方法识别效果优于其他对比算法。     

Framework for COVID-19 Segmentation and Classification Based on Deep Learning of Computed Tomography Lung Images

Corona Virus Disease 2019 (COVID-19) has affected millions of people worldwide and caused more than 6.3 million deaths (World Health Organization, June 2022). Increased attempts have been made to develop deep learning methods to diagnose COVID-19 based on computed tomography (CT) lung images. It is a challenge to reproduce and obtain the CT lung data, because it is not publicly available. This paper introduces a new generalized framework to segment and classify CT images and determine whether a patient is tested positive or negative for COVID-19 based on lung CT images. In this work, many different strategies are explored for the classification task. ResNet50 and VGG16 models are applied to classify CT lung images into COVID-19 positive or negative. Also, VGG16 and ReNet50 combined with U-Net, which is one of the most used architectures in deep learning for image segmentation, are employed to segment CT lung images before the classifying process to increase system performance. Moreover, the image size dependent normalization technique (ISDNT) and Wiener filter are utilized as the preprocessing techniques to enhance images and noise suppression. Additionally, transfer learning and data augmentation techniques are performed to solve the problem of COVID-19 CT lung images deficiency, therefore the over-fitting of deep models can be avoided. The proposed frameworks, which comprised of end-to-end, VGG16, ResNet50, and U-Net with VGG16 or ResNet50, are applied on the dataset that is sourced from COVID-19 lung CT images in Kaggle. The classification results show that using the preprocessed CT lung images as the input for U-Net hybrid with ResNet50 achieves the best performance. The proposed classification model achieves the 98.98% accuracy (ACC), 98.87% area under the ROC curve (AUC), 98.89% sensitivity (Se), 97.99 % precision (Pr), 97.88% F₁-score, and 1.8974-seconds computational time.     

基于联邦学习的无线任务:数据非IID一定影响性能？

作为一种分布式训练框架,联邦学习在无线通信领域有着广阔的应用前景,也面临着多方面的技术挑战,其中之一源于参与训练用户数据集的非独立同分布（Independent and identically distributed,IID）。不少文献提出了解决方法,以减轻户数据集非IID造成的联邦学习性能损失。本文以平均信道增益预测、正交幅度调制信号的解调这两个无线任务以及两个图像分类任务为例,分析用户数据集非IID对联邦学习性能的影响,通过神经网络损失函数的可视化和对模型参数的偏移量进行分析,尝试解释非IID数据集对不同任务影响程度不同的原因。分析结果表明,用户数据集非IID未必导致联邦学习性能的下降。在不同数据集上通过联邦平均算法训练得到的模型参数偏移程度和损失函数形状有很大的差异,二者共同导致了不同任务受数据非IID影响程度的不同;在同一个回归问题中,数据集非IID是否影响联邦学习的性能与引起数据非IID的具体因素有关。      

A dynamic learning neural network for remote sensing applications

The neural network learning process is to adjust the network weights to adapt the selected training data. Based on the polynomial basis function (PBF) modeled neural network that is a modified multilayer perceptrons (MLP) network, a dynamic learning algorithm (DL) is proposed. The presented learning algorithm makes use of the Kalman filtering technique to update the network weights, in the sense that the stochastic characteristics of incoming data sets are implicitly incorporated into the network. The Kalman gains which represent the learning rates of the network weights updating are calculated by using the U-D factorization. By concatenating all of the network weights at each layer to form a long vector such that it can be updated without propagating back, the proposed algorithm improves the performance of convergence to which the backpropagation (BP) learning algorithm often suffers. Numerical illustrations are carried out using two categories of problems: multispectral imagery classification and surface parameters inversion. Results indicates the use of Kalman filtering algorithm not only substantially increases the convergence rate in the learning stage, but also enhances the separability for highly nonlinear boundaries problems, as compared to BP algorithm, suggesting that the proposed DL neural network provides a practical and potential tool for remote sensing applications     

Extreme learning machine with multi-scale local receptive fields for texture classification

Texture classification is a challenging task due to the wide range of natural texture types and large intra-class variations in texture images, such as different rotations, scales, positions and lighting conditions. Many existing methods for extracting texture features are designed carefully by user for specific applications. The extracted texture features are then used as input to various classification methods, such as support vector machines, to classify the textures. The system performance greatly depends on the feature extractor. Unfortunately, there is no systematic approach for feature extractor design. In this paper, we propose a method called extreme learning machine with multi-scale local receptive fields (ELM-MSLRF) to achieve feature learning and classification simultaneously for texture classification. In contrast to traditional methods, the proposed method learns the features by the network itself and can be applied to more general applications. Additionally, it is fast and requires few computations. Experiments on the ALOT texture dataset demonstrate the attractive performance of ELM-MSLRF even compared with the state-of-the-art algorithms. Moreover, the proposed ELM-MSLRF achieves the best performance on the NORB dataset.