Efficient implementation of class of isotropic quadratic filters byusing Walsh-Hadamard transform

A method is proposed for the efficient implementation of a class of second-order Volterra filters where their quadratic kernels are recursively constructed from a set of isotropic subkernels (ISKs). The quadratic kernel of this class of filters can be factorised into a diagonal form by a Walsh-Hadamard transform (WHT) and be implemented using a transform and multiply structure. An algorithm for determining the quadratic ISK kernel for the nonlinear prediction of speech signals is presented     

Nonlinear support vector machine visualization for risk factor analysis using nomograms and localized radial basis function kernels

Nonlinear classifiers, e.g., support vector machines (SVMs) with radial basis function (RBF) kernels, have been used widely for automatic diagnosis of diseases because of their high accuracies. However, it is difficult to visualize the classifiers, and thus difficult to provide intuitive interpretation of results to physicians. We developed a new nonlinear kernel, the localized radial basis function (LRBF) kernel, and new visualization system visualization for risk factor analysis (VRIFA) that applies a nomogram and LRBF kernel to visualize the results of nonlinear SVMs and improve the interpretability of results while maintaining high prediction accuracy. Three representative medical datasets from the University of California, Irvine repository and Statlog dataset-breast cancer, diabetes, and heart disease datasets-were used to evaluate the system. The results showed that the classification performance of the LRBF is comparable with that of the RBF, and the LRBF is easy to visualize via a nomogram. Our study also showed that the LRBF kernel is less sensitive to noise features than the RBF kernel, whereas the LRBF kernel degrades the prediction accuracy more when important features are eliminated. We demonstrated the VRIFA system, which visualizes the results of linear and nonlinear SVMs with LRBF kernels, on the three datasets.     

A tree-structured piecewise linear adaptive filter

The authors propose and analyze a novel architecture for nonlinear adaptive filters. These nonlinear filters are piecewise linear filters obtained by arranging linear filters and thresholds in a tree structure. A training algorithm is used to adaptively update the filter coefficients and thresholds at the nodes of the tree, and to prune the tree. The resulting tree-structured piecewise linear adaptive filter inherits the robust estimation and fast adaptation of linear adaptive filters, along with the approximation and model-fitting properties of tree-structured regression models. A rigorous analysis of the training algorithm for the tree-structured filter is performed. Some techniques are developed for analyzing hierarchically organized stochastic gradient algorithms with fixed gains and nonstationary dependent data. Simulation results show the significant advantages of the tree-structured piecewise linear filter over linear and polynomial filters for adaptive echo cancellation     

General-form 3-3-3 interpolation kernel and its simplified frequency-response derivation

An interpolation kernel is required in a wide variety of signal processing applications such as image interpolation and timing adjustment in digital communications. This article presents a general-form interpolation kernel called 3-3-3 interpolation kernel and derives its frequency response in a closed-form by using a simple derivation method. This closed-form formula is preliminary to designing various 3-3-3 interpolation kernels subject to a set of design constraints. The 3-3-3 interpolation kernel is formed through utilising the third-degree piecewise polynomials, and it is an even-symmetric function. Thus, it will suffice to consider only its right-hand side when deriving its frequency response. Since the right-hand side of the interpolation kernel contains three piecewise polynomials of the third degree, i.e. the degrees of the three piecewise polynomials are (3,3,3), we call it the 3-3-3 interpolation kernel. Once the general-form frequency-response formula is derived, we can systematically formulate the design of various 3-3-3 interpolation kernels subject to a set of design constraints, which are targeted for different interpolation applications. Therefore, the closed-form frequency-response expression is preliminary to the optimal design of various 3-3-3 interpolation kernels. We will use an example to show the optimal design of a 3-3-3 interpolation kernel based on the closed-form frequency-response expression.     

Use of meixner functions in estimation of Volterra kernels of nonlinear systems with delay

Volterra series representation of nonlinear systems is a mathematical analysis tool that has been successfully applied in many areas of biological sciences, especially in the area of modeling of hemodynamic response. In this study, we explored the possibility of using discrete time Meixner basis functions (MBFs) in estimating Volterra kernels of nonlinear systems. The problem of estimation of Volterra kernels can be formulated as a multiple regression problem and solved using least squares estimation. By expanding system kernels with some suitable basis functions, it is possible to reduce the number of parameters to be estimated and obtain better kernel estimates. Thus far, Laguerre basis functions have been widely used in this framework. However, research in signal processing indicates that when the kernels have a slow initial onset or delay, Meixner functions, which can be made to have a slow start, are more suitable in terms of providing a more accurate approximation to the kernels. We, therefore, compared the performance of Meixner functions, in kernel estimation, to that of Laguerre functions in some test cases that we constructed and in a real experimental case where we studied photoreceptor responses of photoreceptor cells of adult fruitflies (Drosophila melanogaster). Our results indicate that when there is a slow initial onset or delay, MBF expansion provides better kernel estimates.     

Sampling signals with finite rate of innovation

The authors consider classes of signals that have a finite number of degrees of freedom per unit of time and call this number the rate of innovation. Examples of signals with a finite rate of innovation include streams of Diracs (e.g., the Poisson process), nonuniform splines, and piecewise polynomials. Even though these signals are not bandlimited, we show that they can be sampled uniformly at (or above) the rate of innovation using an appropriate kernel and then be perfectly reconstructed. Thus, we prove sampling theorems for classes of signals and kernels that generalize the classic "bandlimited and sinc kernel" case. In particular, we show how to sample and reconstruct periodic and finite-length streams of Diracs, nonuniform splines, and piecewise polynomials using sinc and Gaussian kernels. For infinite-length signals with finite local rate of innovation, we show local sampling and reconstruction based on spline kernels. The key in all constructions is to identify the innovative part of a signal (e.g., time instants and weights of Diracs) using an annihilating or locator filter: a device well known in spectral analysis and error-correction coding. This leads to standard computational procedures for solving the sampling problem, which we show through experimental results. Applications of these new sampling results can be found in signal processing, communications systems, and biological systems     

Linear multichannel blind equalizers of nonlinear FIR Volterrachannels

Truncated Volterra expansions model nonlinear systems encountered with satellite communications, magnetic recording channels, and physiological processes. A general approach for blind deconvolution of single-input multiple-output Volterra finite impulse response (FIR) systems is presented. It is shown that such nonlinear systems can be blindly equalized using only linear FIR filters. The approach requires that the Volterra kernels satisfy a certain coprimeness condition and that the input possesses a minimal persistence-of-excitation order. No other special conditions are imposed on the kernel transfer functions or on the input signal, which may be deterministic or random with unknown statistics. The proposed algorithms are corroborated with simulation examples     

Efficient kernel functions for support vector machine regression model for analog circuits’ performance evaluation

Support vector machines (SVMs) have been widely used for creating fast and efficient performance macro-models for quickly predicting the performance parameters of analog circuits. These models have proved to be not only effective and fast but accurate also while predicting the performance. A kernel function is an integral part of SVM to obtain an optimized and accurate model. There is no formal way to decide, which kernel function is suited to a class of regression problem. While most commonly used kernels are radial basis function, polynomial, spline, multilayer perceptron; we have explored many other un-conventional kernel functions and report their efficacy and computational efficiency in this paper. These kernel functions are used with SVM regression models and these macromodels are tested on different analog circuits to check for their robustness and performance. We have used HSPICE for generating the set of learning data. Least Square SVM toolbox along with MATLAB was used for regression. The models which contained modified compositions of kernels were found to be more accurate and thus have lower root mean square error than those containing standard kernels. We have used different CMOS circuits varying in size and complexity as test vehicles—two-stage op amp, cascode op amp, comparator, differential op amp and voltage controlled oscillator.     

Polyphase antialiasing in resampling of images.

Changing resolution of images is a common operation. It is also common to use simple, i.e., small, interpolation kernels satisfying some "smoothness" qualities that are determined in the spatial domain. Typical applications use linear interpolation or piecewise cubic interpolation. These are popular since the interpolation kernels are small and the results are acceptable. However, since the interpolation kernel, i.e., impulse response, has a finite and small length, the frequency domain characteristics are not good. Therefore, when we enlarge the image by a rational factor of (L/M), two effects usually appear and cause a noticeable degradation in the quality of the image. The first is jagged edges and the second is low-frequency modulation of high-frequency components, such as sampling noise. Both effects result from aliasing. Enlarging an image by a factor of (L/M) is represented by first interpolating the image on a grid L times finer than the original sampling grid, and then resampling it every M grid points. While the usual treatment of the aliasing created by the resampling operation is aimed toward improving the interpolation filter in the frequency domain, this paper suggests reducing the aliasing effects using a polyphase representation of the interpolation process and treating the polyphase filters separately. The suggested procedure is simple. A considerable reduction in the aliasing effects is obtained for a small interpolation kernel size. We discuss separable interpolation and so the analysis is conducted for the one-dimensional case.     

Toward an optimal supervised classifier for the analysis of hyperspectral data

In this paper, we propose a supervised classifier based on implementation of the Bayes rule with kernels. The proposed technique first proposes an implicit nonlinear transformation of the data into a feature space seeking to fit normal distributions having a common covariance matrix onto the mapped data. One requirement of this approach is the evaluation of posterior probabilities. We express the discriminant function in dot-product form, and then apply the kernel concept to efficiently evaluate the posterior probabilities. The proposed technique gives the flexibility required to model complex data structures that originate from a wide range of class-conditional distributions. Although we end up with piecewise linear decision boundaries in the feature space, these corresponds to powerful nonlinear boundaries in the original input space. For the data we considered, we have obtained some encouraging results.     

Permutation weighted order statistic filter lattices

We introduce and analyze a new class of nonlinear filters called permutation weighted order statistic (PWOS) filters. These filters extend the concept of weighted order statistic (WOS) filters, in which filter weights associated with the input samples are used to replicate the corresponding samples, and an order statistic is chosen as the filter output. PWOS filters replicate each input sample according to weights determined by the temporal-order and rank-order of samples within a window. Hence, PWOS filters are in essence time-varying WOS filters. By varying the amount of temporal-rank order information used in selecting the output for a given observation window size, we obtain a wide range of filters that are shown to comprise a complete lattice structure. At the simplest level in the lattice, PWOS filters reduce to the well-known WOS filter, but for higher levels in the lattice, the obtained selection filters can model complex nonlinear systems and signal distortions. It is shown that PWOS filters are realizable by a N! piecewise linear threshold logic gate where the coefficients within each partition can be easily optimized using stack filter theory. Simulations are included to show the advantages of PWOS filters for the processing of image and video signals.     

Restoration and reconstruction of AVHRR images

Describes the design of small convolution kernels for the restoration and reconstruction of Advanced Very High Resolution Radiometer (AVHRR) images. The kernels are small enough to be implemented efficiently by convolution, yet effectively correct degradations and increase apparent resolution. The kernel derivation is based on a comprehensive, end-to-end system model that accounts for scene statistics, image acquisition blur, sampling effects, sensor noise, and postfilter reconstruction. The design maximizes image fidelity subject to explicit constraints on the spatial support and resolution of the kernel. The kernels can be designed with finer resolution than the image to perform partial reconstruction for geometric correction and other remapping operations. Experiments demonstrate that small kernels yield fidelity comparable to optimal unconstrained filters with less computation     

Representations of linear periodically time-varying and multirate systems

The linear switched time-varying (LSTV), polyphase (blocked), and alias-component representations of linear periodically time-varying (LPTV) systems are studied. In particular, alias-components are related to the time-shifted versions of the system. It is shown that in general, a filterbank is equivalent to the cascade connection of two LPTV systems. By generalizing some of the results of the LSTV representation of LPTV systems, it is shown that for relatively coprime integers p and m, a p-channel filterbank that has LPTV filters with period m is equivalent to an mp-channel filterbank with LTI filters. The representation of multirate systems that have LPTV kernels is discussed next. Due to the presence of the upsampler and downsampler, there are some degrees of freedom in the choice of the kernel. This redundancy is dealt with by choosing from various subclasses so that there is a one-to-one relationship between a multirate system and its kernel. Then, we find the LPTV kernel that has the least period.     

Discretization of the radon transform and of its inverse by spline convolutions

We present an explicit formula for B-spline convolution kernels; these are defined as the convolution of several B-splines of variable widths h(i) and degrees n(i). We apply our results to derive spline-convolution-based algorithms for two closely related problems: the computation of the Radon transform and of its inverse. First, we present an efficient discrete implementation of the Radon transform that is optimal in the least-squares sense. We then consider the reverse problem and introduce a new spline-convolution version of the filtered back-projection algorithm for tomographic reconstruction. In both cases, our explicit kernel formula allows for the use of high-degree splines; these offer better approximation performance than the conventional lower-degree formulations (e.g., piecewise constant or piecewise linear models). We present multiple experiments to validate our approach and to find the parameters that give the best tradeoff between image quality and computational complexity. In particular, we find that it can be computationally more efficient to increase the approximation degree than to increase the sampling rate.     

Using multirate architectures in realizing quadratic Volterrakernels

Multirate architectures have been used for realizing linear FIR digital filters with reduced computational complexity. The Volterra kernel can be represented as a generalized convolution. It would thus be expected that multirate architectures could be used to advantage in realizing Volterra kernels as well. The quadratic Volterra kernel may be realized in the form of an “LDL structure.” The LDL structure includes a set of FIR filters of increasing length, which may be realized in a computationally efficient manner using multirate architectures     

隐相空间的DUPSO-RPSOVF语音预测模型研究

提出了一种基于二阶Volterra级数的语音信号非线性预测模型.为克服传统的最小均方（Least Mean Square,LMS）算法在模型核系数更新时的固有缺点,引入耗散均匀搜索粒子群优化算法（Dissipative Uniform Particle Swarm Optimization,DUPSO）求解核系数,并构建了DUPSO-SOVF预测模型;为避免传统方法中相空间的重构过程,构建了隐相空间DUPSO-SOVF预测模型,在求解模型核系数时动态地求解出最优嵌入维数和延迟时间;为降低模型复杂度,在误差允许范围内进行模型关键项的提取,从而减少了核系数个数,构建了少参数的DUPSO-RPSOVF（Reduced Parameter SOVF,RPSOVF）预测模型.将英语音素、单词和短语作为实验样本数据进行仿真,结果表明：隐相空间DUPSO-SOVF模型能够准确的计算出相空间重构参数,DUPSO-SOVF和DUPSO-RPSOVF两种预测模型对单帧和多帧语音信号均具有较高的预测精度,优于PSO-SOVF和LMS-SOVF预测模型,并且能够很好地反映语音序列变化的趋势和规律,可以满足语音序列预测的要求.     

Design of multistage separable two-dimensional digital filters with frequency domain specifications

The separable decomposition method for 2-D filter design is examined in the continuous frequency domain. In the general case, improved accuracy and better insight can be obtained in comparison with the discrete counterparts as examined in the literature. Substantial simplifications both for the off-line computational load and the subsequent implementation are possible for frequency response functions being constant on their effective domain. This is achieved through a piecewise linear approximation of the domain boundary which reduces the original eigenvalue problem for operators with 2-D kernel into a solution of decoupled simple boundary-value problems in each direction. The latter are solvable through a simple Newton-Raphson procedure combined with a shooting method. This results into 1-D constituent filters with piecewise sinusoidal frequency responses. Completely analytical results are obtained for fan and directional filter design. Error estimates are given and examples illustrate the applicability of the results.     

Piecewise linear system modeling based on a continuous thresholddecomposition

The continuous threshold decomposition is a segmentation operator used to split a signal into a set of multilevel components. This decomposition method can be used to represent continuous multivariate piecewise linear (PWL) functions and, therefore, can be employed to describe PWL systems defined over a rectangular lattice. The resulting filters are canonical and have a multichannel structure that can be exploited for the development of rapidly convergent algorithms. The optimum design of the class of PWL filters introduced in this paper can be postulated as a least squares problem whose variables separate into a linear and a nonlinear part. Based on this feature, parameter estimation algorithms are developed. First, a block data processing algorithm that combines linear least-squares with grid localization through recursive partitioning is introduced. Second, a time-adaptive method based on the combination of an RLS algorithm for coefficient updating and a signed gradient descent module for threshold adaptation is proposed and analyzed. A system identification problem for wave propagation through a nonlinear multilayer channel serves as a comparative example where the concepts introduced are tested against the linear, Volterra, and neural network alternatives     

一种基于稀疏编码的多核学习图像分类方法

 本文提出一种基于稀疏编码的多核学习图像分类方法.传统稀疏编码方法对图像进行分类时,损失了空间信息,本文采用对图像进行空间金字塔多划分方式为特征加入空间信息限制.在利用非线性SVM方法进行图像分类时,空间金字塔的各层分别形成一个核矩阵,本文使用多核学习方法求解各个核矩阵的权重,通过核矩阵的线性组合来获取能够对整个分类集区分能力最强的核矩阵.实验结果表明了本文所提出图像分类方法的有效性和鲁棒性.对Scene Categories场景数据集可以达到83.10%的分类准确率,这是当前该数据集上能达到的最高分类准确率.