基于信息论准则的信源个数估计算法改进

基于信息论准则(AIC准则,MDL准则)的信源数估计算法及其改进算法在低快怕数和低信噪比情况下大多得不到较好的估计性能.从最基本的MDL准则出发,充分考虑AIC准则与MDL准则估计的优缺点,通过引入影响因子对准则的代价函数进行改进,这样的改进在低快怕数与低信噪比下都能得到较好的估计性能,且具有估计一致性、算法复杂度低等优点.仿真试验验证了该改进算法的有效性.     

SVD-based information theoretic criteria for detection of thenumber of damped/undamped sinusoids and their performance analysis

Following the information-theoretic approach to model selection, the authors develop criteria for detection of the number of damped/undamped sinusoids. These criteria are matched to the singular-value-decomposition (SVD)-based methods, such as modified forward/backward and forward-backward linear prediction, so well that the extra computations needed over and above those required for computing the SVD are marginal. Next, an analytical framework for analyzing the performance of these criteria is developed. In the development of the analysis, some approximations which become better for large signal-to-noise-ratio are made. Simulations are used to verify the usefulness of the analysis, and to compare the performance of the method with that of J.J. Fuchs (1988)     

Nonlinear filtering methods for harmonic retrieval and model orderselection in Gaussian and non-Gaussian noise

This paper addresses the problem of high-resolution parameter estimation (harmonic retrieval) and model-order selection for superimposed sinusoids. The harmonic retrieval problem is analyzed using a nonlinear parameter estimation approach. Estimation is performed using several nonlinear estimators with signals embedded in white and colored Gaussian noise. Simulation results demonstrate that the nonlinear filters perform close to the Cramer-Rao bound. Model order selection is performed in Gaussian and non-Gaussian noise. The problem is formulated using a multiple hypothesis testing approach with assumed known a priori probabilities for each hypothesis. Parameter estimation is performed using the extended Kalman filter when the noise is Gaussian. The extended high-order filter (EHOF) is implemented in non-Gaussian noise. Simulation results demonstrate excellent performance in selecting the correct model order and estimating the signal parameters     

Information theoretic criteria for the determination of the numberof signals in spatially correlated noise

The problem of determining the number of signals in high-resolution array processing when the noise is spatially correlated (having an unknown covariance matrix) is examined. By considering a model in which two sensor arrays are well separated such that their noise outputs are uncorrelated, the authors develop a likelihood function whose maximum can be expressed in a very simple form involving the canonical correlation coefficients. This likelihood function and a choice of penalty functions constitute a number of new information theoretic criteria suitable for the determination of the number of signals in an unknown correlated noise environment. Furthermore, it is demonstrated that the new criteria are applicable in the case when only one sensor array is available     

An information theoretic approach to the distributed detectionproblem

The distributed detection problem is considered from an information-theoretic point of view. An entropy-based cost function is used for system optimization. This cost function maximizes the amount of information transfer between the input and the output. Distributed detection system topologies with and without a fusion center are considered, and an optimal fusion rule and optimal decision rules are derived     

An information theoretic foundation of synchronized detection

The constrained capacity of a coherent coded modulation (CM) digital communication system with data-aided channel estimation and a discrete, equiprobable symbol alphabet is derived under the assumption that the system operates on a flat fading channel and uses an interleaver to combat the bursty nature of the channel. It is shown that linear minimum mean square error channel estimation directly follows from the derivation and links average mutual information to the channel dynamics. Based on the assumption that known training symbols are transmitted, the achievable rate of the system is optimized with respect to the amount of training information needed. Furthermore, the results are compared to the additive white Gaussian noise channel, and the case when ideal channel state information is available at the receiver     

An information theoretic criterion for evaluating the quality of 3-D reconstructions from video

Even though numerous algorithms exist for estimating the three-dimensional (3-D) structure of a scene from its video, the solutions obtained are often of unacceptable quality. To overcome some of the deficiencies, many application systems rely on processing more data than necessary, thus raising the question: how is the accuracy of the solution related to the amount of data processed by the algorithm? Can we automatically recognize situations where the quality of the data is so bad that even a large number of additional observations will not yield the desired solution? Previous efforts to answer this question have used statistical measures like second order moments. They are useful if the estimate of the structure is unbiased and the higher order statistical effects are negligible, which is often not the case. This paper introduces an alternative information-theoretic criterion for evaluating the quality of a 3-D reconstruction. The accuracy of the reconstruction is judged by considering the change in mutual information (MI) (termed as the incremental MI) between a scene and its reconstructions. An example of 3-D reconstruction from a video sequence using optical flow equations and known noise distribution is considered and it is shown how the MI can be computed from first principles. We present simulations on both synthetic and real data to demonstrate the effectiveness of the proposed criterion.     

Control theoretic model for congestion pricing and impact of instability

A control theoretic model for single bit resource marking (SBRM), a TCP variant based on congestion pricing theory, is presented in this article. We also examine the impact of instability on networks and show that instability causes decreased utilization. From our results we conclude that SBRM is more efficient than TCP Reno with RED queues.     

Unsupervised image-set clustering using an information theoretic framework.

In this paper, we combine discrete and continuous image models with information-theoretic-based criteria for unsupervised hierarchical image-set clustering. The continuous image modeling is based on mixture of Gaussian densities. The unsupervised image-set clustering is based on a generalized version of a recently introduced information-theoretic principle, the information bottleneck principle. Images are clustered such that the mutual information between the clusters and the image content is maximally preserved. Experimental results demonstrate the performance of the proposed framework for image clustering on a large image set. Information theoretic tools are used to evaluate cluster quality. Particular emphasis is placed on the application of the clustering for efficient image search and retrieval.     

DTI segmentation using an information theoretic tensor dissimilarity measure

In recent years, diffusion tensor imaging (DTI) has become a popular in vivo diagnostic imaging technique in Radiological sciences. In order for this imaging technique to be more effective, proper image analysis techniques suited for analyzing these high dimensional data need to be developed. In this paper, we present a novel definition of tensor "distance" grounded in concepts from information theory and incorporate it in the segmentation of DTI. In a DTI, the symmetric positive definite (SPD) diffusion tensor at each voxel can be interpreted as the covariance matrix of a local Gaussian distribution. Thus, a natural measure of dissimilarity between SPD tensors would be the Kullback-Leibler (KL) divergence or its relative. We propose the square root of the J-divergence (symmetrized KL) between two Gaussian distributions corresponding to the diffusion tensors being compared and this leads to a novel closed form expression for the "distance" as well as the mean value of a DTI. Unlike the traditional Frobenius norm-based tensor distance, our "distance" is affine invariant, a desirable property in segmentation and many other applications. We then incorporate this new tensor "distance" in a region based active contour model for DTI segmentation. Synthetic and real data experiments are shown to depict the performance of the proposed model.     

Characterization of event related potentials using information theoretic distance measures

Analysis of event-related potentials (ERPs) using signal processing tools has become extremely widespread in recent years. Nonstationary signal processing tools such as wavelets and time-frequency distributions have proven to be especially effective in characterizing the transient phenomena encountered in event-related potentials. In this paper, we focus on the analysis of event-related potentials collected during a psychological experiment where two groups of subjects, spider phobics and snake phobics, are shown the same set of stimulus: A blank stimulus, a neutral stimulus and a spider stimulus. We introduce a new approach, based on time-frequency distributions, for analyzing the ERPs. The difference in brain activity before and after a stimulus is presented is quantified using distance measures as adapted to the time-frequency plane. Three different distance measures, including a new information theoretic distance measure, are applied on the time-frequency plane to discriminate between the responses of the two groups of subjects. The results illustrate the effectiveness of using distance measures combined with time-frequency distributions in differentiating between the two classes of subjects and the different regions of the brain.     

Segmenting object with ambiguous boundary using information theoretic rough sets

In image segmentation, image is divided into regions of similar pixels that satisfy a defined notion of similarity. The complexity of image segmentation is further increased when the separation between neighboring regions is ambiguous. In this paper, we propose an approach that uses the information theoretic rough sets concept (ITRS) to model the ambiguous boundary of the object for further segmentation. The advantage of this approach is incorporating the prior knowledge of the object for effective extraction despite its ambiguous boundary. This approach starts with an assumption that seed points of the regions are available. It then computes the probability of association of the pixels with the seed points. Rough sets theory is invoked on this probability or likelihood map to identify positive, negative, and boundary states of the object. Optimal threshold for the boundary region is determined using histogram based segmentation algorithm for final object extraction. The main contribution relies on the application of ITRS in categorizing the object by combining both the prior and image information. The proposed approach, ITRS segmentation, is compared with different image segmentation methods on simulated brain images, and the result is encouraging with its state-of-the-art performance.     

An information - theoretic proof of Hadamard's inequality (Corresp.)

Hadamard's inequality follows immediately from inspection of both sides of the entropy inequalityh(X_{1}, X_{2},, cdots, X_{n})leq sum h(X_{i}), when(X_{l}, X_{2},cdots, X_{n})is multivariate normal.     

A simple information theoretic proof of the maximum entropyproperty of some Gaussian random fields

A well known result of Burg (1967) and Kunsch (1981) identifies a Gaussian Markov random field with autocovariances specified on a finite part L of the n-dimensional integer lattice, as the random field with maximum entropy among all random fields with same autocovariance values on L. A simple information theoretic proof of a version of the maximum entropy theorem for random fields in n dimensions is presented in the special case that the given autocovariances are compatible with a unilateral autoregressive process.     

Software behavior trust forecast model based on check point scene information

In order to ensure the trustworthiness of software,and evaluate the trusted status of the software after running for a period of time by monitoring software behavior dynamically,a software behavior trust forecast model on checkpoint scene information which was called CBSI-TM was presented.The model set up a number of checkpoints in the software running track,and introduced the time increment of adjacent checkpoints,and the change of CPU utilization rate to define the scene information,and reflected the relationship between adjacent checkpoints scene information.Then the RBF neural network classifier evaluated the status of the current checkpoint to judge the trustworthiness of the software,and the semi weighted Markov model predicted the situation of the next checkpoint to evaluate the trustworthiness of future running trend of the software.The experimental results show that the CBSI-TM model can predict the future trusted status of the software effectively,and verify that the model is more reasonable and effective.     

Magnetic resonance image analysis by information theoretic criteriaand stochastic site models

Quantitative analysis of magnetic resonance (MR) images is a powerful tool for image-guided diagnosis, monitoring, and intervention. The major tasks involve tissue quantification and image segmentation where both the pixel and context images are considered. To extract clinically useful information from images that might be lacking in prior knowledge, the authors introduce an unsupervised tissue characterization algorithm that is both statistically principled and patient specific. The method uses adaptive standard finite normal mixture and inhomogeneous Markov random field models, whose parameters are estimated using expectation-maximization and relaxation labeling algorithms under information theoretic criteria. The authors demonstrate the successful applications of the approach with synthetic data sets and then with real MR brain images