Some Equivalences between Kernel Methods and Information Theoretic Methods

In this paper, we discuss some equivalences between two recently introduced statistical learning schemes, namely Mercer kernel methods and information theoretic methods. We show that Parzen window-based estimators for some information theoretic cost functions are also cost functions in a corresponding Mercer kernel space. The Mercer kernel is directly related to the Parzen window. Furthermore, we analyze a classification rule based on an information theoretic criterion, and show that this corresponds to a linear classifier in the kernel space. By introducing a weighted Parzen window density estimator, we also formulate the support vector machine in this information theoretic perspective.

---

     

Introduction to the Special Issue on Information Theoretic Security

The 26 papers and six items of correspondence in this special issue focus on information theoretic security. The papers and items of correspondence are summarized here.     

Tutorial: Key Elements for WDM Transport Networks

This paper reports a brief review of the key elements for WDM transport networks. In particular, the transmission aspects, the basic elements of WDM networking, and the issues relating to failure restoration and network management are taken into account.     

Information Theoretic Foundations of Adaptive Coded Modulation

In wireless/mobile communications, terminals adapt their rate and transmit power or, more in general, their coding and modulation scheme, depending on the time-varying channel conditions. This paper presents, in a tutorial form, the information theoretic framework underlying such ldquoadaptive modulationrdquo techniques. First, we review fading channel models, channel state information assumptions, and related capacity results. Then, we treat the case of input power constraint, where the optimal input distribution is Gaussian. Finally, we address the case of discrete modulations. In order to treat the latter, we make use of the recently developed method of ldquomercury-waterfillingrdquo, based on the relationship between mutual information and minimum mean-square error (MMSE) estimation of the channel input from the channel output. While the traditional design of adaptive modulation schemes based on uncoded bit-error rate (BER) involves the optimization over a discrete set of signal constellations, when powerful (i.e., capacity approaching) coding schemes are used the corresponding adaptive coded modulation design becomes surprisingly simple. The regime of very powerful coding is justified by the use of modern coding schemes, such as turbo codes and low-density parity-check codes, able to perform close to channel capacity at very small BER.     

Learning from Examples with Information Theoretic Criteria

This paper discusses a framework for learning based on information theoretic criteria. A novel algorithm based on Renyi's quadratic entropy is used to train, directly from a data set, linear or nonlinear mappers for entropy maximization or minimization. We provide an intriguing analogy between the computation and an information potential measuring the interactions among the data samples. We also propose two approximations to the Kulback-Leibler divergence based on quadratic distances (Cauchy-Schwartz inequality and Euclidean distance). These distances can still be computed using the information potential. We test the newly proposed distances in blind source separation (unsupervised learning) and in feature extraction for classification (supervised learning). In blind source separation our algorithm is capable of separating instantaneously mixed sources, and for classification the performance of our classifier is comparable to the support vector machines (SVMs).     

Introduction: WDM Transport Networks: Key Elements and Architectures

Photonic Network Communications -     

两类信息隐匿技术安全性的信息论分析方法

信息隐匿技术是信息安全研究的一个新的领域,它与信息加密有着显著的区别.数字签名中阈下信道是一种传送隐匿信息的安全信道,但每次传送的信息量很少.阈上信道是新近提出的一种在通信双方未共享任何密钥情况下的安全信道,利用阈上信道可以防止中间人攻击.阈上信息可做为公钥,用于保护后续的隐匿信息的传送.本文利用信息理论对这两类信息隐匿技术的安全性作了分析,得出了确保隐匿信息安全的条件.     

Information Theoretic Approach to Traffic Adaptive WDM Networks

WDM networks adapt to the changes in traffic by reconfiguring the virtual topology. Though reconfiguration is done with the objective of utilizing resources efficiently, the resulting disruption in traffic is a cause for concern. Hence, policies are formed to decide on the time (i.e., when) to trigger reconfiguration and the new virtual topology that is most beneficial to the network. We present a simple, general and flexible framework, based on the two conflicting objectives of efficient resource utilization and minimizing traffic disruption, to evaluate reconfiguration policies. Instead of re-determining the reconfiguration policy whenever the traffic changes, we present Incremental Clustering Algorithm (ICA) to pre-plan the reconfiguration policy for a fully predictable finite sequence of traffic matrices. Since full predictability of such a sequence is not possible in practice, we learn the traffic sequences in order to probabilistically predict the future ones. From an information theoretic point of view, we quantify the predictability of traffic sequences and the number of times the reconfiguration policy is re-determined for any WDM network. To optimally predict the future traffic sequences and to incur optimal cost in the re-determination of the reconfiguration policy, we propose Universal Reconfiguration Management System (URMS). A Prediction-based Incremental Clustering Algorithm (PICA) that extends ICA is used by URMS to predict the reconfiguration policy. Within URMS, the probabilities are assigned to the traffic sequences by the prediction schemes of LZ78. We performed extensive simulations to study the effectiveness and efficiency of URMS when compared to the fully predictable and totally unpredictable models. The performance of URMS improves with learning and nearly achieves the performance of a fully predictable model.     

信息理论密码学的新进展及研究问题

本文用信息讼观点对信息理论密码学的新进展及若干未解决的研究问题作一综述，其中也包括作者的某些研究工作。内容包括近年来在Ｓｈａｎｎｏｎ的密码系统和若干不同于Ｓｈａｎｎｏｎ模型的新密码系统方面基于信息论的研究结果和公开问题以及信息理论密码学今后发展的展望。     

电 子 学 报

本文介绍Shannon信息保密理论若干新进展.     

ATM中信元大小的分析

分析在ＡＴＭ技术中ＣＥＬＬ为什么是固定大小的５３字节。     

Information Theoretic Bounds on Authentication Systems in Query Model

Authentication codes provide message integrity guarantees in an information theoretic sense within a symmetric key setting. Information theoretic bounds on the success probability of an adversary who has access to previously authenticated messages have been derived by Simmons and Rosenbaum, among others. In this paper, we consider a strong attack scenario where the adversary is adaptive and has access to authentication and verification oracles. We derive information theoretic bounds on the success probability of the adversary and on the key size of the code. This brings the study of unconditionally secure authentication systems on a par with the study of computationally secure ones. We characterize the codes that meet these bounds and compare our result with the earlier ones.     

信息电视——利用有线电视网传送公众信息

分析了各种电子信息媒体（电脑、电话、广播电视）的特点、介绍了利用有线电视网传送大量公众信息的信息电视概念,信息电视的信息内容、准交互工的检索方式以及优质低价的功能特点,给出了信息电视系统的组成方案,并对信息电视的应用前景作了分析和预测。     

基于非对称信道状态信息的协作通信：一种契约理论建模方法

In relay-assisted cooperative com-munication, relay nodes help forwarding the information of a source node in case of link failure between the source and a destination. Although user cooperation improves the over-all efficiency of the network, it requires incen-tive to stimulate potential relay nodes to assist the source by forwarding its data to the desti-nation. Moreover, the potential relays are bet-ter informed than the source about their chan-nel conditions to destination, which results in asymmetric information between the source and the relays. In this paper, we study the problem of lack of forwarding incentive in cooperative communication when channel state information of relays is private infor-mation and not known by the source. To tackle this problem, we apply the principle of contract theory to a cooperative wireless system. Source first designs incentive compatible and individually rational contract, consisting of a set of power-credit pairs. Then it broadcasts contract items to nearby nodes. Once the source node receives reply messages from the volunteer relays, it chooses one or more relays based on its re-quirements and communication starts. Simulation results show how credit assignment works in order to stimulate relays to cooperate and prevents relays from cheating behavior.     

Information Theoretic Approach to Complexity Reduction of FIR Filter Design

This paper presents a new paradigm of design methodology to reduce the complexity of application-specific finite-impulse response (FIR) digital filters. A new adder graph data structure called the multiroot binary partition graph (MBPG) is proposed for the formulation of the multiple constant multiplication problem of FIR filter design. The set of coefficients in any fixed point representation is partitioned into symbols so that common subexpression identification and elimination become congruent to information parsing for data compression. A minimum number of different pairs or groups of symbols and residues can be used to code a set of coefficients based on their probability and conditional probability of occurrence. This ingenious concept enables the notion of entropy to be applied as a quantitative measure to evaluate the coding density of different compositions of symbols towards a set of coefficients. The minimal vertex set MBPG synthesized by our proposed information theoretic approach results in direct correspondences between the vertices and adders, and edges and physical interconnections. Unlike the common subexpression elimination algorithms based on other graph data structures, the symbol-level information carried in each vertex and the graph isomorphism of MBPG promise further fine-grain optimization in a reduced search space. One such optimization that has been exploited in this paper is the shift-inclusive computation reordering to minimize the width of every two's complement adder to further reduce the implementation cost and the critical path delay of the filter. Experiment results show that the proposed algorithm can contribute up to 19.30% reductions in logic complexity and up to 61.03% reduction in critical path delay over other minimization methods.      

隐高斯树的分层合成——一种基于信息论解决合成问题的方法

文章基于高斯分布提出一种信息论的启发式合成方法:根据给定的联合密度,研究产生随机向量的机制,从而引出(隐)高斯树结构;依赖于已知的树结构,构造分层的连续的合成方案,达到使用足够数量的公共随机变量来合成希望密度的研究目的。算法中所使用的公共随机源由树顶层的隐变量、独立可加性高斯噪声、以及表达变量之间模糊性的伯努利符号三部分组成。文章所提出的方法不但可以揭示数据隐含的内部联系,而且适用于在机器学习领域中产生仿真数据。     

对策论方法在信息网络资源分配中的应用

在研究复杂信息网络的资源分配问题时,网络资源提供者和用户之间对资源的使用存在不同的优化目标,导致对资源分配策略存在多种评价准则。为便于网络管理者进行分布式的资源管理和计费,采用对策论模型分析用户对资源的竞争和处理网络传输拥塞正日益受到广泛的重视。本文首先指出资源分配和计费管理的几个主要难点,比较了中心化与非中心化控制策略的差别,而后综述了采用对策论模型进行资源分配和管理的优点,重点讨论了在非中心化条件下对策均衡解与资源利用效率以及网络计费管理策略之间的关系,最后对基于对策论模型的信息网络流量控制与管理策略进行总体评述。