二态隐马尔可夫过程熵率的逼近算法

基于熵率上下界收敛性,该文提出了一个算法以计算二态隐马尔可夫过程的熵率.该算法能以任意精度逼近熵率的理论值,且可计算最大偏差.算法的复杂度的对数和误差的对数为线性关系,因此其计算代价是可以接受的.该算法为计算一般隐马尔可夫模型的熵率提供了一种新途径.     

Analyticity of Entropy Rate of Hidden Markov Chains

We prove that under mild positivity assumptions the entropy rate of a hidden Markov chain varies analytically as a function of the underlying Markov chain parameters. A general principle to determine the domain of analyticity is stated. An example is given to estimate the radius of convergence for the entropy rate. We then show that the positivity assumptions can be relaxed, and examples are given for the relaxed conditions. We study a special class of hidden Markov chains in more detail: binary hidden Markov chains with an unambiguous symbol, and we give necessary and sufficient conditions for analyticity of the entropy rate for this case. Finally, we show that under the positivity assumptions, the hidden Markov chain itself varies analytically, in a strong sense, as a function of the underlying Markov chain parameters     

Relative Entropy Rate Based Multiple Hidden Markov Model Approximation

This paper proposes a novel relative entropy rate (RER) based approach for multiple HMM (MHMM) approximation of a class of discrete-time uncertain processes. Under different uncertainty assumptions, the model design problem is posed either as a min-max optimisation problem or stochastic minimization problem on the RER between joint laws describing the state and output processes (rather than the more usual RER between output processes). A suitable filter is proposed for which performance results are established which bound conditional mean estimation performance and show that estimation performance improves as the RER is reduced. These filter consistency and convergence bounds are the first results characterizing multiple HMM approximation performance and suggest that joint RER concepts provide a useful model selection criteria. The proposed model design process and MHMM filter are demonstrated on an important image processing dim-target detection problem.     

On the Entropy Rate of Pattern Processes

We study the entropy rate of pattern sequences of stochastic processes, and its relationship to the entropy rate of the original process. We give a complete characterization of this relationship for independent and identically distributed (i.i.d.) processes over arbitrary alphabets, stationary ergodic processes over discrete alphabets, and a broad family of stationary ergodic processes over uncountable alphabets. For cases where the entropy rate of the pattern process is infinite, we characterize the possible growth rate of the block entropy     

Performance Analysis of Block Codes in Hidden Markov Channels

Most investigations on the effect of channel memory on the performance of block codes use a two-state Gilbert-Elliott (GE) model to describe the channel behavior. As there are circumstances that the channel of concern can not be properly described by the GE model, there are some recent works on coded performance that characterize the channel behavior by a general finite-state Markov chain. This letter presents a new efficient systematic approach to analyze the performance of block codes in such a hidden Markov channel (HMC). An application example is given to predict codeword error probability performance of an RS-coded system in a channel with memory. Numerical results are also provided to validate our analytic results.     

On Clutter Rank Observed by Arbitrary Arrays

This paper analyzes the rank and eigenspectrum of the clutter covariance matrix observed by space-time radar systems with arbitrarily configured arrays and varying look geometry. Motivated by recent applications that suggest use of nonuniform antenna arrays, a generalized theory of clutter rank is derived and demonstrated. First, a one-dimensional effective random process is defined by projecting the measurements obtained by an arbitrary space-time radar system into an equivalent one-dimensional sampling structure. Then, this projection and the Karhunen-Loeve representation of random processes are used to predict clutter rank based on effective aperture-bandwidth product. Simulated results are used to confirm the theory over a wide range of scenarios, and along the way, the well-known Brennan's rule for clutter rank is shown to be a special case of the proposed aperture-bandwidth product     

Discrete Memoryless Interference and Broadcast Channels With Confidential Messages: Secrecy Rate Regions

We study information-theoretic security for discrete memoryless interference and broadcast channels with independent confidential messages sent to two receivers. Confidential messages are transmitted to their respective receivers while ensuring mutual information-theoretic secrecy. That is, each receiver is kept in total ignorance with respect to the message intended for the other receiver. The secrecy level is measured by the equivocation rate at the eavesdropping receiver. In this paper, we present inner and outer bounds on secrecy capacity regions for these two communication systems. The derived outer bounds have an identical mutual information expression that applies to both channel models. The difference is in the input distributions over which the expression is optimized. The inner bound rate regions are achieved by random binning techniques. For the broadcast channel, a double-binning coding scheme allows for both joint encoding and preserving of confidentiality. Furthermore, we show that, for a special case of the interference channel, referred to as the switch channel, derived bounds meet. Finally, we describe several transmission schemes for Gaussian interference channels and derive their achievable rate regions while ensuring mutual information-theoretic secrecy. An encoding scheme in which transmitters dedicate some of their power to create artificial noise is proposed and shown to outperform both time-sharing and simple multiplexed transmission of the confidential messages.     

Optimal Evaluation of Discrete Continuous Markov Processes from Observed Digital Signals

Journal of Communications Technology and Electronics - Abstract—On the basis of the theory of conditional Markov processes, recursive optimal and quasi-optimal algorithms for estimating...     

Arbitrary Frame Rate Transcoding Through Temporal and Spatial Complexity

In this paper, an arbitrary frame rate transcoding joint considering temporal and spatial complexity of frames in the adaptive length sliding window is proposed. The length of a sliding window can be adjusted according to bandwidth variation in order to decide the number of skipped frames. The proposed method preserves significant frames and drops non-significant ones using the complexity measurements. Moreover, the motion vector composition algorithm is proposed to reduce the computations of motion estimation process by adopting the coding feature of variable block sizes in H.264/AVC video transcoder. Experimental results show that the proposed method achieves higher visual quality compared to other existing methods. After combining with the proposed fast motion composition algorithm, our proposed algorithm reduces encoding time significantly with slight visual quality degradation.      

二指标隐马尔可夫过程及其在汉字识别中的应用

本文给出了二指标隐马尔可夫过程的定义、基本问题及解法,并提出一种利用二指标隐马尔可夫过程进行汉字识别的方法。     

基于最大熵的隐马尔可夫模型文本信息抽取

文本信息抽取是处理海量文本的重要手段之一.最大熵模型提供了一种自然语言处理的方法.提出了一种基于最大熵的隐马尔可夫模型文本信息抽取算法.该算法结合最大熵模型在处理规则知识上的优势,以及隐马尔可夫模型在序列处理和统计学习上的技术基础,将每个观察文本单元所有特征的加权之和用来调整隐马尔可夫模型中的转移概率参数,实现文本信息抽取.实验结果表明,新的算法在精确度和召回率指标上比简单隐马尔可夫模型具有更好的性能.     

On Information Rates of Time-Varying Fading Channels Modeled as Finite-State Markov Channels

We study information rates of time-varying flatfading channels (FFC) modeled as finite-state Markov channels (FSMC). FSMCs have two main applications for FFCs: modeling channel error bursts and decoding at the receiver. Our main finding in the first application is that receiver observation noise can more adversely affect higher-order FSMCs than lower-order FSMCs, resulting in lower capacities. This is despite the fact that the underlying higher-order FFC and its corresponding FSMC are more predictable. Numerical analysis shows that at low to medium SNR conditions (SNR ≲12 dB) and at medium to fast normalized fading rates (0.01 ≲fDT ≲0.10), FSMC information rates are non-increasing functions of memory order. We conclude that BERs obtained by low-order FSMC modeling can provide optimistic results. To explain the capacity behavior, we present a methodology that enables analytical comparison of FSMC capacities with different memory orders. We establish sufficient conditions that predict higher/lower capacity of a reduced-order FSMC, compared to its original high-order FSMC counterpart. Finally, we investigate the achievable information rates in FSMC-based receivers for FFCs. We observe that high-order FSMC modeling at the receiver side results in a negligible information rate increase for normalized fading rates fDT ≲0.01.     

On Analytic Properties of Entropy Rate

Entropy rate is a real valued functional on the space of discrete random sources for which it exists. However, it lacks existence proofs and/or closed formulas even for classes of random sources which have intuitive parameterizations. A good way to overcome this problem is to examine its analytic properties relative to some reasonable topology. A canonical choice of a topology is that of the norm of total variation as it immediately arises with the idea of a discrete random source as a probability measure on sequence space. It is shown that both upper and lower entropy rate, hence entropy rate itself if it exists, are Lipschitzian relative to this topology, which, by well known facts, is close to differentiability. An application of this theorem leads to a simple and elementary proof of the existence of entropy rate of random sources with finite evolution dimension. This class of sources encompasses arbitrary hidden Markov sources and quantum random walks.     

Outage Behavior of Discrete Memoryless Channels Under Channel Estimation Errors

Communication systems are usually designed by assuming perfect channel state information (CSI). However, in many practical scenarios, only a noisy estimate of the channel is available, which may strongly differ from the true channel. This imperfect CSI scenario is addressed by introducing the notion of estimation-induced outage (EIO). We derive a single-letter characterization of the maximal EIO rate and prove an associated coding theorem and its strong converse for discrete memoryless channels (DMCs). The transmitter and the receiver rely on the channel estimate and the statistics of the estimate to construct codes that guarantee reliable communication with a certain outage probability. This ensures that in the non-outage case the transmission meets the target rate with small error probability, irrespective of the quality of the channel estimate. Applications of the EIO capacity to a single-antenna (nonergodic) Ricean fading channel are considered. The EIO capacity for this case is compared to the EIO rates of a communication system in which the receiver decodes by using a mismatched maximum-likelihood (ML) decoder. The effects of rate-limited feedback to provide the transmitter with quantized CSI are also investigated.      

On Designing Good LDPC Codes for Markov Channels

This paper presents a reduced-complexity approximate density evolution (DE) scheme for low-density parity-check (LDPC) codes in channels with memory in the form of a hidden Markov chain. This approximation is used to design degree sequences representing some of the best known LDPC code ensembles for the Gilbert-Elliott channel, and example optimizations are also given for other Markov channels. The problem of approximating the channel estimation is addressed by obtaining a specially constructed message-passing schedule in which the channel messages all approach their stable densities. It is shown that this new schedule is much easier to approximate than the standard schedule, but has the same ultimate performance in the limits of long block length and many decoding iterations. This result is extended to show that all message-passing schedules that satisfy mild conditions will have the same threshold under density evolution     

Upper Bounds on the Rate of LDPC Codes for a Class of Finite-State Markov Channels

In this correspondence, we consider the class of finite-state Markov channels (FSMCs) in which the channel behaves as a binary symmetric channel (BSC) in each state. Upper bounds on the rate of LDPC codes for reliable communication over this class of FSMCs are found. A simple upper bound for all noninverting FSMCs is first derived. Subsequently, tighter bounds are derived for the special case of Gilbert-Elliott (GE) channels. Tighter bounds are also derived over the class of FSMCs considered. The latter bounds hold almost-surely for any sequence of randomly constructed LDPC codes of given degree distributions. Since the bounds are derived for optimal maximum-likelihood decoding, they also hold for belief propagation decoding. Using the derivations of the bounds on the rate, some lower bounds on the density of parity check matrices for given performance over FSMCs are derived     

The Capacity of Finite Abelian Group Codes Over Symmetric Memoryless Channels

The capacity of finite Abelian group codes over symmetric memoryless channels is determined. For certain important examples, such as m -PSK constellations over additive white Gaussian noise (AWGN) channels, with m a prime power, it is shown that this capacity coincides with the Shannon capacity; i.e., there is no loss in capacity using group codes. (This had previously been known for binary-linear codes used over binary-input output-symmetric memoryless channels.) On the other hand, a counterexample involving a three-dimensional geometrically uniform constellation is presented in which the use of Abelian group codes leads to a loss in capacity. The error exponent of the average group code is determined, and it is shown to be bounded away from the random-coding error exponent, at low rates, for finite Abelian groups not admitting Galois field structure.     

隐马尔可夫模型演化下的隐组查询

针对社会网络图中的隐组查询问题,提出了一种基于隐马尔科夫模型演化的方法。不同于传统方法,文中首先对“微观法则”提出了一些合理的假设,这些法则决定了在某时刻一个个体是否存在于一个特定群体。通过这些假设,可以得到社会个体和群体的动态演化。然后根据群体演化,找出长时间保持通信的群体作为潜在的隐组,再通过进一步分析,确保这些潜在的隐组以一个较高的概率成为理想的结果。为验证算法的有效性,文中分别对模拟和真实的数据进行了测试。     

On the Sum Rate of Downlink NOMA Over Fading Channels

Wireless Personal Communications - Non-orthogonal multiple access (NOMA) is an essential choice for 5G techniques. Especially, the ergodic capacity of NOMA systems is studied extensively to date....     

On the Achievable Rate for Wideband Channels with Estimated CSI

We consider the achievable rate for frequency-selective fading channels when the channel state information (CSI) is to be estimated at the receiver. Since the estimated CSI is not perfect, the achievable rate must be degraded from that with perfect CSI. Using the rate-distortion theory, we study an upper bound on the achievable rate and investigate how the achievable rate can be maximized through an optimization problem by allocating the resources such as degrees of freedom (the transmission time in our work or transmission power) for the exploration of CSI (i.e., the channel estimation) using pilot symbols, and the exploitation of channels to transmit data symbols. Although our study is based on some ideal assumptions, the results could help develop flexible communication systems such as software defined radio (SDR) to achieve a best performance by optimizing radio resources for unknown channels.