Entropy inequalities for discrete channels

The sharp lower boundf(x)on the per-symbol output entropy for a given per-symbol input entropyxis determined for stationary discrete memoryless channels; it is the lower convex envelope of the boundg(x)for a single channel use. The bounds agree for all noiseless channels and all binary channels. However, for nonbinary channels,gis not generally convex so that the bounds differ. Such is the case for the Hamming channels that generalize the binary symmetric channels. The bounds are of interest in connection with multiple-user communication, as exemplified by Wyner's applications of "Mrs. Gerber's lemma" (the bound for binary symmetric channels first obtained by Wyner and Ziv). These applications extend from the binary symmetric case to the. Hamming case. Doubly stochastic channels are characterized by the property of never decreasing entropy.     

Reliability bounds for dependent failures

Upper and lower bounds are obtained for the reliability of a series system with statistically dependent component failures. The method can be applied to series subsystems when these are obtained by decomposing a complex system. Numerical examples illustrate the use of the method     

On classifications of multivariate life distributions in the discrete setup

Attempts have been made to define and study discrete IFR, DFR, IFRA, DFRA, NBU and NWU classes in the multivariate set-up, taking into consideration the specialties of discrete ageing. A few characterizations of a bivariate geometric distribution have been obtained.     

Information-theoretic inequalities for contoured probability distributions

We show that for a special class of probability distributions that we call contoured distributions, information-theoretic invariants and inequalities are equivalent to geometric invariants and inequalities of bodies in Euclidean space associated with the distributions. Using this, we obtain characterizations of contoured distributions with extremal Shannon and Renyi entropy. We also obtain a new reverse information-theoretic inequality for contoured distributions.     

Reliability bounds and critical time for the Birnbaum-Saundersdistribution

The Birnbaum-Saunders distribution, under certain conditions, can be used to model the fatigue failure-time caused by the catastrophic crack size. Reliability bounds for the Birnbaum-Saunders distribution and point and interval estimates for the critical time of the failure (hazard)-rate are discussed. The confidence intervals are constructed under the assumption that both parameters are unknown. This method is not affected by censoring, as long as confidence intervals for the parameters can be established. Numerical examples illustrate the procedure     

Improved multicovering bounds from linear inequalities and supercodes

The multicovering radii of a code are natural generalizations of the covering radius in which the goal is to cover all m-tuples of vectors for some m as cheaply as possible. In this correspondence, we describe several techniques for obtaining lower bounds on the sizes of codes achieving a given multicovering radius. Our main method is a generalization of the method of linear inequalities based on refined weight distributions of the code. We also obtain a linear upper bound on the 2-covering radius. We further study bounds on the sizes of codes with a given multicovering radius that are subcodes of a fixed code. We find, for example, constraints on parity checks for codes with small ordinary covering radius.     

Approximating discrete probability distributions

The method of minimum discrimination information estimation is applied to the problem of estimating ann-dimensional discrete probability distribution in terms of lower order marginal distributions. The procedure provides a convergent iterative algorithm. The method yields regular best asymptotically normal (RBAN) estimates. The general procedure includes as a particular case that proposed by a method using dependence trees. An example is given.     

On sampling continuous aperture distributions for discrete planararrays

The theory of sampling continuous distributions for the purpose of determining planar array excitation coefficients is examined. The cause of pattern degradation associated with conventional sampling is identified, and a new method called integrated sampling is introduced which provides superior results. Implementation details are given for the specific case of a rectangular grid planar array with a roughly circular boundary. Numerical examples are provided for both uniform and Taylor distributions in order to perform comparative studies and to demonstrate the advantages of the new technique     

Hazard rates of nonequispaced discrete distributions

The author proposes a definition of hazard rate of nonequispaced discrete distributions. He presents the relation of the hazard rate of the discrete distribution, obtained by condensing a continuous-life distribution, to the hazard rate of the continuous-life-distribution     

Approximating discrete probability distributions with dependence trees

A method is presented to approximate optimally ann-dimensional discrete probability distribution by a product of second-order distributions, or the distribution of the first-order tree dependence. The problem is to find an optimum set ofn - 1first order dependence relationship among thenvariables. It is shown that the procedure derived in this paper yields an approximation of a minimum difference in information. It is further shown that when this procedure is applied to empirical observations from an unknown distribution of tree dependence, the procedure is the maximum-likelihood estimate of the distribution.     

Shift covariant time-frequency distributions of discrete signals

Many commonly used time-frequency distributions are members of the Cohen (1989) class. This class is defined for continuous signals, and since time-frequency distributions in the Cohen class are quadratic, the formulation for discrete signals is not straightforward. The Cohen class can be derived as the class of all quadratic time-frequency distributions that are covariant to time shifts and frequency shifts. We extend this method to three types of discrete signals to derive what we call the discrete Cohen classes. The properties of the discrete Cohen classes differ from those of the original Cohen class. To illustrate these properties, we also provide explicit relationships between the classical Wigner distribution and the discrete Cohen classes     

Generation of discrete distributions from biased coins

The procedure of Knuth and Yao (1976) to simulate random numbers with specified distribution by parsing sequences of fair coin tosses is generalized to employ discrete distributions of particular form instead of fair coins. Each probability in these distributions is an integral power of some fixed value t. The parse tree for the simulation procedure is closely related to the code trees arising in Karp's optimal variable-length coding algorithm for code symbols of unequal cost     

Reliability analysis of complex models using SURE bounds

As computer and communication systems become more complex it becomes increasingly more difficult to analyze their hardware reliability, because simple models can fail to adequately-capture subtle but important features. This paper describes several ways the authors have addressed this problem for analyses based upon White's SURE theorem. They show: how reliability analysis based on SURE mathematics can attack very large problems by accepting recomputation in order to reduce memory usage; how such analysis can be parallelized both on multiprocessors and on networks of ordinary workstations, and obtain excellent performance gains by doing so; how the SURE theorem supports efficient Monte Carlo based estimation of reliability; and the advantages of the method. Empirical studies of large models solved using these methods show that they are effective in reducing the solution-time of large complex problems     

New bounds on the mutual information for discrete constellations and application to wireless channel estimation

The lack of closed-form expressions of the mutual information for discrete constellations has limited its uses for analyzing reliable communication over wireless fading channels. In order to address this issue, this paper proposes analytically-tractable lower bounds on the mutual information based on Arithmetic-Mean-Geometric-Mean (AM-GM) inequality. The new bounds can apply to a wide range of discrete constellations and reveal some insights into the rate behavior at moderate to high Signal-to-Noise Ratio(SNR) values. The usability of the bounds is further demonstrated to approximate the optimum pilot overhead in stationary fading channels.     

Error bounds on stochastic approximates in discrete nonlinear dynamical systems

A mean-square error bound associated with gains utilised in stochastic approximation procedures is derived here. A specific bound is obtained for the case when the forcing function is a white noise process and the gain is specified.     

New fast method for generating discrete random numbers with arbitrary frequency distributions

A new method for generating discrete random numbers with arbitrary amplitude/frequency distributions is presented. It consists essentially of amplitude manipulation of uniformly distributed, statistically independent sequential numbers. A digital-hardware implementation of the system features fast single-clock operation, with good statistical properties, and is well suited to software implementation.     

Reliability bounds for some static system models using lifetime data on their components

It has become increasingly difficult to obtain a sampling distribution of the reliability function of a system. Therefore, reliability bounds have been studied to facilitate the analysis of reliability characteristics. The bounds for the reliability of the system depend on the mean and variance of the lifetimes recorded on the system. However, in the early stages of design, the lifetime data on a system may either be costly or non-existent. Thus, by using lifetime data on the components of the system, the present study deals with the analysis of reliability bounds for some static system models.     

Random ciphering bounds on a class of secrecy systems and discrete message sources

The problem of enciphering a stationary finite discrete message so that a cryptanalyst is unlikely to decrypt an intercepted cryptogram is considered. Additive-like instantaneous block (ALIB) encipherers are studied that employ a list ofe^{nr}keywords of lengthn, called the cipher. An ALIB encipherer produces a cryptogram word of lengthnfrom a message word and a key word of the same length by combining corresponding message letters and key-word letters. Certain technical restrictions sure placed on the combining function. The decipherer uses a decoder which combines a letter from the key word used in enciphering with a letter from the cryptogram to form a letter of the decoded message. cryptanalyst also decodes letter by letter with an identical decoder; however, he uses a keyword that is not necessarily that used in enciphering. For a given message source and combiner, the design of the cipher consists in choosing the block lengthn, the key rater, and the set ofe^{nr}key words. These are to be chosen so thatp_{w}, the probability of correct decryptment of the message word, andp( Delta), the probability that the per letter nonzero Hamming distance between the decrypted message and the true message is smaller thanDelta, are very small for every cryptanalyst. A set of pairs( Delta,r)for which there exist ciphers with key ratersuch that,p_{w}andp( Delta)can be made arbitrarily small for every cryptanalyst is determined using the concepts of random ciphering and exponential bounding.     

Mean residual life of lifetime distributions

This paper characterizes the general behaviors of the MRL (mean residual lives) for both continuous and discrete lifetime distributions, with respect to their failure rates. For the continuous lifetime distribution with failure rates with only one or two change-points, the characteristic of the MRL depends only on its mean and failure rate at time zero. For failure rates with “roller coaster” behavior, the subsequent behavior of the MRL depends on its MRL and failure-rates at the change points. Using the characterization, their behaviors for the: Weibull; lognormal; Birnbaum-Saunders; inverse Gaussian; and bathtub failure rate distributions are tabulated in terms of their shape parameters. For discrete lifetime distributions, for upside-down bathtub failure rate with only one change point, the characteristic of the MRL depends only on its mean and the probability mass function at time zero