The empirical distribution of rate-constrained source codes

Let X = (X/sub 1/,...) be a stationary ergodic finite-alphabet source, X/sup n/ denote its first n symbols, and Y/sup n/ be the codeword assigned to X/sup n/ by a lossy source code. The empirical kth-order joint distribution Q/spl circ//sup k/[X/sup n/,Y/sup n//spl rceil/(x/sup k/,y/sup k/) is defined as the frequency of appearances of pairs of k-strings (x/sup k/,y/sup k/) along the pair (X/sup n/,Y/sup n/). Our main interest is in the sample behavior of this (random) distribution. Letting I(Q/sup k/) denote the mutual information I(X/sup k/;Y/sup k/) when (X/sup k/,Y/sup k/)/spl sim/Q/sup k/ we show that for any (sequence of) lossy source code(s) of rate /spl les/R lim sup/sub n/spl rarr//spl infin//(1/k)I(Q/spl circ//sup k/[X/sup n/,Y/sup n//spl rfloor/) /spl les/R+(1/k)H (X/sub 1//sup k/)-H~(X) a.s. where H~(X) denotes the entropy rate of X. This is shown to imply, for a large class of sources including all independent and identically distributed (i.i.d.). sources and all sources satisfying the Shannon lower bound with equality, that for any sequence of codes which is good in the sense of asymptotically attaining a point on the rate distortion curve Q/spl circ//sup k/[X/sup n/,Y/sup n//spl rfloor//spl rArr//sup d/P(X/sup k/,Y~/sup k/) a.s. whenever P(/sub X//sup k//sub ,Y//sup k/) is the unique distribution attaining the minimum in the definition of the kth-order rate distortion function. Consequences of these results include a new proof of Kieffer's sample converse to lossy source coding, as well as performance bounds for compression-based denoisers.     

A new binary sequence family with low correlation and large size

For odd n=2l+1 and an integer /spl rho/ with 1/spl les//spl rho//spl les/l, a new family S/sub o/(/spl rho/) of binary sequences of period 2/sup n/-1 is constructed. For a given /spl rho/, S/sub o/(/spl rho/) has maximum correlation 1+2/sup n+2/spl rho/-1/2/, family size 2/sup n/spl rho//, and maximum linear span n(n+1)/2. Similarly, a new family of S/sub e/(/spl rho/) of binary sequences of period 2/sup n/-1 is also presented for even n=2l and an integer /spl rho/ with 1/spl les//spl rho/相似文献   

Compression mappings on primitive sequences over Z/(p/sup e/)

Let Z/(p/sup e/) be the integer residue ring with odd prime p/spl ges/5 and integer e/spl ges/2. For a sequence a_ over Z/(p/sup e/), there is a unique p-adic expansion a_=a_/sub 0/+a_/spl middot/p+...+a_/sub e-1//spl middot/p/sup e-1/, where each a_/sub i/ is a sequence over {0,1,...,p-1}, and can be regarded as a sequence over the finite field GF(p) naturally. Let f(x) be a primitive polynomial over Z/(p/sup e/), and G'(f(x),p/sup e/) the set of all primitive sequences generated by f(x) over Z/(p/sup e/). Set /spl phi//sub e-1/ (x/sub 0/,...,x/sub e-1/) = x/sub e-1//sup k/ + /spl eta//sub e-2,1/(x/sub 0/, x/sub 1/,...,x/sub e-2/) /spl psi//sub e-1/(x/sub 0/,...,x/sub e-1/) = x/sub e-1//sup k/ + /spl eta//sub e-2,2/(x/sub 0/,x/sub 1/,...,x/sub e-2/) where /spl eta//sub e-2,1/ and /spl eta//sub e-2,2/ are arbitrary functions of e-1 variables over GF(p) and 2/spl les/k/spl les/p-1. Then the compression mapping /spl phi//sub e-1/:{G'(f(x),p/sup e/) /spl rarr/ GF(p)/sup /spl infin// a_ /spl rarr/ /spl phi//sub e-1/(a_/sub 0/,...,a_/sub e-1/) is injective, that is, a_ = b_ if and only if /spl phi//sub e-1/(a_/sub 0/,...,a_/sub e-1/) = /spl phi//sub e-1/(b_/sub 0/,...,b_/sub e-1/) for a_,b_ /spl isin/ G'(f(x),p/sup e/). Furthermore, if f(x) is a strongly primitive polynomial over Z/(p/sup e/), then /spl phi//sub e-1/(a_/sub 0/,...,a_/sub e-1/) = /spl psi//sub e-1/(b_/sub 0/,...,b_/sub e-1/) if and only if a_ = b_ and /spl phi//sub e-1/(x/sub 0/,...,x/sub e-1/) = /spl psi//sub e-1/(x/sub 0/,...,x/sub e-1/) for a_,b_ /spl isin/ G'(f(x),p/sup e/).     

Improving the Gilbert-Varshamov bound for q-ary codes

Given positive integers q,n, and d, denote by A/sub q/(n,d) the maximum size of a q-ary code of length n and minimum distance d. The famous Gilbert-Varshamov bound asserts that A/sub q/(n,d+1)/spl ges/q/sup n//V/sub q/(n,d) where V/sub q/(n,d)=/spl Sigma//sub i=0//sup d/ (/sub i//sup n/)(q-1)/sup i/ is the volume of a q-ary sphere of radius d. Extending a recent work of Jiang and Vardy on binary codes, we show that for any positive constant /spl alpha/ less than (q-1)/q there is a positive constant c such that for d/spl les//spl alpha/n A/sub q/(n,d+1)/spl ges/cq/sup n//V/sub q/(n,d)n. This confirms a conjecture by Jiang and Vardy.     

Asymptotic improvement of the Gilbert-Varshamov bound on the size of binary codes

Given positive integers n and d, let A/sub 2/(n,d) denote the maximum size of a binary code of length n and minimum distance d. The well-known Gilbert-Varshamov bound asserts that A/sub 2/(n,d)/spl ges/2/sup n//V(n,d-l), where V(n,d) = /spl sigma//sub i=0//sup d/(/sub i//sup n/) is the volume of a Hamming sphere of radius d. We show that, in fact, there exists a positive constant c such that A/sub 2/(n, d)/spl ges/c2/sup n//V(n,d-1)log/sub 2/V(n, d-1) whenever d/n/spl les/0.499. The result follows by recasting the Gilbert-Varshamov bound into a graph-theoretic framework and using the fact that the corresponding graph is locally sparse. Generalizations and extensions of this result are briefly discussed.     

Optimal linear identifying codes

Identifying codes can be used to locate malfunctioning processors. We say that a code C of length n is a linear (1,/spl les/l)-identifying code if it is a subspace of F/sub 2//sup n/ and for all X,Y/spl sube/F/sub 2//sup n/ such that |X|, |Y|/spl les/l and X/spl ne/Y, we have /spl cup//sub x/spl isin/X/(B(x)/spl cap/C)/spl ne//spl cup/y/spl isin/Y(B(y)/spl cap/C). Strongly (1,/spl les/l)-identifying codes are a variant of identifying codes. We determine the cardinalities of optimal linear (1,/spl les/l)-identifying and strongly (1,/spl les/l)-identifying codes in Hamming spaces of any dimension for locating any at most l malfunctioning processors.     

Lossless and near-lossless source coding for multiple access networks

A multiple access source code (MASC) is a source code designed for the following network configuration: a pair of correlated information sequences {X/sub i/}/sub i=1//sup /spl infin// and {Y/sub i/}/sub i=1//sup /spl infin// is drawn independent and identically distributed (i.i.d.) according to joint probability mass function (p.m.f.) p(x,y); the encoder for each source operates without knowledge of the other source; the decoder jointly decodes the encoded bit streams from both sources. The work of Slepian and Wolf describes all rates achievable by MASCs of infinite coding dimension (n/spl rarr//spl infin/) and asymptotically negligible error probabilities (P/sub e//sup (n)//spl rarr/0). In this paper, we consider the properties of optimal instantaneous MASCs with finite coding dimension (n相似文献   

Characteristics of Inhomogeneous Broadside-Coupled Striplines

A variational method for the analysis of inhomogeneous broadside-coupled striplines is described. The data for even- and odd-mode characteristic impedances, effective dielectric constants, and mode phase velocity ratios are presented. It is found that the phase velocity ratio may be varied over the range 1.14 /spl les/ ( V/sub e/ / V/sub 0/) /spl les/ 3.6 for broadside-coupled suspended microstrip lines (BSML) and 0.36 /spl les/ ( V/sub e/ / V/sub 0/) /spl les/ 0.93 for broadside-coupled inverted microstrip lines (BIML) using materials with dielectric constant less than 16 and S/b /spl ges/ 0.05, W/b /spl les/ 2.0. The effect of nonzero strip thickness is also calculated. It is noticed that the effect of thickness is more pronounced for the odd-mode case than for the even mode. Losses are obtained using the incremental inductance rule of Wheeler. The odd-mode attenuation constant is always higher than the even-mode value.     

Decoding by linear programming

This paper considers a natural error correcting problem with real valued input/output. We wish to recover an input vector f/spl isin/R/sup n/ from corrupted measurements y=Af+e. Here, A is an m by n (coding) matrix and e is an arbitrary and unknown vector of errors. Is it possible to recover f exactly from the data y? We prove that under suitable conditions on the coding matrix A, the input f is the unique solution to the /spl lscr//sub 1/-minimization problem (/spl par/x/spl par//sub /spl lscr/1/:=/spl Sigma//sub i/|x/sub i/|) min(g/spl isin/R/sup n/) /spl par/y - Ag/spl par//sub /spl lscr/1/ provided that the support of the vector of errors is not too large, /spl par/e/spl par//sub /spl lscr/0/:=|{i:e/sub i/ /spl ne/ 0}|/spl les//spl rho//spl middot/m for some /spl rho/>0. In short, f can be recovered exactly by solving a simple convex optimization problem (which one can recast as a linear program). In addition, numerical experiments suggest that this recovery procedure works unreasonably well; f is recovered exactly even in situations where a significant fraction of the output is corrupted. This work is related to the problem of finding sparse solutions to vastly underdetermined systems of linear equations. There are also significant connections with the problem of recovering signals from highly incomplete measurements. In fact, the results introduced in this paper improve on our earlier work. Finally, underlying the success of /spl lscr//sub 1/ is a crucial property we call the uniform uncertainty principle that we shall describe in detail.     

Cyclic codes over GR(4/sup m/) which are also cyclic over /spl Zopf//sub 4/

Let GR(4/sup m/) be the Galois ring of characteristic 4 and cardinality 4/sup m/, and /spl alpha/_={/spl alpha//sub 0/,/spl alpha//sub 1/,...,/spl alpha//sub m-1/} be a basis of GR(4/sup m/) over /spl Zopf//sub 4/ when we regard GR(4/sup m/) as a free /spl Zopf//sub 4/-module of rank m. Define the map d/sub /spl alpha/_/ from GR(4/sup m/)[z]/(z/sup n/-1) into /spl Zopf//sub 4/[z]/(z/sup mn/-1) by d/spl alpha/_(a(z))=/spl Sigma//sub i=0//sup m-1//spl Sigma//sub j=0//sup n-1/a/sub ij/z/sup mj+i/ where a(z)=/spl Sigma//sub j=0//sup n-1/a/sub j/z/sup j/ and a/sub j/=/spl Sigma//sub i=0//sup m-1/a/sub ij//spl alpha//sub i/, a/sub ij//spl isin//spl Zopf//sub 4/. Then, for any linear code C of length n over GR(4/sup m/), its image d/sub /spl alpha/_/(C) is a /spl Zopf//sub 4/-linear code of length mn. In this article, for n and m being odd integers, it is determined all pairs (/spl alpha/_,C) such that d/sub /spl alpha/_/(C) is /spl Zopf//sub 4/-cyclic, where /spl alpha/_ is a basis of GR(4/sup m/) over /spl Zopf//sub 4/, and C is a cyclic code of length n over GR(4/sup m/).     

On the filtering problem for stationary random /spl Zopf//sup 2/-fields

It is shown that whenever a stationary random field (Z/sub n,m/)/sub n,m/spl isin/z/ is given by a Borel function f:/spl Ropf//sup z/ /spl times/ /spl Ropf//sup z/ /spl rarr/ /spl Ropf/ of two stationary processes (X/sub n/)/sub n/spl isin/z/ and (Y/sub m/)/sub m/spl isin/z/ i.e., then (Z/sub n, m/) = (f((X/sub n+k/)/sub k/spl epsi/z/, (Y/sub m + /spl lscr// )/sub /spl lscr/ /spl epsi/z/)) under a mild first coordinate univalence assumption on f, the process (X/sub n/)/sub n/spl isin/z/ is measurable with respect to (Z/sub n,m/)/sub n,m/spl epsi/z/ whenever the process (Y/sub m/)/sub m/spl isin/z/ is ergodic. The notion of universal filtering property of an ergodic stationary process is introduced, and then using ergodic theory methods it is shown that an ergodic stationary process has this property if and only if the centralizer of the dynamical system canonically associated with the process does not contain a nontrivial compact subgroup.     

List decoding of q-ary Reed-Muller codes

The q-ary Reed-Muller (RM) codes RM/sub q/(u,m) of length n=q/sup m/ are a generalization of Reed-Solomon (RS) codes, which use polynomials in m variables to encode messages through functional encoding. Using an idea of reducing the multivariate case to the univariate case, randomized list-decoding algorithms for RM codes were given in and . The algorithm in Sudan et al. (1999) is an improvement of the algorithm in , it is applicable to codes RM/sub q/(u,m) with u相似文献   

Low-Temperature Materials and Thin Film Transistors for Flexible Electronics

This paper addresses the low-temperature deposition processes and electronic properties of silicon based thin film semiconductors and dielectrics to enable the fabrication of mechanically flexible electronic devices on plastic substrates. Device quality amorphous hydrogenated silicon (a-Si:H), nanocrystalline silicon (nc-Si), and amorphous silicon nitride (a-SiN/sub x/) films and thin film transistors (TFTs) were made using existing industrial plasma deposition equipment at the process temperatures as low as 75/spl deg/C and 120/spl deg/C. The a-Si:H TFTs fabricated at 120/spl deg/C demonstrate performance similar to their high-temperature counterparts, including the field effect mobility (/spl mu//sub FE/) of 0.8 cm/sup 2/V/sup -1/s/sup -1/, the threshold voltage (V/sub T/) of 4.5 V, and the subthreshold slope of 0.5 V/dec, and can be used in active matrix (AM) displays including organic light emitting diode (OLED) displays. The a-Si:H TFTs fabricated at 75/spl deg/C exhibit /spl mu//sub FE/ of 0.6 cm/sup 2/V/sup -1/s/sup -1/, and V/sub T/ of 4 V. It is shown that further improvement in TFT performance can be achieved by using n/sup +/ nc-Si contact layers and plasma treatments of the interface between the gate dielectric and the channel layer. The results demonstrate that with appropriate process optimization, the large area thin film Si technology suits well the fabrication of electronic devices on low-cost plastic substrates.     

Mode Propagation in a Nonuniform Cylindrical Medium (Correspondence)

A simplified form of the coupling coefficient C(/spl beta//sub p/, /spl beta//sub q/) resulting from a coupled mode theory analysis of wave propagation in a nonuniform medium is derived. It is found for most situations of interest that C(/spl beta//sub p/, /spl beta//sub q/) is proportional to 1/(/spl beta//sub p/-/spl beta//sub q/) and the power transfer between two modes is proportional to 1/(/spl beta//sub p/ - /spl beta//sub q/)/sup 4/. /spl beta//sub p/ and /spl beta//sub q/ are the two different modal propagation constants. For a dielectric rod C(/spl beta//sub p/, /spl beta//sub q/) is a simple line integral around the rod boundary. Approximate forms are presented for optical waveguides.     

The inequalities of quantum information theory

Let /spl rho/ denote the density matrix of a quantum state having n parts 1, ..., n. For I/spl sube/N={1, ..., n}, let /spl rho//sub I/=Tr/sub N/spl bsol/I/(/spl rho/) denote the density matrix of the state comprising those parts i such that i/spl isin/I, and let S(/spl rho//sub I/) denote the von Neumann (1927) entropy of the state /spl rho//sub I/. The collection of /spl nu/=2/sup n/ numbers {S(/spl rho//sub I/)}/sub I/spl sube/N/ may be regarded as a point, called the allocation of entropy for /spl rho/, in the vector space R/sup /spl nu//. Let A/sub n/ denote the set of points in R/sup /spl nu// that are allocations of entropy for n-part quantum states. We show that A~/sub n/~ (the topological closure of A/sub n/) is a closed convex cone in R/sup /spl nu//. This implies that the approximate achievability of a point as an allocation of entropy is determined by the linear inequalities that it satisfies. Lieb and Ruskai (1973) have established a number of inequalities for multipartite quantum states (strong subadditivity and weak monotonicity). We give a finite set of instances of these inequalities that is complete (in the sense that any valid linear inequality for allocations of entropy can be deduced from them by taking positive linear combinations) and independent (in the sense that none of them can be deduced from the others by taking positive linear combinations). Let B/sub n/ denote the polyhedral cone in R/sup /spl nu// determined by these inequalities. We show that A~/sub n/~=B/sub n/ for n/spl les/3. The status of this equality is open for n/spl ges/4. We also consider a symmetric version of this situation, in which S(/spl rho//sub I/) depends on I only through the number i=/spl ne/I of indexes in I and can thus be denoted S(/spl rho//sub i/). In this case, we give for each n a finite complete and independent set of inequalities governing the symmetric allocations of entropy {S(/spl rho//sub i/)}/sub 0/spl les/i/spl les/n/ in R/sup n+1/.     

Order statistics from a logistic distribution and applications to survival and reliability analysis

Joint moments involving arbitrary powers of order statistics are the main concern. Consider order statistics u/sub 1/ /spl les/ u/sub 2/ /spl les/ /spl middot//spl middot//spl middot/ /spl les/ u/sub k/ coming from a simple random sample of size n from a real continuous population where u/sub 1/ = x/sub r(1):n/ is order-statistic #r/sub 1/, u/sub 2/ = x/sub r(1)+r(2):n/ is order statistic #(r/sub 1/ + r/sub 2/), et al., and u/sub k/ = x/sub r(1)+/spl middot//spl middot//spl middot/+r(k):n/ is order statistic #(r/sub 1/ +/spl middot//spl middot//spl middot/+ r/sub k/). Product moments are examined of the type E[u/sub 1//sup /spl alpha/(1)/ /spl middot/ u/sub 2//sup /spl alpha/(2)//sub /spl middot/ /spl middot//spl middot//spl middot//spl middot//u/sub k//sup /spl alpha/(k)/] where /spl alpha//sub 1/, ..., /spl alpha//sub k/ are arbitrary quantities that might be complex numbers, and E[/spl middot/] denotes the s-expected value. Some explicit evaluations are considered for a logistic population. Detailed evaluations of all integer moments of u/sub 1/ and recurrence relations, recurring only on the order of the moments, are given. Connections to survival functions in survival analysis, hazard functions in reliability situations, real type-1, type-2 /spl beta/ and Dirichlet distributions are also examined. Arbitrary product moments for the survival functions are evaluated. Very general results are obtained which can be used in many problems in various areas.     

Entropy and the law of small numbers

Two new information-theoretic methods are introduced for establishing Poisson approximation inequalities. First, using only elementary information-theoretic techniques it is shown that, when S/sub n/=/spl Sigma//sub i=1//sup n/X/sub i/ is the sum of the (possibly dependent) binary random variables X/sub 1/,X/sub 2/,...,X/sub n/, with E(X/sub i/)=p/sub i/ and E(S/sub n/)=/spl lambda/, then D(P(S/sub n/)/spl par/Po(/spl lambda/)) /spl les//spl Sigma//sub i=1//sup n/p/sub i//sup 2/+[/spl Sigma//sub i=1//sup n/H(X/sub i/)-H(X/sub 1/,X/sub 2/,...,X/sub n/)] where D(P(S/sub n/)/spl par/Po(/spl lambda/)) is the relative entropy between the distribution of S/sub n/ and the Poisson (/spl lambda/) distribution. The first term in this bound measures the individual smallness of the X/sub i/ and the second term measures their dependence. A general method is outlined for obtaining corresponding bounds when approximating the distribution of a sum of general discrete random variables by an infinitely divisible distribution. Second, in the particular case when the X/sub i/ are independent, the following sharper bound is established: D(P(S/sub n/)/spl par/Po(/spl lambda/))/spl les/1//spl lambda/ /spl Sigma//sub i=1//sup n/ ((p/sub i//sup 3/)/(1-p/sub i/)) and it is also generalized to the case when the X/sub i/ are general integer-valued random variables. Its proof is based on the derivation of a subadditivity property for a new discrete version of the Fisher information, and uses a recent logarithmic Sobolev inequality for the Poisson distribution.     

Asymptotic properties on codeword lengths of an optimal FV code for general sources

This correspondence is concerned with asymptotic properties on the codeword length of a fixed-to-variable length code (FV code) for a general source {X/sup n/}/sub n=1//sup /spl infin// with a finite or countably infinite alphabet. Suppose that for each n /spl ges/ 1 X/sup n/ is encoded to a binary codeword /spl phi//sub n/(X/sup n/) of length l(/spl phi//sub n/(X/sup n/)). Letting /spl epsiv//sub n/ denote the decoding error probability, we consider the following two criteria on FV codes: i) /spl epsiv//sub n/ = 0 for all n /spl ges/ 1 and ii) lim sup/sub n/spl rarr//spl infin///spl epsiv//sub n/ /spl les/ /spl epsiv/ for an arbitrarily given /spl epsiv/ /spl isin/ [0,1). Under criterion i), we show that, if X/sup n/ is encoded by an arbitrary prefix-free FV code asymptotically achieving the entropy, 1/nl(/spl phi//sub n/(X/sup n/)) - 1/nlog/sub 2/ 1/PX/sup n/(X/sup n/) /spl rarr/ 0 in probability as n /spl rarr/ /spl infin/ under a certain condition, where P/sub X//sup n/ denotes the probability distribution of X/sup n/. Under criterion ii), we first determine the minimum rate achieved by FV codes. Next, we show that 1/nl(/spl phi//sub n/(X/sup n/)) of an arbitrary FV code achieving the minimum rate in a certain sense has a property similar to the lossless case.     

Identifying and locating-dominating codes: NP-completeness results for directed graphs

Let G=(V, A) be a directed, asymmetric graph and C a subset of vertices, and let B/sub r//sup -/(v) denote the set of all vertices x such that there exists a directed path from x to v with at most r arcs. If the sets B/sub r//sup -/(v) /spl cap/ C, v /spl isin/ V (respectively, v /spl isin/ V/spl bsol/C), are all nonempty and different, we call C an r-identifying code (respectively, an r-locating-dominating code) of G. In other words, if C is an r-identifying code, then one can uniquely identify a vertex v /spl isin/ V only by knowing which codewords belong to B/sub r//sup -/(v), and if C is r-locating-dominating, the same is true for the vertices v in V/spl bsol/C. We prove that, given a directed, asymmetric graph G and an integer k, the decision problem of the existence of an r-identifying code, or of an r-locating-dominating code, of size at most k in G, is NP-complete for any r/spl ges/1 and remains so even when restricted to strongly connected, directed, asymmetric, bipartite graphs or to directed, asymmetric, bipartite graphs without directed cycles.