On the capacity of certain additive non-Gaussian channels

A model of an additive non-Gaussian noise channel with generalized average input energy constraint is considered. The asymptotic channel capacityC_{zeta}(S), for large signal-to-noise ratioS, is found under certain conditions on the entropyH_{ tilde{ zeta}}( zeta)of the measure induced in function space by the noise processzeta, relative to the measure induced bytilde{zeta}, where is a Gaussian process with the same covariance as that ofzeta. IfH_{ tilde{zeta}}( zeta) < inftyand the channel input signal is of dimensionM< infty, thenC_{ zeta}(S)= frac{1}{2}M ln(1 + S/M) + Q_{zeta}( M ) + {o}(1), where0 leq Q_{ zeta}( M ) leq H_{ tilde{ zeta}}( zeta). If the channel input signal is of infinite dimension andH_{ tilde{ zeta}}( zeta) rightarrow 0forS rightarrow infty, thenC_{ zeta}(S) = frac{1}{2}S+{o}(1).     

Improvements of Winograd's result on computation in the presence of noise (Corresp.)

Winograd's result concerning Elias' model of computation in the presence of noise can be stated without reference to computation. If a codevarphi: {0,1}^{k} rightarrow {0,1}^{n}is min-preserving(varphi (a wedge b) = varphi (a) wedge varphi (b)fora,b in {0,1}^{k})andepsilon n-error correcting, then the ratek/n rightarrow 0ask rightarrow infty. This result is improved and extended in two directions. begin{enumerate} item For min-preserving codes with {em fixed} maximal (and also average) error probability on a binary symmetric channel againk/n rightarrow 0ask rightarrow infty(strong converses). item Second, codes with lattice properties without reference to computing are studied for their own sake. Already for monotone codes( varphi (a) leq varphi (b)fora leq b)the results in direction 1) hold for maximal errors. end{enumerate} These results provide examples of coding theorems in which entropy plays no role, and they can be reconsidered from the viewpoint of multiuser information theory.     

When does the modular distance induce a metric in the binary case? (Corresp.)

The modular distance induces a metric if and only if the nonadjacent form of the modulusMhas one of the following forms:1) 2^{n}+2^{n-2} pm 2^{i}, wheren-igeq 4; 2) 2^{n} - 2^{j} pm 2^{i}, where2 leq n -j leq 5andj-igeq 2; 3) 2^{n} pm 2^{j}, wheren -j geq 2; 4) 2^{n}.     

An upper bound for codes for the noisy two-access binary adder channel (Corresp.)

Using earlier methods a combinatorial upper bound is derived for|C|. cdot |D|, where(C,D)is adelta-decodable code pair for the noisy two-access binary adder channel. Asymptotically, this bound reduces toR_{1}=R_{2} leq frac{3}{2} + elog_{2} e - (frac{1}{2} + e) log_{2} (1 + 2e)= frac{1}{2} - e + H(frac{1}{2} - e) - frac{1}{2}H(2e),wheree = lfloor (delta - 1)/2 rfloor /n, n rightarrow inftyandR_{1}resp.R_{2}is the rate of the codeCresp.D.     

Robust prediction of band-limited signals from past samples (Corresp.)

For a complex-valued deterministic signal of finite energy band-limited to the normalized frequency band|w| leq piexplicit coefficients{a_{kn}}are found such that for anyTsatisfying0 < T leq 1/2,left| f(t)-sum^{2n}_{k=1}a_{kn}f(t - kT)right| leq E_{f}cdot beta^{n}whereE_{f}is the signal energy andbeta doteq 0.6863. Thus the estimate off(t)in terms of2npast samples taken at a rate equal to or in excess of twice the Nyquist rate converges uniformly at a geometric rate tof(t)on(- infty , infty). The suboptimal coefficients{a_{kn}}have the desirable property of being pure numbers independent of both the particular band-limited signal and of the selected sampling rate1/T. It is also shown that these same coefficients can be used to estimate the value ofx(t)of a wide-sense stationary random process in terms of past samples.     

On theepsilon-entropy and the rate-distortion function of certain non-Gaussian processes

Letxi = {xi(t), 0 leq t leq T}be a process with covariance functionK(s,t)andE int_0^T xi^2(t) dt < infty. It is proved that for everyvarepsilon > 0thevarepsilon-entropyH_{varepsilon}(xi)satisfies begin{equation} H_{varepsilon}(xi_g) - mathcal{H}_{xi_g} (xi) leq H_{varepsilon}(xi) leq H_{varepsilon}(xi_g) end{equation} wherexi_gis a Gaussian process with the covarianeeK(s,t)andmathcal{H}_{xi_g}(xi)is the entropy of the measure induced byxi(in function space) with respect to that induced byxi_g. It is also shown that ifmathcal{H}_{xi_g}(xi) < inftythen, asvarepsilon rightarrow 0begin{equation} H_{varepsilon}(xi) = H_{varepsilon}(xi_g) - mathcal{H}_{xi_g}(xi) + o(1). end{equation} Furthermore, ff there exists a Gaussian processg = { g(t); 0 leq t leq T }such thatmathcal{H}_g(xi) < infty, then the ratio betweenH_{varepsilon}(xi)andH_{varepsilon}(g)goes to one asvarepsilongoes to zero. Similar results are given for the rate-distortion function, and some particular examples are worked out in detail. Some cases for whichmathcal_{xi_g}(xi) = inftyare discussed, and asymptotic bounds onH_{varepsilon}(xi), expressed in terms ofH_{varepsilon}(xi_g), are derived.     

Effect of gap length on the input admittance of center fed coaxial waveguides and infinite dipoles

The input admittance of a coaxial waveguide fed by a gap of length2din the center conductor is evaluated using the dyadic Green's function of the guide and a band of equivalent magnetic surface current proportional to the gap's axial electric field via the equivalence principle. The axial electric field is expressed in terms of a rapidly convergent series of ultraspherical polynomials whose weighting function satisfies the edge conditions at each end of the gap. If the inner and outer radii of the coaxial guide areaandb, respectively, then the limiting case ofb rightarrow inftyis an infinite dipole in free space. Numerical results for the admittance are given as a function ofka (0.01 leq ka leq 0.50)with parameterb/a = 2, 5,10and 50 for the coaxial guide. For the infinite dipole the admittance is presented as a function ofd/a (10^{-3} leq d/a leq 10)withkaas a parameter (0.001 leq ka leq 0.1).     

On MDS extensions of generalized Reed- Solomon codes

An(n, k, d)linear code overF=GF(q)is said to be {em maximum distance separable} (MDS) ifd = n - k + 1. It is shown that an(n, k, n - k + 1)generalized Reed-Solomon code such that2leq k leq n - lfloor (q - 1)/2 rfloor (k neq 3 {rm if} qis even) can be extended by one digit while preserving the MDS property if and only if the resulting extended code is also a generalized Reed-Solomon code. It follows that a generalized Reed-Solomon code withkin the above range can be {em uniquely} extended to a maximal MDS code of lengthq + 1, and that generalized Reed-Solomon codes of lengthq + 1and dimension2leq k leq lfloor q/2 rfloor + 2 (k neq 3 {rm if} qis even) do not have MDS extensions. Hence, in cases where the(q + 1, k)MDS code is essentially unique,(n, k)MDS codes withn > q + 1do not exist.     

Weight enumerator for second-order Reed-Muller codes

In this paper, we establish the following result. Theorem:A_i, the number of codewords of weightiin the second-order binary Reed-Muller code of length2^mis given byA_i = 0unlessi = 2^{m-1}or2^{m-1} pm 2^{m-l-j}, for somej, 0 leq j leq [m/2], A_0 = A_{2^m} = 1, and begin{equation} begin{split} A_{2^{m-1} pm 2^{m-1-j}} = 2^{j(j+1)} &{frac{(2^m - 1) (2^{m-1} - 1 )}{4-1} } \ .&{frac{(2^{m-2} - 1)(2^{m-3} -1)}{4^2 - 1} } cdots \ .&{frac{(2^{m-2j+2} -1)(2^{m-2j+1} -1)}{4^j -1} } , \ & 1 leq j leq [m/2] \ end{split} end{equation} begin{equation} A_{2^{m-1}} = 2 { 2^{m(m+1)/2} - sum_{j=0}^{[m/2]} A_{2^{m-1} - 2^{m-1-j}} }. end{equation}     

On single-sample robust detection of known signals with additive unknown-mean amplitude-bounded random interference--II: The randomized decision rule solution (Corresp.)

A randomized decision rule is derived and proved to be the saddlepoint solution of the robust detection problem for known signals in independent unknown-mean amplitude-bounded noise. The saddlepoint solutionphi^{0}uses an equaUy likely mixed strategy to chose one ofNBayesian single-threshold decision rulesphi_{i}^{0}, i = 1,cdots , Nhaving been obtained previously by the author. These decision rules are also all optimal against the maximin (least-favorable) nonrandomized noise probability densityf_{0}, wheref_{0}is a picket fence function withNpickets on its domain. Thee pair(phi^{0}, f_{0})is shown to satisfy the saddlepoint condition for probability of error, i.e.,P_{e}(phi^{0} , f) leq P_{e}(phi^{0} , f_{0}) leq P_{e}(phi, f_{0})holds for allfandphi. The decision rulephi^{0}is also shown to be an eqoaliir rule, i.e.,P_{e}(phi^{0}, f ) = P_{e}(phi^{0},f_{0}), for allf, with4^{-1} leq P_{e}(phi^{0},f_{0})=2^{-1}(1-N^{-1})leq2^{-1} , N geq 2. Thus nature can force the communicator to use an {em optimal} randomized decision rule that generates a large probability of error and does not improve when less pernicious conditions prevail.     

Fitting continuous probability density functions over [0,infty) using information theory ideas (Corresp.)

The problem of estimating a probability density functionp(x)over[0, infty)when several low-order moments are known is considered. A computational procedure, based on information theory, is used to obtain the maximum entropy estimates ofp(x)in a number of cases. The situation when the first two moments only are known is considered in some detail. A table is included for estimatingp(x)when givenmu_1 ^{prime} , mu_2 ^{prime}withmu_2 ^{prime} leq 2(mu_1 ^{prime}) ^ 2.     

A Gaussian channel with slow fading (Corresp.)

An interleaved fading channel whose state is known to the receiver is analyzed. The reliability functionE(R)is obtained for ratesRin the rangeR_c leq R leq C. The capacity is shown to beC = E_A { frac{1}{2} ln (1 + A^2 n)}whereAis a factor describing the fading mechanism anduis the signal-to-noise ratio per dimension.     

Asymptotic properties of delay-time-weighted probability of error (Corresp.)

Asymptotic properties of expected distortion are studied for the delay-time-weighted probability of error distortion measured_n(x,tilde{x}) = n^{-1} sum_{t=0}^{n-1} f(t + n)[l - delta(x_t,tilde{x}_t)],, wherex = (x_0,x_1,cdots,x_{n-1})andtilde{x} = (tilde{x}_0,tilde{x}_1,cdots,tilde{x}_{n-1})are source and reproducing vectors, respectively, anddelta (cdot, cdot)is the Kronecker delta. With reasonable block coding and transmission constraintsx_tis reproduced astilde{x}_twith a delay oft + ntime units. It is shown that if the channel capacity is greater than the source entropyC > H(X), then there exists a sequence of block lengthncodes such thatE[d_n(X,tilde{X})] rigjhtarrow 0asn rightarrow inftyeven iff(t) rightarrow inftyat an exponential rate. However, iff(t)grows at too fast an exponential rate, thenE[d_n(X,tilde{X})] rightarrow inftyasn rightarrow infty. Also, ifC < H(X)andf(t) rightarrow inftythenE[d_n(X,tilde{X})] rightarrow inftyasn rightarrow inftyno matter how slowlyf(t)grows.     

Complexity of acceptors for prefix codes (Corresp.)

For a given finite set of messages and their assigned probabilities, Huffman's procedure gives a method of computing a length set (a set of codeword lengths) that is optimal in the sense that the average word length is minimized. Corresponding to a particular length set, however, there may be more than one code. LetL(n)consist of all length sets with largest termn, and, for anyell in L(n), let{cal S}( ell)be the smallest number of states in any finite-state acceptor which accepts a set of codewords with length setell. It is shown that, for alln > 1,n^{2}/(16 log_{2} n) leq max {cal S}(ell) leq 0(n^{2}).ell in L(n)     

An approximation to the weight distribution of binary primitive BCH codes with designed distances 9 and 11 (Corresp.)

Recently Kasami {em et al.} presented a linear programming approach to the weight distribution of binary linear codes [2]. Their approach to compute upper and lower bounds on the weight distribution of binary primitive BCH codes of length2^{m} - 1withm geq 8and designed distance2t + 1with4 leq t leq 5is improved. From these results, the relative deviation of the number of codewords of weightjleq 2^{m-1}from the binomial distribution2^{-mt} left( stackrel{2^{m}-1}{j} right)is shown to be less than 1 percent for the following cases: (1)t = 4, j geq 2t + 1andm geq 16; (2)t = 4, j geq 2t + 3and10 leq m leq 15; (3)t=4, j geq 2t+5and8 leq m leq 9; (4)t=5,j geq 2t+ 1andm geq 20; (5)t=5, j geq 2t+ 3and12 leq m leq 19; (6)t=5, j geq 2t+ 5and10 leq m leq 11; (7)t=5, j geq 2t + 7andm=9; (8)t= 5, j geq 2t+ 9andm = 8.     

On the Schalkwijk-Kailath coding scheme with a peak energy constraint

In a recent series of papers, [2]-[4] Schalkwijk and Kailath have proposed a block coding scheme for transmission over the additive white Gaussian noise channel with one-sided spectral densityN_{0}using a noiseless delayless feedback link. The signals have bandwidthW (W leq infty)and average powerbar{P}. They show how to communicate at ratesR < C = W log (1 + bar{P}/N_{0}W), the channel capacity, with error probabilityP_{e} = exp {-e^{2(C-R)T+o(T)}}(whereTis the coding delay), a "double exponential" decay. In their scheme the signal energy (in aT-second transmission) is a random variable with only its expectation constrained to bebar{P}T. In this paper we consider the effect of imposing a peak energy constraint on the transmitter such that whenever the Schalkwijk-Kailath scheme requires energy exceeding abar{P}T(wherea > 1is a fixed parameter) transmission stops and an error is declared. We show that the error probability is degraded to a "single exponential" formP_{e} = e^{-E(a)T+o(T)}and find the exponentE(a). In the caseW = infty , E(a) = (a - 1)^{2}/4a C. For finiteW, E(a)is given by a more complicated expression.     

Design of single mesh flashlamp driving circuits with resistive losses

The normalized nonlinear equation(dI/dtau) pm [alpha + beta|I|^{1/2}] + intmin{0}max{tau}I dbar{tau}= 1describing current evolution in a single mesh flashlamp driving circuit has been solved using numerical methods for the case of a critically damped pulse (alpha simeq 0.84) and beta = r/Z_{0} neq 0. Normalized current, power, and energy waveforms for values of β in the range0 leq beta leq 0.5are obtained. The energy efficiency ηE, which is numerically equal to the flashlamp dissipated energy di-Vided by the energy stored in the driving circuit, is also shown as a function of β. Increased numerical accuracy has improved the previously used value of α for critical damping and leads to a slightly different form for the curve representing the locus of values (α β) to obtain critically damped pulses in the presence of restrictive losses.     

Bounds for codes over the unit circle

LetCbe a code of lengthnand rateRover the alphabetA(Q)={ exp (2pi ir/Q): r=O,1, cdots ,Q-1}, and letd(C)be the minimum Euclidean distance ofC. For largen, the lower and upper bounds are obtained in parametric form on the achievable pairs(R, delta), wheredelta = d^{2}(C)/nholds. To obtain these bounds, the arguments leading to the Gilbert bound and the Elias bound, respectively, are applied to the alphabetA(Q). ForQ rightarrow infty, they are shown to be expressible in terms of the modified Bessel function of the first kind. The Elias type bound is compared with the Kabatyanskii-Levenshtein (K-L) bound that holds for less restrictive alphabets. It turns out that our upper bound improves the K-L bound fordelta leq 0.93.     

On lower bounds for the nonlinear filtering problem (Corresp.)

The system modeldx_{t}=m(x_{t})dt+sigma_{o}dw_{t}and the observationdy_{t}=h(x_{t})dt+rho dv_{t}are considered, and a lower bound on the error for the least-square estimate of a function ofx_{t}, sayz(x_{t}), is derived. This bound is then applied to derive a lower bound on the error in estimatingx_{t}in the case where the constantsigma_{o}in the equation for the system dynamics is replaced bysigma (x_{t}).     

An estimate of the variation of a band-limited process

Upper and lower bounds are established for the mean-square variation of a stationary processX(t)whose power spectrum is bounded byomega_{c}, in terms of its average powerP_{0}and the average powerP_{1}of its derivative. It is shown thatleft( frac{2}{pi} right)^{2} P_{1} tau^{2} leq E {|X(t+tau )-X(t)|^{2}} leq P_{1} tau^{2} leq omega_{c}^{2}P_{0}tau^{2}where the upper bounds are valid for anytauand the lower bound fortau < pi / omega_{c}. These estimates are applied to the mean-square variation of the envelope of a quasi-monochromatic process.