An upper bound to the capacity of the band-limited Gaussian autoregressive channel with noiseless feedback

Upper bounds to the capacity of band-limited Gaussianmth-order autoregressive channels with feedback and average energy constraintEare derived. These are the only known hounds on one- and two-way autoregressive channels of order greater than one. They are the tightest known for the first-order case. In this case letalpha_1be the regression coefficient,sigma^2the innovation variance,Nthe number of channel iterations per source symbol, ande = E/N; then the first-order capacityC^1is bounded by begin{equation} C^1 leq begin{cases} frac{1}{2} ln [frac{e}{sigma^2}(1+ mid alpha_1 mid ) ^ 2 +1], & frac{e}{sigma^2} leq frac{1}{1- alpha_1^2} \ frac{1}{2} ln [frac{e}{sigma^2} + frac{2mid alpha_1 mid}{sqrt{1-alpha_1^2}} sqrt{frac{e}{simga^2}} + frac{1}{1-alpha_1^2}], & text{elsewhere}.\ end{cases} end{equation} This is equal to capacity without feedback for very low and very highe/sigma^2and is less than twice this one-way capacity everywhere.     

Weight enumerator for second-order Reed-Muller codes

In this paper, we establish the following result. Theorem:A_i, the number of codewords of weightiin the second-order binary Reed-Muller code of length2^mis given byA_i = 0unlessi = 2^{m-1}or2^{m-1} pm 2^{m-l-j}, for somej, 0 leq j leq [m/2], A_0 = A_{2^m} = 1, and begin{equation} begin{split} A_{2^{m-1} pm 2^{m-1-j}} = 2^{j(j+1)} &{frac{(2^m - 1) (2^{m-1} - 1 )}{4-1} } \ .&{frac{(2^{m-2} - 1)(2^{m-3} -1)}{4^2 - 1} } cdots \ .&{frac{(2^{m-2j+2} -1)(2^{m-2j+1} -1)}{4^j -1} } , \ & 1 leq j leq [m/2] \ end{split} end{equation} begin{equation} A_{2^{m-1}} = 2 { 2^{m(m+1)/2} - sum_{j=0}^{[m/2]} A_{2^{m-1} - 2^{m-1-j}} }. end{equation}     

An approximation to the weight distribution of binary primitive BCH codes with designed distances 9 and 11 (Corresp.)

Recently Kasami {em et al.} presented a linear programming approach to the weight distribution of binary linear codes [2]. Their approach to compute upper and lower bounds on the weight distribution of binary primitive BCH codes of length2^{m} - 1withm geq 8and designed distance2t + 1with4 leq t leq 5is improved. From these results, the relative deviation of the number of codewords of weightjleq 2^{m-1}from the binomial distribution2^{-mt} left( stackrel{2^{m}-1}{j} right)is shown to be less than 1 percent for the following cases: (1)t = 4, j geq 2t + 1andm geq 16; (2)t = 4, j geq 2t + 3and10 leq m leq 15; (3)t=4, j geq 2t+5and8 leq m leq 9; (4)t=5,j geq 2t+ 1andm geq 20; (5)t=5, j geq 2t+ 3and12 leq m leq 19; (6)t=5, j geq 2t+ 5and10 leq m leq 11; (7)t=5, j geq 2t + 7andm=9; (8)t= 5, j geq 2t+ 9andm = 8.     

Linear feedback rate bounds for regressive channels (Corresp.)

This article presents new tighter upper bounds on the rate of Gaussian autoregressive channels with linear feedback. The separation between the upper and lower bounds is small. We havefrac{1}{2} ln left( 1 + rho left( 1+ sum_{k=1}^{m} alpha_{k} x^{- k} right)^{2} right) leq C_{L} leq frac{1}{2} ln left( 1+ rho left( 1+ sum_{k = 1}^{m} alpha_{k} / sqrt{1 + rho} right)^{2} right), mbox{all rho}, whererho = P/N_{0}W, alpha_{l}, cdots, alpha_{m}are regression coefficients,Pis power,Wis bandwidth,N_{0}is the one-sided innovation spectrum, andxis a root of the polynomial(X^{2} - 1)x^{2m} - rho left( x^{m} + sum^{m}_{k=1} alpha_{k} x^{m - k} right)^{2} = 0.It is conjectured that the lower bound is the feedback capacity.     

An upper bound for codes for the noisy two-access binary adder channel (Corresp.)

Using earlier methods a combinatorial upper bound is derived for|C|. cdot |D|, where(C,D)is adelta-decodable code pair for the noisy two-access binary adder channel. Asymptotically, this bound reduces toR_{1}=R_{2} leq frac{3}{2} + elog_{2} e - (frac{1}{2} + e) log_{2} (1 + 2e)= frac{1}{2} - e + H(frac{1}{2} - e) - frac{1}{2}H(2e),wheree = lfloor (delta - 1)/2 rfloor /n, n rightarrow inftyandR_{1}resp.R_{2}is the rate of the codeCresp.D.     

On certain projective geometry codes (Corresp.)

LetVbe an(n, k, d)binary projective geometry code withn = (q^{m}-1)/(q - 1), q = 2^{s}, andd geq [(q^{m-r}-1)/(q - 1)] + 1. This code isr-step majority-logic decodable. With reference to the GF(q^{m}) = {0, 1, alpha , alpha^{2} , cdots , alpha^{n(q-1)-1} }, the generator polynomialg(X), ofV, hasalpha^{nu}as a root if and only ifnuhas the formnu = i(q - 1)andmax_{0 leq l < s} W_{q}(2^{l} nu) leq (m - r - 1)(q - 1), whereW_{q}(x)indicates the weight of the radix-qrepresentation of the numberx. LetSbe the set of nonzero numbersnu, such thatalpha^{nu}is a root ofg(X). LetC_{1}, C_{2}, cdots, C_{nu}be the cyclotomic cosets such thatSis the union of these cosets. It is clear that the process of findingg(X)becomes simpler if we can find a representative from eachC_{i}, since we can then refer to a table, of irreducible factors, as given by, say, Peterson and Weldon. In this correspondence it was determined that the coset representatives for the cases ofm-r = 2, withs = 2, 3, andm-r=3, withs=2.     

On the weight structure of Reed-Muller codes

The following theorem is proved. Letf(x_1,cdots, x_m)be a binary nonzero polynomial ofmvariables of degreenu. H the number of binarym-tuples(a_1,cdots, a_m)withf(a_1, cdots, a_m)= 1 is less than2^{m-nu+1}, thenfcan be reduced by an invertible affme transformation of its variables to one of the following forms. begin{equation} f = y_1 cdots y_{nu - mu} (y_{nu-mu+1} cdots y_{nu} + y_{nu+1} cdots y_{nu+mu}), end{equation} wherem geq nu+muandnu geq mu geq 3. begin{equation} f = y_1 cdots y_{nu-2}(y_{nu-1} y_{nu} + y_{nu+1} y_{nu+2} + cdots + y_{nu+2mu -3} y_{nu+2mu-2}), end{equation} This theorem completely characterizes the codewords of thenuth-order Reed-Muller code whose weights are less than twice the minimum weight and leads to the weight enumerators for those codewords. These weight formulas are extensions of Berlekamp and Sloane's results.     

A fundamental inequality between the probabilities of binary subgroups and cosets

The probability of a set of binaryn-tuples is defined to be the sum of the probabilities of the individualn-tuples when each digit is chosen independently with the same probabilitypof being a "one." It is shown that, under such a definition, the ratio between the probability of a subgroup of order2^{k}and any of its proper cosets is always greater than or equal to a functionF_{k}(p), whereF_{k}(p) geq 1forp leq frac{1}{2}with equality when and only whenp = frac{1}{2}. It is further shown thatF_{k}(p)is the greatest lower bound on this ratio, since a subgroup and proper coset of order2^{k}can always be found such that the ratio between their probabilities is exactlyF_{k}(p). It is then demonstrated that for a linear code on a binary symmetric channel the "tall-zero" syndrome is more probable than any other syndrome. This result is applied to the problem of error propagation in convolutional codes.     

Optimal source codes for geometrically distributed integer alphabets (Corresp.)

LetP(i)= (1 - theta)theta^ibe a probability assignment on the set of nonnegative integers wherethetais an arbitrary real number,0 < theta < 1. We show that an optimal binary source code for this probability assignment is constructed as follows. Letlbe the integer satisfyingtheta^l + theta^{l+1} leq 1 < theta^l + theta^{l-1}and represent each nonnegative integeriasi = lj + rwhenj = lfloor i/l rfloor, the integer part ofi/l, andr = [i] mod l. Encodejby a unary code (i.e.,jzeros followed by a single one), and encoderby a Huffman code, using codewords of lengthlfloor log_2 l rfloor, forr < 2^{lfloor log l+1 rfloor} - l, and lengthlfloor log_2 l rfloor + 1otherwise. An optimal code for the nonnegative integers is the concatenation of those two codes.     

An improved error bound for equal-strength diversity channels (Corresp.)

An upper bound on the minimum probability of error for an equal-strength diversity channel is simply derived that improves a previously known bound by the factor[4(1 - p)]^(-1),0 leq p leq frac{1}{2}.     

Sequences achieving the boundary of the entropy region for a two-source are virtually memoryless (Corresp.)

For a joint distribution{rm dist}(X,Y), the functionT(t)=min { H(Y|U): I(U wedge Y|X)=O, H(X|U)geq t}is an important characteristic. It equals the asymptotic minimum of(1/n)H(Y^{n})for random pairs of sequences(X^{n}, Y^{n}), wherefrac{1}{n} sum ^{n}_{i=1}{rm dist} X_{i} sim {rm dist} X, {rm dist} Y^{n}|X^{n} = ({rm dist} Y|X)^{n}, frac{1}{n}H(X^{n})geq t.We show that if, for(X^{n}, Y^{n})as given, the rate pair[(1/n)H(X^{n}),(1/n)H(Y^{n})]approaches the nonlinear part of the curve(t,T(t)), then the sequenceX^{n}is virtually memoryless. Using this, we determine some extremal sections of the rate region of entropy characterization problems and find a nontrivial invariant for weak asymptotic isomorphy of discrete memoryless correlated sources.     

A new class of codes meeting the Griesmer bound

An infinite sequence ofk-dimensional binary linear block codes is constructed with parametersn=2^{k}+2^{k-2}-15,d=2^{k-1}+2^{k-3}-8,k geq 7. Fork geq 8these codes are unique, while there are five nonisomorphic codes fork=7. By shortening these codes in an appropriate way, one finds codes meeting the Griesmer bound for2^{k-1}+2^{k-3}-15 leq d leq 2^{k-1}+2^{k-3}-8; k geq 7.     

A universal chart for use in relating ionospheric absorption to phase path and group path

The absorption of a radio wave in the ionosphere can be approximated byA = frac{omega}{c}frac{1}{2}intfrac{Z(mu' - mu)}{1 + g} cos alpha ds, where the integral is along the ray that exists when the normalized collision frequencyZ = 0, mu'andmuare the group and phase refractive indices, respectively, andalphathe angle between the wave normal and ray direction. Graphs are presented from whichgcan be obtained for any values of the ionospheric plasma parametersXandY.     

On the capacity of certain additive non-Gaussian channels

A model of an additive non-Gaussian noise channel with generalized average input energy constraint is considered. The asymptotic channel capacityC_{zeta}(S), for large signal-to-noise ratioS, is found under certain conditions on the entropyH_{ tilde{ zeta}}( zeta)of the measure induced in function space by the noise processzeta, relative to the measure induced bytilde{zeta}, where is a Gaussian process with the same covariance as that ofzeta. IfH_{ tilde{zeta}}( zeta) < inftyand the channel input signal is of dimensionM< infty, thenC_{ zeta}(S)= frac{1}{2}M ln(1 + S/M) + Q_{zeta}( M ) + {o}(1), where0 leq Q_{ zeta}( M ) leq H_{ tilde{ zeta}}( zeta). If the channel input signal is of infinite dimension andH_{ tilde{ zeta}}( zeta) rightarrow 0forS rightarrow infty, thenC_{ zeta}(S) = frac{1}{2}S+{o}(1).     

A bound on lightweight sequences with application to definite decoding (Corresp.)

It is shown that the numberMof binary-valuedn-tuples having fractional weightdeltaor less,0 < delta leq frac{1}{3}, such that no twon-tuples agree in anyLconsecutive positions, is bounded by2^{2LH(delta)+1}. A set ofn-tuples is constructed to show that this bound is not likely to be improved upon by any significant factor. This bound is used to show that the ratiod_{DD}/n_{DD}of definite-decoding minimum distance to definite-decoding constraint length is lower bounded byH^{-l}[frac{1}{6} cdot (1 - R)/ (1+R)]asn_{DD}grows without bound.     

A Gilbert bound for periodic binary convolutional codes

A Gilbert bound for periodic binary convolutional (PBC) codes is established. This bound shows that regardless of previous decoding decisions any fraction of errors less thanalpha/2can be corrected in a constraint length by some PBC code if the constraint length is sufficiently large andR, the code rate, is less than[1 - H(alpha)]/2, 0 leq alpha < frac{1}{2}.     

Further classifications of codes meeting the Griesmer bound (Corresp.)

For any(n, k, d)binary linear code, the Griesmer bound says thatn geq sum_{i=0}^{k-1} lceil d/2^{i} rceil, wherelceil x rceildenotes the smallest integergeq x. We consider codes meeting the Griesmer bound with equality. These codes have parametersleft( s(2^{k} - 1) - sum_{i=1}^{p} (2^{u_{i}} - 1), k, s2^{k-1} - sum_{i=1}^{p} 2^{u_{i} -1} right), wherek > u_{1} > cdots > u_{p} geq 1. We characterize all such codes whenp = 2oru_{i-1}-u_{i} geq 2for2 leq i leq p.     

On theepsilon-entropy and the rate-distortion function of certain non-Gaussian processes

Letxi = {xi(t), 0 leq t leq T}be a process with covariance functionK(s,t)andE int_0^T xi^2(t) dt < infty. It is proved that for everyvarepsilon > 0thevarepsilon-entropyH_{varepsilon}(xi)satisfies begin{equation} H_{varepsilon}(xi_g) - mathcal{H}_{xi_g} (xi) leq H_{varepsilon}(xi) leq H_{varepsilon}(xi_g) end{equation} wherexi_gis a Gaussian process with the covarianeeK(s,t)andmathcal{H}_{xi_g}(xi)is the entropy of the measure induced byxi(in function space) with respect to that induced byxi_g. It is also shown that ifmathcal{H}_{xi_g}(xi) < inftythen, asvarepsilon rightarrow 0begin{equation} H_{varepsilon}(xi) = H_{varepsilon}(xi_g) - mathcal{H}_{xi_g}(xi) + o(1). end{equation} Furthermore, ff there exists a Gaussian processg = { g(t); 0 leq t leq T }such thatmathcal{H}_g(xi) < infty, then the ratio betweenH_{varepsilon}(xi)andH_{varepsilon}(g)goes to one asvarepsilongoes to zero. Similar results are given for the rate-distortion function, and some particular examples are worked out in detail. Some cases for whichmathcal_{xi_g}(xi) = inftyare discussed, and asymptotic bounds onH_{varepsilon}(xi), expressed in terms ofH_{varepsilon}(xi_g), are derived.     

An estimate of the variation of a band-limited process

Upper and lower bounds are established for the mean-square variation of a stationary processX(t)whose power spectrum is bounded byomega_{c}, in terms of its average powerP_{0}and the average powerP_{1}of its derivative. It is shown thatleft( frac{2}{pi} right)^{2} P_{1} tau^{2} leq E {|X(t+tau )-X(t)|^{2}} leq P_{1} tau^{2} leq omega_{c}^{2}P_{0}tau^{2}where the upper bounds are valid for anytauand the lower bound fortau < pi / omega_{c}. These estimates are applied to the mean-square variation of the envelope of a quasi-monochromatic process.