Some cross correlation properties for distorted signals

For a nondecreasing distortion characteristicphi(cdot)and a given signalx(cdot), the "cross correlation" function defined byR_{phi} (tau) triangleq int_{-infty}^{infty} phi[x(t)]x(t - tau) dtis shown to satisfy the inequalityR_{phi}(tau) leq R_{phi}(0), for alltau, generalizing an earlier result of Richardson that requiredphi(cdot)to be continuous and strictly increasing. The methods of the paper also show that, under weak conditions, begin{equation} R_{phi,psi}(tau) triangleq int_{-infty}^{infty} phi[x(t)]psi[x(t - tau)] dt leq R_{phi,psi}(0) end{equation} whenpsiis strictly increasing andphiis nondecreasing. In the case of hounded signals (e.g., periodic functions), the appropriate cross correlation function is begin{equation} mathcal{R}_{phi,psi}(tau} triangleq lim_{T rightarrow infty} (2T)^{-l} int_{-T}^T phi[x(t)]psi[x(t - tau)] dt. end{equation} For this case it is shown thatmathcal{R}_{phi,psi} (tau) leq mathcal{R}_{phi,psi}(0)for any nondecreasing (or nonincreasing) distortion functionsphiandpsi. The result is then applied to generalize an inequality on correlation functions for periodic signals due to Prosser. Noise signals are treated and inequalities of a similar nature are obtained for ensemble-average cross correlation functions under suitable hypotheses on the statistical properties of the noise. Inequalities of this type are the basis of a well-known method of estimating the unknown time delay of an observed signal. The extension to nondecreasing discontinuous distortion functions allows the use of hard limiting or quantization to facilitate the cross correlation calculation.     

On the capacity of certain additive non-Gaussian channels

A model of an additive non-Gaussian noise channel with generalized average input energy constraint is considered. The asymptotic channel capacityC_{zeta}(S), for large signal-to-noise ratioS, is found under certain conditions on the entropyH_{ tilde{ zeta}}( zeta)of the measure induced in function space by the noise processzeta, relative to the measure induced bytilde{zeta}, where is a Gaussian process with the same covariance as that ofzeta. IfH_{ tilde{zeta}}( zeta) < inftyand the channel input signal is of dimensionM< infty, thenC_{ zeta}(S)= frac{1}{2}M ln(1 + S/M) + Q_{zeta}( M ) + {o}(1), where0 leq Q_{ zeta}( M ) leq H_{ tilde{ zeta}}( zeta). If the channel input signal is of infinite dimension andH_{ tilde{ zeta}}( zeta) rightarrow 0forS rightarrow infty, thenC_{ zeta}(S) = frac{1}{2}S+{o}(1).     

Asymptotic properties of delay-time-weighted probability of error (Corresp.)

Asymptotic properties of expected distortion are studied for the delay-time-weighted probability of error distortion measured_n(x,tilde{x}) = n^{-1} sum_{t=0}^{n-1} f(t + n)[l - delta(x_t,tilde{x}_t)],, wherex = (x_0,x_1,cdots,x_{n-1})andtilde{x} = (tilde{x}_0,tilde{x}_1,cdots,tilde{x}_{n-1})are source and reproducing vectors, respectively, anddelta (cdot, cdot)is the Kronecker delta. With reasonable block coding and transmission constraintsx_tis reproduced astilde{x}_twith a delay oft + ntime units. It is shown that if the channel capacity is greater than the source entropyC > H(X), then there exists a sequence of block lengthncodes such thatE[d_n(X,tilde{X})] rigjhtarrow 0asn rightarrow inftyeven iff(t) rightarrow inftyat an exponential rate. However, iff(t)grows at too fast an exponential rate, thenE[d_n(X,tilde{X})] rightarrow inftyasn rightarrow infty. Also, ifC < H(X)andf(t) rightarrow inftythenE[d_n(X,tilde{X})] rightarrow inftyasn rightarrow inftyno matter how slowlyf(t)grows.     

Mutual information of the white Gaussian channel with and without feedback

The following model for the white Gaussian channel with or without feedback is considered: begin{equation} Y(t) = int_o ^{t} phi (s, Y_o ^{s} ,m) ds + W(t) end{equation} wheremdenotes the message,Y(t)denotes the channel output at timet,Y_o ^ {t}denotes the sample pathY(theta), 0 leq theta leq t. W(t)is the Brownian motion representing noise, andphi(s, y_o ^ {s} ,m)is the channel input (modulator output). It is shown that, under some general assumptions, the amount of mutual informationI(Y_o ^{T} ,m)between the messagemand the output pathY_o ^ {T}is directly related to the mean-square causal filtering error of estimatingphi (t, Y_o ^{t} ,m)from the received dataY_o ^{T} , 0 leq t leq T. It follows, as a corollary to the result forI(Y_o ^ {T} ,m), that feedback can not increase the capacity of the nonband-limited additive white Gaussian noise channel.     

On mean-square aliasing error in the cardinal series expansion of random processes (Corresp.)

An upper bound is derived for the mean-square error involved when a non-band-limited, wide-sense stationary random processx(t)(possessing an integrable power spectral density) is approximated by a cardinal series expansion of the formsum^{infty}_{-infty}x(n/2W)sinc2W(t-n/2W), a sampling expansion based on the choice of some nominal bandwidthW > 0. It is proved thatlim_{N rightarrow infty} E {|x(t) - x_{N}(t)|^{2}} leq frac{2}{pi}int_{| omega | > 2 pi W}S_{x}( omega) d omega,wherex_{N}(t) = sum_{-N}^{N}x(n/2W)sinc2W(t-n/2W), andS_{x}(omega)is the power spectral density forx(t). Further, the constant2/ piis shown to be the best possible one if a bound of this type (involving the power contained in the frequency region lying outside the arbitrarily chosen band) is to hold uniformly int. Possible reductions of the multiplicative constant as a function oftare also discussed, and a formula is given for the optimal value of this constant.     

On single-sample robust detection of known signals with additive unknown-mean amplitude-bounded random interference--II: The randomized decision rule solution (Corresp.)

A randomized decision rule is derived and proved to be the saddlepoint solution of the robust detection problem for known signals in independent unknown-mean amplitude-bounded noise. The saddlepoint solutionphi^{0}uses an equaUy likely mixed strategy to chose one ofNBayesian single-threshold decision rulesphi_{i}^{0}, i = 1,cdots , Nhaving been obtained previously by the author. These decision rules are also all optimal against the maximin (least-favorable) nonrandomized noise probability densityf_{0}, wheref_{0}is a picket fence function withNpickets on its domain. Thee pair(phi^{0}, f_{0})is shown to satisfy the saddlepoint condition for probability of error, i.e.,P_{e}(phi^{0} , f) leq P_{e}(phi^{0} , f_{0}) leq P_{e}(phi, f_{0})holds for allfandphi. The decision rulephi^{0}is also shown to be an eqoaliir rule, i.e.,P_{e}(phi^{0}, f ) = P_{e}(phi^{0},f_{0}), for allf, with4^{-1} leq P_{e}(phi^{0},f_{0})=2^{-1}(1-N^{-1})leq2^{-1} , N geq 2. Thus nature can force the communicator to use an {em optimal} randomized decision rule that generates a large probability of error and does not improve when less pernicious conditions prevail.     

Optimum waveform signal sets with amplitude and energy constraints

It is desirable to choose the waveforms making up a signaling alphabet so that they are maximally separated one from another. This problem is considered, in the space of square-integrable functions, for signals which have finite duration, and are constrained in the ranges of their values as well as in energy. Corresponding to each of the following cases, we establish sharp bounds for the minimum distance and for the average distance between elements of a fixed size signal set, and construct sets of signals that attain both bounds simultaneously. begin{list} item {em Case A (Energy Constraint Only):} The average energy of the waveforms in the signal set is at mostsigma, where0 leq sigma < infty. item {em Case B (Energy and Peak Amplitude Constraints):} The average energy of the waveforms in the signal set isleq sigma (0 leq sigma < 1), and the absolute value of each waveform is at most1. item {em Case C (Energy and Value Constraints):} The average energy of the waveforms in the signal set is at mostb^{2}sigma + a^{2}(1 - sigma), and each waveform takes values in the set[a, b], where0 leq a < b < infty, and0 leq sigma leq 1. end{list} Cases A and B are applicable to signal design for communication in channels with additive noise (say Gaussian), and Case C is applicable to signal design for optical channels, where the signal represents the intensity of a photon stream. The general character of the results is that the minimum distance behaves likegamma sigmain Cases A and B, and likegamma sigma (1 - sigma)in Case C, withgammaa suitable constant.     

An upper bound for codes for the noisy two-access binary adder channel (Corresp.)

Using earlier methods a combinatorial upper bound is derived for|C|. cdot |D|, where(C,D)is adelta-decodable code pair for the noisy two-access binary adder channel. Asymptotically, this bound reduces toR_{1}=R_{2} leq frac{3}{2} + elog_{2} e - (frac{1}{2} + e) log_{2} (1 + 2e)= frac{1}{2} - e + H(frac{1}{2} - e) - frac{1}{2}H(2e),wheree = lfloor (delta - 1)/2 rfloor /n, n rightarrow inftyandR_{1}resp.R_{2}is the rate of the codeCresp.D.     

Hypothesis testing with multiterminal data compression

The multiterminal hypothesis testingH: XYagainstH̄: X̄Ȳis considered whereX^{n} (X̄^{n})andY^{n} (Ȳ^{n})are separately encoded at ratesR_{1}andR_{2}, respectively. The problem is to determine the minimumbeta_{n}of the second kind of error probability, under the condition that the first kind of error probabilityalpha_{n} leq epsilonfor a prescribed0 < epsilon < 1. A good lower boundtheta_{L}(R_{1}, R_{2})on the power exponenttheta (R_{1}, R_{2},epsilon)= lim inf_{n rightarrow infty}(-1/n log beta_{n})is given and several interesting properties are revealed. The lower bound is tighter than that of Ahlswede and Csiszár. Furthermore, in the special case of testing against independence, this bound turns out to coincide with that given by them. The main arguments are devoted to the special case withR_{2} = inftycorresponding to full side information forY^{n}(Ȳ^{n}). In particular, the compact solution is established to the complete data compression cases, which are useful in statistics from the practical point of view.     

Performance of anM-ary orthogonal communication system using stationary stochastic signals

This paper considers the performance of a communication system which transmits forTseconds the real part of a sample function of one ofMstationary complex Gaussian processes whose spectral densities are all frequency translations of the functionS_{xi (f). At the receiver white Gaussian noise of one-sided densityN_{0}is added. The center frequencies of the processes are assumed to be sufficiently separated that theMcovariance functions are orthogonal overT. Exponently tight bounds are obtained for the error probability of the maximum likelihood receiver. It is shown that the error probability approaches zero exponentially withTfor all ratesR = (ln M)/Tup toC= int_{-infty}^{infty} [S_{xi (f)/N_{0}] df - int_{- infty}^{infty} ln [1 + S_{xi}(f)/N_{0}] dfwhich is shown to be the channel capacity. Similar results are obtained for the case of stochastic signals with specular components.     

Improvements of Winograd's result on computation in the presence of noise (Corresp.)

Winograd's result concerning Elias' model of computation in the presence of noise can be stated without reference to computation. If a codevarphi: {0,1}^{k} rightarrow {0,1}^{n}is min-preserving(varphi (a wedge b) = varphi (a) wedge varphi (b)fora,b in {0,1}^{k})andepsilon n-error correcting, then the ratek/n rightarrow 0ask rightarrow infty. This result is improved and extended in two directions. begin{enumerate} item For min-preserving codes with {em fixed} maximal (and also average) error probability on a binary symmetric channel againk/n rightarrow 0ask rightarrow infty(strong converses). item Second, codes with lattice properties without reference to computing are studied for their own sake. Already for monotone codes( varphi (a) leq varphi (b)fora leq b)the results in direction 1) hold for maximal errors. end{enumerate} These results provide examples of coding theorems in which entropy plays no role, and they can be reconsidered from the viewpoint of multiuser information theory.     

On certain projective geometry codes (Corresp.)

LetVbe an(n, k, d)binary projective geometry code withn = (q^{m}-1)/(q - 1), q = 2^{s}, andd geq [(q^{m-r}-1)/(q - 1)] + 1. This code isr-step majority-logic decodable. With reference to the GF(q^{m}) = {0, 1, alpha , alpha^{2} , cdots , alpha^{n(q-1)-1} }, the generator polynomialg(X), ofV, hasalpha^{nu}as a root if and only ifnuhas the formnu = i(q - 1)andmax_{0 leq l < s} W_{q}(2^{l} nu) leq (m - r - 1)(q - 1), whereW_{q}(x)indicates the weight of the radix-qrepresentation of the numberx. LetSbe the set of nonzero numbersnu, such thatalpha^{nu}is a root ofg(X). LetC_{1}, C_{2}, cdots, C_{nu}be the cyclotomic cosets such thatSis the union of these cosets. It is clear that the process of findingg(X)becomes simpler if we can find a representative from eachC_{i}, since we can then refer to a table, of irreducible factors, as given by, say, Peterson and Weldon. In this correspondence it was determined that the coset representatives for the cases ofm-r = 2, withs = 2, 3, andm-r=3, withs=2.     

The solutions of nonaxisymmetric fields in SELFOC fibers with longitudinal imperfections

The solutions of nonaxisymmetric fields in SELFOC fibers with longitudinal imperfections are investigated. The refractive index is assumed to be of the formK = varepsilon/varepsilon_{0} = K_{0} - K_{2}(z) r^{2}(K_{2}(z)is a gradually varying function ofz). The analytic solution is obtained. When K2is a constant, the solution is reduced to that of perfect SEL-FOC fibers, which is well known. Further the relations of modes conversion in imperfect fibers are obtained.     

Linear feedback rate bounds for regressive channels (Corresp.)

This article presents new tighter upper bounds on the rate of Gaussian autoregressive channels with linear feedback. The separation between the upper and lower bounds is small. We havefrac{1}{2} ln left( 1 + rho left( 1+ sum_{k=1}^{m} alpha_{k} x^{- k} right)^{2} right) leq C_{L} leq frac{1}{2} ln left( 1+ rho left( 1+ sum_{k = 1}^{m} alpha_{k} / sqrt{1 + rho} right)^{2} right), mbox{all rho}, whererho = P/N_{0}W, alpha_{l}, cdots, alpha_{m}are regression coefficients,Pis power,Wis bandwidth,N_{0}is the one-sided innovation spectrum, andxis a root of the polynomial(X^{2} - 1)x^{2m} - rho left( x^{m} + sum^{m}_{k=1} alpha_{k} x^{m - k} right)^{2} = 0.It is conjectured that the lower bound is the feedback capacity.     

Robust prediction of band-limited signals from past samples (Corresp.)

For a complex-valued deterministic signal of finite energy band-limited to the normalized frequency band|w| leq piexplicit coefficients{a_{kn}}are found such that for anyTsatisfying0 < T leq 1/2,left| f(t)-sum^{2n}_{k=1}a_{kn}f(t - kT)right| leq E_{f}cdot beta^{n}whereE_{f}is the signal energy andbeta doteq 0.6863. Thus the estimate off(t)in terms of2npast samples taken at a rate equal to or in excess of twice the Nyquist rate converges uniformly at a geometric rate tof(t)on(- infty , infty). The suboptimal coefficients{a_{kn}}have the desirable property of being pure numbers independent of both the particular band-limited signal and of the selected sampling rate1/T. It is also shown that these same coefficients can be used to estimate the value ofx(t)of a wide-sense stationary random process in terms of past samples.     

An approximation to the weight distribution of binary primitive BCH codes with designed distances 9 and 11 (Corresp.)

Recently Kasami {em et al.} presented a linear programming approach to the weight distribution of binary linear codes [2]. Their approach to compute upper and lower bounds on the weight distribution of binary primitive BCH codes of length2^{m} - 1withm geq 8and designed distance2t + 1with4 leq t leq 5is improved. From these results, the relative deviation of the number of codewords of weightjleq 2^{m-1}from the binomial distribution2^{-mt} left( stackrel{2^{m}-1}{j} right)is shown to be less than 1 percent for the following cases: (1)t = 4, j geq 2t + 1andm geq 16; (2)t = 4, j geq 2t + 3and10 leq m leq 15; (3)t=4, j geq 2t+5and8 leq m leq 9; (4)t=5,j geq 2t+ 1andm geq 20; (5)t=5, j geq 2t+ 3and12 leq m leq 19; (6)t=5, j geq 2t+ 5and10 leq m leq 11; (7)t=5, j geq 2t + 7andm=9; (8)t= 5, j geq 2t+ 9andm = 8.     

Sequences achieving the boundary of the entropy region for a two-source are virtually memoryless (Corresp.)

For a joint distribution{rm dist}(X,Y), the functionT(t)=min { H(Y|U): I(U wedge Y|X)=O, H(X|U)geq t}is an important characteristic. It equals the asymptotic minimum of(1/n)H(Y^{n})for random pairs of sequences(X^{n}, Y^{n}), wherefrac{1}{n} sum ^{n}_{i=1}{rm dist} X_{i} sim {rm dist} X, {rm dist} Y^{n}|X^{n} = ({rm dist} Y|X)^{n}, frac{1}{n}H(X^{n})geq t.We show that if, for(X^{n}, Y^{n})as given, the rate pair[(1/n)H(X^{n}),(1/n)H(Y^{n})]approaches the nonlinear part of the curve(t,T(t)), then the sequenceX^{n}is virtually memoryless. Using this, we determine some extremal sections of the rate region of entropy characterization problems and find a nontrivial invariant for weak asymptotic isomorphy of discrete memoryless correlated sources.     

Writing on dirty paper (Corresp.)

A channel with outputY = X + S + Zis examined, The stateS sim N(0, QI)and the noiseZ sim N(0, NI)are multivariate Gaussian random variables (Iis the identity matrix.). The inputX in R^{n}satisfies the power constraint(l/n) sum_{i=1}^{n}X_{i}^{2} leq P. IfSis unknown to both transmitter and receiver then the capacity isfrac{1}{2} ln (1 + P/( N + Q))nats per channel use. However, if the stateSis known to the encoder, the capacity is shown to beC^{ast} =frac{1}{2} ln (1 + P/N), independent ofQ. This is also the capacity of a standard Gaussian channel with signal-to-noise power ratioP/N. Therefore, the stateSdoes not affect the capacity of the channel, even thoughSis unknown to the receiver. It is shown that the optimal transmitter adapts its signal to the stateSrather than attempting to cancel it.     

Evaluation of the Q Function

A convenient method of evaluating theQfunction over the parameter space quarter plane is presented. TheQfunction is first expressed as an infinite series. TheNterm truncated seriesQ_{N}(a, b)is used to approximateQ(a, b)fora^{2} + b^{2} leq Rwhere the choice ofNdepends on the accuracy desired andRis determined by considerations such as computer bit capacity, computational time, and accuracy. Fora^{2} + b^{2} > R, alternate expressions are used. Whenb - a geq d, we approximateQby 0, and whenb - a leq d, we approximateQby 1. The accuracy is dependent on the choice of the constantd. In the reniainder of the quarter plane,a^{2} + b^{2} > Rand| b - a < d, and an efficient expression is used, but it is of limited accuracy (from 10^-5to 10^-9) near the linea = b.     

Asymptotics and Error Rate Bounds for M-ary DPSK

The symbol error probabilityP_{E}(M)forM-ary DPSK is shown to be bounded in terms of a recent asymptotic approximationP_{asym}(M)by the inequalitiesP_{asym}(M) < P_{E}(M) < 1.03P_{asym}(M);M geq 4, E_{b}/N_{0} geq 1whereE_{b}/N_{0}is the bit energy-to-noise spectral density ratio. Aside from the wide range of validity and the closeness of the lower and upper bounds, this result is striking in light of the often held view that such asymptotic approximations are primarily of value only in the limitE_{b}/N_{0} rightarrow infty; thus, one of the goals of this note is to demonstrate that asymptotic methods can lead to extremely good error rate approximations in lieu of the more traditional and more widely used bounding techniques. The results are also noted to be applicable in other similar situations which commonly occur.