A fundamental inequality between the probabilities of binary subgroups and cosets

The probability of a set of binaryn-tuples is defined to be the sum of the probabilities of the individualn-tuples when each digit is chosen independently with the same probabilitypof being a "one." It is shown that, under such a definition, the ratio between the probability of a subgroup of order2^{k}and any of its proper cosets is always greater than or equal to a functionF_{k}(p), whereF_{k}(p) geq 1forp leq frac{1}{2}with equality when and only whenp = frac{1}{2}. It is further shown thatF_{k}(p)is the greatest lower bound on this ratio, since a subgroup and proper coset of order2^{k}can always be found such that the ratio between their probabilities is exactlyF_{k}(p). It is then demonstrated that for a linear code on a binary symmetric channel the "tall-zero" syndrome is more probable than any other syndrome. This result is applied to the problem of error propagation in convolutional codes.     

An upper bound for codes for the noisy two-access binary adder channel (Corresp.)

Using earlier methods a combinatorial upper bound is derived for|C|. cdot |D|, where(C,D)is adelta-decodable code pair for the noisy two-access binary adder channel. Asymptotically, this bound reduces toR_{1}=R_{2} leq frac{3}{2} + elog_{2} e - (frac{1}{2} + e) log_{2} (1 + 2e)= frac{1}{2} - e + H(frac{1}{2} - e) - frac{1}{2}H(2e),wheree = lfloor (delta - 1)/2 rfloor /n, n rightarrow inftyandR_{1}resp.R_{2}is the rate of the codeCresp.D.     

An estimate of the variation of a band-limited process

Upper and lower bounds are established for the mean-square variation of a stationary processX(t)whose power spectrum is bounded byomega_{c}, in terms of its average powerP_{0}and the average powerP_{1}of its derivative. It is shown thatleft( frac{2}{pi} right)^{2} P_{1} tau^{2} leq E {|X(t+tau )-X(t)|^{2}} leq P_{1} tau^{2} leq omega_{c}^{2}P_{0}tau^{2}where the upper bounds are valid for anytauand the lower bound fortau < pi / omega_{c}. These estimates are applied to the mean-square variation of the envelope of a quasi-monochromatic process.     

On the capacity of certain additive non-Gaussian channels

A model of an additive non-Gaussian noise channel with generalized average input energy constraint is considered. The asymptotic channel capacityC_{zeta}(S), for large signal-to-noise ratioS, is found under certain conditions on the entropyH_{ tilde{ zeta}}( zeta)of the measure induced in function space by the noise processzeta, relative to the measure induced bytilde{zeta}, where is a Gaussian process with the same covariance as that ofzeta. IfH_{ tilde{zeta}}( zeta) < inftyand the channel input signal is of dimensionM< infty, thenC_{ zeta}(S)= frac{1}{2}M ln(1 + S/M) + Q_{zeta}( M ) + {o}(1), where0 leq Q_{ zeta}( M ) leq H_{ tilde{ zeta}}( zeta). If the channel input signal is of infinite dimension andH_{ tilde{ zeta}}( zeta) rightarrow 0forS rightarrow infty, thenC_{ zeta}(S) = frac{1}{2}S+{o}(1).     

Construction of de Bruijn sequences of minimal complexity

It is well known that the linear complexity of a de Bruijn sequenceSof length2^{n}is bounded below by2^{n- 1} + nforn geq 3. It is shown that this lower bound is attainable for alln.     

On the truncation error of the cardinal sampling expansion

Modern communication theory and practice are heavily dependent on the representation of continuous parameter signals by linear combinations, involving a denumerable set of random variables. Among the best known and most useful is the cardinal seriesf_{n} (t) = sum^{+n}_{-n} f(k) frac{sin pi (t - k)}{ pi ( t - k )}for deterministic functions and wide-sense stationary stochastic processes bandlimited to(-pi, pi). When, as invariably occurs in applications, samplesf(k)are available only over a finite period, the resulting finite approximation is subject to a truncation error. For functions which areL_{1}Fourier transforms supported on[-pi + delta, + pi - delta], uniform trunction error bounds of the formO(n^{-1})are known. We prove that analogousO(n^{-1})bounds remain valid without the guard banddeltaand for Fourier-Stieltjes transforms; we require only a bounded variation condition in the vicinity of the endpoints- piand+ piof the basic interval. Our methods depend on a Dirichlet kernel representation forf_{n}(t)and on properties of functions of bounded variation; this contrasts with earlier approaches involving series or complex variable theory. Other integral kernels (such as the Fejer kernel) yield certain weighted truncated cardinal series whose errors can also be bounded. A mean-square trunction error bound is obtained for bandlimited wide-sense stationary stochastic processes. This error estimate requires a guard band, and leads to a uniformO(n^{-2})bound. The approach again employs the Dirichlet kernel and draws heavily on the arguments applied to deterministic functions.     

Temperature dependence of MOSFET characteristics in weak inversion

A knowledge of the MOSFET operating in weak inversion is important for circuits with low leakage specifications. This paper discusses the effect of temperature on the MOSFET in weak inversion. The reciprocal slopenof the log IDSversus VGSrelationship between source-drain current IDSand gate bias VGSmay be given byfrac{1}{(n - 1 - gamma)^{2}} = frac{2Cmin{ox}max{2}}{qepsilon_{s}N_{B}} [frac{3}{4} frac{E_{g^{0}}{q} - (frac{3}{2}alpha + frac{k}{q})T]withalpha equiv (k/q)(38.2 - ln N_{B} + (3/2) ln T)and γ ≡C_{ss}/C_{ox}, where Coxis the oxide capacitance per unit area, Cssthe surface states capacitance per unit area,qthe electronic charge, εsthe permittivity of silicon, NBthe bulk doping concentration,kthe Boltzmann's constant,Tthe absolute temperature, andE_{g0}the extrapolated value of the energy gap of lightly doped silicon atT = 0K. This theoretical formula was in good agreement with experimental results in a temperature range of interest.     

Equipartition laws of thermal noise

Classically, the thermal noise in electricalRCcircuits andLCRseries circuits is governed by the equipartition lawfrac{1}{2}overline{CV^{2}} = frac{1}{2}kT, whereV(t)is the noise voltage developed acrossC. When quantum effects are taken into account, the equipartition law no longer holds forRCcircuits, although an equipartition law can be deemed for the measured mean square noise voltage under certain conditions. InLCRseries circuits the equipartition lawfrac{1}{2}overline{CV^{2}} = frac{1}{2}kT, changes intofrac{1}{2}overline{CV^{2}} = frac{1}{2}bar{E}(f_{0})for high-Qtuned circuits, wherebar{E}(f_{0})is the average energy of a harmonic oscillator tuned at the tuning frequency of the tuned circuit.     

Narrow-band systems and Gaussianity

The approach to Gaussianity of the outputy(t)of a narrow-band systemh(t)is investigated. It is assumed that the inputx(t)is ana-dependent process, in the sense that the random variablesx(t)andx(t + u)are independent foru > a. WithF(y)andG(y)the distribution functions ofy(t)and of a suitable normal process, a realistic boundBon the differenceF(y) -- G(y)is determined, and it is shown thatB rightarrow 0as the bandwidthomega_oof the system tends to zero. In the special case of the shot noise process begin{equation} y(t) = sum_i h(t - t_i) end{equation} it is shown that begin{equation} mid F(y) - G(y) mid < (omega_o/lambda) frac{1}{2} end{equation} wherelambda_iis the average density of the Poisson pointst_i.     

An improved error bound for equal-strength diversity channels (Corresp.)

An upper bound on the minimum probability of error for an equal-strength diversity channel is simply derived that improves a previously known bound by the factor[4(1 - p)]^(-1),0 leq p leq frac{1}{2}.     

Bounds for truncation error in sampling expansions of band-limited signals

Forf(t)a real-valued signal band-limited to- pi r leq omega leq pi r (0 < r < 1)and represented by its Fourier integral, upper bounds are established for the magnitude of the truncation error whenf(t)is approximated at a generic timetby an appropriate selection ofN_{1} + N_{2} + 1terms from its Shannon sampling series expansion, the latter expansion being associated with the full band[-pi, pi]and thus involving samples offtaken at the integer points. Results are presented for two cases: 1) the Fourier transformF(omega)is such that|F(omega)|^{2}is integrable on[-pi, pi r](finite energy case), and 2)|F(omega)|is integrable on[-pi r, pi r]. In case 1) it is shown that the truncation error magnitude is bounded above byg(r, t) cdot sqrt{E} cdot left( frac{1}{N_{1}} + frac{1}{N_{2}} right)whereEdenotes the signal energy andgis independent ofN_{1}, N_{2}and the particular band-limited signal being approximated. Correspondingly, in case 2) the error is bounded above byh(r, t) cdot M cdot left( frac{1}{N_{1}} + frac{1}{N_{2}} right)whereMis the maximum signal amplitude andhis independent ofN_{1}, N_{2}and the signal. These estimates possess the same asymptotic behavior as those exhibited earlier by Yao and Thomas [2], but are derived here using only real variable methods in conjunction with the signal representation. In case 1), the estimate obtained represents a sharpening of the Yao-Thomas bound for values ofrdose to unity.     

A First-Order Markov Model for Understanding Delta Modulation Noise Spectra

A first-order Markov process is used to model the sequence of quantization noise samples in delta modulation. An autocorrelation parameterCin the Markov model controls the shape of the noise spectrum, and asCdecreases from 1 to 0 and then to -1, the spectrum changes from a low-pass to a flat, and then to a high-pass characteristic. One can also use the Markov model to predict the so-called out-of-band noise rejection that is obtained when delta modulation is performed with an oversampled input, and the resulting quantization noise is lowpass filtered to the input band. The noise rejectionGis a function ofCas well as an oversampling factorFand an interesting asymptotic result is thatG=frac{1-C}{1+C} dot FifF gg frac{1+C}{1-C} dot frac{pi}{2}. Delta modulation literature has noted the importance of the specialGvalues,Fand2F. These correspond to autocorrelation values of 0 and -1/3.     

An achievable bound for optimal noiseless coding of a random variable (Corresp.)

For a discreteN-valued random variable (Npossibly denumerably infinite) Leung-Yan-Cheong and Cover have given bounds for the minimal expected length of a one-to-one (not necessarily uniquely decodable) codeL_{1:1}=sum_{i=1}^{N} p_{i} log left( frac{1}{2} + 1 right).It is shown that the best possible case occurs for certain denumerably infinite sets of nonzero probabilities. This absolute bound is related to the Shannon entropyHof the distribution by(h (cdot)is the binary entropy function).     

Short convolutional codes with maximal free distance for rates 1/2, 1/3, and 1/4 (Corresp.)

This paper gives a tabulation of binary convolutional codes with maximum free distance for ratesfrac{1}{2}, frac{1}{3}, andfrac{1}{4}for all constraint lengths (measured in information digits)nuup to and includingnu = 14. These codes should be of practical interest in connection with Viterbi decoders.     

A Gilbert bound for periodic binary convolutional codes

A Gilbert bound for periodic binary convolutional (PBC) codes is established. This bound shows that regardless of previous decoding decisions any fraction of errors less thanalpha/2can be corrected in a constraint length by some PBC code if the constraint length is sufficiently large andR, the code rate, is less than[1 - H(alpha)]/2, 0 leq alpha < frac{1}{2}.     

An upper bound to the capacity of the band-limited Gaussian autoregressive channel with noiseless feedback

Upper bounds to the capacity of band-limited Gaussianmth-order autoregressive channels with feedback and average energy constraintEare derived. These are the only known hounds on one- and two-way autoregressive channels of order greater than one. They are the tightest known for the first-order case. In this case letalpha_1be the regression coefficient,sigma^2the innovation variance,Nthe number of channel iterations per source symbol, ande = E/N; then the first-order capacityC^1is bounded by begin{equation} C^1 leq begin{cases} frac{1}{2} ln [frac{e}{sigma^2}(1+ mid alpha_1 mid ) ^ 2 +1], & frac{e}{sigma^2} leq frac{1}{1- alpha_1^2} \ frac{1}{2} ln [frac{e}{sigma^2} + frac{2mid alpha_1 mid}{sqrt{1-alpha_1^2}} sqrt{frac{e}{simga^2}} + frac{1}{1-alpha_1^2}], & text{elsewhere}.\ end{cases} end{equation} This is equal to capacity without feedback for very low and very highe/sigma^2and is less than twice this one-way capacity everywhere.     

A theorem on the entropy of certain binary sequences and applications--I

In this, the first part of a two-part paper, we establish a theorem concerning the entropy of a certain sequence of binary random variables. In the sequel we will apply this result to the solution of three problems in multi-user communication, two of which have been open for some time. Specifically we show the following. LetXandYbe binary randomn-vectors, which are the input and output, respectively, of a binary symmetric channel with "crossover" probabilityp_0. LetH{X}andH{ Y}be the entropies ofXandY, respectively. Then begin{equation} begin{split} frac{1}{n} H{X} geq h(alpha_0), qquad 0 leq alpha_0 &leq 1, Rightarrow \ qquad qquad &qquad frac{1}{n}H{Y} geq h(alpha_0(1 - p_0) + (1 - alpha_0)p_0) end{split} end{equation} whereh(lambda) = -lambda log lambda - (1 - lambda) log(l - lambda), 0 leq lambda leq 1.     

A Gaussian channel with slow fading (Corresp.)

An interleaved fading channel whose state is known to the receiver is analyzed. The reliability functionE(R)is obtained for ratesRin the rangeR_c leq R leq C. The capacity is shown to beC = E_A { frac{1}{2} ln (1 + A^2 n)}whereAis a factor describing the fading mechanism anduis the signal-to-noise ratio per dimension.     

Sequences achieving the boundary of the entropy region for a two-source are virtually memoryless (Corresp.)

For a joint distribution{rm dist}(X,Y), the functionT(t)=min { H(Y|U): I(U wedge Y|X)=O, H(X|U)geq t}is an important characteristic. It equals the asymptotic minimum of(1/n)H(Y^{n})for random pairs of sequences(X^{n}, Y^{n}), wherefrac{1}{n} sum ^{n}_{i=1}{rm dist} X_{i} sim {rm dist} X, {rm dist} Y^{n}|X^{n} = ({rm dist} Y|X)^{n}, frac{1}{n}H(X^{n})geq t.We show that if, for(X^{n}, Y^{n})as given, the rate pair[(1/n)H(X^{n}),(1/n)H(Y^{n})]approaches the nonlinear part of the curve(t,T(t)), then the sequenceX^{n}is virtually memoryless. Using this, we determine some extremal sections of the rate region of entropy characterization problems and find a nontrivial invariant for weak asymptotic isomorphy of discrete memoryless correlated sources.