On the design of high-complexity cosine-modulated transmultiplexers based on the frequency-response masking approach

Two efficient techniques exploiting the frequency-response masking (FRM) approach are proposed in order to make it feasible to design prototype filters for highly selective nearly perfect-reconstruction cosine-modulated transmultiplexers and filter banks (CMTs and CMFBs) having a very large number of channels. In these design schemes, the number of unknowns is drastically reduced when compared with the corresponding techniques for designing direct-form prototype filters. Furthermore, in the proposed techniques, the main figures of merits, that is, the intersymbol interference and the interchannel interference for CMTs and the overall and aliasing distortions for CMFBs are taken into account in a controlled manner. In order to speed up the convergence of these two optimization techniques, simplifications for computing the resulting nonlinear constraints and the corresponding gradient vectors are proposed. They differ from each other in the sense that the first and second ones utilize the frequency-domain and time-domain constraints for controlling the figures of merit, respectively. Combining these two techniques results in numerically efficient algorithms for designing optimized CMTs (or CMFBs) with a reduced computational complexity (number of arithmetic operations per output sample), particularly when both branches of the FRM structure are required. Design examples are included illustrating the efficiency of the design methods and the high performance of the resulting CMT structures.