Balancing and model reduction for second-order form linear systems

Model reduction of second-order form linear systems is considered where a second-order form reduced model is desired, The focus is on reduction methods that employ or mimic Moore's balance and truncate (1981). First, we examine second-order form model reduction by conversion to first-order form and obtain a complete solution for this problem. Then, new Gramians and input/output (I/O) invariants for second-order systems are motivated and defined. Based on these, direct second-order balancing methods are developed. This leads naturally to direct second-order form analogs for the well-known first-order form balance and truncate model reduction method. Explicit algorithms are given throughout the paper     

The communication-hiding pipelined BiCGstab method for the parallel solution of large unsymmetric linear systems

A High Performance Computing alternative to traditional Krylov subspace methods, pipelined Krylov subspace solvers offer better scalability in the strong scaling limit compared to standard Krylov subspace methods for large and sparse linear systems. The typical synchronization bottleneck is mitigated by overlapping time-consuming global communication phases with local computations in the algorithm. This paper describes a general framework for deriving the pipelined variant of any Krylov subspace algorithm. The proposed framework was implicitly used to derive the pipelined Conjugate Gradient (p-CG) method in Hiding global synchronization latency in the preconditioned Conjugate Gradient algorithm by P. Ghysels and W. Vanroose, Parallel Computing, 40(7):224–238, 2014. The pipelining framework is subsequently illustrated by formulating a pipelined version of the BiCGStab method for the solution of large unsymmetric linear systems on parallel hardware. A residual replacement strategy is proposed to account for the possible loss of attainable accuracy and robustness by the pipelined BiCGStab method. It is shown that the pipelined algorithm improves scalability on distributed memory machines, leading to significant speedups compared to standard preconditioned BiCGStab.     

Pipelined robust M-estimate adaptive second-order Volterra filter against impulsive noise

The main limits on adaptive Volterra filters are their computational complexity in practical implementation and significant performance degradation under the impulsive noise environment. In this paper, a low-complexity pipelined robust M-estimate second-order Volterra (PRMSOV) filter is proposed to reduce the computational burdens of the Volterra filter and enhance the robustness against impulsive noise. The PRMSOV filter consists of a number of extended second-order Volterra (SOV) modules without feedback input cascaded in a chained form. To apply to the modular architecture, the modified normalized least mean M-estimate (NLMM) algorithms are derived to suppress the effect of impulsive noise on the nonlinear and linear combiner subsections, respectively. Since the SOV-NLMM modules in the PRMSOV can operate simultaneously in a pipelined parallelism fashion, they can give a significant improvement of computational efficiency and robustness against impulsive noise. The stability and convergence on nonlinear and linear combiner subsections are also analyzed with the contaminated Gaussian (CG) noise model. Simulations on nonlinear system identification and speech prediction show the proposed PRMSOV filter has better performance than the conventional SOV filter and joint process pipelined SOV (JPPSOV) filter under impulsive noise environment. The initial convergence, steady-state error, robustness and computational complexity are also better than the SOV and JPPSOV filters.     

A second-order maximum principle for discrete-time bilinear control systems with applications to discrete-time linear switched systems

A powerful approach for analyzing the stability of continuous-time switched systems is based on using optimal control theory to characterize the “most unstable” switching law. This reduces the problem of determining stability under arbitrary switching to analyzing stability for the specific “most unstable” switching law. For discrete-time switched systems, the variational approach received considerably less attention. This approach is based on using a first-order necessary optimality condition in the form of a maximum principle (MP), and typically this is not enough to completely characterize the “most unstable” switching law. In this paper, we provide a simple and self-contained derivation of a second-order necessary optimality condition for discrete-time bilinear control systems. This provides new information that cannot be derived using the first-order MP. We demonstrate several applications of this second-order MP to the stability analysis of discrete-time linear switched systems.     

Subanalytic solutions of linear difference equations and multidimensional hypergeometric sequences

We consider linear difference equations with polynomial coefficients over C and their solutions in the form of sequences indexed by the integers (sequential solutions). We investigate the C-linear space of subanalytic solutions, i.e., those sequential solutions that are the restrictions to Z of some analytic solutions of the original equation. It is shown that this space coincides with the space of the restrictions to Z of entire solutions and that the dimension of this space is equal to the order of the original equation.We also consider d-dimensional (d≥1) hypergeometric sequences, i.e., sequential and subanalytic solutions of consistent systems of first-order difference equations for a single unknown function. We show that the dimension of the space of subanalytic solutions is always at most 1, and that this dimension may be equal to 0 for some systems (although the dimension of the space of all sequential solutions is always positive).Subanalytic solutions have applications in computer algebra. We show that some implementations of certain well-known summation algorithms in existing computer algebra systems work correctly when the input sequence is a subanalytic solution of an equation or a system, but can give incorrect results for some sequential solutions.     

A maximally pipelined tridiagonal linear equation solver

A new method of solving tridiagonal systems of linear equations is proposed which introduces parallelism in a way that may be effectively exploited by a suitable parallel computer architecture. The scheme is based on the program transformation techniques which can produce machine code to be executed in a maximally pipelined fashion. Compared to the existing parallel solution techniques such as the cyclic reduction algorithms, the new method has the following advantages: (1) It eliminates the substantial data rearrangement overhead incurred by many existing parallel algorithms; (2) it sustains a relatively constant parallelism during various phases of program execution; (3) the code generation is independent of the length of the vectors to be computed, hence is more flexible; (4) in general the size of machine code is much smaller and hence is more efficient in terms of memory usage. The new method is also numerically stable. Using the new method, we outline the code structure of a maximally pipelined tridiagonal linear equation solver for a static data flow super-computer.     

Vector processing efficiency of plasma MHD codes by use of the FACOM 230-75 APU

In the framework of pipelined vector architecture, the efficiency of vector processing is assessed with respect to plasma MHD codes in nuclear fusion research. By using a vector processor, the FACOM 230-75 APU, the limit of the enhancement factor due to parallelism of current vector machines is examined for three numerical codes based on a fluid model. Reasonable speed-up factors of approximately 6,6 and 4 times faster than the highly optimized scalar version are obtained for ERATO (linear stability code), AEOLUS-R1 (nonlinear stability code) and APOLLO (1-1/2D transport code), respectively. Problems of the pipelined vector processors are discussed from the viewpoint of restructuring, optimization and choice of algorithms. In conclusion, the important concept of “concurrency within pipelined parallelism” is emphasized.     

Compiling techniques for first-order linear recurrences on a vector computer

Linear recurrences are the most important class of nonvectorizable problems in typical scientific/engineering calculations. This work discusses high-performance methods for solving first-order linear recurrences on a vector computer, investigates automatic transformations, and develops compiling techniques for first-order linear recurrence problems. The results show that the improved vector code generated by the vectorizing compiler on the HITAC S-820 supercomputer runs at the rate of 150 MFLOPS (million floating operations per second) for moderate loop lengths (>1000) and over 200 MFLOPS for long loop lengths (> 10000). Also, overall performance improvements of 69% in the 14 Lawrence Livermore Loops and 25 % in the 24 Lawrence Livermore Loops, as measured by the harmonic mean, are attained.     

递归最小二乘循环神经网络

针对循环神经网络(Recurrent neural networks, RNNs)一阶优化算法学习效率不高和二阶优化算法时空开销过大, 提出一种新的迷你批递归最小二乘优化算法. 所提算法采用非激活线性输出误差替代传统的激活输出误差反向传播, 并结合加权线性最小二乘目标函数关于隐藏层线性输出的等效梯度, 逐层导出RNNs参数的迷你批递归最小二乘解. 相较随机梯度下降算法, 所提算法只在RNNs的隐藏层和输出层分别增加了一个协方差矩阵, 其时间复杂度和空间复杂度仅为随机梯度下降算法的3倍左右. 此外, 本文还就所提算法的遗忘因子自适应问题和过拟合问题分别给出一种解决办法. 仿真结果表明, 无论是对序列数据的分类问题还是预测问题, 所提算法的收敛速度要优于现有主流一阶优化算法, 而且在超参数的设置上具有较好的鲁棒性.     

改进混合策略的多分辨率3D医学图像配准

基于混合策略的多分辨率算法是当前3D医学图像刚体配准中普遍采用的方法,不过其仅仅是优化算法的混合。通过研究不同分辨率对一阶互信息（常称为互信息）和二阶互信息配准的影响,在二级多分辨率策略的配准中,各级采用相对更适合的相似性测度,提出了混合优化算法和混合测度的改进算法。实验表明,改进算法在配准精度上达到了亚体素级,且明显优于基于单一测度的算法,在配准速度上远远快于基于二阶互信息单一测度的算法,略慢于基于一阶互信息单一测度的算法。     

Sequential operators in computability logic

Computability logic (CL) is a semantical platform and research program for redeveloping logic as a formal theory of computability, as opposed to the formal theory of truth which it has more traditionally been. Formulas in CL stand for (interactive) computational problems, understood as games between a machine and its environment; logical operators represent operations on such entities; and “truth” is understood as existence of an effective solution, i.e., of an algorithmic winning strategy.The formalism of CL is open-ended, and may undergo series of extensions as the study of the subject advances. The main groups of operators on which CL has been focused so far are the parallel, choice, branching, and blind operators, with the logical behaviors of the first three groups resembling those of the multiplicatives, additives and exponentials of linear logic, respectively. The present paper introduces a new important group of operators, called sequential. The latter come in the form of sequential conjunction and disjunction, sequential quantifiers, and sequential recurrences (“exponentials”). As the name may suggest, the algorithmic intuitions associated with this group are those of sequential computations, as opposed to the intuitions of parallel computations associated with the parallel group of operations. Specifically, while playing a parallel combination of games means playing all components of the combination simultaneously, playing a sequential combination means playing the components in a sequential fashion, one after one.The main technical result of the present paper is a sound and complete axiomatization of the propositional fragment of computability logic whose vocabulary, together with negation, includes all three — parallel, choice and sequential — sorts of conjunction and disjunction. An extension of this result to the first-order level is also outlined.     

三维非定常不可压涡量—速度Navier—Stokes方程组的有限差分法

提出了一种数值求解三维非定常涡量—速度形式的不可压Navier-Stokes方程组的有限差分方法,该方法在空间方向上具有二阶精度,并且系数矩阵具有对角占优性,因此适合高雷诺数问题的数值求解．同时,给出了适合的二阶涡量边界条件．通过对有精确解的狄利克雷边值问题和典型的驱动方腔流问题的数值实验,验证了本文格式的精确性、稳定性和有效性．     

基于序贯线性贝叶斯的RFID标签数量估计算法

为解决现有标签数量估计算法中估计精度与复杂度之间的矛盾,在分析比较现有算法的基础上,提出一种基于序贯线性贝叶斯的射频识别（RFID）标签数量估计算法。首先,基于线性贝叶斯理论,充分利用空闲、成功和碰撞时隙数量观测值及相关性,建立了标签数量估计问题的线性模型;然后,推导了标签数量估计值的闭式表达式,给出了表达式各阶统计量的序贯式求解方法;最后,对序贯式贝叶斯算法的计算复杂度进行了分析和对比。仿真结果表明,所提算法通过序贯贝叶斯方法提高了估计精度和识别效率,当观测时隙数为帧长一半时估计误差仅为4%。该算法以线性解析式形式更新标签数量估计值,避免了穷举搜索,与高精度的最大后验概率和马氏距离算法相比,计算复杂度由O（n²）和O（n）下降为O（1）。经理论分析和仿真验证,基于序贯线性贝叶斯的RFID标签数量估计算法兼具高精度和低复杂度的特性,能很好地满足硬件资源受限应用场景下对标签数量的估计需求。     

Parallel Recurrence Solvers for Vector and SIMD Supercomputers

In this paper we present two algorithms for the parallel solution of first-order linear recurrences, We show that the algorithms can be used to efficiently solve both scalar and blocked versions of the problem on vector and SIMD architectures. The first algorithm is a parallel approach whose resulting code can be explicitly vectorized, making it suitable for efficient execution on vector architectures such as the Cray 2. The second algorithm is a modified recursive approach designed to reduce the communication overhead encountered in SIMD architectures such as the Connection Machine 2 (CM-2). We present the performance exhibited by the parallel algorithm implementations on the Cray 2 and CM-2 for both scalar and blocked versions of the recurrence problem.     

A team algorithm for robust stability analysis and control design of uncertain time‐varying linear systems using piecewise quadratic Lyapunov functions

A team algorithm based on piecewise quadratic simultaneous Lyapunov functions for robust stability analysis and control design of uncertain time‐varying linear systems is introduced. The objective is to use robust stability criteria that are less conservative than the usual quadratic stability criterion. The use of piecewise quadratic Lyapunov functions leads to a non‐convex optimization problem, which is decomposed into a convex subproblem in a selected subset of decision variables, and a lower‐dimensional non‐convex subproblem in the remaining decision variables. A team algorithm that combines genetic algorithms (GA) for the non‐convex subproblem and interior‐point methods for the solution of linear matrix inequalities (LMI), which form the convex subproblem, is proposed. Numerical examples are given, showing the advantages of the proposed method. Copyright © 2001 John Wiley & Sons, Ltd.     

Confidence-weighted Learning for Feature Evolution

Several recent works have studied feature evolvable learning. They usually assume that features would not vanish or appear in an arbitrary way; instead, old features vanish and new features emerge as the hardware device collecting the data features is replaced. However, the existing learning algorithms for feature evolution only utilize the first-order information of data streams and ignore the second-order information which can reveal the correlations between features and thus significantly improve the classification performance. We propose a Confidence-Weighted learning for Feature Evolution (CWFE) algorithm to solve the aforementioned problem. First, second-order confidence-weighted learning is introduced to update the prediction model. Next, to make full use of the learned model, a linear mapping is learned in the overlapping period to recover the old features. Then, the existing model is updated with the recovered old features and, at the same time, a new prediction model is learned with the new features. Furthermore, two ensemble methods are introduced to utilize the two models. Finally, experimental studies show that the proposed algorithms outperform existing feature evolvable learning algorithms.     

A class of explicit two-step superstable methods for second-order linear initial value problems

A class of explicit two-step superstable methods of fourth algebraic order for the numerical solution of second-order linear initial value problems is presented in this article. We need Taylor expansion at an internal grid point and collocation formulae for the derivatives of the solution to derive a method and then modify it into a class of methods having the desired stability properties. Computational results are presented to demonstrate the applicability of the methods to some standard problems.     

FSR Methods for Second-Order Regression Models

Most variable selection techniques focus on first-order linear regression models. Often, interaction and quadratic terms are also of interest, but the number of candidate predictors grows very fast with the number of original predictors, making variable selection more difficult. Forward selection algorithms are thus developed that enforce natural hierarchies in second-order models to control the entry rate of uninformative effects and to equalize the false selection rates from first-order and second-order terms. Method performance is compared through Monte Carlo simulation and illustrated with data from a Cox regression and from a response surface experiment.     

Controllability of second-order infinite-dimensional systems

In the paper, the approximate controllability of linear abstract second-order infinite-dimensional dynamical systems is considered. It is proved using the frequency-domain method, that approximate controllability of second-order system can be verified by the approximate controllability conditions for the corresponding simplified first-order system. General results are then applied for approximate controllability investigation of a vibratory dynamical system modeling flexible mechanical structure. Some special cases are also considered. Moreover, remarks and comments on the relationships between different concepts of controllability are given. The paper extends earlier results on approximate controllability of second-order abstract dynamical systems.     

Estimating the concentration of optically active constituents of sea water by Takagi-Sugeno models with quadratic rule consequents

Determining the concentrations of chlorophyll, suspended particulate matter and coloured dissolved organic matter in the sea water is basic to support the monitoring of upwelling phenomena, algae blooms, and changes in the marine ecosystem. Since these concentrations affect the spectral distribution of the solar light back-scattered by the water body, their estimation can be computed by using a set of remotely sensed multispectral measurements of the reflected sunlight. In this paper, the relation between the concentrations of interest and the average subsurface reflectances is modelled by means of a set of second-order Takagi-Sugeno (TS) fuzzy rules. Unlike first-order TS rules, which adopt linear functions as consequent, second-order TS rules exploit quadratic functions, thus improving the modelling capability of the rule in the subspace determined by the antecedent. First, we show how we can build a second-order TS model through a simple transformation, which allows estimating the consequent parameters using standard linear least-squares algorithms, and by adopting one of the most used methods proposed in the literature to generate first-order TS models. Then, we compare first-order and second-order TS models against mean square error and interpretability of rules. We highlight how second-order TS models allow us to achieve better approximation than first-order TS models though maintaining interpretability of the rules. Finally, we show how second-order TS models perform considerably better (the mean square error is lower by two orders of magnitude) than the specific implementations of radial basis function networks and multi-layer perceptron networks used in previous papers for the same application domain.