Ensemble of surrogates

The custom in surrogate-based modeling of complex engineering problems is to fit one or more surrogate models and select the one surrogate model that performs best. In this paper, we extend the utility of an ensemble of surrogates to (1) identify regions of possible high errors at locations where predictions of surrogates widely differ, and (2) provide a more robust approximation approach. We explore the possibility of using the best surrogate or a weighted average surrogate model instead of individual surrogate models. The weights associated with each surrogate model are determined based on the errors in surrogates. We demonstrate the advantages of an ensemble of surrogates using analytical problems and one engineering problem. We show that for a single problem the choice of test surrogate can depend on the design of experiments.     

Real time average and variance with modulo arithmetic

The arithmetic mean, and sample variance, are the most common summary statistics of a data set. There are circumstances (within computational loops, and in real-time data analysis, for examples) where it is desirable to maintain a current estimate of the mean and variance. The “lossy” (sometimes called the “running”) mean and variance are frequently useful also. We present an algorithm for these applications, and show the provisions necessary for use with cyclical (e.g. phase-angle) data.     

小波去噪与平均算法去噪在拉曼光谱中的应用

拉曼光谱分析中,噪声的存在常影响分析的准确度和检测限,现有滤波方法在光谱信号除噪方面有种种缺陷.使用自适应小波阈值函数滤噪法和平移不变量小波去噪法两种方法,并分别与传统方法平均算法相结合,实现了信号与噪音的有效分离,均取得了很好的效果.即使对信噪比小于1的高噪声信号也能够很好地保留了信号的细节,获取满意的处理结果.     

Specifying recursive agents with GDTs

The purpose of this article is to formalise the notion of recursive agent by extending the Goal Decomposition Tree formalism (GDT). A formal semantics of this decomposition is given, as well as the definition of operators to introduce various ways of recursively defining agents. Design patterns, that show various use cases for recursive agents, are also presented. Finally, to preserve the essential GDT characteristics (that is to allow the verification of agents behaviours), we give proof schemas that allow a proof of the correctness of a recursive agent.     

Robustized recursive estimation with applications

In this paper two convergence theorems for the robustized scalar versions of the Robbins-Monro and the Kiefer-Wolfowitz procedures are presented. In addition, two information processing oriented examples are included which illustrate these recursive estimators and their robustness.     

Failure of pseudo-periodic surrogates

A surrogate test is a method for determining the properties of time-series data. Some methods have been proposed to generate surrogate data that can be used to determine whether a pseudo-periodic time series has deterministic properties beyond pseudo-periodicity. Luo’s method is one such method. In this article, Luo’s method and the problems associated with it are discussed. In this method, surrogate datasets are produced by adding the time-shifted data to the original data. Consequently, the pseudo-periodicity of the time series is presumably preserved, but the fine structure related to the deterministic properties is destroyed. Luo’s method gives correct results for many data. However, it generates incorrect results for certain time series, for example, the time series of the Rössler chaotic attractor and phase-shifted sinusoidal waves. To overcome this problem, we propose an alternative method based on the Poincaré section.     

Fast surrogates of U-statistics

U-statistics have long been known as a class of nonparametric estimators with good theoretical properties such as unbiasedness and asymptotic normality. However, their applications in modern statistical analysis are limited due to the high computational complexity, especially when massive data sets are becoming more and more common nowadays. In this paper, using the “divide-and-conquer” technique, we developed two surrogates of the U-statistics, aggregated U-statistics and average aggregated U-statistics, both of which are shown asymptotically equivalent to U-statistics and computationally much more efficient. When dividing the raw data set into K subsets, the two proposed estimators reduce the computational complexity from O(N^m) to O(K(N/K)^m), which results in significant time reduction as long as K=o(N) and m≥2. The merit of the two proposed statistics is demonstrated by both simulation studies and real data examples.     

基于多核集成的在线半监督学习方法

在很多实时预测任务中,学习器需对实时采集到的数据在线地进行学习.由于数据采集的实时性,往往难以为采集到的所有数据提供标记.然而,目前的在线学习方法并不能利用未标记数据进行学习,致使学得的模型并不能即时反映数据的动态变化,降低其实时响应能力.提出一种基于多核集成的在线半监督学习方法,使得在线学习器即使在接收到没有标记的数据时也能进行在线学习.该方法采用多个定义在不同RKHS中的函数对未标记数据预测的一致程度作为正则化项,在此基础上导出了多核集成在线半监督学习的即时风险函数,然后借助在线凸规划技术进行求解.在UCl数据集上的实验结果以及在网络入侵检测上的应用表明,该方法能够有效利用数据流中未标记数据来提升在线学习的性能.     

Parallelizing programs with recursive data structures

A study is made of the problem of estimating interference in an imperative language with dynamic data structures. The authors focus on developing efficient and implementable methods for recursive data structures. In particular, they present interference analysis tools and parallelization techniques for imperative programs that contain dynamically updatable trees and directed acyclic graphs. The analysis methods are based on a regular-expression-like representation of the relationship between accessible nodes in the data structure. They authors have implemented their analysis, and they present some concrete examples that have been processed by this system     

Ensemble of metamodels with optimized weight factors

Approximate mathematical models (metamodels) are often used as surrogates for more computationally intensive simulations. The common practice is to construct multiple metamodels based on a common training data set, evaluate their accuracy, and then to use only a single model perceived as the best while discarding the rest. This practice has some shortcomings as it does not take full advantage of the resources devoted to constructing different metamodels, and it is based on the assumption that changes in the training data set will not jeopardize the accuracy of the selected model. It is possible to overcome these drawbacks and to improve the prediction accuracy of the surrogate model if the separate stand-alone metamodels are combined to form an ensemble. Motivated by previous research on committee of neural networks and ensemble of surrogate models, a technique for developing a more accurate ensemble of multiple metamodels is presented in this paper. Here, the selection of weight factors in the general weighted-sum formulation of an ensemble is treated as an optimization problem with the desired solution being one that minimizes a selected error metric. The proposed technique is evaluated by considering one industrial and four benchmark problems. The effect of different metrics for estimating the prediction error at either the training data set or a few validation points is also explored. The results show that the optimized ensemble provides more accurate predictions than the stand-alone metamodels and for most problems even surpassing the previously reported ensemble approaches.     

Fusion of recursive programs with computational effects

Fusion laws permit to eliminate various of the intermediate data structures that are created in function compositions. The fusion laws associated with the traditional recursive operators on datatypes cannot, in general, be used to transform recursive programs with effects. Motivated by this fact, this paper addresses the definition of two recursive operators on datatypes that capture functional programs with effects. Effects are assumed to be modeled by monads. The main goal is thus the derivation of fusion laws for the new operators. One of the new operators is called monadic unfold. It captures programs (with effects) that generate a data structure in a standard way. The other operator is called monadic hylomorphism, and corresponds to programs formed by the composition of a monadic unfold followed by a function defined by structural induction on the data structure that the monadic unfold generates.     

Propositional dynamic logic with recursive programs

We extend the propositional dynamic logic PDL of Fischer and Ladner with a restricted kind of recursive programs using the formalism of visibly pushdown automata [R. Alur, P. Madhusudan, Visibly pushdown languages, in: Procceings of the 36th Annual ACM Symposium on Theory of Computing (STOC 2004), 2004, ACM, pp. 202–211]. We show that the satisfiability problem for this extension remains decidable, generalising known decidability results for extensions of PDL by non-regular programs. Our decision procedure establishes a 2-ExpTime upper complexity bound, and we prove a matching lower bound that applies already to rather weak extensions of PDL with non-regular programs. Thus, we also show that such extensions tend to be more complex than standard PDL.     

Synthesising recursive functions with side effects

Automatic Program Synthesis involves the automatic generation of a program (or plan) to achieve a specific goal. This means that smaller pre-defined (or previously synthesised) program segments are combined or modified to achieve the total goal. To guarantee that the required goal is actually achieved, the interactions between these program segments must be identified and considered. This paper shows how the side effects of segments can be derived and constructively used in achieving the required goal, and in guaranteeing the correctness of the resulting program. A program synthesising system PROSYN using these principles will then be described. A trace of a sample program synthesis, which generates a program that solves a general set of linear simultaneous equations, is given in the appendix.     

Ensemble strategies with adaptive evolutionary programming

Mutation operators such as Gaussian, Lévy and Cauchy have been used with evolutionary programming (EP). According to the no free lunch theorem, it is impossible for EP with a single mutation operator to outperform always. For example, Classical EP (CEP) with Gaussian mutation is better at searching in a local neighborhood while the Fast EP (FEP) with the Cauchy mutation performs better over a larger neighborhood. Motivated by these observations, we propose an ensemble approach where each mutation operator has its associated population and every population benefits from every function call. This approach enables us to benefit from different mutation operators with different parameter values whenever they are effective during different stages of the search process. In addition, the recently proposed Adaptive EP (AEP) using Gaussian (ACEP) and Cauchy (AFEP) mutations is also evaluated. In the AEP, the strategy parameter values are adapted based on the search performance in the previous few generations. The performance of ensemble is compared with a mixed mutation strategy, which integrates several mutation operators into a single algorithm as well as against the AEP with a single mutation operator. Improved performance of the ensemble over the single mutation-based algorithms and mixed mutation algorithm is verified using statistical tests.     

Code generation with a recursive optimizer

This paper describes a general purpose optimizer used to construct a new code generator for an existing Coral 66 compiler. The optimizer is recursive and gives the one pass code generator some of the advantages of a multiple pass scheme by ‘pipelining’ short sequences of linear intermediate code at each activation level.     

Uncertainty quantification in reliability estimation with limit state surrogates

Model-based reliability analysis is affected by different types of epistemic uncertainty, due to inadequate data and modeling errors. When the physics-based simulation model is computationally expensive, a surrogate has often been used in reliability analysis, introducing additional uncertainty due to the surrogate. This paper proposes a framework to include statistical uncertainty and model uncertainty in surrogate-based reliability analysis. Two types of surrogates have been considered: (1) general-purpose surrogate models that compute the system model output over the desired ranges of the random variables; and (2) limit-state surrogates. A unified approach to connect the model calibration analysis using the Kennedy and O’Hagan (KOH) framework to the construction of limit state surrogate and to estimating the uncertainty in reliability analysis is developed. The Gaussian Process (GP) general-purpose surrogate of the physics-based simulation model obtained from the KOH calibration analysis is further refined at the limit state (local refinement) to construct the limit state surrogate, which is used for reliability analysis. An efficient single-loop sampling approach using the probability integral transform is used for sampling the input variables with statistical uncertainty. The variability in the GP prediction (surrogate uncertainty) is included in reliability analysis through correlated sampling of the model predictions at different inputs. The Monte Carlo sampling (MCS) error, which represents the error due to limited Monte Carlo samples, is quantified by constructing a probability density function. All the different sources of epistemic uncertainty are quantified and aggregated to estimate the uncertainty in the reliability analysis. Two examples are used to demonstrate the proposed techniques.     

Hierarchical matrix arithmetic with accumulated updates

Hierarchical matrices can be used to construct efficient preconditioners for partial differential and integral equations by taking advantage of low-rank structures in triangular factorizations and inverses of the corresponding stiffness matrices. The setup phase of these preconditioners relies heavily on low-rank updates that are responsible for a large part of the algorithm’s total run-time, particularly for matrices resulting from three-dimensional problems. This article presents a new algorithm that significantly reduces the number of low-rank updates and can shorten the setup time by 50% or more.

     

Ensemble control of linear systems with parameter uncertainties

In this paper, we study the optimal control problem for a class of four-dimensional linear systems based on quaternionic and Fourier analysis. When the control is unconstrained, the optimal ensemble controller for this linear ensemble control systems is given in terms of prolate spheroidal wave functions. For the constrained convex optimisation problem of such systems, the quadratic programming is presented to obtain the optimal control laws. Simulations are given to verity the effectiveness of the proposed theory.