基于状态-动作图测地高斯基的策略迭代强化学习

在策略迭代强化学习中,基函数构造是影响动作值函数逼近精度的一个重要因素.为了给动作值函数逼近提供合适的基函数,提出一种基于状态-动作图测地高斯基的策略迭代强化学习方法.首先,根据离策略方法建立马尔可夫决策过程的状态-动作图论描述;然后,在状态-动作图上定义测地高斯核函数,利用基于近似线性相关的核稀疏方法自动选择测地高斯...     

利用椭圆性质提取椭圆

基于在文献［１］中开发的曲线提取算法的框架，并利用由椭圆的极点极线性质开发的三点组到椭圆参数的收敛映射，我们开发了一个鲁棒地从直接提取椭圆的以椭机Ｈｏｕｇｈ变换（ＲＨＴ）和椭圆性质为基础的椭圆提取新技术。用精确仿真的计算机图象和真实图象进行的大量实验证明了该技术的正确性、有效性和实用性。     

Efficient Jarratt-like methods for solving systems of nonlinear equations

We present the iterative methods of fourth and sixth order convergence for solving systems of nonlinear equations. Fourth order method is composed of two Jarratt-like steps and requires the evaluations of one function, two first derivatives and one matrix inversion in each iteration. Sixth order method is the composition of three Jarratt-like steps of which the first two steps are that of the proposed fourth order scheme and requires one extra function evaluation in addition to the evaluations of fourth order method. Computational efficiency in its general form is discussed. A comparison between the efficiencies of proposed techniques with existing methods of similar nature is made. The performance is tested through numerical examples. Moreover, theoretical results concerning order of convergence and computational efficiency are confirmed in the examples. It is shown that the present methods are more efficient than their existing counterparts, particularly when applied to the large systems of equations.     

An iteration method with maximal order based on standard information

For finding a root of a function f, Halley's iteration family is a higher generalization of Newton's iteration function. In every step, it uses the values of f and its first number of derivatives, called standard information. Based on the standard information, we obtain an iteration method with maximal order of convergence. It is a natural generalization of Halley's iteration family in terms of divided differences. An explicit construction for this method is also obtained. Numerical experiments are given demonstrating the importance of the proposed approach.     

Optimal Newton–Secant like methods without memory for solving nonlinear equations with its dynamics

We construct two optimal Newton–Secant like iterative methods for solving nonlinear equations. The proposed classes have convergence order four and eight and cost only three and four function evaluations per iteration, respectively. These methods support the Kung and Traub conjecture and possess a high computational efficiency. The new methods are illustrated by numerical experiments and a comparison with some existing optimal methods. We conclude with an investigation of the basins of attraction of the solutions in the complex plane.     

Discrete search methods for optimizing stochastic systems

In simulation practice, although estimating the performance of a complex stochastic system is of great value to the decision maker, it is not always enough. For example, a warehouse manager may be interested in finding out the probability that all demands are met from on-hand inventory under a certain system configuration of a fixed safety stock and a fixed order quantity. But he might be more interested in finding out what values of safety stock and order quantity will maximize this probability. In this paper we develop three strategies of a new iterative search procedure for finding the optimal parameters of a stochastic system, where the objective function cannot be evaluated exactly but must be estimated through Monte Carlo simulation. In each iteration, two neighboring configurations are compared and the one that appears to be the better one is passed on to the next iteration. The first strategy of the proposed method uses a single observation of each configuration in every iteration, while the second strategy uses a fixed number of observations of each configuration in every iteration. The third strategy uses sequential sampling with fixed boundaries. We show that, for all of these three strategies, the search process satisfies local balance equations and its equilibrium distribution gives most weight to the optimal point (when suitably normalized by the size of the neighborhoods). We also show that the configuration that has been visited most often in the first m iterations converges almost surely to an optimum solution.     

Feature-Based Aggregation and Deep Reinforcement Learning: A Survey and Some New Implementations

In this paper we discuss policy iteration methods for approximate solution of a finite-state discounted Markov decision problem, with a focus on feature-based aggregation methods and their connection with deep reinforcement learning schemes. We introduce features of the states of the original problem, and we formulate a smaller "aggregate" Markov decision problem, whose states relate to the features. We discuss properties and possible implementations of this type of aggregation, including a new approach to approximate policy iteration. In this approach the policy improvement operation combines feature-based aggregation with feature construction using deep neural networks or other calculations. We argue that the cost function of a policy may be approximated much more accurately by the nonlinear function of the features provided by aggregation, than by the linear function of the features provided by neural networkbased reinforcement learning, thereby potentially leading to more effective policy improvement.      

Rank-two residue iteration method for nonnegative matrix factorization

Rank-one residue iteration (RRI) is a recently developed block coordinate method for nonnegative matrix factorization (NMF). Numerical results show that the decomposed matrices generated by RRI method may have several columns, which are zero vectors. In this paper, by studying two special kinds of quadratic programming, we develop two block coordinate methods for NMF, rank-two residue iteration (RTRI) method and rank-two modified residue iteration (RTMRI) method. In the two algorithms, the exact solution of the subproblem can be obtained directly. We also provide that the consequence generated by our proposed algorithms can converge to a stationary point. Numerical results show that the RTRI method and the RTMRI method can yield better solutions, especially RTMRI method can remedy the limitation of the RRI method.     

Accuracy,Speed and Robustness of Policy Function Iteration

Policy function iteration methods for solving and analyzing dynamic stochastic general equilibrium models are powerful from a theoretical and computational perspective. Despite obvious theoretical appeal, significant startup costs and a reliance on grid-based methods have limited the use of policy function iteration as a solution algorithm. We reduce these costs by providing a user-friendly suite of MATLAB functions that introduce multi-core processing and Fortran via MATLAB’s executable function. Within the class of policy function iteration methods, we advocate using time iteration with linear interpolation. We examine a canonical real business cycle model and a new Keynesian model that features regime switching in policy parameters, Epstein–Zin preferences, and monetary policy that occasionally hits the zero-lower bound on the nominal interest rate to highlight the attractiveness of our methodology. We compare our advocated approach to other familiar iteration and approximation methods, highlighting the tradeoffs between accuracy, speed and robustness.     

A note on neta's family of sixth-order methods for solving equations

It has been shown that some three-step methods exist which, as well as Neta's methods, require one derivative and three function evaluations per iteration, but have an asymptotic convergence rate 7 which is better than the 6 of Neta.     

Input-to-state stability of infinite-dimensional control systems

We develop tools for investigation of input-to-state stability (ISS) of infinite-dimensional control systems. We show that for certain classes of admissible inputs, the existence of an ISS-Lyapunov function implies the ISS of a system. Then for the case of systems described by abstract equations in Banach spaces, we develop two methods of construction of local and global ISS-Lyapunov functions. We prove a linearization principle that allows a construction of a local ISS-Lyapunov function for a system, the linear approximation of which is ISS. In order to study the interconnections of nonlinear infinite-dimensional systems, we generalize the small-gain theorem to the case of infinite-dimensional systems and provide a way to construct an ISS-Lyapunov function for an entire interconnection, if ISS-Lyapunov functions for subsystems are known and the small-gain condition is satisfied. We illustrate the theory on examples of linear and semilinear reaction-diffusion equations.     

A hybrid multi-objective optimization algorithm for content based image retrieval

Relevance feedback methods in CBIR (Content Based Image Retrieval) iteratively use relevance information from the user to search the space for other relevant samples. As several regions of interest may be scattered through the space, an effective search algorithm should balance the exploration of the space to find new potential regions of interest and the exploitation of areas around samples which are known relevant. However, many algorithms concentrate the search on areas which are close to the images that the user has marked as relevant, according to a distance function in the (possibly deformed) multidimensional feature space. This maximizes the number of relevant images retrieved at the first iterations, but limits the discovery of new regions of interest and may leave unexplored a large section of the space. In this paper, we propose a novel hybrid approach that uses a scattered search algorithm based on NSGA II (Non-dominated Sorting Genetic Algorithm) only at the first iteration of the relevance feedback process, and then switches to an exploitation algorithm. The combined approach has been tested on three databases and in combination with several other methods. When the hybrid method does not produce better results from the first iteration, it soon catches up and improves both precision and recall.     

Numerical methods for the QCD overlap operator: III. Nested iterations

The numerical and computational aspects of chiral fermions in lattice quantum chromodynamics are extremely demanding. In the overlap framework, the computation of the fermion propagator leads to a nested iteration where the matrix vector multiplications in each step of an outer iteration have to be accomplished by an inner iteration; the latter approximates the product of the sign function of the hermitian Wilson fermion matrix with a vector.In this paper we investigate aspects of this nested paradigm. We examine several Krylov subspace methods to be used as an outer iteration for both propagator computations and the Hybrid Monte-Carlo scheme. We establish criteria on the accuracy of the inner iteration which allow to preserve an a priori given precision for the overall computation. It will turn out that the accuracy of the sign function can be relaxed as the outer iteration proceeds. Furthermore, we consider preconditioning strategies, where the preconditioner is built upon an inaccurate approximation to the sign function. Relaxation combined with preconditioning allows for considerable savings in computational efforts up to a factor of 4 as our numerical experiments illustrate. We also discuss the possibility of projecting the squared overlap operator into one chiral sector.     

Partitioned and Implicit–Explicit General Linear Methods for Ordinary Differential Equations

Implicit–explicit (IMEX) time stepping methods can efficiently solve differential equations with both stiff and nonstiff components. IMEX Runge–Kutta methods and IMEX linear multistep methods have been studied in the literature. In this paper we study new implicit–explicit methods of general linear type. We develop an order conditions theory for high stage order partitioned general linear methods (GLMs) that share the same abscissae, and show that no additional coupling order conditions are needed. Consequently, GLMs offer an excellent framework for the construction of multi-method integration algorithms. Next, we propose a family of IMEX schemes based on diagonally-implicit multi-stage integration methods and construct practical schemes of order up to three. Numerical results confirm the theoretical findings.     

INFFC: An iterative class noise filter based on the fusion of classifiers with noise sensitivity control

In classification, noise may deteriorate the system performance and increase the complexity of the models built. In order to mitigate its consequences, several approaches have been proposed in the literature. Among them, noise filtering, which removes noisy examples from the training data, is one of the most used techniques. This paper proposes a new noise filtering method that combines several filtering strategies in order to increase the accuracy of the classification algorithms used after the filtering process. The filtering is based on the fusion of the predictions of several classifiers used to detect the presence of noise. We translate the idea behind multiple classifier systems, where the information gathered from different models is combined, to noise filtering. In this way, we consider the combination of classifiers instead of using only one to detect noise. Additionally, the proposed method follows an iterative noise filtering scheme that allows us to avoid the usage of detected noisy examples in each new iteration of the filtering process. Finally, we introduce a noisy score to control the filtering sensitivity, in such a way that the amount of noisy examples removed in each iteration can be adapted to the necessities of the practitioner. The first two strategies (use of multiple classifiers and iterative filtering) are used to improve the filtering accuracy, whereas the last one (the noisy score) controls the level of conservation of the filter removing potentially noisy examples. The validity of the proposed method is studied in an exhaustive experimental study. We compare the new filtering method against several state-of-the-art methods to deal with datasets with class noise and study their efficacy in three classifiers with different sensitivity to noise.     

加权光滑投影孪生支持向量回归算法

现有双边移位投影孪生支持向量回归（PPTSVR）算法在训练阶段没有考虑不同位置样本对超平面构造的影响,当样本中存在异常点时会降低算法拟合性能。针对该问题,提出一种加权光滑投影孪生支持向量回归算法。采用孤立森林法赋予每个样本不同的权值,并且赋予样本中异常点很小的权值,通过将权值引入算法目标函数,削弱异常点对超平面构造的影响。为直接在原空间中寻求最优超平面,引入正号函数,将有约束优化问题转化为无约束优化问题,并采用Sigmoid光滑函数对目标函数进行光滑处理,证明其任意阶可微且严格凸的特性,进而在原空间中采用牛顿迭代法进行求解。在基准数据集和人工测试函数上的实验结果表明,该算法相比于现有代表性回归算法具备更好的拟合性能和更快的训练速度,尤其当训练样本中存在异常点时,相比于PPTSVR算法拟合性能提升更明显。     

Interval iterations for including a set of solutions

For including a set of solutions of a function strip we apply interval iterations. To construct a sequence of intervals converging to a fix-interval or a pseudofix-interval we make use of three kinds of interval operators with different properties and two iteration methods which are in accordance with the assumptions.     

面向欠约束几何系统的一种同伦求解方法

针对几何约束系统的数值求解过程中，经常发生的数值不稳定性问题，构造了一种面向欠约束系统的同伦方法，并将其与现有的求解与分解方法有机地结合起来，提出了一种牛顿－同伦混合方法，在牛顿迭代失败的位置自动调用欠约束同伦法，既提高了几何约束求解器的效率，同时又保证了求解的效率。     

Efficient preconditioned NHSS iteration methods for solving complex symmetric linear systems

Based on the new HSS (NHSS) iteration method introduced by Pour and Goughery (2015), we propose a preconditioned variant of NHSS (P*NHSS) and an efficient parameterized P*NHSS (PPNHSS) iteration methods for solving a class of complex symmetric linear systems. The convergence properties of the P*NHSS and the PPNHSS iteration methods show that the iterative sequences are convergent to the unique solution of the linear system for any initial guess when the parameters are properly chosen. Moreover, we discuss the quasi-optimal parameters which minimize the upper bounds for the spectral radius of the iteration matrices. Numerical results show that the PPNHSS iteration method is superior to several iteration methods whether the experimental optimal parameters are used or not.     

Improved Tabu Search and Simulated Annealing methods for nonlinear data assimilation

Nonlinear data assimilation can be a very challenging task. Four local search methods are proposed for nonlinear data assimilation in this paper. The methods work as follows: At each iteration, the observation operator is linearized around the current solution, and a gradient approximation of the three dimensional variational (3D-Var) cost function is obtained. Then, samples along potential steepest descent directions of the 3D-Var cost function are generated, and the acceptance/rejection criteria for such samples are similar to those proposed by the Tabu Search and the Simulated Annealing framework. In addition, such samples can be drawn within certain sub-spaces so as to reduce the computational effort of computing search directions. Once a posterior mode is estimated, matrix-free ensemble Kalman filter approaches can be implemented to estimate posterior members. Furthermore, the convergence of the proposed methods is theoretically proven based on the necessary assumptions and conditions. Numerical experiments have been performed by using the Lorenz-96 model. The numerical results show that the cost function values on average can be reduced by several orders of magnitudes by using the proposed methods. Even more, the proposed methods can converge faster to posterior modes when sub-space approximations are employed to reduce the computational efforts among iterations.