An adaptive nonmonotone trust region algorithm

Based on an eigenvalue analysis conducted on the scaled memoryless quasi-Newton updating formulas BFGS and DFP, an adaptive choice for the trust region radius is proposed. Then, using a trust region ratio obtained from a nonmonotone line search strategy, an adaptive nonmonotone trust region algorithm is developed. Under proper conditions, it is briefly shown that the proposed algorithm is globally and locally superlinearly convergent. Numerical experiments are done on a set of unconstrained optimization test problems of the CUTEr collection, using the Dolan–Moré performance profile. They show efficiency of the proposed algorithm.     

Numerical nonlinear observers using pseudo‐Newton‐type solvers

In constructing a globally convergent numerical nonlinear observer of Newton‐type for a continuous‐time nonlinear system, a globally convergent nonlinear equation solver with a guaranteed rate of convergence is necessary. In particular, the solver should be Jacobian free, because an analytic form of the state transition map of the nonlinear system is generally unavailable. In this paper, two Jacobian‐free nonlinear equation solvers of pseudo‐Newton type that fulfill these requirements are proposed. One of them is based on the finite difference approximation of the Jacobian with variable step size together with the line search. The other uses a similar idea, but the estimate of the Jacobian is mostly updated through a BFGS‐type law. Then, by using these solvers, globally stable numerical nonlinear observers are constructed. Numerical results are included to illustrate the effectiveness of the proposed methods. Copyright © 2007 John Wiley & Sons, Ltd.     

On the numerical solution of Fredholm integral equations utilizing the local radial basis function method

The current investigation describes a computational technique to solve one- and two-dimensional Fredholm integral equations of the second kind. The method estimates the solution using the discrete collocation method by combining locally supported radial basis functions (RBFs) constructed on a small set of nodes instead of all points over the analysed domain. In this work, we employ the Gauss–Legendre integration rule on the influence domains of shape functions to approximate the local integrals appearing in the method. In comparison with the globally supported RBFs for solving integral equations, the proposed method is stable and uses much less computer memory. The scheme does not require any cell structures, so it is meshless. We also obtain the error analysis of the proposed method and demonstrate that the convergence rate of the approach is high. Illustrative examples clearly show the reliability and efficiency of the new method.     

解大规模无约束优化的自适应过滤信赖域法

提出一种解大规模无约束优化问题的自适应过滤信赖域法。用目标函数的梯度及迭代点的信息来构造目标函数海赛矩阵的近似数量矩阵,引进了过滤技术和自适应技术,大大提高了计算效率。从理论上证明了新算法的全局收敛性,数值试验结果也表明了新算法的有效性。     

Globally convergent algorithms with local learning rates

A novel generalized theoretical result is presented that underpins the development of globally convergent first-order batch training algorithms which employ local learning rates. This result allows us to equip algorithms of this class with a strategy for adapting the overall direction of search to a descent one. In this way, a decrease of the batch-error measure at each training iteration is ensured, and convergence of the sequence of weight iterates to a local minimizer of the batch error function is obtained from remote initial weights. The effectiveness of the theoretical result is illustrated in three application examples by comparing two well-known training algorithms with local learning rates to their globally convergent modifications.     

A new modified scaled conjugate gradient method for large-scale unconstrained optimization with non-convex objective function

In this paper, according to the fifth-order Taylor expansion of the objective function and the modified secant equation suggested by Li and Fukushima, a new modified secant equation is presented. Also, a new modification of the scaled memoryless BFGS preconditioned conjugate gradient algorithm is suggested which is the idea to compute the scaling parameter based on a two-point approximation of our new modified secant equation. A remarkable feature of the proposed method is that it possesses a globally convergent even without convexity assumption on the objective function. Numerical results show that the proposed new modification of scaled conjugate gradient is efficient.     

求解非线性方程组的蛙跳和 BFGS 混合算法

混合蛙跳算法具有算法简单、控制参数少、易于实现等优点,但缺乏良好的局部细化搜索能力,使得求解精度不高。借鉴BFGS算法强的局部搜索能力,将BFGS算法与混合蛙跳算法有机融合,形成性能更优的混合优化算法,并用来求解非线性方程组。通过3个非线性方程组的实验表明,该混合算法收敛精度较高,收敛速度较快,是一种较好的求解非线性方程组的方法。     

A non-monotone inexact non-interior continuation method based on a parametric smoothing function for LWCP

This paper considers the linear weighted complementarity problem (denoted by LWCP). We introduce a parametric smoothing function which is a broad class of smoothing functions for the LWCP and enjoys some favourable properties. Based on this function, we propose a new non-interior continuation method for solving the LWCP. In general, the non-interior continuation method consists of finding an exact solution of a system of equations at each iteration, which may be cumbersome if one is solving a large-scale problem. To overcome this difficulty, our method uses an inexact Newton method to solve the corresponding linear system approximately and adopts a non-monotone line search to obtain a step size. Under suitable assumptions, we show that the proposed method is globally and locally quadratically convergent. Preliminary numerical results are also reported.     

A new algorithm based on differential transform method for solving multi-point boundary value problems

In this work, an efficient algorithm based on the differential transform method is applied to solve the multi-point boundary value problems. The solution obtained by using the proposed method takes the form of a convergent series with easily computable components. Several numerical examples, both linear and nonlinear, are given to testify the validity and applicability of the proposed method. Comparisons are made between the present method and the other existing methods.     

线性支持向量机优化问题的一种光滑算法

线性支持向量机的无约束优化模型的目标函数不是一个二阶可微函数,因此不能应用一些快速牛顿算法来求解。提出了目标函数的一种光滑化技巧,从而得到了相应的光滑线性支持向量机模型,并给出了求解该光滑线性支持向量机模型的Newton-Armijo算法,该算法是全局收敛的和二次收敛的。     

Fault diagnosis based on high-gain observer with an update law for a class of nonlinear systems

In this paper, by using the well-known high-gain observer design, an update law for the gain and an adaptive estimation of parameters, a new method of fault diagnosis for a class of nonlinear systems is presented. Without resort to any transformation for the parameters, the estimation errors of the states and the parameters are guaranteed to be globally exponentially convergent by a persistent excitation condition. Compared to the existing results, it can be applied to nonlinear systems with nonlinear terms admitting an incremental rate depending on the measured output. A case study further verifies the validity of the proposed research.     

A new spectral conjugate gradient method for large-scale unconstrained optimization

The spectral conjugate gradient methods, with simple construction and nice numerical performance, are a kind of effective methods for solving large-scale unconstrained optimization problems. In this paper, based on quasi-Newton direction and quasi-Newton condition, and motivated by the idea of spectral conjugate gradient method as well as Dai-Kou's selecting technique for conjugate parameter [SIAM J. Optim. 23 (2013), pp. 296–320], a new approach for generating spectral parameters is presented, where a new double-truncating technique, which can ensure both the sufficient descent property of the search directions and the bounded property of the sequence of spectral parameters, is introduced. Then a new associated spectral conjugate gradient method for large-scale unconstrained optimization is proposed. Under either the strong Wolfe line search or the generalized Wolfe line search, the proposed method is always globally convergent. Finally, a large number of comparison numerical experiments on large-scale instances from one thousand to two million variables are reported. The numerical results show that the proposed method is more promising.     

Convergence of an inexact generalized Newton method with a scaled residual control

The inexact generalized Newton method is an iterative method for solving systems of nonsmooth equations. In this paper, the iterative process with a relative residual control is presented and the conditions for local convergence to a solution are provided. These results can be applied to solve Lipschitz continuous equations under some mild assumptions. Moreover, a globally convergent version of the algorithm with a damped approach based on the Armijo rule is considered.     

An efficient method for unconstrained optimization problems ofnonlinear large mesh-interconnected systems

Presents an efficient method for solving unconstrained optimization problems for nonlinear large mesh-interconnected systems. This method combines an approximate scaled gradient method with a block Gauss-Seidel with line search method which is used to obtain an approximate solution of the unconstrained quadratic programming subproblem. The authors prove that their method is globally convergent and demonstrate by several numerical examples its superior efficiency compared to a sparse matrix technique based method. In an example of a system of more than 200 variables, the authors observe that their method is 3.45 times faster than the sparse matrix technique based Newton-like method and about 50 times faster than the Newton-like method without the sparse matrix technique     

A sufficient descent conjugate gradient method and its global convergence

In this paper, a DL-type conjugate gradient method is presented. The given method is a modification of the Dai–Liao conjugate gradient method. It can also be considered as a modified LS conjugate gradient method. For general objective functions, the proposed method possesses the sufficient descent condition under the Wolfe line search and is globally convergent. Numerical comparisons show that the proposed algorithm slightly outperforms the PRP+ and CG-descent gradient algorithms as well as the Barzilai–Borwein gradient algorithm.     

应用非单调线搜索求解一类互补问题

考虑一类含非Lipschtizian连续函数的非线性互补问题。引入plus函数的一类广义光滑函数,讨论其性质。应用所引入函数将互补问题重构为一系列光滑方程组,提出一个具有非单调线搜索的Newton算法求解重构的方程组以得到原问题的解。在很弱的条件下,该算法具有全局收敛性和局部二次收敛性。利用该算法求解一自由边界问题,其数值结果显示该算法是有效的。     

Solving general convex nonlinear optimization problems by an efficient neurodynamic model

In this paper, a neural network model is constructed on the basis of the duality theory, optimization theory, convex analysis theory, Lyapunov stability theory and LaSalle invariance principle to solve general convex nonlinear programming (GCNLP) problems. Based on the Saddle point theorem, the equilibrium point of the proposed neural network is proved to be equivalent to the optimal solution of the GCNLP problem. By employing Lyapunov function approach, it is also shown that the proposed neural network model is stable in the sense of Lyapunov and it is globally convergent to an exact optimal solution of the original problem. The simulation results also show that the proposed neural network is feasible and efficient.     

Qualitative analysis of a recurrent neural network for nonlinear continuously differentiable convex minimization over a nonempty closed convex subset.

We investigate the qualitative properties of a recurrent neural network (RNN) for minimizing a nonlinear continuously differentiable and convex objective function over any given nonempty, closed, and convex subset which may be bounded or unbounded, by exploiting some key inequalities in mathematical programming. The global existence and boundedness of the solution of the RNN are proved when the objective function is convex and has a nonempty constrained minimum set. Under the same assumption, the RNN is shown to be globally convergent in the sense that every trajectory of the RNN converges to some equilibrium point of the RNN. If the objective function itself is uniformly convex and its gradient vector is a locally Lipschitz continuous mapping, then the RNN is globally exponentially convergent in the sense that every trajectory of the RNN converges to the unique equilibrium point of the RNN exponentially. These qualitative properties of the RNN render the network model well suitable for solving the convex minimization over any given nonempty, closed, and convex subset, no matter whether the given constrained subset is bounded or not.     

Convexification of a 3-D coefficient inverse scattering problem

A version of the so-called “convexification” numerical method for a coefficient inverse scattering problem for the 3D Helmholtz equation is developed analytically and tested numerically. Backscattering data are used, which result from a single direction of the propagation of the incident plane wave on an interval of frequencies. The method converges globally. The idea is to construct a weighted Tikhonov-like functional. The key element of this functional is the presence of the so-called Carleman Weight Function (CWF). This is the function which is involved in the Carleman estimate for the Laplace operator. This functional is strictly convex on any appropriate ball in a Hilbert space for an appropriate choice of the parameters of the CWF. Thus, both the absence of local minima and convergence of minimizers to the exact solution are guaranteed. Numerical tests demonstrate a good performance of the resulting algorithm. Unlikeprevious the so-called tail functions globally convergent method, we neither do not impose the smallness assumption of the interval of wavenumbers, nor we do not iterate with respect to the so-called tail functions.     

Universal Practical Tracking Control of a Planar Underactuated Vehicle

In this article, a universal controller is proposed for a planar underactuated vehicle to track arbitrary trajectories including feasible/non‐feasible ones and fixed points. The controller design relies on several coordinate/input transformations, auxiliary trajectory design and the back‐stepping technique. The stability analysis shows that the position and orientation tracking errors are uniformly globally practically asymptotically convergent (UGPAC), and the velocity tracking errors are uniformly globally asymptotically convergent (UGAC) to a ball of origin. Moreover, if the tracked target is in uniform rectilinear motion or motionless, the whole closed‐loop tracking error system is uniformly globally practically asymptotically stable (UGPAS). The effectiveness of proposed control law is verified by simulation examples.