Estimation of distribution algorithms with Kikuchi approximations

The question of finding feasible ways for estimating probability distributions is one of the main challenges for Estimation of Distribution Algorithms (EDAs). To estimate the distribution of the selected solutions, EDAs use factorizations constructed according to graphical models. The class of factorizations that can be obtained from these probability models is highly constrained. Expanding the class of factorizations that could be employed for probability approximation is a necessary step for the conception of more robust EDAs. In this paper we introduce a method for learning a more general class of probability factorizations. The method combines a reformulation of a probability approximation procedure known in statistical physics as the Kikuchi approximation of energy, with a novel approach for finding graph decompositions. We present the Markov Network Estimation of Distribution Algorithm (MN-EDA), an EDA that uses Kikuchi approximations to estimate the distribution, and Gibbs Sampling (GS) to generate new points. A systematic empirical evaluation of MN-EDA is done in comparison with different Bayesian network based EDAs. From our experiments we conclude that the algorithm can outperform other EDAs that use traditional methods of probability approximation in the optimization of functions with strong interactions among their variables.     

Time domain approximation using digital methods

Two digital methods are proposed for the solution of the time-domain approximation problem. Tho first method, based on a reeursivo algorithm for tho matrix pseudo-iriverso, determines the pulse transfer function of a discrete-timo system of a prescribed order providing the best fit in tho least squares sense at tho samplo points. Tho continuous-time system transfer function is then derived by using a simple transformation provided that tho sampling frequence oxeeeds tho Nyquist rate. Tho method is direct and computationally efficient. The second method is based on using an offieiont gradient search algorithm providing the best fit in the least pti sense at tho samplo points. A number of examples illustrato the methods     

Arch shape optimization using force approximation methods

The objective of this paper is to provide a method of optimizing areas of the members as well as the shape of both two-hinged and fixed arches. The design process includes satisfaction of combined stress constraints under the assumption that the arch ribs can be approximated by a finite number of straight members.In order to reduce the number of detailed finite element analyses, the Force Approximization Method is used. A finite element analysis of the initial structure is performed and the gradients of the member end forces (axial, bending moment) are calculated with respect to the areas and nodal coordinates. The gradients are used to form an approximate structural analysis based on first order Taylor series expansions of the member end forces. Using move limits, a numerical optimizer minimizes the volume of the arch with information from the approximate structural analysis.Numerical examples are presented to demonstrate the efficiency and reliablity of the proposed method for shape optimization. It is shown that the number of finite element analysis is minimal and the procedure provides a highly efficient method of arch shape optimization.     

Iterative methods for domain decomposition using spectral tau and collocation approximation

In this paper we consider the numerical approximation of elliptic problems by spectral methods in domains subdivided into substructures. We review an iterative procedure with interface relaxation, reducing the given differential problem to a sequence of Dirichlet and mixed Neumann-Dirichlet problems on each subdomain. The iterative procedure is applied to both tau and collocation spectral approximations. Two optimal strategies for the automatic selection of the relaxation parameter are indicated. We present several numerical experiments showing the convergence properties of the iterative scheme with respect to the decomposition. A multilevel technique for domain decomposition methods is proposed.     

Estimation methods for passage times using one-dependent cycles

The lengths of certain passage-time intervals (random time intervals) in discrete-event stochastic systems correspond to delays in computer, communication, manufacturing, and transportation systems. Simulation is often the only available means for analyzing a sequence of such lengths. It is sometimes possible to obtain meaningful estimates for the limiting average delay indirectly, that is, without measuring lengths of individual passage-time intervals. For general time-average limits of a sequence of delays, however, it is necessary to measure individual lengths and combine them to form point and interval estimates. We consider sequences of delays determined by state transitions of a generalized semi-Markov process and introduce a recursively-generated sequence of realvalued random vectors, called start vectors, to provide the link between the starts and terminations of passage-time intervals. This method of start vectors for measuring delays avoids the need to tag entities in the system. We show that if the generalized semi-Markov process has a recurrent single-state, then the sample paths of any sequence of delays can be decomposed into one-dependent, identically distributed cycles. We then show that an extension of the regenerative method for analysis of simulation output can be used to obtain meaningful point estimates and confidence intervals for time-average limits. This estimation procedure is valid not only when there are no ongoing passage times at any regeneration point but, unlike previous methods, also when the sequence of delays does not inherit regenerative structure. Application of these methods to a manufacturing cell with robots is discussed.     

Convergence of trust region augmented Lagrangian methods using variable fidelity approximation data

To date the primary focus of most constrained approximate optimization strategies is that application of the method should lead to improved designs. Few researchers have focused on the development of constrained approximate optimization strategies that are assured of converging to a Karush-Kuhn-Tucker (KKT) point for the problem. Recent work by the authors based on a trust region model management strategy has shown promise in managing the convergence of constrained approximate optimization in application to a suite of single level optimization test problems. Using a trust-region model management strategy, coupled with an augmented Lagrangian approach for constrained approximate optimization, the authors have shown in application studies that the approximate optimization process converges to a KKT point for the problem. The approximate optimization strategy sequentially builds a cumulative response surface approximation of the augmented Lagrangian which is then optimized subject to a trust region constraint. In this research the authors develop a formal proof of convergence for the response surface approximation based optimization algorithm. Previous application studies were conducted on single level optimization problems for which response surface approximations were developed using conventional statistical response sampling techniques such as central composite design to query a high fidelity model over the design space. In this research the authors extend the scope of application studies to include the class of multidisciplinary design optimization (MDO) test problems. More importantly the authors show that response surface approximations constructed from variable fidelity data generated during concurrent subspace optimization (CSSOs) can be effectively managed by the trust region model management strategy. Results for two multidisciplinary test problems are presented in which convergence to a KKT point is observed. The formal proof of convergence and the successful MDO application of the algorithm using variable fidelity data generated by CSSO are original contributions to the growing body of research in MDO.Nomenclature k Lagrangian iteration - s approximate minimization iteration - i, j, l variable indices - m number of inequality constraints - n number of design variables - p number of equality constraints - f(x) objective function - g(x) inequality constraint vector - g _j (x) j-th inequality constraint - h(x) equality constraint vector - h _j (x) i-th equality constraint - c(x) generalized constraint vector - c _i (x) i-th generalized constraint - c ₁,c ₂,c ₃,c ₄ real constants - m(x) approximate model - q(x) approximate model - q(x) piecewise approximation - r _p penalty parameter - t, t ₁,t ₂ step size length - x design vector, dimensionn - x _l l-th design variable - x ^U upper bound vector, dimensionn - x _l ^U l-th design upper bound - x ^L lower bound vector, dimensionn - x _l ^L l-th design lower bound - B approximation of the Hessian - K constraints residual - S design space - , ₁, ₂, scalars - ₁, ₂ convergence tolerances - ₀, ₁, ₂, , trust region parameters - Lagrange multiplier vector, dimensionm+p - _i i-th Lagrange multiplier - trust region ratio - (x) alternative form for inequality constraints - (x, ,r _p ) augmented Lagrangian function - approximation of the augmented Lagrangian function - fidelity control - . Euclidean norm - , inner product - gradient operator with respect to design vector x - P(y(x)) projection operator; projects the vector y onto the set of feasible directions at x - trust region radius - x step size     

Comparing offset curve approximation methods

Offset curves have diverse engineering applications, spurring extensive research on various offset techniques. In a paper on offset curve approximation (Lee et al., 1996), we suggested a new approach based on approximating the offset circle instead of the offset curve itself. To demonstrate the effectiveness of this approach, we compared it extensively with previous methods. To our surprise, Tiller and Hanson's (1984) simple method outperforms other methods for offsetting (piecewise) quadratic curves, even though its performance is not as good for high degree curves. The experimental results revealed other interesting facts, too. Had these details been reported several years ago, we believe offset approximation research might have developed somewhat differently. This article is intended to fill an important gap in the literature. We conducted qualitative as well as quantitative comparisons employing various contemporary offset approximation methods for freeform curves in the plane. We measured the efficiency of the offset approximation in terms of the number of control points generated while making the approximations within a prescribed tolerance     

Computational methods for blending surface approximation

Blending surfaces form a smooth transition between two distinct, intersecting surfaces or smoothly join two or more disconnected surfaces and are normally procedural surfaces which are difficult to exchange and to interrogate in a reliable and efficient manner. In this paper, an approximation method for blending surfaces which are curvature continuous to the underlying surfaces with a non-uniform rational B-spline (NURBS) surface is presented. The use of NURBS is important since it facilitates the exchange of geometric information between various computer aided design and manufacturing systems. In the method, linkage curves on the underlying surfaces are approximated to within a specified tolerance and cross-link curves are created using the linkage curves, a directional curve and the parametric partial derivatives of the underlying surfaces. Cross-link curves are lofted to form the blending surface and an adaptive sampling procedure is used to test the blending surface against specified tolerances. Cross-link curves are added, where necessary, and the surface relofted until the continuity conditions are satisfied to within specified tolerances. Examples illustrate the applicability of the method.     

Image registration by local approximation methods

Image registration is approached as an approximation problem. Two locally sensitive transformation functions are proposed for image registration. These transformation functions are obtained by the weighted least-squares method and the local weighted mean method. The former is a global method and uses information about all control points to establish correspondence between local areas in the images; nearby control points are, however, given higher weights to make the process locally sensitive. The latter is a local method and uses information about local control points only to register local areas in the images.     

Multivariate stochastic approximation using a simultaneousperturbation gradient approximation

The problem of finding a root of the multivariate gradient equation that arises in function minimization is considered. When only noisy measurements of the function are available, a stochastic approximation (SA) algorithm for the general Kiefer-Wolfowitz type is appropriate for estimating the root. The paper presents an SA algorithm that is based on a simultaneous perturbation gradient approximation instead of the standard finite-difference approximation of Keifer-Wolfowitz type procedures. Theory and numerical experience indicate that the algorithm can be significantly more efficient than the standard algorithms in large-dimensional problems     

Stochastic approximation methods for identification and control--A survey

Stochastic search techniques have been the essential part for most identification and self-organizing or learning control algorithms for stochastic systems. Stochastic approximation search algorithms have been very popular among the researchers in these areas because of their simplicity of implementation, convergence properties, as well as intuitive appeal to the investigator. This paper presents an exposition of the stochastic approximation algorithms and their application to various parameter identification and self-organizing control algorithms.     

Shape approximation using quadtrees

The quadtree representation encodes a 2″ by 2″ binary image as a set of maximal blocks of 1's or 0's whose sizes and positions are powers of 2. With the aid of the quadtree, a hierarchy of approximations to the image can be defined. Several ways of doing this are described. The accuracy of these approximations is empirically evaluated by studying how fast estimates of the first few moments of the image, computed from the approximations, converge to the true values, using a database of 112 airplane silhouettes. Approaches to the problem of fast shape matching using these approximations are also discussed.     

Note on the equivalence of the Routh and Hurwitz methods of approximation

A method of reduction based on the use of Hurwitz polynomials is shown to be equivalent to other methods of reduction which are based on the Routh table.     

Estimation of habit-related information from male voice data using machine learning-based methods

Artificial Life and Robotics - According to a survey on the cause of death among Japanese people, lifestyle-related diseases (such as malignant neoplasms, cardiovascular diseases, and pneumonia)...     

On two methods for discreteL p approximation

A recent paper by Merle and Späth [3] gives some computational experience with two algorithms for linear discreteL _p approximation. In this note, we establish a theoretical basis for some convergence properties observed by these authors.     

Integrated multiquadric radial basis function approximation methods

Promising numerical results using once and twice integrated radial basis functions have been recently presented. In this work we investigate the integrated radial basis function (IRBF) concept in greater detail, connect to the existing RBF theory, and make conjectures about the properties of IRBF approximation methods. The IRBF methods are used to solve PDEs.     

Most Relevant Explanation: computational complexity and approximation methods

Most Relevant Explanation (MRE) is the problem of finding a partial instantiation of a set of target variables that maximizes the generalized Bayes factor as the explanation for given evidence in a Bayesian network. MRE has a huge solution space and is extremely difficult to solve in large Bayesian networks. In this paper, we first prove that MRE is at least NP-hard. We then define a subproblem of MRE called MRE _k that finds the most relevant k-ary explanation and prove that the decision problem of MRE _k is NP^PPNP^{\it PP}-complete. Since MRE needs to find the best solution by MRE _k over all k, and we can also show that MRE is in NP^PPNP^{\it PP}, we conjecture that a decision problem of MRE is NP^PPNP^{\it PP}-complete as well. Furthermore, we show that MRE remains in NP^PPNP^{\it PP} even if we restrict the number of target variables to be within a log factor of the number of all unobserved variables. These complexity results prompt us to develop a suite of approximation algorithms for solving MRE, One algorithm finds an MRE solution by integrating reversible-jump MCMC and simulated annealing in simulating a non-homogeneous Markov chain that eventually concentrates its mass on the mode of a distribution of the GBF scores of all solutions. The other algorithms are all instances of local search methods, including forward search, backward search, and tabu search. We tested these algorithms on a set of benchmark diagnostic Bayesian networks. Our empirical results show that these methods could find optimal MRE solutions for most of the test cases in our experiments efficiently.     

Function approximation using generalized adalines

This paper proposes neural organization of generalized adalines (gadalines) for data driven function approximation. By generalizing the threshold function of adalines, we achieve the K-state transfer function of gadalines which responds a unitary vector of K binary values to the projection of a predictor on a receptive field. A generative component that uses the K-state activation of a gadaline to trigger K posterior independent normal variables is employed to emulate stochastic predictor-oriented target generation. The fitness of a generative component to a set of paired data mathematically translates to a mixed integer and linear programming. Since consisting of continuous and discrete variables, the mathematical framework is resolved by a hybrid of the mean field annealing and gradient descent methods. Following the leave-one-out learning strategy, the obtained learning method is extended for optimizing multiple generative components. The learning result leads to parameters of a deterministic gadaline network for function approximation. Numerical simulations further test the proposed learning method with paired data oriented from a variety of target functions. The result shows that the proposed learning method outperforms the MLP and RBF learning methods for data driven function approximation.