Evolutionary design of neural network architectures: a review of three decades of research

We present a comprehensive review of the evolutionary design of neural network architectures. This work is motivated by the fact that the success of an Artificial Neural Network (ANN) highly depends on its architecture and among many approaches Evolutionary Computation, which is a set of global-search methods inspired by biological evolution has been proved to be an efficient approach for optimizing neural network structures. Initial attempts for automating architecture design by applying evolutionary approaches start in the late 1980s and have attracted significant interest until today. In this context, we examined the historical progress and analyzed all relevant scientific papers with a special emphasis on how evolutionary computation techniques were adopted and various encoding strategies proposed. We summarized key aspects of methodology, discussed common challenges, and investigated the works in chronological order by dividing the entire timeframe into three periods. The first period covers early works focusing on the optimization of simple ANN architectures with a variety of solutions proposed on chromosome representation. In the second period, the rise of more powerful methods and hybrid approaches were surveyed. In parallel with the recent advances, the last period covers the Deep Learning Era, in which research direction is shifted towards configuring advanced models of deep neural networks. Finally, we propose open problems for future research in the field of neural architecture search and provide insights for fully automated machine learning. Our aim is to provide a complete reference of works in this subject and guide researchers towards promising directions.

     

Anticipating Cutoff Diameters in Deterministic Lateral Displacement (DLD) Microfluidic Devices for an Optimized Particle Separation

Deterministic lateral displacement (DLD) devices enable to separate nanometer to micrometer‐sized particles around a cutoff diameter, during their transport through a microfluidic channel with slanted rows of pillars. In order to design appropriate DLD geometries for specific separation sizes, robust models are required to anticipate the value of the cutoff diameter. So far, the proposed models result in a single cutoff diameter for a given DLD geometry. This paper shows that the cutoff diameter actually varies along the DLD channel, especially in narrow pillar arrays. Experimental and numerical results reveal that the variation of the cutoff diameter is induced by boundary effects at the channel side walls, called the wall effect. The wall effect generates unexpected particle trajectories that may compromise the separation efficiency. In order to anticipate the wall effect when designing DLD devices, a predictive model is proposed in this work and has been validated experimentally. In addition to the usual geometrical parameters, a new parameter, the number of pillars in the channel cross dimension, is considered in this model to investigate its influence on the particle trajectories.     

Achieving robust design through statistical effect screening

This work proposes a method for statistical effect screening to identify design parameters of a numerical simulation that are influential to performance while simultaneously being robust to epistemic uncertainty introduced by calibration variables. Design parameters are controlled by the analyst, but the optimal design is often uncertain, while calibration variables are introduced by modeling choices. We argue that uncertainty introduced by design parameters and calibration variables should be treated differently, despite potential interactions between the two sets. Herein, a robustness criterion is embedded in our effect screening to guarantee the influence of design parameters, irrespective of values used for calibration variables. The Morris screening method is utilized to explore the design space, while robustness to uncertainty is quantified in the context of info‐gap decision theory. The proposed method is applied to the National Aeronautics and Space Administration Multidisciplinary Uncertainty Quantification Challenge Problem, which is a black‐box code for aeronautic flight guidance that requires 35 input parameters. The application demonstrates that a large number of variables can be handled without formulating simplifying assumptions about the potential coupling between calibration variables and design parameters. Because of the computational efficiency of the Morris screening method, we conclude that the analysis can be applied to even larger‐dimensional problems. (Approved for unlimited, public release on October 9, 2013, LA‐UR‐13‐27839, Unclassified.) Copyright © 2015 John Wiley & Sons, Ltd.     

Coping with production time variability via dynamic lead-time quotation

In this paper, we propose two dynamic lead-time quotation policies in an M/GI/1 type make-to-stock queueing system serving lead-time sensitive customers with a single type of product. Incorporating non-exponential service times in an exact method for make-to-stock queues is usually deemed difficult. Our analysis of the proposed policies is exact and requires the numerical inversion of the Laplace transform of the sojourn time of an order to be placed. The first policy assures that the long-run probability of delivering the product within the quoted lead-time is the same for all backlogged customers. The second policy is a refinement of the first which improves the profitability if customers are oversensitive to even short delays in delivery. Numerical results show that both policies perform close to the optimal policy that was characterized only for exponential service times. The new insight gained is that the worsening impact of the production time variability, which is felt significantly in systems accepting all customers by quoting zero lead times, decreases when dynamic lead-time quotation policies are employed.     

A critical discussion on rebar electrical continuity and usual interpretation thresholds in the field of half-cell potential measurements in steel reinforced concrete

Corrosion of steel in reinforced concrete structures is a recurrent problem affecting civil engineering structures and costing the world billions of dollars per year. This physical phenomenon mainly results from chloride ingress or concrete carbonation. Corrosion can be diagnosed through a nondestructive method such as half-cell potential measurements. The present paper studies this method on a reinforced concrete wall containing eighteen unconnected steel bars and subjected to chloride-induced macrocell corrosion. Three corrosion systems with different configurations of connections between the steel bars are generated, involving three different anode-to-cathode surface ratios. Then, half-cell potential variations are observed versus macrocell corrosion current. The results lead to a critical discussion regarding the physical relevance of the usual potential threshold method to detect corroding rebars in reinforced concrete structures. In addition, the experiments demonstrate that electrical continuity between reinforcing steel bars is not necessary to get meaningful information about the macrocell corrosion system. At last, the paper show that the electric field (potential gradient) relative to a macrocell corrosion system may be measured by connecting the measurement system (reference electrode?+?voltmeter) to any electrochemical system in electrolytic contact with the concrete.     

Formation of Cu Nanodots on Diamond Surface to Improve Heat Transfer in Cu/D Composites

Diamond‐dispersed copper matrix (Cu/D) composite materials with different interfacial configurations are fabricated through powder metallurgy and their thermal performances are evaluated. An innovative solution to chemically bond copper (Cu) to diamond (D) has been investigated and compared to the traditional Cu/D bonding process involving carbide‐forming additives such as boron (B) or chromium (Cr). The proposed solution consists of coating diamond reinforcements with Cu particles through a gas–solid nucleation and growth process. The Cu particle‐coating acts as a chemical bonding agent at the Cu–D interface during hot pressing, leading to cohesive and thermally conductive Cu/D composites with no carbide‐forming additives. Investigation of the microstructure of the Cu/D materials through scanning electron microscopy, transmission electron microscopy, and atomic force microscopy analyses is coupled with thermal performance evaluations through thermal diffusivity, dilatometry, and thermal cycling. Cu/D composites fabricated with 40 vol% of Cu‐coated diamonds exhibit a thermal conductivity of 475 W m^?1 K^?1 and a thermal expansion coefficient of 12 × 10^?6 °C^?1. These promising thermal performances are superior to that of B‐carbide‐bonded Cu/D composites and similar to that of Cr‐carbide‐bonded Cu/D composites fabricated in this study. Moreover, the Cu/D composites fabricated with Cu‐coated diamonds exhibit higher thermal cycling resistance than carbide‐bonded materials, which are affected by the brittleness of the carbide interphase upon repeated heating and cooling cycles. The as‐developed materials can be applicable as heat spreaders for thermal management of power electronic packages. The copper‐carbon chemical bonding solution proposed in this article may also be found interesting to other areas of electronic packaging, such as brazing solders, direct bonded copper substrates, and polymer coatings.