We present a comprehensive review of the evolutionary design of neural network architectures. This work is motivated by the fact that the success of an Artificial Neural Network (ANN) highly depends on its architecture and among many approaches Evolutionary Computation, which is a set of global-search methods inspired by biological evolution has been proved to be an efficient approach for optimizing neural network structures. Initial attempts for automating architecture design by applying evolutionary approaches start in the late 1980s and have attracted significant interest until today. In this context, we examined the historical progress and analyzed all relevant scientific papers with a special emphasis on how evolutionary computation techniques were adopted and various encoding strategies proposed. We summarized key aspects of methodology, discussed common challenges, and investigated the works in chronological order by dividing the entire timeframe into three periods. The first period covers early works focusing on the optimization of simple ANN architectures with a variety of solutions proposed on chromosome representation. In the second period, the rise of more powerful methods and hybrid approaches were surveyed. In parallel with the recent advances, the last period covers the Deep Learning Era, in which research direction is shifted towards configuring advanced models of deep neural networks. Finally, we propose open problems for future research in the field of neural architecture search and provide insights for fully automated machine learning. Our aim is to provide a complete reference of works in this subject and guide researchers towards promising directions.
相似文献Neuroevolution is the name given to a field of computer science that applies evolutionary computation for evolving some aspects of neural networks. After the AI Winter came to an end, neural networks reemerged to solve a great variety of problems. However, their usage requires designing their topology, a decision with a potentially high impact on performance. Whereas many works have tried to suggest rules-of-thumb for designing topologies, the truth is that there are not analytic procedures for determining the optimal one for a given problem, and trial-and-error is often used instead. Neuroevolution arose almost 3 decades ago, with some works focusing on the evolutionary design of the topology and most works describing techniques for learning connection weights. Since then, evolutionary computation has been proved to be a convenient approach for determining the topology and weights of neural networks, and neuroevolution has been applied to a great variety of fields. However, for more than 2 decades neuroevolution has mainly focused on simple artificial neural networks models, far from today’s deep learning standards. This is insufficient for determining good architectures for modern networks extensively used nowadays, which involve multiple hidden layers, recurrent cells, etc. More importantly, deep and convolutional neural networks have become a de facto standard in representation learning for solving many different problems, and neuroevolution has only focused in this kind of networks in very recent years, with many works being presented in 2017 onward. In this paper, we review the field of neuroevolution during the last 3 decades. We will put the focus on very recent works on the evolution of deep and convolutional neural networks, which is a new but growing field of study. To the best of our knowledge, this is the best survey reviewing the literature in this field, and we have described the features of each work as well as their performance on well-known databases when available. This work aims to provide a complete reference of all works related to neuroevolution of convolutional neural networks up to the date. Finally, we will provide some future directions for the advancement of this research area.
相似文献Deep learning is the most active research topic amongst data scientists and analysts these days. It is because deep learning has provided very high accuracy in various domains such as speech recognition, image processing and natural language processing. Researchers are actively working to deploy deep learning on information retrieval. Due to large-scale data generated by social media and sensor networks, it is quite difficult to train unstructured and highly complex data. Recommender system is intelligent information filtering technique which assists the user to find topic of interest within complex overloaded information. In this paper, our motive is to improve recommendation accuracy for large-scale heterogeneous complex data by integrating deep learning architecture. In our proposed approach ratings, direct and indirect trust values are fed in neural network using shared layer in autoencoder. Comprehensive experiment analysis on three public datasets proves that RMSE and MAE are improved significantly by using our proposed approach.
相似文献Deep reinforcement learning augments the reinforcement learning framework and utilizes the powerful representation of deep neural networks. Recent works have demonstrated the remarkable successes of deep reinforcement learning in various domains including finance, medicine, healthcare, video games, robotics, and computer vision. In this work, we provide a detailed review of recent and state-of-the-art research advances of deep reinforcement learning in computer vision. We start with comprehending the theories of deep learning, reinforcement learning, and deep reinforcement learning. We then propose a categorization of deep reinforcement learning methodologies and discuss their advantages and limitations. In particular, we divide deep reinforcement learning into seven main categories according to their applications in computer vision, i.e. (i) landmark localization (ii) object detection; (iii) object tracking; (iv) registration on both 2D image and 3D image volumetric data (v) image segmentation; (vi) videos analysis; and (vii) other applications. Each of these categories is further analyzed with reinforcement learning techniques, network design, and performance. Moreover, we provide a comprehensive analysis of the existing publicly available datasets and examine source code availability. Finally, we present some open issues and discuss future research directions on deep reinforcement learning in computer vision.
相似文献The advances in reinforcement learning have recorded sublime success in various domains. Although the multi-agent domain has been overshadowed by its single-agent counterpart during this progress, multi-agent reinforcement learning gains rapid traction, and the latest accomplishments address problems with real-world complexity. This article provides an overview of the current developments in the field of multi-agent deep reinforcement learning. We focus primarily on literature from recent years that combines deep reinforcement learning methods with a multi-agent scenario. To survey the works that constitute the contemporary landscape, the main contents are divided into three parts. First, we analyze the structure of training schemes that are applied to train multiple agents. Second, we consider the emergent patterns of agent behavior in cooperative, competitive and mixed scenarios. Third, we systematically enumerate challenges that exclusively arise in the multi-agent domain and review methods that are leveraged to cope with these challenges. To conclude this survey, we discuss advances, identify trends, and outline possible directions for future work in this research area.
相似文献With the continuous mining and gradual reduction of shallow deposits, deep prospecting has become a new global prospecting trend. In addition, with the development of artificial intelligence, deep learning provides a favorable means for geological big data analysis. This paper, researches the No. II Orebody of the Xiongcun deposit. First, based on previous research results and metallogenic regularity, prospecting information, namely, lithology, Au-Ag-Cu chemical elements and wall rock alteration is extracted, and the block model is established by combining the Kriging interpolation structure. Second, the datasets are divided into dataset I and dataset II according to “randomness” and “depth”. Third, deep prospecting prediction models based on deep neural networks (DNN) and the convolutional neural networks (CNN) is constructed, and the model parameters are optimized. Finally, the models are applied to the deep prediction of the Xiongcun No. II Orebody. The results show that the accuracy rate and recall rate of the prediction model based on the DNN algorithm are 96.15% and 89.23%, respectively, and the AUC is 96.39%, which are higher values than those of the CNN algorithm, indicating that the performance of the prediction model based on the DNN algorithm is better. The accuracy of prediction model based on dataset I is higher than that of dataset II. The accuracy of deep metallogenic prediction based on the DNN algorithm is approximately 89%, that based on the CNN is approximately 87%, and that based on prospecting information method is approximately 61.27%. The prediction results of the DNN algorithm are relatively consistent in the spatial location and scale of the orebody. Therefore, based on the work done in this paper, it is feasible to use a deep learning method to carry out deep mineral prediction.
相似文献Technology-based education of children with special needs has become the focus of many research works in recent years. The wide range of different disabilities that are encompassed by the term “special needs”, together with the educational requirements of the children affected, represent an enormous multidisciplinary challenge for the research community. In this article, we present a systematic literature review of technology-enhanced and game-based learning systems and methods applied on children with special needs. The article analyzes the state-of-the-art of the research in this field by selecting a group of primary studies and answering a set of research questions. Although there are some previous systematic reviews, it is still not clear what the best tools, games or academic subjects (with technology-enhanced, game-based learning) are, out of those that have obtained good results with children with special needs. The 18 articles selected (carefully filtered out of 614 contributions) have been used to reveal the most frequent disabilities, the different technologies used in the prototypes, the number of learning subjects, and the kind of learning games used. The article also summarizes research opportunities identified in the primary studies.
相似文献