Predicting the performance measures of a message-passing multiprocessor architecture using artificial neural networks

In this paper, we develop multi-layer feed-forward artificial neural network (MFANN) models for predicting the performance measures of a message-passing multiprocessor architecture interconnected by the simultaneous optical multiprocessor exchange bus (SOME-Bus), which is a fiber-optic interconnection network. OPNET Modeler is used to simulate the SOME-Bus multiprocessor architecture and to create the training and testing datasets. The performance of the MFANN prediction models is evaluated using standard error of estimate (SEE) and multiple correlation coefficient (R). Also, the results of the MFANN models are compared with the ones obtained by generalized regression neural network (GRNN), support vector regression (SVR), and multiple linear regression (MLR). It is shown that MFANN models perform better (i.e., lower SEE and higher R) than GRNN-based, SVR-based, and MLR-based models for predicting the performance measures of a message-passing multiprocessor architecture.     

Planning with a functional neural network architecture

Introduces the concept of planning in an interactive environment between two systems: the challenger and the responder. The responder's task is to produce behavior that relates to the challenger's behavior through some response function. In this setup, we concentrate planning on the responder's actions and use the produced plan in order to control the responder. In general, the responder is assumed to be a nonlinear system whose input-output (I/O) map may be expressed by a Volterra series. The planner uses an estimate of the challenger's future output sequence, the response function, and a model of the responder's I/O relation implemented through a functional artificial neural network (FANN) architecture, in order to produce the input sequence that will be applied to the responder in the future, in parallel-time with the challenger's corresponding output sequence. The responder accepts input from the planner, which may be combined with feedback information, in order to produce an output sequence that relates to the challenger's output sequence according to the response function. The importance of planning for the generation of smooth behavior is discussed, and the effectiveness of the planner's implementation using neural network technology is demonstrated with an example.     

Recognition of coloured and textured images through a multi-scale neural architecture with orientational filtering and chromatic diffusion

The aim of this paper is to outline a multiple scale neural model to recognise colour images of textured scenes. This model combines colour and textural information in order to recognise colour texture images through the operation of two main components: a segmentation component composed of the colour opponent system (COS) and the chromatic segmentation system (CSS); and a recognition component formed by an ARTMAP-based neural network with scale and orientation-invariance properties. Segmentation is achieved by perceptual contour extraction and diffusion processes on the colour opponent channels based on the human psychophysical theory of colour perception. This colour regions enhancement along with their local textural features constitutes the recognition pattern to be sent to the supervised neural classifier. The CSS accomplishes the colour region enhancement through a multiple scale loop of oriented filters and competition–cooperation mechanisms. Afterwards, the neural architecture performs an attentive recognition of the scene using those oriented filters responses and the chromatic diffusions. Some comparative tests with other models are included in order to prove the recognition capabilities of this neural architecture and how the use of colour information encourages the texture classification and the accuracy of the boundary detection.     

Navigating mobile robots with a modular neural architecture

Neural architectures have been proposed to navigate mobile robots within several environment definitions. In this paper a new neural modular constructive approach to navigate mobile robots in unknown environments is presented. The problem, in its basic form, consists of defining and executing a trajectory to a pre-defined goal while avoiding all obstacles, in an unknown environment. Some crucial issues arise when trying to solve this problem, such as an overflow of sensorial information and conflicting objectives. Most neural network (NN) approaches to this problem focus on a monolithic system, i.e., a system with only one neural network that receives and analyses all available information, resulting in conflicting training patterns, long training times and poor generalisation. The work presented in this article circumvents these problems by the use of a constructive modular NN. Navigation capabilities were proven with the NOMAD 200 mobile robot.     

自适应模糊神经网络研究

模糊神经网络提供了从人工神经网络中模糊规则的抽取。本文研究模糊神经网络的自适应学习，规则插入和抽取及神经－模糊推理的FuNN模型，把遗传算法作为系统模糊规则选择的自适应策略之一。     

Scalable communications for a million-core neural processing architecture

The design of a new high-performance computing platform to model biological neural networks requires scalable, layered communications in both hardware and software. SpiNNaker’s hardware is based upon Multi-Processor System-on-Chips (MPSoCs) with flexible, power-efficient, custom communication between processors and chips. The architecture scales from a single 18-processor chip to over 1 million processors and to simulations of billion-neuron, trillion-synapse models, with tens of trillions of neural spike-event packets conveyed each second. The communication networks and overlying protocols are key to the successful operation of the SpiNNaker architecture, designed together to maximise performance and minimise the power demands of the platform. SpiNNaker is a work in progress, having recently reached a major milestone with the delivery of the first MPSoCs. This paper presents the architectural justification, which is now supported by preliminary measured results of silicon performance, indicating that it is indeed scalable to a million-plus processor system.     

基于通道性能度量的神经网络结构搜索算法

卷积神经网络的表征与预测能力往往依赖结构合理性,但其主流结构均由人工设计,存在设计难度高、算力要求强、时间开销大等问题.如何让神经网络自主搜索合理结构并节约计算资源是当前的研究重点.目前,基于部分通道连接的可微分结构搜索算法,以其高效的显存利用率在搜索速度和分类性能上表现良好.然而,其针对通道的随机采样策略易造成重要信息丢失,当通道连接不足时性能明显下降.为此,提出一种基于通道性能度量的神经网络结构搜索算法,利用注意力机制提取通道重要性系数,并以此对通道进行排序采样.此外,考虑到预热阶段导致搜索不充分,产生较大离散化误差,在结构权重连续化的过程中设计温度正则化系数,提升权重差异.实验表明,所提算法能够在节约计算资源的基础上搜索出更优的卷积神经网络结构.     

Predicting terrain contours using a feed-forward neural network

Wheeled or tracked vehicles cannot move easily over much of the land surface of the earth. This paper describes research work to create walking machines that are able to travel when the terrain makes wheeled or tracked vehicles ineffective. These legged walking vehicles must be able to negotiate unknown environments with little or no knowledge of the terrain. A predictive terrain contour mapping strategy is proposed that uses a feed-forward neural network trained using a back-propagation algorithm to predict contours based on leg positions and orientations. The strategy is tested using the abilities of a tele-operated eight-legged robot named “Robug IV”. Predicted performance is an improvement on previous implementations and a summarised comparison of the results for the four terrains is provided.     

Forecasting time series with a new architecture for polynomial artificial neural network

Polynomial artificial neural networks (PANN) have been shown to be powerful for forecasting nonlinear time series. The training time is small compared to the time used by other algorithms of artificial neural networks and the capacity to compute relations between the inputs and outputs represented by every term of the polynomial. In this paper a new structure of polynomial is presented that improves the performance of this type of network considering only non-integers exponents. The architecture adaptation uses genetic algorithm (GA) to find the optimal architecture for every example. Some examples of sunspots and chaotic time series are presented.     

Convergence properties of eigenvalues of space discretized diffusion equations

A theorem is presented in this paper to establish the convergence of eigenvalues of space discretized Diffusion equation. The computational results confirm this.     

Deformation invariant visual object recognition: Experiments with a self-organising neural architecture

A self-organising neural network architecture for grey-scale visual object rcognition is presented. The network is composed of three processing layers with an architecture designed to give deformation tolerance. The processing layers involve feature extraction, sub-pattern detection and classification. Training is generally performed on-line in an unsupervised manner, classes being created when objects are presented that cannot be classified. The results given show the effect of the two discrimination parameters when the network is applied to two very different sets of images, namely hand written numerals and hand gestures images. The sensitivity of the network to the parameters that govern the size of detectable patterns and the areas over which they are detected is also tested. The robustness of the network to the order of image presentation is also demonstrated. The results show that parameter choice is not critical and heuristically chosen parameters provide near optimum performance.     

A neural architecture for a class of abduction problems

The general task of abduction is to infer a hypothesis that best explains a set of data. A typical subtask of this is to synthesize a composite hypothesis that best explains the entire data from elementary hypotheses which can explain portions of it. The synthesis subtask of abduction is computationally expensive, more so in the presence of certain types of interactions between the elementary hypotheses. In this paper, we first formulate the abduction task as a nonmonotonic constrained-optimization problem. We then consider a special version of the general abduction task that is linear and monotonic. Next, we describe a neural network based on the Hopfield model of computation for the special version of the abduction task. The connections in this network are symmetric, the energy function contains product forms, and the minimization of this function requires a network of order greater than two. We then discuss another neural architecture which is composed of functional modules that reflect the structure of the abduction task. The connections in this second-order network are asymmetric. We conclude with a discussion of how the second architecture may be extended to address the general abduction task.     

深度神经网络结构搜索综述

深度神经网络在图像识别、语言识别和机器翻译等人工智能任务中取得了巨大进展,很大程度上归功于优秀的神经网络结构设计。神经网络大都由手工设计,需要专业的机器学习知识以及大量的试错。为此,自动化的神经网络结构搜索成为研究热点。神经网络结构搜索（neural architecture search,NAS）主要由搜索空间、搜索策略与性能评估方法3部分组成。在搜索空间设计上,出于计算量的考虑,通常不会搜索整个网络结构,而是先将网络分成几块,然后搜索块中的结构。根据实际情况的不同,可以共享不同块中的结构,也可以对每个块单独搜索不同的结构。在搜索策略上,主流的优化方法包含强化学习、进化算法、贝叶斯优化和基于梯度的优化等。在性能评估上,为了节省计算时间,通常不会将每一个网络都充分训练到收敛,而是通过权值共享、早停等方法尽可能减小单个网络的训练时间。与手工设计的网络相比,神经网络结构搜索得到的深度神经网络具有更好的性能。在ImageNet分类任务上,与手工设计的MobileNetV2相比,通过神经网络结构搜索得到的MobileNetV3减少了近30%的计算量,并且top-1分类精度提升了3.2%;在Cityscapes语义分割任务上,与手工设计的DeepLabv3+相比,通过神经网络结构搜索得到的Auto-DeepLab-L可以在没有ImageNet预训练的情况下,达到比DeepLabv3+更高的平均交并比（mean intersection over union,mIOU）,同时减小一半以上的计算量。神经网络结构搜索得到的深度神经网络通常比手工设计的神经网络有着更好的表现,是未来神经网络设计的发展趋势。     

An analog processor architecture for a neural network classifier

Many neural-like algorithms currently under study support classification tasks. Several of these algorithms base their functionality on LVQ-like procedures to find locations of centroids in the data space, and on kernel (or radial-basis) functions centered on these centroids to approximate functions or probability densities. A generic analog chip could implement in a parallel way all basic functions found in these algorithms, permitting construction of a fast, portable classification system     

Predicting sun spots using a layered perceptron neural network

Interest in neural networks has expanded rapidly in recent years. Selecting the best structure for a given task, however, remains a critical issue in neural-network design. Although the performance of a network clearly depends on its structure, the procedure for selecting the optimal structure has not been thoroughly investigated, it is well known that the number of hidden units must be sufficient to discriminate each observation correctly. A large number of hidden units requires extensive computational time for training and often times prediction results may not be as accurate as expected. This study attempts to apply the principal component analysis (PCA) to determine the structure of a multilayered neural network for time series forecasting problems. The main focus is to determine the number of hidden units for a multilayered feedforward network. One empirical experiment with sunspot data is used to demonstrate the usefulness of the proposed approach.     

Predicting box-office success of motion pictures with neural networks

Predicting box-office receipts of a particular motion picture has intrigued many scholars and industry leaders as a difficult and challenging problem. In this study, the use of neural networks in predicting the financial performance of a movie at the box-office before its theatrical release is explored. In our model, the forecasting problem is converted into a classification problem-rather than forecasting the point estimate of box-office receipts, a movie based on its box-office receipts in one of nine categories is classified, ranging from a ‘flop’ to a ‘blockbuster.’ Because our model is designed to predict the expected revenue range of a movie before its theatrical release, it can be used as a powerful decision aid by studios, distributors, and exhibitors. Our prediction results is presented using two performance measures: average percent success rate of classifying a movie's success exactly, or within one class of its actual performance. Comparison of our neural network to models proposed in the recent literature as well as other statistical techniques using a 10-fold cross validation methodology shows that the neural networks do a much better job of predicting in this setting.     

RealNet: a neural network architecture for real-time systems scheduling

Real-time embedded systems are spreading to more and more new fields and their scope and complexity have grown dramatically in the last few years. Nowadays, real-time embedded computers or controllers can be found everywhere, both in very simple devices used in everyday life and in professional environments. Real-time embedded systems have to take into account robustness, safety and timeliness. The most-used schedulability analysis is the worst-case response time proposed by Joseph and Pandya (Comput J 29:390–395,1986). This test provides a bivaluated response (yes/no) indicating whether the processes will meet their corresponding deadlines or not. Nevertheless, sometimes the real-time designer might want to know, more exactly, the probability of the processes meeting their deadlines, in order to assess the risk of a failed scheduling depending on critical requirements of the processes. This paper presents RealNet, a neural network architecture that will generate schedules from timing requirements of a real-time system. The RealNet simulator will provide the designer, after iterating and averaging over some trials, an estimation of the probability that the system will not meet the deadlines. Moreover, the knowledge of the critical processes in these schedules will allow the designer to decide whether changes in the implementation are required.This revised version was published online in November 2004 with a correction to the accepted date.     

EfficientPose: Efficient human pose estimation with neural architecture search

Computational Visual Media - Human pose estimation from image and video is a key task in many multimedia applications. Previous methods achieve great performance but rarely take efficiency into...     

Evolutionary approximation and neural architecture search

Genetic Programming and Evolvable Machines - Automated neural architecture search (NAS) methods are now employed to routinely deliver high-quality neural network architectures for various...