Weakly pulse-coupled oscillators, FM interactions, synchronization,and oscillatory associative memory

We study pulse-coupled neural networks that satisfy only two assumptions: each isolated neuron fires periodically, and the neurons are weakly connected. Each such network can be transformed by a piece-wise continuous change of variables into a phase model, whose synchronization behavior and oscillatory associative properties are easier to analyze and understand. Using the phase model, we can predict whether a given pulse-coupled network has oscillatory associative memory, or what minimal adjustments should be made so that it can acquire memory. In the search for such minimal adjustments we obtain a large class of simple pulse-coupled neural networks that ran memorize and reproduce synchronized temporal patterns the same way a Hopfield network does with static patterns. The learning occurs via modification of synaptic weights and/or synaptic transmission delays.     

Efficient computation of the frame-based extended union model and its application in speech recognition against partial temporal corruptions

The extended union model (EUM) was recently proposed and shown to be effective in handling short time temporal corruption. Because of the computational complexity, the EUM probability can only be computed over groups of consecutive observations (called segments) and recognition can only be performed under N-best re-scoring paradigm. In this paper, we introduce a hidden variable called “pattern of corruption” and re-formulate the extended union model as marginalizing over possible patterns of corruption with likelihood computed via the missing feature theory. We then introduce a recursive relationship between the EUM probabilities of two successive observation sequences that can greatly simplify the EUM probability computation. This makes it possible to compute the EUM probability over a long sequence. Using this recursive relationship, the EUM probability over frames, called the “frame-based EUM” can easily be computed. To simplify the EUM-based recognition, we propose an approximated, dynamic programming-based EUM recognition algorithm, called the Frame-based EUM Viterbi algorithm (FEVA), that performs recognition directly instead of via N-best re-scoring. Experimental results on digit recognition under added impulsive noises show that both the frame-base EUM and the FEVA outperform the segment-based EUM.     

Time-delay recurrent neural network for temporal correlations and prediction

This paper presents a time-delay recurrent neural network (TDRNN) for temporal correlations and prediction. The TDRNN employs adaptive time delays and recurrences where the adaptive time delays make the network choose the optimal values of time delays for the temporal location of the important information in the input sequence and the recurrences enable the network to encode and integrate temporal context information of sequences. The TDRNN and multiple recurrent neural network(MRNN) described in this paper, adaptive time-delay neural network (ATNN) proposed by Lin, and time-delay neural network (TDNN) introduced by Waibel were simulated and applied to the chaotic time series prediction of Mackey–Glass delay-differential equation and the Korean stock market index prediction. The simulation results suggest that employing time delayed recurrences in the layered network is more effective for temporal correlations and prediction than putting multiple time delays into the neurons or their connections. The best performance is attained by the TDRNN. The TDRNN will be well applicable for temporal signal recognition, prediction and identification.     

Influence of order payment delays on the efficiency of multinomenclature reserve control models

For the multinomenclature EOQ reserve control model with renting storage space and delayed payment for the orders, we propose an approach to optimization that accounts for the temporal value of money. We estimate the influence of these delays both on the parameters of the optimal control strategy and on the efficiency of using floating capital. The study has been done for situations when the said delays let one pay for orders from the revenue on repeat order intervals. We establish a necessary and sufficient condition imposed on the duration of delays that guarantees that this order payment from the revenue is possible. We illustrate characteristic features of optimization procedures with numerical computations.     

Discrete-time backpropagation for training synaptic delay-basedartificial neural networks

The aim of the paper is to endow a well-known structure for processing time-dependent information, synaptic delay-based ANNs, with a reliable and easy to implement algorithm suitable for training temporal decision processes. In fact, we extend the backpropagation algorithm to discrete-time feedforward networks that include adaptable internal time delays in the synapses. The structure of the network is similar to the one presented by Day and Davenport (1993), that is, in addition to the weights modeling the transmission capabilities of the synaptic connections, we model their length by means of a parameter that indicates the delay a discrete-event suffers when going from the origin neuron to the target neuron through a synaptic connection. Like the weights, these delays are also trainable, and a training algorithm can be derived that is almost as simple as the backpropagation algorithm, and which is really an extension of it. We present examples of the application of these networks and algorithm to the prediction of time series and to the recognition of patterns in electrocardiographic signals. In the first case, we employ the temporal reasoning characteristics of these networks for the prediction of future values in a benchmark example of a time series: the one governed by the Mackey-Glass chaotic equation. In the second case, we provide a real life example. The problem consists in identifying different types of beats through two levels of temporal processing, one relating the morphological features which make up the beat in time and another one that relates the positions of beats in time, that is, considers rhythm characteristics of the ECG signal. In order to do this, the network receives the signal sequentially, no windowing, segmentation, or thresholding are applied     

Selective Quantitative Analysis and Interval Model Checking: Verifying Different Facets of a System

In this work we propose a verification methodology consisting of selective quantitative timing analysis and interval model checking. Our methods can aid not only in determining if a system works correctly, but also in understanding how well the system works. The selective quantitative algorithms compute minimum and maximum delays over a selected subset of system executions. A linear-time temporal logic (LTL) formula is used to select either infinite paths or finite intervals over which the computation is performed. We show how tableau for LTL formulas can be used for selecting either paths or intervals and also for model checking formulas interpreted over paths or intervals.To demonstrate the usefulness of our methods we have verified a complex and realistic distributed real-time system. Our tool has been able to analyze the system and to compute the response time of the various components. Moreover, we have been able to identify inefficiencies that caused the response time to increase significantly (about 50%). After changing the design we not only verified that the response time was lower, but were also able to determine the causes for the poor performance of the original model using interval model checking.     

Intraneuronal information processing, directional selectivity and memory for spatio-temporal sequences

Interacting intracellular signalling pathways can perform computations on a scale that is slower, but more fine-grained, than the interactions between neurons upon which we normally build our computational models of the brain (Bray D 1995 Nature 376 307-12). What computations might these potentially powerful intraneuronal mechanisms be performing? The answer suggested here is: storage of spatio-temporal sequences of synaptic excitation so that each individual neuron can recognize recurrent patterns that have excited it in the past. The experimental facts about directionally selective neurons in the visual system show that neurons do not integrate separately in space and time, but along straight spatio-temporal trajectories; thus, neurons have some of the capacities required to perform such a task. In the retina, it is suggested that calcium-induced calcium release (CICR) may provide the basis for directional selectivity. In the cortex, if activation mechanisms with different delays could be separately reinforced at individual synapses, then each such Hebbian super-synapse would store a memory trace of the delay between pre- and post-synaptic activity, forming an ideal basis for the memory and response to phase sequences.     

Modelling and temporal performances evaluation of networked control systems using (max, +) algebra

In this paper, we address the problem of temporal performances evaluation of producer/consumer networked control systems. The aim is to develop a formal method for evaluating the response time of this type of control systems. Our approach consists on modelling, using Petri nets classes, the behaviour of the whole architecture including the switches that support multicast communications used by this protocol. (max, +) algebra formalism is then exploited to obtain analytical formulas of the response time and the maximal and minimal bounds. The main novelty is that our approach takes into account all delays experienced at the different stages of networked automation systems. Finally, we show how to apply the obtained results through an example of networked control system.     

Invariant Recognition of Spatio-Temporal Patterns in The Olfactory System Model

This paper presents a model of a network of integrate-and-fire neurons with time delay weights, capable of invariant spatio-temporal pattern recognition. Spatio-temporal patterns are formed by spikes according to the encoding principle that the phase shifts of the spikes encode the input stimulus intensity which corresponds to the concentration of constituent molecules of an odor. We applied the Hopfield's phase shift encoding principle at the output level for spatio-temporal pattern recognition: Firing of an output neuron indicates that corresponding odor is recognized and phase shift of its firing encodes the concentration of the recognized odor. The temporal structure of the model provides the base for the modeling of higher level tasks, where temporal correlation is involved, such as feature binding and segmentation, object recognition, etc.     

Combining automatic acquisition of knowledge with machine learning approaches for multilingual temporal recognition and normalization

This paper presents an improvement in the temporal expression (TE) recognition phase of a knowledge based system at a multilingual level. For this purpose, the combination of different approaches applied to the recognition of temporal expressions are studied. In this work, for the recognition task, a knowledge based system that recognizes temporal expressions and had been automatically extended to other languages (TERSEO system) was combined with a system that recognizes temporal expressions using machine learning techniques. In particular, two different techniques were applied: maximum entropy model (ME) and hidden Markov model (HMM), using two different types of tagging of the training corpus: (1) BIO model tagging of literal temporal expressions and (2) BIO model tagging of simple patterns of temporal expressions. Each system was first evaluated independently and then combined in order to: (a) analyze if the combination gives better results without increasing the number of erroneous expressions in the same percentage and (b) decide which machine learning approach performs this task better. When the TERSEO system is combined with the maximum entropy approach the best results for F-measure (89%) are obtained, improving TERSEO recognition by 4.5 points and ME recognition by 7.     

基于CTC模型的无分割文本验证码识别

验证码安全性是保障网络安全的重要一环,本文利用深度学习,提出长短期记忆（Long Short-Term Memory, LSTM）网络和连接时序分类（Connectionist Temporal Classification, CTC）模型对主流的验证码图片进行智能识别,利用开源CAPTCHA验证码库生成数据集,简化验证码识别模型,统一语音识别和文本识别方法,实现端到端模型识别。本文提出的方法在较小训练集情况下有更优秀的性能。     

Dynamic Query Operator Scheduling for Wide-Area Remote Access

Distributed databases operating over wide-area networks such as the Internet, must deal with the unpredictable nature of the performance of communication. The response times of accessing remote sources can vary widely due to network congestion, link failure, and other problems. In such an unpredictable environment, the traditional iterator-based query execution model performs poorly. We have developed a class of methods, called query scrambling, for dealing explicitly with the problem of unpredictable response times. Query scrambling dynamically modifies query execution plans on-the-fly in reaction to unexpected delays in data access. In this paper we focus on the dynamic scheduling of query operators in the context of query scrambling. We explore various choices for dynamic scheduling and examine, through a detailed simulation, the effects of these choices. Our experimental environment considers pipelined and non-pipelined join processing in a client with multiple remote data sources and delayed or possibly bursty arrivals of data. Our performance results show that scrambling rescheduling is effective in hiding the impact of delays on query response time for a number of different delay scenarios.     

IPM-Model: AI and metaheuristic-enabled face recognition using image partial matching for multimedia forensics investigation with genetic algorithm

The rapid enhancement in the development of information technology has driven the development of human facial image recognition. Recently, facial recognition has been successfully applied in several distinct domains with the help of computing and information technology. This kind of application plays a significant role in the process of digital forensics investigation, recognizing the patterns of a human face based on the partial matching of images that would be in 24-bit color image format, including the spacing of the eyes, the bridging of the nose, the contour of the lips, ears, and chin. In this paper, we have proposed and implemented an image recognition model based on principal component analysis, genetic algorithms, and neural networks, in which PCA reduces the dimension of the benchmark dataset, while genetic algorithms and neural nets optimize the searching patterns of image matching and provide highly efficient output with a minimal amount of time. Through the experiment results on the human facial images dataset of the Georgia Institute of Technology, the overall match showed that the proposed model can achieve the recognition of human face images with an accuracy rate of 93.7%. Moreover, this model helps to examine, analyze, and detect individuals by partial matching with reidentification in the procedure of forensics investigation. The experimental result shows the robustness of the proposed model in terms of efficiency compared to other state-of-the-art methods.

     

Integrated design of residual generation and evaluation for fault detection of networked control systems

In this paper, integrated design of residual generation and evaluation is proposed to fault detection (FD) of networked control systems. Both the imperfect network transmissions and the sampling effects are considered under the assumption that no stochastic network model is available. By deriving a linear discrete time‐varying system equivalent to the sampled plant, the continuous‐time behaviors of disturbances and faults during sampling intervals are captured. Moreover, the effects of unknown but bounded packet delays and dropouts in the controller‐to‐actuator link are described by uncertain input delays with finite number of possibilities. A parity relation‐based FD module with multiple residuals is constructed for the derived system model. For the constructed FD module, the set of undetectable faults is obtained from integrated analysis of residual generation and evaluation and then designed to achieve its worst‐case minimal geometrical size while guaranteeing zero false alarms. Simulation results on a networked three‐tank system are provided to show the merits of the proposed integrated design approach. Copyright © 2015 John Wiley & Sons, Ltd.     

Predicting temporal violations for parallel business cloud workflows

Workflow temporal violations, namely, intermediate workflow runtime delays, often occur and have a serious impact on the on‐time completion of massive concurrent requests. Therefore, accurate prediction of cloud workflow temporal violations is critical as its result can serve as an essential reference for temporal violation prevention and handling strategies. Conventional studies mainly focus on the time delays of a single workflow activity or a single workflow instance but overlook the propagation of time delays among them. This is a serious problem as time delays can propagate in cloud workflow system due to resource sharing and the dependencies among workflow activities. This paper first proposes a novel temporal violation transmission model inspired by an epidemic model to model the dynamics of time delay propagation. Afterward, a novel temporal violation prediction strategy is presented to estimate the number of temporal violations that may occur and determine the number of violations that must be handled to achieve the target service‐level agreement, namely, the on‐time completion rate. To the best of our knowledge, this is the first attempt to predict cloud workflow temporal violations at the workflow build‐time stage by analyzing the propagation of temporal violations. Experimental results demonstrate that our strategy can make highly accurate predictions and is scalable for a large batch of parallel workflows running in the cloud.     

Recognition-directed recovering of temporal information from handwriting images

This paper analyses a handwriting recognition system for offline cursive words based on HMMs. It compares two approaches for transforming offline handwriting available as two-dimensional images into one-dimensional input signals that can be processed by HMMs. In the first approach, a left–right scan of the word is performed resulting in a sequence of feature vectors. In the second approach, a more subtle process attempts to recover the temporal order of the strokes that form words as they were written. This is accomplished by a graph model that generates a set of paths, each path being a possible temporal order of the handwriting. The recognition process then selects the most likely temporal stroke order based on knowledge that has been acquired from a large set of handwriting samples for which the temporal information was available. We show experimentally that such an offline recognition system using the recovered temporal order can achieve recognition performances that are much better than those obtained with the simple left–right order, and that come close to those of an online recognition system. We have been able to assess the ordering quality of handwriting when comparing true ordering and recovered one, and we also analyze the situations where offline and online information differ and what the consequences are on the recognition performances. For these evaluations, we have used about 30,000 words from the IRONOFF database that features both the online signal and offline signal for each word.     

Incorporating space and time into random forest models for analyzing geospatial patterns of drug-related crime incidents in a major U.S. metropolitan area

The opioid crisis has hit American cities hard, and research on spatial and temporal patterns of drug-related activities including detecting and predicting clusters of crime incidents involving particular types of drugs is useful for distinguishing hot zones where drugs are present that in turn can further provide a basis for assessing and providing related treatment services. In this study, we investigated spatiotemporal patterns of more than 52,000 reported incidents of drug-related crime at block group granularity in Chicago, IL between 2016 and 2019. We applied a space-time analysis framework and machine learning approaches to build a model using training data that identified whether certain locations and built environment and sociodemographic factors were correlated with drug-related crime incident patterns, and establish the top contributing factors that underlaid the trends. Space and time, together with multiple driving factors, were incorporated into a random forest model to analyze these changing patterns. We accommodated both spatial and temporal autocorrelation in the model learning process to assist with capturing the changes over time and tested the capabilities of the space-time random forest model by predicting drug-related activity hot zones. We focused particularly on crime incidents that involved heroin and synthetic drugs as these have been key drug types that have highly impacted cities during the opioid crisis in the U.S.     

Global dynamics of an HIV-1 infection model with distributed intracellular delays

In this paper, an HIV-1 infection model with distributed intracellular delays is investigated, where the intracellular delays account for the time the target cells are contacted by the virus particles and the time the contacted cells become actively infected meaning that the contacting virions enter cells and the time the virus has penetrated into a cell and the time the new virions are created within the cell and are released from the cell, respectively. By analyzing the characteristic equations, the local stability of an infection-free equilibrium and a chronic-infection equilibrium of the model is established. By using suitable Lyapunov functionals and LaSalle’s invariance principle, it is proved that if the basic reproduction ratio is less than unity, the infection-free equilibrium is globally asymptotically stable; and if the basic reproduction ratio is greater than unity, the chronic-infection equilibrium is globally asymptotically stable.     

A MATHEMATICAL MODEL THAT LEARNS AN ADAPTIVELY GENERATED NOVEL PATTERN IN QUADRUPED LOCOMOTION

Quadrupeds show several locomotion patterns when adapting to environmental conditions. An immediate transition among walk, trot, and gallop implies the existence of a memory for locomotion patterns. In this article, we postulate that motion pattern learning necessitates the repetitive presentation of the same environmental conditions and aim at constructing a mathematical model for new pattern learning. The model construction considers a decerebrate cat experiment in which only the left forelimb is driven at higher speed by a belt on a treadmill. A central pattern generator (CPG) model that qualitatively describes this decerebrate cat's behavior has already been proposed. In developing this model, we introduce a memory mechanism to store the locomotion pattern, where the memory is represented as the minimal point of the potential function. The recollection process is described as a gradient system of this potential function, while in the memorization process a new pattern learning is regarded as a new minimal point generation by the bifurcation from an already existing minimal point. Finally, we discuss the generalization of this model to motion adaptation and learning.     

A new graph-like classification method applied to ancient handwritten musical symbols

Several algorithms have been proposed in the past to solve the problem of binary pattern recognition. The problem of finding features that clearly distinguish two or more different patterns is a key issue in the design of such algorithms. In this paper, a graph-like recognition process is proposed that combines a number of different classifiers to simplify the type of features and classifiers used in each classification step. The graph-like classification method is applied to ancient music optical recogniti on, and a high degree of accuracy has been achieved.Received: 27 December 2002, Accepted: 10 January 2003, Published online: 4 July 2003