Cognitive engineering based knowledge representation in neural networks

A model of a human neural knowledge processing system is presented that suggests the following. First, an entity in the outside world lends to be locally encoded in neural networks so that the conceptual information structure is mirrored in its physical implementation. Second, the knowledge of problem solving is implemented in a quite implicit way in the internal structure of the neural network (a functional group of associated hidden neurons and their connections to entity neurons) not in individual neurons or connections. Third, the knowledge system is organized and implemented in a modular fashion in neural networks according to the local specialization of problem solving where a module of neural network implements an inter-related group of knowledge such as a schema, and different modules have similar processing mechanisms, but differ in their input and output patterns. A neural network module can be tuned just as a schema structure can be adapted for changing environments. Three experiments were conducted to try to validate the suggested cognitive engineering based knowledge structure in neural networks through computer simulation. The experiments, which were based on a task of modulo arithmetic, provided some insights into the plausibility of the suggested model of a neural knowledge processing system.     

Symbolic representation of neural networks

Neural networks often surpass decision trees in predicting pattern classifications, but their predictions cannot be explained. This algorithm's symbolic representations make each prediction explicit and understandable. Our approach to understanding a neural network uses symbolic rules to represent the network decision process. The algorithm, NeuroRule, extracts these rules from a neural network. The network can be interpreted by the rules which, in general, preserve network accuracy and explain the prediction process. We based NeuroRule on a standard three layer feed forward network. NeuroRule consists of four phases. First, it builds a weight decay backpropagation network so that weights reflect the importance of the network's connections. Second, it prunes the network to remove irrelevant connections and units while maintaining the network's predictive accuracy. Third, it discretizes the hidden unit activation values by clustering. Finally, it extracts rules from the network with discretized hidden unit activation values     

Neural networks and knowledge engineering

A brief overview is given of the nature of present day artificial neural net computing and the authors emphasize the theme that in this mode of computing, knowledge is not represented symbolically, but in the form of distributed processing and localized decision rules. The authors propose that in neural-net computing, the processing is the representation. In other words, the very nature of the processing encodes the knowledge. There is no place and no need for a separate body of global rules to be used by the network for inferencing. If rules exist at all, they are in the nature of local processing steps carried out at individual processors in response to stimuli from other neurons. The authors develop this theme together with a theme which is closely related to it. The second notion is that neural networks may also be thought of and implemented in terms of heterogeneous networks, rather than always or only in terms of massive arrays of identical elemental processors     

Graph representation learning via simple jumping knowledge networks

Applied Intelligence - Recent graph neural networks for graph representation learning depend on a neighborhood aggregation process. Several works focus on simplifying the neighborhood aggregation...     

Reorganizing knowledge in neural networks: an explanatory mechanismfor neural networks in data classification problems

We propose an explanatory mechanism for multilayered neural networks (NN). In spite of the effective learning capability as a uniform function approximator, the multilayered NN suffers from unreadability, i.e., it is difficult for the user to interpret or understand the "knowledge" that the NN has by looking at the connection weights and thresholds obtained by backpropagation (BP). This unreadability comes from the distributed nature of the knowledge representation in the NN. In this paper, we propose a method that reorganizes the distributed knowledge in the NN to extract approximate classification rules. Our rule extraction method is based on the analysis of the function that the NN has learned, rather than on the direct interpretation of connection weights as correlation information. More specifically, our method divides the input space into "monotonic regions" where a monotonic region is a set of input patterns that belongs to the same class with the same sensitivity pattern. Approximate classification rules are generated by projecting these monotonic regions.     

Time-delay neural networks: representation and induction offinite-state machines

In this work, we characterize and contrast the capabilities of the general class of time-delay neural networks (TDNNs) with input delay neural networks (IDNNs), the subclass of TDNNs with delays limited to the inputs. Each class of networks is capable of representing the same set of languages, those embodied by the definite memory machines (DMMs), a subclass of finite-state machines. We demonstrate the close affinity between TDNNs and DMM languages by learning a very large DMM (2048 states) using only a few training examples. Even though both architectures are capable of representing the same class of languages, they have distinguishable learning biases. Intuition suggests that general TDNNs which include delays in hidden layers should perform well, compared to IDNNs, on problems in which the output can be expressed as a function on narrow input windows which repeat in time. On the other hand, these general TDNNs should perform poorly when the input windows are wide, or there is little repetition. We confirm these hypotheses via a set of simulations and statistical analysis.     

From knowledge based software engineering to knowware based software engineering

The first part of this paper reviews our efforts on knowledge-based software engineering, namely PROMIS, started from 1990s. The key point of PROMIS is to generate applications automatically based on domain knowledge as well as software knowledge. That is featured by separating the development of domain knowledge from the development of software. But in PROMIS, we did not find an appropriate representation for the domain knowledge. Fortunately, in our recent work, we found such a carrier for knowledge modules, i.e. knowware. Knowware is a commercialized form of domain knowledge. This paper briefly introduces the basic definitions of knowware, knowledge middleware and knowware engineering. Three life circle models of knowware engineering and the design of corresponding knowware implementations are given. Finally we discuss application system automatic generation and domain knowledge modeling on the J2EE platform, which combines the techniques of PROMIS, knowware and J2EE, and the development and deployment framework, i.e. PROMIS/KW＊＊.     

基于径向基概率神经网络的工程图纸图形符号识别

基于径向基概率神经网络，提出一种扫描工程图纸图像分割后的图形符号识别方法，针对已分割的扫描工程图纸图形符号图像，首先进行二值化处理，然后对二值图形符号图像进行Hu不变矩特征提取，再使用一种新型的径向基概率神经网络进行分类，从而实现图像识别．为加快径向基概率神经网络的收敛速度，采用递归最小二乘算法进行训练．实验结果表明，径向基概率神经网络在识别性能与速度等方面非常适合于工程图纸的图形符号识别。     

Frame-based representation of philosophical systems using a knowledge engineering tool

This article addresses the methodological problem of the non-linear representation of philosophical systems in a computerized knowledge base. It is a problem of knowledge representation as defined in the field of artificial intelligence. Instead of a purely theoretical discussion of the issue, we present selected results of a practical experiment which has in itself some theoretical significance. We show how one can represent different philosophies using CODE, a knowledge engineering system developed by artificial intelligence researchers. The hypothesis is that such a computer based representation of philosophical systems can give insight into their conceptual structure. We argue that computer aided text analysis can apply knowledge representation tools and techniques developed in artificial intelligence and we estimate how philosophers as well as knowledge engineers could gain from this cross-fertilization. This paper should be considered as an experiment report on the use of knowledge representation techniques in computer aided text analysis. It is part of a much broader project on the representation of conceptual structures in an expert system. However, we intentionally avoided technical issues related to either Computer Science or History of Philosophy to focus on the benefit to enhance traditional humanistic studies with tools and methods developed in AI on the one hand and the need to develop more appropriate tools on the other. Gilbert Boss is professor of Philosophy at Université Laval, Québec. He is the author of several books, including Les machines à penser. L'homme et l'ordinateur,Zurich: Grand Midi, 1987, and John Stuart Mill. Induction et utilité,Paris: PUF, 1990. His main fields of research are modern philosophy, philosophy of culture, philosophical discourse and systems, artificial intelligence. Maryvonne Longeart is professor of Computer Science at UQAH, Hull, Québec. Her research interests include object oriented design methodologies and knowledge representation. She received a PhD in Philosophy from the University of Ottawa in 1978 and a BSc in computer science in 1987. She contributed to the Encyclopédie philosophique universelle,PUF, 1992 and published several papers on the representation of complex conceptual systems.Douglas Skuce is professor of computer science at the University of Ottawa. He has worked in the area of knowledge engineering since his PhD (McGill, 1977). During 1978-present he has been developing the CODE system for various applications, including terminology and software development. Currently, his interests include designing ontologies for knowledge exchange and coupling large corpora to systems such as CODE.     

Incorporating symbolic domain knowledge into graph neural networks

Machine Learning - Our interest is in scientific problems with the following characteristics: (1) Data are naturally represented as graphs; (2) The amount of data available is typically small; and...     

Special issue: engineering applications of neural networks

Neural Computing and Applications -     

How knowledge representation meets software engineering (and often databases)

This paper surveys a selection of personal research projects which addressed problems related to Software Engineering, and whose solution was inspired by ideas from the field of Knowledge Representation and Reasoning. Surprisingly often, the research was also related to problems in Databases. We discuss, in part, to what extent did the KR ideas provide ready-made solutions to SE and DB problems, and how frequently we had to invent new KR techniques.     

Fuzzy systems and neural networks in software engineering project management

To make reasonable estimates of resources, costs, and schedules, software project managers need to be provided with models that furnish the essential framework for software project planning and control by supplying important management numbers concerning the state and parameters of the project that are critical for resource allocation. Understanding that software development is not a mechanistic process brings about the realization that parameters that characterize the development of software possess an inherent fuzziness, thus providing the rationale for the development of realistic models based on fuzzy set or neural theories.Fuzzy and neural approaches offer a key advantage over traditional modeling approaches in that they aremodel-free estimators. This article opens up the possibility of applying fuzzy estimation theory and neural networks for the purpose of software engineering project management and control, using Putnam's manpower buildup index (MBI) estimation model as an example. It is shown that the MBI selection process can be based upon 64 different fuzzy associative memory (FAM) rules. The same rules are used to generate 64 training patterns for a feedforward neural network. The fuzzy associative memory and neural network approaches are compared qualitatively through estimation surfaces. The FAM estimation surfaces are stepped, whereas those from the neural system are smooth. Also, the FAM system sets up much faster than the neural system. FAM rules obtained from logical antecedent-consequent pairs are maintained distinct, giving the user the ability to determine which FAM rule contributed how much membership activation to a concluded output.     

Representing knowledge by neural networks for qualitative analysisand reasoning

A systematic approach has been developed to construct neural networks for qualitative analysis and reasoning. These neural networks are used as specialized parallel distributed processors for solving constraint satisfaction problems. A typical application of such a neural network is to determine a reasonable change of a system after one or more of its variables are changed. A six-node neural network is developed to represent fundamental qualitative relations. A larger neural network can be constructed hierarchically for a system to be modeled by using six-node neural networks as building blocks. The complexity of the neural network building process is thus kept manageable. An example of developing a neural network reasoning model for a transistor equivalent circuit is demonstrated. The use of this neural network model in the equivalent circuit parameter extraction process is also described     

Minimax classifiers based on neural networks

The problem of designing a classifier when prior probabilities are not known or are not representative of the underlying data distribution is discussed in this paper. Traditional learning approaches based on the assumption that class priors are stationary lead to sub-optimal solutions if there is a mismatch between training and future (real) priors. To protect against this uncertainty, a minimax approach may be desirable. We address the problem of designing a neural-based minimax classifier and propose two different algorithms: a learning rate scaling algorithm and a gradient-based algorithm. Experimental results show that both succeed in finding the minimax solution and it is also pointed out the differences between common approaches to cope with this uncertainty in priors and the minimax classifier.     

Three-dimensional object representation and invariant recognitionusing continuous distance transform neural networks

3D object recognition under partial object viewing is a difficult pattern recognition task. In this paper, we introduce a neural-network solution that is robust to partial viewing of objects and noise corruption. This method directly utilizes the acquired 3D data and requires no feature extraction. The object is first parametrically represented by a continuous distance transform neural network (CDTNN) trained by the surface points of the exemplar object. The CDTNN maps any 3D coordinate into a value that corresponds to the distance between the point and the nearest surface point of the object. Therefore, a mismatch between the exemplar object and an unknown object can be easily computed. When encountered with deformed objects, this mismatch information can be backpropagated through the CDTNN to iteratively determine the deformation in terms of affine transform. Application to 3D heart contour delineation and invariant recognition of 3D rigid-body objects is presented.     

Content-based image retrieval by combining convolutional neural networks and sparse representation

Multimedia Tools and Applications - As stored data and images on memory disks increase, image retrieval has a necessary task on image processing. Although lots of researches have been reported for...     

A new complicated-knowledge representation approach based on knowledge meshes

This paper presents a new complicated-knowledge representation method for the self-reconfiguration of complex systems such as complex software systems, complex manufacturing systems, and knowledgeable manufacturing systems. Herein, new concepts of a knowledge mesh (KM) and an agent mesh (AM) are proposed along with a new KM-based approach to complicated-knowledge representation. KM is the representation of such complicated macroknowledge as an advanced manufacturing mode, focusing on knowledge about the structure, functions, and information flows of an advanced manufacturing system. The multiple set, KM, and the mapping relationships between both, are then formally defined. The union, intersection, and minus operations on the multiple sets are proposed, and their properties proved. Then, the perfectness of a KM, the redundancy set between the two KMs, and the multiple redundancy set on the redundancy set are defined. Three examples are provided to illustrate the concepts of the KM, multiple set, multiple redundancy set, and logical operations. On the basis of the above, the KM-based inference engine is presented. In logical operations on KMs, each KM is taken as an operand. A new KM obtained by operations on KM multiple sets can be mapped into an AM for automatic reconfiguration of complex software systems. Finally, the combination of two real management modes is exemplified for the effective application of the new KM-based method to the self-reconfiguration of complex systems. It is worth mentioning that KM multiple sets can also be taken as a new formal representation of software systems if their corresponding AMs are the real software systems.