Evolutionary Multi-task Learning for Modular Knowledge Representation in Neural Networks

The brain can be viewed as a complex modular structure with features of information processing through knowledge storage and retrieval. Modularity ensures that the knowledge is stored in a manner where any complications in certain modules do not affect the overall functionality of the brain. Although artificial neural networks have been very promising in prediction and recognition tasks, they are limited in terms of learning algorithms that can provide modularity in knowledge representation that could be helpful in using knowledge modules when needed. Multi-task learning enables learning algorithms to feature knowledge in general representation from several related tasks. There has not been much work done that incorporates multi-task learning for modular knowledge representation in neural networks. In this paper, we present multi-task learning for modular knowledge representation in neural networks via modular network topologies. In the proposed method, each task is defined by the selected regions in a network topology (module). Modular knowledge representation would be effective even if some of the neurons and connections are disrupted or removed from selected modules in the network. We demonstrate the effectiveness of the method using single hidden layer feedforward networks to learn selected n-bit parity problems of varying levels of difficulty. Furthermore, we apply the method to benchmark pattern classification problems. The simulation and experimental results, in general, show that the proposed method retains performance quality although the knowledge is represented as modules.     

使用程序聚类技术的模块重构风险分析方法

随着软件系统的演化,其模块化结构会逐渐退化。软件重构是调整系统结构的重要手段,但哪些模块最需要重构却难以预测。提出了一种基于程序聚类技术的模块重构风险分析方法,该方法通过对目标系统进行结构聚类和语义聚类获得其参考模块化结构,然后比较现实模块化结构与参考模块化结构之间的差异,对程序模块的设计质量进行评价,识别出系统中重构风险较高的模块。实验以三个开源软件的演化历史作为研究对象,与传统的模块化度量方法进行了比较,结果表明采用所提方法获得的预测结果与实际重构活动有较好的吻合度,从而验证了该方法的有效性。     

加权Fast Newman模块化算法在人脑结构网络中的应用

针对二值人脑结构网络的模块化方法不足以反映复杂的人脑生理特征这一问题,提出一种基于Fast Newman二值算法的加权脑网络模块化算法。该算法以凝聚节点的层次聚类思想为基础,以脑网络中单个脑区节点的权重值和脑网络总权重值为主要依据构建加权模块度评价指标,并将其增量作为度量值来确定加权脑网络中节点的合并从而实现模块划分。将该算法应用于60个健康人的组平均数据中的实验结果显示,与二值人脑网络模块化结果相对比,所提算法得到的模块度提高了28%,并且模块内部和模块外部的特征区分更加明显,所得到的人脑模块也更符合已知的人脑生理特性;而与现有的两种加权模块化算法实验对比结果表明,所提算法在合理划分人脑网络模块结构的同时也小幅提高了模块度。     

基于社团检测的复杂网络中心性方法

论证了社团检测函数模块密度的优化进程能转化为核矩阵的特征谱分.基于核矩阵最大特征值对应的特征向量,提出了一种新的中心性方法,称为模块密度中心性方法.与以往中心性度量方法不同,这种方法以模块密度检测复杂网络中的社团结构为基础,度量了第一个节点到它分配社团上的贡献,对社团的贡献越大,该节点的中心性值越高,反之亦然.通过合成网络和标准数据集网络,验证了该方法,并同其他中心性方法进行了比较,实验表明提出的模块密度中心性方法对网络中关键节点有更好的解和稳定性.进一步在计算机产生的两个大的随机网络和来自现实世界的两个大的复杂网络中,研究了模块密度中心性方法的统计分布.结果表明了提出的中心性方法能够刻画复杂网络的拓扑结构属性.     

PolyTop: a Matlab implementation of a general topology optimization framework using unstructured polygonal finite element meshes

We present an efficient Matlab code for structural topology optimization that includes a general finite element routine based on isoparametric polygonal elements which can be viewed as the extension of linear triangles and bilinear quads. The code also features a modular structure in which the analysis routine and the optimization algorithm are separated from the specific choice of topology optimization formulation. Within this framework, the finite element and sensitivity analysis routines contain no information related to the formulation and thus can be extended, developed and modified independently. We address issues pertaining to the use of unstructured meshes and arbitrary design domains in topology optimization that have received little attention in the literature. Also, as part of our examination of the topology optimization problem, we review the various steps taken in casting the optimal shape problem as a sizing optimization problem. This endeavor allows us to isolate the finite element and geometric analysis parameters and how they are related to the design variables of the discrete optimization problem. The Matlab code is explained in detail and numerical examples are presented to illustrate the capabilities of the code.     

Modularity and the Number of Design Choices

This paper analyzes one aspect of “modularity” in the architectural literature. Arguments can be made in favor of modularity, but the authors use mathematics to prove their argument against empty modularity. If we have a large quantity of structural information, then modular design can organize this information to prevent randomness and sensory overload. In that case, the module is not an empty module, but a rich, complex module containing a considerable amount of substructure. Empty modules, on the other hand, eliminate internal information, and their repetition eliminates information from the entire region that they cover. Modularity works in a positive sense only when there is substructure to organize.     

基于复杂网络理论的产甲烷常温菌和嗜热菌的代谢网络的拓扑与模块化的研究

近几年,复杂网络的研究正成为广泛关注的热点,代谢网络是复杂网络的一个例子。本文以产甲烷的常温古细菌Methanosarcina acetivorans(M.acetivorans)和嗜热古细菌Methanopyrus kandleri(M.kandleri)的代谢网络为对象,从拓扑参数以及模块化两方面进行比较研究。结果表明:M.acetivorans与M.kandleri的代谢网络均具有较高的模块化结构。同时发现它们模块化后的代谢网络中的Hub模块均属于氨基酸代谢和碳水化合物代谢,表明这些网络模块均具有一定的功能意义。最后将Hub模块与最紧密的k-核心网络相比较,发现它们节点完全相同,此结果表明代谢网络的最紧密k-核心网络部分也是不同网络比较的重要因素。     

A control theoretic framework for modular analysis and design of biomolecular networks

Control theory has been instrumental for the analysis and design of a number of engineering systems, including aerospace and transportation systems, robotics and intelligent machines, manufacturing chains, electrical, power, and information networks. In the past several years, the ability of de novo creating biomolecular networks and of measuring key physical quantities has come to a point in which quantitative analysis and design of biological systems is possible. While a modular approach to analyze and design complex systems has proven critical in most control theory applications, it is still subject of debate whether a modular approach is viable in biomolecular networks. In fact, biomolecular networks display context-dependent behavior, that is, the input/output dynamical properties of a module change once this is part of a network. One cause of context dependence, similar to what found in many engineering systems, is retroactivity, that is, the effect of loads applied on a module by downstream systems. In this paper, we focus on retroactivity and review techniques, based on nonlinear control and dynamical systems theory, that we have developed to quantify the extent of modularity of biomolecular systems and to establish modular analysis and design techniques.     

Dual modularity optimization for detecting overlapping communities in bipartite networks

Many algorithms have been designed to discover community structure in networks. These algorithms are mostly dedicated to detecting disjoint communities. Very few of them are intended to discover overlapping communities, particularly the bipartite networks have hardly been explored for the detection of such communities. In this paper, we describe a new approach which consists in forming overlapping mixed communities in a bipartite network based on dual optimization of modularity. To this end, we propose two algorithms. The first one is an evolutionary algorithm dedicated for global optimization of the Newman’s modularity on the line graph. This algorithm has been tested on well-known real benchmark networks and compared with several other existing methods of community detection in networks. The second one is an algorithm that locally optimizes the graph Mancoridis modularity, and we have adapted to a bipartite graph. Specifically, this second algorithm is applied to the decomposition of vertices, resulting from the evolutionary process, and also characterizes the overlapping communities taking into account their semantic aspect. Our approach requires a priori no knowledge on the number of communities searched in the network. We show its interest on two datasets, namely, a group of synthetic networks and real-world network whose structure is also difficult to understand.     

Evolutionary Learning of Modular Neural Networks with Genetic Programming

Evolutionary design of neural networks has shown a great potential as a powerful optimization tool. However, most evolutionary neural networks have not taken advantage of the fact that they can evolve from modules. This paper presents a hybrid method of modular neural networks and genetic programming as a promising model for evolutionary learning. This paper describes the concepts and methodologies for the evolvable model of modular neural networks, which might not only develop new functionality spontaneously, but also grow and evolve its own structure autonomously. We show the potential of the method by applying an evolved modular network to a visual categorization task with handwritten digits. Sophisticated network architectures as well as functional subsystems emerge from an initial set of randomly-connected networks. Moreover, the evolved neural network has reproduced some of the characteristics of natural visual system, such as the organization of coarse and fine processing of stimuli in separate pathways.     

Multi-level learning based memetic algorithm for community detection

Complex network has become an important way to analyze the massive disordered information of complex systems, and its community structure property is indispensable to discover the potential functionality of these systems. The research on uncovering the community structure of networks has attracted great attentions from various fields in recent years. Many community detection approaches have been proposed based on the modularity optimization. Among them, the algorithms which optimize one initial solution to a better one are easy to get into local optima. Moreover, the algorithms which are susceptible to the optimized order are easy to obtain unstable solutions. In addition, the algorithms which simultaneously optimize a population of solutions have high computational complexity, and thus they are difficult to apply to practical problems. To solve the above problems, in this study, we propose a fast memetic algorithm with multi-level learning strategies for community detection by optimizing modularity. The proposed algorithm adopts genetic algorithm to optimize a population of solutions and uses the proposed multi-level learning strategies to accelerate the optimization process. The multi-level learning strategies are devised based on the potential knowledge of the node, community and partition structures of networks, and they work on the network at nodes, communities and network partitions levels, respectively. Extensive experiments on both benchmarks and real-world networks demonstrate that compared with the state-of-the-art community detection algorithms, the proposed algorithm has effective performance on discovering the community structure of networks.     

二谱分聚类的复杂网络社团检测方法

近年来,复杂网络中的社团发现越来越受到研究人员的关注并且许多方法被提了出来。为有效地检测复杂网络中的社团结构,优化了评估与发现社团的模块密度函数(即D值)。通过模块密度的优化进程,证明了模块密度函数能写成模块密度矩阵迹的最大化表达形式。利用模块密度矩阵的谱分分解,提出了一种新的二谱分的聚类检测复杂网络社团方法。在LFR标准人工模型网络中验证了二谱分方法的有效性。实验结果显示这种新的方法在发现复杂网络社团上有较高的准确性。     

Using augmenting modular neural networks to evolve neuro-controllers for a team of underwater vehicles

The paper presents a new generative neuro-evolutionary method called augmenting modular neural networks (AMNN). As the name of the method implies, its purpose is to construct neural networks with a modular architecture. In addition to the modularity itself, neural networks evolving according to AMNN are also characterized by gradually expanding architecture. In the beginning of the evolutionary process, all networks consist of only output modules (or a single module). After some time, if the architecture of all networks is insufficient to effectively perform a task, all of them are augmented by one hidden module. In the following generations, further hidden modules are also added and this procedure is continued until some stopping criterion is satisfied. To test performance of AMNN, the method was used to evolve neuro-controllers for a team of underwater vehicles whose common goal was to capture other vehicle behaving by a deterministic strategy (predator–prey problem). The experiments were carried out in simulation, whereas their results were used to compare AMNN with neuro-evolutionary methods designed for building monolithic neural networks.     

基于最小割图分割的社区发现算法

该文证明了模块度最大化问题可以被转换成为原网络上的最小割图分割问题,并且基于该证明提出了一种高效的社区发现算法。同时,该文创新性地将模块度理论与当今比较流行的统计推理模型相结合: 首先,这些统计推理模型被转化为模块度最大化问题中的零模型;其次,统计推理模型中的目标函数被修改并应用于本文的最优化算法中。实验结果显示,无论是在真实世界网络还是在人工生成网络中,该文提出的算法均具有高效和稳定的发现社区的能力。     

复杂网络中的社团结构发现方法

社团结构是真实复杂网络异质性与模块化特性的反映。深入研究网络的社团结构有助于揭示错综复杂的真 实网络是怎样由许多相对独立而又互相关联的社区形成的,使人们更好地理解系统不同层次的结构和功能,具有广泛 的实用价值。总结了目前常用的社区发现方法,包括经典的GN算法、模块度优化算法、基于网络动力学的方法以及 统计推断方法;用社区划分基准测试网络Zachary对上述算法进行了实验,对这几类算法的时间复杂度和优缺点进行 了比较分析。最后,对复杂网络的社区结构发现算法的研究进行了展望。     

A multiobjective box-covering algorithm for fractal modularity on complex networks

The box-covering method is widely used on measuring the fractal property on complex networks. The problem of finding the minimum number of boxes to tile a network is known as a NP-hard problem. Many algorithms have been proposed to solve this problem. All the current box-covering algorithms regard the box number minimization as the only objective. However, the fractal modularity of the network partition divided by the box-covering method, has been proved to be strongly related to the information transportation in complex networks. Maximizing the fractal modularity is also important in the box-covering method, which can be divided into two objectives: maximization of ratio association and minimization of ratio cut. In this paper, to solve the dilemma of minimizing the box number and maximizing the fractal modularity at the same time, a multiobjective discrete particle swarm optimization box-covering (MOPSOBC) algorithm is proposed. The MOPSOBC algorithm applies the decomposition approach on the two objectives to approximate the Pareto front. The proposed MOPSOBC algorithm has been applied to six benchmark networks and compared with the state-of-the-art algorithms, including two classical box-covering algorithms, four single objective optimization algorithms and six multiobjective optimization algorithms. The experimental results show that the MOPSOBC algorithm can get similar box numbers with the current best algorithm, and it outperforms the state-of-the-art algorithms on the fractal modularity and normalized mutual information.     

PSO-optimized modular neural network trained by OWO-HWO algorithm for fault location in analog circuits

Fault diagnosis of analog circuits is a key problem in the theory of circuit networks and has been investigated by many researchers in recent decades. In this paper, an active filter circuit is used as the circuit under test (CUT) and is simulated in both fault-free and faulty conditions. A modular neural network model is proposed in this paper for soft fault diagnosis of the CUT. To optimize the structure of neural network modules in the proposed scheme, particle swarm optimization (PSO) algorithm is used to determine the number of hidden layer nodes of neural network modules. In addition, the output weight optimization–hidden weight optimization (OWO-HWO) training algorithm is employed, instead of conventional output weight optimization–backpropagation (OWO-BP) algorithm, to improve convergence speed in training of the neural network modules in proposed modular model. The performance of the proposed method is compared to that of monolithic multilayer perceptrons (MLPs) trained by OWO-BP and OWO-HWO algorithms, K-nearest neighbor (KNN) classifier and a related system with the same CUT. Experimental results show that the PSO-optimized modular neural network model which is trained by the OWO-HWO algorithm offers higher correct fault location rate in analog circuit fault diagnosis application as compared to the classic and monolithic investigated neural models.     

Cost-weight trades for modular composite structures

A design approach for airframe structures is formulated based on the concept of modularity allowing trade-offs and optimization between cost and weight. A modular structure can be created by replacing a collection of parts which all have a unique design by a collection of parts where the same design repeats multiple times. Structures with high levels of modularity have higher weight since it is harder to design a weight-efficient structure when the amount of design options is limited, but this weight increase might be worth the associated decrease in manufacturing cost. In modular design, cost reductions are achieved through learning curve effects and through reduction of the non-recurring cost, for example, due to lower tooling costs. Based on dynamic programming, an approach to determine the optimum number of repeating designs was determined and applied to a composite fuselage structure. Two examples are given where the cost-weight efficiency at different modularity levels is assessed for a composite airframe: the stringers and the frames in a fuselage. The corresponding cost-weight diagrams indicated that the modularity concept provides a useful methodology for designing more cost- weight efficient structures. In both cases it was possible to replace a large amount of designs and increase the level of modularity of the structure, yielding significant reductions in recurring and non-recurring manufacturing costs while keeping the associated weight increase of the structure to a minimum.     

复杂网络大数据中重叠社区检测算法

大数据时代互联网用户数量呈爆炸性增长,社交网络、电商交易网络等复杂网络规模快速发展,准确有效地检测复杂网络大数据中重叠社区结构对用户兴趣点推荐和热点传播具有重要意义。提出一种新的面向复杂网络大数据的重叠社区检测算法DOC（Detecting Overlapping Communities over complex network big data）,时间复杂度为O（nlog²（n））,算法基于模块度聚类和图计算思想应用新的节点和边的更新方法,利用平衡二叉树对模块度增量建立索引,基于模块度最优的思想设计一种新的重叠社区检测算法。相对于传统重叠节点检测算法,对每个节点分析的频率大大降低,可以在较低的算法运行时间下获得较高的识别准确率。复杂网络大数据集上的算法测试结果表明：DOC算法能够有效地检测出网络重叠社区,社区识别准确率较高,在大规模LFR基准数据集上其重叠社区检测标准化互信息指标NMI最高能达到0.97,重叠节点检测指标F-score的平均值在0.91以上,且复杂网络大数据下的运行时间明显优于传统算法。     

Discovering relationships between modularity and cost

While much has changed in product modularity research in the 18 years since the independence axiom, some basic questions remain unanswered. Perhaps the most fundamental of those questions is whether increasing modularity actually saves money. The goal of the research behind this paper was to clearly define the fundamental relationship between product modularity and product cost. Our previous work in modular product design provided a complete package of a product modularity measure and a modular design method. The “best” measure was created and verified after correcting common performance problems among the seven measures, finally subtracting the averaged relationships external to modules from the averaged relationship within modules. After comparing and finding better design elements among four representative modular design methods, the “best” method was developed that includes product decomposition, multi-component reconfiguration and elimination, and an extended limiting factor identification. The “best” method/measure package quickly yields redesign products with higher modularity. To seek out relationships between product life-cycle modularity and product life-cycle cost, modular product design experiments were implemented for four off-the-shelf products using the new measure/method package applied to increase both functional and retirement modularity. The modularity data recorded for each redesign included retirement modularity, manufacturing modularity and assembly modularity. Each redesign’s life-cycle cost was also obtained based on several classical cost models. The cost data recorded for each redesign included retirement cost, manufacturing cost, and assembly cost. The best relationships came from the retirement viewpoint. However, there is not a significant relationship between any life-cycle modularity and any life-cycle cost unless there are significantly large modularity changes. Life-cycle modularity-cost relationships are more likely to exist in data pools generated from that life-cycle redesign viewpoint. The beginning of modular redesign, where greater modularity improvements are seen, is more effective at reducing costs. Cost savings depend the appropriateness of the modularity matrix’s product architecture representation from a cost savings viewpoint.