Using distributed ledger technology to democratize neural network training

Artificial Intelligence has regained research interest, primarily because of big data. Internet expansion, social networks and online sensors led to the generation of an enormous amount of information daily. This unprecedented data availability boosted Machine Learning. A research area that has greatly benefited from this fact is Deep Neural Networks. Nowadays many use cases require huge models with millions of parameters and big data are proven to be essential to their proper training. The scientific community has proposed several methods to generate more accurate models. Usually, these methods need high performance infrastructure, which limits their applicability to large organizations and institutions that have the required funds. Another source of concern is privacy; anyone using the leased processing power of a remote data center, must trust another entity with their data. Unfortunately, in many cases sensitive data were leaked, either for financial exploitation or due to security issues. However, there is a lack of research studies when it comes to open communities of individuals with commodity hardware, who wish to join forces in a way that is non-binding and without the need for a central authority. Our work on LEARNAE attempts to fill this gap, by creating a way of providing training in Artificial Neural Networks, featuring decentralization, data ownership and fault tolerance. This article adds some important pieces to the puzzle: It studies the resilience of LEARNAE when dealing with network disruptions and proposes a novel way of embedding low-energy sensors that reside in the Internet of Things domain, retaining at the same time the established distributed philosophy.

     

A distributed approach to node clustering in decentralized peer-to-peer networks

Connectivity-based node clustering has wide-ranging applications in decentralized peer-to-peer (P2P) networks such as P2P file sharing systems, mobile ad-hoc networks, P2P sensor networks, and so forth. This paper describes a connectivity-based distributed node clustering scheme (CDC). This scheme presents a scalable and efficient solution for discovering connectivity-based clusters in peer networks. In contrast to centralized graph clustering algorithms, the CDC scheme is completely decentralized and it only assumes the knowledge of neighbor nodes instead of requiring a global knowledge of the network (graph) to be available. An important feature of the CDC scheme is its ability to cluster the entire network automatically or to discover clusters around a given set of nodes. To cope with the typical dynamics of P2P networks, we provide mechanisms to allow new nodes to be incorporated into appropriate existing clusters and to gracefully handle the departure of nodes in the clusters. These mechanisms enable the CDC scheme to be extensible and adaptable in the sense that the clustering structure of the network adjusts automatically as nodes join or leave the system. We provide detailed experimental evaluations of the CDC scheme, addressing its effectiveness in discovering good quality clusters and handling the node dynamics. We further study the types of topologies that can benefit best from the connectivity-based distributed clustering algorithms like CDC. Our experiments show that utilizing message-based connectivity structure can considerably reduce the messaging cost and provide better utilization of resources, which in turn improves the quality of service of the applications executing over decentralized peer-to-peer networks.     

SKY: efficient peer-to-peer networks based on distributed Kautz graphs

Many proposed P2P networks are based on traditional interconnection topologies. Given a static topology, the maintenance mechanism for node join/departure is critical to designing an efficient P2P network. Kautz graphs have many good properties such as constant degree, low congestion and optimal diameter. Due to the complexity in topology maintenance, however, to date there have been no effective P2P networks that are proposed based on Kautz graphs with base > 2. To address this problem, this paper presents the “distributed Kautz (D-Kautz) graphs”, which adapt Kautz graphs to the characteristics of P2P networks. Using the D-Kautz graphs we further propose SKY, the first effective P2P network based on Kautz graphs with arbitrary base. The effectiveness of SKY is demonstrated through analysis and simulations. Supported partially by the National Natural Science Foundation of China (Grant Nos. 60673167 and 60703072), the Hunan Provincial Natural Science Foundation of China (Grant No. 08JJ3125), and the National Basic Research Program of China (973) (Grant No. 2005CB321801)     

基于分布式多子网神经网络的可燃气体分析

根据催化传感器在不同的电场条件下具有不同气体检测灵敏度的特点,介绍了采用单一的热催化传感器在不同的电场强度下,通过分布式多子网神经网络对含未知气体的可燃混合气体进行分析的新方法。应用分布式神经网络,通过训练建立了信号识别的模型,并以3种混合气体为对象进行实验,结果证明了分析方法的可行性。实验表明：该网络在泛化能力与学习速度等均优于BP和RBF网络,其多子网、自动分解任务的特点尤其适用于复杂样本的学习,具有很好的应用前景。     

Using a distributed quadtree index in peer-to-peer networks

Peer-to-peer (P2P) networks have become a powerful means for online data exchange. Currently, users are primarily utilizing these networks to perform exact-match queries and retrieve complete files. However, future more data intensive applications, such as P2P auction networks, P2P job-search networks, P2P multiplayer games, will require the capability to respond to more complex queries such as range queries involving numerous data types including those that have a spatial component. In this paper, a distributed quadtree index that adapts the MX-CIF quadtree is described that enables more powerful accesses to data in P2P networks. This index has been implemented for various prototype P2P applications and results of experiments are presented. Our index is easy to use, scalable, and exhibits good load-balancing properties. Similar indices can be constructed for various multidimensional data types with both spatial and non-spatial components.     

Plant identification based on very deep convolutional neural networks

Multimedia Tools and Applications - Plant identification is a critical step in protecting plant diversity. However, many existing identification systems prohibitively rely on hand-crafted features...     

Background subtraction based on deep convolutional neural networks features

Multimedia Tools and Applications - Background modeling and subtraction, the task to detect moving objects in a scene, is a fundamental and critical step for many high level computer vision tasks....     

基于概率神经网络的入侵检测技术

提出一种基于概率神经网络的高效入侵检测技术。对网络数据处理、概率神经网络的训练与检测及其算法进行分析。在网络训练中,提出一种基于实验数据选择概率神经网络关键参数的方法,分析该方法的可行性。实验表明通过此方法能使入侵检测系统具有更高的检测精度和效率。     

MT-混合数据的深度网络快速训练方法

为加速神经网络的训练,提出一种名为MT (mix training)的模型训练方法,并从理论与实验的角度来解释这种方法。该方法直接加权平均两张不同的图片为一张,对标签以同样的权值进行加权平均。由于只使用融合后的图片进行训练,该方法能够有效地加速网络的训练。使用DenseNet-40 (k=12)作为网络结构,在CIFAR-10、CIFAR-100、SVHN这3个数据集上验证了该方法能够节约一半的训练时间,在CIFAR-10、CIFAR-100上分别达到了93.51%、73.40%的识别率,高于未使用该方法的识别率93.00%、72.45%。     

基于神经网络的图像盲水印技术

将水印嵌入到宿主图像的小波变换域的低频分量;利用BP神经网络的自学习、自适应的特性和一段已知序列训练神经网络,根据确定的神经网络模型可实现水印的盲提取;在神经网络的输入信号计算上提出选择邻域窗口为3＊3方形窗口比十字窗口具有更好的实验效果.仿真实验结果表明该算法对常用的图像处理如JPEG压缩、剪切、加噪和滤波等攻击具有较好的鲁棒性和不可见性.     

一种基于自适应遗传算法的神经网络学习算法

结合遗传算法与梯度下降法优点,提出了一种训练神经网络权值的混合优化算法,同时能够优化网络的结构.首先利用全局搜索能力可靠的遗传算法,采用递阶编码方案和自适应变异概率,同时优化网络的权值和结构,在进化结束时,能够寻到全局最优点附近的点.在遗传算法搜索结果的基础上,利用局部寻优能力较强的梯度下降法,从此点出发,进行局部搜索,最终达到网络的训练目标.与单一的遗传算法或者梯度下降法比较而言,混合优化算法的收敛速度明显提高.     

基于非结构化对等网络的分散式桌面网格平台

描述了一个基于非结构化对等网络、可以共享网络上空闲资源的JVMs虚拟机的桌面网格平台UDDG(unstructureddecentralized desktop grid)。提出了对等实体的最小活跃邻居节点数、更新时间域值等概念,每个对等实体维护了一个最小活跃邻居数的列表,结合非结构化对等网络的网络跳数的机制,通过广播查找消息来寻找资源的虚拟机。通过构建评测环境,运行并行案例程序计算结果表明,UDDG提供了一种构建高性能的桌面网格平台的新思路。     

Introducing artificial evolution into peer-to-peer networks with the distributed remodeling framework

A peer-to-peer (P2P) network is a complex system whose elements (peer nodes, or simply peers) cooperate to implement scalable distributed services. From a general point of view, the activities of a P2P system are consequences of external inputs coming from the environment, and of the internal feedbacks among nodes. The reaction of a peer to direct or indirect inputs from the environment is dictated by its functional structure, which is usually defined in terms of static rules (protocols) shared among peers. The introduction of artificial evolution mechanisms may improve the efficiency of P2P networks, with respect to resource consumption, while preserving high performance in response to the environmental needs. In this paper, we propose the distributed remodeling framework (DRF), a general approach for the design of efficient environment-driven peer-to-peer networks. As a case study, we consider an ultra-large-scale storage and computing system whose nodes perform lookups for resources provided by other nodes, to cope with task execution requests that cannot be fulfilled locally. Thanks to the DRF, workload modifications trigger reconfigurations at the level of single peers, from which global system adaptation emerges without centralized control.     

Accelerated training algorithm for feedforward neural networks based on least squares method

A least squares based training algorithm for feedforward neural networks is presented. By decomposing each neuron of the network into a linear part and a nonlinear part, the learning error can then be minimized on each neuron by applying the least squares method to solve the linear part of the neuron. In all the problems investigated, the proposed algorithm is capable of achieving the required error level in one training iteration. Comparing to the conventional backpropagation algorithm and other fast training algorithms, the proposed training algorithm provides a major breakthrough in speeding up the training process.     

An improved model training method for residual convolutional neural networks in deep learning

Multimedia Tools and Applications - Residual convolutional neural network (R-CNN) has become a promising method for image recognition in deep learning applications. The application accuracy, as a...     

A decentralized control of interconnected systems using neural networks.

We develop a decentralized neural-network (NN) controller for a class of large-scale nonlinear systems with the high-order interconnections. The controller is a mixed NN comprised of a conventional NN and a special NN. The conventional NN is used to approximate the unknown nonlinearities in the subsystem, while a special NN is used to counter the high-order interconnections. We prove that this NN structure can achieve a stable controller for the large-scale systems.     

Stratified pooling based deep convolutional neural networks for human action recognition

Video based human action recognition is an active and challenging topic in computer vision. Over the last few years, deep convolutional neural networks (CNN) has become the most popular method and achieved the state-of-the-art performance on several datasets, such as HMDB-51 and UCF-101. Since each video has a various number of frame-level features, how to combine these features to acquire good video-level feature becomes a challenging task. Therefore, this paper proposed a novel action recognition method named stratified pooling, which is based on deep convolutional neural networks (SP-CNN). The process is mainly composed of five parts: (i) fine-tuning a pre-trained CNN on the target dataset, (ii) frame-level features extraction; (iii) the principal component analysis (PCA) method for feature dimensionality reduction; (iv) stratified pooling frame-level features to get video-level feature; and (v) SVM for multiclass classification. Finally, the experimental results conducted on HMDB-51 and UCF-101 datasets show that the proposed method outperforms the state-of-the-art.     

An efficient incentive scheme with a distributed authority infrastructure in peer-to-peer networks

Today’s peer-to-peer networks are designed based on the assumption that the participating nodes are cooperative, which does not hold in reality. Incentive mechanisms that promote cooperation must be introduced. However, the existing incentive schemes (using either reputation or virtual currency) suffer from various attacks based on false reports. Even worse, a colluding group of malicious nodes in a peer-to-peer network can manipulate the history information of its own members, and the damaging power increases dramatically with the group size. Such malicious nodes/collusions are difficult to detect, especially in a large network without a centralized authority. In this paper, we propose a new distributed incentive scheme, in which the amount that a node can benefit from the network is proportional to its contribution, malicious nodes can only attack others at the cost of their own interests, and a colluding group cannot gain advantage by cooperation regardless of its size. Consequently, the damaging power of colluding groups is strictly limited. The proposed scheme includes three major components: a distributed authority infrastructure, a key sharing protocol, and a contract verification protocol.     

Sporadic decentralized resource maintenance for P2P distributed storage networks

In this paper, we propose a novel decentralized resource maintenance strategy for peer-to-peer (P2P) distributed storage networks. Our strategy relies on the Wuala overlay network architecture, (The WUALA Project). While the latter is based, for the resource distribution among peers, on the use of erasure codes, e.g., Reed–Solomon codes, here we investigate the system behavior when a simple randomized network coding strategy is applied. We propose to replace the Wuala regular and centralized strategy for resource maintenance with a decentralized strategy, where users regenerate new fragments sporadically, namely every time a resource is retrieved. Both strategies are analyzed, analytically and through simulations, in the presence of either erasure and network coding. It will be shown that the novel sporadic maintenance strategy, when used with randomized network coding, leads to a fully decentralized solution with management complexity much lower than common centralized solutions.