Data fusion and transfer learning empowered granular trust evaluation for Internet of Things

In the Internet of Things (IoT), a huge amount of valuable data is generated by various IoT applications. As the IoT technologies become more complex, the attack methods are more diversified and can cause serious damages. Thus, establishing a secure IoT network based on user trust evaluation to defend against security threats and ensure the reliability of data source of collected data have become urgent issues, in this paper, a Data Fusion and transfer learning empowered granular Trust Evaluation mechanism (DFTE) is proposed to address the above challenges. Specifically, to meet the granularity demands of trust evaluation, time–space empowered fine/coarse grained trust evaluation models are built utilizing deep transfer learning algorithms based on data fusion. Moreover, to prevent privacy leakage and task sabotage, a dynamic reward and punishment mechanism is developed to encourage honest users by dynamically adjusting the scale of reward or punishment and accurately evaluating users’ trusts. The extensive experiments show that: (i) the proposed DFTE achieves high accuracy of trust evaluation under different granular demands through efficient data fusion; (ii) DFTE performs excellently in participation rate and data reliability.     

Efficient Flexible M-Tree Bulk Loading Using FastMap and Space-Filling Curves

Many database applications currently deal with objects in a metric space. Examples of such objects include unstructured multimedia objects and points of interest (POIs) in a road network. The M-tree is a dynamic index structure that facilitates an efficient search for objects in a metric space. Studies have been conducted on the bulk loading of large datasets in an M-tree. However, because previous algorithms involve excessive distance computations and disk accesses, they perform poorly in terms of their index construction and search capability. This study proposes two efficient M-tree bulk loading algorithms. Our algorithms minimize the number of distance computations and disk accesses using FastMap and a space-filling curve, thereby significantly improving the index construction and search performance. Our second algorithm is an extension of the first, and it incorporates a partitioning clustering technique and flexible node architecture to further improve the search performance. Through the use of various synthetic and real-world datasets, the experimental results demonstrated that our algorithms improved the index construction performance by up to three orders of magnitude and the search performance by up to 20.3 times over the previous algorithm.     

Deep reinforcement learning-based cooperative interactions among heterogeneous vehicular networks

Most real-world vehicle nodes can be structured into an interconnected network of vehicles. Through structuring these services and vehicle device interactions into multiple types, such internet of vehicles becomes multidimensional heterogeneous overlay networks. The heterogeneousness of the overlays makes it difficult for the overlay networks to coordinate with each other to improve their performance. Therefore, it poses an interesting but critical challenge to the effective analysis of heterogeneous virtual vehicular networks. A variety of virtual vehicular networks can be easily deployed onto the native network by applying the concept of SDN (Software Defined Networking). These virtual networks reflect their heterogeneousness due to their different performance goals, and they compete for the same physical resources of the underlying network, so that a sub-optimal performance of the virtual networks may be achieved. Therefore, we propose a Deep Reinforcement Learning (DRL) approach to make the virtual networks cooperate with each other through the SDN controller. A cooperative solution based on the asymmetric Nash bargaining is proposed for co-existing virtual networks to improve their performance. Moreover, the Markov Chain model and DRL resolution are introduced to leverage the heterogeneous performance goals of virtual networks. The implementation of the approach is introduced, and simulation results confirm the performance improvement of the latency sensitive, loss-rate sensitive and throughput sensitive heterogeneous vehicular networks using our cooperative solution.     

Vertical cyclic loading response of geosynthetic-encased stone column in soft clay

Dynamic responses of the geosynthetic-encased stone column (GESC) supported embankment under traffic loads have become a hot topic. This study investigates the responses of GESC improved ground under vertical cyclic loading. A series of laboratory tests in a designed model test tank have been carried out with different loading parameters (varied loading amplitudes and frequencies), different column dimensions (varied encasement lengths and column diameters). In the tests, the soil-column stress distribution, accumulated settlement of loading plate, excess pore water pressure in the surrounding soil and lateral bulging of the stone column are monitored. Experimental results indicate that the vertical stress on the stone column increases with the increment of encasement length, and decreases with the increment of column diameter, loading amplitude and loading frequency. The increasing stress on the surrounding soil leads to a greater accumulated settlement of the loading plate and excess pore water pressure, while the increasing stress on the column leads to larger lateral bulging of the column. Excess pore water pressure dissipates effectively through vertical and horizontal drainage channels provided by the stone column and the sand bed. The geosynthetic encasement prevents the clay from obstructing the drainage channel by filtration and guarantees the drainage effect.     

Stress distribution in thick-walled cylinder due to non-uniform radial pressure on the example of reinforcement corrosion in concrete

This paper presents an analytical solution to the non-uniform pressure on thick-walled cylinder. The formulation is based on the linear elasticity theory (plain strain) and stress function method. As an example, the proposed solution is used to model the stress distribution due to non-uniform steel reinforcement corrosion in concrete. The model is formulated considering different scenarios of corrosion pressure distribution. It is validated against the finite element model for different cases of non-uniform pressure distributions. The results show that the corrosion-induced cracks are likely to start just beyond the anodic zone. This is confirmed by the experimental tests on concrete cylinder exposed to non-uniform accelerated corrosion of steel reinforcement. The model can be effectively used to calculate the distribution of corrosion-induced stresses in concrete.     

Influence of domain switching process on the electrical fatigue behavior of ferroelectrics

We report an experimental approach, designed based on the recent findings that domain switching in ferroelectric ceramics can be separated into three regimes during antiparallel electric field loading, to investigate the influence of domain switching process on the electrical fatigue behavior of ferroelectrics. Uniaxial compressive stress (−2 MPã -100 MPa) and thermal loading (20 °C–150 °C) were used to tune the domain switching process. Under the same loading condition, the bipolar electrical fatigue behavior of soft lead zirconate titanate ceramics was systematically characterized. The amplitude and frequency of the applied electric field are 2 kV/mm and 10 Hz, respectively. By analyzing the evolution of the domain switching process, combined with the measured polarization and strain response, as well as the cracks observed on the surface of the specimen, it is found that the fatigue of ferroelectric ceramics was mainly related to the domain switching process near the coercive electric field: the regime 2 defined in this paper. The underlying mechanism was further discussed by considering the interplay between the domain switching process with the main factors affecting the electrical fatigue of ferroelectrics, namely defect redistribution, charge carrier injection, and crack initiation.     

When Does Communication Learning Need Hierarchical Multi-Agent Deep Reinforcement Learning

Abstract

Multi-agent systems need to communicate to coordinate a shared task. We show that a recurrent neural network (RNN) can learn a communication protocol for coordination, even if the actions to coordinate are performed steps after the communication phase. We show that a separation of tasks with different temporal scale is necessary for successful learning. We contribute a hierarchical deep reinforcement learning model for multi-agent systems that separates the communication and coordination task from the action picking through a hierarchical policy. We further on show, that a separation of concerns in communication is beneficial but not necessary. As a testbed, we propose the Dungeon Lever Game and we extend the Differentiable Inter-Agent Learning (DIAL) framework. We present and compare results from different model variations on the Dungeon Lever Game.     

Integration of imitation learning using GAIL and reinforcement learning using task-achievement rewards via probabilistic graphical model

The integration of reinforcement learning (RL) and imitation learning (IL) is an important problem that has long been studied in the field of intelligent robotics. RL optimizes policies to maximize the cumulative reward, whereas IL attempts to extract general knowledge about the trajectories demonstrated by experts, i.e, demonstrators. Because each has its own drawbacks, many methods combining them and compensating for each set of drawbacks have been explored thus far. However, many of these methods are heuristic and do not have a solid theoretical basis. This paper presents a new theory for integrating RL and IL by extending the probabilistic graphical model (PGM) framework for RL, control as inference. We develop a new PGM for RL with multiple types of rewards, called probabilistic graphical model for Markov decision processes with multiple optimality emissions (pMDP-MO). Furthermore, we demonstrate that the integrated learning method of RL and IL can be formulated as a probabilistic inference of policies on pMDP-MO by considering the discriminator in generative adversarial imitation learning (GAIL) as an additional optimality emission. We adapt the GAIL and task-achievement reward to our proposed framework, achieving significantly better performance than policies trained with baseline methods.     

Seismic behavior of geosynthetic-reinforced retaining walls backfilled with cohesive soil

This study investigates the seismic performance of geosynthetic-reinforced modular block retaining walls backfilled with cohesive, fine grained clay-sand soil mixture. Shaking table tests were performed for three ½ scaled (wall height 190 cm) and ¼ scaled model walls to investigate the effects of backfill type, the influence of reinforcement length and reinforcement stiffness effects. The El Centro and Kobe earthquake records of varying amplitudes were used as base acceleration. Displacement of the front wall, accelerations at different locations, strains on the reinforcements, and the visual observations of the facing and the backfill surface were used to evaluate the seismic performance of model walls. The model walls were subjected to rigorous shaking and the walls did not exhibit any stability problems or signs of impending failure. The maximum deformations observed on the models with cohesive backfill was less than half of the deformation of the sand model. The load transfers between the geogrid and cohesive soil was comparable to that of sand and hence the needed reinforcement length was similar as well. As a result; the model walls with cohesive backfills performed within acceptable limits under seismic loading conditions when compared with granular backfilled counterparts.     

A Novel Beam Search to Improve Neural Machine Translation for English-Chinese

Neural Machine Translation (NMT) is an end-to-end learning approach for automated translation, overcoming the weaknesses of conventional phrase-based translation systems. Although NMT based systems have gained their popularity in commercial translation applications, there is still plenty of room for improvement. Being the most popular search algorithm in NMT, beam search is vital to the translation result. However, traditional beam search can produce duplicate or missing translation due to its target sequence selection strategy. Aiming to alleviate this problem, this paper proposed neural machine translation improvements based on a novel beam search evaluation function. And we use reinforcement learning to train a translation evaluation system to select better candidate words for generating translations. In the experiments, we conducted extensive experiments to evaluate our methods. CASIA corpus and the 1,000,000 pairs of bilingual corpora of NiuTrans are used in our experiments. The experiment results prove that the proposed methods can effectively improve the English to Chinese translation quality.