Data fusion and transfer learning empowered granular trust evaluation for Internet of Things

In the Internet of Things (IoT), a huge amount of valuable data is generated by various IoT applications. As the IoT technologies become more complex, the attack methods are more diversified and can cause serious damages. Thus, establishing a secure IoT network based on user trust evaluation to defend against security threats and ensure the reliability of data source of collected data have become urgent issues, in this paper, a Data Fusion and transfer learning empowered granular Trust Evaluation mechanism (DFTE) is proposed to address the above challenges. Specifically, to meet the granularity demands of trust evaluation, time–space empowered fine/coarse grained trust evaluation models are built utilizing deep transfer learning algorithms based on data fusion. Moreover, to prevent privacy leakage and task sabotage, a dynamic reward and punishment mechanism is developed to encourage honest users by dynamically adjusting the scale of reward or punishment and accurately evaluating users’ trusts. The extensive experiments show that: (i) the proposed DFTE achieves high accuracy of trust evaluation under different granular demands through efficient data fusion; (ii) DFTE performs excellently in participation rate and data reliability.     

Deep reinforcement learning-based cooperative interactions among heterogeneous vehicular networks

Most real-world vehicle nodes can be structured into an interconnected network of vehicles. Through structuring these services and vehicle device interactions into multiple types, such internet of vehicles becomes multidimensional heterogeneous overlay networks. The heterogeneousness of the overlays makes it difficult for the overlay networks to coordinate with each other to improve their performance. Therefore, it poses an interesting but critical challenge to the effective analysis of heterogeneous virtual vehicular networks. A variety of virtual vehicular networks can be easily deployed onto the native network by applying the concept of SDN (Software Defined Networking). These virtual networks reflect their heterogeneousness due to their different performance goals, and they compete for the same physical resources of the underlying network, so that a sub-optimal performance of the virtual networks may be achieved. Therefore, we propose a Deep Reinforcement Learning (DRL) approach to make the virtual networks cooperate with each other through the SDN controller. A cooperative solution based on the asymmetric Nash bargaining is proposed for co-existing virtual networks to improve their performance. Moreover, the Markov Chain model and DRL resolution are introduced to leverage the heterogeneous performance goals of virtual networks. The implementation of the approach is introduced, and simulation results confirm the performance improvement of the latency sensitive, loss-rate sensitive and throughput sensitive heterogeneous vehicular networks using our cooperative solution.     

Stress distribution in thick-walled cylinder due to non-uniform radial pressure on the example of reinforcement corrosion in concrete

This paper presents an analytical solution to the non-uniform pressure on thick-walled cylinder. The formulation is based on the linear elasticity theory (plain strain) and stress function method. As an example, the proposed solution is used to model the stress distribution due to non-uniform steel reinforcement corrosion in concrete. The model is formulated considering different scenarios of corrosion pressure distribution. It is validated against the finite element model for different cases of non-uniform pressure distributions. The results show that the corrosion-induced cracks are likely to start just beyond the anodic zone. This is confirmed by the experimental tests on concrete cylinder exposed to non-uniform accelerated corrosion of steel reinforcement. The model can be effectively used to calculate the distribution of corrosion-induced stresses in concrete.     

When Does Communication Learning Need Hierarchical Multi-Agent Deep Reinforcement Learning

Abstract

Multi-agent systems need to communicate to coordinate a shared task. We show that a recurrent neural network (RNN) can learn a communication protocol for coordination, even if the actions to coordinate are performed steps after the communication phase. We show that a separation of tasks with different temporal scale is necessary for successful learning. We contribute a hierarchical deep reinforcement learning model for multi-agent systems that separates the communication and coordination task from the action picking through a hierarchical policy. We further on show, that a separation of concerns in communication is beneficial but not necessary. As a testbed, we propose the Dungeon Lever Game and we extend the Differentiable Inter-Agent Learning (DIAL) framework. We present and compare results from different model variations on the Dungeon Lever Game.     

Integration of imitation learning using GAIL and reinforcement learning using task-achievement rewards via probabilistic graphical model

The integration of reinforcement learning (RL) and imitation learning (IL) is an important problem that has long been studied in the field of intelligent robotics. RL optimizes policies to maximize the cumulative reward, whereas IL attempts to extract general knowledge about the trajectories demonstrated by experts, i.e, demonstrators. Because each has its own drawbacks, many methods combining them and compensating for each set of drawbacks have been explored thus far. However, many of these methods are heuristic and do not have a solid theoretical basis. This paper presents a new theory for integrating RL and IL by extending the probabilistic graphical model (PGM) framework for RL, control as inference. We develop a new PGM for RL with multiple types of rewards, called probabilistic graphical model for Markov decision processes with multiple optimality emissions (pMDP-MO). Furthermore, we demonstrate that the integrated learning method of RL and IL can be formulated as a probabilistic inference of policies on pMDP-MO by considering the discriminator in generative adversarial imitation learning (GAIL) as an additional optimality emission. We adapt the GAIL and task-achievement reward to our proposed framework, achieving significantly better performance than policies trained with baseline methods.     

A Novel Beam Search to Improve Neural Machine Translation for English-Chinese

Neural Machine Translation (NMT) is an end-to-end learning approach for automated translation, overcoming the weaknesses of conventional phrase-based translation systems. Although NMT based systems have gained their popularity in commercial translation applications, there is still plenty of room for improvement. Being the most popular search algorithm in NMT, beam search is vital to the translation result. However, traditional beam search can produce duplicate or missing translation due to its target sequence selection strategy. Aiming to alleviate this problem, this paper proposed neural machine translation improvements based on a novel beam search evaluation function. And we use reinforcement learning to train a translation evaluation system to select better candidate words for generating translations. In the experiments, we conducted extensive experiments to evaluate our methods. CASIA corpus and the 1,000,000 pairs of bilingual corpora of NiuTrans are used in our experiments. The experiment results prove that the proposed methods can effectively improve the English to Chinese translation quality.     

Assessment of the effectiveness of steel fibre reinforcement for the punching resistance of flat slabs by experimental research and design approach

The present paper deals with the experimental assessment of the effectiveness of steel fibre reinforcement in terms of punching resistance of centrically loaded flat slabs, and to the development of an analytical model capable of predicting the punching behaviour of this type of structures. For this purpose, eight slabs of 2550 × 2550 × 150 mm³ dimensions were tested up to failure, by investigating the influence of the content of steel fibres (0, 60, 75 and 90 kg/m³) and concrete strength class (50 and 70 MPa). Two reference slabs without fibre reinforcement, one for each concrete strength class, and one slab for each fibre content and each strength class compose the experimental program. All slabs were flexurally reinforced with a grid of ribbed steel bars in a percentage to assure punching failure mode for the reference slabs. Hooked ends steel fibres provided the unique shear reinforcement. The results have revealed that steel fibres are very effective in converting brittle punching failure into ductile flexural failure, by increasing both the ultimate load and deflection, as long as adequate fibre reinforcement is assured. An analytical model was developed based on the most recent concepts proposed by the fib Mode Code 2010 for predicting the punching resistance of flat slabs and for the characterization of the behaviour of fibre reinforced concrete. The most refined version of this model was capable of predicting the punching resistance of the tested slabs with excellent accuracy and coefficient of variation of about 5%.     

掘进巷道快速过断层施工技术应用

基于四候煤矿3108回风顺槽过F3断层期间,顶板出现严重破碎、下沉现象,提出了俯斜台阶法施工工艺,并对断层破碎带顶板采取MF-2型化学材料注浆加固以及"钢筋锚索网+U29梯形棚"联合支护措施。支护效果检验结果表明,联合支护有效控制了断层破碎区顶板下沉、破碎现象,保证了巷道顶板稳定性。     

基于Q-learning的不确定环境BDI Agent最优策略规划研究

BDI模型能够很好地解决在特定环境下的Agent的推理和决策问题,但在动态和不确定环境下缺少决策和学习的能力。强化学习解决了Agent在未知环境下的决策问题,却缺少BDI模型中的规则描述和逻辑推理。针对BDI在未知和动态环境下的策略规划问题,提出基于强化学习Q-learning算法来实现BDI Agent学习和规划的方法,并针对BDI的实现模型ASL的决策机制做出了改进,最后在ASL的仿真平台Jason上建立了迷宫的仿真,仿真实验表明,在加入Q-learning学习机制后的新的ASL系统中,Agent在不确定环境下依然可以完成任务。     

Dissociating uncertainty responses and reinforcement signals in the comparative study of uncertainty monitoring.

Although researchers are exploring animals' capacity for monitoring their states of uncertainty, the use of some paradigms allows the criticism that animals map avoidance responses to error-causing stimuli not because of uncertainty monitored but because of feedback signals and stimulus aversion. The authors addressed this criticism with an uncertainty-monitoring task in which participants completed blocks of trials with feedback deferred so that they could not associate reinforcement signals to particular stimuli or stimulus-response pairs. Humans and 1 of 2 monkeys were able to make cognitive, decisional uncertainty responses that were independent of feedback or reinforcement history within a task. This finding unifies the comparative literature on uncertainty monitoring. The dissociation of performance from reinforcement has theoretical implications, and the deferred-feedback technique has many applications. (PsycINFO Database Record (c) 2010 APA, all rights reserved)