Transfer in variable-reward hierarchical reinforcement learning

Transfer learning seeks to leverage previously learned tasks to achieve faster learning in a new task. In this paper, we consider transfer learning in the context of related but distinct Reinforcement Learning (RL) problems. In particular, our RL problems are derived from Semi-Markov Decision Processes (SMDPs) that share the same transition dynamics but have different reward functions that are linear in a set of reward features. We formally define the transfer learning problem in the context of RL as learning an efficient algorithm to solve any SMDP drawn from a fixed distribution after experiencing a finite number of them. Furthermore, we introduce an online algorithm to solve this problem, Variable-Reward Reinforcement Learning (VRRL), that compactly stores the optimal value functions for several SMDPs, and uses them to optimally initialize the value function for a new SMDP. We generalize our method to a hierarchical RL setting where the different SMDPs share the same task hierarchy. Our experimental results in a simplified real-time strategy domain show that significant transfer learning occurs in both flat and hierarchical settings. Transfer is especially effective in the hierarchical setting where the overall value functions are decomposed into subtask value functions which are more widely amenable to transfer across different SMDPs.     

Applicability of feed-forward and recurrent neural networks to Boolean function complexity modeling

In this paper, we present the feed-forward neural network (FFNN) and recurrent neural network (RNN) models for predicting Boolean function complexity (BFC). In order to acquire the training data for the neural networks (NNs), we conducted experiments for a large number of randomly generated single output Boolean functions (BFs) and derived the simulated graphs for number of min-terms against the BFC for different number of variables. For NN model (NNM) development, we looked at three data transformation techniques for pre-processing the NN-training and validation data. The trained NNMs are used for complexity estimation for the Boolean logic expressions with a given number of variables and sum of products (SOP) terms. Both FFNNs and RNNs were evaluated against the ISCAS benchmark results. Our FFNNs and RNNs were able to predict the BFC with correlations of 0.811 and 0.629 with the benchmark results, respectively.     

Efficient Design of Rounding Based Static Segment Imprecise Multipliers for Error Tolerance Application

Journal of Electronic Testing - Error-Tolerant applications regularly accomplish more data adaption. Approximate computing is one of the optimum strategies for data manipulation in several...     

Control of bandwidth,resonant frequency,and modelling of bandpass filter using open stub resonator for K-band application

Microsystem Technologies - In this paper, we present the implementation of a new compact end coupled bandpass filter (BPF) in coplanar waveguide (CPW) technology. The proposed BPF is exposed with...     

Design,simulation and analysis of RF MEMS capacitive shunt switches with high isolation and low pull-in-voltage

Microsystem Technologies - This paper presents the design a capacitive shunt type RF-MEMS switch with high isolation, high switching speed and low actuation voltage for Ka-band applications. The...     

Laser-treated austenitic steel and nickel alloy for human implants

The recent research in biocompatible materials has been useful in replacing and supporting the fractured natural human bones/joints. Under some condition, negative reaction like release of ions from the bare metal toward the human body fluid leads to corrosion. In this proposed research paper, the biocompatibility of the laser surface-modified austenitic stainless steel (SS316L) and nickel-based superalloy (Inconel 718) was studied. The investigation on laser-modified surfaces is evaluated through electrochemical polarization analysis using simulated body fluid (SBF). The samples subjected to electrochemical polarization analysis were characterized by optical image analysis, SEM, EDS, and XRD analysis. It was inferred that laser surface-modified materials provided enhanced corrosion resistance and bare nickel alloy is more susceptible to corrosion by SBF.     

Biosynthesised silver and copper nanoformulation as foliar spray to control bird's eye spot disease in tea plantations

Tea leaves have economic importance in preparation of the popular beverage of the world “tea”. Bird’s eye spot disease of tea leaves creates significant revenue loss in tea trade of many tea plant cultivating countries. Management of this disease by silver (AgNps) and copper (CuNps) nanoparticles that are biosynthesised by efficient antagonists was studied. The biocontrol agents like Pseudomonas fluorescens, Trichoderma atroviride and Streptomyces sannanensis were evaluated for nanoparticle synthesis against Cercospora theae isolates namely KC10, MC24 and VC38. Initially, the freshly prepared extracellular AgNps showed high disease control (59.42 – 79.76%), but the stability of antagonistic property in stored nanoparticles were significantly high in CuNps (58.71 – 73.81%). Greenhouse studies on various treatments imposed also showed reduced disease incidence percentage of 13.4, 7.57 and 10.11% when treated with CuNps synthesized by P. fluorescens, T. atroviride and S. sannanensis respectively. Various treatment schedule in fields suggested the use of Bionanocopper@1.5 ppm for highest yield (3743 kg/ha) with 66.1% disease prevention. The results suggest the use of biosynthesised CuNps using Streptomyces sannanensis for controlling the tea plant pathogens causing foliar disease with higher stability in releasing the antagonistic activity during sporadic disease incidence of bird’s eye spot disease in tea plants.Inspec keywords: silver, copper, crops, plant diseases, nanoparticles, air pollution, agrochemicals, nanobiotechnologyOther keywords: biosynthesised silver, biosynthesised copper, nanoformulation, foliar spray, bird eye spot disease control, tea plantations, tea leaves, economic importance, revenue loss, tea trade, tea plant cultivating countries, silver nanoparticles, AgNps, copper nanoparticles, CuNps, biocontrol agents, nanoparticle synthesis, Cercospora theae isolates, KC10, MC24, VC38, greenhouse studies, antagonistic property, P. fluorescens, T. atroviride, S. sannanensis, fungicides, synthetic nanomaterials, bionanomaterials, disease prevention, green leaf yield, BionanoCu, tea plant pathogens, foliar disease     

Foley catheters functionalised with a synergistic combination of antibiotics and silver nanoparticles resist biofilm formation

Foley catheters are inevitable in health care unit. Pathogens colonise and form biofilm on catheter causing catheter‐associated urinary tract infection. Therefore, the authors aimed to functionalise catheter to resist biofilm formation. The authors impregnated urinary catheters with a synergistic combination of antibiotics and silver nanoparticles (SNPs) to evaluate antibiofilm efficacy in vitro and in vivo. SNPs were synthesised using Spirulina platensis. Synergy between the SNPs and antibiotics was determined by the checker‐board method. In vivo efficacy of the functionalised catheters was assessed in mice. Liver and kidney function tests of mice were performed. The in vitro anti‐adherence activity of the functionalised catheters was evaluated after 2 years. Nanoparticle sizes were 42–75 nm. Synergistic activity was observed among SNPs (2 µg/ml), amikacin (6.25 µg/ml), and nitrofurantoin (31.25 µg/ml). In mice, catheters functionalised with combinations of antibiotics and SNPs exhibited no colonisation until Day 14. Blood, liver, and kidney tests were normal. After 2 years, catheters functionalised with antibiotics exhibited 25% inhibition of bacterial adhesion, and catheters functionalised with the nanoparticle‐antibiotic combination exhibited 90% inhibition. Impregnation of urinary catheters with a synergistic combination of antibiotics and SNPs is an efficient and promising method for preventing biofilm formation.Inspec keywords: catheters, drugs, silver, nanoparticles, nanomedicine, liver, kidney, blood, microorganisms, adhesion, biomechanics, cellular biophysicsOther keywords: Foley catheters, synergistic nanoparticle‐antibiotics combination, silver nanoparticles, biofilm formation resitance, health care unit, pathogens, urinary tract infection, SNP, Spirulina platensis, checker‐board method, liver function, kidney function, vitro antiadherence activity, amikacin, nitrofurantoin, blood, bacterial adhesion, size 42 nm to 75 nm, Ag     

Automatic text classification using BPLion-neural network and semantic word processing

Text mining has become a major research topic in which text classification is the important task for finding the relevant information from the new document. Accordingly, this paper presents a semantic word processing technique for text categorization that utilizes semantic keywords, instead of using independent features of the keywords in the documents. Hence, the dimensionality of the search space can be reduced. Here, the Back Propagation Lion algorithm (BP Lion algorithm) is also proposed to overcome the problem in updating the neuron weight. The proposed text classification methodology is experimented over two data sets, namely, 20 Newsgroup and Reuter. The performance of the proposed BPLion is analysed, in terms of sensitivity, specificity, and accuracy, and compared with the performance of the existing works. The result shows that the proposed BPLion algorithm and semantic processing methodology classifies the documents with less training time and more classification accuracy of 90.9%.     

An improved anti-forensics JPEG compression using Least Cuckoo Search algorithm

The lossy nature of the JPEG compression leaves traces which are utilized by the forensic agents to identify the local tampering in the image. In this paper, a tricky anti-forensic method has been proposed to remove the traces left by the JPEG compression in both the spatial domain and discrete cosine transform domain. A novel Least Cuckoo Search algorithm is devised in the proposed anti-forensic compression scheme. Moreover, a new fitness function called histogram deviation is formulated in the optimization algorithm. The experimentation of the proposed anti-forensic compression scheme is performed over uncompressed images from UCID database. The performance of the proposed method is evaluated, and it is compared with the existing methods using PSNR, MSE and classification accuracy as measures. The experimentation ensued with promising results, i.e. accuracy of 0.97, PSNR of 44.34?dB, and MSE of 0.1789 which prove the efficacy of the proposed method.