Transfer in variable-reward hierarchical reinforcement learning

Transfer learning seeks to leverage previously learned tasks to achieve faster learning in a new task. In this paper, we consider transfer learning in the context of related but distinct Reinforcement Learning (RL) problems. In particular, our RL problems are derived from Semi-Markov Decision Processes (SMDPs) that share the same transition dynamics but have different reward functions that are linear in a set of reward features. We formally define the transfer learning problem in the context of RL as learning an efficient algorithm to solve any SMDP drawn from a fixed distribution after experiencing a finite number of them. Furthermore, we introduce an online algorithm to solve this problem, Variable-Reward Reinforcement Learning (VRRL), that compactly stores the optimal value functions for several SMDPs, and uses them to optimally initialize the value function for a new SMDP. We generalize our method to a hierarchical RL setting where the different SMDPs share the same task hierarchy. Our experimental results in a simplified real-time strategy domain show that significant transfer learning occurs in both flat and hierarchical settings. Transfer is especially effective in the hierarchical setting where the overall value functions are decomposed into subtask value functions which are more widely amenable to transfer across different SMDPs.     

Applicability of feed-forward and recurrent neural networks to Boolean function complexity modeling

In this paper, we present the feed-forward neural network (FFNN) and recurrent neural network (RNN) models for predicting Boolean function complexity (BFC). In order to acquire the training data for the neural networks (NNs), we conducted experiments for a large number of randomly generated single output Boolean functions (BFs) and derived the simulated graphs for number of min-terms against the BFC for different number of variables. For NN model (NNM) development, we looked at three data transformation techniques for pre-processing the NN-training and validation data. The trained NNMs are used for complexity estimation for the Boolean logic expressions with a given number of variables and sum of products (SOP) terms. Both FFNNs and RNNs were evaluated against the ISCAS benchmark results. Our FFNNs and RNNs were able to predict the BFC with correlations of 0.811 and 0.629 with the benchmark results, respectively.     

Viscoelastic material design with negative stiffness components using topology optimization

An application of topology optimization to design viscoelastic composite materials with elastic moduli that soften with frequency is presented. The material is a two-phase composite whose first constituent is isotropic and viscoelastic while the other is an orthotropic material with negative stiffness but stable. A concept for this material based on a lumped parameter model is used. The performance of the topology optimization approach in this context is illustrated using three examples.     

High-temperature X-ray structural,thermal and dielectric characteristics of ferroelectric Bi2VO5.5

The X-ray powder diffraction, dielectric and thermal studies of bismuth vanadate (Bi₂VO_5.5) ceramic have been carried out as a function of temperature (300–900 K). The hightemperature X-ray studies, supported by differential scanning calorimetry, clearly demonstrate that Bi₂VO_5.5 undergoes two major phase transitions at 730 and 835 K. It was found that the one at 730 K is associated with both the ferroelectric and the crystallographic transition, while at 835 K, Bi₂VO_5.5 undergoes only the crystallographic transition. Anomalies in both the dielectric constant and specific heat curves have been observed at 730 and 835 K. The total heat, Q, and entropy, S, associated with the transition at 730 K were found to be higher than those at 835 K.     

A nano-plasmonic HMIM waveguide based concurrent dual-band BPF using circular ring resonator

This article analyzes the transmission line characteristics of plasmonic hybrid metal insulator metal (HMIM) waveguide, circular ring resonator (CRR) based dual-band band-pass filters with two transmission poles in both pass-bands in the optical regime using coupled line feed. The transmission line characteristics of an HMIM waveguide, such as characteristic impedance (ZPV), effective refractive index (neff) and propagation length (Lspp), have been obtained by using full wave simulation. Using basic HMIM slot waveguide, a CRR with periodic loading of double- and triple-ring CRR is numerically analyzed. Two input ports have been used for excitation, which are located at the separation of 180° positions along the CRR, and are coupled with the ring by parallel coupled lines, producing the dual pass-bands with the synchronous excitation of two transmission poles. The proposed double-ring dual-band band-pass filter (DR-DB-BPF) offers 35 dB extinction ratio (ER), 299.69 nm free spectral range (FSR) and narrow band full width half maximum (FWHM) of 78.057—112.43 nm. The triple-ring DB-BPF (TR-DB-BPF) has 22.5 dB ER, FSR of 292.18 nm and FWHM of 42.751—59.58 nm. The proposed filters are very useful in the development of dual-band filters for electronic photonic integrated circuits (EPICs), as the optical signals are filtered at two wavelengths simultaneously.     

Learning Noise-Assisted Robust Image Features for Fine-Grained Image Retrieval

Fine-grained image search is one of the most challenging tasks in computer vision that aims to retrieve similar images at the fine-grained level for a given query image. The key objective is to learn discriminative fine-grained features by training deep models such that similar images are clustered, and dissimilar images are separated in the low embedding space. Previous works primarily focused on defining local structure loss functions like triplet loss, pairwise loss, etc. However, training via these approaches takes a long training time, and they have poor accuracy. Additionally, representations learned through it tend to tighten up in the embedded space and lose generalizability to unseen classes. This paper proposes a noise-assisted representation learning method for fine-grained image retrieval to mitigate these issues. In the proposed work, class manifold learning is performed in which positive pairs are created with noise insertion operation instead of tightening class clusters. And other instances are treated as negatives within the same cluster. Then a loss function is defined to penalize when the distance between instances of the same class becomes too small relative to the noise pair in that class in embedded space. The proposed approach is validated on CARS-196 and CUB-200 datasets and achieved better retrieval results (85.38% recall@1 for CARS-196% and 70.13% recall@1 for CUB-200) compared to other existing methods.     

Toward Stable Cycling of a Cost-Effective Cation-Disordered Rocksalt Cathode via Fluorination

The recently developed Li-excess cation-disordered rock salts (DRXs) exhibit an excellent chemical diversity for the development of alternative Co/Ni-free high-energy cathodes. Herein, the synthesis of a highly fluorinated DRX cathode, Li_1.2Mn_0.6Ti_0.2O_1.8F_0.2, based on cost-effective and earth-abundant transition metals, via a solid-state reaction, is reported. The fluorinated DRX cathode using ammonium fluoride precursor exhibits more uniform particle size and delivers a specific discharge capacity of 233 mAh g⁻¹ and specific energy of 754 Wh kg⁻¹, with 206 mAh g⁻¹ retained after 200 cycles. The combined synchrotron X-ray absorption spectroscopy and resonant inelastic X-ray scattering spectroscopy analysis reveals that the remarkable cycling performance is attributed to the high fluorination and thus enhanced Mn content, enabling the utilization of more Mn redox than the oxide analog. This study demonstrates a great promise to develop next-generation cost-effective DRX cathodes with enhanced capacity retention for high-energy Li-ion batteries.     

Laser-treated austenitic steel and nickel alloy for human implants

The recent research in biocompatible materials has been useful in replacing and supporting the fractured natural human bones/joints. Under some condition, negative reaction like release of ions from the bare metal toward the human body fluid leads to corrosion. In this proposed research paper, the biocompatibility of the laser surface-modified austenitic stainless steel (SS316L) and nickel-based superalloy (Inconel 718) was studied. The investigation on laser-modified surfaces is evaluated through electrochemical polarization analysis using simulated body fluid (SBF). The samples subjected to electrochemical polarization analysis were characterized by optical image analysis, SEM, EDS, and XRD analysis. It was inferred that laser surface-modified materials provided enhanced corrosion resistance and bare nickel alloy is more susceptible to corrosion by SBF.     

Synthesis and characterisation of neem leaf extract, 2, 3‐dehydrosalanol and quercetin dihydrate mediated silver nano particles for therapeutic applications

The utility of green silver nanoparticles (AgNPs) in veterinary medicine is steadily increasing as they have many therapeutic applications against pathogens and arthropods of livestock. In this study, green AgNPs using neem (N‐AgNPs), 2,3‐dehydrosalanol (2,3‐DHS‐AgNPs) and quercetin dihydrate (QDH‐AgNPs) were synthesised and characterised. Synthesised compounds were characterised by UV‐Vis spectroscopy and the peak absorbance was recorded at 370 nm for neem extract. For N‐AgNPs, 2,3‐DHS‐AgNPs and QDH‐AgNPs, the maximum absorbance peaks were at 430, 230 and 220 nm, respectively. The FTIR analysis confirmed the synthesis of green AgNPs. The XRD pattern of N‐AgNPs showed the peaks corresponding to whole spectra of 2 θ values ranging from 10–80. The relatively higher intensity of (111, 222) planes in face centred cubic crystalline structure supports the formation of synthesised AgNPs. In DLS analysis, the hydrodynamic diameter of neem leaf extract was found to be 259.8 nm, followed by 5.3, 6.7 and 261.8 nm for 2,3‐DHS‐AgNPs, N‐AgNPs and QDH‐AgNPs, respectively. Based on the transmission electron microscopy and scanning electron microscopy image analyses, confirmed the formation of N‐AgNPs, 2,3‐DHS‐AgNPs and QDH‐AgNPs. These eco‐friendly phyto‐AgNPs may be of use as an effective alternative to chemical control methods against the arthropods of livestock.Inspec keywords: nanoparticles, silver, nanomedicine, biomedical materials, nanofabrication, Fourier transform infrared spectra, ultraviolet spectra, visible spectra, X‐ray diffraction, light scattering, transmission electron microscopy, scanning electron microscopy, aggregation, veterinary medicineOther keywords: 2,3‐dehydrosalanol mediated silver nanoparticles, quercetin dihydrate mediated silver nanoparticles, therapeutic applications, green silver nanoparticles, veterinary medicine, Azadirachta indica, UV‐visible spectroscopy, Fourier transformed infrared analysis, X‐ray diffraction, (111) planes, (222) planes, face centred cubic crystalline structure, dynamic light scattering, hydrodynamic diameter, aqueous neem leaf extract, transmission electron microscopy, hexagonal shape, pencil head shape, cuboid shape, scanning electron microscopy, aggregation, arthropod infesting livestock, Ag, in‐vivo antiectoparasitic activity, in‐vitro antiectoparasitic activity     

Biosynthesised silver and copper nanoformulation as foliar spray to control bird's eye spot disease in tea plantations

Tea leaves have economic importance in preparation of the popular beverage of the world “tea”. Bird’s eye spot disease of tea leaves creates significant revenue loss in tea trade of many tea plant cultivating countries. Management of this disease by silver (AgNps) and copper (CuNps) nanoparticles that are biosynthesised by efficient antagonists was studied. The biocontrol agents like Pseudomonas fluorescens, Trichoderma atroviride and Streptomyces sannanensis were evaluated for nanoparticle synthesis against Cercospora theae isolates namely KC10, MC24 and VC38. Initially, the freshly prepared extracellular AgNps showed high disease control (59.42 – 79.76%), but the stability of antagonistic property in stored nanoparticles were significantly high in CuNps (58.71 – 73.81%). Greenhouse studies on various treatments imposed also showed reduced disease incidence percentage of 13.4, 7.57 and 10.11% when treated with CuNps synthesized by P. fluorescens, T. atroviride and S. sannanensis respectively. Various treatment schedule in fields suggested the use of Bionanocopper@1.5 ppm for highest yield (3743 kg/ha) with 66.1% disease prevention. The results suggest the use of biosynthesised CuNps using Streptomyces sannanensis for controlling the tea plant pathogens causing foliar disease with higher stability in releasing the antagonistic activity during sporadic disease incidence of bird’s eye spot disease in tea plants.Inspec keywords: silver, copper, crops, plant diseases, nanoparticles, air pollution, agrochemicals, nanobiotechnologyOther keywords: biosynthesised silver, biosynthesised copper, nanoformulation, foliar spray, bird eye spot disease control, tea plantations, tea leaves, economic importance, revenue loss, tea trade, tea plant cultivating countries, silver nanoparticles, AgNps, copper nanoparticles, CuNps, biocontrol agents, nanoparticle synthesis, Cercospora theae isolates, KC10, MC24, VC38, greenhouse studies, antagonistic property, P. fluorescens, T. atroviride, S. sannanensis, fungicides, synthetic nanomaterials, bionanomaterials, disease prevention, green leaf yield, BionanoCu, tea plant pathogens, foliar disease