Transfer in variable-reward hierarchical reinforcement learning

Transfer learning seeks to leverage previously learned tasks to achieve faster learning in a new task. In this paper, we consider transfer learning in the context of related but distinct Reinforcement Learning (RL) problems. In particular, our RL problems are derived from Semi-Markov Decision Processes (SMDPs) that share the same transition dynamics but have different reward functions that are linear in a set of reward features. We formally define the transfer learning problem in the context of RL as learning an efficient algorithm to solve any SMDP drawn from a fixed distribution after experiencing a finite number of them. Furthermore, we introduce an online algorithm to solve this problem, Variable-Reward Reinforcement Learning (VRRL), that compactly stores the optimal value functions for several SMDPs, and uses them to optimally initialize the value function for a new SMDP. We generalize our method to a hierarchical RL setting where the different SMDPs share the same task hierarchy. Our experimental results in a simplified real-time strategy domain show that significant transfer learning occurs in both flat and hierarchical settings. Transfer is especially effective in the hierarchical setting where the overall value functions are decomposed into subtask value functions which are more widely amenable to transfer across different SMDPs.     

Applicability of feed-forward and recurrent neural networks to Boolean function complexity modeling

In this paper, we present the feed-forward neural network (FFNN) and recurrent neural network (RNN) models for predicting Boolean function complexity (BFC). In order to acquire the training data for the neural networks (NNs), we conducted experiments for a large number of randomly generated single output Boolean functions (BFs) and derived the simulated graphs for number of min-terms against the BFC for different number of variables. For NN model (NNM) development, we looked at three data transformation techniques for pre-processing the NN-training and validation data. The trained NNMs are used for complexity estimation for the Boolean logic expressions with a given number of variables and sum of products (SOP) terms. Both FFNNs and RNNs were evaluated against the ISCAS benchmark results. Our FFNNs and RNNs were able to predict the BFC with correlations of 0.811 and 0.629 with the benchmark results, respectively.     

BLAST-SSAHA Hybridization for Credit Card Fraud Detection

A phenomenal growth in the number of credit card transactions, especially for online purchases, has recently led to a substantial rise in fraudulent activities. Implementation of efficient fraud detection systems has thus become imperative for all credit card issuing banks to minimize their losses. In real life, fraudulent transactions are interspersed with genuine transactions and simple pattern matching is not often sufficient to detect them accurately. Thus, there is a need for combining both anomaly detection as well as misuse detection techniques. In this paper, we propose to use two-stage sequence alignment in which a profile analyzer (PA) first determines the similarity of an incoming sequence of transactions on a given credit card with the genuine cardholder's past spending sequences. The unusual transactions traced by the profile analyzer are next passed on to a deviation analyzer (DA) for possible alignment with past fraudulent behavior. The final decision about the nature of a transaction is taken on the basis of the observations by these two analyzers. In order to achieve online response time for both PA and DA, we suggest a new approach for combining two sequence alignment algorithms BLAST and SSAHA.     

Viscoelastic material design with negative stiffness components using topology optimization

An application of topology optimization to design viscoelastic composite materials with elastic moduli that soften with frequency is presented. The material is a two-phase composite whose first constituent is isotropic and viscoelastic while the other is an orthotropic material with negative stiffness but stable. A concept for this material based on a lumped parameter model is used. The performance of the topology optimization approach in this context is illustrated using three examples.     

High-temperature X-ray structural,thermal and dielectric characteristics of ferroelectric Bi2VO5.5

The X-ray powder diffraction, dielectric and thermal studies of bismuth vanadate (Bi₂VO_5.5) ceramic have been carried out as a function of temperature (300–900 K). The hightemperature X-ray studies, supported by differential scanning calorimetry, clearly demonstrate that Bi₂VO_5.5 undergoes two major phase transitions at 730 and 835 K. It was found that the one at 730 K is associated with both the ferroelectric and the crystallographic transition, while at 835 K, Bi₂VO_5.5 undergoes only the crystallographic transition. Anomalies in both the dielectric constant and specific heat curves have been observed at 730 and 835 K. The total heat, Q, and entropy, S, associated with the transition at 730 K were found to be higher than those at 835 K.     

Learning Noise-Assisted Robust Image Features for Fine-Grained Image Retrieval

Fine-grained image search is one of the most challenging tasks in computer vision that aims to retrieve similar images at the fine-grained level for a given query image. The key objective is to learn discriminative fine-grained features by training deep models such that similar images are clustered, and dissimilar images are separated in the low embedding space. Previous works primarily focused on defining local structure loss functions like triplet loss, pairwise loss, etc. However, training via these approaches takes a long training time, and they have poor accuracy. Additionally, representations learned through it tend to tighten up in the embedded space and lose generalizability to unseen classes. This paper proposes a noise-assisted representation learning method for fine-grained image retrieval to mitigate these issues. In the proposed work, class manifold learning is performed in which positive pairs are created with noise insertion operation instead of tightening class clusters. And other instances are treated as negatives within the same cluster. Then a loss function is defined to penalize when the distance between instances of the same class becomes too small relative to the noise pair in that class in embedded space. The proposed approach is validated on CARS-196 and CUB-200 datasets and achieved better retrieval results (85.38% recall@1 for CARS-196% and 70.13% recall@1 for CUB-200) compared to other existing methods.     

BS-SC Model: A Novel Method for Predicting Child Abuse Using Borderline-SMOTE Enabled Stacking Classifier

For a long time, legal entities have developed and used crime prediction methodologies. The techniques are frequently updated based on crime evaluations and responses from scientific communities. There is a need to develop type-based crime prediction methodologies that can be used to address issues at the subgroup level. Child maltreatment is not adequately addressed because children are voiceless. As a result, the possibility of developing a model for predicting child abuse was investigated in this study. Various exploratory analysis methods were used to examine the city of Chicago’s child abuse events. The data set was balanced using the Borderline-SMOTE technique, and then a stacking classifier was employed to ensemble multiple algorithms to predict various types of child abuse. The proposed approach successfully predicted crime types with 93% of accuracy, precision, recall, and F1-Score. The AUC value of the same was 0.989. However, when compared to the Extra Trees model (17.55), which is the second best, the proposed model’s execution time was significantly longer (476.63). We discovered that Machine Learning methods effectively evaluate the demographic and spatial-temporal characteristics of the crimes and predict the occurrences of various subtypes of child abuse. The results indicated that the proposed Borderline-SMOTE enabled Stacking Classifier model (BS-SC Model) would be effective in the real-time child abuse prediction and prevention process.     

Toward Stable Cycling of a Cost-Effective Cation-Disordered Rocksalt Cathode via Fluorination

The recently developed Li-excess cation-disordered rock salts (DRXs) exhibit an excellent chemical diversity for the development of alternative Co/Ni-free high-energy cathodes. Herein, the synthesis of a highly fluorinated DRX cathode, Li_1.2Mn_0.6Ti_0.2O_1.8F_0.2, based on cost-effective and earth-abundant transition metals, via a solid-state reaction, is reported. The fluorinated DRX cathode using ammonium fluoride precursor exhibits more uniform particle size and delivers a specific discharge capacity of 233 mAh g⁻¹ and specific energy of 754 Wh kg⁻¹, with 206 mAh g⁻¹ retained after 200 cycles. The combined synchrotron X-ray absorption spectroscopy and resonant inelastic X-ray scattering spectroscopy analysis reveals that the remarkable cycling performance is attributed to the high fluorination and thus enhanced Mn content, enabling the utilization of more Mn redox than the oxide analog. This study demonstrates a great promise to develop next-generation cost-effective DRX cathodes with enhanced capacity retention for high-energy Li-ion batteries.     

Detection of Malicious Profiles and Protecting Users in Online Social Networks

Wireless Personal Communications - Everyone today actively uses online social networks to get in touch with their friends, for career opportunities, and business also. Some of the most popular...