REINFORCEMENT LEARNING FOR POMDP USING STATE CLASSIFICATION

Reinforcement learning (RL) has been widely used to solve problems with a little feedback from environment. Q learning can solve Markov decision processes (MDPs) quite well. For partially observable Markov decision processes (POMDPs), a recurrent neural network (RNN) can be used to approximate Q values. However, learning time for these problems is typically very long. We present a new combination of RL and RNN to find a good policy for POMDPs in a shorter learning time. This method contains two phases: firstly, state space is divided into two groups (fully observable state group and hidden state group); secondly, a Q value table is used to store values of fully observable states and an RNN is used to approximate values for hidden states. Results of experiments in two grid world problems show that the proposed method enables an agent to acquire a policy with better learning performance compared to the method using only a RNN.     

Estimating aboveground biomass of a mangrove plantation on the Northern coast of Vietnam using machine learning techniques with an integration of ALOS-2 PALSAR-2 and Sentinel-2A data

ABSTRACT

Aboveground biomass (AGB) of mangrove forest plays a crucial role in global carbon cycle by reducing greenhouse gas emissions and mitigating climate change impacts. Monitoring mangrove forests biomass accurately still remains challenging compared to other forest ecosystems. We investigated the usability of machine learning techniques for the estimation of AGB of mangrove plantation at a coastal area of Hai Phong city (Vietnam). The study employed a GIS database and support vector regression (SVR) to build and verify a model of AGB, drawing upon data from a survey in 25 sampling plots and an integration of Advanced Land Observing Satellite-2 Phased Array Type L-band Synthetic Aperture Radar-2 (ALOS-2 PALSAR-2) dual-polarization horizontal transmitting and horizontal receiving (HH) and horizontal transmitting and vertical receiving (HV) and Sentinel-2A multispectral data. The performance of the model was assessed using root mean square error (RMSE), mean absolute error (MAE), coefficient of determination (R²), and leave-one-out cross-validation. Usability of the SVR model was assessed by comparing with four state-of-the-art machine learning techniques, i.e. radial basis function neural networks, multi-layer perceptron neural networks, Gaussian process, and random forest. The SVR model shows a satisfactory result (R² = 0.596, RMSE = 0.187, MAE = 0.123) and outperforms the four machine learning models. The SVR model-estimated AGB ranged between 36.22 and 230.14 Mg ha^?1 (average = 87.67 Mg ha^?1). We conclude that an integration of ALOS-2 PALSAR-2 and Sentinel-2A data used with SVR model can improve the AGB accuracy estimation of mangrove plantations in tropical areas.     

Strategic parameter-driven routing models for multidestinationtraffic in telecommunication networks

We present mathematical models that determine the optimal parameters for strategically routing multidestination traffic in an end-to-end network setting. Multidestination traffic refers to a traffic type that can be routed to any one of a multiple number of destinations. A growing number of communication services is based on multidestination routing. In this parameter-driven approach, a multidestination call is routed to one of the candidate destination nodes in accordance with predetermined decision parameters associated with each candidate node. We present three different approaches: (1) a link utilization (LU) approach, (2) a network cost (NC) approach, and (3) a combined parametric (CP) approach. The LU approach provides the solution that would result in an optimally balanced link utilization, whereas the NC approach provides the least expensive way to route traffic to destinations. The CP approach, on the other hand, provides multiple solutions that help leverage link utilization and cost. The LU approach has in fact been implemented by a long distance carrier resulting in a considerable efficiency improvement in its international direct services, as summarized.     

Metal protection by anticorrosion papers in humid tropical climate

Results of 3-year full-scale field climatic tests on application of five types of inhibiting papers for anticorrosion protection of ferrous and nonferrous metals during storage in a humid tropical climate have been carried out by the Joint Russian and Vietnamese Tropical Research and Technology Center and presented here. It has been shown that the inhibiting paper IPCAS-118 can be recommended for anticorrosion protection of equipment made of steel, copper, and aluminum during 3-year storage in sheds in humid tropical climate conditions     

Automatic mesh generation for finite element analysis

The finite element analysis in engineering applications comprises three phases: domain discretization, equation solving and error analysis. The domain discretization or mesh generation is the pre-processing phase which plays an important role in the achievement of accurate solutions. In this paper, the improvement of one particularly promising technique for generating two-dimensional meshes is presented. Our technique shows advantages and efficiency over some currently available mesh generators.     

Microlens Arrays by Direct-Writing Inkjet Print for LCD Backlighting Applications

In this paper, we demonstrated the Inkjet print method to fabricate a ultraviolet (UV)-curable epoxy microlens deflecting array with controllable curvature and filling factor, by which a 7-in light guide plate with 82% uniformity and 70% light efficiency was successfully implemented. The microlenses can be directly formed by plastic discharged by inkjet head, eliminating the need for molds and slashing development time. Proposed method will certainly has a promising impact on rapid prototyping and other specialized microfeatures due to its simplicity and versatility.     

Effect of Ag addition on the corrosion properties of Sn‐based solder alloys

Corrosion properties of three different Sn‐Ag lead free solder alloys have been investigated in 0.3 wt% Na₂SO₄ solution as corrosive environment. As cast solder alloy was analyzed by X‐ray diffraction (XRD) and scanning electron microscopy (SEM). Volume fractions of the Ag₃Sn in the solders were determined by image analysis technique. Pitting potential and corrosion potential for the alloys were determined by potentiodynamic tests. Electrochemical impedance spectroscopy (EIS) was carried out to measure the film and charge transfer resistance. Alloys with lower Ag content have been found as better corrosion resistance material.     

Effect of Particle Deposition on Granular Aerosol Filtration: A Comparative Study of Methods in Evaluating and Interpreting Experimental Data

Three methods which can be applied to extract quantitative information on the effect of deposition on filter performance from effluent concentration and pressure drop data were developed. It was found that all three methods are capable of describing the deposition effect in terms of the amount of deposited particles. Furthermore, the method of extrapolation was found to give the most consistent, if not the most accurate, interpretation of experimental data.     

Robust stability of Metzler operators under parameter perturbations

In this paper we study how the spectral bound of Metzler operators changes under parameter perturbations. Characterizations of the stability radii of Metzler operator with respect to this type of disturbances are established. The results generalize those obtained in (Vietnam J. Math. 2006; 34 :357–368; Vietnam J. Math. 1998; 26 :147–163). Copyright © 2009 John Wiley & Sons, Ltd.