Two timescale convergent Q-learning for sleep-scheduling in wireless sensor networks

In this paper, we consider an intrusion detection application for Wireless Sensor Networks. We study the problem of scheduling the sleep times of the individual sensors, where the objective is to maximize the network lifetime while keeping the tracking error to a minimum. We formulate this problem as a partially-observable Markov decision process (POMDP) with continuous state-action spaces, in a manner similar to Fuemmeler and Veeravalli (IEEE Trans Signal Process 56(5), 2091–2101, 2008). However, unlike their formulation, we consider infinite horizon discounted and average cost objectives as performance criteria. For each criterion, we propose a convergent on-policy Q-learning algorithm that operates on two timescales, while employing function approximation. Feature-based representations and function approximation is necessary to handle the curse of dimensionality associated with the underlying POMDP. Our proposed algorithm incorporates a policy gradient update using a one-simulation simultaneous perturbation stochastic approximation estimate on the faster timescale, while the Q-value parameter (arising from a linear function approximation architecture for the Q-values) is updated in an on-policy temporal difference algorithm-like fashion on the slower timescale. The feature selection scheme employed in each of our algorithms manages the energy and tracking components in a manner that assists the search for the optimal sleep-scheduling policy. For the sake of comparison, in both discounted and average settings, we also develop a function approximation analogue of the Q-learning algorithm. This algorithm, unlike the two-timescale variant, does not possess theoretical convergence guarantees. Finally, we also adapt our algorithms to include a stochastic iterative estimation scheme for the intruder’s mobility model and this is useful in settings where the latter is not known. Our simulation results on a synthetic 2-dimensional network setting suggest that our algorithms result in better tracking accuracy at the cost of only a few additional sensors, in comparison to a recent prior work.     

Stochastic-Reinforcement Learning Assisted Dynamic Power Management Model for Zone-Routing Protocol in Mobile Ad Hoc Networks

Wireless Personal Communications - The Zone Routing Protocol of Mobile Ad Hoc Networks is one of the most reliable and efficient routing protocols. However, maintaining Quality of Service,...     

MARS: Link-layer rate selection for multicast transmissions in wireless mesh networks

IEEE 802.11 devices dynamically choose among different modulation schemes and bit-rates for frame transmissions. This rate adaptation, however, is restricted only to unicast frames. Multicast (and broadcast) frames are constrained to use a fixed low bit-rate modulation, resulting in low throughput for multicast streams. Availability of high bandwidth and efficient use of the medium is crucial to support multimedia multicast streaming applications such as IPTV, especially in multihop mesh networks. To address this problem, we propose a rate adaptation algorithm for multicast transmissions in these networks. The proposed algorithm, MARS, is distributed in nature, and relies on local network measurements to select a transmission bit-rate for a given multicast group. The algorithm also facilitates the joint use of bit-rate selection and link-layer mechanisms such as acknowledgements and retransmissions to improve reliability of high throughput multicast streams. Based on implementation and evaluation on a testbed, the algorithm provides up to 600% gain in throughput compared to traditional 802.11 networks in some scenarios. Additionally, the algorithm can support multicast streams while consuming a small fraction (20%) of the resources compared to the basic 802.11 operation.     

Intraoperative brain shift compensation: accounting for dural septa

Biomechanical models that describe soft tissue deformation provide a relatively inexpensive way to correct registration errors in image-guided neurosurgical systems caused by nonrigid brain shift. Quantifying the factors that cause this deformation to sufficient precision is a challenging task. To circumvent this difficulty, atlas-based methods have been developed recently that allow for uncertainty, yet still capture the first-order effects associated with deformation. The inverse solution is driven by sparse intraoperative surface measurements, which could bias the reconstruction and affect the subsurface accuracy of the model prediction. Studies using intraoperative MR have shown that the deformation in the midline, tentorium, and contralateral hemisphere is relatively small. The dural septa act as rigid membranes supporting the brain parenchyma and compartmentalizing the brain. Accounting for these structures in models may be an important key to improving subsurface shift accuracy. A novel method to segment the tentorium cerebelli will be described, along with the procedure for modeling the dural septa. Results in seven clinical cases show a qualitative improvement in subsurface shift accuracy making the predicted deformation more congruous with previous observations in the literature. The results also suggest a considerably more important role for hyperosmotic drug modeling for the intraoperative shift correction environment.     

Novel electrically conducting polyurethanes with oligoanilines: Synthesis,conductivity, and electrochemical properties

New class of conducting polyurethanes (CPUs) containing oligoanilines, namely tetraaniline (TAni) or trianiline (TriAni), in the backbone have been synthesized and characterized by formal spectral techniques. The unique properties of these CPUs, viz., electronic conductivity and electrochemical activity arising from the presence of oligoaniline units have been evaluated. The basic polyurethane backbone is derived from toluene diisocyanate, isophorone diisocyanate or hexamethylene diisocyanate, and polypropylene glycols of molecular weight 425 and 2000. In the first category of polyurethanes, the prepolymers obtained from the above reactants were chain terminated by TAni in emeraldine base oxidation state. The conductivity of these CPUs films ranged from 1.2 × 10^?5 to 1.77 × 10^?3 S cm^?1. These polymers showed lower conductivity due to the presence of nonconjugated polyurethane segments. These CPUs exhibited slightly different electrochemical activity than that of TAni. The second category of CPUs is obtained from prepolymers by chain extension with TriAni. The conductivity of these polymers is similar to the TAni analogues but are electrochemically inactive. The anticorrosion properties of two of these polymers have also been evaluated in this study. © 2014 Wiley Periodicals, Inc. J. Appl. Polym. Sci. 2014 , 131, 40794.     

Highly active robust oxide solid solution electro-catalysts for oxygen reduction reaction for proton exchange membrane fuel cell and direct methanol fuel cell cathodes

The identification and development of novel non-noble metals based electro-catalyst exhibiting excellent electrochemical activity and stability than noble metal electro-catalyst is important for commercial development of proton exchange membrane fuel cells (PEMFCs). Such non-noble electro-catalyst with unique electronic structure and superior electrochemical performance will immensely contribute to lowering the capital cost of PEMFCs. Herein, we have identified solid solution electro-catalysts of WO₃ and IrO₂ for oxygen reduction reaction (ORR) in PEMFCs exploiting theoretical first principles approaches. The theoretical results were experimentally validated by generation of nanostructured (W_1-xIr_x)O_y (x = 0.2, 0.3; y = 2.7–2.8) electro-catalysts for ORR. (W_0.7Ir_0.3)O_y demonstrated ～43% improved electrochemical activity than Pt/C with similar loading at 0.9 V (vs RHE), respectively. Moreover, single full cell PEMFC study revealed an acceptable ～81% improved maximum power density for (W_0.7Ir_0.3)O_y than Pt/C combined with excellent long term stability. These results thus, show the potential of (W_0.7Ir_0.3)O_y as ORR electro-catalyst for replacing of Pt/C in PEMFCs and direct methanol fuel cells on the additional grounds of superior methanol tolerance.     

An analysis of the etiology of anemia and iron deficiency in young women of low socioeconomic status in Bangalore, India

BACKGROUND: Anemia and iron deficiency are significant public health problems in India, particularly among women and children. Recent figures suggest that nearly 50% of young Indian women are anemic. OBJECTIVES: Few studies have comprehensively assessed etiologic factors contributing to anemia and iron deficiency in India. Hence, this study assessed the relative importance of various factors contributing to these problems in young women of low socioeconomic status in Bangalore, India. METHODS: A random sample of 100 nonpregnant, nonlactating women 18 to 35 years of age, selected from among 511 women living in a poor urban settlement, participated in this study. Data were obtained on demography, socioeconomic status, anthropometry, three-day dietary intake, blood hemoglobin, hemoglobinopathies, serum ferritin, serum C-reactive protein, and stool parasites. RESULTS: The prevalence rates of anemia and iron deficiency were 39% and 62%, respectively; 95% of the anemic women were iron deficient. The mean dietary iron intake was 9.5 mg per day, predominantly from the consumption of cereals, pulses, and vegetables (77%). The estimated bioavailability of nonheme iron in this diet was 2.8%. Dietary intakes were suboptimal for several nutrients. Blood hemoglobin was significantly correlated with dietary intake of fat, riboflavin, milk and yogurt, and coffee. Serum ferritin was significantly correlated with intake of niacin, vitamin B12, and selenium. Parasitic infestation was low. CONCLUSIONS: An inadequate intake of dietary iron, its poor bioavailability, and concurrent inadequate intake of dietary micronutrients appear to be the primary factors responsible for the high prevalence of anemia and iron deficiency in this population.