Rollout sampling approximate policy iteration

Several researchers have recently investigated the connection between reinforcement learning and classification. We are motivated by proposals of approximate policy iteration schemes without value functions, which focus on policy representation using classifiers and address policy learning as a supervised learning problem. This paper proposes variants of an improved policy iteration scheme which addresses the core sampling problem in evaluating a policy through simulation as a multi-armed bandit machine. The resulting algorithm offers comparable performance to the previous algorithm achieved, however, with significantly less computational effort. An order of magnitude improvement is demonstrated experimentally in two standard reinforcement learning domains: inverted pendulum and mountain-car.     

Development of a Mortality Prediction Model in Hospitalised SARS-CoV-2 Positive Patients Based on Routine Kidney Biomarkers

Acute kidney injury (AKI) is a prevalent complication in severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) positive inpatients, which is linked to an increased mortality rate compared to patients without AKI. Here we analysed the difference in kidney blood biomarkers in SARS-CoV-2 positive patients with non-fatal or fatal outcome, in order to develop a mortality prediction model for hospitalised SARS-CoV-2 positive patients. A retrospective cohort study including data from suspected SARS-CoV-2 positive patients admitted to a large National Health Service (NHS) Foundation Trust hospital in the Yorkshire and Humber regions, United Kingdom, between 1 March 2020 and 30 August 2020. Hospitalised adult patients (aged ≥ 18 years) with at least one confirmed positive RT-PCR test for SARS-CoV-2 and blood tests of kidney biomarkers within 36 h of the RT-PCR test were included. The main outcome measure was 90-day in-hospital mortality in SARS-CoV-2 infected patients. The logistic regression and random forest (RF) models incorporated six predictors including three routine kidney function tests (sodium, urea; creatinine only in RF), along with age, sex, and ethnicity. The mortality prediction performance of the logistic regression model achieved an area under receiver operating characteristic (AUROC) curve of 0.772 in the test dataset (95% CI: 0.694–0.823), while the RF model attained the AUROC of 0.820 in the same test cohort (95% CI: 0.740–0.870). The resulting validated prediction model is the first to focus on kidney biomarkers specifically on in-hospital mortality over a 90-day period.     

Power Efficient Video Multipath Transmission over Wireless Multimedia Sensor Networks

This paper proposes a power efficient multipath video packet scheduling scheme for minimum video distortion transmission (optimised Video QoS) over wireless multimedia sensor networks. The transmission of video packets over multiple paths in a wireless sensor network improves the aggregate data rate of the network and minimizes the traffic load handled by each node. However, due to the lossy behavior of the wireless channel the aggregate transmission rate cannot always support the requested video source data rate. In such cases a packet scheduling algorithm is applied that can selectively drop combinations of video packets prior to transmission to adapt the source requirements to the channel capacity. The scheduling algorithm selects the less important video packets to drop using a recursive distortion prediction model. This model predicts accurately the resulting video distortion in case of isolated errors, burst of errors and errors separated by a lag. Two scheduling algorithms are proposed in this paper. The Baseline scheme is a simplified scheduler that can only decide upon which packet can be dropped prior to transmission based on the packet’s impact on the video distortion. This algorithm is compared against the Power aware packet scheduling that is an extension of the Baseline capable of estimating the power that will be consumed by each node in every available path depending on its traffic load, during the transmission. The proposed Power aware packet scheduling is able to identify the available paths connecting the video source to the receiver and schedule the packet transmission among the selected paths according to the perceived video QoS (Peak Signal to Noise Ratio—PSNR) and the energy efficiency of the participating wireless video sensor nodes, by dropping packets if necessary based on the distortion prediction model. The simulation results indicate that the proposed Power aware video packet scheduling can achieve energy efficiency in the wireless multimedia sensor network by minimizing the power dissipation across all nodes, while the perceived video quality is kept to very high levels even at extreme network conditions (many sensor nodes dropped due to power consumption and high background noise in the channel).     

On the Impact of Performance Faults in Modern Microprocessors

Modern microprocessors incorporate a variety of architectural features, such as branch prediction and speculative execution, which are not critical to the correctness of their operation yet are essential towards improving performance. Accordingly, while faults in the corresponding hardware may not necessarily affect functional correctness, they may, nevertheless, adversely impact performance. In this paper, we investigate quantitatively the performance impact of such faults using a superscalar, dynamically-scheduled, out-of-order, Alpha-like microprocessor, on which we execute SPEC2000 integer benchmarks. We provide extensive fault simulation-based experimental results that elucidate the various aspects of performance faults and we discuss how this information may guide the inclusion of additional hardware for performance loss recovery and yield enhancement.     

Possibilities of the Computer-Controlled Detonation Spraying method: A chemistry viewpoint

This article is aimed to discuss the chemical aspects of detonation spraying of powder materials. In this method of coating deposition, ceramic, metallic or composite powders are injected into the barrel of a detonation gun filled with an explosive gaseous mixture. When the latter is ignited, the powders are heated and accelerated toward the substrate. Subjected to high temperatures, the powders are prone to chemical reactions, the reaction products possibly becoming the major phase constituents of the coatings. What types of reactions are possible? Can these reactions be carried out in a controlled manner? We answer these questions considering the interactions of the sprayed powders with the gaseous environment of the barrel as well as those between the phases of a composite feedstock powder. In Computer-Controlled Detonation Spraying (CCDS), the explosive charge and stoichiometry of the fuel-oxygen mixtures are precisely measured and can be flexibly changed. Our studies demonstrate that with the introduction of a highly flexible process of CCDS, detonation spraying has entered a new development stage, at which it can be considered as a powerful method of composition and microstructure tailoring of thermally sprayed coatings. During CCDS of TiO₂-containing powders, chemical reduction of titanium dioxide can be carried out to different levels to form either oxygen-deficient TiO_2−x or Ti₃O₅ suboxide. CCDS of Ti₃Al can produce titanium oxide coatings when oxidation by the detonation products dominates or titanium nitride-titanium aluminide coatings when oxidation is hindered but the interaction of the powders with nitrogen—a carrier gas component—is favored. During detonation spraying of Ti₃SiC₂–Cu composites, the Ti₃SiC₂ phase is preserved only in cold conditions; otherwise, Si de-intercalates from the Ti₃SiC₂ phase and dissolves in Cu resulting in the formation of the TiC_x–Cu(Si) composite coatings.     

Reversible Cluster Aggregation and Growth Model for Graphene Suspensions

We present a reversible cluster aggregation model for 2‐D macromolecules represented by line segments in 2‐D; and, we use it to describe the aggregation process of functionalized graphene particles in an aqueous SDS surfactant solution. The model produces clusters with similar sizes and structures as a function of SDS concentration in agreement with experiments and predicts the existence of a critical surfactant concentration (C_crit) beyond which thermodynamically stable graphene suspensions form. Around C_crit, particles form dense clusters rapidly and sediment. At C ? C_crit, a contiguous ramified network of graphene gel forms which also densifies, but at a slower rate, and sediments with time. The deaggregation–reaggregation mechanism of our model captures the restructuring of the large aggregates towards a graphite‐like structure for the low SDS concentrations. © 2017 American Institute of Chemical Engineers AIChE J, 63: 5462–5473, 2017     

Synthesis of amphiphilic (meth)acrylates with oligo(ethylene glycol) and (or) oligo(propylene glycol) blocks by the esterification of (meth)acrylic acid

Thermoresponsive PEG-based (PEG stands polyethylene glycol) polymers are unique for use in medicine because of their low toxicity, good biocompatibility and biodegradability, but usually more hydrophobic and more toxic comonomers are used to adjust lower critical solution temperature (LCST). A convenient way to overcome this problem and to finely tune LCST is to use alkoxy oligo(ethylene glycol)- or alkoxy oligo(propylene glycol) (meth)acrylates as starting comonomers. Here we report on the conditions for the simple and affordable synthesis of methoxy oligo(propylene glycol) (meth)acrylate- and methoxy oligo(propylene glycol)-block-oligo(ethylene glycol) (meth)acrylate-based macromonomers with high yields (80%–98.7%) by the acid-catalyzed esterification of (meth)acrylic acid with alkoxy oligo(alkylene glycols) containing oligo(ethylene glycol) (OEG) and/or oligo(propylene glycol) (OPG) blocks. p-Toluene sulphonic acid (pTSA), alkyl(C₁₂–C₁₄)benzene sulfonic acid (ABSA) and H₂SO₄ were used as catalysts. It has been shown that pTSA and ABSA are practically the same in catalytic activity and are superior to sulfuric acid. The reaction orders with respect to catalyst was found to be close to 1 in all cases. It has been shown that the reaction is actually insensitive to the lengths of OEG and OPG blocks, as well as to the structure of the terminal alkyl group, while the esterification of acrylic acid (AA) proceeds much faster compared to methacrylic acid (MAA) one under the same conditions. The influence of temperature on the equilibrium conversions of alcohols was determined, which were found to be 89%–93% for the esterification of AA and 61%–86% for MAA in the temperature range of 60–120°C. A further increase in conversion was achieved by introducing an azeotropic agent (toluene), its optimal concentration was found to be 10%–15%.     

Oxygen-Generating Cryogels Restore T Cell Mediated Cytotoxicity in Hypoxic Tumors

Solid tumors are protected from antitumor immune responses due to their hypoxic microenvironments. Weakening hypoxia-driven immunosuppression by hyperoxic breathing of 60% oxygen has shown to be effective in unleashing antitumor immune cells against solid tumors. However, efficacy of systemic oxygenation is limited against solid tumors outside of lungs and has been associated with unwanted side effects. As a result, it is essential to develop targeted oxygenation alternatives to weaken tumor hypoxia as novel approaches to restore immune responses against cancer. Herein, injectable oxygen-generating cryogels (O₂-cryogels) to reverse tumor-induced hypoxia are reported. These macroporous biomaterials are designed to locally deliver oxygen, inhibit the expression of hypoxia-inducible genes in hypoxic melanoma cells, and reduce the accumulation of immunosuppressive extracellular adenosine. The data show that O₂-cryogels enhance T cell-mediated secretion of cytotoxic proteins, restoring the killing ability of tumor-specific cytotoxic T lymphocytes, both in vitro and in vivo. In summary, O₂-cryogels provide a unique and safe platform to supply oxygen as a coadjuvant in hypoxic tumors and have the potential to improve cancer immunotherapies.     

The GRAINS thermodynamic and kinetic code for modeling nebular condensation

The GRAINS thermodynamic and kinetic code for modeling condensation and grain growth in the solar nebula is described. The code calculates CWPI-type (Condensation With Partial Isolation) equilibrium partitioning of 33 chemical elements among 242 gaseous and 520 condensed phases, and growth in the nebula of metal and olivine grains by condensation from the nebular gas accompanied by diffusional redistribution of major and trace elements in those grains. Several examples of the GRAINS application to cosmochemical problems are briefly discussed.