Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning (RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming (ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively. Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks, showing how they promote ADP formulation significantly. Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has demonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.     

基于业务基础平台快速搭建业务应用系统

为了帮助企业实现快速搭建业务应用系统,业务基础平台的出现是一个必然的选择。首先,对企业信息化建设所存在的问题进行了探讨;然后,对ADP平台进行了概述,并就ADP平台为快速搭建业务应用系统提供的能力进行重点阐述;最后,讨论了如何在平台上快速搭建业务系统。     

Combined 1H and 31P MR spectroscopic imaging: impaired energy metabolism in severe carotid stenosis and changes upon treatment

OBJECT: To evaluate if combined (1)H and (31)P MR spectroscopic imaging (MRSI) before and after treatment of severe internal carotid artery (ICA) stenosis detects significant changes in energy metabolism in the basal ganglia of both hemispheres. MATERIALS AND METHODS: A group of 14 patients with high-grade ICA stenosis and 11 healthy control subjects were examined with 2D (1)H MRSI and 3D (31)P MRSI at 3 T before and after treatment of severe ICA stenosis. Spectroscopic data were processed with LCModel and jMRUI software. Changes of the phosphorylated metabolites, pH, N-acetyl-acetate, creatine and choline-containing compounds prior/post intervention were analyzed and patients' data were compared with that of control subjects. RESULTS: Untreated patients had significantly higher Adenosindiphosphate (ADP) in basal ganglia ipsi- and contralateral to the side of ACI stenosis compared to controls. After treatment, ADP of both hemispheres significantly decreased by approximately 20% compared to the pre-treatment values. Further, significant decreases of phosphorylated metabolites prior/post intervention were found for patients compared to controls. CONCLUSION: This spectroscopic study reveals that unilateral high-grade ICA stenosis has an effect on cerebral high-energy metabolism of both hemispheres, which is at least partially reversible after treatment. Therefore the restoration of blood flow in high-grade ICA stenosis recovers the impaired energy balance of the brain.     

基于ADP3806的高功率LED驱动电路设计

设计了一种基于ADP3806的高功率发光二极管(LED)的高效驱动电路。ADP3806是一款开关模式电源控制器,拥有双环路恒定电压和恒定电流控制、远程精确电流检测以及关断和可编程可同步开关频率,能提供恒定电流。同时在设计中利用单端原边电感转换器(SEPIC),其可以提供一种可以高于或低于输入电压的输出电压,在适当的占空比下工作,使连续传导模式(CCM)和脉冲宽度调制(PWM)控制变得简单,提高了效率,并且避免由变压器泄漏电感带来的电压尖峰和振铃。从而在需要进行升压和降压转换来同时驱动多个高功率LED的场合,这个设计是非常适合的。     

The Local Environment of Loop Switch 1 Modulates the Rate of ATP-Induced Dissociation of Human Cardiac Actomyosin

Two isoforms of human cardiac myosin, alpha and beta, share significant sequence similarities but show different kinetics. The alpha isoform is a faster motor; it spends less time being strongly bound to actin during the actomyosin cycle. With alpha isoform, actomyosin dissociates faster upon ATP binding, and the affinity of ADP to actomyosin is weaker. One can suggest that the isoform-specific actomyosin kinetics is regulated at the nucleotide binding site of human cardiac myosin. Myosin is a P-loop ATPase; the nucleotide-binding site consists of P-loop and loops switch 1 and 2. All three loops position MgATP for successful hydrolysis. Loops sequence is conserved in both myosin isoforms, and we hypothesize that the isoform-specific structural element near the active site regulates the rate of nucleotide binding and release. Previously we ran molecular dynamics simulations and found that loop S291-E317 near loop switch 1 is more compact and exhibits larger fluctuations of the position of amino acid residues in beta isoform than in alpha. In alpha isoform, the loop forms a salt bridge with loop switch 1, the bridge is not present in beta isoform. Two isoleucines I303 and I313 of loop S291-E317 are replaced with valines in alpha isoform. We introduced a double mutation I303V:I313V in beta isoform background and studied how the mutation affects the rate of ATP binding and ADP dissociation from actomyosin. We found that ATP-induced actomyosin dissociation occurs faster in the mutant, but the rate of ADP release remains the same as in the wild-type beta isoform. Due to the proximity of loop S291-E317 and loop switch 1, a faster rate of ATP-induced actomyosin dissociation indicates that loop S291-E317 affects structural dynamics of loop switch 1, and that loop switch 1 controls ATP binding to the active site. A similar rate of ADP dissociation from actomyosin in the mutant and wild-type myosin constructs indicates that loop switch 1 does not control ADP release from actomyosin.     

丹酚酸对ADP诱导的血小板聚集的影响

目的 研究丹酚酸(salvianolic acid,SA)及其代谢物对腺苷-5'二磷酸(adenosine-5-diphosphate,ADP)诱导的血小板聚集的影响,为寻找到高效低毒的抗血栓药物提供依据.方法 取健康雄性SD大鼠动脉血浆,以药物替格瑞洛为阳性对照,ADP为诱导剂,考察SA对血小板聚集的体外作用,用比浊...     

The role of PaAAC1 encoding a mitochondrial ADP/ATP carrier in the biosynthesis of extracellular glycolipids,mannosylerythritol lipids,in the basidiomycetous yeast Pseudozyma antarctica

Pseudozyma antarctica produces large amounts of the glycolipid biosurfactants known as mannosylerythritol lipids (MEL), which show not only excellent surface‐active properties but also versatile biochemical actions. A gene homologous with a mitochondrial ADP/ATP carrier was dominantly expressed in P. antarctica under MEL‐producing conditions on the basis of previous gene expression analysis. The gene encoding the mitochondrial ADP/ATP carrier of P. antarctica (PaAAC1) contained a putative open reading frame of 954 bp and encodes a polypeptide of 317 amino acids. The deduced translation product shared high identity of 66%, 70%, 69%, 74%, 75% and 52% with the mitochondrial ADP/ATP carrier of Saccharomyces cerevisiae (AAC1), S. cerevisiae (AAC2), S. cerevisiae (AAC3), Kluyveromyces lactis (KlAAC), Neurospora crassa (NcAAC) and human (ANT1), respectively, and conserved the consensus sequences of all ADP/ATP carrier proteins. The gene expression by introducing a plasmid pUXV1‐PaAAC1 into the yeast cells increased the MEL production. In addition, the expression of PaAAC1 in which the conserved arginine and leucine required for ATP transport activity were replaced with isoleucine and serine, respectively, failed to increase MEL production. Accordingly, these results suggest that PaAAC1 encoding a mitochondrial ADP/ATP carrier should be involved in MEL biosynthesis in the yeast. Copyright © 2010 John Wiley & Sons, Ltd.     

On the way to selective PARP-2 inhibitors. Design, synthesis, and preliminary evaluation of a series of isoquinolinone derivatives

PARP-1 and PARP-2 are members of the family of poly(ADP-ribose)polymerases, which are involved in the maintenance of genomic integrity under conditions of genotoxic stimuli. The different roles of the two isoforms under pathophysiological conditions have not yet been fully clarified, and this is partially due to the lack of selective inhibitors. We report herein the synthesis and preliminary pharmacological evaluation of a large series of isoquinolinone derivatives as PARP-1/PARP-2 inhibitors. Among them, we identified the 5-benzoyloxyisoquinolin-1(2 H)-one derivative as the most selective PARP-2 inhibitor reported so far, with a PARP-2/PARP-1 selectivity index greater than 60.     

Inhibitory Mechanisms of Lusianthridin on Human Platelet Aggregation

Lusianthridin is a phenanthrene derivative isolated from Dendrobium venustum. Some phenanthrene compounds have antiplatelet aggregation activities via undefined pathways. This study aims to determine the inhibitory effects and potential mechanisms of lusianthridin on platelet aggregation. The results indicated that lusianthridin inhibited arachidonic acid, collagen, and adenosine diphosphate (ADP)-stimulated platelet aggregation (IC₅₀ of 0.02 ± 0.001 mM, 0.14 ± 0.018 mM, and 0.22 ± 0.046 mM, respectively). Lusianthridin also increased the delaying time of arachidonic acid-stimulated and the lag time of collagen-stimulated and showed a more selective effect on the secondary wave of ADP-stimulated aggregations. Molecular docking studies revealed that lusianthridin bound to the entrance site of the cyclooxygenase-1 (COX-1) enzyme and probably the active region of the cyclooxygenase-2 (COX-2) enzyme. In addition, lusianthridin showed inhibitory effects on both COX-1 and COX-2 enzymatic activities (IC₅₀ value of 10.81 ± 1.12 µM and 0.17 ± 1.62 µM, respectively). Furthermore, lusianthridin significantly inhibited ADP-induced suppression of cAMP formation in platelets at 0.4 mM concentration (p < 0.05). These findings suggested that possible mechanisms of lusianthridin on the antiplatelet effects might act via arachidonic acid-thromboxane and adenylate cyclase pathways.     

500K、1 000K型ADP与ADCP比测分析

随着电子技术高速发展,声学多普勒测流技术也在不断升级,不同的仪器设备厂商生产不同型号、频率的声学多普勒流速剖面仪,有ADCP、ADP等一些声学多普勒流速剖面仪。本文对SonTek/YSI公司的500K、1 000K型走航式ADP与Rio Grande600K型走航式ADCP作比测分析,提供设备推广应用作参考。