Magnetic Field-Based Reward Shaping for Goal-Conditioned Reinforcement Learning

Goal-conditioned reinforcement learning (RL) is an interesting extension of the traditional RL framework, where the dynamic environment and reward sparsity can cause conventional learning algorithms to fail. Reward shaping is a practical approach to improving sample efficiency by embedding human domain knowledge into the learning process. Existing reward shaping methods for goal-conditioned RL are typically built on distance metrics with a linear and isotropic distribution, which may fail to provide sufficient information about the ever-changing environment with high complexity. This paper proposes a novel magnetic field-based reward shaping (MFRS) method for goal-conditioned RL tasks with dynamic target and obstacles. Inspired by the physical properties of magnets, we consider the target and obstacles as permanent magnets and establish the reward function according to the intensity values of the magnetic field generated by these magnets. The nonlinear and anisotropic distribution of the magnetic field intensity can provide more accessible and conducive information about the optimization landscape, thus introducing a more sophisticated magnetic reward compared to the distance-based setting. Further, we transform our magnetic reward to the form of potential-based reward shaping by learning a secondary potential function concurrently to ensure the optimal policy invariance of our method. Experiments results in both simulated and real-world robotic manipulation tasks demonstrate that MFRS outperforms relevant existing methods and effectively improves the sample efficiency of RL algorithms in goal-conditioned tasks with various dynamics of the target and obstacles.

     

通信基站的雷电防护

可靠的雷电防护是保证通信基站安全运行的重要保障。本文全面分析了雷电侵入基站进而损坏站内设备的途径,进而研究了通信基站的雷击防护技术。     

工程师短缺抑或缺少其他——一个行动者网络视角的透视

我国工程教育发展史映射了工业化进程。在我国工程教育中,存在着培养目标不清、政府政策导向不明、工科院校培养模式失衡和企业角色缺位等问题。在借用行动者网络理论分析框架对这些问题进行检讨的基础上,指出了重构我国工程教育行动者网络的必要性。我国工程教育的培养目标应定位于培养工业化发展过程中需要的工程师,政府在这一网络重构过程中发挥主导作用,引导工科院校建立多层次、多规格的人才培养体系。政府应加大投入,改善_T-科院校办学条件。利用减免税等政策工具,吸引企业积极参与网络构建;通过实施认证制度和激励手段,规范工科院校和企业的行为。     

Enhanced Magnetoresistance Effect and Magnetic Properties of Double Perovskite La2−xSrxMnRuO6 (0 ≤ x ≤ 1.0)

Journal of Superconductivity and Novel Magnetism - Double perovskite La2?xSrxMnRuO6 (x?=?0, 0.3, 0.5, 0.8, 1.0) samples were synthesized by the standard solid-state reaction...     

一种三参数统一化动量方法及其最优收敛速率

动量方法由于能够改善SGD(stochastic gradient descent)的收敛性能而倍受机器学习研究者的关注.随着其在深度学习的成功应用,动量方法出现了众多形式的变体.特别地,产生了SUM(stochastic unified momentum)和QHM(quasi-hyperbolic momentum)两种统一框架.但是,即使是对非光滑凸优化问题,其最优平均收敛性的获得仍然存在着固定迭代步数和无约束等不合理限制.为此,提出了一种更一般的含三参数的统一化动量方法TPUM(triple-parameters unified momentum),能够同时包含SUM和QHM;其次,针对约束的非光滑凸优化问题,在采取时变步长的条件下,证明了所提出的TPUM具有最优的平均收敛速率,并将其推广到随机情况,从而保证了添加动量不会影响标准梯度下降法的收敛性能以及动量方法对机器学习问题的可应用性.典型的L1范数约束hinge损失函数优化问题实验验证了理论分析的正确性.     

结构抗火试验室通风状况的数值模拟及分析

利用Fluent软件对结构抗火试验室内的气流组织进行了模拟,分析了影响试验室内污染物分布的因素,结合实测对模拟结果进行了验证.提出了不同的通风改造方案,并在数值模拟的基础上优选出最终方案,以改善试验室人员的工作环境.     

Spatiotemporalized Hydrogel Microspheres Promote Vascularized Osteogenesis via Ultrasound Oxygen Delivery

Disturbance of spatiotemporal oxygen balance is the main cause of delayed healing or nonhealing of large bone defects. The accurate administration of oxygen to regulate disruptions in the spatiotemporal oxygen equilibrium during 9 h of hypoxia is imperative for bone tissue regeneration. Herein, oxygen-loaded nanobubbles prepared by double emulsification are successfully embedded in GelMA/HepMA microsphere macromolecular meshwork by microfluidic techniques, and a spatiotemporalized hydrogel microsphere is constructed by noncovalently binding bone morphogenetic protein 2 (BMP-2). The spatiotemporalized hydrogel microspheres precisely “remote control” oxygen release by ultrasound in vitro 9 h after bone injury to regulate spatiotemporal oxygen homeostasis disorder, maintain a high level of vascular endothelial growth factor (VEGF) expression, and accelerate bone repair. The spatiotemporalized hydrogel microspheres possess good oxygen-carrying capacity and ultrasonic responsiveness, and the oxygen concentration increases to 1.63, 1.95, 2.11, and 2.29 times under the ultrasound action at different intensities of 1, 2, 3, and 4 W, respectively, providing the conditions for the precise regulation of spatiotemporal oxygen balance disorder by ultrasound. In the in vitro hypoxia model and in vivo rat femoral defect model, the spatiotemporal hydrogel microspheres show good vascularization and osteogenesis capabilities, which provide a new strategy for the clinical treatment of large bone defects.     

桥梁结构参数化设计研究

参数化设计是基于桥梁结构设计内在联系与约束关系进行三维正向设计的一种方法,该方法可有效提高结构的调整与优化效率。在方案初步设计阶段,参数化设计理念可用于桥梁结构合理性、经济性及美观性的优化,满足现阶段桥梁造型多样化发展的社会需求;在施工图设计阶段通过调整自变量参数,可灵活地进行桥梁结构细部构造调整,并与二维设计图进行自动交互,大幅提升设计效率及质量。本文基于“R+GH”参数化平台,提出桥梁结构参数化设计流程,并通过工程案例的应用进一步验证该设计思想的实用性和高效性。     

Injectable silk/hydroxyapatite nanocomposite hydrogels with vascularization capacity for bone regeneration

Localized and sustained osteogenic-angiogenic stimulation to bone defects is critical for effective bone repair.Here,desferrioxamine(DFO)was loaded on silk fibroin nanofibers and blended with hydroxyapatite nanorods(HA),forming injectable DFO-loaded silk fibroin-HA nanocomposite hydrogels.The composite hydrogels remained homogeneous distribution of HA with high ratio(60%)and also higher stiffness than that of pure silk fibroin nanofiber hydrogels,which provided stable osteogenic stimulation niches for tissue regeneration.Without the scarify of injectability,the hydrogels achieved slow delivery of DFO for above 60 days,resulting in suitable angiogenesis in vitro and in vivo and better osteogenesis than DFO-free systems.Compared to previous injectable silk fibroin-HA hydrogels,the introduction of vascularization capacity further stimulated the osteogenic differentiation of stem cells and accelerated new bone formation.Quicker and better bone healing were detected at defect sites after the injection of DFO-loaded nanocomposite hydrogels,indicating the effective synergistic effect of osteogenic and angiogenic cues.This work provides a simple and effective strategy of introducing angiogenic cues to bone matrices.We believe that the injectable nanocomposite hydrogels are suitable for the regeneration of bone tissues.     

TMP工艺与催化剂的配套性分析

在重油催化裂解多产丙烯（简称TMP）技术中,将两段催化裂解工艺与配套的增产丙烯催化剩结合,以大庆常压渣油为原料可以得到20％的丙烯收率,并能够很好兼顾汽柴油的质量。试验结果表明：以大庆常压渣油为原料,使用配套开发的LCC-300催化剂,干气加焦炭收率。小于15％、丙烯的收率大于20％、汽油的辛烷值RON96、烯烃30％（v）左右。