引导性权重驱动的图表问答重定位关系网络

目的 图表问答是计算机视觉多模态学习的一项重要研究任务,传统关系网络(relation network,RN)模型简单的两两配对方法可以包含所有像素之间的关系,因此取得了不错的结果,但此方法不仅包含冗余信息,而且平方式增长的关系配对的特征数量会给后续的推理网络在计算量和参数量上带来很大的负担。针对这个问题,提出了一种基于融合语义特征提取的引导性权重驱动的重定位关系网络模型来改善不足。方法 首先通过融合场景任务的低级和高级图像特征来提取更丰富的统计图语义信息,同时提出了一种基于注意力机制的文本编码器,实现融合语义的特征提取,然后对引导性权重进行排序进一步重构图像的位置,从而构建了重定位的关系网络模型。结果 在2个数据集上进行实验比较,在FigureQA(an annotated figure dataset for visual reasoning)数据集中,相较于IMG+QUES(image+questions)、RN和ARN(appearance and relation networks),本文方法的整体准确率分别提升了26.4%,8.1%,0.46%,在单一验证集上,相较于LEA...     

LMR-CBT: learning modality-fused representations with CB-Transformer for multimodal emotion recognition from unaligned multimodal sequences

Learning modality-fused representations and processing unaligned multimodal sequences are meaningful and challenging in multimodal emotion recognition. Existing approaches use directional pairwise attention or a message hub to fuse language, visual, and audio modalities. However, these fusion methods are often quadratic in complexity with respect to the modal sequence length, bring redundant information and are not efficient. In this paper, we propose an efficient neural network to learn modality-fused representations with CB-Transformer (LMR-CBT) for multimodal emotion recognition from unaligned multi-modal sequences. Specifically, we first perform feature extraction for the three modalities respectively to obtain the local structure of the sequences. Then, we design an innovative asymmetric transformer with cross-modal blocks (CB-Transformer) that enables complementary learning of different modalities, mainly divided into local temporal learning, cross-modal feature fusion and global self-attention representations. In addition, we splice the fused features with the original features to classify the emotions of the sequences. Finally, we conduct word-aligned and unaligned experiments on three challenging datasets, IEMOCAP, CMU-MOSI, and CMU-MOSEI. The experimental results show the superiority and efficiency of our proposed method in both settings. Compared with the mainstream methods, our approach reaches the state-of-the-art with a minimum number of parameters.     

Recent Advances and Clinical Potential of Near Infrared Photothermal Conversion Materials for Photothermal Hepatocellular Carcinoma Therapy

Growing concern about photothermal tumor therapy (PTT) as a promising alternative to conventional liver cancer treatment, which is a treatment strategy that utilizes near-infrared (NIR) light-induced photothermal agents (PTAs) to yield photothermal effects to localize thermal damage for tumors. Herein, given the gap between experimental research and clinical application, this review seeks to timely summarize and highlight the recent progress of PTAs used for photothermal treatment of liver cancer in vivo and in vitro in the last five-year. The implications of various PTAs on the multifunctional photothermal conversion capability, the structure-performance correlations of PTT, together with the evaluation of their potential in application are systematically dissected to further dig out what the buried mechanism is. Besides, higher requirements are put forward for the discrepancies and crucial issues faced by different PTAs in PTT with related medical technical obstacles being conquered, which lays a solid theoretical foundation for the medical field of oncology treatment as a whole, especially liver cancer. Finally, it is expected that this review can present valuable guidance for the design of efficient, photostability, and biosafety-aware PTAs for anticancer therapy while stepping into the fast traffic lane for the conversion from bench to bedside in the foreseeable future.     

基于多模态脑机接口的智能小车自动驾驶系统

在传统的控制系统当中, 人们依赖于使用手柄、操纵杆等设备来与外部设备实现人机交互, 这对于具有运动障碍的患者来说是具有挑战的. 而脑机接口(BCI)技术可通过脑环将脑电信号转化为对外界设备的控制命令, 使这些患者可以由大脑“意识”直接控制外部设备. 本文提出一种基于多模态脑机接口的智能小车自动驾驶系统, 该系统融合了受试者的脑电信号、眼电信号和陀螺仪信号3种模态的信号来控制小车. 其中, 脑电信号用于控制小车的速度, 眼电信号用于控制小车的启停, 陀螺仪信号则用于控制小车的转向功能. 此外, 我们还融合了计算机视觉技术, 为智能小车增加了自动驾驶功能, 使得控制更加智能化. 经过实验表明, 10名受试者使用该系统控制小车的平均准确率达到了92.47%, 平均响应时间为1.55 s, 平均信息传递速率达到了55.94 bit/min, 从而说明该控制系统是有效且高效的. 此外, 为了验证小车的自动驾驶功能, 我们设置了多个对比实验进行验证. 实验结果表明, 与手动驾驶相比, 虽然该自动驾驶系统在操控小车的速度上存在劣势, 但是在准确率与稳定性上具有更好的性能优势. 证明该系统可以为残障人士带来更好的操控体验, 在脑控应用和自动驾驶领域具有广阔的应用前景.