Soft Robotic-Adapted Multimodal Sensors Derived from Entirely Intrinsic Self-Healing and Stretchable Cross-Linked Networks

Flexible sensing technologies that play a pivotal role in endowing robots with detection capabilities and monitoring their motions are impulsively desired for intelligent robotics systems. However, integrating and constructing reliable and sustainable flexible sensors with multifunctionality for robots remains an everlasting challenge. Herein, an entirely intrinsic self-healing, stretchable, and attachable multimodal sensor is developed that can be conformally integrated with soft robots to identify diverse signals. The dynamic bonds cross-linked networks including the insulating polymer and conductive hydrogel with good comprehensive performances are designed to fabricate the sensor with prolonged lifespan and improved reliability. Benefiting from the self-adhesiveness of the hydrogel, strong interfacial bonding can be formed on various surfaces, which promotes the conformable integration of the sensor with robots. Due to the ionic transportation mechanism, the sensor can detect strain and temperature based on piezoresistive and thermoresistive effect, respectively. Moreover, the sensor can work in triboelectric mode to achieve self-powered sensing. Various information can be identified from the electrical signals generated by the sensor, including hand gestures, soft robot crawling motions, a message of code, the temperature of objects, and the type of materials, holding great promise in the fields of environmental detection, wearable devices, human-machine interfacing, and robotics.     

Petromyzontidae-Biomimetic Multimodal Microneedles-Integrated Bioelectronic Catheters for Theranostic Endoscopic Surgery

Current catheter devices in minimally invasive surgery still possess limited functional options, lacking multimodal integration of both sensing and therapy. Catheter devices usually operate outside the tissue, incapable to detect intra-tissue biochemical information for accurate localization and assessment of lesions during surgery. Inspired by the feature and functions of Petromyzontidae, here a multimodal core-shell microneedles-integrated bioelectronic catheter (MNIBC) for tissue-penetrating theranostics in endoscopic surgery is developed. The microneedle (MN) device possesses individually addressable functionality at single-MN tip resolution, enabling multiplex functions (a total of 11 functions distributed in three types of catheters) including biochemical sensing, myoelectric modulation, electroporation, and drug delivery in a submucosal environment. The MNIBC is prepared through hybrid fabrication and dimensionality reduction strategies, where the MN electrodes are functionalized with an MXene-carbon nanotube (MXene-CNT)-based electron mediator, addressing the challenge of reduced electrode sensitivity on ultra-small MN tip. The functionalities of MNIBC are demonstrated both ex vivo and in vivo on anesthetized rabbits via laparoscopy, simulated cystoscopy, and laparotomy. The MNIBC can effectively detect intra-tissue biochemical signals in the bladder, and offers localized electroporation and intra-tissue drug delivery for precise treatments of lesions. The versatile features of the MNIBC present a highly advanced platform for precise surgeries.     

基于多模态图像融合的早期蕈样肉芽肿识别

早期蕈样肉芽肿（Mycosis fungoid, MF）可表现为红斑鳞屑性皮损,很难从银屑病及慢性湿疹等良性炎症性皮肤病中鉴别出来。本文提出了一种基于多模态图像融合的早期蕈样肉芽肿识别方法。该方法基于皮肤镜图像和临床图像,采用ResNet18网络提取单模态图像的特征;设计跨模态的注意力模块,实现两种模态图像的特征融合;并且设计自注意力模块提取融合特征中的关键信息,改善信息冗余,从而提高蕈样肉芽肿智能识别的准确度。实验结果表明,本文所提出的智能诊断模型优于对比算法。将本文模型应用于皮肤科医生的实际临床诊断,通过实验组医生和对照组医生平均诊断准确率的变化证实了本文模型能够有效提升临床诊断水平。     

融合语音、脑电和人脸表情的多模态情绪识别

本文提出了一种多模态情绪识别方法, 该方法融合语音、脑电及人脸的情绪识别结果来从多个角度综合判断人的情绪, 有效地解决了过去研究中准确率低、模型鲁棒性差的问题. 对于语音信号, 本文设计了一个轻量级全卷积神经网络, 该网络能够很好地学习语音情绪特征且在轻量级方面拥有绝对的优势. 对于脑电信号, 本文提出了一个树状LSTM模型, 可以全面学习每个阶段的情绪特征. 对于人脸信号, 本文使用GhostNet进行特征学习, 并改进了GhostNet的结构使其性能大幅提升. 此外, 我们设计了一个最优权重分布算法来搜寻各模态识别结果的可信度来进行决策级融合, 从而得到更全面、更准确的结果. 上述方法在EMO-DB与CK+数据集上分别达到了94.36%与98.27%的准确率, 且提出的融合方法在MAHNOB-HCI数据库的唤醒效价两个维度上分别得到了90.25%与89.33%的准确率. 我们的实验结果表明, 与使用单一模态以及传统的融合方式进行情绪识别相比, 本文提出的多模态情绪识别方法有效地提高了识别准确率.     

广告视觉隐喻的符号意义生成机制

目的解析广告视觉隐喻的符号运作机制与组合方式。方法以符号意义的生成理论为原点,以索绪尔符号的价值理论为方法,分析广告聚合段上的符号关系与视觉隐喻修辞的内在联系,提出意义二元值的同向替代机制,以及广告视觉隐喻符号的组合方式。结论广告视觉隐喻是通过聚合段上的符号互相替代而形成的,其符号组合有本体融入喻体、喻体融入本体、本喻体互融3种模式。     

多模态隐喻视角下的产品设计理论方法研究

目的研究多模态隐喻在产品设计中的设计方法。方法归纳多模态隐喻在产品设计中的表现形式,深入分析其在产品设计中的创建机制与方法。结论提出以产品图像、动作、声音、空间为元素的多模式的产品设计创建方法。     

汉语名词的隐喻知识表示及获取研究

隐喻知识的表示和获取是进行隐喻计算的基础。该文把隐喻知识看作是本体和喻体的特征和属性之间的关联。主要通过惯用语导入和句法模式识别两种机制,来获取名词的隐喻知识。惯用语的隐喻比较固定和单一,从专门的词典释义中就能获取准确的隐喻特征和属性。而一般名词的隐喻知识比较复杂,该文主要依托语料库和搜索引擎,通过关键词和句法匹配来获取同一名词不同的隐喻特征及对应属性。该项工作的结果,对隐喻句隐喻意义的获取和名词语义属性特征的描述体系构建具有一定的价值。