具身与离身认知在多模态交互环境下的交互语法研究

目的 针对目前具身交互设计领域中具身认知和离身认知割裂的现状，将心理学第一代认知科学的离身认知与第二代认知科学的具身认知相结合，构建在多模态交互环境下的交互语法，指导智能体与人的具身交互和离身交互。方法 以智能体为多模态交互环境的媒介，通过把人类的具身性逐步转化为智能体的离身性，运用隐喻、转喻、隐转喻的方式，使智能体的离身性激发人类交互主体，产生新的具身认知。结论 在多模态交互环境中，把具身交互中的示能性、交互行为、交互前馈与离身交互中的符号、语义、交互反馈进行组建，形成完整交互语法体系。将交互事件中的3个要素进行映射，匹配到人与智能体的多模态交互关系中。

Interaction Grammar of Embodied & Disembodied Cognition in Multi-modal Interactive Environment

The work aims to combine the disembodied cognition in the first generation context of cognitive science with the embodied cognition in the second generation context of cognitive science for the separation of embodied cognition and disembodied cognition in the field of embodied interaction design and construct the interaction design grammar in multi-modal interactive environment to guide the embodied and disembodied interaction between agent and human. By taking the agent as the media of the multimodal interactive environment and transforming the embodiment of human beings into the disembodiment of the agent, metaphor, metonymy and metaphtonymy were used to make the embodiment of the agent stimulate the feedback interactive subject of human beings and generate new feedforward embodied cognition. In the context of multimodal interaction, affordance, interaction behavior and interaction feedforward mapping are constructed with symbol, semantics and interaction feedback in embodied interaction to form a complete interactive grammar system. Three elements in the interactive event are mapped and matched to the multimodal interactive relationship between human and agent.