Speech and Language Processing for Multimodal Human-Computer Interaction

In this paper, we describe our recent work at Microsoft Research, in the project codenamed Dr. Who, aimed at the development of enabling technologies for speech-centric multimodal human-computer interaction. In particular, we present in detail MiPad as the first Dr. Who's application that addresses specifically the mobile user interaction scenario. MiPad is a wireless mobile PDA prototype that enables users to accomplish many common tasks using a multimodal spoken language interface and wireless-data technologies. It fully integrates continuous speech recognition and spoken language understanding, and provides a novel solution to the current prevailing problem of pecking with tiny styluses or typing on minuscule keyboards in today's PDAs or smart phones. Despite its current incomplete implementation, we have observed that speech and pen have the potential to significantly improve user experience in our user study reported in this paper. We describe in this system-oriented paper the main components of MiPad, with a focus on the robust speech processing and spoken language understanding aspects. The detailed MiPad components discussed include: distributed speech recognition considerations for the speech processing algorithm design; a stereo-based speech feature enhancement algorithm used for noise-robust front-end speech processing; Aurora2 evaluation results for this front-end processing; speech feature compression (source coding) and error protection (channel coding) for distributed speech recognition in MiPad; HMM-based acoustic modeling for continuous speech recognition decoding; a unified language model integrating context-free grammar and N-gram model for the speech decoding; schema-based knowledge representation for the MiPad's personal information management task; a unified statistical framework that integrates speech recognition, spoken language understanding and dialogue management; the robust natural language parser used in MiPad to process the speech recognizer's output; a machine-aided grammar learning and development used for spoken language understanding for the MiPad task; Tap & Talk multimodal interaction and user interface design; back channel communication and MiPad's error repair strategy; and finally, user study results that demonstrate the superior throughput achieved by the Tap & Talk multimodal interaction over the existing pen-only PDA interface. These user study results highlight the crucial role played by speech in enhancing the overall user experience in MiPad-like human-computer interaction devices.     

一款雷达信号处理SOC芯片的存储器内建自测试设计

内建自测试(BIST)为嵌入式存储器提供了一种有效的测试方法.详细介绍了存储器故障类型及内建自测试常用的March算法和ROM算法.在一款雷达信号处理SOC芯片中BIST被采用作为芯片内嵌RAM和ROM的可测试性设计的解决方案.利用BIST原理成功地为芯片内部5块RAM和2块ROM设计了自测试电路,并在芯片的实际测试过程中成功完成对存储器的测试并证明内嵌存储器不存在故障.     

人机自然交互过程中情态的识别与融合

人机自然的交互过程中,在传递交互内容的显性信息同时也伴随有关于情态的隐性信息,即通过语言表达的情感和通过人脸显现的表情.本文阐述了情态识别的研究意义和实用价值;给出了考虑显性信道和隐性信道的识别构架;讨论和评价了一些识别人脸表情的途径与方法;研究了情态识别融合问题,并给出了一些在实验室中得到的结果.     

基于消息的监控软件人机交互平台的开发

文章简要说明了传统人机交互系统所存在的问题并提出了一种新的人机交互系统的设计平台。应用组件设计的思想以及模拟消息映射的技术，采用客户／中间层朋＆务器的结构模式，开发了基于该平台的新一代监控软件的服务器程序及客户程序。由于服务器程序和客户程序遵循平台的协议规范，不同的客户程序与服务器程序之间可以使用通用的方式进行数据交换，从而大大加强人机交互程序的扩展性和灵活性。     

WLAN SOC芯片BX501的FPGA验证平台设计与实现

系统芯片（SOC）设计是以模块复用和软硬件协同设计为基础，基于FPGA的验证平台是一种有效的验证途径。文章讨论了WLANSOC芯片BX501的验证平台的两种实现方案，介绍了采用Xilinx Virtex-Ⅱ系列FPGA的设计实现；同时对SOC设计的FPGA验证问题进行了分析和探讨。     

CA 技术的发展及机顶盒高级安全芯片原理

结合对智能卡CAS的安全漏洞及破解方法的分析,介绍了CA技术从普通CAS、高级安全CAS到DCAS 的发展变迁,重点讲述了高级安全CAS和DCAS的安全机制及机顶盒高级安全芯片原理。     

具备兼容性和层次性的SOC测试控制结构设计

IP核的集成问题是SOC设计的关键,测试集成更是无法回避的难题.因此,灵活高效的测试控制结构成为SOC可测性设计的重要研究内容.文章分析了IEEE Std 1149.1对传统IC芯片内部和外部测试的整体控制能力;剖析了IEEE Std 1500TM对嵌入式IP核测试所做规定的标准性和可配置性.在此基础上,提出了一种复用芯片级测试控制器的测试控制结构,该结构能兼容不同类型的IP核,并且有助于实现复杂SOC的层次性测试控制.     

A Fast MAP Algorithm for High-Resolution Image Reconstruction with Multisensors

In many applications, it is required to reconstruct a high-resolution image from multiple, undersampled and shifted noisy images. Using the regularization techniques such as the classical Tikhonov regularization and maximum a posteriori (MAP) procedure, a high-resolution image reconstruction algorithm is developed. Because of the blurring process, the boundary values of the low-resolution image are not completely determined by the original image inside the scene. This paper addresses how to use (i) the Neumann boundary condition on the image, i.e., we assume that the scene immediately outside is a reflection of the original scene at the boundary, and (ii) the preconditioned conjugate gradient method with cosine transform preconditioners to solve linear systems arising from the high-resolution image reconstruction with multisensors. The usefulness of the algorithm is demonstrated through simulated examples.     

基于AdaBoost算法和光流匹配的实时手势识别

着眼于更宽泛和更便捷的应用需要,提出了基于AdaBoost算法和光流匹配的实时手势识别方案.只需连接到计算机的摄像头读取二维手势视频片段就能对手势作为较为准确的识别.其中,采用AdaBoost算法遍历图像,完成静态手势的识别工作;在动态手势的识别过程中,运用了光流法结合模板匹配的方法.整个系统对静态和动态手势的识别均具有较强的鲁棒性.