User interfaces for communication bridges across the digital divide

Connecting people across the “digital divide” is as much a social effort as a technological one. We are developing a community-centred approach to learn how interaction techniques can compensate for poor communication across the digital divide. We have incorporated the lessons learned regarding social intelligence design in an abstraction and in a device called the SoftBridge. The SoftBridge allows communication to flow from endpoints through adapters, getting converted if necessary, and out to destination endpoints. Field trials are underway with two communities in South Africa: disadvantaged Deaf users and an isolated rural community. Initial lessons learned show that we have to design user interfaces that allow users to understand and cope with delay. We also learned that social concerns are often more important than the technical issues in designing such systems.     

An adaptive enhancement method for generic multi-modal high‐bit RAW grayscale image displaying on low-bit monitors

Aiming at the poor adaptability and robustness of existing pseudo-color methods, most of which can only deal with an 8-bit grayscale, an adaptive enhancement algorithm of high grayscale images based on priori knowledge was proposed. Firstly, aiming at the problem that the original RGB color space is not easy to adjust dynamically, the power adjustment was integrated into the constructed high-bit chromatogram, and a power adaptive adjustment function based on the brightness priority over the original grayscale image was designed. Secondly, aiming at the problems of over-exposure, under-exposure, and poor gradients in RGB space, an adaptive grayscale correction algorithm was designed according to the priori knowledge distribution of RGB perceived brightness. Finally, to guarantee the color balance of the enhanced image, a color balance correction algorithm based on CMY space was designed. To verify the effectiveness of this method, it was applied to the pseudo-color enhancement of 16-bit pipeline CR images, 14-bit infrared images, 24-bit pipeline DR weld images, 8-bit or 24-bit rail crack images, and 16-bit remote sensing images respectively. The subjective and objective experimental results show that the design method has stronger adaptability, which has obvious advantages compared with the existing advanced high grayscale image enhancement methods. The enhancement effect is more coordinated, the processing result is more in line with human visual perception, and the details and texture information of the original image can be better preserved.     

The effect of sub-threshold forces on human performance in multi-modal computer-aided design

In traditional CAD (computer-aided design) systems, the manipulation of points and lines is often difficult because designers manipulate virtual objects through their vision system. Nowadays, designers can explore and manipulate virtual objects in haptic-enabled CAD systems using haptic devices. Haptic devices can present force feedbacks to pull or push the users’ hands into desirable targets. Of course the intent is for the user to experience the same sensations in the virtual realm as they would in the real world. Thus, sub-threshold forces, which cannot be perceived by users, should be incorporated in the control of users’ movements. As a result, our attention is directed to study the effect of sub-threshold forces on the accuracy of movement in a haptic-enabled virtual reality (VR) system. In this study, our goal is to manipulate users’ hands using controlled forces such that users cannot notice the forces. With this in mind, we have constructed a haptic-enabled virtual environment (VE) to carry out a multi-modal Fitts’ type task. In the task, subjects could see the position of the haptic probe in the VE where forces were applied on their hands. Basically, the accuracy of subjects was measured using a performance index when the intensity and direction of forces changed. A psychophysical method was utilized to ensure that the forces were below the force threshold of the human force perception. Results indicate that the accuracy is affected by the intensity and direction of sub-threshold forces even when users are allowed to control their actions through visual feedbacks.     

Combining feature spaces for classification

In this paper we offer a variational Bayes approximation to the multinomial probit model for basis expansion and kernel combination. Our model is well-founded within a hierarchical Bayesian framework and is able to instructively combine available sources of information for multinomial classification. The proposed framework enables informative integration of possibly heterogeneous sources in a multitude of ways, from the simple summation of feature expansions to weighted product of kernels, and it is shown to match and in certain cases outperform the well-known ensemble learning approaches of combining individual classifiers. At the same time the approximation reduces considerably the CPU time and resources required with respect to both the ensemble learning methods and the full Markov chain Monte Carlo, Metropolis-Hastings within Gibbs solution of our model. We present our proposed framework together with extensive experimental studies on synthetic and benchmark datasets and also for the first time report a comparison between summation and product of individual kernels as possible different methods for constructing the composite kernel matrix.     

Segmentation of news videos based on audio-video information

In this paper, we propose an innovative architecture to segment a news video into the so-called “stories” by both using the included video and audio information. Segmentation of news into stories is one of the key issues for achieving efficient treatment of news-based digital libraries. While the relevance of this research problem is widely recognized in the scientific community, we are in presence of a few established solutions in the field. In our approach, the segmentation is performed in two steps: first, shots are classified by combining three different anchor shot detection algorithms using video information only. Then, the shot classification is improved by using a novel anchor shot detection method based on features extracted from the audio track. Tests on a large database confirm that the proposed system outperforms each single video-based method as well as their combination.

求解带约束的多峰函数优化问题的新演化算法

求解带约束的多峰函数优化问题在科学研究和工程应用中具有重要的现实意义。本文在对郭涛算法和“两阶段子空间演化算法”改进的基础上,通过采用混沌初始化和混沌变异算子,给出了一种求解带约束条件的多峰函数优化问题的有效的演化算法。通过对具有多峰的复杂函数的测试表明:该算法在求解质量和收敛速度方面都得到了很好的结果,表现出较强的鲁棒性。     

Structurally noise resistant classifier for multi-modal person verification

In this letter we propose a piece-wise linear (PL) classifier for use as the decision stage in a two-modal verification system, comprised of a face and a speech expert. The classifier utilizes a fixed decision boundary that has been specifically designed to account for the effects of noisy audio conditions. Experimental results on the VidTIMIT database show that in clean conditions, the proposed classifier is outperformed by a traditional weighted summation decision stage (using both fixed and adaptive weights). Using white Gaussian noise to corrupt the audio data resulted in the PL classifier obtaining better performance than the fixed approach and similar performance to the adaptive approach. Using a more realistic noise type, namely “operations room” noise from the NOISEX-92 corpus, resulted in the PL classifier obtaining better performance than both the fixed and adaptive approaches. The better results in this case stem from the PL classifier not making a direct assumption about the type of noise that causes the mismatch between training and testing conditions (unlike the adaptive approach). Moreover, the PL classifier has the advantage of having a fixed (non-adaptive, thus simpler) structure.     

跨媒体分析与推理技术研究综述

当前,以网络数据为代表的跨媒体数据呈现爆炸式增长的趋势,呈现出了跨模态、跨数据源的复杂关联及动态演化特性,跨媒体分析与推理技术针对多模态信息理解、交互、内容管理等需求,通过构建跨模态、跨平台的语义贯通与统一表征机制,进一步实现分析和推理以及对复杂认知目标的不断逼近,建立语义层级的逻辑推理机制,最终实现跨媒体类人智能推理...     

基于多通道交互技术的计算机辅助需求分析系统的研发

近年来,人机交互向着认知系统方向发展,以便提供自然与有效的交互原则。认知系统依靠来自多个传感器的数据的处理和融合算法的结合,通过动态地适应外界环境来改善人机交互体验。本原型作品在现有的面向问题的计算机辅助软件需求工程(Computer-Aided Requirements Engineering,CARE)工具的基础上,以及在人机交互理论和设计原则的指导下,针对需求展示的场合,提出了一种基于手势操作的自然交互方式,并在追踪识别手势的过程中运用多通道技术辅助追踪,从而大大加快了追踪与识别的速度,提高了精确度,提供了更好的用户体验。