Visual saliency assistance mechanism based on visually impaired navigation systems

Visually impaired people face significant challenges in their work, studies, and daily lives. Nowadays, numerous visually impaired navigation devices are available to help visually impaired people solve daily life problems. These devices typically include modules for target recognition, distance measurement, and text reading aloud. They are designed to help visually impaired people avoid obstacles and understand the presence of things around them through text reading aloud. Due to the need to avoid all potential obstacles, the target recognition algorithms embedded in these devices must recognize a wide range of targets. However, the text reading aloud module cannot read all of them. Therefore, we designed a visual saliency assistance mechanism that simulates the regions that humans may pay the most attention to in the whole picture. The output of the visual saliency assistance mechanism is overlaid with the target recognition result, which can greatly reduce the target number of text reading aloud. This way, the visually impaired navigation device can not only help to avoid obstacles but also help visually impaired people understand the interest targets of most people in the whole picture. The visual saliency assistance mechanism we designed consists of three components: a spatio-temporal feature extraction (STFE) module, a spatio-temporal feature fusion (STFF) module, and a multi-scale feature fusion (MSFF) module. The STFF module fuses long-term spatio-temporal features and improves the temporal memory between frames. The MSFF module fully integrates information at different scales to improve the accuracy of saliency prediction. Therefore, this proposed visual saliency model can assist in the efficient operation of visually impaired navigation systems. The area under roc curve judd (AUC-J) metric of the proposed model was 93.9%, 93.8%, and 91.5% on three widely used saliency datasets: Holly-wood2, UCF Sports, and DHF1K, respectively. The results show that our proposed model outperforms the current state-of-the-art models.     

一种视相关的网格简化方法*

针对网格模型的实时显示问题提出了一种视相关的网格简化方法。在网格简化中,将影响图形对象重要性的视觉敏感度与图形对象在屏幕上投影面积两者有效结合,实现网格模型的实时大幅度简化, 实验结果表明,简化速度快,但显示并无明显失真。     

基于局部对比和全局稀有度的显著性检测

为了提高显著图的精确度,能够准确地提取显著目标,提出了结合局部对比和全局稀有度的显著性检测算法.该方法通过两个方面来衡量显著性:局部对比和全局稀有度.采用多尺度高斯差分模拟“中央周边差”计算中心点与其周围点的各特征之间的差异来描述局部对比,采用多尺度高斯卷积图像的特征及特征方差的概率计算全局稀有度,将局部对比和全局稀有度融合和归一化得到最终综合显著图.实验结果表明,该方法能较好地检测图像中的显著区域,在突出图像中不同物体边缘的同时,也能突出图像中同质区域的显著性,说明了结合局部和全局的算法能够得到更好的显著图.     

自适应融合局部和全局稀疏表示的图像显著性检测

针对基于局部或全局稀疏表示的图像显著性检测方法频繁出现提取对象不完整、边界不光滑及噪声消除不干净等问题,提出自适应融合局部和全局稀疏表示的图像显著性检测方法。首先,对原始图像进行分块处理,利用图像块代替像素操作,降低算法复杂度;其次,对分块后的图像进行局部稀疏表示,即：针对每一个图像块,选取其周围的若干图像块生成过完备字典,基于该字典对图像块进行稀疏重构,得到原始图像的初始局部显著图,该显著图能够有效提取显著性目标的边缘信息;接着,对分块后的图像进行全局稀疏表示,与局部稀疏表示过程类似,不同的是针对每一个图像块所生成的字典来源于图像四周边界处的图像块,这样可以得到能有效检测出显著性目标内部区域的初始全局显著图;最后,将初始局部和全局显著图进行自适应融合,生成最终显著图。实验结果表明,提出算法在查准率（precision）、查全率（recall）及F-measure等指标上优于几种经典的图像显著性检测方法。     

Human detection using a mobile platform and novel features derived from a visual saliency mechanism

Human detection is a key ability to an increasing number of applications that operates in human inhabited environments or needs to interact with a human user. Currently, most successful approaches to human detection are based on background substraction techniques that apply only to the case of static cameras or cameras with highly constrained motions. Furthermore, many applications rely on features derived from specific human poses, such as systems based on features derived from the human face which is only visible when a person is facing the detecting camera. In this work, we present a new computer vision algorithm designed to operate with moving cameras and to detect humans in different poses under partial or complete view of the human body. We follow a standard pattern recognition approach based on four main steps: (i) preprocessing to achieve color constancy and stereo pair calibration, (ii) segmentation using depth continuity information, (iii) feature extraction based on visual saliency, and (iv) classification using a neural network. The main novelty of our approach lies in the feature extraction step, where we propose novel features derived from a visual saliency mechanism. In contrast to previous works, we do not use a pyramidal decomposition to run the saliency algorithm, but we implement this at the original image resolution using the so-called integral image. Our results indicate that our method: (i) outperforms state-of-the-art techniques for human detection based on face detectors, (ii) outperforms state-of-the-art techniques for complete human body detection based on different set of visual features, and (iii) operates in real time onboard a mobile platform, such as a mobile robot (15 fps).     

物体显著性排名感知网络用于高效图像检索

针对目前图像检索领域主要依靠语义相似性检索图片而忽略了场景中物体重要性关系问题,提出了一种基于场景感知的物体显著性排名算法SASR,使图像检索更关注场景中物体的相互关系。SASR分为两个阶段,在第一阶段,提出了基于视点数据的“组合阈值”物体级显著性排名真值标签标注方法,该方法简化了排名标签的标注;在第二阶段,提出了基于图卷积网络的物体级显著性排序网络,该网络解决了多个在物体级排序问题中存在的特异性难点。该算法改善了目前显著性排名标签生成方式并进行了大量对比实验,在现有SALICON数据集上的实验结果表明,其提升了显著性排名的性能,在NUS-WIDE数据集上的实验结果表明在该算法的支撑下,图像检索性能平均提升了2%,证明了其有效性。     

Visual saliency guided normal enhancement technique for 3D shape depiction

Visual saliency can effectively guide the viewer's visual attention to salient regions of a 3D shape. Incorporating the visual saliency measure of a polygonal mesh into the normal enhancement operation, a novel saliency guided shading scheme for shape depiction is developed in this paper. Due to the visual saliency measure of the 3D shape, our approach will adjust the illumination and shading to enhance the geometric salient features of the underlying model by dynamically perturbing the surface normals. The experimental results demonstrate that our non-photorealistic shading scheme can enhance the depiction of the underlying shape and the visual perception of its salient features for expressive rendering. Compared with previous normal enhancement techniques, our approach can effectively convey surface details to improve shape depiction without impairing the desired appearance.     

Hexahedral mesh smoothing via local element regularization and global mesh optimization

The quality of finite element meshes is one of the key factors that affect the accuracy and reliability of finite element analysis results. In order to improve the quality of hexahedral meshes, we present a novel hexahedral mesh smoothing algorithm which combines a local regularization for each hexahedral mesh, using dual element based geometric transformation, with a global optimization operator for all hexahedral meshes. The global optimization operator is composed of three main terms, including the volumetric Laplacian operator of hexahedral meshes and the geometric constraints of surface meshes which keep the volumetric details and the surface details, and another is the transformed node displacements condition which maintains the regularity of all elements. The global optimization operator is formulated as a quadratic optimization problem, which is easily solved by solving a sparse linear system. Several experimental results are presented to demonstrate that our method obtains higher quality results than other state-of-the-art approaches.     

基于多级全局信息传递模型的视觉显著性检测

对神经网络中的卷积特征采用分层处理的思想能明显提升显著目标检测的性能。然而,在集成分层特征时,如何获得丰富的全局信息以及有效融合较高层特征空间的全局信息和底层细节信息仍是一个没有解决的问题。为此,提出了一种基于多级全局信息传递模型的显著性检测算法。为了提取丰富的多尺度全局信息,在较高层级引入了多尺度全局特征聚合模块（MGFAM）,并且将多层级提取出的全局信息进行特征融合操作;此外,为了同时获得高层特征空间的全局信息和丰富的底层细节信息,将提取到的有判别力的高级全局语义信息以特征传递的方式和较低层次特征进行融合。这些操作可以最大限度提取到高级全局语义信息,同时避免了这些信息在逐步传递到较低层时产生的损失。在ECSSD、PASCAL-S、SOD、HKU-IS等4个数据集上进行实验,实验结果表明,所提算法相较于较先进的NLDF模型,其F-measure（F）值分别提高了0.028、0.05、0.035和0.013,平均绝对误差（MAE）分别降低了0.023、0.03、0.023和0.007。同时,所提算法在准确率、召回率、F-measure值及MAE等指标上也优于几种经典的图像显著性检测方法。     

Material-Discontinuity Preserving Progressive Mesh Using Vertex-Collapsing Simplification

Level Of Detail (LOD) modelling or mesh reduction has been found useful in interactive walkthrough applications. Progressive meshing techniques based on edge or triangle collapsing have been recognised useful in continuous LOD, progressive refinement, and progressive transmission. We present a vertex-collapsing mesh reduction scheme that effectively takes shape and feature preserving as well as material-discontinuity preserving into account, and produces a progressive mesh which generally has more vertices collapsed between adjacent levels of detail than methods based on edge-collapsing and triangle collapsing.     

基于边顶点重要度的网格简化算法及应用

基于三角形网格边折叠简化思想，提出一种基于边顶点重要度简化算法，采用一种改进的三角形网格数据结构，在预处理中利用二叉树对顶点重要度进行快速排序并记录三角形合并关系，得到所需分辨率下的近似网格模型。     

结合整体一致性和局部差异性的图像目标显著性检测

目的 为准确描述图像的显著信息,提出一种结合整体一致性和局部差异性的显著性检测方法,并将显著性特征融入到目标分割中。方法 首先,利用频率调谐法(IG)对目标整体特征的一致性进行显著性检测。然后,引入NIF算法检测显著目标的局部差异性。最后结合两种算法形成最终的显著性检测方法,并应用于图像目标分割。结果 在公认的Weizmann数据集上验证本文方法显示目标的绝对效率并与其他算法对比,实验结果表明本文方法在精确率,召回率,F1-measure(分别为0.445 6,0.751 2,0.576 4)等方面优于当前流行的算法。并且在融合显著性的图像目标分割中,取得满意的实验结果。结论 提出一种新的显著性检测算法,综合体现目标的整体和局部特征,并在公开数据集上取得较高的统计评价。实验结果表明,该算法能够对自然图像进行较准确的显著性检测,并成功地应用于自然图像的目标分割。     

基于图像显著性的路面裂缝检测

有效的视觉显著性方法能准确快速地帮助人们在大量视觉信息中找到感兴趣的物体.针对实际路面图像噪声成分复杂、覆盖面广的特点,提出一种基于图像显著性的路面裂缝检测算法.该算法对路面裂缝图像分块灰度校正后,根据灰度稀疏性、全局对比度计算粗尺度下的裂缝显著值,然后由裂缝局部亮度、边缘特性、连续性特点进行不断扩张的细尺度的局部邻域显著性增强,再经空间显著性加强后,采用自适应阈值分割提取裂缝.大量的实验结果表明,该算法比传统算法更能正确、有效地检测出裂缝整体区域,抗噪声能力强,漏检率和误检率很低,具有和人类视觉特性相符合的检测结果.     

基于视觉显著性的均值漂移跟踪算法

为解决突变运动下的目标跟踪问题,提出了一种基于视觉显著性的均值漂移跟踪算法,将视觉注意机制运用到均值漂移跟踪框架中,利用时空显著性算法对视频序列进行检测,生成视觉显著图,从视觉显著图对应的显著性区域中建立目标的颜色特征表示模型来实现运动目标跟踪.实验结果表明:该算法在摄像机摇晃等动态场景下可以较准确检测出时空均显著的目标,有效克服了在运动目标发生丢失和遮挡等情况下跟踪不稳定的问题,具有较强的鲁棒性,从而实现复杂场景下目标较准确的跟踪.     

一种基于视觉特性的三角网格简化算法

在计算机图形学和几何造型中,实体模型经常采用多边形网格描述,由于绘制时间和存储量与网格的数量成正比,因此复杂的网格模型通常并不实用,从而必须进行模型简化。因为任意多边形可以很方便地被剖分为三角形,由此该文提出一种新的基于视觉特性的三角形网格简化算法。该算法基于人类的视觉特性对三角形网格进行重要性分析,模型细节的选择取决于整个模型对视觉效果的贡献程度,在用户指定的尺度范围内,通过采用收缩三角形以达到迅速简化的目的,以较小的图形生成代价获取丰富的图形视觉效果。实验结果表明,该算法具有实现简单,速度快的特点,能有效地支持细节层次模型的表示。     

骨骼蒙皮动画中网格优化的研究与实现

在骨骼蒙皮动画中需要处理大量的网格三角形,而在很多骨骼蒙皮动画的应用中都需要很高的动画实时性,为了提高骨骼蒙皮动画的实时性,对网格的优化是其中一项重要的工作.本文提出了一种优化网格的方法,并予以实现.     

Prediction of mobile image saliency and quality under cloud computing environment

Recent years have witnessed the explosive growth of multimedia applications over networks and increasingly high requirements of consumers for multimedia signals in terms of quality of experience (QoE). Effective and efficient yet energy-saving saliency detection model and quality prediction method are eagerly desired, since they play critical roles in raising users' QoE and promoting the progress of green multimedia communication. Current studies of saliency detection and quality evaluation are far from ideal yet. In this paper we investigate the influence of complexity on visual saliency and quality. Complexity is an essential concept in human perception to visual stimulus, but it is substantially abstract and hard to be endowed with a clear definition. We suppose that brain systematically combines global and local features during the whole process of human perception. Global features lead a dominant position in seeking salient areas under the condition that image complexity is high, namely without obviously isolated foreground objects, whereas local features play a key role in an opposite situation. With this consideration, this paper establishes a novel framework for detecting visual saliency based on image complexity estimation before complexity-adaptive merging of global and local features. Furthermore, the concept of complexity is deployed for blind photographic image quality assessment (IQA) by means of saliency-based weighting. Features which refer to contrast, artifacts, brightness and natural scene statistics (NSS) are modified and integrated to derive a blind IQA model and predict the quality of photos. Based on the above two technologies, this paper introduces smart phones as mobile terminals, cloud platforms for speed-up and energy-saving, and wireless networks for transmission, and provides a practical mobile multimedia application. Comparative experiments validate that, within this application system, our proposed saliency detection model and blind photographic IQA method implement better than existing relevant competitors in terms of effectiveness and efficiency comparison.     

结合区域协方差分析的图像显著性检测

目的 图像显著性检测的目的是为了获得高质量的能够反映图像不同区域显著性程度的显著图,利用图像显著图可以快速有效地处理图像中的视觉显著区域。图像的区域协方差分析将图像块的多维特征信息表述为一个协方差矩阵,并用协方差距离来度量两个图像块特征信息的差异大小。结合区域协方差分析,提出一种新的图像显著性检测方法。方法 该方法首先将输入的图像进行超像素分割预处理;然后基于像素块的区域协方差距离计算像素块的显著度;最后对像素块进行上采样用以计算图像像素点的显著度。结果 利用本文显著性检测方法对THUS10000数据集上随机选取的200幅图像进行了显著性检测并与4种不同方法进行了对比,本文方法估计得到的显著性检测结果更接近人工标定效果,尤其是对具有复杂背景的图像以及前背景颜色接近的图像均能达到较好的检测效果。结论 本文方法将图像像素点信息和像素块信息相结合,避免了单个噪声像素点引起图像显著性检测的不准确性,提高了检测精确度;同时,利用协方差矩阵来表示图像特征信息,避免了特征点的数量、顺序、光照等对显著性检测的影响。该方法可以很好地应用到显著目标提取和图像分割应用中。