3D model retrieval using weighted bipartite graph matching

In this paper, we propose a view-based 3D model retrieval algorithm, where many-to-many matching method, weighted bipartite graph matching, is employed for comparison between two 3D models. In this work, each 3D model is represented by a set of 2D views. Representative views are first selected from the query model and the corresponding initial weights are provided. These initial weights are further updated based on the relationship among these representative views. The weighted bipartite graph is built with these selected 2D views, and the matching result is used to measure the similarity between two 3D models. Experimental results and comparison with existing methods show the effectiveness of the proposed algorithm.     

Collaborative Distribution Alignment for 2D image-based 3D shape retrieval

Retrieving 3D shapes with 2D images has become a popular research area nowadays, and a great deal of work has been devoted to reducing the discrepancy between 3D shapes and 2D images to improve retrieval performance. However, most approaches ignore the semantic information and decision boundaries of the two domains, and cannot achieve both domain alignment and category alignment in one module. In this paper, a novel Collaborative Distribution Alignment (CDA) model is developed to address the above existing challenges. Specifically, we first adopt a dual-stream CNN, following a similarity guided constraint module, to generate discriminative embeddings for input 2D images and 3D shapes (described as multiple views). Subsequently, we explicitly introduce a joint domain-class alignment module to dynamically learn a class-discriminative and domain-agnostic feature space, which can narrow the distance between 2D image and 3D shape instances of the same underlying category, while pushing apart the instances from different categories. Furthermore, we apply a decision boundary refinement module to avoid generating class-ambiguity embeddings by dynamically adjusting inconsistencies between two discriminators. Extensive experiments and evaluations on two challenging benchmarks, MI3DOR and MI3DOR-2, demonstrate the superiority of the proposed CDA method for 2D image-based 3D shape retrieval task.     

Wasserstein distance feature alignment learning for 2D image-based 3D model retrieval

2D image-based 3D model retrieval has become a hotspot topic in recent years. However, the current existing methods are limited by two aspects. Firstly, they are mostly based on the supervised learning, which limits their application because of the high time and cost consuming of manual annotation. Secondly, the mainstream methods narrow the discrepancy between 2D and 3D domains mainly by the image-level alignment, which may bring the additional noise during the image transformation and influence cross-domain effect. Consequently, we propose a Wasserstein distance feature alignment learning (WDFAL) for this retrieval task. First of all, we describe 3D models through a series of virtual views and use CNNs to extract features. Secondly, we design a domain critic network based on the Wasserstein distance to narrow the discrepancy between two domains. Compared to the image-level alignment, we reduce the domain gap by the feature-level distribution alignment to avoid introducing additional noise. Finally, we extract the visual features from 2D and 3D domains, and calculate their similarity by utilizing Euclidean distance. The extensive experiments can validate the superiority of the WDFAL method.     

3D object retrieval based on sparse coding in weak supervision

With the rapid development of computer vision and digital capture equipment, we can easily record the 3D information of objects. In the recent years, more and more 3D data are generated, which makes it desirable to develop effective 3D retrieval algorithms. In this paper, we apply the sparse coding method in a weakly supervision manner to address 3D model retrieval. First, each 3D object, which is represented by a set of 2D images, is used to learn dictionary. Then, sparse coding is used to compute the reconstruction residual for each query object. Finally, the residual between the query model and the candidate model is used for 3D model retrieval. In the experiment, ETH, NTU and ALOL dataset are used to evaluate the performance of the proposed method. The results demonstrate the superiority of the proposed method.     

Hierarchical deep semantic alignment for cross-domain 3D model retrieval

With the development of deep learning and the widespread application of 3D modeling technology, image-based cross-domain 3D model retrieval has attracted more and more researchers’ attention. Existing methods have achieved success by aligning the feature distributions from different domains. However, previous methods just statistically align the domain-level or class-level feature distributions, leaving sample discriminability a margin to be improved for retrieval. To address this issue, this paper proposes a Hierarchical Deep Semantic Alignment Network (HDSAN) for cross-domain 3D model retrieval, which combines the proposed sample-level semantic enhancement with global domain alignment and class semantic alignment. Concretely, we adopt adversarial domain adaptation at the domain level and dynamically align the class centers of two domains at the class level. To further improve sample discriminability, we design intra-domain and cross-domain triplet center alignment to enhance the semantic representation ability at the sample level. Experiments on two commonly-used cross-domain 3D model retrieval datasets MI3DOR-1 and MI3DOR-2 demonstrate the effectiveness of the proposed method.     

Cross-domain representation learning by domain-migration generative adversarial network for sketch based image retrieval

Sketch based image retrieval (SBIR), which uses free-hand sketches to search the images containing similar objects/scenes, is attracting more and more attentions as sketches could be got more easily with the development of touch devices. However, this task is difficult as the huge differences between sketches and images. In this paper, we propose a cross-domain representation learning framework to reduce these differences for SBIR. This framework aims to transfer sketches to images with the information learned both in the sketch domain and image domain by the proposed domain migration generative adversarial network (DMGAN). Furthermore, to reduce the representation gap between the generated images and natural images, a similarity learning network (SLN) is also proposed with the new designed loss function incorporating semantic information. Extensive experiments have been done from different aspects, including comparison with state-of-the-art methods. The results show that the proposed DMGAN and SLN really work for SBIR.     

裸眼3D视频教育应用价值分析

本文在介绍裸眼3D视频相关概念的基础上,通过介绍四种常见3D显示技术在教育中的应用情况及美国两个典型的裸眼3D视频教学实验案例,分析了裸眼3D视频在教育应用中的效果。进而总结了裸眼3D视频应用于教育的优势,论证了裸眼3D视频在教育中应用的价值所在。     

Colorful 3D reconstruction at high resolution using multi-view representation

High-quality 3D models should contain accurate shapes, as well as other correct attributes, such as realistic surface color. However, current researches were mostly focused on the reconstruction of shapes. We present a method to reconstruct high-resolution colorful 3D models from single images. Shapes and colors are learned separately, using a coarse-to-fine strategy in which the 3D color is expressed as 3-channel volumes. Colorful volumes share the same spatial dimension with generated shape volumes. We propose orthographic colorful maps to retain and recover projected coordinates and corresponding color for 3D surface points. To achieve a fine granularity increase in the quality of maps from low-resolution to high-resolution, we introduce 2D super resolution during reconstructing 3D shapes and color volumes. Models are carved by utilizing predicted high-resolution silhouette, depth and color details. Experimental results in a subset of the ShapeNet dataset and the Colorful Human dataset show the effectiveness of our method.     

电镜重构病毒的三维显示与三维任意区域分割

利用冷冻电镜单颗粒三维重建方法获得分辨率1.5nm家蚕质多角体（BmCPV）;2.5nm,中蜂囊状幼虫病病毒（CSBV）;1.4nm,伊蚊C6/36细胞浓核病毒（C6/36DNV）的三维结构体数据,对三维结构数据进行了数据归并,并找到对显示细节较为敏感的向量维度,用表面显示方法对病毒体数据进行三维显示,用基于空间的方法完成了任意区域的交互分割。显示时间和信噪比与原有显示方法有了提高。病毒表面细节和轴上突起清晰可见。所分割的部分结构准确完整,并可以对所分割的部分进行量化分析。为冷冻电镜单颗粒重构提供了新的三维可视与分析方法。     

Fast 3D video stabilization using ROI-based warping

With the rapid development of portable digital video equipment, such as camcorders, digital cameras and smart phones, video stabilization techniques for camera de-shaking are strongly required. The cutting-edge video stabilization techniques provide outstanding visual quality by utilizing 3D motion, while early video stabilization is based on 2D motion only. Recently, a content-preserving warping algorithm has been acknowledged as state-of-the-art thanks to its superior stabilization performance. However, the huge computational cost of this technique is a serious burden in spite of its excellent performance. Thus, we propose a fast video stabilization algorithm that provides significantly reduced computational complexity over the state-of-the-art with the same stabilization performance. First, we estimate the 3D information of the feature points in each input frame and define the region of interest (ROI) based on the estimated 3D information. Next, if the number of feature points in the ROI is sufficient, we apply the proposed ROI-based pre-warping and content-preserving warping sequentially to the input frame. Otherwise, conventional full-frame warping is applied. From intensive simulation results, we find that the proposed algorithm reduces computational complexity to 14% of that of the state-of-the-art method, while keeping almost equivalent stabilization performance.     

Direct method for rotation estimation from spherical images using 3D mesh surfaces with SPHARM representation

Since a large field of view obviously bears important advantages, the use of spherical images is becoming increasingly important in various computer vision and image processing applications. This paper presents a novel rotation estimation approach for spherical images based on 3D mesh representation of gray level intensity. Once the 3D meshes of the underlying spherical images are obtained, the 3D rotation can be estimated directly and efficiently, without feature extraction and matching process. Subsequently, we propose a direct method for 3D object rotation estimation using spherical harmonics representation with SVD decomposition and ICP algorithm for estimation refinement. Experimental results validate our approach and prove its suitability and robustness for rotation estimation. Moreover, it performs well against noisy images, brightness changes, image compression and occlusions. A comparative study of our proposed approach with four similar methods for 3D rotation estimation between spherical images, is realized to prove its effectiveness.     

基于3D-DWT及混沌映射的加密医用体数据鲁棒检索算法研究

随着医疗数据的共享日益频繁,数据安全问题变得日益突出.安全起见,把医学图像加密后存储到服务器或云端.如何实现对加密医学图像检索成为一个亟待解决的问题.提出一种基于混沌和三维小波变换的加密医用体数据鲁棒检索算法.其主要步骤为:首先,通过三维小波变换和混沌映射对医用体数据进行加密;然后,对加密体数据进行小波分解,得到子带图像,再对子带图像进行DCT变换,提取DCT变换能量聚集的低频系数作为加密体数据的特征向量,建立加密图像的特征向量数据库;最后,通过计算加密图像的特征向量的相似度来检索目标图像,将检索到的加密图像返回,并解密所检索到的医用体数据.实验结果表明,该检索算法对加密体数据的常规攻击和几何攻击具有较强的鲁棒性,并提高了图像检索的安全性和准确性.     

三维小波多分等级树视频压缩编码

提出了基于三维小波多发等级树视频压缩编码方法。通过将原始图象序列进行帧分组,并对每个图象帧组在时间、水平和垂直三维空间上进行小波变换。然后,对变换域的各个空间－时间频率了带按照其方向的不同。构成多个小波等级树,并对每个等级树分别进行ＳＰＩＨＴ编码算法。仿真实验表明了该方法的有效性。     

Detecting spatiotemporal irregularities in videos via a 3D convolutional autoencoder

Spatiotemporal irregularities (i.e., the uncommon appearance and motion patterns) in videos are difficult to detect, as they are usually not well defined and appear rarely in videos. We tackle this problem by learning normal patterns from regular videos, while treating irregularities as deviations from normal patterns. To this end, we introduce a 3D fully convolutional autoencoder (3D-FCAE) that is trainable in an end-to-end manner to detect both temporal and spatiotemporal irregularities in videos using limited training data. Subsequently, temporal irregularities can be detected as frames with high reconstruction errors, and irregular spatiotemporal patterns can be detected as blurry regions that are not well reconstructed. Our approach can accurately locate temporal and spatiotemporal irregularities thanks to the 3D fully convolutional autoencoder and the explored effective architecture. We evaluate the proposed autoencoder for detecting irregular patterns on benchmark video datasets with weak supervision. Comparisons with state-of-the-art approaches demonstrate the effectiveness of our approach. Moreover, the learned autoencoder shows good generalizability across multiple datasets.     

多稀疏表示分类器决策融合修正距离的图像检索

针对当前许多图 像检索方法的检索精度不理想的问题,本文为增强图像特征的表达能力,通过统计图像的颜 色矩、多尺度分块 局部二值模式、灰度共生矩阵、尺度不变特征变换以及空间位置信息,提取5类能从不同角 度表征图像本 质特性的特征,并根据图像库中各训练图像的类别信息,以此5类特征构造5个稀疏表示分类 器,同时引 入决策融合思想,根据每个子分类器的分类性能,通过一个自适应迭代运算过程确定各子分 类器的融合权 值,以刻画不同类别特征的图像表达能力,并据此构造距离修正因子对不同特征所描述的图 像间距离进行 修正,从而得到综合各类特征表达能力的图像间的修正距离,实现图像的相似性评价,获得 检索结果。实 验结果表明,基于Corel-1000图像库,本文提出的方法平均查准率 为82.1%,比现有的方法平均提升10个百分点 ,而且鲁棒性更强。     

3D model retrieval by sample based alignment

In 3D model retrieval, preprocessing of 3D models is needed, in which alignment is a key factor that significantly affects retrieval performance. In particular, the anti-rotation image feature can obtain the alignment effect of 3D model views. In practice, the focus of many users of 3D models is not just on retrieval performance, but the use of aligned models for different purposes. In this paper, we propose a method, namely Sample Based Alignment (SBA) for better 3D model alignment and retrieval. In SBA, given a class, a sample model is used as the target for alignment, after which each 3D model in this class is then aligned one by one, i.e., the 3D model is actually rotated. Our experimental results, based on two 3D model datasets and performance comparisons with other methods, demonstrate the superiority of the SBA method over state-of-the-art methods in terms of 3D model retrieval and classification.     

三维卷积神经网络及其在视频理解领域中的应用研究

3维卷积神经网络(3D CNN)是近几年来深度学习研究中的热点,在计算机视觉领域取得了诸多成就。虽然研究多年且成果丰富,但目前仍缺少关于此内容全面、细致的综述。基于此,该文从以下几个方面对其进行综述：首先阐述3维卷积神经网络的基本原理和模型结构,接着从网络结构、网络内部和优化方法总结3维卷积神经网络的相关改进工作,然后对3维卷积神经网络在视频理解领域中的应用进行总结,最后总结全文内容并对未来发展方向进行展望。该文针对3维卷积神经网络的最新研究进展以及在视频理解领域中的应用进行了系统的综述,对3维卷积神经网络的研究发展具有一定的积极意义。