High-quality as-is 3D thermal modeling in MEP systems using a deep convolutional network

With the growing need for automated condition monitoring and analysis in existing buildings, significant effort has been spent on the development of three-dimensional (3D) thermal models. However, little attention has been paid to ensuring the quality of these 3D thermal models, which can directly impact the accuracy of condition monitoring and analysis results. This study aims to propose a method to generate a high-quality 3D thermal model for mechanical, electrical, and plumbing (MEP) systems by bridging the quality discrepancy between high-resolution laser scan data and low-resolution thermal images using a deep convolutional neural network. The proposed method consists of two main parts: (1) improving the resolution of thermal images based on a deep convolutional network and (2) generating a high-quality 3D thermal model by mapping improved thermal images. The performance of the thermal image resolution improvement was validated using a dataset consisting of 312 thermal images. The results demonstrated that the quality of the improved thermal images based on a deep convolutional network was higher than conventional bicubic interpolation in terms of root mean square error (RMSE), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM). Qualitative analysis of a 3D thermal model utilizing the resolution-improved thermal images was also conducted. This was further qualitatively analyzed to have resulted in improved overall quality of the 3D thermal model. The ability to generate a high-quality 3D thermal model can help auditors to perform automated condition monitoring and analysis in buildings based on objective and accurate data.     

A 3D face and hand biometric system for robust user-friendly authentication

A complete authentication system based on fusion of 3D face and hand biometrics is presented and evaluated in this paper. The system relies on a low cost real-time sensor, which can simultaneously acquire a pair of depth and color images of the scene. By combining 2D and 3D facial and hand geometry features, we are able to provide highly reliable user authentication robust to appearance and environmental variations. The design of the proposed system addresses two basic requirements of biometric technologies: dependable performance under real-world conditions along with user convenience. Experimental evaluation on an extensive database recorded in a real working environment demonstrates the superiority of the proposed multimodal scheme against unimodal classifiers in the presence of numerous appearance and environmental variations, thus making the proposed system an ideal solution for a wide range of real-world applications, from high-security to personalization of services and attendance control.     

基于三维数据与MMSV特征的二维人脸识别*

针对二维人脸识别对姿态与光照变化较为敏感的问题,提出了一种基于三维数据与混合多尺度奇异值特征MMSV(mixture of multi-scale singular value,MMSV)的二维人脸识别方法。在训练阶段,利用三维人脸数据与光照模型获取大量具有不同姿态和光照条件的二维虚拟图像,为构造完备的特征模板奠定基础;同时,通过子集划分有效地缓解了人脸特征提取过程中的非线性问题;最后对人脸图像进行MMSV特征提取,从而对人脸的全局与局部特征进行融合。在识别阶段,通过计算MMSV特征子空间距离完成分类识别。实验证明,提取到的MMSV特征包含有更多的鉴别信息,对姿态和光照变化具有理想的鲁棒性。该方法在WHU-3D数据库上取得了约98.4%的识别率。     

Constructing 3D human model from front and side images

Constructing 3D human model from 2D images provides a cost-effective approach to visualize digital human in virtual environment. This paper presents a systematic approach for constructing 3D human model using the front and side images of a person. The silhouettes of human body are first detected and the feature points on the silhouettes are subsequently identified. The feature points are further used to obtain the body dimensions that are necessary for identifying a template 3D human model. The shape of the template human model can be modified by the free-form deformation method. Moreover, the proposed approach has been applied for constructing the 3D human models of 30 subjects. The comparisons between the constructed 3D models and the 3D scanning models of the 30 subjects indicate that the proposed system is very effective and robust.     

Automatic 3D face recognition from depth and intensity Gabor features

As is well known, traditional 2D face recognition based on optical (intensity or color) images faces many challenges, such as illumination, expression, and pose variation. In fact, the human face generates not only 2D texture information but also 3D shape information. In this paper, we investigate what contributions depth and intensity information makes to face recognition when expression and pose variations are taken into account, and we propose a novel system for combining depth and intensity information to improve face recognition systems. In our system, local features described by Gabor wavelets are extracted from depth and intensity images, which are obtained from 3D data after fine alignment. Then a novel hierarchical selecting scheme embedded in linear discriminant analysis (LDA) and AdaBoost learning is proposed to select the most effective and most robust features and to construct a strong classifier. Experiments are performed on the CASIA 3D face database and the FRGC V2.0 database, two data sets with complex variations, including expressions, poses and long time lapses between two scans. Experimental results demonstrate the promising performance of the proposed method. In our system, all processes are performed automatically, thus providing a prototype of automatic face recognition combining depth and intensity information.     

Automatic facial feature extraction and 3D face modeling using two orthogonal views with application to 3D face recognition

We present a fully automated algorithm for facial feature extraction and 3D face modeling from a pair of orthogonal frontal and profile view images of a person's face taken by calibrated cameras. The algorithm starts by automatically extracting corresponding 2D landmark facial features from both view images, then compute their 3D coordinates. Further, we estimate the coordinates of the features that are hidden in the profile view based on the visible features extracted in the two orthogonal face images. The 3D coordinates of the selected feature points obtained from the images are used first to align, then to locally deform the corresponding facial vertices of the generic 3D model. Preliminary experiments to assess the applicability of the resulted models for face recognition show encouraging results.     

Circular camera array system for 3‐D displays based on the reconstruction of parallax rays

Abstract— A circular camera system employing an image‐based rendering technique that captures light‐ray data needed for reconstructing three‐dimensional (3‐D) images by using reconstruction of parallax rays from multiple images captured from multiple viewpoints around a real object in order to display a 3‐D image of a real object that can be observed from multiple surrounding viewing points on a 3‐D display is proposed. An interpolation algorithm that is effective in reducing the number of component cameras in the system is also proposed. The interpolation and experimental results which were performed on our previously proposed 3‐D display system based on the reconstruction of parallax rays will be described. When the radius of the proposed circular camera array was 1100 mm, the central angle of the camera array was 40°, and the radius of a real 3‐D object was between 60 and 100 mm, the proposed camera system, consisting of 14 cameras, could obtain sufficient 3‐D light‐ray data to reconstruct 3‐D images on the 3‐D display.     

Multi-class US traffic signs 3D recognition and localization via image-based point cloud model using color candidate extraction and texture-based recognition

Continuous condition monitoring and inspection of traffic signs are essential to ensure that safety and performance criteria are met. The use of 3D point cloud modeling by the construction industry has been significantly increased in recent years especially for recording the as-is conditions of facilities. The high-precision and dense 3D point clouds generated by photogrammetry can facilitate the process of asset condition assessment. This paper presents an automated computer-vision based method that detects, classifies, and localizes traffic signs via street-level image-based 3D point cloud models. The proposed pipeline integrates 3D object detection algorithm. An improved Structure-from-Motion (SfM) procedure is developed to create a 3D point cloud of roadway assets from the street level imagery. In order to assist with accurate 3D recognition and localization by color and texture features extraction, an automated process of point cloud cleaning and noise removal is proposed. Using camera pose information from SfM, the points within the bounding box of detected traffic signs are then projected into the cleaned point cloud by using the triangulation method (linear and non-linear) and the 3D points corresponding to the traffic sign in question are labeled and visualized in 3D. The proposed framework is validated using real-life data, which represent the most common types of traffic signs. The robustness of the proposed pipeline is evaluated by analyzing the accuracy in detection of traffic signs as well as the accuracy in localization in 3D point cloud model. The results promise to better and more accurate visualize the location of the traffic signs with respect to other roadway assets in 3D environment.     

基于图像辅助的三维模型形状检索

随着3D建模技术的快速发展,互联网上可用的3D模型库出现了爆炸式增长,越来越多的3D模型可以方便地通过网络下载使用.这直接促使了3D形状检索技术的发展,即给出特定的搜索信息,要求系统搜索出符合要求的、相似的3D模型.文章提出了一种新的3D形状检索方法,以3D模型作为输入,系统将会从模型数据库中自动检索出与输入形状最相似的模型.对于给定的输入模型以及数据库中的每一个模型,首先由计算机生成多幅在不同视角下的2D草图;然后,应用Gabor滤波器对每一幅2D草图提取图像上的局部特征,并对特征进行量化,从而得到代表该图像特征的直方图,这样对于每一个3D模型将得到多个代表该模型的直方图;最终,通过对比两个模型之间直方图的相似性,可以得到它们的相似性值,从而检索出与输入模型最相似的模型.文章所提出的方法通过采取2D图像分析方法提取能反映3D模型的特征并计算出模型之间的相似性值.经过测试,在一些公开的数据集上得到了较好的效果.     

基于3维立体技术的视觉加密

视觉加密是新的加密研究领域,其优点是利用人眼视觉系统的特性直接对被加密的内容进行解码,而不需要用专用软件或硬件进行复杂的解密计算。近年来,由于自动立体显示技术和立体眼镜技术的发展,使得3维显示的应用日趋广泛,而传统的视觉加密方法在3维视频领域内尚无法取得令人满意的效果,因此3维立体视觉加密研究具有重要意义。为了更有效地对3维视频进行加密,利用3维显示的原理和特点,将其与视觉加密的优点相结合,提出了一种3维立体视觉加密新方法,该方法利用人眼视觉特性中的视差特性将隐藏信息巧妙地隐藏在右视图中,实验结果表明,该方法是有效的。     

Robust palmprint verification using 2D and 3D features

This paper presents a new personal authentication system that simultaneously exploits 2D and 3D palmprint features. The objective of our work is to improve accuracy and robustness of existing palmprint authentication systems using 3D palmprint features. The proposed multilevel framework for personal authentication efficiently utilizes the robustness (against spoof attacks) of the 3D features and the high discriminating power of the 2D features. The developed system uses an active stereo technique, structured light, to simultaneously capture 3D image or range data and a registered intensity image of the palm. The surface curvature feature based method is investigated for 3D palmprint feature extraction while Gabor feature based competitive coding scheme is used for 2D representation. We comparatively analyze these representations for their individual performance and attempt to achieve performance improvement using the proposed multilevel matcher that utilizes fixed score level combination scheme to integrate information. Our experiments on a database of 108 subjects achieved significant improvement in performance with the integration of 3D features as compared to the case when 2D palmprint features alone are employed. We also present experimental results to demonstrate that the proposed biometric system is extremely difficult to circumvent, as compared to the currently proposed palmprint authentication approaches in the literature.     

动态多视角复杂3D人体行为数据库及行为识别

提供了一个较大规模的基于RGB-D摄像机的人体复杂行为数据库DMV (Dynamic and multi-view) action3D,从2个固定视角和一台移动机器人动态视角录制人体行为。数据库现有31个不同的行为类,包括日常行为、交互行为和异常行为类等三大类动作,收集了超过620个行为视频约60万帧彩色图像和深度图像,为机器人寻找最佳视角提供了可供验证的数据库。为验证数据集的可靠性和实用性,本文采取4种方法进行人体行为识别,分别是基于关节点信息特征、基于卷积神经网络（Convolutional neural networks,CNN)和条件随机场(Conditional random field,CRF)结合的CRFasRNN方法提取的彩色图像HOG3D特征,然后采用支持向量机(Support vector machine,SVM)方法进行了人体行为识别;基于3维卷积网络（C3D）和3D密集连接残差网络提取时空特征,通过softmax层以预测动作标签。实验结果表明：DMV action3D人体行为数据库由于场景多变、动作复杂等特点,识别的难度也大幅增大。DMV action3D数据集对于研究真实环境下的人体行为具有较大的优势,为服务机器人识别真实环境下的人体行为提供了一个较佳的资源。     

Recognition of 3D facial expression dynamics

In this paper we propose a method that exploits 3D motion-based features between frames of 3D facial geometry sequences for dynamic facial expression recognition. An expressive sequence is modelled to contain an onset followed by an apex and an offset. Feature selection methods are applied in order to extract features for each of the onset and offset segments of the expression. These features are then used to train GentleBoost classifiers and build a Hidden Markov Model in order to model the full temporal dynamics of the expression. The proposed fully automatic system was employed on the BU-4DFE database for distinguishing between the six universal expressions: Happy, Sad, Angry, Disgust, Surprise and Fear. Comparisons with a similar 2D system based on the motion extracted from facial intensity images was also performed. The attained results suggest that the use of the 3D information does indeed improve the recognition accuracy when compared to the 2D data in a fully automatic manner.     

An automated vision-based method for rapid 3D energy performance modeling of existing buildings using thermal and digital imagery

Modeling the energy performance of existing buildings enables quick identification and reporting of potential areas for building retrofit. However, current modeling practices of using energy simulation tools do not model the energy performance of buildings at their element level. As a result, potential retrofit candidates caused by construction defects and degradations are not represented. Furthermore, due to manual modeling and calibration processes, their application is often time-consuming. Current application of 2D thermography for building diagnostics is also facing several challenges due to a large number of unordered and non-geo-tagged images. To address these limitations, this paper presents a new computer vision-based method for automated 3D energy performance modeling of existing buildings using thermal and digital imagery captured by a single thermal camera. First, using a new image-based 3D reconstruction pipeline which consists of Graphic Processing Unit (GPU)-based Structure-from-Motion (SfM) and Multi-View Stereo (MVS) algorithms, the geometrical conditions of an existing building is reconstructed in 3D. Next, a 3D thermal point cloud model of the building is generated by using a new 3D thermal modeling algorithm. This algorithm involves a one-time thermal camera calibration, deriving the relative transformation by forming the Epipolar geometry between thermal and digital images, and the MVS algorithm for dense reconstruction. By automatically superimposing the 3D building and thermal point cloud models, 3D spatio-thermal models are formed, which enable the users to visualize, query, and analyze temperatures at the level of 3D points. The underlying algorithms for generating and visualizing the 3D spatio-thermal models and the 3D-registered digital and thermal images are presented in detail. The proposed method is validated for several interior and exterior locations of a typical residential building and an instructional facility. The experimental results show that inexpensive digital and thermal imagery can be converted into ubiquitous reporters of the actual energy performance of existing buildings. The proposed method expedites the modeling process and has the potential to be used as a rapid and robust building diagnostic tool.     

基于小波变换的三维医学图象压缩技术的研究

满足诊断要求的医学影像无失真压缩编码技术的研究是图象压缩编码研究的新领域,也是ＰＡＣＳ系统和远程医疗诊断系统研究的热点之一。该文正是以ＣＴ,ＭＲＩ等医疗成像设备在临床应用时一次检查产生的人体某一部位的多帧断层图象组为对象,取目前编码领域最具潜力的小波变换技术,并将小波变换由二维推广至三维,同时将二维工邓列图象组视为三维图象而开展的医学影像编码技术研究。实验结果表明,在经临床医师确认的失真门限附近,     

Floating autostereoscopic display with in situ interaction

Floating three‐dimensional (3D) display implements direct interaction between human hands and virtual 3D images, which offers natural and effective augmented reality interaction. In this study, we propose a novel floating autostereoscopic display, combining head tracking lenticular display with an image projection system, to offer the observers with an accurate 3D image floating in midair without any optical elements between observers and the virtual 3D image. Combined with a gesture recognition device, the proposed system can achieve in situ augmented reality interaction with the floating 3D image. A distortion correction method is developed to achieve 3D display with accurate spatial information. Moreover, a coordinate calibration method is designed to improve the accuracy in the in situ interaction. Experiments were performed to prove the feasibility of the proposed system, and the good results show the potential of human‐computer interaction in medicine and life sciences.     

Level-set segmentation of brain tumors using a threshold-based speed function

The level set approach can be used as a powerful tool for 3D segmentation of a tumor to achieve an accurate estimation of its volume. A major challenge of such algorithms is to set the equation parameters, especially the speed function. In this paper, we introduce a threshold-based scheme that uses level sets for 3D tumor segmentation (TLS). In this scheme, the level set speed function is designed using a global threshold. This threshold is defined based on the idea of confidence interval and is iteratively updated throughout the evolution process. We propose two threshold-updating schemes, search-based and adaptive, that require different degrees of user involvement. TLS does not require explicit knowledge about the tumor and non-tumor density functions and can be implemented in an automatic or semi-automatic form depending on the complexity of the tumor shape. The proposed algorithm has been tested on magnetic resonance images of the head for tumor segmentation and its performance evaluated visually and quantitatively. The experimental results confirm the effectiveness of TLS and its superior performance when compared with a region-competition based method.     

Automated cell analysis in 2D and 3D: A comparative study

Optical projection tomographic microscopy is a technique that allows 3D analysis of individual cells. Theoretically, 3D morphometry would more accurately capture cellular features than 2D morphometry. To evaluate this thesis, classifiers based on 3D reconstructions of cell nuclei were compared with 2D images from the same nuclei. Human adenocarcinoma and normal lung epithelium cells were used. Testing demonstrated a three-fold reduction in the false negative rate for adenocarcinoma detection in 3D versus 2D at the same high specificity. We conclude that 3D imaging will potentially expand the horizon for automated cell analysis with broad applications in the biological sciences.     

异常步态3维人体建模和可变视角识别

目的 运用视觉和机器学习方法对步态进行研究已成为当前热点,但多集中在身份识别领域。本文从不同的视角对其进行研究,探讨一种基于点云数据和人体语义特征模型的异常步态3维人体建模和可变视角识别方法。方法 运用非刚性变形和蒙皮方法,构建基于形体和姿态语义特征的参数化3维人体模型;以红外结构光传感器获取的人体异常步态点云数据为观测目标,构建其对应形体和姿态特征的3维人体模型。通过ConvGRU（convolution gated necurrent unit）卷积循环神经网络来提取其投影深度图像的时空特征,并将样本划分为正样本、负样本和自身样本三元组,对异常步态分类器进行训练,以提高分类器对细小差异的鉴别能力。同时对异常步态数据获取难度大和训练视角少的问题,提出了一种基于形体、姿态和视角变换的训练样本扩充方法,以提高模型在面对视角变化时的泛化能力。结果 使用CSU（Central South University）3维异常步态数据库和DHA（depth-included human action video）深度人体行为数据库进行实验,并对比了不同异常步态或行为识别方法的效果。结果表明,本文方法在CSU异常步态库实验中,0°、45°和90°视角下对异常步态的综合检测识别率达到了96.6%,特别是在90°到0°交叉和变换视角实验中,比使用DMHI（difference motion history image）和DMM-CNN（depth motion map-convolutional neural network）等步态动作特征要高出25%以上。在DHA深度人体运动数据库实验中,本文方法识别率接近98%,比DMM等相关算法高出2%~3%。结论 提出的3维异常步态识别方法综合了3维人体先验知识、循环卷积网络的时空特性和虚拟视角样本合成方法的优点,不仅能提高异常步态在面对视角变换时的识别准确性,同时也为3维异常步态检测和识别提供一种新思路。     

基于网络的手机定制系统的实现方法研究*

首先分析了基于网络的手机定制特点和现状,并从用户个性化在线配置角度分析了目前主要基于文本和二维图形的网络化定制系统中存在的问题,进而对其系统结构、三维模型构建、系统集成等关键问题进行了研究并分析了其中的关键技术。结合典型手机产品开发了原型系统,该系统用一种比VRML更加优越的新的Web3D实现技术——Cult3D实现了网络三维产品的虚拟展示。经实际应用证明,该系统可有效地提高定制产品的客户满意度,进而提高企业的竞争力。