An automated vision-based method for rapid 3D energy performance modeling of existing buildings using thermal and digital imagery

Modeling the energy performance of existing buildings enables quick identification and reporting of potential areas for building retrofit. However, current modeling practices of using energy simulation tools do not model the energy performance of buildings at their element level. As a result, potential retrofit candidates caused by construction defects and degradations are not represented. Furthermore, due to manual modeling and calibration processes, their application is often time-consuming. Current application of 2D thermography for building diagnostics is also facing several challenges due to a large number of unordered and non-geo-tagged images. To address these limitations, this paper presents a new computer vision-based method for automated 3D energy performance modeling of existing buildings using thermal and digital imagery captured by a single thermal camera. First, using a new image-based 3D reconstruction pipeline which consists of Graphic Processing Unit (GPU)-based Structure-from-Motion (SfM) and Multi-View Stereo (MVS) algorithms, the geometrical conditions of an existing building is reconstructed in 3D. Next, a 3D thermal point cloud model of the building is generated by using a new 3D thermal modeling algorithm. This algorithm involves a one-time thermal camera calibration, deriving the relative transformation by forming the Epipolar geometry between thermal and digital images, and the MVS algorithm for dense reconstruction. By automatically superimposing the 3D building and thermal point cloud models, 3D spatio-thermal models are formed, which enable the users to visualize, query, and analyze temperatures at the level of 3D points. The underlying algorithms for generating and visualizing the 3D spatio-thermal models and the 3D-registered digital and thermal images are presented in detail. The proposed method is validated for several interior and exterior locations of a typical residential building and an instructional facility. The experimental results show that inexpensive digital and thermal imagery can be converted into ubiquitous reporters of the actual energy performance of existing buildings. The proposed method expedites the modeling process and has the potential to be used as a rapid and robust building diagnostic tool.     

A review of 3D reconstruction techniques in civil engineering and their applications

Three-dimensional (3D) reconstruction techniques have been used to obtain the 3D representations of objects in civil engineering in the form of point cloud models, mesh models and geometric models more often than ever, among which, point cloud models are the basis. In order to clarify the status quo of the research and application of the techniques in civil engineering, literature retrieval is implemented by using major literature databases in the world and the result is summarized by analyzing the abstracts or the full papers when required. First, the research methodology is introduced, and the framework of 3D reconstruction techniques is established. Second, 3D reconstruction techniques for generating point clouds and processing point clouds along with the corresponding algorithms and methods are reviewed respectively. Third, their applications in reconstructing and managing construction sites and reconstructing pipelines of Mechanical, Electrical and Plumbing (MEP) systems, are presented as typical examples, and the achievements are highlighted. Finally, the challenges are discussed and the key research directions to be addressed in the future are proposed. This paper contributes to the knowledge body of 3D reconstruction in two aspects, i.e. summarizing systematically the up-to-date achievements and challenges for the applications of 3D reconstruction techniques in civil engineering, and proposing key future research directions to be addressed in the field.     

Multi-class US traffic signs 3D recognition and localization via image-based point cloud model using color candidate extraction and texture-based recognition

Continuous condition monitoring and inspection of traffic signs are essential to ensure that safety and performance criteria are met. The use of 3D point cloud modeling by the construction industry has been significantly increased in recent years especially for recording the as-is conditions of facilities. The high-precision and dense 3D point clouds generated by photogrammetry can facilitate the process of asset condition assessment. This paper presents an automated computer-vision based method that detects, classifies, and localizes traffic signs via street-level image-based 3D point cloud models. The proposed pipeline integrates 3D object detection algorithm. An improved Structure-from-Motion (SfM) procedure is developed to create a 3D point cloud of roadway assets from the street level imagery. In order to assist with accurate 3D recognition and localization by color and texture features extraction, an automated process of point cloud cleaning and noise removal is proposed. Using camera pose information from SfM, the points within the bounding box of detected traffic signs are then projected into the cleaned point cloud by using the triangulation method (linear and non-linear) and the 3D points corresponding to the traffic sign in question are labeled and visualized in 3D. The proposed framework is validated using real-life data, which represent the most common types of traffic signs. The robustness of the proposed pipeline is evaluated by analyzing the accuracy in detection of traffic signs as well as the accuracy in localization in 3D point cloud model. The results promise to better and more accurate visualize the location of the traffic signs with respect to other roadway assets in 3D environment.     

飞行时间3维相机的多视角散乱点云优化配准

针对目前基于飞行时间(TOF)原理的3维相机实现物体完整表面的3维点云重建过程中,多视角散乱点云配准精度低的问题,提出一种优化配准方法。该方法通过构建一个目标功能函数,并结合相邻点云的转换矩阵对该目标函数进行最小化求解,直接获取任意位置的点云到基准点云所处坐标系的绝对转换矩阵,避免了对连续点云之间的配准而引起误差的累加。对不同的物体进行实验,实验结果表明,该方法在保证点云配准速度的同时,提高了多视角点云配准的精度,物体点云模型重建效果较好,有利于实现后期3维曲面网格的重建。     

Three-dimensional (3D) reconstruction of structures and landscapes: A new point-and-line fusion method

The technique of three-dimensional (3D) reconstruction is widely used to develop infrastructure and landscape models to manage cities and assets better. Accurately reconstructing 3D structures (e.g., planes or lines) is a core step in rebuilding a model, especially within a built environment, where piece-wise planar/linear structures predominately prevail. As high-resolution images of large areas have become increasingly accessible, this paper develops an improved 3D reconstruction pipeline using the combination of point and line features. By introducing a dense reconstruction algorithm, which is an improved patch based stereo matching algorithm, this paper presents a robust approach that can be used to overcome the inaccuracies, integrity and reconstruction inefficiencies associated with point clouds. A 3D line extraction method is added to reconstruct accurate edges of buildings. The experimental results demonstrate that the proposed method visually improves the reconstruction effect of a 3D structure and a model's visualization.     

结合视觉惯性模组的室内三维布局鲁棒重建方法

针对使用传统单目相机的全自动三维重建方法结果精确度差和整体结构理解缺失等问题,提出一种结合视觉惯性里程计和由运动到结构的全自动室内三维布局重建系统.首先利用视觉里程计获得关键帧图像序列和对应空间位置姿态,并利用运动恢复结构算法计算精确相机位姿;然后利用多图视立体几何算法生成高质量稠密点云;最后基于曼哈顿世界假设,针对典型的现代建筑室内场景,设计一种基于规则的自底向上的布局重建方法,得到最终房间外轮廓布局.使用浙江大学CAD&CG实验室场景现场扫描数据集和人工合成的稠密点云数据集作为实验数据,在Ubuntu 16.04和PCL 1.9环境下进行实验.结果表明,文中方法对三维点云噪声容忍度高,能够有效地重建出室内场景的三维外轮廓布局.     

Urban building reconstruction from raw LiDAR point data

We present a method for automatic reconstruction of the volumetric structures of urban buildings, directly from raw LiDAR point clouds. Given the large-scale LiDAR data from a group of urban buildings, we take advantage of the “divide-and-conquer” strategy to decompose the entire point clouds into a number of subsets, each of which corresponds to an individual building. For each urban building, we determine its upward direction and partition the corresponding point data into a series of consecutive blocks, achieved by investigating the distributions of feature points of the building along the upward direction. Next, we propose a novel algorithm, Spectral Residual Clustering (SRC), to extract the primitive elements within the contours of blocks from the sectional point set, which is formed by registering the series of consecutive slicing points. Subsequently, we detect the geometric constraints among primitive elements through individual fitting, and perform constrained fitting over all primitive elements to obtain the accurate contour. On this basis, we execute 3D modeling operations, like extrusion, lofting or sweeping, to generate the 3D models of blocks. The final accurate 3D models are generated by applying the union Boolean operations over the block models. We evaluate our reconstruction method on a variety of raw LiDAR scans to verify its robustness and effectiveness.     

一种基于SfM重建点云的三角网格化算法*

针对SfM重建点云的曲面建模问题,提出一种改进的区域增长网格化算法。定义k近邻影响域提高拓扑稳定性,引入二叉排序树高效地组织候选三角片,采用无向环搜索策略完成孔洞的检测,最终获得完整的三角网格面。实验结果表明,该算法相比于Possion曲面重建,在获得高的重建精度的同时显著提高了计算效率,有助于提升3D曲面重建与模型表现的性能。     

Feature Preserving Mesh Generation from 3D Point Clouds

We address the problem of generating quality surface triangle meshes from 3D point clouds sampled on piecewise smooth surfaces. Using a feature detection process based on the covariance matrices of Voronoi cells, we first extract from the point cloud a set of sharp features. Our algorithm also runs on the input point cloud a reconstruction process, such as Poisson reconstruction, providing an implicit surface. A feature preserving variant of a Delaunay refinement process is then used to generate a mesh approximating the implicit surface and containing a faithful representation of the extracted sharp edges. Such a mesh provides an enhanced trade‐off between accuracy and mesh complexity. The whole process is robust to noise and made versatile through a small set of parameters which govern the mesh sizing, approximation error and shape of the elements. We demonstrate the effectiveness of our method on a variety of models including laser scanned datasets ranging from indoor to outdoor scenes.     

空地融合的城市道路基础设施数字化技术研究

城市道路基础设施三维模型的重构,在城市道路BIM应用与数字化领域具有重大意义;针对城市交通基础设施数字化重构的需求,对车载激光扫描与无人机倾斜摄影采集技术进行综合运用,提出一套从信息采集、空地点云配准、点云分割到三维重构的完整技术方法;首先使用车载激光扫描技术和无人机倾斜摄影技术对交通基础设施信息进行采集,并使用运动恢复结构算法(SfM,structure from motion)生成基础设施空地点云;其次使用迭代最近点法(ICP,iterative closest point)对空地点云进行精配准,然后利用基于PointNet网络的方法对融合点云进行语义分割;最后对分割出的交通基础设施对象进行三维重构;提出的空地融合的城市交通基础设施数字化技术能够高效地实现交通基础设施重构,为城市交通基础设施数字化提供基础、为后续交通专业领域的应用研究提供便利.     

股骨NURBS三维实体重建及有限元分析

研究人体股骨是各个分支学科继续发展的需要,能够对人体骨骼进行精确的重建则是研究的基础和关键,股骨的NURBS实体重建有着重大的意义。本文详细介绍了一种通过逆向工程软件Geomagic与三维CAD软件UG相结合,实现股骨从点云到NURBS曲面到实体的三维模型精确重建方法;最后利用ANSYS对重建模型进行了分析,证明了所建模型的正确性与有效性。     

PointCleanNet: Learning to Denoise and Remove Outliers from Dense Point Clouds

Point clouds obtained with 3D scanners or by image-based reconstruction techniques are often corrupted with significant amount of noise and outliers. Traditional methods for point cloud denoising largely rely on local surface fitting (e.g. jets or MLS surfaces), local or non-local averaging or on statistical assumptions about the underlying noise model. In contrast, we develop a simple data-driven method for removing outliers and reducing noise in unordered point clouds. We base our approach on a deep learning architecture adapted from PCPNet, which was recently proposed for estimating local 3D shape properties in point clouds. Our method first classifies and discards outlier samples, and then estimates correction vectors that project noisy points onto the original clean surfaces. The approach is efficient and robust to varying amounts of noise and outliers, while being able to handle large densely sampled point clouds. In our extensive evaluation, both on synthetic and real data, we show an increased robustness to strong noise levels compared to various state-of-the-art methods, enabling accurate surface reconstruction from extremely noisy real data obtained by range scans. Finally, the simplicity and universality of our approach makes it very easy to integrate in any existing geometry processing pipeline. Both the code and pre-trained networks can be found on the project page ( https://github.com/mrakotosaon/pointcleannet ).     

点云算法在医学领域的研究进展

点云作为一种重要的3维数据,能够直观地模拟生物器官、组织等的3维结构,基于医学点云数据的分类、分割、配准、目标检测等任务可以辅助医生进行更为准确的诊断和治疗,在临床医学以及个性化医疗器械辅助设计与3D打印有着重要的应用价值。随着深度学习的发展,越来越多的点云算法逐步由传统算法扩展到深度学习算法中。本文对点云算法在医学领域的研究及其应用进行综述,旨在总结目前用于医学领域的点云方法,包括医学点云的特点、获取途径以及数据转换方法;医学点云分割中的传统算法和深度学习算法;以及医学点云的配准任务定义、意义,以及基于有/无特征的配准方法。总结了医学点云在临床应用中仍存在的限制和挑战：1）医学图像重建的人体器官点云分布稀疏且包含噪音、误差;2）医学点云数据集标注困难、制作成本高,可用于训练深度学习模型的公开数据集非常稀少;3）前沿的点云处理算法大都基于自然场景点云数据集训练,这些算法在医学点云处理中的鲁棒性和泛化能力还有待验证。随着医学点云数据集质量和数量的提升,医学点云处理算法的研究将会吸引更多的研究者。     

基于 AISI 网络的 BIM 三维重建方法研究

自动从点云数据生成建筑信息模型(BIM)一直是建筑自动化领域的研究热点。基于 传统算法的建筑自动三维重建的缺点包括人工设计特征,识别过程复杂,应用场景有限等。随 着三维机器学习领域的不断成熟,处理点云便有了新的手段。通过引入实例分割中的 ASIS 网 络框架对点云进行处理,即从扫描点云场景中自动分割和分类建筑构建元素并得到实例分割矩 阵。接着,基于包围盒假设从得到的实例分割矩阵中提取建筑构件外轮廓参数,并将外轮廓参 数和分割的语义分类结果作为 BIM 建模的构件参数。最后,将这些提取的构件参数输入到自制 的 IFC 生成器中,自动生成基于工业基础类(IFC)标准的 BIM 模型。实验表明,利用无噪点点 云方法,可实现基于曼哈顿世界假设下的室内单房间的三维重建。     

误差云映射的深度融合3维重建

描述了一种应用于视频序列的3维稠密重建方法,主要针对基于多深度图恢复3维物体时存在的多对一映射重复投影问题,提出基于误差云概念的3维定点优化方法.该方法首先通过深度矫正,增加视频序列深度图的一致性;其次利用投影过滤分类算法,把所有投影点按照多对一映射的关系,以3维空间中所有不同点为类型,进行投影点依次映射,将每一点划分在各自的误差云类中以求得投影点与3维点的对应关系;随后采用空间高斯分布求出每个误差云所恢复出来的点坐标.最后通过多边形技术对恢复的点云数据进行网格化,使其重建出精确的目标物体或场景的3维轮廓.从实验结果可以观察到,本文3维重建方法可以有效减少深度图融合多对一映射重复投影问题所带来的负面影响,使重建结果更加接近于真实物体,达到较好的效果.     

Reconstructing Animated Meshes from Time‐Varying Point Clouds

In this paper, we describe a novel approach for the reconstruction of animated meshes from a series of time‐deforming point clouds. Given a set of unordered point clouds that have been captured by a fast 3‐D scanner, our algorithm is able to compute coherent meshes which approximate the input data at arbitrary time instances. Our method is based on the computation of an implicit function in ?⁴ that approximates the time‐space surface of the time‐varying point cloud. We then use the four‐dimensional implicit function to reconstruct a polygonal model for the first time‐step. By sliding this template mesh along the time‐space surface in an as‐rigid‐as‐possible manner, we obtain reconstructions for further time‐steps which have the same connectivity as the previously extracted mesh while recovering rigid motion exactly. The resulting animated meshes allow accurate motion tracking of arbitrary points and are well suited for animation compression. We demonstrate the qualities of the proposed method by applying it to several data sets acquired by real‐time 3‐D scanners.     

基于激光点云数据的客家土楼三维建模

传统测量方法难以获得完整准确的几何信息,且在测量过程中对文化遗产的频繁接触也存在造成文化遗产损害的隐患。利用地面激光扫描技术开展了客家土楼真实感、精细化三维建模的应用研究。首先,在现场点云数据采集的基础上提出了基于点云数据的客家土楼三维高精度重建的具体过程;其次,对点云数据处理和三维实体重建两个核心步骤,重点对点云数据的配准拼接、去噪简化和分站处理、构件轮廓线提取、三维几何建模和纹理贴图等步骤进行了详细论述。另外,对建模过程中纹理优化处理这一难点进行分析探讨,给出具体解决途径。应用结果表明:三维激光扫描技术适用于具有高复杂度几何特征的客家土楼精确化、真实感建模,为客家土楼建筑文化遗产未来的开发与保护过程的损害情况监测、恢复重建和虚拟展示等提供了高精度的数据基础。目前,技术方法已应用于世界遗产地—福建客家土楼的数字化旅游信息服务中,成效明显。     

结构光视觉三维点云逐层三角网格化算法

针对结构光视觉恢复的大规模三维点云的可投影特点,提出一种基于投影网格的底边驱动逐层网格化曲面重建算法。该算法首先将点云投影到一个二维平面上;然后基于点云投影区域建立规则投影网格,并将投影点映射到规则二维投影网格上,建立二维网格点与三维点云间的映射关系;接着对投影网格进行底边驱动的逐层网格化,建立二维三角网格;最后根据二维投影点与三维点的对应关系及二维三角网格拓扑关系获得最终的三维网格曲面。实验结果表明,算法曲面重建速度快,可较好地保持曲面细节特征。     

飞行时间三维相机的多视角散乱点云优化配准方法研究

针对目前基于飞行时间(TOF)原理的三维相机实现物体完整表面的三维点云重建过程中,多视角散乱点云配准精度低的问题,本文提出了一种多视角散乱点云优化配准方法。该方法通过构建一个目标功能函数,并结合相邻点云的变换矩阵对该目标函数进行最小化求解,直接获取任意位置的点云到基准坐标系的绝对变换矩阵,避免了对相邻点云的变换矩阵进行累积而引起误差的累加。实验结果表明,该方法提高了多视角点云配准的精度,同时增强了物体点云模型重建的效果,在三维曲面重建中具有较强的实用性。     

End-to-end point cloud-based segmentation of building members for automating dimensional quality control

A frequent and accurate quality inspection procedure to assess the quality requirements during the life cycle of buildings is crucial. Among different quality measures, the dimensional quality that involves spatial features of buildings is of significant importance. However, the traditional manual inspection of dimensional quality in buildings is unreliable and tedious. Thus, this study presents an end-to-end method for quality inspection of building structural members using point cloud datasets. The proposed method, first, detects and labels structural members within the point cloud based on a set of domain-specific geometric and semantic definitions. Then, each structural member's section width, height, and length are obtained with the proposed bounding box method. Experiments on three real-world buildings' point clouds with various geometric features and noise levels, occlusion, and outliers were also conducted, illustrating the performance efficiency and accuracy of the proposed model for dimensional quality inspection of building structural members.