3D as-is building energy modeling and diagnostics: A review of the state-of-the-art

To achieve ambitious cuts in energy consumptions of the building sector, recent efforts have focused on devising methods that can provide accurate representation of the as-is conditions for existing buildings and in turn increase the quality and effectiveness of building retrofits. Today, 3D geometrical models produced by computer vision and laser scanning methods can be used as the basis of energy modeling purposes. Several methods are also introduced to facilitate the diagnostics and measurement of the thermal and other environmental conditions. To this end, this paper extensively reviews the state-of-the-art techniques that can semi-automatically or automatically create as-is geometrical and thermal models for building energy modeling and retrofit assessment purposes. It also provides an overview on the main algorithms used by these methods for representing spatio-thermal point clouds, automatically converting these point clouds into semantic Building Information Models (BIM) in gbXML format for as-is energy modeling purposes, and also contrasting them with expected energy performance models. The underlying formulations and methods for measuring actual thermal resistance of the building assemblies and mapping them into gbXML-based representations are also presented. The most recent works in the IT-driven building automation system (BAS) for energy conservation purposes are also reviewed. Finally, the technology gaps that need to be addressed in future research are identified and discussed.     

Textured mesh surface reconstruction of large buildings with multi-view stereo

There are three main approaches for reconstructing 3D models of buildings. Laser scanning is accurate but expensive and limited by the laser’s range. Structure-from-motion (SfM) and multi-view stereo (MVS) recover 3D point clouds from multiple views of a building. MVS methods, especially patch-based MVS, can achieve higher density than do SfM methods. Sophisticated algorithms need to be applied to the point clouds to construct mesh surfaces. The recovered point clouds can be sparse in areas that lack features for accurate reconstruction, making recovery of complete surfaces difficult. Moreover, segmentation of the building’s surfaces from surrounding surfaces almost always requires some form of manual inputs, diminishing the ease of practical application of automatic 3D reconstruction algorithms. This paper presents an alternative approach for reconstructing textured mesh surfaces from point cloud recovered by patch-based MVS method. To a good first approximation, a building’s surfaces can be modeled by planes or curve surfaces which are fitted to the point cloud. 3D points are resampled on the fitted surfaces in an orderly pattern, whose colors are obtained from the input images. This approach is simple, inexpensive, and effective for reconstructing textured mesh surfaces of large buildings. Test results show that the reconstructed 3D models are sufficiently accurate and realistic for 3D visualization in various applications.     

Complete residential urban area reconstruction from dense aerial LiDAR point clouds

We present an automatic system to reconstruct 3D urban models for residential areas from aerial LiDAR scans. The key difference between downtown area modeling and residential area modeling is that the latter usually contains rich vegetation. Thus, we propose a robust classification algorithm that effectively classifies LiDAR points into trees, buildings, and ground. The classification algorithm adopts an energy minimization scheme based on the 2.5D characteristic of building structures: buildings are composed of opaque skyward roof surfaces and vertical walls, making the interior of building structures invisible to laser scans; in contrast, trees do not possess such characteristic and thus point samples can exist underneath tree crowns. Once the point cloud is successfully classified, our system reconstructs buildings and trees respectively, resulting in a hybrid model representing the 3D urban reality of residential areas.     

Automatic detection of building roofs from point clouds produced by the dense image matching technique

Three-dimensional (3D) spatial information of object points is a vital requirement for many disciplines. Laser scanning technology and techniques based on image matching have been used extensively to produce 3D dense point clouds. These data are used frequently in various applications, such as the generation of digital surface model (DSM)/digital terrain model (DTM), extracting objects (e.g., buildings, trees, and roads), 3D modelling, and detecting changes. The aim of this study was to extract the building roof points automatically from the 3D point cloud data created via the image matching techniques with optical aerial images (with red, green, and blue band (RGB) and infrared (IR)). In the first stage of the study, as an alternative to laser scanning technology, which is more expensive than optical imaging systems, the 3D point clouds were produced by matching high-resolution images using a Semi Global Matching algorithm. The normalized difference vegetation index (NDVI) values for each point were calculated using the spectral information (RGB + IR) in the 3D point cloud data, and the points that represented the vegetation cover were determined using these values. In the second stage, existing ground and non-ground points that were free of vegetation cover were determined within the point cloud. Subsequently, only the points on the roof of the building were detected automatically using the proposed algorithm. Thus, points of the roofs of buildings located in areas with different topographic characteristics were detected automatically detected using only images. It was determined that the average values of correctness (Corr), completeness (Comp), and quality (Q) of the pixel-based accuracy analysis metrics were 95%, 98%, and 93%, respectively, in the selected test areas. According to the results of the accuracy analysis, it is clear that the proposed algorithm is very successful in automatic extraction of building roof points.     

结合体元数据结构的机载LIDAR建筑物检测

目的 目前,点云、栅格格网及不规则三角网等建筑物检测中常用的离散机载激光雷达（LIDAR）点云数据表达方式存在模型表达复杂、算法开发困难、结果表达不准确及难以表达多返回数据等缺点。为此,针对LIDAR点云体元结构模型构建及在此基础上的建筑物检测展开研究,提出一种基于体元的建筑物检测算法。方法 首先将点云数据规则化为二值（即1、0值,分别表示体元中是否包含有激光点）3D体元结构。然后利用3D滤波算法将上述体元结构中表征数据点的体元分类为地面和非地面体元。最后,依据建筑物边缘的接近直线、跳变特性从非地面体元中搜寻建筑物边缘作为种子体元进而标记与其3D连通的非地面体元集合为建筑物体元。结果 实验基于ISPRS（international society for photogrammetry and remote sensing）提供的包含了不同的建筑物类型的城区LIDAR点云数据测试了"邻域尺度"参数的敏感性及提出算法的精度。定量评价的结果表明：56邻域为最佳邻域尺度;建筑物的检测质量可达到95%以上——平均完整度可达到95.61%、平均正确率可达95.97%。定性评价的结果表明：对大型、密集、不规则形状、高低混合及其他屋顶类型比较特殊的复杂建筑物均可成功检测。结论 本文提出的建筑物检测算法采用基于体元空间邻域关系的搜索标记方式,可有效实现对各类建筑目标特别是城市建筑目标的检测,检测结果易于建模3D建筑物模型。     

基于透视式增强现实的BIM建筑三维重建仿真

为了提高三维建筑模型的精准度,需要深入研究BIM建筑三维重建方法。当前方法耗时较长,得到的三维建筑模型与实际建筑之间的误差较大,存在效率低和精准度低的问题。将透视式增强现实技术应用到BIM建筑三维重建中,提出基于透视式增强现实的BIM建筑三维重建方法,通过BIM构建初始三维建筑模型,采用直接线性变换算法计算摄像机的内部参数和外部参数,完成摄像机标定。在摄像机标定结果的基础上采用LK光流计算方法得到像素在图像中的光流,根据光流的方向阈值和光流的大小筛选图像中的光流,提取到图像的匹配点,基于初始三维建筑模型针对建筑图像匹配点构成空间三维点云,采用Delaunay方法对空间三维点云进行三角化处理,针对处理后的建筑图像通过贴纹理完成BIM建筑三维重建。仿真结果表明,所提方法的效率高、精准度高。     

Alignment of continuous video onto 3D point clouds

We propose a general framework for aligning continuous (oblique) video onto 3D sensor data. We align a point cloud computed from the video onto the point cloud directly obtained from a 3D sensor. This is in contrast to existing techniques where the 2D images are aligned to a 3D model derived from the 3D sensor data. Using point clouds enables the alignment for scenes full of objects that are difficult to model; for example, trees. To compute 3D point clouds from video, motion stereo is used along with a state-of-the-art algorithm for camera pose estimation. Our experiments with real data demonstrate the advantages of the proposed registration algorithm for texturing models in large-scale semi-urban environments. The capability to align video before a 3D model is built from the 3D sensor data offers new practical opportunities for 3D modeling. We introduce a novel modeling-through-registration approach that fuses 3D information from both the 3D sensor and the video. Initial experiments with real data illustrate the potential of the proposed approach.     

基于激光点云数据的客家土楼三维建模

传统测量方法难以获得完整准确的几何信息,且在测量过程中对文化遗产的频繁接触也存在造成文化遗产损害的隐患。利用地面激光扫描技术开展了客家土楼真实感、精细化三维建模的应用研究。首先,在现场点云数据采集的基础上提出了基于点云数据的客家土楼三维高精度重建的具体过程;其次,对点云数据处理和三维实体重建两个核心步骤,重点对点云数据的配准拼接、去噪简化和分站处理、构件轮廓线提取、三维几何建模和纹理贴图等步骤进行了详细论述。另外,对建模过程中纹理优化处理这一难点进行分析探讨,给出具体解决途径。应用结果表明:三维激光扫描技术适用于具有高复杂度几何特征的客家土楼精确化、真实感建模,为客家土楼建筑文化遗产未来的开发与保护过程的损害情况监测、恢复重建和虚拟展示等提供了高精度的数据基础。目前,技术方法已应用于世界遗产地—福建客家土楼的数字化旅游信息服务中,成效明显。     

城市建筑物框架轮廓三维自动重构研究

建筑物是城市景观中一个重要组成部分,因此快速自动实现城市建筑物的三维可视化对于建立数码城市具有重大意义。针对传统三维建模人工劳动强度大、自动化程度低和花费高等问题,提出了一套自动化程度较高的基于线、面信息的城市建筑物框架轮廓快速三维重构方法。通过普通数码相机获取影像数据大大降低了实际应用成本,而且采用线面基元进行三维重构相比于经典的点云重构方法不仅提高了效率,而且获得了清晰的三维边缘轮廓。通过实验验证了方法的有效性和可行性。     

High-quality as-is 3D thermal modeling in MEP systems using a deep convolutional network

With the growing need for automated condition monitoring and analysis in existing buildings, significant effort has been spent on the development of three-dimensional (3D) thermal models. However, little attention has been paid to ensuring the quality of these 3D thermal models, which can directly impact the accuracy of condition monitoring and analysis results. This study aims to propose a method to generate a high-quality 3D thermal model for mechanical, electrical, and plumbing (MEP) systems by bridging the quality discrepancy between high-resolution laser scan data and low-resolution thermal images using a deep convolutional neural network. The proposed method consists of two main parts: (1) improving the resolution of thermal images based on a deep convolutional network and (2) generating a high-quality 3D thermal model by mapping improved thermal images. The performance of the thermal image resolution improvement was validated using a dataset consisting of 312 thermal images. The results demonstrated that the quality of the improved thermal images based on a deep convolutional network was higher than conventional bicubic interpolation in terms of root mean square error (RMSE), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM). Qualitative analysis of a 3D thermal model utilizing the resolution-improved thermal images was also conducted. This was further qualitatively analyzed to have resulted in improved overall quality of the 3D thermal model. The ability to generate a high-quality 3D thermal model can help auditors to perform automated condition monitoring and analysis in buildings based on objective and accurate data.     

A robust hybrid image-based modeling system

This paper presents a new robust image-based modeling system for creating high-quality 3D models of complex objects from a sequence of unconstrained photographs. The images can be acquired by a video camera or hand-held digital camera without the need of camera calibration. In contrast to previous methods, we integrate correspondence-based and silhouette-based approaches, which significantly enhances the reconstruction of objects with few visual features (e.g., uni-colored objects) and improves surface smoothness. Our solution uses a mesh segmentation and charting approach in order to create a low-distortion mesh parameterization suitable for objects of arbitrary genus. A high-quality texture is produced by first parameterizing the reconstructed objects using a segmentation and charting approach, projecting suitable sections of input images onto the model, and combining them using a graph-cut technique. Holes in the texture due to surface patches without projecting input images are filled using a novel exemplar-based inpainting method which exploits appearance space attributes to improve patch search, and blends patches using Poisson-guided interpolation. We analyzed the effect of different algorithm parameters, and compared our system with a laser scanning-based reconstruction and existing commercial systems. Our results indicate that our system is robust, superior to other image-based modeling techniques, and can achieve a reconstruction quality visually not discernible from that of a laser scanner.     

Automated segmentation of RGB-D images into a comprehensive set of building components using deep learning

Building information modeling (BIM) has a semantic scope that encompasses all building systems, e.g. architectural, structural, mechanical, electrical, and plumbing. Automated, comprehensive digital modeling of buildings will require methods for semantic segmentation of images and 3D reconstructions capable of recognizing all building component classes. However, prior building component recognition methods have had limited semantic coverage and are not easily combined or scaled. Here we show that a deep neural network can semantically segment RGB-D (i.e. color and depth) images into 13 building component classes simultaneously despite the use of a small training dataset with only 1490 object instances. For this task, the method achieves an average intersection over union (IoU) of 0.5. The dataset was designed using a common building taxonomy to ensure comprehensive semantic coverage and was collected from a diversity of buildings to ensure intra-class diversity. As a consequence of its semantic scope, it was necessary to perform pre-segmentation and 3D to 2D projection as leverage for dataset annotation. In creating our deep learning pipeline, we found that transfer learning, class balancing, and prevention of overfitting effectively overcame the dataset’s borderline adequate class representation. Our results demonstrate how the semantic coverage of a building component recognition method can be scaled to include a larger diversity of building systems. We anticipate our method to be a starting point for broadening the scope of the semantic segmentation methods involved in digital modeling of buildings.     

A type-2 fuzzy system-based approach for image data fusion to create building information models

Building Information Modelling (BIM) is a standard digital process that fuses buildings information from different sources into a 3D model during their lifecycle. For new construction sites using BIM, it is possible to monitor the cost, schedule, and changes throughout the lifecycle; however, existing buildings do not have a BIM model. Manually creating the BIM models for existing buildings is a high-cost task, both in time and money, hence there is a need for extracting information from available paper-based documentation and fuse it into a BIM model. The struggle of facility management and utility companies to fully adopt a BIM process (due to their high volumes of paper-based documentation of existing buildings) has led to the research on creating these 3D BIM models from 2D floor plan images.This paper presents a novel processing pipeline to extract 2D digital information from floorplans, fusing it into a 3D BIM model. The work focuses on fusing the available information to create the structure of the building in BIM format, which is considered the essential step before looking on working with other sources of data. In this process, we introduce a type-2 fuzzy logic based Explainable Artificial Intelligence (XAI) approach for the semantic segmentation step. The approach consists of using the outputs of type-2 fuzzy logic systems to classify a pixel as wall or background, by using information around and from the pixel of interest as the inputs to the system. After the semantic segmentation step, the output of the type-2 fuzzy logic goes through a noise removal process and finally a transformation from 2D to 3D by assigning the corresponding BIM tag to each identified element. The proposed type-2 fuzzy logic semantic segmentation approach produced comparable results (97.3% mean Intersection over Union (IoU) performance metric value) to the opaque box model approach based on Convolutional Neural Network (CNN) (99.3% mean IoU performance metric value). However, the type-2 fuzzy XAI system benefits from being an augmentable and interpretable model, which means that human users can understand the decision process and modify the model using their expert knowledge.     

BIM技术在古建筑保护中的应用研究

传统测绘方法获取的二维图纸难以满足现有古建筑修缮和保护的要求,本文采用三维激光扫描技术与BIM结合,通过对点云数据的处理可以高效的建立高精度的古建筑三维立体模型,使古建筑三维模型结构化及数据信息化,便于古建筑的数字化存档,为古建筑的修缮和保护提供数据支持。     

Application of clustering for the development of retrofit strategies for large building stocks

In order to reduce energy consumption and emissions from the built environment, it is vital to transform the existing building stock and develop retrofit strategies to achieve energy efficiency and building-integrated renewable energy supply. Compared to developing cost-optimal retrofit strategies for one building, the development of strategies for 100 to up to 10,000 buildings remains a major challenge. This paper presents a method to cluster buildings based on their sensitivity to different retrofit measures, focusing on the cost-effectiveness. Derived from algorithmic clustering and combined with time and cost data, a tailored development of retrofit strategies for large building stocks becomes possible. Improved identification of retrofit measures and strategies, in contrast to the conventional classification based on building type and age, is demonstrated. The method is illustrated, using the data from the case study project ‘Zernez Energia 2020’, which aims to achieve carbon neutrality of a Swiss alpine village.     

虚拟游戏场景中的模型研究

游戏游戏建筑模型是虚拟虚拟游戏模型中的基本对象,已有的对游戏建筑的表示局限于某一层次级别,比如块模型、带屋顶的模型、结构模型和室内模型.引入了连续层次级别这一概念来对地理信息系统中的建筑进行表示和建模,而连续层次级别方法统一了虚拟三维游戏模型中不同类型的建筑的表示.     

基于二维点云图的三维人体建模方法

近年来基于二维图像的三维建模方法取得了快速发展,但就人体建模而言,由于摄像头采集到的二维人体图像包含衣物、发丝等大量的纹理信息,而像虚拟试衣等相关应用需要将人体表面的衣物褶皱等纹理信息去除,同时考虑到裸体数据采集侵犯了用户的隐私,因此提出一种基于二维点云图像到三维人体模型的新型建模方法。与摄像机等辅助设备进行二维图片数据集的采集不同,该算法的输入是由三维人体点云模型以顶点模式绘制的二维点云渲染图。主要工作是建立一个由二维点云图和相应的人体黑白二值图构成的数据集,并训练一个由前者生成后者的生成对抗网络模型。该模型将二维点云图转化为相应的黑白二值图。将该二值图输入一个训练好的卷积神经网络,用于评估二维图像到三维人体模型构建的效果。考虑到由不完整三维点云数据重建完整的三维人体网格模型是一个具有挑战性的问题,因此通过模拟二维点云的破损和残缺状态,使得算法能够处理不完整的二维点云图。大量的实验结果表明,该方法重建出的三维人体模型能够有效实现视觉上的真实感,为了对重建后的精度进行定量的分析,选取了人体特征中具有代表性的腰围特征作为误差评估;为了增加三维人体模型库中人体形态的多样性,还引入一种便捷的三维人体模型数据增强技术。实验结果表明,该算法只需要输入一张二维点云图像,就能快速创建出相应的数字化人体模型。     

基于自适应几何分析的高效交互式导航

三维场景的虚拟导航需要同时保证相机视角的光滑性和智能性,在算法设计上极具挑战。文章研发了一套实时自动生成光滑连续且高效的相机路径导航系统。首先通过线下模型几何分析和语义分析,包含分析场景中建筑物模型、道路模型以及非建筑的重要模型等,求得模型重要性值;其次是自适应地获取路线采样点以及高效存储采样点的视角图片;最后是利用动态规划的方式求出每个采样点的最佳视角并光滑地连接这些采样点形成相机运行轨迹。4 个不同三维场景的实验结果表明了所提算法的高效性和智能性,同时文章所进行的用户调查也充分反映了所提方法具有的明显优势。     

基于机载视频量测的3维城市建模

3维城市模型的快速获取及更新是近十年来计算机视觉及数字摄影测量领域研究的热点。从实用、经济的角度出发，提出了集成城市数字地图、LIDAR data以及机载视频序列影像多数据源，基于数字摄影测量理论的半自动获取3维城市模型的解决办法，并在3维导航的数据生产实践中进行了验证，取得较好的生产效率及效果。