Image Registration in a Coarse Three-Dimensional Virtual Environment

In recent years, the availability of off‐the‐shelf geometric data for an urban environment has increased. During rendering, ground level images are mapped onto the façades of the buildings to improve the visual quality of the scene. This paper focuses on a technique that enables ground level images to be automatically integrated into an existing coarse three‐dimensional environment. The approach utilises the planar nature of architectural scenes to enable the automatic extraction of a building façade from an image and its registration into the virtual environment.     

Geometry and Bonding Rules Position Analysis in the Formation of Traditional Iranian Architectural Brick Facades

Over time, architectural ornaments have had a variety of forms that are created from a combination of different types of materials and factors. Brick decorations are one such example, a type of ornament that is often formed on the exterior façade of buildings. Investigating covert rules and brick bonds geometry can reveal a part of the ancient architectural secrets and ways used to reduce the ravages upon new brick façades. Shaped bricks in traditional Iranian architecture directly affect the placement, size and brick arrangement rules. This article is the first attempt to study the effects of geometry and the principal rules of arrangement that help to shape the formation of brick façades specifically, as well as analysing the samples of brick façades taken from monuments, field recordings and discussion with the workmen themselves. The analysis has revealed that permanent rules govern the bonding of brick façades, along with the size and dimensions of the brick effects, which shape the final look of the decoration.     

Procedural Modeling of a Building from a Single Image

Creating a virtual city is demanded for computer games, movies, and urban planning, but it takes a lot of time to create numerous 3D building models. Procedural modeling has become popular in recent years to overcome this issue, but creating a grammar to get a desired output is difficult and time consuming even for expert users. In this paper, we present an interactive tool that allows users to automatically generate such a grammar from a single image of a building. The user selects a photograph and highlights the silhouette of the target building as input to our method. Our pipeline automatically generates the building components, from large‐scale building mass to fine‐scale windows and doors geometry. Each stage of our pipeline combines convolutional neural networks (CNNs) and optimization to select and parameterize procedural grammars that reproduce the building elements of the picture. In the first stage, our method jointly estimates camera parameters and building mass shape. Once known, the building mass enables the rectification of the façades, which are given as input to the second stage that recovers the façade layout. This layout allows us to extract individual windows and doors that are subsequently fed to the last stage of the pipeline that selects procedural grammars for windows and doors. Finally, the grammars are combined to generate a complete procedural building as output. We devise a common methodology to make each stage of this pipeline tractable. This methodology consists in simplifying the input image to match the visual appearance of synthetic training data, and in using optimization to refine the parameters estimated by CNNs. We used our method to generate a variety of procedural models of buildings from existing photographs.     

Enhancing Urban Façades via LiDAR‐Based Sculpting

Buildings with symmetrical façades are ubiquitous in urban landscapes and detailed models of these buildings enhance the visual realism of digital urban scenes. However, a vast majority of the existing urban building models in web‐based 3D maps such as Google earth are either less detailed or heavily rely on texturing to render the details. We present a new framework for enhancing the details of such coarse models, using the geometry and symmetry inferred from the light detection and ranging (LiDAR) scans and 2D templates. The user‐defined 2D templates, referred to as coded planar meshes (CPMs), encodes the geometry of the smallest repeating 3D structures of the façades via face codes. Our encoding scheme, take into account the directions, type as well as the offset distance of the sculpting to be applied at the respective locations on the coarse model. In our approach, LiDAR scan is registered with the coarse models taken from Google earth 3D or Bing maps 3D and decomposed into dominant planar segments (each representing the frontal or lateral walls of the building). The façade segments are then split into horizontal and vertical tiles using a weighted point count function defined over the window or door boundaries. This is followed by an automatic identification of CPM locations with the help of a template fitting algorithm that respects the alignment regularity as well as the inter‐element spacing on the façade layout. Finally, 3D boolean sculpting operations are applied over the boxes induced by CPMs and the coarse model, and a detailed 3D model is generated. The proposed framework is capable of modelling details even with occluded scans and enhances not only the frontal façades (facing to the streets) but also the lateral façades of the buildings. We demonstrate the potentials of the proposed framework by providing several examples of enhanced Google earth models and highlight the advantages of our method when designing photo‐realistic urban façades.     

Building renovation adopts mass customization

This work is motivated by an industrial need of manufacturing façades insulating envelopes in order to reduce energy consumption in residential buildings. An insulating envelope is a configuration of a set of rectangular panels that respects a set of limitations. Due to the number of façades to be renovated and the number of possible configurations for a single façade, the envelope configuration is both a mass customization problem as well as a combinatorial one. The paper then introduces a decision support system based on the framework of constraint satisfaction, as it fits neatly the constrained nature of the problem. Two configuration tasks have been identified as prerequisite to envelopes configurations: (1) the configuration of a questionnaire for information inputs and (2) the configuration of a constraint satisfaction problem for each one of the façades to be renovated. The system architecture promotes maintenance, modularity and efficiency as different configuration tasks are divided into web-services. Conception and implementation of the massive building thermal renovation are then supported.     

3D city models for urban farming site identification in buildings

Studies have suggested that there is farming potential in urban residential buildings. However, these studies are limited in scope, require field visits and time-consuming measurements. Furthermore, they have not suggested ways to identify suitable sites on a larger scale let alone means of surveying numerous micro-locations across the same building. Using a case study area focused on high-rise buildings in Singapore, this paper examines a novel application of three-dimensional (3D) city models to identify suitable farming micro-locations (level and orientation) in residential buildings. We specifically investigate whether the vertical spaces of these buildings comprising outdoor corridors, façades and windows receive sufficient photosynthetically active radiation (PAR) for growing food crops and do so at a high resolution. We also analyze the spatio-temporal characteristics of PAR, and the impact of shadows and different weather conditions on PAR in the building. Environmental simulations on the 3D model of the study area indicated that the cumulative daily PAR or Daily Light Integral (DLI) at a location in the building was dependent on its orientation and shape, sun's diurnal and annual motion, weather conditions, and shadowing effects of the building's own façades and surrounding buildings. The DLI in the study area generally increased with building's levels and, depending on the particular micro-location, was found suitable for growing moderately light-demanding crops such as lettuce and sweet pepper. These variations in DLI at different locations of the same building affirmed the need for such simulations. The simulations were validated with field measurements of PAR, and correlation coefficients between them exceeded 0.5 in most cases thus, making a case that 3D city models offer a promising practical solution to identifying suitable farming locations in residential buildings, and have the potential for urban-scale applications.     

Nighttime image dehazing using color cast removal and dual path multi-scale fusion strategy

Nighttime image dehazing aims to remove the effect of haze on the images captured in nighttime, which however, raises new challenges such as severe color distortion, more complex lighting conditions, and lower contrast. Instead of estimating the transmission map and atmospheric light that are difficult to be accurately acquired in nighttime, we propose a nighttime image dehazing method composed of a color cast removal and a dual path multi-scale fusion algorithm. We first propose a human visual system (HVS) inspired color correction model, which is effective for removing the color deviation on nighttime hazy images. Then, we propose to use dual path strategy that includes an underexposure and a contrast enhancement path for multi-scale fusion, where the weight maps are achieved by selecting appropriate exposed areas under Gaussian pyramids. Extensive experiments demonstrate that the visual effect of the hazy nighttime images in real-world datasets can be significantly improved by our method regarding contrast, color fidelity, and visibility. In addition, our method outperforms the state-of-the-art methods qualitatively and quantitatively.     

Interaction with Media Façades

Media façades are a prominent example of the digital augmentation of urban spaces. They denote the concept of turning the surface of a building into a large-scale urban screen. Because of their enormous size, they require interaction at a distance and they have a high level of visibility. Additionally, they are situated in a highly dynamic urban environment with rapidly changing conditions, which results in settings which are neither comparable, nor reproducible. Altogether, this makes the development of interactive media façade installations a challenging task.     

融合目标增强与稀疏重构的显著性检测

目的 为了解决图像显著性检测中存在的边界模糊,检测准确度不够的问题,提出一种基于目标增强引导和稀疏重构的显著检测算法（OESR）。方法 基于超像素,首先从前景角度计算超像素的中心加权颜色空间分布图,作为前景显著图;由图像边界的超像素构建背景模板并对模板进行预处理,以优化后的背景模板作为稀疏表示的字典,计算稀疏重构误差,并利用误差传播方式进行重构误差的校正,得到背景差异图;最后,利用快速目标检测方法获取一定数量的建议窗口,由窗口的对象性得分计算目标增强系数,以此来引导两种显著图的融合,得到最终显著检测结果。结果 实验在公开数据集上与其他12种流行算法进行比较,所提算法对具有不同背景复杂度的图像能够较准确的检测出显著区域,对显著对象的提取也较为完整,并且在评价指标检测上与其他算法相比,在MSRA10k数据集上平均召回率提高4.1%,在VOC2007数据集上,平均召回率和F检验分别提高18.5%和3.1%。结论 本文提出一种新的显著检测方法,分别利用颜色分布与对比度方法构建显著图,并且在显著图融合时采用一种目标增强系数,提高了显著图的准确性。实验结果表明,本文算法能够检测出更符合视觉特性的显著区域,显著区域更加准确,适用于自然图像的显著性目标检测、目标分割或基于显著性分析的图像标注。     

Productive high-complexity 3D city modeling with point clouds collected from terrestrial LiDAR

To satisfy the needs of photo-realistic and ground-based representation of three-dimensional (3D) city models for a variety of applications, significant efforts have been made to automatically reconstruct detailed 3D building façades from terrestrial LiDAR data. Nonetheless, in real-world applications for high-quality 3D city modeling, three major problems are typically encountered: (1) very low productivity due to fully manual operation, (2) low geometric accuracy of 3D modeling resulting from the process of reducing original LiDAR data, and (3) system failure when importing huge LiDAR data to 3D drawing software. To overcome these limitations, the present study proposes a semi-automatic method entailing a plane component detection based on RANSAC segmentation, boundary tracing of the planar components, and manual drawing of details using the remaining, significantly reduced points. The proposed method was applied to point clouds of various buildings in a high-density area in Korea. In comparison with manual operation, the proposed method was proved to improve modeling productivity in the time-consumption aspect and to facilitate operators’ accurate object drawing. However, for additional automation and completeness of 3D modeling, further study is necessary. The proposed method requires a segmentation algorithm to heuristically determine parameters for the most desirable results as well as to detect curvilinear surfaces in modeling complex and curved façades.     

Generating a Visual Language of Performance-Driven Configurations for the Principal Façade of a Prototype Sustainable House

A performance-driven application of shape grammars is presented. A parametric shape grammar that generates a language of pattern designs for the principal façade of a prototype house—featuring a 5 × 20 matrix of electrochromic windows—based equally on performance and aesthetic criteria is described. The adjustment of the chromatism and light transmittance of each individual windowpane on the façade enables the adjustment of solar radiation at the house interior. The novel aspect of the grammar is that it encodes performance constraints of interior daylight illuminance and associates them to visual, symmetry principles of two-dimensional pattern generation. Twelve parametric rules account for the generation of the façade pattern language and five subclasses account for the symmetry of the patterns in the language.     

一种鲁棒的夜间图像显著性对象检测模型

基于人类视觉注意机制的显著性对象检测模型作为能主动感知图像中重要信息的有效方法,对探索视觉早期认知过程的大范围知觉信息组织具有重要意义.然而由于夜间图像具有低信噪比和低对比度特性,现有的视觉显著性对象检测模型在夜间场景中容易受到噪声干扰、弱纹理模糊等多方面因素的影响.有鉴于此,本文提出了一种基于区域协方差和全局搜索的夜间图像显著性对象检测方法.首先将输入图像分割为超像素块,并分别计算它们的协方差.然后使用超像素块协方差的差异性作为适应度函数,并结合全局搜索算法来优化各个超像素块的显著值.最后通过图扩散方法来精炼显著图结果.实验测试采用了5个公开图像数据集和1个夜间图像数据集,通过与11种目前主流的视觉显著性对象检测模型进行对比,综合评价了本文所提出模型的性能.     

Impacts of tree and building shades on the urban heat island: Combining remote sensing, 3D digital city and spatial regression approaches

The continued increase in average and extreme temperatures around the globe is expected to strike urban communities more harshly because of the urban heat island (UHI). Devising natural and design-based solutions to stem the rising heat has become an important urban planning issue. Recent studies have examined the impacts of 2D/3D urban land-use structures on land surface temperature (LST), but with little attention to the shades cast by 3D objects, such as buildings and trees. It is, however, known that shades are particularly relevant for controlling summertime temperatures. This study examines the role of urban shades created by trees and buildings, focusing on the effects of shade extent and location on LST mitigation. A realistic 3D digital representation of urban and suburban landscapes, combined with detailed 2D land cover information, is developed. Shadows projected on horizontal and vertical surfaces are obtained through GIS analysis, and then quantified as independent variables explaining LST variations over grids of varying sizes with spatial regression models. The estimation results show that the shades on different 3D surfaces, including building rooftops, sun-facing façades, not-sun-facing façades, and on 2D surfaces including roadways, other paved covers, and grass, have cooling effects of varying impact, showing that shades clearly modify the thermal effects of urban built-up surfaces. Tree canopy volume has distinct effects on LST via evapotranspiration. One of the estimated models is used, after validation, to simulate the LST impacts of neighborhood scenarios involving additional greening. The findings illustrate how urban planners can use the proposed methodology to design 3D land-use solutions for effective heat mitigation.     

Salient Region Detection by Modeling Distributions of Color and Orientation

We present a robust salient region detection framework based on the color and orientation distribution in images. The proposed framework consists of a color saliency framework and an orientation saliency framework. The color saliency framework detects salient regions based on the spatial distribution of the component colors in the image space and their remoteness in the color space. The dominant hues in the image are used to initialize an expectation-maximization (EM) algorithm to fit a Gaussian mixture model in the hue-saturation (H-S) space. The mixture of Gaussians framework in H-S space is used to compute the inter-cluster distance in the H-S domain as well as the relative spread among the corresponding colors in the spatial domain. Orientation saliency framework detects salient regions in images based on the global and local behavior of different orientations in the image. The oriented spectral information from the Fourier transform of the local patches in the image is used to obtain the local orientation histogram of the image. Salient regions are further detected by identifying spatially confined orientations and with the local patches that possess high orientation entropy contrast. The final saliency map is selected as either color saliency map or orientation saliency map by automatically identifying which of the maps leads to the correct identification of the salient region. The experiments are carried out on a large image database annotated with ldquoground-truthrdquo salient regions, provided by Microsoft Research Asia, which enables us to conduct robust objective level comparisons with other salient region detection algorithms.     

融合似物性前景对象与背景先验的图像显著性检测

为了在复杂背景图像中准确地提取出图像的显著区域,提出一种结合似物性前景对象与背景先验知识的图像显著性检测方法（OFOBP）。该方法首先对图像进行超像素分割,计算超像素颜色空间分布,得到初始显著图;利用似物性检测方法获取多个目标窗口,由窗口建立搜索区域,结合二值化的初始显著图优化目标窗口;再利用多窗口特征对超像素做前景对象预测,获取前景显著图;其次建立背景模板,计算稀疏重构误差获取背景先验图;最后融合两种显著图,得到最终显著检测结果。在公开数据集上与11种算法进行比较,本文算法能够较为准确地检测出显著区域,尤其是在复杂背景下对多个显著目标的检测,存在明显的优势。     

图像增强对显著性目标检测的影响研究

目的 雾霾、雨雪天气和水下等非理想环境因素会引起图像退化,导致出现低质图像,从而影响人类主观视觉感受及机器视觉应用任务的性能,因此,低质图像被利用之前进行图像增强成为惯常的预处理过程。然而,图像增强能否提高图像机器视觉应用任务的性能及影响程度等问题鲜有系统性研究。针对上述问题,本文以图像显著性目标检测这一机器视觉应用为例,研究图像增强对显著性目标检测性能的影响。方法 首先利用包括5种传统方法、6种深度学习方法等共11种典型图像增强方法对图像进行增强处理,然后利用8种典型的显著性目标检测方法对增强前后的图像分别进行显著性目标检测实验,并对比分析其结果。结果 实验表明,图像增强对低质图像显著性目标检测方法性能的促进作用不明显,某些增强方法甚至表现出负面影响,也存在同一增强方法对不同的显著性目标检测方法作用不同的现象。结论 图像增强对于显著性目标检测及其他的机器视觉应用的实际效果值得进一步研究,如何根据图像机器视觉应用的需求来选择和设计有效的增强方法需进一步探讨。     

融合显著物体检测的图像显示适配

目的 图像显著适配旨在自动调节图像尺寸,对图像内容进行非均匀缩放,以便在受限的展示空间内更好地保留显著物体。为了解决显示适配过程中显著物体部分扭曲的问题,提出一种基于显著物体检测的图像显示适配方法。方法 本文方法采用显著物体分割结果来替代显著性图,以改进显示适配结果。首先,采用显著性融合和传播的方法生成显著性图;接着,结合输入图像和显著性图,采用自适应三阈值方法实现显著物体分割;然后,以此为基础,生成输入图像的曲边网格表示;最后,通过对不同网格的非均匀缩放,生成符合目标尺寸的适配结果。结果 在面向图像显示适配的公开数据集RetargetMe上,将本文方法与现有的10种代表性显示适配方法的结果进行了人工评估和比较。本文方法可以有效地减少显著物体出现部分扭曲的现象,能在48.8%的图像上取得无明显缺陷的适配效果,比现有最好的方法提高了5%。结论 基于显著物体检测的图像显示适配方法有助于提高显示适配过程中对显著物体处理的一致性,减少由于显著物体部分扭曲而引起的明显人工处理痕迹,从而达到提升显示适配效果的目的。     

Perception of Visual Artifacts in Image‐Based Rendering of Façades

Image‐based rendering (IBR) techniques allow users to create interactive 3D visualizations of scenes by taking a few snapshots. However, despite substantial progress in the field, the main barrier to better quality and more efficient IBR visualizations are several types of common, visually objectionable artifacts. These occur when scene geometry is approximate or viewpoints differ from the original shots, leading to parallax distortions, blurring, ghosting and popping errors that detract from the appearance of the scene. We argue that a better understanding of the causes and perceptual impact of these artifacts is the key to improving IBR methods. In this study we present a series of psychophysical experiments in which we systematically map out the perception of artifacts in IBR visualizations of façades as a function of the most common causes. We separate artifacts into different classes and measure how they impact visual appearance as a function of the number of images available, the geometry of the scene and the viewpoint. The results reveal a number of counter‐intuitive effects in the perception of artifacts. We summarize our results in terms of practical guidelines for improving existing and future IBR techniques.