Enhancing Urban Façades via LiDAR‐Based Sculpting

Buildings with symmetrical façades are ubiquitous in urban landscapes and detailed models of these buildings enhance the visual realism of digital urban scenes. However, a vast majority of the existing urban building models in web‐based 3D maps such as Google earth are either less detailed or heavily rely on texturing to render the details. We present a new framework for enhancing the details of such coarse models, using the geometry and symmetry inferred from the light detection and ranging (LiDAR) scans and 2D templates. The user‐defined 2D templates, referred to as coded planar meshes (CPMs), encodes the geometry of the smallest repeating 3D structures of the façades via face codes. Our encoding scheme, take into account the directions, type as well as the offset distance of the sculpting to be applied at the respective locations on the coarse model. In our approach, LiDAR scan is registered with the coarse models taken from Google earth 3D or Bing maps 3D and decomposed into dominant planar segments (each representing the frontal or lateral walls of the building). The façade segments are then split into horizontal and vertical tiles using a weighted point count function defined over the window or door boundaries. This is followed by an automatic identification of CPM locations with the help of a template fitting algorithm that respects the alignment regularity as well as the inter‐element spacing on the façade layout. Finally, 3D boolean sculpting operations are applied over the boxes induced by CPMs and the coarse model, and a detailed 3D model is generated. The proposed framework is capable of modelling details even with occluded scans and enhances not only the frontal façades (facing to the streets) but also the lateral façades of the buildings. We demonstrate the potentials of the proposed framework by providing several examples of enhanced Google earth models and highlight the advantages of our method when designing photo‐realistic urban façades.     

Interaction with Media Façades

Media façades are a prominent example of the digital augmentation of urban spaces. They denote the concept of turning the surface of a building into a large-scale urban screen. Because of their enormous size, they require interaction at a distance and they have a high level of visibility. Additionally, they are situated in a highly dynamic urban environment with rapidly changing conditions, which results in settings which are neither comparable, nor reproducible. Altogether, this makes the development of interactive media façade installations a challenging task.     

Automatic façade recovery from single nighttime image

Nighttime images are difficult to process due to insufficient brightness,lots of noise,and lack of details.Therefore,they are always removed from time-lapsed image analysis.It is interesting that nighttime images have a unique and wonderful building features that have robust and salient lighting cues from human activities.Lighting variation depicts both the statistical and individual habitation,and it has an inherent man-made repetitive structure from architectural theory.Inspired by this,we propose an automatic nighttime fa?ade recovery method that exploits the lattice structures of window lighting.First,a simple but efficient classification method is employed to determine the salient bright regions,which may be lit windows.Then we groupwindows into multiple lattice proposals with respect to fa?ades by patch matching,followed by greedily removing overlapping lattices.Using the horizon constraint,we solve the ambiguous proposals problem and obtain the correct orientation.Finally,we complete the generated fa?ades by filling in the missing windows.This method is well suited for use in urban environments,and the results can be used as a good single-view compensation method for daytime images.The method also acts as a semantic input to other learning-based 3D image reconstruction techniques.The experiment demonstrates that our method works well in nighttime image datasets,and we obtain a high lattice detection rate of 82.1%of 82 challenging images with a low mean orientation error of 12.1±4.5 degrees.     

Building renovation adopts mass customization

This work is motivated by an industrial need of manufacturing façades insulating envelopes in order to reduce energy consumption in residential buildings. An insulating envelope is a configuration of a set of rectangular panels that respects a set of limitations. Due to the number of façades to be renovated and the number of possible configurations for a single façade, the envelope configuration is both a mass customization problem as well as a combinatorial one. The paper then introduces a decision support system based on the framework of constraint satisfaction, as it fits neatly the constrained nature of the problem. Two configuration tasks have been identified as prerequisite to envelopes configurations: (1) the configuration of a questionnaire for information inputs and (2) the configuration of a constraint satisfaction problem for each one of the façades to be renovated. The system architecture promotes maintenance, modularity and efficiency as different configuration tasks are divided into web-services. Conception and implementation of the massive building thermal renovation are then supported.     

A new method of data transformation for satellite images: I. Methodology and transformation equations for TM images

These two papers deal with a new method of data transformation. By analysing grey level curves (broken lines) of various ground features in image bands of different satellites, we have found that, inherent in 3‐ or 4‐band satellite images (SPOT, IKONOS, Quick Bird, OrbView, FORMOSAT and MSS) there are three basic remote‐sensing characteristics as follows: (1) the general radiance level L; (2) the visible–infrared radiation balance B; and (3) the band radiance variation vector (direction and speed) V. However, inherent in 5‐ or 7‐band satellite images (NOAA, TM), besides the above three, there is an extra basic remote‐sensing characteristic, i.e. the thermal radiation intensity I. This is denoted by thermal bands, i.e. the TM band 6 or NOAA (AVHRR) bands 4 and 5, which are relatively independent and can be used directly, and hence are unnecessary for information extraction or data transformation. Therefore, the data transformation only lies in extracting the L, B and V from original satellite images. Furthermore, we have also found that there are three basic ground‐cover elements on the Earth's surface: i.e. the bare land (in a broad sense), the vegetation and the water body, which, in different proportions, constitute all ground cover. Moreover, there are three basic (primitive) colours on colour image (including colour composite of satellite images): i.e. red, green and blue, which generate all colours on the colour image. Further research has revealed that the three basic remote‐sensing characteristics, the three basic ground‐cover elements and the three basic colours on the composite can conceptually constitute a three‐to‐three corresponding regular triangle scheme. Perhaps a good method of data transformation should make the scheme realistic, i.e. make the three ‘threes’ all correspond to each other. The research presented here has completed this task by regression calculations and selection of specific variables. First, the methodology and transformation equations for TM images are discussed. The transformed L, B and V images have relatively independent and equally distributed information as well as clear and definite physical, mathematical and geographical significance. They can be used effectively for generating high‐quality colour composites, on which the red, green, blue, yellow, pink, cyan and other colours of various kinds are all generated and all pure, saturated, equilibrated, meaning‐definite and close to the colours of ground features in nature. As a result, interpretations and discriminations of ground features can be easier and conducted not only by experience, but also by logic. The L, B and V images can also be used effectively for classification and digital analysis of ground features. With regard to the transformation equations for SPOT, NOAA, IKONOS, Quick Bird, OrbView, FORMOSAT and MSS images and the method application will be dealt with in the second paper.     

Geometry and Bonding Rules Position Analysis in the Formation of Traditional Iranian Architectural Brick Facades

Over time, architectural ornaments have had a variety of forms that are created from a combination of different types of materials and factors. Brick decorations are one such example, a type of ornament that is often formed on the exterior façade of buildings. Investigating covert rules and brick bonds geometry can reveal a part of the ancient architectural secrets and ways used to reduce the ravages upon new brick façades. Shaped bricks in traditional Iranian architecture directly affect the placement, size and brick arrangement rules. This article is the first attempt to study the effects of geometry and the principal rules of arrangement that help to shape the formation of brick façades specifically, as well as analysing the samples of brick façades taken from monuments, field recordings and discussion with the workmen themselves. The analysis has revealed that permanent rules govern the bonding of brick façades, along with the size and dimensions of the brick effects, which shape the final look of the decoration.     

A new method of data transformation for satellite images: II. Transformation equations for SPOT,NOAA, IKONOS,Quick Bird,ASTER, MSS and other images and application

By applying the same principles and techniques as described in the first paper, the data transformation equations for SPOT, NOAA, IKONOS, Quick Bird ASTER, and MSS are discussed and presented here. As in TM images, the general radiance level L, the visible–infrared radiation balance B and the band radiance variation vector (direction and speed) V images are all extracted from the above satellite images. All transformed black and white L, B and V images and their colour composites have their universality and individuality. The universality is that the transformed images accord with the three‐to‐three corresponding regular triangle scheme described in paper I and that the colours on their colour composite are pure, saturated, plentiful, brilliant, information‐equilibrated, meaning‐definite and close to colours of ground features in nature. The individuality is that the transformed images all have their own regional and seasonal characteristics. The universality is only determined by the coefficients of transformation equations and not dependent upon the original gray level values of image scene in any region and any season. Meanwhile the individuality is only determined by the preservation of regional and seasonal characteristics contained in the original gray level values of concrete image scenes. This paper also deals with the application of the transformed images in image enhancement, ground features interpretation, classification and digital analysis. In addition, some application examples are also offered. Finally the peculiarities and some shortcomings of this method are also summarized.     

Automatic generation of fabrication drawings for façade mullions and transoms through BIM models

Fabrication drawings are essential for manufacturing, design evaluation and inspection of building components, especially for building façade structural components. In order to clearly represent the physical characteristics of the façade structural components, a large number of section views need to be produced, which is very time-consuming and labor intensive. Therefore, automatic generation of fabrication drawings for building façade components (such as mullions and transoms) is of paramount importance. In this paper, attempts have been made to develop an efficient framework in order to automatically generate fabrication drawings for building façade structural components, including mullions and transoms. To represent the complex physical characteristics (such as holes and notches) on mullions and transoms using minimum number of drawing views, a computational algorithm based on graph theory is developed to eliminate duplicated section views. Another methodology regarding the generation of breaks for top views is also proposed to further improve the quality of drawing layouts. The obtained drawing views are then automatically arranged using a developed approach. In addition, primary dimensions of the drawing views focusing on the physical features are also generated. Furthermore, in order to maintain the consistency of drawing formats across multiple drawings, a methodology is proposed to determine the scaling factors of the drawings by using clustering technique. In an illustrative example, the proposed framework is used to generate the fabrication drawings for a typical BIM model containing façade structural components, and saving in time is observed.     

Vertical motion control of building façade maintenance robot with built-in guide rail

Recently, the number of high-rise building has increased along with the development of technology to cope with the increase in population. Because of this, many researches on an automatic building façade maintenance system have been conducted to satisfy the increasing demands of façade maintenance. However, most researches have focused on the mechanism and system composition, while working safety issues have not been sufficiently dealt with. This paper deals with the motion control issues of the building façade maintenance robot system which is composed of a vertical robot and a horizontal robot moving along the rail of the façade. With consideration for the vertical robot, these issues include the safety of docking process and the stability of vertical motion. During the docking process for the inter-floor circulation of the horizontal robot, shocks and positioning errors are generated due to increasing load. To solve this, the rail brake system is operated to suppress the shock during the docking process, and a re-leveling process is conducted to compensate the gap which is equal to the positioning error between the built-in transom rail of the robot and the transom rail of the building. In addition, many noises are generated from the surroundings that significantly affect the motion of the vertical robot due to vibration. To enhance the motion stability of the vertical robot, vibration suppression control is developed in this paper, using the state estimation which considers the dynamic properties of the wire rope. For the feasibility of this algorithm, the field experiment of the building façade maintenance robot is conducted.     

Generating a Visual Language of Performance-Driven Configurations for the Principal Façade of a Prototype Sustainable House

A performance-driven application of shape grammars is presented. A parametric shape grammar that generates a language of pattern designs for the principal façade of a prototype house—featuring a 5 × 20 matrix of electrochromic windows—based equally on performance and aesthetic criteria is described. The adjustment of the chromatism and light transmittance of each individual windowpane on the façade enables the adjustment of solar radiation at the house interior. The novel aspect of the grammar is that it encodes performance constraints of interior daylight illuminance and associates them to visual, symmetry principles of two-dimensional pattern generation. Twelve parametric rules account for the generation of the façade pattern language and five subclasses account for the symmetry of the patterns in the language.     

Perception of Visual Artifacts in Image‐Based Rendering of Façades

Image‐based rendering (IBR) techniques allow users to create interactive 3D visualizations of scenes by taking a few snapshots. However, despite substantial progress in the field, the main barrier to better quality and more efficient IBR visualizations are several types of common, visually objectionable artifacts. These occur when scene geometry is approximate or viewpoints differ from the original shots, leading to parallax distortions, blurring, ghosting and popping errors that detract from the appearance of the scene. We argue that a better understanding of the causes and perceptual impact of these artifacts is the key to improving IBR methods. In this study we present a series of psychophysical experiments in which we systematically map out the perception of artifacts in IBR visualizations of façades as a function of the most common causes. We separate artifacts into different classes and measure how they impact visual appearance as a function of the number of images available, the geometry of the scene and the viewpoint. The results reveal a number of counter‐intuitive effects in the perception of artifacts. We summarize our results in terms of practical guidelines for improving existing and future IBR techniques.     

A Neglected Harbinger of the Triple-Storey Façade of Guarini’s Santissima Annunziata in Messina

The triple-storey façade, one of the most original inventions of the Baroque period in terms of form and proportion, arose in Sicily and quickly spread throughout Italy, to Europe and beyond. Guarini’s design for the façade of Santissima Annunziata in Messina paved the way for its general acceptance. The roots for the concept, however, may be found in the work of Giacomo Del Duca.     

Channel invariant online visibility enhancement for visual SLAM in a turbid environment

This paper presents a real‐time and channel‐invariant visibility enhancement algorithm using a hybrid image enhancement approach. The proposed method is initially motivated by an underwater visual simultaneous localization and mapping (SLAM) failure in a turbid medium. The environments studied contain various particles and are dominated by a different image degradation model. Targeting image enhancement for degraded images but not being limited to it, the proposed method provides a highly effective solution for both color and gray images with substantial improvement in the process time compared to conventional methods. The proposed method introduces a hybrid scheme of two image enhancement modules: a model‐based (extensive) enhancement and a model‐free (immediate) enhancement. The proposed method is validated by using simulated synthetic color images and real‐world color and grayscale underwater images. Real‐world validation is performed in various environments such as hazy indoor, smoky indoor, and underwater. Using the ground truth trajectory or clear images acquired from the same area but without turbidity, we evaluate the proposed visibility enhancement and camera registration improvement for a feature based (ORB‐SLAM2), a direct (LSD‐SLAM), and a visual underwater SLAM application.     

Generative Methods for Long-Term Place Recognition in Dynamic Scenes

This paper proposes a new framework for visual place recognition that incrementally learns models of each place and offers adaptability to dynamic elements in the scene. Traditional Bag-Of-Words (BOW) image-retrieval approaches to place recognition typically treat images in a holistic manner and are not capable of dealing with sub-scene dynamics, such as structural changes to a building façade or seasonal effects on foliage. However, by treating local features as observations of real-world landmarks in a scene that is observed repeatedly over a period of time, such dynamics can be modelled at a local level, and the spatio-temporal properties of each landmark can be independently updated incrementally. The method proposed models each place as a set of such landmarks and their geometric relationships. A new BOW filtering stage and geometric verification scheme are introduced to compute a similarity score between a query image and each scene model. As further training images are acquired for each place, the landmark properties are updated over time and in the long term, the model can adapt to dynamic behaviour in the scene. Results on an outdoor dataset of images captured along a 7 km path, over a period of 5 months, show an improvement in recognition performance when compared to state-of-the-art image retrieval approaches to place recognition.     

Land cover change detection with a cross‐correlogram spectral matching algorithm

Timely extraction of reliable land cover change information is increasingly needed at a wide continuum of scales. Few methods developed from previous studies have proved to be robust when noise, changes in atmospheric and illumination conditions, and other scene‐ and sensor‐dependent variables are present in the multitemporal images. In this study, we developed a new method based on cross‐correlogram spectral matching (CCSM) with the aim of identifying interannual land cover changes from time‐series Normalized Difference Vegetation Index (NDVI) data. In addition, a new change index is proposed with integration of two parameters that are measured from the cross‐correlogram: the root mean square (RMS) and (1?R _max), where R _max is the maximum correlation coefficient in a correlogram. Subsequently, a method was proposed to derive the optimal threshold for judging ‘change’ or ‘non‐change’ with the acquired change index. A pilot study was carried out using SPOT VGT‐S images acquired in 1998 and 2000 at Xianghai Park in Jilin Province. The results indicate that CCSM is superior to a traditional Change Vector Analysis (CVA) when noise is present with the data. Because of an error associated with the ground truthing data, a more comprehensive assessment of the developed method is still in process using better ground truthing data and images at a larger time interval. It is worth noting that this method can be applied not only to NDVI datasets but also to other index datasets reflecting surface conditions sampled at different time intervals. In addition, it can be applied to datasets from different satellites without the need to normalize sensor differences.     

Progressive Virtual Beam Lights

A recent technique that forms virtual ray lights (VRLs) from path segments in media, reduces the artifacts common to VPL approaches in participating media, however, distracting singularities still remain. We present Virtual Beam Lights (VBLs), a progressive many‐lights algorithm for rendering complex indirect transport paths in, from, and to media. VBLs are efficient and can handle heterogeneous media, anisotropic scattering, and moderately glossy surfaces, while provably converging to ground truth. We inflate ray lights into beam lights with finite thicknesses to eliminate the remaining singularities. Furthermore, we devise several practical schemes for importance sampling the various transport contributions between camera rays, light rays, and surface points. VBLs produce artifact‐free images faster than VRLs, especially when glossy surfaces and/or anisotropic phase functions are present. Lastly, we employ a progressive thickness reduction scheme for VBLs in order to render results that converge to ground truth.     

Procedural Modeling of a Building from a Single Image

Creating a virtual city is demanded for computer games, movies, and urban planning, but it takes a lot of time to create numerous 3D building models. Procedural modeling has become popular in recent years to overcome this issue, but creating a grammar to get a desired output is difficult and time consuming even for expert users. In this paper, we present an interactive tool that allows users to automatically generate such a grammar from a single image of a building. The user selects a photograph and highlights the silhouette of the target building as input to our method. Our pipeline automatically generates the building components, from large‐scale building mass to fine‐scale windows and doors geometry. Each stage of our pipeline combines convolutional neural networks (CNNs) and optimization to select and parameterize procedural grammars that reproduce the building elements of the picture. In the first stage, our method jointly estimates camera parameters and building mass shape. Once known, the building mass enables the rectification of the façades, which are given as input to the second stage that recovers the façade layout. This layout allows us to extract individual windows and doors that are subsequently fed to the last stage of the pipeline that selects procedural grammars for windows and doors. Finally, the grammars are combined to generate a complete procedural building as output. We devise a common methodology to make each stage of this pipeline tractable. This methodology consists in simplifying the input image to match the visual appearance of synthetic training data, and in using optimization to refine the parameters estimated by CNNs. We used our method to generate a variety of procedural models of buildings from existing photographs.