Mixed Reality-Based Visualization Interfaces for Architecture, Engineering, and Construction Industry

Varied computing devices and automated sensors will enable new human-computer interface paradigms for interacting with digitally managed project information. The writers therefore propose the development of Mixed Reality (MR)-based computer interfaces, and especially Augmented Reality systems, for the architecture, engineering, and construction industry and describe the technologies and principles for applying such computer interfaces to support all phases of the constructed facility project life cycle. An Augmented Reality computer-aided drawing prototype is described as an experimental platform to study the human factors issues in interacting with Augmented Reality three-dimensional digital design models. Two critical research needs are cited for realizing effective Augmented Reality systems: (1) human factors research for development of visualization tools to enhance design comprehension and support collaborative work, and (2) the development of a technology infrastructure for “augmented” control and inspection interfaces to directly access digital project plan and site information that may be spatially referenced and displayed in the field. Research should be geared to advance knowledge regarding spatial cognition, human–computer interfaces, and computer-mediated human-to-human interactions, and it should address application of MR to all phases of the project life cycle.     

Evaluation of Augmented Reality for Rapid Assessment of Earthquake-Induced Building Damage

This paper discusses the feasibility of using augmented reality (AR) to evaluate earthquake-induced building damage. In the proposed approach, previously stored building information is superimposed onto a real structure in AR. Structural damage can then be quantified by measuring and interpreting key differences between the real and augmented views of the facility. Proof-of-concept experiments were performed in conjunction with large-scale cyclic shear wall tests. In these, CAD images of the walls were superimposed onto the wall specimens. Then, as the wall specimens were deformed under applied loading, the horizontal drifts between the walls and the augmented images were computed using two different techniques and compared with actual wall drifts. The obtained results highlight the potential of using AR for rapid damage detection and indicate that the accuracy of structural displacements measured using AR is a direct function of the accuracy with which augmented images can be registered with the real world. The limitations of the technology, considerations for field implementation, and the potential for other related applications of AR are also discussed.     

Automated Generation of Operations Level Construction Animations in Outdoor Augmented Reality

Three-dimensional (3D) visualization is an effective tool for communicating, verifying, and validating the results of a simulated operation. Traditional visualization tools used for this purpose are typically based on the paradigm of virtual reality. Augmented reality (AR) is a relatively newer visualization paradigm whose engineering applications have been explored by a limited number of researchers. In this paper, the problem of generating smooth and continuous AR animations from the results of running discrete event simulation models and a general purpose methodology to overcome this challenge are discussed. The structure of an AR animation authoring language developed by the writers to create a logical link between a running simulation model and its corresponding 3D visualization in AR is described. In order to validate the functionality and effectiveness of the designed methods and animation language, an AR-based visualization application was developed and the designed algorithms were successfully tested using different simulation scenarios of varying visual and operational complexity.     

Integrating Robot Mapping and Augmented Building Simulation

This paper discusses a framework for integrated augmented reality (AR) architecture for indoor thermal performance data visualization that utilizes a mobile robot to generate environment maps. It consists of three modules: robot mapping, computational fluid dynamics (CFD) simulation, and AR visualization. The robot mapping module enables the modeling of spatial geometry using a mobile robot. In order to generate steady approximations to scanned three-dimensional data sets, the paper presents a novel “split-and-merge expectation-maximization patch fitting” (SMEMPF) planar approximation method. It allows for precise adjustment of patches independent from the initial model. The final result is a set of patches identifying planar macrostructures that consist of a collection of supported tiles. These patches are used to model the spatial geometry under investigation. The CFD simulation module facilitates the prediction of building performance databased on the spatial data generated using the SMEMPF method. The AR visualization module assists in interactive and immersive visualization of CFD simulation results. Such an integrated AR architecture will facilitate rapid multiroom mobile AR visualizations.     

Georeferenced Registration of Construction Graphics in Mobile Outdoor Augmented Reality

This paper describes research that investigated the application of the global positioning system and 3 degree-of-freedom (3-DOF) angular tracking to address the registration problem during interactive visualization of construction graphics in outdoor augmented reality (AR) environments. The global position and the three-dimensional (3D) orientation of a user’s viewpoint are tracked, and this information is reconciled with the known global position and orientation of superimposed computer-aided design (CAD) objects. Based on this computation, the relative translation and axial rotations between the user’s viewpoint and the CAD objects are continually calculated. The relative geometric transformations are then applied to the CAD objects inside a virtual viewing frustum that is coincided with the real world space that is in the user’s view. The result is an augmented outdoor environment where superimposed graphical objects stay fixed to their real world locations as the user navigates. The algorithms are implemented in a software tool called UM-AR-GPS-ROVER that is capable of interactively placing static and dynamic 3D models at any location in outdoor augmented space. The concept and prototype are demonstrated with an example in which scheduled construction activities for the erection of a structural steel frame are graphically simulated in outdoor AR.     

CAD and Management of Construction Projects

The increasing interest in computer‐aided design (CAD) has prompted research that is aimed at identifying the opportunities for construction managers and building contractors. It has been found that the use of CAD systems in the U.K. is mainly confined to the production of detailed drawings. Indeed, most of the systems used are 2‐D drafting tools and incapable of supporting the integration of even modest amounts of nongraphical (construction) data. On the other hand, many 3‐D modeling systems have the potential to integrate construction data, although they appear to be almost ignored. The use of 3‐D modeling systems is considered to be the most suitable vehicle for successfully integrating these data. However, this is likely to necessitate the introduction of separate databases, preferably of the relational type. The use of 3‐D modeling systems in assessing the construction implications of outline designs also presents interesting possibilities and is discussed.     

Research in Visualization Techniques for Field Construction

Field construction can be planned, monitored, and controlled at two distinct levels: (1)?the activity or schedule level; and (2)?the operation or process level. Graphical three-dimensional (3D) visualization can serve as an effective communication method at both levels. Many research efforts in visualizing construction are rooted in scheduling. They typically involve linking activity-based construction schedules and 3D computer-aided design (CAD) models of facilities to describe discretely evolving construction product visualizations (often referred to as four-dimensional CAD). The focus is on communicating what components are built where and when, with the intention of studying the optimal activity sequence, spatial, and temporal interferences. The construction processes or operations actually involved in building the components are usually implied. A second approach in visualizing construction is rooted in discrete-event simulation that, in addition to visualizing evolving construction products, also concerns the visualization of the operations and processes that are performed in building them. In addition to what is built where and when, the approach communicates who builds it and how by depicting the interaction between involved machines, resources, and materials. This paper introduces the two approaches and describes the differences in concept, form, and content between activity level and operations level construction visualization. An example of a structural steel framing operation is presented to elucidate the comparison. This work was originally published in the proceedings of the 2002 IEEE Winter Simulation Conference. This paper expands on the original work by describing recent advances in both activity and operations level construction visualization.     

Distributed Augmented Reality for Visualizing Collaborative Construction Tasks

Augmented reality is a visualization method in which virtual objects are aligned with the real world and the viewer can interact with the virtual objects in real time. In this paper, a new methodology called distributed augmented reality for visualizing collaborative construction tasks (DARCC) is proposed. Using this methodology, virtual models of construction equipment can be operated and viewed by several operators to interactively simulate construction activities on the construction site in augmented reality mode. The paper investigates the design issues of DARCC including tracking and registration, object modeling, engineering constraints, and interaction and communication methods. The DARCC methodology is implemented in a prototype system and tested in a case study about a bridge deck rehabilitation project.     

Promoting Efficient Use of Visualization Tools through Education

Three-dimensional visualization is getting increasingly popular. However, for realizing the full benefits of information technology (IT), we need to go beyond visualization and adopt modeling techniques that allow reuse of information for other tasks in the project life cycle. Already it is possible to use visualization models for tasks such as simulation. However, widespread use of these integrated systems is hindered by the lack of education in the fundamentals of IT in the engineering curriculum. This paper analyzes the dependencies between concepts related to visualization and lists a minimum set of topics that should be part of the curriculum. Experiences in teaching these in a new course module related to visualization are described. Feedback data collected shows that most students appreciated the use of IT tools in education. However, comments by many students indicated inherent problems related to IT and visualization. Nevertheless, it is expected that when more universities start teaching fundamentals we should see more IT adoption in the industry.     

Visualization of Construction Progress Monitoring with 4D Simulation Model Overlaid on Time-Lapsed Photographs

The ability to effectively communicate progress information and represent as-built and as-planned progress discrepancies are identified as key components for successful project management that allow corrective decisions to be made in a timely manner. However, current formats of reporting (e.g., textual progress reports, progress curves, and photographs) may not properly and quickly communicate project progress. Current monitoring methods also require manual data collection and extensive data extraction from different construction documents, which distract managers from the important task of decision making. Therefore, to facilitate progress monitoring, this paper proposes visualization of performance metrics that aims to represent progress deviations through superimposition of four-dimensional (4D) as-planned model over time-lapsed photographs in single and comprehensive visual imagery. As a part of the developed system, registration of the 4D model with photographs, augmenting photographs, and occlusion removal for progress images are presented. While contextual information is preserved, the as-built photographs are enhanced and augmented with 4D as-planned model in which the performance metrics are visualized. The augmented photographs provide a consistent platform for representing as-planned, as-built, and progress discrepancies information and facilitate communication and reporting processes.     

Applicability of 4D CAD in Civil Engineering Construction: Case Study of a Cable-Stayed Bridge Project

Four-dimensional (4D) computer-aided design (CAD) has been credited with improving construction planning procedures. The integration of three-dimensional CAD with schedule information has enabled the effective detection of design and planning flaws in many construction projects. However, the benefit of 4D CAD has been centered on architectural constructions, as other areas such as civil infrastructure have seldom been the target of 4D CAD application. This paper presents a case study in which a cable-stayed bridge construction was analyzed and modeled using the 4D graphic simulation approach. The cable-stayed bridge was chosen for the case study because it suitably represents the complex nature of modern civil infrastructure. 4D CAD models were developed at three different levels of detail: activity, discrete operation, and continuous operation. The clear definitions of the three levels of detail of 4D CAD and their application results for the cable-stayed bridge are presented herein.     

Automated Recognition of 3D CAD Objects in Site Laser Scans for Project 3D Status Visualization and Performance Control

This paper presents a new approach that allows automated recognition of three-dimensional (3D) computer-aided design (CAD) objects from 3D site laser scans. This approach provides a robust and efficient means to recognize objects in a scene by integrating planning technologies, such as multidimensional CAD modeling, and field technologies, such as 3D laser scanning. Using such an approach, it would be possible to visualize the 3D status of a project and automate some tasks related to project control. These tasks include 3D progress tracking, productivity tracking, and construction dimensional quality assessment and quality control. This paper provides an overview of the developed approach and demonstrates its performance in object recognition and project 3D status visualization, with data collected from a construction job site.     

Diffusion of Computer Aided Design Technology in Architectural Design Practice

Computer aided design (CAD) technology is one of the most influential information technology (IT) innovations of the last four decades. This paper studies the factors that influence the spread of this important IT innovation in the context of the Turkish architectural design practice. It builds on the innovation diffusion theory which proposes that internal (i.e., copying behaviors of others) and external influence (i.e., complying with clients’ requirements, changes in government regulations, demand conditions, and consulting firms’ suggestions) factors drive diffusion of an innovation. The paper empirically tests the propositions of innovation diffusion theory by using three mathematical models: The internal influence model, the external influence model, and the mixed influence model. Research findings point out that the mixed influence model has the highest exploratory power. They show that the diffusion of CAD technology in architectural design practice is primarily driven by internal rather than external influence factors. This study is of importance to researchers because this is the first application of the influence models to the study of the diffusion of CAD technology in architectural design practice. It is also of relevance to design practitioners since the findings should provide a useful guide in their decision to adopt or not to adopt CAD technology.     

Visualization of Work Flow to Support Lean Construction

Implementation of advanced production management techniques, such as lean construction concepts like filtering of work packages to stabilize work flows, pull flow of teams and materials, and in-process quality control, demands effective and timely flows of information both to and from the workface. The key requirement—making the process state transparent to all participants—is more difficult to achieve in construction than in manufacturing, because work crews move continuously within a physical environment that is itself changing. Novel computer-aided visualization tools can fulfill the needs that simpler tools, such as Kanban cards, fulfill in manufacturing. Two prototypes with user interfaces designed to facilitate process flow have been devised and implemented within the context of building information modeling (BIM) software systems. They demonstrate aspects of the synergy between BIM and lean construction. Given the dynamic and dispersed physical environments and the fractured contracting arrangements typical of construction, BIM-based visualization interfaces are important tools for providing process transparency.     

Construction Scheduling and Progress Control Using Geographical Information Systems

Traditional scheduling and progress control techniques such as bar charts and the critical path method (CPM) fail to provide information pertaining to the spatial aspects of a construction project. A system called PMS-GIS (Progress Monitoring System with Geographical Information Systems) was developed to represent construction progress not only in terms of a CPM schedule but also in terms of a graphical representation of the construction that is synchronized with the work schedule. In PMS-GIS, the architectural design is executed using a computer-aided drafting (CAD) program (AutoCAD), the work schedule is generated using a project management software (P3), the design and schedule information (including percent complete information) are plugged into a GIS package (ArcViewGIS), and for every update, the system produces a CPM-generated bar chart alongside a 3D rendering of the project marked for progress. The GIS-based system developed in this study helps to effectively communicate the schedule∕progress information to the parties involved in the project, because they will be able to see in detail the spatial aspects of the project alongside the schedule.     

Development of Computerized Specifications

This paper describes the history, development, implementation, and benefits of a computerized master specification system established in a consulting engineering∕architecture company. Besides reviewing the problems encountered in developing such a system, the paper quantifies computer costs and word processing labor hours expended in preparing technical specifications for several projects betwen 1980 and 1983. It compares these costs and labor hours to those expended using the traditional system of preparing specifications. The computerized specification system allows engineers to prepare technical specifications in one‐third to one‐half the time it takes using the traditional methods. Word processing or typing productivity increases by a factor of three to six.     

Graphical 3D Visualization of Highway Bridge Ratings

Aging highway infrastructure requires effective rating methodologies to prioritize bridges for rehabilitation and repair. To aid engineers in decision making regarding bridge maintenance, a three-dimensional (3D) visualization system is developed for rating reinforced concrete deck-girder bridge. Color codings show the most probable mode of failure for girder cross sections under combined moment-shear forces and allow an engineer to determine a rehabilitation strategy. The visualization system relies on 3D finite-element analyses using the open source framework OpenSees, making the system readily extensible to a wide range of bridge types and loading scenarios, as well as emergent reliability-based rating methodologies. Important features of the visualization system are emphasized, including the use of lighting and feature edge detection to improve the visual quality of a bridge model. Recent developments in scientific visualization are discussed for potential application to civil engineering problems.     

Interactive 3D CAD for Effective Derrick Crane Operation in a Cable-Stayed Bridge Construction

The high expectation of esthetic and functional quality in modern civil infrastructure has resulted in the increased demand for long span bridges. In advanced or developing countries, long span bridges such as cable-stayed and suspension bridges are considered even as landmarks that symbolize the prosperity or culture of the region. These long span bridges require higher level of design and construction technologies than other types of bridges. In particular, the construction of cable-stayed bridges involves precise and sophisticated operation of construction equipment such as derrick cranes. However, it is not easy to plan the operations of a derrick crane before the actual construction process takes place. Unexpected spatial constraints in the construction site may hinder the smooth operation of a derrick crane, which leads to lower than expected productivity and safety. This study applies interactive three-dimensional (3D) computer aided design (CAD) to the derrick crane operation for the purpose of identifying potential problems. Construction managers can have the two way process with the 3D CAD system to interactively test their construction plans and scenarios. The case study shows that the interactive 3D CAD system significantly improves the constructability of the cable-stayed bridge construction.     

Use of Tangible and Augmented Reality Models in Engineering Graphics Courses

Engineering graphics courses are typically a requirement for engineering students around the world. Besides understanding and depicting graphic representation of engineering objects, the goal of these courses is to provide students with an understanding of the relationship between three-dimensional (3D) objects and their projections. However, in the classroom, where time is limited, it is very difficult to explain 3D geometry using only drawings on paper or at the blackboard. The research presented herein aims to develop two teaching aids; a tangible model and an augmented reality (AR) model, to help students better understand the relationship between 3D objects and their projections. Tangible models refer to the physical objects which are comprised of a set of differently shaped pieces. The tangible model we developed includes eight wooden blocks that include all the main geometrical features with respect to their 3D projections. The AR models are the virtual models which can superimpose 3D graphics of typical geometries on real-time video and dynamically vary view perspective in real-time to be seen as real objects. The AR model was developed using the ARToolKitPlus library and includes all the geometrical features generally taught in engineering graphics courses or technical drawing courses. To verify the effectiveness and applicability of the models we developed, we conducted a user test on 35 engineering-major students. The statistical results indicated that the tangible model significantly increased the learning performance of students in their abilities to transfer 3D objects onto two-dimensional (2D) projections. Students also demonstrated higher engagement with the AR model during the learning process. Compared to using the screen-based orthogonal and pictorial images, the tangible model and augmented reality model were evaluated to be more effective teaching aids for engineering graphics courses.