Robot System of DRC‐HUBO+ and Control Strategy of Team KAIST in DARPA Robotics Challenge Finals

This paper summarizes how Team KAIST prepared for the DARPA Robotics Challenge (DRC) Finals, especially in terms of the robot system and control strategy. To imitate the Fukushima nuclear disaster situation, the DRC performed eight tasks and degraded communication conditions. This competition demanded various robotic technologies, such as manipulation, mobility, telemetry, autonomy, and localization. Their systematic integration and the overall system robustness were also important issues in completing the challenge. In this sense, this paper presents a hardware and software system for the DRC‐HUBO+, a humanoid robot that was used for the DRC; it also presents control methods, such as inverse kinematics, compliance control, a walking algorithm, and a vision algorithm, all of which were implemented to accomplish the tasks. The strategies and operations for each task are briefly explained with vision algorithms. This paper summarizes what we learned from the DRC before the conclusion. In the competition, 25 international teams participated with their various robot platforms. We competed in this challenge using the DRC‐HUBO+ and won first place in the competition.     

Team THOR's Entry in the DARPA Robotics Challenge Finals 2015

This paper describes Team THOR's approach to human‐in‐the‐loop disaster response robotics for the 2015 DARPA Robotics Challenge (DRC) Finals. Under the duress of unpredictable networking and terrain, fluid operator interactions and dynamic disturbance rejection become major concerns for effective teleoperation. We present a humanoid robot designed to effectively traverse a disaster environment while allowing for a wide range of manipulation abilities. To complement the robot hardware, a hierarchical software foundation implements network strategies that provide real‐time feedback to an operator under restricted bandwidth using layered user interfaces. Our strategy for humanoid locomotion includes a backward‐facing knee configuration paired with specialized toe and heel lifting strategies that allow the robot to traverse difficult surfaces while rejecting external perturbations. With an upper body planner that encodes operator preferences, predictable motion plans are executed in unforeseen circumstances. These plans are critical for manipulation in unknown environments. Our approach was validated during the DRC Finals competition, where Team THOR scored three points in 18 min of operation time, and the results are presented along with an analysis of each task.     

Team VALOR's ESCHER: A Novel Electromechanical Biped for the DARPA Robotics Challenge

The Electric Series Compliant Humanoid for Emergency Response (ESCHER) platform represents the culmination of four years of development at Virginia Tech to produce a full‐sized force‐controlled humanoid robot capable of operating in unstructured environments. ESCHER's locomotion capability was demonstrated at the DARPA Robotics Challenge (DRC) Finals when it successfully navigated the 61 m loose dirt course. Team VALOR, a Track A team, developed ESCHER leveraging and improving upon bipedal humanoid technologies implemented in previous research efforts, specifically for traversing uneven terrain and sustained untethered operation. This paper presents the hardware platform, software, and control systems developed to field ESCHER at the DRC Finals. ESCHER's unique features include custom linear series elastic actuators in both single and dual actuator configurations and a whole‐body control framework supporting compliant locomotion across variable and shifting terrain. A high‐level software system designed using the robot operating system integrated various open‐source packages and interfaced with the existing whole‐body motion controller. The paper discusses a detailed analysis of challenges encountered during the competition, along with lessons learned that are critical for transitioning research contributions to a fielded robot. Empirical data collected before, during, and after the DRC Finals validate ESCHER's performance in fielded environments.     

An Architecture for Human‐Guided Autonomy: Team TROOPER at the DARPA Robotics Challenge Finals

Recent robotics efforts have automated simple, repetitive tasks to increase execution speed and lessen an operator's cognitive load, allowing them to focus on higher‐level objectives. However, an autonomous system will eventually encounter something unexpected, and if this exceeds the tolerance of automated solutions, there must be a way to fall back to teleoperation. Our solution is a largely autonomous system with the ability to determine when it is necessary to ask a human operator for guidance. We call this approach human‐guided autonomy. Our design emphasizes human‐on‐the‐loop control where an operator expresses a desired high‐level goal for which the reasoning component assembles an appropriate chain of subtasks. We introduce our work in the context of the DARPA Robotics Challenge (DRC) Finals. We describe the software architecture Team TROOPER developed and used to control an Atlas humanoid robot. We employ perception, planning, and control automation for execution of subtasks. If subtasks fail, or if changing environmental conditions invalidate the planned subtasks, the system automatically generates a new task chain. The operator is able to intervene at any stage of execution, to provide input and adjustment to any control layer, enabling operator involvement to increase as confidence in automation decreases. We present our performance at the DRC Finals and a discussion about lessons learned.     

WALK‐MAN: A High‐Performance Humanoid Platform for Realistic Environments

In this work, we present WALK‐MAN, a humanoid platform that has been developed to operate in realistic unstructured environment, and demonstrate new skills including powerful manipulation, robust balanced locomotion, high‐strength capabilities, and physical sturdiness. To enable these capabilities, WALK‐MAN design and actuation are based on the most recent advancements of series elastic actuator drives with unique performance features that differentiate the robot from previous state‐of‐the‐art compliant actuated robots. Physical interaction performance is benefited by both active and passive adaptation, thanks to WALK‐MAN actuation that combines customized high‐performance modules with tuned torque/velocity curves and transmission elasticity for high‐speed adaptation response and motion reactions to disturbances. WALK‐MAN design also includes innovative design optimization features that consider the selection of kinematic structure and the placement of the actuators with the body structure to maximize the robot performance. Physical robustness is ensured with the integration of elastic transmission, proprioceptive sensing, and control. The WALK‐MAN hardware was designed and built in 11 months, and the prototype of the robot was ready four months before DARPA Robotics Challenge (DRC) Finals. The motion generation of WALK‐MAN is based on the unified motion‐generation framework of whole‐body locomotion and manipulation (termed loco‐manipulation). WALK‐MAN is able to execute simple loco‐manipulation behaviors synthesized by combining different primitives defining the behavior of the center of gravity, the motion of the hands, legs, and head, the body attitude and posture, and the constrained body parts such as joint limits and contacts. The motion‐generation framework including the specific motion modules and software architecture is discussed in detail. A rich perception system allows the robot to perceive and generate 3D representations of the environment as well as detect contacts and sense physical interaction force and moments. The operator station that pilots use to control the robot provides a rich pilot interface with different control modes and a number of teleoperated or semiautonomous command features. The capability of the robot and the performance of the individual motion control and perception modules were validated during the DRC in which the robot was able to demonstrate exceptional physical resilience and execute some of the tasks during the competition.     

A distributed problem-solving approach to collaborative facility engineering

A framework for collaborative facility engineering is presented. The framework is based on a distributed problem-solving approach to collaborative facility engineering and employs an integration approach called Agent-Based Software Engineering as an implementation vehicle of this approach. The focal entity of this framework is a Multiagent Design Team (MDT) that comprises a collection of software agents (e.g. design software applications with a certain standard communication interface) and a design specialist, which together perform specific design tasks. Multiagent design teams are autonomous and form an organizational structure based on a federation architecture. Every multiagent design team surrenders its autonomy to a system program called facilitator, which coordinates the interaction among software agents in the federation architecture. Facilitators can be viewed as representatives of one or more teams that facilitate the exchange of design information and knowledge in support of the design tasks they perform. In the federation architecture, design specialists collaborate by exchanging design information with others via their software agents, and by identifying and resolving design conflicts by negotiation. In addition to a discussion of the framework's primary components, its realization in an integrated distributed environment for collaborative building engineering is described.     

HyperMoVal: Interactive Visual Validation of Regression Models for Real‐Time Simulation

During the development of car engines, regression models that are based on machine learning techniques are increasingly important for tasks which require a prediction of results in real‐time. While the validation of a model is a key part of its identification process, existing computation‐ or visualization‐based techniques do not adequately support all aspects of model validation. The main contribution of this paper is an interactive approach called HyperMoVal that is designed to support multiple tasks related to model validation: 1) comparing known and predicted results, 2) analyzing regions with a bad fit, 3) assessing the physical plausibility of models also outside regions covered by validation data, and 4) comparing multiple models. The key idea is to visually relate one or more n‐dimensional scalar functions to known validation data within a combined visualization. HyperMoVal lays out multiple 2D and 3D sub‐projections of the n‐dimensional function space around a focal point. We describe how linking HyperMoVal to other views further extends the possibilities for model validation. Based on this integration, we discuss steps towards supporting the entire workflow of identifying regression models. An evaluation illustrates a typical workflow in the application context of car‐engine design and reports general feedback of domain experts and users of our approach. These results indicate that our approach significantly accelerates the identification of regression models and increases the confidence in the overall engineering process.     

Distributed Coordination in Heterogeneous Multi-Robot Systems

Coordination in multi-robot systems is a very active research field in Artificial Intelligence and Robotics, since through coordination one can achieve a more effective execution of the robots' tasks. In this paper we present an approach to distributed coordination of a multi-robot system that is based on dynamic role assignment. The approach relies on the broadcast communication of utility functions that define the capability for every robot to perform a task and on the execution of a coordination protocol for dynamic role assignment. The presented method is robust to communication failures and suitable for application in dynamic environments. In addition to experimental results showing the effectiveness of our approach, the method has been successfully implemented within the team of heterogeneous robots Azzurra Robot Team in a very dynamic hostile environment provided by the RoboCup robotic soccer competitions.     

Acquisition, Synthesis, and Rendering of Bidirectional Texture Functions

One of the main challenges in computer graphics is still the realistic rendering of complex materials such as fabric or skin. The difficulty arises from the complex meso structure and reflectance behavior defining the unique look‐and‐feel of a material. A wide class of such realistic materials can be described as 2D‐texture under varying light‐ and view direction, namely, the Bidirectional Texture Function (BTF). Since an easy and general method for modeling BTFs is not available, current research concentrates on image‐based methods, which rely on measured BTFs (acquired real‐world data) in combination with appropriate synthesis methods. Recent results have shown that this approach greatly improves the visual quality of rendered surfaces and therefore the quality of applications such as virtual prototyping. This state‐of‐the‐art report (STAR) will present the techniques for the main tasks involved in producing photo‐realistic renderings using measured BTFs in details.     

Stability analysis of nonholonomic multiagent coordinate‐free formation control subject to communication delays

This paper investigates the convergence of nonholonomic multiagent coordinate‐free formation control to a prescribed target formation subject to communication delays by means of Lyapunov‐Krasovskii approach and smooth state‐feedback control laws. As a result, an iterative algorithm based on linear matrix inequalities is provided to obtain the worst‐case point‐to‐point delay under which the multiagent system is guaranteed to be stable. It is worth mentioning that: (i) the given algorithm holds for any connected communication topology and (ii) the formation control is coordinate‐free, that is, a common frame is not required to be shared between agents. The effectiveness of the given method is illustrated through simulation results.     

A framework for the development of videogames

A framework is a high‐level solution for the reuse of software pieces, a step forward in simple library‐based reuse, that allows the sharing of not only common functions but also the generic logic of a domain application. It also ensures a better level of quality for the final product, given the fact that an important fraction of the application is already found within the framework and has therefore already been tested. This case study takes the systematic generation of hot‐spot subsystems approach as a reference point to describe the underlying concepts in the design of a framework for the development of 2D action videogames for low‐performance machines. The main goal of this paper is to show the applicability of framework‐based reuse to videogames. Both standard and framework‐based game implementations are compared and the results are analysed. Special attention is paid to the (potential) benefits that the use of frameworks brings to the fulfillment of maintenance tasks along the game's life cycle, a stage that normally consumes most resources in software projects. At the end of the paper, based on the implementation results, this study shows the predicted conditions under which building a framework is cost effective for the development of videogames similar to the ones from the studied domain. Copyright © 2001 John Wiley & Sons, Ltd.     

Reliable Rolling‐guided Point Normal Filtering for Surface Texture Removal

Semantic surface decomposition (SSD) facilitates various geometry processing and product re‐design tasks. Filter‐based techniques are meaningful and widely used to achieve the SSD, which however often leads to surface either under‐fitting or over‐fitting. In this paper, we propose a reliable rolling‐guided point normal filtering method to decompose textures from a captured point cloud surface. Our method is built on the geometry assumption that 3D surfaces are comprised of an underlying shape (US) and a variety of bump ups and downs (BUDs) on the US. We have three core contributions. First, by considering the BUDs as surface textures, we present a RANSAC‐based sub‐neighborhood detection scheme to distinguish the US and the textures. Second, to better preserve the US (especially the prominent structures), we introduce a patch shift scheme to estimate the guidance normal for feeding the rolling‐guided filter. Third, we formulate a new position updating scheme to alleviate the common uneven distribution of points. Both visual and numerical experiments demonstrate that our method is comparable to state‐of‐the‐art methods in terms of the robustness of texture removal and the effectiveness of the underlying shape preservation.     

Interprocedural Compilation of Fortran D

Fortran D is a version of Fortran extended with data decomposition specifications. It is designed to provide a machine-independent programming model for data-parallel applications and has heavily influenced the design of High Performance Fortran (HPF). In previous work we described Fortran D compilation algorithms for individual procedures. This paper presents an interprocedural approach to analyze data and computation partitions, optimize communication, support dynamic data decomposition, and perform other tasks required to compile Fortran D programs. Our algorithms are designed to make interprocedural compilation efficient. First, we collect summary information after edits to solve important data-flow problems in a separate interprocedural propagation phase. Second, for nonrecursive programs we compile procedures in reverse topological order to propagate additional interprocedural information during code generation. We thus limit compilation to a single pass over each procedure body. We also perform optimizations across procedure boundaries by delaying instantiation of the computation partition, communication, and dynamic data decomposition. Empirical results show that interprocedural optimization is crucial in achieving acceptable performance for a common application code.     

Mobile robot motion estimation by 2D scan matching with genetic and iterative closest point algorithms

The paper reports on mobile robot motion estimation based on matching points from successive two‐dimensional (2D) laser scans. This ego‐motion approach is well suited to unstructured and dynamic environments because it directly uses raw laser points rather than extracted features. We have analyzed the application of two methods that are very different in essence: (i) A 2D version of iterative closest point (ICP), which is widely used for surface registration; (ii) a genetic algorithm (GA), which is a novel approach for this kind of problem. Their performance in terms of real‐time applicability and accuracy has been compared in outdoor experiments with nonstop motion under diverse realistic navigation conditions. Based on this analysis, we propose a hybrid GA‐ICP algorithm that combines the best characteristics of these pure methods. The experiments have been carried out with the tracked mobile robot Auriga‐α and an on‐board 2D laser scanner. © 2006 Wiley Periodicals, Inc.     

Metropolitan intelligent surveillance systems for urban areas by harnessing IoT and edge computing paradigms

Recent technological advances led to the rapid and uncontrolled proliferation of intelligent surveillance systems (ISSs), serving to supervise urban areas. Driven by pressing public safety and security requirements, modern cities are being transformed into tangled cyber‐physical environments, consisting of numerous heterogeneous ISSs under different administrative domains with low or no capabilities for reuse and interaction. This isolated pattern renders itself unsustainable in city‐wide scenarios that typically require to aggregate, manage, and process multiple video streams continuously generated by distributed ISS sources. A coordinated approach is therefore required to enable an interoperable ISS for metropolitan areas, facilitating technological sustainability to prevent network bandwidth saturation. To meet these requirements, this paper combines several approaches and technologies, namely the Internet of Things, cloud computing, edge computing and big data, into a common framework to enable a unified approach to implementing an ISS at an urban scale, thus paving the way for the metropolitan intelligent surveillance system (MISS). The proposed solution aims to push data management and processing tasks as close to data sources as possible, thus increasing performance and security levels that are usually critical to surveillance systems. To demonstrate the feasibility and the effectiveness of this approach, the paper presents a case study based on a distributed ISS scenario in a crowded urban area, implemented on clustered edge devices that are able to off‐load tasks in a “horizontal” manner in the context of the developed MISS framework. As demonstrated by the initial experiments, the MISS prototype is able to obtain face recognition results 8 times faster compared with the traditional off‐loading pattern, where processing tasks are pushed “vertically” to the cloud.     

PCPNet Learning Local Shape Properties from Raw Point Clouds

In this paper, we propose PCPNET , a deep‐learning based approach for estimating local 3D shape properties in point clouds. In contrast to the majority of prior techniques that concentrate on global or mid‐level attributes, e.g., for shape classification or semantic labeling, we suggest a patch‐based learning method, in which a series of local patches at multiple scales around each point is encoded in a structured manner. Our approach is especially well‐adapted for estimating local shape properties such as normals (both unoriented and oriented) and curvature from raw point clouds in the presence of strong noise and multi‐scale features. Our main contributions include both a novel multi‐scale variant of the recently proposed PointNet architecture with emphasis on local shape information, and a series of novel applications in which we demonstrate how learning from training data arising from well‐structured triangle meshes, and applying the trained model to noisy point clouds can produce superior results compared to specialized state‐of‐the‐art techniques. Finally, we demonstrate the utility of our approach in the context of shape reconstruction, by showing how it can be used to extract normal orientation information from point clouds.     

Reconstructing Animated Meshes from Time‐Varying Point Clouds

In this paper, we describe a novel approach for the reconstruction of animated meshes from a series of time‐deforming point clouds. Given a set of unordered point clouds that have been captured by a fast 3‐D scanner, our algorithm is able to compute coherent meshes which approximate the input data at arbitrary time instances. Our method is based on the computation of an implicit function in ?⁴ that approximates the time‐space surface of the time‐varying point cloud. We then use the four‐dimensional implicit function to reconstruct a polygonal model for the first time‐step. By sliding this template mesh along the time‐space surface in an as‐rigid‐as‐possible manner, we obtain reconstructions for further time‐steps which have the same connectivity as the previously extracted mesh while recovering rigid motion exactly. The resulting animated meshes allow accurate motion tracking of arbitrary points and are well suited for animation compression. We demonstrate the qualities of the proposed method by applying it to several data sets acquired by real‐time 3‐D scanners.