一种基于运动特征的快速有效的视频镜头全拼图方法

由于全拼图能比关键帧提供更多的视觉信息,因此它已经成为视觉计算中一个重要的分析工具。为了提高全拼图的质量和拼图速度,提出了一种基于运动特征的快速有效的全拼图生成方法。该方法首先给定一个视频镜头,并基于运动相位熵的分析方法决定该镜头内容是否适合生成全拼图;然后,对于适合生成全拼图的镜头,通过构造全局运动路径的方法,仅需要挑选全部视频帧的一个子集用来生成高质量的全拼图。实验结果表明,与传统的全拼图方法相比,该新方法在提高全拼图的视觉质量的同时,显著地降低了计算时间。     

Manifold Modeling and Its Application to Tubular Scene Manifold Mosaicing Algorithm

Manifold mosaicing is arguably the most important class of image mosaicing methods. The existing manifold mosaicing methods work reasonably well only for scenes with simple plane structure and for images taken in a usual way such that the camera??s motion direction is perpendicular to its optical axis. A novel manifold modeling theoretical framework is presented to unify the current image mosaicing methods. Based on this framework, an effective mosaicing algorithm is also proposed to mosaic tubular scenes which have failed most existing methods. We adopt a two-step shaping strategy to parallelize the optical flows and change the topological genus of the image strips by M?bius mappings and circular ring extending. We performed computational experiments via image sequences acquired from tubular scenes and obtained excellent panoramas. The theoretical framework and algorithms in this paper have immediate applications to many practical tubular scene mosaicing problems in medical imaging, industrial inspection, gaming, virtual reality and robotics.     

Fast topology estimation for image mosaicing using adaptive information thresholding

Over the past decade, several image mosaicing methods have been proposed in robotic mapping and remote sensing applications. Owing to rapid developments in obtaining optical data from areas beyond human reach, there is a high demand from different science fields for creating large-area image mosaics, often using images as the only source of information. One of the most important steps in the mosaicing process is motion estimation between overlapping images to obtain the topology, i.e., the spatial relationships between images.In this paper, we propose a generic framework for feature-based image mosaicing capable of obtaining the topology with a reduced number of matching attempts and of getting the best possible trajectory estimation. Innovative aspects include the use of a fast image similarity criterion combined with a Minimum Spanning Tree (MST) solution, to obtain a tentative topology and information theory principles to decide when to update trajectory estimation. Unlike previous approaches for large-area mosaicing, our framework is able to naturally deal with the cases where time-consecutive images cannot be matched successfully, such as completely unordered sets. This characteristic also makes our approach robust to sensor failure. The performance of the method is illustrated with experimental results obtained from different challenging underwater image sequences.     

EXPERIMENTS OF MORPHOGENESIS IN SWARMS OF SIMPLE MOBILE ROBOTS

In this paper, we focus on the problem of having a multitude of very simple mobile robots self-organize their relative positions so as to obtain a variety of spatial configurations. The problem has a variety of applications in mobile robotics, modular robots, sensor networks, and computational self-assembly. The approach we investigate in this paper attempts at minimizing the local capability of robots and at verifying how and to what extent a variety of global shapes can be obtained by exploiting simple self-organizing algorithms and emergent behaviors. Several experiments are reported showing the effectiveness of the approach.     

Efficient exploration of unknown indoor environments using a team of mobile robots

Whenever multiple robots have to solve a common task, they need to coordinate their actions to carry out the task efficiently and to avoid interferences between individual robots. This is especially the case when considering the problem of exploring an unknown environment with a team of mobile robots. To achieve efficient terrain coverage with the sensors of the robots, one first needs to identify unknown areas in the environment. Second, one has to assign target locations to the individual robots so that they gather new and relevant information about the environment with their sensors. This assignment should lead to a distribution of the robots over the environment in a way that they avoid redundant work and do not interfere with each other by, for example, blocking their paths. In this paper, we address the problem of efficiently coordinating a large team of mobile robots. To better distribute the robots over the environment and to avoid redundant work, we take into account the type of place a potential target is located in (e.g., a corridor or a room). This knowledge allows us to improve the distribution of robots over the environment compared to approaches lacking this capability. To autonomously determine the type of a place, we apply a classifier learned using the AdaBoost algorithm. The resulting classifier takes laser range data as input and is able to classify the current location with high accuracy. We additionally use a hidden Markov model to consider the spatial dependencies between nearby locations. Our approach to incorporate the information about the type of places in the assignment process has been implemented and tested in different environments. The experiments illustrate that our system effectively distributes the robots over the environment and allows them to accomplish their mission faster compared to approaches that ignore the place labels.     

Coordinating heterogeneous teams of robots using temporal symbolic planning

The efficient coordination of a team of heterogeneous robots is an important requirement for exploration, rescue, and disaster recovery missions. In this paper, we present a novel approach to target assignment for heterogeneous teams of robots. It goes beyond existing target assignment algorithms in that it explicitly takes symbolic actions into account. Such actions include the deployment and retrieval of other robots or manipulation tasks. Our method integrates a temporal planning approach with a traditional cost-based planner. The proposed approach was implemented and evaluated in two distinct settings. First, we coordinated teams of marsupial robots. Such robots are able to deploy and pickup smaller robots. Second, we simulated a disaster scenario where the task is to clear blockades and reach certain critical locations in the environment. A similar setting was also investigated using a team of real robots. The results show that our approach outperforms ad-hoc extensions of state-of-the-art cost-based coordination methods and that the approach is able to efficiently coordinate teams of heterogeneous robots and to consider symbolic actions.     

Tracking multiple moving targets with swarms of mobile robots

This paper presents a distributed approach to enable mobile robot swarms to track multiple targets moving unpredictably. The proposed approach consists of two constituent algorithms: local interaction and target tracking. When the robots are faster than the targets, Lyapunov theory can be applied to show that the robots converge asymptotically to each vertex of the desired equilateral triangular configurations while tracking the targets. Toward practical implementation of the algorithms, it is important to realize the observation capability of individual robots in an inexpensive and efficient way. A new proximity sensor that we call dual rotating infrared (DRIr) sensor is developed to meet these requirements. Both our simulation and experimental results employing the proposed algorithms and DRIr sensors confirm that the proposed distributed multi-target tracking method for a swarm of robots is effective and easy to implement.     

Multi-robot mobility enhanced hop-count based localization in ad hoc networks

The localization problem is important in mobile robots and wireless sensor network and has been studied for many years. Among many localization methods, the hop-count based approach is simple and scalable; however, the localization accuracy is not satisfactory if the node density is low. To solve this problem, in this paper a multi-robot approach is proposed to utilize the cooperation and mobility of the robots to improve the node distribution (density), thus enhancing the hop-count based localization. By an auction-based task allocation scheme, the robots can negotiate with the static sensor nodes and then select the most suitable robots to move to the area of sparse node density, thus increasing the localization accuracy for the static sensor nodes. On the other hand, the robots also can localize themselves with the help of the static sensor nodes. The efficacy of this approach is shown by simulation.     

Rectified Surface Mosaics

We approach mosaicing as a camera tracking problem within a known parameterized surface. From a video of a camera moving within a surface, we compute a mosaic representing the texture of that surface, flattened onto a planar image. Our approach works by defining a warp between images as a function of surface geometry and camera pose. Globally optimizing this warp to maximize alignment across all frames determines the camera trajectory, and the corresponding flattened mosaic image. In contrast to previous mosaicing methods which assume planar or distant scenes, or controlled camera motion, our approach enables mosaicing in cases where the camera moves unpredictably through proximal surfaces, such as in medical endoscopy applications.     

Generalized Mosaicing: High Dynamic Range in a Wide Field of View

We present an approach that significantly enhances the capabilities of traditional image mosaicking. The key observation is that as a camera moves, it senses each scene point multiple times. We rigidly attach to the camera an optical filter with spatially varying properties, so that multiple measurements are obtained for each scene point under different optical settings. Fusing the data captured in the multiple images yields an image mosaic that includes additional information about the scene. We refer to this approach as generalized mosaicing. In this paper we show that this approach can significantly extend the optical dynamic range of any given imaging system by exploiting vignetting effects. We derive the optimal vignetting configuration and implement it using an external filter with spatially varying transmittance. We also derive efficient scene sampling conditions as well as ways to self calibrate the vignetting effects. Maximum likelihood is used for image registration and fusion. In an experiment we mounted such a filter on a standard 8-bit video camera, to obtain an image panorama with dynamic range comparable to imaging with a 16-bit camera.     

Distributed multirobot exploration, mapping, and task allocation

We present an integrated approach to multirobot exploration, mapping and searching suitable for large teams of robots operating in unknown areas lacking an existing supporting communications infrastructure. We present a set of algorithms that have been both implemented and experimentally verified on teams—of what we refer to as Centibots—consisting of as many as 100 robots. The results that we present involve search tasks that can be divided into a mapping stage in which robots must jointly explore a large unknown area with the goal of generating a consistent map from the fragment, a search stage in which robots are deployed within the environment in order to systematically search for an object of interest, and a protection phase in which robots are distributed to track any intruders in the search area. During the first stage, the robots actively seek to verify their relative locations in order to ensure consistency when combining data into shared maps; they must also coordinate their exploration strategies so as to maximize the efficiency of exploration. In the second and third stages, robots allocate search tasks among themselves; since tasks are not defined a priori, the robots first produce a topological graph of the area of interest and then generate a set of tasks that reflect spatial and communication constraints. Our system was evaluated under extremely realistic real-world conditions. An outside evaluation team found the system to be highly efficient and robust.     

Distributed and resilient localization algorithm for Swarm Robotic Systems

Many applications of Swarm Robotic Systems (SRSs) require each robot to be able to discover its own position. To provide such capability, some localization methods have been proposed, in which the positions of the robots are estimated based on a set of reference nodes in the swarm. In this paper, a distributed and resilient localization algorithm is proposed based on the BSA–MMA algorithm, which uses the Backtracking Search Algorithm (BSA) and the Min–Max Area (MMA) confidence factor. It is designed in a novel four-stage approach, where a new method, called Multi-hop Collaborative Min–Max Localization (MCMM), is included to improve the resilience in case of failures during the recognition of the reference nodes. The results, obtained with real Kilobot robots, show 28–36% of performance improvement obtained by the MCMM. Also, it is shown that the final result of the localization process is better when the MCMM is executed than if it is not executed. The experiments outcomes demonstrate that the novel four-stage approach and the use of the MCMM algorithm represents a progress in the design of distributed localization algorithms for SRS, especially with regard to its resilience.     

Robust joint stem detection and crop‐weed classification using image sequences for plant‐specific treatment in precision farming

Conventional farming still relies on large quantities of agrochemicals for weed management which have several negative side‐effects on the environment. Autonomous robots offer the potential to reduce the amount of chemicals applied, as robots can monitor and treat each plant in the field individually and thereby circumventing the uniform chemical treatment of the whole field. Such agricultural robots need the ability to identify individual crops and weeds in the field using sensor data and must additionally select effective treatment methods based on the type of weed. For example, certain types of weeds can only be effectively treated mechanically due to their resistance to herbicides, whereas other types can be treated trough selective spraying. In this article, we present a novel system that provides the necessary information for effective plant‐specific treatment. It estimates the stem location for weeds, which enables the robots to perform precise mechanical treatment, and at the same time provides the pixel‐accurate area covered by weeds for treatment through selective spraying. The major challenge in developing such a system is the large variability in the visual appearance that occurs in different fields. Thus, an effective classification system has to robustly handle substantial environmental changes including varying weed pressure, various weed types, different growth stages, changing visual appearance of the plants and the soil. Our approach uses an end‐to‐end trainable fully convolutional network that simultaneously estimates plant stem positions as well as the spatial extent of crop plants and weeds. It jointly learns how to detect the stems and the pixel‐wise semantic segmentation and incorporates spatial information by considering image sequences of local field strips. The jointly learned feature representation for both tasks furthermore exploits the crop arrangement information that is often present in crop fields. This information is considered even if it is only observable from the image sequences and not a single image. Such image sequences, as typically provided by robots navigating over the field along crop rows, enable our approach to robustly estimate the semantic segmentation and stem positions despite the large variations encountered in different fields. We implemented and thoroughly tested our approach on images from multiple farms in different countries. The experiments show that our system generalizes well to previously unseen fields under varying environmental conditions—a key capability to deploy such systems in the real world. Compared to state‐of‐the‐art approaches, our approach generalizes well to unseen fields and not only substantially improves the stem detection accuracy, that is, distinguishing crop and weed stems, but also improves the semantic segmentation performance.     

Structure and event mining in sports video with efficient mosaic

Video is an information-intensive media with much redundancy. Therefore, it is desirable to be able to mine structure or semantics of video data for efficient browsing, summarization and highlight extraction. In this paper, we propose a mosaic based approach to key-event as well as structure mining, which is regarded as a complementary view for sports video analysis. Mosaic is generated for each shot by a novel efficient mosaicing scheme, which constructs a global motion path and selects a best subset of frames for mosaicing. These improved mosaics are then used as the representative image of shot content. Based on mosaic, the structure and event in sports video are mined by the methods with prior knowledge and without prior knowledge. Without prior knowledge, our system is able to locate global view shots taken by dominant camera. If prior knowledge is available, the events in these global view shots are detected using robust features extracted from mosaics. For global view mining, the experiments compared with key-frame-based scheme have demonstrated that this mosaic-based scheme presents better results in several kinds of sports videos; for events mining, the detection of key-plays and key-events in the specific-domain of soccer videos have proved its effectiveness.

Graph based construction of textured large field of view mosaics for bladder cancer diagnosis

Large field-of-view panoramic images greatly facilitate bladder cancer diagnosis and follow-up. Such 2D mosaics can be obtained by registering the images of a video-sequence acquired during cystoscopic examinations. The scientific challenge in the registration process lies in the strong inter- and intra-patient texture variability of the images, from which primitives cannot be robustly extracted. State-of-the-art registration methods are not at the same time robust and accurate, especially for image pairs with a small amount of overlap (less than 90%) or strong perspective transformations. Moreover, no previous contribution to cystoscopy mosaicing presents panoramic images created from multiple overlapping sequences (e.g. “zigzags” or loop trajectories). We show how such overlapping sections can be automatically detected and present a novel registration algorithm that robustly superimposes non-consecutive image pairs, which are related by stronger perspective transformations and share less overlap than consecutive images (less than 50%). Globally coherent panoramic images are constructed using a non-linear optimization and a novel contrast-enhancing stitching method. Results on both phantom and patient data are obtained using constant algorithm parameters, which demonstrate the robustness of the proposed method. While the methods presented in this contribution are specifically designed for cystoscopy mosaicing, they can also be applied to more general mosaicing problems. We demonstrate this on a traditional stitching application, where a set of pictures of a building are stitched into a seamless, globally coherent panoramic image.     

Transferring knowledge across robots: A risk sensitive approach

One of the most impressive characteristics of human perception is its domain adaptation capability. Humans can recognize objects and places simply by transferring knowledge from their past experience. Inspired by that, current research in robotics is addressing a great challenge: building robots able to sense and interpret the surrounding world by reusing information previously collected, gathered by other robots or obtained from the web. But, how can a robot automatically understand what is useful among a large amount of information and perform knowledge transfer? In this paper we address the domain adaptation problem in the context of visual place recognition. We consider the scenario where a robot equipped with a monocular camera explores a new environment. In this situation traditional approaches based on supervised learning perform poorly, as no annotated data are provided in the new environment and the models learned from data collected in other places are inappropriate due to the large variability of visual information. To overcome these problems we introduce a novel transfer learning approach. With our algorithm the robot is given only some training data (annotated images collected in different environments by other robots) and is able to decide whether, and how much, this knowledge is useful in the current scenario. At the base of our approach there is a transfer risk measure which quantifies the similarity between the given and the new visual data. To improve the performance, we also extend our framework to take into account multiple visual cues. Our experiments on three publicly available datasets demonstrate the effectiveness of the proposed approach.     

Data-based dynamical modeling of externally observed actuators-only robots

This paper describes the derivation of dynamical models for mobile robots using system-identification methods and shows how such models can be used in the control of autonomous devices. Unlike traditional approaches that are based on the system physics alone, in this paper, an autoregressive model structure is derived from the physics of the process, and its parameters are estimated using real input and output data. From the resulting model, efficient control can be obtained for centralized and remotely controlled robots (typically externally observed) with little or no local "intelligence," thus overcoming many of the severe shortcomings presented by such systems. System dead time is one example of a serious problem that can be efficaciously minimized by using the model as an online predictor. Several experimental results using small differential-driven robots are included to demonstrate the applicability and robustness of the suggested procedure in actual real-world scenarios.     

Routing-based navigation of dense mobile robots

In the future, many teams of robots will navigate in home or office environments, similar to dense crowds operating currently in different scenarios. The paper aims to route a large number of robots so as to avoid build-up of congestions, similar to the problem of route planning of traffic systems. In this paper, first probabilistic roadmap approach is used to get a roadmap for online motion planning of robots. A graph search-based technique is used for motion planning. In the literature, typically the search algorithms consider only the static obstacles during this stage, which results in too many robots being scheduled on popular/shorter routes. The algorithm used here therefore penalizes roadmap edges that lie in regions with large robot densities so as to judiciously route the robots. This planning is done continuously to adapt the path to changing robotic densities. The search returns a deliberative trajectory to act as a guide for the navigation of the robot. A point at a distant of the deliberative path becomes the immediate goal of the reactive system. A ‘centre of area’-based reactive navigation technique is used to reactively avoid robots and other dynamic obstacles. In order to avoid two robots blocking each other and causing a deadlock, a deadlock avoidance scheme is designed that detects deadlocks, makes the robots wait for a random time and then allows them to make a few random steps. Experimental results show efficient navigation of a large number of robots. Further, routing results in effectively managing the robot densities so as to enable an efficient navigation.     

Connection Mechanism for Autonomous Self-Assembly in Mobile Robots

This paper presents a connection mechanism for autonomous self-assembly in mobile robots. Using this connection mechanism, mobile robots can be autonomously connected and disconnected. The purpose of self-assembly in mobile robotics is to add a new capability to mobile robots, thus, improving their performance to best fit the terrain conditions. Construction of a reconnectable joint is of primary concern in such systems. In this paper, first the geometric conditions and force equations of a general docking mechanism are studied. Then, we discuss the design details of our connection mechanism and present some experimental results that show that the proposed mechanism overcomes significant alignment errors and is considerably power efficient.