Rawseeds ground truth collection systems for indoor self-localization and mapping

A trustable and accurate ground truth is a key requirement for benchmarking self-localization and mapping algorithms; on the other hand, collection of ground truth is a complex and daunting task, and its validation is a challenging issue. In this paper we propose two techniques for indoor ground truth collection, developed in the framework of the European project Rawseeds, which are mutually independent and also independent on the sensors onboard the robot. These techniques are based, respectively, on a network of fixed cameras, and on a network of fixed laser scanners. We show how these systems are implemented and deployed, and, most importantly, we evaluate their performance; moreover, we investigate the possible fusion of their outputs.     

Design and development of a spectro-polarimeter for ground truth data collection

Abstract

A spectro-polarimeter operating in ten spectral bands in the visible and near-IR range has been designed and developed for the measurement of Stokes parameters of reflected radiation from the terrain surfaces under in situ conditions. The radiometric performance and the data quality of the unit have been evaluated and explained. The instrument was extensively used in the field for the collection of spectral signatures of wheat canopies. The unit was also used in the laboratory. All spectral bands were found to be useful for the measurement of absolute values of spectral irradiance and spectral radiance, while bands 1-8 were found to be useful for the measurement of polarization. This article describes the design features and specifications and gives a typical set of field observations.     

A ground truth based vanishing point detection algorithm

A Bayesian probability-based vanishing point detection algorithm is presented which introduces the use of multiple features and training with ground truth data to determine vanishing point locations. The vanishing points of 352 images were manually identified to create ground truth data. Each intersection is assigned a probability of being coincident with a ground truth vanishing point, based upon conditional probabilities of a number of features. The results of this algorithm are demonstrated to be superior to the results of a similar algorithm where each intersection is considered to be of equal importance. The advantage of this algorithm is that multiple features derived from ground truth training are used to determine vanishing point location.     

Navigating with ranging radios: Five data sets with ground truth

Here we present five large data sets with range‐only measurements between a mobile robot and stationary nodes. Each data set consists of range measurements, surveyed locations of the stationary radio nodes, dead‐reckoned trajectory of the robot, and ground truth from a sophisticated inertial navigation system/global positioning system mounted on a robot traveling several kilometers at a time. Range measurements are made with two radio‐based ranging systems: a RFID tag‐based ranging system and an ultra–wide band ranging system. All the data are accurately time‐stamped and presented in standard formats (i.e., text files). In addition to the raw data, we present some noise characteristics of the two different ranging systems to offer insight into the quality of the range data from each system. © 2009 Wiley Periodicals, Inc.     

A seamless ground truth detection for enhancing localization on mobile robots

Robot localization mechanism is an essential feature to determine the position of the corresponding robot within an environment, particularly in the field of Standard Platform League (SPL) at the RoboCup. Despite the available input from the onboard sensors, the ground truth information is necessary for a real-time localization system. This study proposes an efficient color-based segmentation scheme using an overhead projective camera with an autonomous calibration procedure. This enhances the system robustness against lighting changes and different labeling setups for the field environment. The experimental results show that the proposed method localizes and recognizes objects with a detection rate of 96.4%.     

基于Labelme的参考图像的手工分割

图像分割算法虽然已经有了大量的研究,但没有适用于所有图像的通用分割算法。因此针对不同图像的实际情况,需要进行大量的实验来选择最优的分割算法。通过比较算法分割得到的结果图像与对应手工分割得到的参考图像之间的差异,可以得到一种图像分割算法的性能评价,因此本文提出一种参考图像的获取方法。通过结合算法和Label Me在线注释软件,该方法能够方便地完成图像中各种目标边界的定位并保存图像。     

Effect of errors in ground truth on classification accuracy

The effect of errors in ground truth on the estimated thematic accuracy of a classifier is considered. A relationship is derived between the true accuracy of a classifier relative to ground truth without errors, the actual accuracy of the ground truth used, and the measured accuracy of the classifier as a function of the number of classes. We show that if the accuracy of the ground truth is known or can be estimated, the true accuracy of a classifier can be estimated from the measured accuracy. In a series of simulations our method is shown to produce unbiased estimates of the true accuracy of the classifier with an uncertainty that depends on the number of samples and the accuracy of the ground truth. A method for determining the relative performance of two or more classifiers over the same area is then discussed. The results indicate that, as the number of samples increases, the performance of the classifiers can be effectively differentiated using inaccurate ground truth. It is argued that relative accuracies computed using a large number of inaccurate ground truth points are more representative of the true relative performance of the classifiers as they are being evaluated over a larger portion of the scene. An example is presented that uses this method to evaluate the relative performance of two Landsat classifiers.     

CLEVR-XAI: A benchmark dataset for the ground truth evaluation of neural network explanations

The rise of deep learning in today’s applications entailed an increasing need in explaining the model’s decisions beyond prediction performances in order to foster trust and accountability. Recently, the field of explainable AI (XAI) has developed methods that provide such explanations for already trained neural networks. In computer vision tasks such explanations, termed heatmaps, visualize the contributions of individual pixels to the prediction. So far XAI methods along with their heatmaps were mainly validated qualitatively via human-based assessment, or evaluated through auxiliary proxy tasks such as pixel perturbation, weak object localization or randomization tests. Due to the lack of an objective and commonly accepted quality measure for heatmaps, it was debatable which XAI method performs best and whether explanations can be trusted at all. In the present work, we tackle the problem by proposing a ground truth based evaluation framework for XAI methods based on the CLEVR visual question answering task. Our framework provides a (1) selective, (2) controlled and (3) realistic testbed for the evaluation of neural network explanations. We compare ten different explanation methods, resulting in new insights about the quality and properties of XAI methods, sometimes contradicting with conclusions from previous comparative studies. The CLEVR-XAI dataset and the benchmarking code can be found at https://github.com/ahmedmagdiosman/clevr-xai.     

Comparison of land surface temperatures derived from satellite observations with ground truth during FIFE

Surface temperatures of the FIFE (First ISLSCP Field Experiment) experimental area derived from thermal infrared radiances recorded from different satellite platforms at different scales were compared with reference observations by means of infrared thermometers at ground stations distributed over the area. FIFE was conducted during late spring, summer and fall over an area of 15 km by 15 km in a hilly tall-grass prairie region in northeastern Kansas. The data available for this purpose were produced by AVHRR and TOVS instruments aboard NOAA-9 and NOAA-10, the TM instrument aboard Landsat-5 and VISSR instrument aboard GOES-7. The scales covered by these instruments span a wide range, namely between hundreds of metres (Landsat TM) and hundreds of kilometres (TOVS). The data are analysed both with and without the application of an atmospheric correction.     

A portable multiband radiometer for ground data collection in remote sensing

Abstract.

The design and calibration of a rugged, low-cost, portable multiband radiometer is described. The instrument measures simultaneously in four bands, the spectral coverage of each being determined by interchangeable absorption filters. Simple band-pass type radiometers are seen to be complementary to conventional spectroradiometers for ground data collection in remote sensing. They have the inherent advantages of portability and speed of operation which make them particularly suitable for fieldwork in areas of complex terrain. An example of the practical use of the instrument to measure the spectral reflectance of partially vegetated surfaces in an area of complex terrain in southern Italy is presented. The increased spectral and spatial resolution of second generation sensing systems will make remote sensing of complex areas more feasible and extensive ground-based spectral measurements in such areas are a necessary step to understanding and utilizing such remotely sensed data.     

Measuring sea surface temperature from satellites: a ground truth approach

Determining the correction for atmospheric attenuation is a major problem in processing thermal infrared digital data from very high resolution radiometers aboard NOAA Polar Orbiting Satellites. An empirical equation for estimating this correction is developed. The coefficients of the equation are determined by using regression techniques and comparing satellite observations to sea surface temperature measurements. Although there is not sufficient data to fully evaluate this procedure, initial satellite measurements are within 0.5°C of independent sea surface temperature measurements.     

Evaluation of 3D interest point detection techniques via human-generated ground truth

In this paper, we present an evaluation strategy based on human-generated ground truth to measure the performance of 3D interest point detection techniques. We provide quantitative evaluation measures that relate automatically detected interest points to human-marked points, which were collected through a web-based application. We give visual demonstrations and a discussion on the results of the subjective experiments. We use a voting-based method to construct ground truth for 3D models and propose three evaluation measures, namely False Positive and False Negative Errors, and Weighted Miss Error to compare interest point detection algorithms.     

A new feature commonly observed from air and ground for outdoor localization with elevation map built by aerial mapping system

Monte Carlo localization (MCL) uses a reference map to estimate a pose of a ground robot in outdoor environments. However, MCL shows low performance when it uses an elevation map built by an aerial mapping system because three‐dimensional (3D) environments are observed differently from the air and the ground and such an elevation map cannot represent outdoor environments in detail. Although other types of maps have been proposed to improve localization performance, an elevation map is still used as the main reference map in some applications. Therefore, we propose a new feature to improve localization performance with an elevation map. This feature is extracted from 3D range data and represents the part of an object that can be commonly observed from both the air and the ground. Therefore, this feature is likely to be accurately matched with an elevation map, and the average error of this feature is much smaller than that of unclassified sensing data. Experimental results in real environments show that the success rate of global localization increased and the error of local tracking decreased. Thus, the proposed feature can be very useful for localization of an outdoor ground robot when an elevation map is used as a reference map. © 2010 Wiley Periodicals, Inc.     

A system of robotic grasping with experience acquisition

Robotic grasping has played a fundamental role in the robotic manipulation, while grasping an unknown object is still a challenge. A successful grasp is largely determined by the object representation and the corresponding grasp planning strategy. With the help of RGBD camera, the point cloud of the object can be obtained conveniently. However, the large amount of point cloud is often unorganized with some inevitable noise. It may result in the geometry of the object imprecise and lead to some poor grasp planning. In this paper, a parametric model--superquadric is chosen to represent the shape of an object. We firstly recover the superquadric of an object from the raw point cloud in a single view with conjugate gradient method. Then a force-closure grasp planning strategy is applied to this object to obtain stable grasp configurations. Finally we store the grasp parameters as grasp experience in a grasp dataset which can be used for future grasping tasks. The performance of the proposed grasping system is represented both in simulation and actual experiment scenario successfully.     

A new accelerated proximal technique for regression with high-dimensional datasets

We consider the problem of minimization of the sum of two convex functions, one of which is a smooth function, while another one may be a nonsmooth function. Many high-dimensional learning problems (classification/regression) can be designed using such frameworks, which can be efficiently solved with the help of first-order proximal-based methods. Due to slow convergence of traditional proximal methods, a recent trend is to introduce acceleration to such methods, which increases the speed of convergence. Such proximal gradient methods belong to a wider class of the forward–backward algorithms, which mathematically can be interpreted as fixed-point iterative schemes. In this paper, we design few new proximal gradient methods corresponding to few state-of-the-art fixed-point iterative schemes and compare their performances on the regression problem. In addition, we propose a new accelerated proximal gradient algorithm, which outperforms earlier traditional methods in terms of convergence speed and regression error. To demonstrate the applicability of our method, we conducted experiments for the problem of regression with several publicly available high-dimensional real datasets taken from different application domains. Empirical results exhibit that the proposed method outperforms the previous methods in terms of convergence, accuracy, and objective function values.     

A novel robotic leg design with hybrid dynamics

A new leg mechanism is designed to achieve optimal behaviour in both the stance and swing phases. The leg is designed such that its dynamics and kinematics optimally mimic those of simpler mechanisms, namely the spring-loaded inverted pendulum (SLIP) and the double pendulum, for the stance and swing phase, respectively. Controllers based on the simpler mechanisms can thus be used, resulting in an effective control strategy that is simple and transparent.     

Exploring spatial datasets with histograms

As online spatial datasets grow both in number and sophistication, it becomes increasingly difficult for users to decide whether a dataset is suitable for their tasks, especially when they do not have prior knowledge of the dataset. In this paper, we propose browsing as an effective and efficient way to explore the content of a spatial dataset. Browsing allows users to view the size of a result set before evaluating the query at the database, thereby avoiding zero-hit/mega-hit queries and saving time and resources. Although the underlying technique supporting browsing is similar to range query aggregation and selectivity estimation, spatial dataset browsing poses some unique challenges. In this paper, we identify a set of spatial relations that need to be supported in browsing applications, namely, the contains, contained and the overlap relations. We prove a lower bound on the storage required to answer queries about the contains relation accurately at a given resolution. We then present three storage-efficient approximation algorithms which we believe to be the first to estimate query results about these spatial relations. We evaluate these algorithms with both synthetic and real world datasets and show that they provide highly accurate estimates for datasets with various characteristics. Recommended by: Sunil Prabhakar Work supported by NSF grants IIS 02-23022 and CNF 04-23336. An earlier version of this paper appeared in the 17th International Conference on Data Engineering (ICDE 2001).