Depth camera based collision avoidance via active robot control

A new type of depth cameras can improve the effectiveness of safety monitoring in human–robot collaborative environment. Especially on today's manufacturing shop floors, safe human–robot collaboration is of paramount importance for enhanced work efficiency, flexibility, and overall productivity. Within this context, this paper presents a depth camera based approach for cost-effective real-time safety monitoring of a human–robot collaborative assembly cell. The approach is further demonstrated in adaptive robot control. Stationary and known objects are first removed from the scene for efficient detection of obstacles in a monitored area. The collision detection is processed between a virtual model driven by real sensors, and 3D point cloud data of obstacles to allow different safety scenarios. The results show that this approach can be applied to real-time work cell monitoring.     

IRS-P6 LISS-3和Landsat-5 TM两种多光谱传感器影像数据的对比分析

随着载有各种新型传感器的卫星相继发射升空,不同传感器之间的相互比较成了一个研究热点。首先从传感器的轨道特征、光谱范围等方面对IRS-P6 LISS-3和Landsat-5 TM进行了机理方面的对比分析,选取3对同日过空的遥感影像,研究了IRS-P6 LISS-3和Landsat-5 TM遥感数据在可见光-近红外、短红外各对应光谱波段之间的关系,建立TM和LISS-3各波段之间的相互转换公式,与Chander等提出的转换公式进行对比分析。结果表明:实际TM和模拟TM多光谱数据之间具有较强的相关关系,决定系数R²均大于0.97;模拟TM与实际TM的水体指数(MNDWI)和归一化植被指数(NDVI)空间散点分布和实际LISS-3与实际TM的空间散点分布相比,具有更强的相关关系,其决定系数R²有一定提高,散点分布趋于对称。因此,所求的关系转换方程具有较高的精度和有效性,效果优于Chander等提出的转换公式。     

Multi-modality imagery database for plant phenotyping

Among many applications of machine vision, plant image analysis has recently began to gain more attention due to its potential impact on plant visual phenotyping, particularly in understanding plant growth, assessing the quality/performance of crop plants, and improving crop yield. Despite its importance, the lack of publicly available research databases containing plant imagery has substantially hindered the advancement of plant image analysis. To alleviate this issue, this paper presents a new multi-modality plant imagery database named “MSU-PID,” with two distinct properties. First, MSU-PID is captured using four types of imaging sensors, fluorescence, infrared, RGB color, and depth. Second, the imaging setup and the variety of manual labels allow MSU-PID to be suitable for a diverse set of plant image analysis applications, such as leaf segmentation, leaf counting, leaf alignment, and leaf tracking. We provide detailed information on the plants, imaging sensors, calibration, labeling, and baseline performances of this new database.     

Synthetic satellite imagery for current and future environmental satellites

During the next decade, data from a new generation of US geostationary and polar orbiting satellites will become available. To prepare for these data, representative imagery of these satellites is desirable. Two independent methods have been developed to create imagery from future satellites before they are placed into orbit. One method uses data from current operational and experimental satellites. Data obtained this way are referred to as simulated imagery. Another method generates satellite imagery by using numerical models. Data obtained by this method are referred to as synthetic imagery. Each method has some weaknesses that can be overcome by using both methods together. Synthetic imagery for two future US sensors is introduced in this paper. Emphasis is placed on a severe thunderstorm event.     

A survey of energy-efficient context recognition systems using wearable sensors for healthcare applications

Human context recognition (HCR) from on-body sensor networks is an important and challenging task for many healthcare applications because it offers continuous monitoring capability of both personal and environmental parameters. However, these systems still face a major energy issue that prevent their wide adoption. Indeed, in healthcare applications, sensors are used to capture data during daily life or extended stays in hospital. Thus, continuous sampling and communication tasks quickly deplete sensors’ battery reserves, and frequent battery replacement is not convenient. Therefore, there is a need to develop energy-efficient solutions for long-term monitoring applications in order to foster the acceptance of these technologies by the patients. In this paper, we survey existing energy-efficient approaches designed for HCR based on wearable sensor networks. We propose a new classification of the energy-efficient mechanisms for health-related human context recognition applications and we review the related works in detail. Moreover, we provide a qualitative comparison of these solutions in terms of energy-consumption, recognition accuracy and latency. Finally, we discuss open research issue and give directions for future works.     

三维视觉前沿进展

在自动驾驶、机器人、数字城市以及虚拟/混合现实等应用的驱动下,三维视觉得到了广泛的关注。三维视觉研究主要围绕深度图像获取、视觉定位与制图、三维建模及三维理解等任务而展开。本文围绕上述三维视觉任务,对国内外研究进展进行了综合评述和对比分析。首先,针对深度图像获取任务,从非端到端立体匹配、端到端立体匹配及无监督立体匹配3个方面对立体匹配研究进展进行了回顾,从深度回归网络和深度补全网络两个方面对单目深度估计研究进展进行了回顾。其次,针对视觉定位与制图任务,从端到端视觉定位和非端到端视觉定位两个方面对大场景下的视觉定位研究进展进行了回顾,并从视觉同步定位与地图构建和融合其他传感器的同步定位与地图构建两个方面对同步定位与地图构建的研究进展进行了回顾。再次,针对三维建模任务,从深度三维表征学习、深度三维生成模型、结构化表征学习与生成模型以及基于深度学习的三维重建等4个方面对三维几何建模研究进展进行了回顾,并从多视RGB重建、单深度相机和多深度相机方法以及单视图RGB方法等3个方面对人体动态建模研究进展进行了回顾。最后,针对三维理解任务,从点云语义分割和点云实例分割两个方面对点云语义理解研究进展进行了回顾。在此基础上,给出了三维视觉研究的未来发展趋势,旨在为相关研究者提供参考。     

Multi-sensor information fusion based on machine learning for real applications in human activity recognition: State-of-the-art and research challenges

This paper firstly introduces common wearable sensors, smart wearable devices and the key application areas. Since multi-sensor is defined by the presence of more than one model or channel, e.g. visual, audio, environmental and physiological signals. Hence, the fusion methods of multi-modality and multi-location sensors are proposed. Despite it has been contributed several works reviewing the stateoftheart on information fusion or deep learning, all of them only tackled one aspect of the sensor fusion applications, which leads to a lack of comprehensive understanding about it. Therefore, we propose using a more holistic approach in order to provide a more suitable starting point from which to develop a full understanding of the fusion methods of wearable sensors. Specifically, this review attempts to provide a more comprehensive survey of the most important aspects of multi-sensor applications for human activity recognition, including those recently added to the field for unsupervised learning and transfer learning. Finally, the open research issues that need further research and improvement are identified and discussed.     

Computer games technology and higher education

The motivation of this paper is to debate the issue of how we should treat computer games technology in higher education. Certainly it is the case that computer games technology is a new and demanding profession. It is also the case that there has been a slow realisation within the academic community of both the depth and the importance of the topic. The issues are obvious. What exactly comprises computer games technology and should it be considered as a separate academic topic for a degree course or be a set of options within an established undergraduate degree structure? And who is responsible for pushing forward research in the subject? Within the framework of this debate we will present our work at the University of Sheffield.     

Understanding human behavior from motion imagery

Computer vision is gradually making the transition from image understanding to video understanding. This is due to the enormous success in analyzing sequences of images that has been achieved in recent years. The main shift in the paradigm has been from recognition followed by reconstruction (shape from X) to motion-based recognition. Since most videos are about people, this work has focused on the analysis of human motion. In this paper, I present my perspective on understanding human behavior. Automatically understanding human behavior from motion imagery involves extraction of relevant visual information from a video sequence, representation of that information in a suitable form, and interpretation of visual information for the purpose of recognition and learning about human behavior. Significant progress has been made in human tracking over the last few years. As compared with tracking, not much progress has been made in understanding human behavior, and the issue of representation has largely been ignored. I present my opinion on possible reasons and hurdles for slower progress in understanding human behavior, briefly present our work in tracking, representation, and recognition, and comment on the next steps in all three areas.Published online: 28 August 2003     

Toward evaluation of visual navigation algorithms on RGB-D data from the first- and second-generation Kinect

Although the introduction of commercial RGB-D sensors has enabled significant progress in the visual navigation methods for mobile robots, the structured-light-based sensors, like Microsoft Kinect and Asus Xtion Pro Live, have some important limitations with respect to their range, field of view, and depth measurements accuracy. The recent introduction of the second- generation Kinect, which is based on the time-of-flight measurement principle, brought to the robotics and computer vision researchers a sensor that overcomes some of these limitations. However, as the new Kinect is, just like the older one, intended for computer games and human motion capture rather than for navigation, it is unclear how much the navigation methods, such as visual odometry and SLAM, can benefit from the improved parameters. While there are many publicly available RGB-D data sets, only few of them provide ground truth information necessary for evaluating navigation methods, and to the best of our knowledge, none of them contains sequences registered with the new version of Kinect. Therefore, this paper describes a new RGB-D data set, which is a first attempt to systematically evaluate the indoor navigation algorithms on data from two different sensors in the same environment and along the same trajectories. This data set contains synchronized RGB-D frames from both sensors and the appropriate ground truth from an external motion capture system based on distributed cameras. We describe in details the data registration procedure and then evaluate our RGB-D visual odometry algorithm on the obtained sequences, investigating how the specific properties and limitations of both sensors influence the performance of this navigation method.     

Fine spatial resolution satellite sensors for the next decade

Following the end of the Cold War governmental restrictions on the commercial availability of fine spatial resolution satellite sensor imagery have been relaxed world-wide. This, combined with marked reductions in the costs of developing, launching and operating satellites, has led to considerable research activity in this field by a number of private remote sensing organisations. Within the next few years, imagery with a spatial resolution as fine as 1 m in panchromatic mode and 4 m in multispectral mode will be available widely. This Letter presents a review of fine spatial resolution satellite sensors in operation or planned for operation within the next decade. Details of both commercial and governmental systems are provided. The emphasis is on commercially available data and so data collected for defence applications only are not included. A variety of both instrument and data specifications are highlighted, including spatial and spectral capabilities, and characteristics of viewing geometry, satellite orbit, data collection and supply. Typically, these systems are characterized not only by their fine spatial resolution, but also by high geometric precision, short revisit intervals and rapid data supply.     

Online continual learning for human activity recognition

Sensor-based human activity recognition (HAR), with the ability to recognise human activities from wearable or embedded sensors, has been playing an important role in many applications including personal health monitoring, smart home, and manufacturing. The real-world, long-term deployment of these HAR systems drives a critical research question: how to evolve the HAR model automatically over time to accommodate changes in an environment or activity patterns. This paper presents an online continual learning (OCL) scenario for HAR, where sensor data arrives in a streaming manner which contains unlabelled samples from already learnt activities or new activities. We propose a technique, OCL-HAR, making a real-time prediction on the streaming sensor data while at the same time discovering and learning new activities. We have empirically evaluated OCL-HAR on four third-party, publicly available HAR datasets. Our results have shown that this OCL scenario is challenging to state-of-the-art continual learning techniques that have significantly underperformed. Our technique OCL-HAR has consistently outperformed them in all experiment setups, leading up to 0.17 and 0.23 improvements in micro and macro F1 scores.     

Detection of rural electrification in Africa using DMSP-OLS night lights imagery

We report on the first systematic ground-based validation of the US Air Force Defense Meteorological Satellite Program’s Operational Linescan System (DMSP-OLS) night lights imagery to detect rural electrification in the developing world. Drawing upon a unique survey of villages in Senegal and Mali, this study compares night-time light output from the DMSP-OLS against ground-based survey data on electricity use in 232 electrified villages and additional administrative data on 899 unelectrified villages. The analysis reveals that electrified villages are consistently brighter than unelectrified villages across annual composites, monthly composites, and a time series of nightly imagery. Electrified villages appear brighter because of the presence of streetlights, and brighter villages tend to have more streetlights. By contrast, the correlation of light output with household electricity use and access is low. We further demonstrate that a detection algorithm using data on night-time light output and the geographic location of settlements can accurately classify electrified villages. This research highlights the potential to use night lights imagery for the planning and monitoring of ongoing efforts to connect the 1.4 billion people who lack electricity around the world.     

Commodity cluster-based parallel processing of hyperspectral imagery

The rapid development of space and computer technologies has made possible to store a large amount of remotely sensed image data, collected from heterogeneous sources. In particular, NASA is continuously gathering imagery data with hyperspectral Earth observing sensors such as the Airborne Visible-Infrared Imaging Spectrometer (AVIRIS) or the Hyperion imager aboard Earth Observing-1 (EO-1) spacecraft. The development of fast techniques for transforming the massive amount of collected data into scientific understanding is critical for space-based Earth science and planetary exploration. This paper describes commodity cluster-based parallel data analysis strategies for hyperspectral imagery, a new class of image data that comprises hundreds of spectral bands at different wavelength channels for the same area on the surface of the Earth. An unsupervised technique that integrates the spatial and spectral information in the image data using multi-channel morphological transformations is parallelized and compared to other available parallel algorithms. The code's portability, reusability and scalability are illustrated by using two high-performance parallel computing architectures: a distributed memory, multiple instruction multiple data (MIMD)-style multicomputer at European Center for Parallelism of Barcelona, and a Beowulf cluster at NASA's Goddard Space Flight Center. Experimental results suggest that Beowulf clusters are a source of computational power that is both accessible and applicable to obtaining results in valid response times in information extraction applications from hyperspectral imagery.     

Are low cost Brain Computer Interface headsets ready for motor imagery applications?

Low cost electroencephalography (EEG) headset devices for brain data capturing are fast becoming a key instrument on Brain Computer Interface (BCI) applications. In spite of being a controller device initially developed for gaming, the research community has adopted them as a key element to gather EEG data. However, there have been little discussion about their performance when being compared with professional and research EEG headsets.This paper provides an assessment of one of these devices, the Emotiv EPOC, on a motor imagery problem. As a benchmark, the data and results presented for the Data Set V of the BCI Competition III have been used, which were recorded with a professional Biosemi Active 2 EEG headset. From the perspective of a final working application, it is shown that the performance of this headset is comparable to that found in professional devices when using the same number of sensors and sensor positions for a three status motor imagery cognitive process. This finding implies an increase on the number of EEG headsets the researchers and manufacturers of BCI systems applied to motor imagery problems can integrate and a reduction of their cost.As part of this paper the Emotiv EPOC recorded raw and pre-processed datasets are published to allow further improvements and comparisons.     

The sonification space: A reference system for sonification tasks

Sonification is a fairly new term to scientists who are unaware of its multiple use cases. Even if some general definitions of the concept of sonification are commonly accepted, heterogeneous techniques – significantly different as it regards approaches, means and goals – are available. In this work we propose a reference system useful to interpret already-existing sonification instances and to plan new sonification tasks. This work aims to present a reference system for sonification using the inherent properties in the sonic output rather than the data itself. Validation has been conducted by automatically analyzing available experiments and examples, and placing them on the proposed sonification space, according to time-granularity and abstraction-level dimensions. This work can constitute the starting point for future research on computer-assisted sonification. It will be beneficial to a wide range of readers, in particular those from different disciplines looking at new ways to present and analyze data.     

A Survey of Information Visualization Books

Information visualization is a rapidly evolving field with a growing volume of scientific literature and texts continually published. To keep abreast of the latest developments in the domain, survey papers and state‐of‐the‐art reviews provide valuable tools for managing the large quantity of scientific literature. Recently, a survey of survey papers was published to keep track of the quantity of refereed survey papers in information visualization conferences and journals. However, no such resources exist to inform readers of the large volume of books being published on the subject, leaving the possibility of valuable knowledge being overlooked. We present the first literature survey of information visualization books that addresses this challenge by surveying the large volume of books on the topic of information visualization and visual analytics. This unique survey addresses some special challenges associated with collections of books (as opposed to research papers) including searching, browsing and cost. This paper features a novel two‐level classification based on both books and chapter topics examined in each book, enabling the reader to quickly identify to what depth a topic of interest is covered within a particular book. Readers can use this survey to identify the most relevant book for their needs amongst a quickly expanding collection. In indexing the landscape of information visualization books, this survey provides a valuable resource to both experienced researchers and newcomers in the data visualization discipline.     

Reconstruction of sand wave bathymetry using both satellite imagery and multi-beam bathymetric data: a case study of the Taiwan Banks

Sand waves are a widespread bed-form in the tidal environment. Their formation, migration, and other properties have great significance in the fields of geology and oceanography. Currently, research on sand waves is mainly based on multi-beam bathymetric (MB) data. For large sand wave regions, however, the acquisition ability of MB data becomes limited; instead, interval-line measurement is adopted but with the inherent problem that it cannot achieve the full spatial coverage required. Reconstructed sand wave bathymetry (SWB) using interpolation algorithms cannot reflect the real SWB either. In this article, we propose a new approach in using both satellite imagery and MB data to reconstruct SWB, which takes advantage of characteristic information (ripple and crest) of SWB rendered in satellite imagery and MB data. The new approach is exemplified by a case study of the Taiwan Banks. We use the imagery obtained by the Charge-Coupled Device from the Small Satellite Constellation for Environment and Disaster Monitoring and Forecasting, and the MB data from R2Sonic to reconstruct the digital elevation model. The results show that the root mean square error of the reconstructed water depth is 1.47 m (compared to the MB data not using this approach), suggesting that the new approach is effective in reconstructing SWB. This approach allows a reduction in MB data track density, which may lead to an improvement in efficiency.