Long-term high-frequency validation of MODIS LST products in a Mediterranean mountain environment

The accuracy of Moderate-resolution Imaging Spectroradiometer (MODIS) level 3 1 km land surface temperature (LST) products was assessed through long-term validation carried out in a mountainous site in Sierra Nevada, southeast Spain. A total of 1458 day and night thermal images, acquired by Terra and Aqua satellites during 2008, were processed and compared to ground-truth data recorded at the meteorological station of Robledal de Cañar with a frequency of one measurement every 10 min. The purpose of this investigation was to understand whether MODIS LST data can be used as input for climate models to be constructed for mountainous environments. Several trends in the MODIS LST data were observed, including the underestimation of daytime values and the overestimation of night-time values. Although all the data sets (Terra and Aqua, diurnal and nocturnal) showed high correlation coefficients with ground measurements, only night values maintained a relatively high accuracy of approximately 2°C of annual average error. Factors that may cause errors in the MODIS LST data, like acquisition angle, cloud, and snow cover, were analysed without conclusive results. High accuracy levels, i.e. close to 1°C, similar to other validation studies carried out over simpler and much more homogenous land-cover types such as cultivated fields, have been achieved for night images acquired during the summer months, thus making these datasets reliable for their use in climatic models over mountainous regions.     

Ten years of forest cover change in the Sierra Nevada detected using Landsat satellite image analysis

The Sierra Nevada of California is a region where large wildfires have been suppressed for over a century. A detailed geographic record of recent vegetation regrowth and disturbance patterns in forests of the Sierra Nevada remains a gap that can be filled with remote-sensing data. Landsat Thematic Mapper imagery was analysed to detect 10 years of recent changes (between 2000 and 2009) in forest vegetation cover for areas burned by wildfires between years of 1995 and 1999 in the region. Results confirmed the prevalence of regrowing forest vegetation during the period 2000 and 2009 over 17% of the combined burned areas. Classification of these regrowing forest vegetation areas by the Landsat normalized burn ratio (NBR) showed that there was a marked increase in average disturbance index (ΔDI) values in the transitions from low to moderate to high burn severity classes. Within the five combined wildfire perimeters, 45% of the high burn severity area delineated by the RdNBR analysis was covered by regrowing forest detected between 2000 and 2009. In contrast, a notable fraction (12%) of the entire 5 km (unburned) buffer area outside the 1995–1999 fires perimeters showed decline in forest cover, and not nearly as many regrowing forest areas, covering only 3% of all the 1995–1999 buffer areas combined. Based on comparison of these results to ground-based survey data and high-resolution aerial images, the Landsat difference index (ΔDI) methodology can fulfil much of the need for consistent, low-cost monitoring of changes due to climate and biological factors in western forest regrowth following stand-replacing disturbances.     

A SOM prototype-based cluster analysis methodology

Data clustering is aimed at finding groups of data that share common hidden properties. These kinds of techniques are especially critical at early stages of data analysis where no information about the dataset is available. One of the mayor shortcomings of the clustering algorithms is the difficulty for non-experts users to configure them and, in some cases, interpret the results. In this work a computational approach with a two-layer structure based on Self-Organizing Map (SOM) is presented for cluster analysis. In the first level, a quantization of the data samples using topology-preserving metrics to automatically determine the number of units in the SOM is proposed. In the second level the obtained SOM prototypes are clustered by means of a connectivity analysis to explore the quality of the partitioning with different number of clusters. The most important benefit of this two-layer procedure is that computational load decreases considerably in comparison with data based clustering methods, making it possible to cluster large data sets and to consider several different clustering alternatives in a limited time. This methodology produces a two-dimensional map representation of the, usually, high dimensional input space, along with quantitative information on viable clustering alternatives, which facilitates the exploration of the possible partitions in a dataset. The efficiency and interpretation of the methodology is illustrated by its application to artificial, benchmark and real complex biological datasets. The experimental results demonstrate the ability of the method to identify possible segmentations in a dataset, compared to algorithms that only yield a single clustering solution. The proposed algorithm tackles the intrinsic limitations of SOM and the parameter settings associated with the clustering methodology, without requiring the number of clusters or the SOM architecture as a prerequisite, among others. This way, it makes possible its application even by researchers with a limited expertise in machine learning.     

Visualizing the Impact of Geographical Variations on Multivariate Clustering

Traditional multivariate clustering approaches are common in many geovisualization applications. These algorithms are used to define geodemographic profiles, ecosystems and various other land use patterns that are based on multivariate measures. Cluster labels are then projected onto a choropleth map to enable analysts to explore spatial dependencies and heterogeneity within the multivariate attributes. However, local variations in the data and choices of clustering parameters can greatly impact the resultant visualization. In this work, we develop a visual analytics framework for exploring and comparing the impact of geographical variations for multivariate clustering. Our framework employs a variety of graphical configurations and summary statistics to explore the spatial extents of clustering. It also allows users to discover patterns that can be concealed by traditional global clustering via several interactive visualization techniques including a novel drag & drop clustering difference view. We demonstrate the applicability of our framework over a demographics dataset containing quick facts about counties in the continental United States and demonstrate the need for analytical tools that can enable users to explore and compare clustering results over varying geographical features and scales.     

Spatiotemporal evolution analysis of time-series land use change using self-organizing map to examine the zoning and scale effects

In this study, we applied a self-organizing map (SOM) neural network method to analyze the spatiotemporal evolution of land-use in Beijing using five time-period classification data from 2005 to 2013. We conducted a spatiotemporal integrated expression and a comparative analysis of the time-series of land use data at 5 km grid level. The experiments at the township level and three different grid levels (20 km, 10 km and 1 km) were simultaneously conducted as the comparison study to analysis the modifiable areal unit problem (MAUP). The land use structure data of analysis unit over 5 years were used as input data for SOM. After training the SOM network, the aggregation modes for different land use types were identified on the output plane. Then, the second-step cluster of the output neurons of the SOM was analyzed to construct a series of land use change trajectories that enabled us to get the spatiotemporal patterns of land use change. The results showed five spatial aggregation patterns and three spatiotemporal change patterns of land use 2005 to 2013. The three patterns of spatiotemporal change represent (1) the expansion of urban areas onto farmland in the southeast plains, (2) the development of forest land in the northwest mountainous areas, and (3) the development of piedmont mixed type land use structures. The results of the comparison experiments showed the zoning effect and the scale effect of MAUP, which were: the 5 km grid-based analysis could provide more precise spatiotemporal evolution patterns in the mountainous area, whereas the township level analysis was more appropriate in the plain area; the pattern of forest land development could be better revealed on 20 km and 10 km grid level, while the pattern of built-up land development could be better revealed on 5 km and 1 km grid level.     

Analysing coastal ocean model outputs using competitive-learning pattern recognition techniques

To assist in interpreting the hydrodynamics of a complex coastal environment, a Self Organizing Map (SOM) has been constructed using output from a three-dimensional hydrodynamic model of the Huon-D'Entrecasteaux region in South-East Tasmania, over a one-year period. Interpretation of the SOM enabled nine characteristic or prototype states to be identified. As expected, the dominant forcing mechanisms were freshwater input via riverine discharge and input from oceanic waters. While these mechanisms are well understood, subtle features associated with the interaction of the two forcing mechanisms and the transitions between meta-stable states, were revealed by visualizing the SOM output. Further investigation was undertaken to determine how effective the SOM would be in identifying these prototype states given sensor data from a sensor network being designed for future deployment within the region. This research has demonstrated that SOM analysis can be a useful tool for identifying and interpreting patterns in large oceanographic datasets.     

Market segmentation of organ donors in Egypt: a bio-inspired computational intelligence approach

First performed in 1954, organ transplantation is a universally practiced clinical procedure. This study uses ant colony optimization (ACO), radial basis function neural network (RBFNN), Kohonen’s self-organizing maps (SOM), and support vector machines (SVMs) to examine the effect of various cognitive, psychographic, and attitudinal factors on organ donation. ACO, RBFNN, SOM, and SVMs are compared to a standard statistical method (linear discriminant analysis [LDA]). The variable sets considered are altruistic values, perceived risks/benefits, knowledge, attitudes toward organ donation, and intention to donate organs. The paper shows how it is possible to identify various dimensions of organ donation behavior by uncovering complex patterns in the dataset and also shows the classification and clustering abilities of machine-learning systems.     

Swarm optimized organizing map (SWOM): A swarm intelligence basedoptimization of self-organizing map

This work studies the optimization of SOM algorithm in terms of reducing its training time by the use of a swarm intelligence method, i.e. particle swarm optimization (PSO).Our novel algorithm optimizes SOM with PSO and reduces computational time of the training phase of SOM significantly. The performance of the algorithms has been tested with genomic datasets, biomedical datasets and an artificial dataset to show the efficiency of swarm optimized SOM, i.e. SWOM. The experimental comparison between SOM and SWOM, has demonstrated significant reduction in training time of SWOM with preservation of clustering quality.     

Space‐in‐Time and Time‐in‐Space Self‐Organizing Maps for Exploring Spatiotemporal Patterns

Spatiotemporal data pose serious challenges to analysts in geographic and other domains. Owing to the complexity of the geospatial and temporal components, this kind of data cannot be analyzed by fully automatic methods but require the involvement of the human analyst's expertise. For a comprehensive analysis, the data need to be considered from two complementary perspectives: (1) as spatial distributions (situations) changing over time and (2) as profiles of local temporal variation distributed over space. In order to support the visual analysis of spatiotemporal data, we suggest a framework based on the “Self‐Organizing Map” (SOM) method combined with a set of interactive visual tools supporting both analytic perspectives. SOM can be considered as a combination of clustering and dimensionality reduction. In the first perspective, SOM is applied to the spatial situations at different time moments or intervals. In the other perspective, SOM is applied to the local temporal evolution profiles. The integrated visual analytics environment includes interactive coordinated displays enabling various transformations of spatiotemporal data and post‐processing of SOM results. The SOM matrix display offers an overview of the groupings of data objects and their two‐dimensional arrangement by similarity. This view is linked to a cartographic map display, a time series graph, and a periodic pattern view. The linkage of these views supports the analysis of SOM results in both the spatial and temporal contexts. The variable SOM grid coloring serves as an instrument for linking the SOM with the corresponding items in the other displays. The framework has been validated on a large dataset with real city traffic data, where expected spatiotemporal patterns have been successfully uncovered. We also describe the use of the framework for discovery of previously unknown patterns in 41‐years time series of 7 crime rate attributes in the states of the USA.     

A comparison of 18 winter seasons of in situ and passive microwave-derived snow water equivalent estimates in Western Canada

The Meteorological Service of Canada (MSC) has developed an operational snow water equivalent (SWE) retrieval algorithm suite for western Canada that can be applied to both Scanning Multichannel Microwave Radiometer (SMMR) and Special Sensor Microwave/Imager (SSM/I) data. Separate algorithms derive SWE for open environments, deciduous, coniferous, and sparse forest cover. A final SWE value represents the area-weighted average based on the proportional land cover within each pixel. The combined SSM/I and SMMR time series of dual polarized, multichannel, spaceborne passive microwave brightness temperatures extends back to 1978, providing a lengthy time series for algorithm assessment. In this study, 5-day average (pentad) passive microwave-derived SWE imagery for 18 winter seasons (December, January, February 1978/79 through 1995/96) was compared to SWE estimates taken from a distributed network of surface measurements throughout western Canada.Results indicated both vegetative and snowpack controls on the performance of MSC algorithms. In regions of open and low-density forest cover, the in situ and passive microwave SWE data exhibited both strong agreement and similar levels of interannual variability. In locations where winter season SWE typically exceeded 75 mm, and/or dense vegetative cover was present, dataset agreement weakened appreciably, with little interannual variability in the passive microwave SWE retrievals. These results have important implications for extending the SWE monitoring capability of the MSC algorithm suite to northern regions such as the Mackenzie River basin.     

Incorporating self-organizing map with text mining techniques for text hierarchy generation

Self-organizing maps (SOM) have been applied on numerous data clustering and visualization tasks and received much attention on their success. One major shortage of classical SOM learning algorithm is the necessity of predefined map topology. Furthermore, hierarchical relationships among data are also difficult to be found. Several approaches have been devised to conquer these deficiencies. In this work, we propose a novel SOM learning algorithm which incorporates several text mining techniques in expanding the map both laterally and hierarchically. On training a set of text documents, the proposed algorithm will first cluster them using classical SOM algorithm. We then identify the topics of each cluster. These topics are then used to evaluate the criteria on expanding the map. The major characteristic of the proposed approach is to combine the learning process with text mining process and makes it suitable for automatic organization of text documents. We applied the algorithm on the Reuters-21578 dataset in text clustering and categorization tasks. Our method outperforms two comparing models in hierarchy quality according to users’ evaluation. It also receives better F1-scores than two other models in text categorization task.     

A psycho-cognitive segmentation of organ donors in Egypt using Kohonen’s self-organizing maps

This study uses self-organizing maps (SOM) to examine the effect of various psychographic and cognitive factors on organ donation in Egypt. SOM is a machine learning method that can be used to explore patterns in large and complex datasets for linear and nonlinear patterns. The results show that major variables affecting organ donation are related to perceived benefits/risks of organ donation, organ donation knowledge, attitudes toward organ donation, and intention to donate organs. The study also shows that SOM models are capable of improving clustering quality while extracting valuable information from multidimensional data.     

Shades of green: A psychographic segmentation of the green consumer in Kuwait using self-organizing maps

This study uses self-organizing maps (SOM) to examine the effect of various psychographic and cognitive factors on green consumption in Kuwait. SOM is a machine learning method that can be used to explore patterns in large and complex datasets for linear and non-linear patterns. The results show that major variables affecting green consumption are related to altruistic values, environmental concern, environmental knowledge, skepticism towards environmental claims, attitudes toward green consumption, and intention to buy green products. The study also shows that SOM models are capable of improving clustering quality while extracting valuable information from multidimensional data.     

A Scalable Parallel Algorithm for Self-Organizing Maps with Applications to Sparse Data Mining Problems

We describe a scalable parallel implementation of the self organizing map (SOM) suitable for data-mining applications involving clustering or segmentation against large data sets such as those encountered in the analysis of customer spending patterns. The parallel algorithm is based on the batch SOM formulation in which the neural weights are updated at the end of each pass over the training data. The underlying serial algorithm is enhanced to take advantage of the sparseness often encountered in these data sets. Analysis of a realistic test problem shows that the batch SOM algorithm captures key features observed using the conventional on-line algorithm, with comparable convergence rates.Performance measurements on an SP2 parallel computer are given for two retail data sets and a publicly available set of census data.These results demonstrate essentially linear speedup for the parallel batch SOM algorithm, using both a memory-contained sparse formulation as well as a separate implementation in which the mining data is accessed directly from a parallel file system. We also present visualizations of the census data to illustrate the value of the clustering information obtained via the parallel SOM method.     

An unsupervised method of classifying remotely sensed images using Kohonen self‐organizing maps and agglomerative hierarchical clustering methods

Unlike conventional unsupervised classification methods, such as K‐means and ISODATA, which are based on partitional clustering techniques, the methodology proposed in this work attempts to take advantage of the properties of Kohonen's self‐organizing map (SOM) together with agglomerative hierarchical clustering methods to perform the automatic classification of remotely sensed images. The key point of the proposed method is to execute the cluster analysis process by means of a set of SOM prototypes, instead of working directly with the original patterns of the image. This strategy significantly reduces the complexity of the data analysis, making it possible to use techniques that have not normally been considered viable in the processing of remotely sensed images, such as hierarchical clustering methods and cluster validation indices. Through the use of the SOM, the proposed method maps the original patterns of the image to a two‐dimensional neural grid, attempting to preserve the probability distribution and topology of the input space. Afterwards, an agglomerative hierarchical clustering method with restricted connectivity is applied to the trained neural grid, generating a simplified dendrogram for the image data. Utilizing SOM statistic properties, the method employs modified versions of cluster validation indices to automatically determine the ideal number of clusters for the image. The experimental results show examples of the application of the proposed methodology and compare its performance to the K‐means algorithm.     

Time series pattern discovery by a PIP-based evolutionary approach

Time series are an important and interesting research field due to their many different applications. In our previous work, we proposed a time-series segmentation approach by combining a clustering technique, discrete wavelet transformation (DWT) and a genetic algorithm to automatically find segments and patterns from a time series. In this paper, we propose a perceptually important points (PIP)-based evolutionary approach, which uses PIP instead of DWT, to effectively adjust the length of subsequences and find appropriate segments and patterns, as well as avoid some problems that arose in the previous approach. To achieve this, an enhanced suitability factor in the fitness function is designed, modified from the previous approach. The experimental results on a real financial dataset show the effectiveness of the proposed approach.     

Global analysis of empirical relations between annual climate and seasonality of NDVI

This paper describes the use of satellite data to calibrate a new climate vegetation greenness relation for global change studies. We examined statistical relations between annual climate indexes (temperature, precipitation, and surface radiation) and seasonal attributes of the AVHRR Normalized Difference Vegetation Index (NDVI) time series for the mid-1980s in order to refine our understanding of intra-annual patterns and global controls on natural vegetation dynamics. Multiple linear regression results using global 1 gridded data sets suggest that three climate indexes: degree days (growing/chilling), annual precipitation total, and an annual moisture index together can account to 70-80% of the geographical variation in the NDVI seasonal extremes (maximum and minimum values) for the calibration year 1984. Inclusion of the same annual climate index values from the previous year explains no substantial additional portion of the global scale variation in NDVI seasonal extremes. The monthly timing of NDVI extremes is closely associated with seasonal patterns in maximum and minimum temperature and rainfall, with lag times of 1 to 2 months. We separated well-drained areas from 1 grid cells mapped as greater than 25% inundated coverage for estimation of both the magnitude and timing of seasonal NDVI maximum values. Predicted monthly NDVI, derived from our climate-based regression equations and Fourier smoothing algorithms, shows good agreement with observed NDVI for several different years at a series of ecosystem test locations from around the globe. Regions in which NDVI seasonal extremes are not accurately predicted are mainly high latitude zones, mixed and disturbed vegetation types, and other remote locations where climate station data are sparse.     

Impact of Fuzzy Normalization on Clustering Microarray Temporal Datasets Using Cuckoo Search

Microarrays have reformed biotechnological research in the past decade. Deciphering the hidden patterns in gene expression data proffers a prodigious preference to strengthen the understanding of functional genomics. The complexity of biological networks with larger volume of genes also increases the challenges of comprehending and interpretation of the resulting mass of data. Clustering addresses these challenges, which is essential in the data mining process to reveal natural structures and identify interesting patterns in the underlying data. The clustering of gene expression data has been proven to be useful in making known the natural structure inherent in gene expression data, understanding gene functions, cellular processes, and molecular functions. Clustering techniques are used to examine gene expression data to extract groups of genes from the tested samples based on a similarity criterion. Subspace clustering broadens the traditional clustering by extracting the groups of genes that are highly correlated in different subspace within the dataset. Mining the temporal patterns in high dimensional data is done with computational effort and thus normalization is needed. In this work, normalization using fuzzy logic is applied to the data before clustering. The multi-objective cuckoo search optimization is implemented to extract co-expressed genes over different subspaces. The proposed methods are applied to the real life temporal gene expression datasets in which it extracts the genes that are responsible for the disease grouped in a same cluster. The experiment results prove that the impact of fuzzy normalization on the dataset improves the clustering.     

基于GHSOM网络的时间序列聚类方法

提出了一种基于增长型分层自组织映射(GHSOM)的时间序列聚类方法，给出了该方法的基本原理和具体算法步骤，对实测时间序列数据进行了聚类验证和分析。研究结果表明，增长型分层自组织映射能根据对象特征无监督地对时间序列进行正确聚类，由于具有动态增长及分层特性，能分析对象内在的层次结构并实现由粗到精的聚类，可以扩展应用于大型乃至巨量时间序列数据库的模式发现。     

Cluster ensembles in collaborative filtering recommendation

Recommender systems, which recommend items of information that are likely to be of interest to the users, and filter out less favored data items, have been developed. Collaborative filtering is a widely used recommendation technique. It is based on the assumption that people who share the same preferences on some items tend to share the same preferences on other items. Clustering techniques are commonly used for collaborative filtering recommendation. While cluster ensembles have been shown to outperform many single clustering techniques in the literature, the performance of cluster ensembles for recommendation has not been fully examined. Thus, the aim of this paper is to assess the applicability of cluster ensembles to collaborative filtering recommendation. In particular, two well-known clustering techniques (self-organizing maps (SOM) and k-means), and three ensemble methods (the cluster-based similarity partitioning algorithm (CSPA), hypergraph partitioning algorithm (HGPA), and majority voting) are used. The experimental results based on the Movielens dataset show that cluster ensembles can provide better recommendation performance than single clustering techniques in terms of recommendation accuracy and precision. In addition, there are no statistically significant differences between either the three SOM ensembles or the three k-means ensembles. Either the SOM or k-means ensembles could be considered in the future as the baseline collaborative filtering technique.