A view-dependent spatiotemporal saliency-driven approach for time varying volumetric data in geovisualization

Geospatial datasets from satellite observations and model simulations are becoming more accessible. These spatiotemporal datasets are relatively massive for visualization to support advanced analysis and decision making. A challenge to visualizing massive geospatial datasets is identifying critical spatial and temporal changes reflected in the data while maintaining high interactive rendering speed, even when data are accessed remotely. We propose a view-dependent spatiotemporal saliency-driven approach that facilitates the discovery of regions showing high levels of spatiotemporal variability and reduces the rendering intensity of interactive visualization. Our method is based on a novel definition of data saliency, a spatiotemporal tree structure to store visual saliency values, as well as a saliency-driven view-dependent level-of-detail (LOD) control. To demonstrate its applicability, we have implemented the approach with an open-source remote visualization package and conducted experiments with spatiotemporal datasets produced by a regional dust storm simulation model. The results show that the proposed method may not be outstanding in some specific situations, but it consistently performs very well across different settings according to different criteria.     

Multimodal fusion of biomedical data at different temporal and dimensional scales

The introduction of integrative approaches to biomedical research (integrative biology, physiome, Virtual Physiological Human, etc.) poses original problems to computer aided medicine: the need to operate with large amounts of data that are strongly heterogeneous in structure, format and even in the knowledge domain that generated them; the need to integrate all of these data into a coherent whole; the further complication imposed by the fact that more and more frequently these data are captured at very different dimensional and/or temporal scales. The present study describes a first attempt at providing an interactive visualisation environment for homogeneous biomedical data defined over radically different spatial or temporal scales. In particular, we describe new strategies for the management of the dimensional information of highly heterogeneous data types; the management of temporal multiscaling; for 3D unstructured spatial multiscale visualisation and the related interaction paradigms and user interface. Preliminary results with a prototype implementation based on the OpenMAF application framework (http://www.openmaf.org) indicate that it is possible to develop effective environments for interactive visualisation of multiscale biomedical data.     

Spatial and semantical label inference for social media

Exploring the spatial and semantical knowledge from messages in social media offers us an opportunity to get a deeper understanding about the mobility and activity of users, which can be leveraged to improve the service quality of online applications like recommender systems. In this paper, we investigate the problem of the spatial and semantical label inference, where the challenges come from three aspects: diverse heterogeneous information, uncertainty of individual mobility, and large-scale sparse data. We address the challenges by exploring two types of data fusion, the fusion of heterogeneous social networks and the fusion of heterogeneous features. We build a 4-dimensional tensor, called spatial–temporal semantical tensor (STST), to model the individual mobility and activity by fusing two heterogeneous social networks, a social media network and a location-based social network (LBSN). To address the challenge arising from diverse heterogeneous information and the uncertainty of individual mobility, we construct three types of heterogeneous features and fuse them with STST by exploring their interdependency relationships. Particularly, a spatial tendency feature is constructed to constrain the inference of individual mobility and reduce the uncertainty. To deal with large-scale sparse data, we propose a parallel contextual tensor factorization (PCTF) to concurrently factorize STST. Finally, we integrate these components into an inference framework, called spatial and semantical label inference SSLI. The results of extensive experiments conducted on real datasets and synthetic datasets verify the effectiveness and efficiency of SSLI.     

Flowstrates: An Approach for Visual Exploration of Temporal Origin‐Destination Data

Many origin‐destination datasets have become available in the recent years, e.g. flows of people, animals, money, material, or network traffic between pairs of locations, but appropriate techniques for their exploration still have to be developed. Especially, supporting the analysis of datasets with a temporal dimension remains a significant challenge. Many techniques for the exploration of spatio‐temporal data have been developed, but they prove to be only of limited use when applied to temporal origin‐destination datasets. We present Flowstrates , a new interactive visualization approach in which the origins and the destinations of the flows are displayed in two separate maps, and the changes over time of the flow magnitudes are represented in a separate heatmap view in the middle. This allows the users to perform spatial visual queries, focusing on different regions of interest for the origins and destinations, and to analyze the changes over time provided with the means of flow ordering, filtering and aggregation in the heatmap. In this paper, we discuss the challenges associated with the visualization of temporal origin‐destination data, introduce our solution, and present several usage scenarios showing how the tool we have developed supports them.     

Continuous range-based skyline queries in road networks

With the ever-growing popularity of smartphone devices in recent years, skyline queries over spatial Web objects in road networks have received increasing attention. In the literature, various techniques have been developed to tackle skyline queries that take both spatial and non-spatial attributes into consideration. However, the existing solutions only focus on solving point-based queries, where the query location is a spatial point. We observe that in many real-life applications, the user location is often represented by a spatial range. Thus, in this paper, we study a new problem of range-based skyline queries (CRSQs) in road networks. Two efficient algorithms named landmark-based (LBA) and index-based (IBA) algorithms are proposed. We also present incremental versions of LBA and IBA to handle continuous range-based skyline queries over moving objects. Extensive experiments using real road network datasets demonstrate the effectiveness and efficiency of our proposed algorithms.     

Distributed system anomaly detection using deep learning-based log analysis

Anomaly detection is a key step in ensuring the security and reliability of large-scale distributed systems. Analyzing system logs through artificial intelligence methods can quickly detect anomalies and thus help maintenance personnel to maintain system security. Most of the current works only focus on the temporal or spatial features of distributed system logs, and they cannot sufficiently extract the global features of distributed system logs to achieve a good correct rate of anomaly detection. To further address the shortcomings of existing methods, this paper proposes a deep learning model with global spatiotemporal features to detect the presence of anomalies in distributed system logs. First, we extract semi-structured log events from log templates and model them as natural language. In addition, we focus on the temporal characteristics of logs using the bidirectional long short-term memory network and the spatial invocation characteristics of logs using the Transformer. Extensive experimental evaluations show the advantages of our proposed model for distributed system log anomaly detection tasks. The optimal F1-Score on three open-source datasets and our own collected distributed system datasets reach 98.04%, 94.34%, 88.16%, and 97.40%, respectively.     

Visual Analysis of Multi‐Joint Kinematic Data

Kinematics is the analysis of motions without regarding forces or inertial effects, with the purpose of understanding joint behaviour. Kinematic data of linked joints, for example the upper extremity, i.e. the shoulder and arm joints, contains many related degrees of freedom that complicate numerical analysis. Visualisation techniques enhance the analysis process, thus improving the effectiveness of kinematic experiments. This paper describes a new visualisation system specifically designed for the analysis of multi‐joint kinematic data of the upper extremity. The challenge inherent in the data is that the upper extremity is comprised of five cooperating joints with a total of fifteen degrees of freedom. The range of motion may be affected by subtle deficiencies of individual joints that are difficult to pinpoint. To highlight these subtleties our approach combines interactive filtering and multiple visualisation techniques. Our system is further differentiated by the fact that it integrates simultaneous acquisition and visual analysis of biokinematic data. Also, to facilitate complex queries, we have designed a visual query interface with visualisation and interaction elements that are based on the domain‐specific anatomical representation of the data. The combination of these techniques form an effective approach specifically tailored for the investigation and comparison of large collections of kinematic data. This claim is supported by an evaluation experiment where the technique was used to inspect the kinematics of the left and right arm of a patient with a healed proximal humerus fracture, i.e. a healed shoulder fracture.     

A systematic review of shared visualisation to achieve common ground

This paper reports a systematic review of shared visualisation based on fifteen papers from 2000 to 2013. The findings identified five shared visualisation strategies that represent the ways implemented to process data sharing and knowledge to arrive at the desired level of understanding. Four visualisation techniques were also identified to show how shared cognition is made possible in designing tools for mediating data or knowledge among the users involved. These findings provide research opportunities in integrating rich interactive data visualisation for mobile-based technologies as an effective mean in supporting collaborative work. Finally, social, task and cognitive elements which can be significantly supported by shared visualisation and a guideline for future researchers seeking to design shared visualisation-based systems are presented.     

Towards insight-driven sampling for big data visualisation

ABSTRACT

Creating an interactive, accurate, and low-latency big data visualisation is challenging due to the volume, variety, and velocity of the data. Visualisation options range from visualising the entire big dataset, which could take a long time and be taxing to the system, to visualising a small subset of the dataset, which could be fast and less taxing to the system but could also lead to a less-beneficial visualisation as a result of information loss. The main research questions investigated by this work are what effect sampling has on visualisation insight and how to provide guidance to users in navigating this trade-off. To investigate these issues, we study an initial case of simple estimation tasks on histogram visualisations of sampled big data, in hopes that these results may generalise. Leveraging sampling, we generate subsets of large datasets and create visualisations for a crowd-sourced study involving a simple cognitive visualisation task. Using the results of this study, we quantify insight, sampling, visualisation, and perception error in comparison to the full dataset. We use these results to model the relationship between sample size and insight error, and we propose the use of our model to guide big data visualisation sampling.     

Configuring Hierarchical Layouts to Address Research Questions

We explore the effects of selecting alternative layouts in hierarchical displays that show multiple aspects of large multivariate datasets, including spatial and temporal characteristics. Hierarchical displays of this type condition a dataset by multiple discrete variable values, creating nested graphical summaries of the resulting subsets in which size, shape and colour can be used to show subset properties. These 'small multiples' are ordered by the conditioning variable values and are laid out hierarchically using dimensional stacking. Crucially, we consider the use of different layouts at different hierarchical levels, so that the coordinates of the plane can be used more effectively to draw attention to trends and anomalies in the data. We argue that these layouts should be informed by the type of conditioning variable and by the research question being explored. We focus on space-filling rectangular layouts that provide data-dense and rich overviews of data to address research questions posed in our exploratory analysis of spatial and temporal aspects of property sales in London. We develop a notation ('HiVE') that describes visualisation and layout states and provides reconfiguration operators, demonstrate its use for reconfiguring layouts to pursue research questions and provide guidelines for this process. We demonstrate how layouts can be related through animated transitions to reduce the cognitive load associated with their reconfiguration whilst supporting the exploratory process.     

In situ ray tracing and computational steering for interactive blood flow simulation

Recent algorithm and hardware developments have significantly improved our capability to interactively visualise time-varying flow fields. However, when visualising very large dynamically varying datasets interactively there are still limitations in the scalability and efficiency of these methods. Here we present a rendering pipeline which employs an efficient in situ ray tracing technique to visualise flow fields as they are simulated. The ray casting approach is particularly well suited for the visualisation of large and sparse time-varying datasets, where it is capable of rendering fluid flow fields at high image resolutions and at interactive frame rates on a single multi-core processor using OpenMP. The parallel implementation of our in situ visualisation method relies on MPI, requires no specialised hardware support, and employs the same underlying spatial decomposition as the fluid simulator. The visualisation pipeline allows the user to operate on a commodity computer and explore the simulation output interactively. Our simulation environment incorporates numerous features that can be utilised in a wide variety of research contexts.     

A data driven anonymization system for information rich online social network graphs

In recent years, online social networks have become a part of everyday life for millions of individuals. Also, data analysts have found a fertile field for analyzing user behavior at individual and collective levels, for academic and commercial reasons. On the other hand, there are many risks for user privacy, as information a user may wish to remain private becomes evident upon analysis. However, when data is anonymized to make it safe for publication in the public domain, information is inevitably lost with respect to the original version, a significant aspect of social networks being the local neighborhood of a user and its associated data. Current anonymization techniques are good at identifying risks and minimizing them, but not so good at maintaining local contextual data which relate users in a social network. Thus, improving this aspect will have a high impact on the data utility of anonymized social networks. Also, there is a lack of systems which facilitate the work of a data analyst in anonymizing this type of data structures and performing empirical experiments in a controlled manner on different datasets. Hence, in the present work we address these issues by designing and implementing a sophisticated synthetic data generator together with an anonymization processor with strict privacy guarantees and which takes into account the local neighborhood when anonymizing. All this is done for a complex dataset which can be fitted to a real dataset in terms of data profiles and distributions. In the empirical section we perform experiments to demonstrate the scalability of the method and the improvement in terms of reduction of information loss with respect to approaches which do not consider the local neighborhood context when anonymizing.     

T-GAN: A deep learning framework for prediction of temporal complex networks with adaptive graph convolution and attention mechanism

Complex network is graph network with non-trivial topological features often occurring in real systems, such as video monitoring networks, social networks and sensor networks. While there is growing research study on complex networks, the main focus has been on the analysis and modeling of large networks with static topology. Predicting and control of temporal complex networks with evolving patterns are urgently needed but have been rarely studied. In view of the research gaps we are motivated to propose a novel end-to-end deep learning based network model, which is called temporal graph convolution and attention (T-GAN) for prediction of temporal complex networks. To joint extract both spatial and temporal features of complex networks, we design new adaptive graph convolution and integrate it with Long Short-Term Memory (LSTM) cells. An encoder-decoder framework is applied to achieve the objectives of predicting properties and trends of complex networks. And we proposed a dual attention block to improve the sensitivity of the model to different time slices. Our proposed T-GAN architecture is general and scalable, which can be used for a wide range of real applications. We demonstrate the applications of T-GAN to three prediction tasks for evolving complex networks, namely, node classification, feature forecasting and topology prediction over 6 open datasets. Our T-GAN based approach significantly outperforms the existing models, achieving improvement of more than 4.7% in recall and 25.1% in precision. Additional experiments are also conducted to show the generalization of the proposed model on learning the characteristic of time-series images. Extensive experiments demonstrate the effectiveness of T-GAN in learning spatial and temporal feature and predicting properties for complex networks.     

Incremental and interactive mining of web traversal patterns

Web mining involves the application of data mining techniques to large amounts of web-related data in order to improve web services. Web traversal pattern mining involves discovering users’ access patterns from web server access logs. This information can provide navigation suggestions for web users indicating appropriate actions that can be taken. However, web logs keep growing continuously, and some web logs may become out of date over time. The users’ behaviors may change as web logs are updated, or when the web site structure is changed. Additionally, it can be difficult to determine a perfect minimum support threshold during the data mining process to find interesting rules. Accordingly, we must constantly adjust the minimum support threshold until satisfactory data mining results can be found.The essence of incremental data mining and interactive data mining is the ability to use previous mining results in order to reduce unnecessary processes when web logs or web site structures are updated, or when the minimum support is changed. In this paper, we propose efficient incremental and interactive data mining algorithms to discover web traversal patterns that match users’ requirements. The experimental results show that our algorithms are more efficient than other comparable approaches.     

Extensions of parallel coordinates for interactive exploration of large multi-timepoint data sets

Parallel coordinate plots (PCPs) are commonly used in information visualization to provide insight into multi-variate data. These plots help to spot correlations between variables. PCPs have been successfully applied to unstructured datasets up to a few millions of points. In this paper, we present techniques to enhance the usability of PCPs for the exploration of large, multi-timepoint volumetric data sets, containing tens of millions of points per timestep. The main difficulties that arise when applying PCPs to large numbers of data points are visual clutter and slow performance, making interactive exploration infeasible. Moreover, the spatial context of the volumetric data is usually lost. We describe techniques for preprocessing using data quantization and compression, and for fast GPU-based rendering of PCPs using joint density distributions for each pair of consecutive variables, resulting in a smooth, continuous visualization. Also, fast brushing techniques are proposed for interactive data selection in multiple linked views, including a 3D spatial volume view. These techniques have been successfully applied to three large data sets: Hurricane Isabel (Vis'04 contest), the ionization front instability data set (Vis'08 design contest), and data from a large-eddy simulation of cumulus clouds. With these data, we show how PCPs can be extended to successfully visualize and interactively explore multi-timepoint volumetric datasets with an order of magnitude more data points.     

结合混合池化的双流人脸活体检测网络

目的 人脸识别技术在很多领域起着重要作用,但大量的欺诈攻击对人脸识别产生了威胁,比如打印攻击和重放攻击。传统的活体检测方法是以手工方式提取特征且缺乏对时间维度的考虑,导致检测效果不佳。针对以上问题,提出一种结合混合池化的双流活体检测网络。方法 对数据集提取光流图像并进行面部检测,得到双流网络的两个输入;在双流网络末端加入空间金字塔和全局平均混合池化,利用全连接层对池化后的特征进行分类并进行分数层面的融合;对空间流网络和时间流网络进行融合得到一个最优结果,同时考虑了不同颜色空间对检测性能的影响。结果 在CASIA-FASD （CASIA face anti-spoofing database）和replay-attack两个数据集上做了多组对比实验,在CASIA-FASD数据集上,等错误率（equal error rate,EER）为1.701%;在replay-attack数据集上,等错误率和半错误率（half total error rate,HTER）分别为0.091%和0.082%。结论 结合混合池化的双流活体检测网络充分考虑时间维度,提出的空间金字塔和全局平均混合池化策略能有效地利用特征。针对包含多种攻击类型、图像质量差异较大的数据集,本文提出的网络模型均能取得较低的错误率。     

Quantitative Evaluation of Time-Dependent Multidimensional Projection Techniques

Dimensionality reduction methods are an essential tool for multidimensional data analysis, and many interesting processes can be studied as time-dependent multivariate datasets. There are, however, few studies and proposals that leverage on the concise power of expression of projections in the context of dynamic/temporal data. In this paper, we aim at providing an approach to assess projection techniques for dynamic data and understand the relationship between visual quality and stability. Our approach relies on an experimental setup that consists of existing techniques designed for time-dependent data and new variations of static methods. To support the evaluation of these techniques, we provide a collection of datasets that has a wide variety of traits that encode dynamic patterns, as well as a set of spatial and temporal stability metrics that assess the quality of the layouts. We present an evaluation of 9 methods, 10 datasets, and 12 quality metrics, and elect the best-suited methods for projecting time-dependent multivariate data, exploring the design choices and characteristics of each method. Additional results can be found in the online benchmark repository. We designed our evaluation pipeline and benchmark specifically to be a live resource, open to all researchers who can further add their favorite datasets and techniques at any point in the future.     

A visualisation technique for large temporal social network datasets in Hyperbolic space

Visualisations of temporal social network datasets have the potential to be complex and require a lot of cognitive input. In this paper, we present a novel visualisation approach that depicts both relational and statistical information of evolving social structures. The underlying framework is implemented by the usage of Hyperbolic Geometry to support focus context rendering. The proposed method guarantees representing prominent social actors through scaling their representations, preserves user's mental map, and provides the user to reduce visual clutter by means of filtering.     

基于时空循环神经网络的下一个兴趣点推荐方法

下一个兴趣点推荐已经成为基于位置的社交网络(location-based social networks,LBSNs)中一个重要任务。现有的模型没有深入考虑相邻签到兴趣点之间的转移时空信息,无法对用户访问下一个兴趣点的长短时间偏好和远近距离偏好进行有效建模。本文通过对循环神经网络(recurrent neural network, RNN)进行扩展,提出一个新的基于会话的时空循环神经网络模型(sesson-based spatial-temporal recurrent neural network, SST-RNN)用于下一个兴趣点推荐。该模型通过设置时间转移矩阵和空间转移矩阵分别对用户的时间和空间偏好信息进行建模,综合考虑连续签到兴趣点的序列信息、时空信息以及用户偏好进行下一个兴趣点推荐。通过在2个真实公开的数据集上进行实验,结果显示本文提出的SST-RNN模型的推荐效果比主流的推荐模型有显著提升。在Foursquare和CA数据集上,ACC@5评价指标分别提升了36.38%和13.81%,MAP评价指标分别提升了30.72%和17.26%。     

Real-time recovery of moving 3D faces for emerging applications

Existing face imaging systems are not suitable to meet the face representation and recognition demands for emerging applications in areas such as interactive gaming, enhanced learning environments and directed advertising. This is mainly due to the poor capture and characterisation of facial data that compromises their spatial and temporal precision. For emerging applications it is not only necessary to have a high level of precision for the representation of facial data, but also to characterise dynamic faces as naturally as possible and in a timely manner. This study proposes a new framework for capturing and recovering dynamic facial information in real-time at significantly high order of spatial and temporal accuracy to capture and model subtle facial changes for enhanced realism in 3D face visualisation and higher precision for face recognition applications. We also present a novel, fast, and robust correspondence mapping approach for 3D registration of moving 3D faces.