Observations Data Model 2: A community information model for spatially discrete Earth observations

Integrated access to and analysis of data for cross-domain synthesis studies are hindered because common characteristics of observational data, including time, location, provenance, methods, and units are described differently within different information models, including physical implementations and exchange schema. We describe a new information model for spatially discrete Earth observations called the Observations Data Model Version 2 (ODM2) aimed at facilitating greater interoperability across scientific disciplines and domain cyberinfrastructures. ODM2 integrates concepts from ODM1 and other existing cyberinfrastructures to expand capacity to consistently describe, store, manage, and encode observational datasets for archival and transfer over the Internet. Compared to other systems, it accommodates a wider range of observational data derived from both sensors and specimens. We describe the identification of community information requirements for ODM2 and then present the core information model and demonstrate how it can be formally extended to accommodate a range of information requirements and use cases.     

A framework for user driven data management

Scientists within the materials engineering community produce a wide variety of data, with datasets differing in size and complexity. Examples include large 3D volume densitometry files (voxel) generated by microfocus computer tomography (μCT) and simple text files containing results from a tensile test. Increasingly, there is a need to share this data as part of international collaborations. The design of a suitable database schema and the architecture of a system that can cope with the varying information is a continuing problem in the management of heterogeneous data. We present a model flexible enough to meet users’ diverse requirements. Metadata is held using a database and its design allows users to control their own data structures. Data is held in a file store which, in combination with the metadata, gives huge flexibility. Using examples from materials engineering we illustrate how the model can be applied.     

A simulation tool for managing environmental flows in regulated rivers

Environmental flows provide river flow regimes to restore and conserve aquatic ecosystems, creating considerably different demands compared to conventional water extraction. With increasing incorporation of environmental flows in water planning worldwide, governments require decision support tools to manage these flows in regulated rivers. We developed the Environmental Water Allocation Simulator with Hydrology (eWASH), a fast, flexible and user-friendly scenario-based hydrological modelling tool, supporting environmental flow management decisions for single- or multi-reservoir systems. Environmental flow demands and management rules are easily specified via the graphical user interface, and batch processing functions aid in uncertainty assessment. eWASH modelled main processes of complex regulated rivers and the tool is widely applicable. We calibrated eWASH for the Gwydir and Macquarie Rivers of Australia's Murray–Darling Basin. Modelled monthly environmental flow allocations exhibited Nash–Sutcliffe efficiencies of 0.55 for the Gwydir and 0.72 for the Macquarie catchments respectively when validated.     

Automatic processing, quality assurance and serving of real-time weather data

Recent advances in technology have produced a significant increase in the availability of free sensor data over the Internet. With affordable weather monitoring stations now available to individual meteorology enthusiasts, a reservoir of real time data such as temperature, rainfall and wind speed can now be obtained for most of the world. Despite the abundance of available data, the production of usable information about the weather in individual local neighbourhoods requires complex processing that poses several challenges.This paper discusses a collection of technologies and applications that harvest, refine and process this data, culminating in information that has been tailored toward the user. In this instance, this allows a user to make direct queries about the weather at any location, even when this is not directly instrumented, using interpolation methods provided by the INTAMAP project. A simplified example illustrates how the INTAMAP web processing service can be employed as part of a quality control procedure to estimate the bias and residual variance of user contributed temperature observations, using a reference standard based on temperature observations with carefully controlled quality. We also consider how the uncertainty introduced by the interpolation can be communicated to the user of the system, using UncertML, a developing standard for uncertainty representation.     

DOLAP data warehouse research over two decades: Trends and challenges

This paper introduces to the Information Systems special issue, including the four best papers submitted to DOLAP 2018. Additionally, the 20th anniversary of DOLAP motivated the analysis of DOLAP topics, as follows. First, the recent 5-years DOLAP topics were confronted with those of VLDB, SIGMOD, and ICDE. Next, the DOLAP topics were analyzed within its 20 years of history. Finally, the analysis is concluded with the list of the most frequent research topics of the aforementioned conferences and still open research problems.     

A fuzzy object-oriented data model for managing vague and uncertain information

In this paper, a fuzzy Object-Oriented Data model (FOOD) is defined based on the extension of a Graph-based Object model (D. Lucarella and A. Zanzi “A graph-oriented data model,” in Database and Expert Systems Applications, R. Wagner and H. Toma, Eds., Springer-Verlag, Berlin, 1996, pp. 197–206), in order to manage both crisp and imperfect information. These capabilities are requisites of many current applications dealing with data of different nature and with complex interrelationships. The model is based on a visual paradigm which supports both the representation of the data semantics and the direct browsing of the information. In the extended model both the database scheme and instances are represented as directed labeled graphs in which the fuzzy and uncertain information has its own representation. ©1999 John Wiley & Sons, Inc.     

Towards an integrated Flood Information System: Centralized data access,analysis, and visualization

The Iowa Flood Information System (IFIS) is a web-based platform developed at the Iowa Flood Center (IFC) in order to provide access to flood inundation maps, real-time flood conditions, flood forecasts, flood-related data, information, applications, and interactive visualizations for communities in Iowa. The IFIS provides community-centric watershed and river characteristics, rainfall conditions, and stream-flow data and visualization tools. Interactive interfaces allow access to inundation maps for different stage and return period values as well as to flooding scenarios with contributions from multiple rivers. Real-time and historical data of water levels, gauge heights, hourly and seasonal flood forecasts, and rainfall conditions are made available by integrating data from NEXRAD radars, IFC stream sensors, and USGS and National Weather Service (NWS) stream gauges. The IFIS provides customized flood-related data, information, and visualization for over 1000 communities in Iowa. To help reduce the damage from floods, the IFIS helps communities make better-informed decisions about the occurrence of floods and alerts communities in advance using NWS and IFC forecasts. The integrated and modular design and structure of the IFIS allows easy adaptation of the system in other regional and scientific domains. This paper provides an overview of the design and capabilities of the IFIS that was developed as a platform to provide one-stop access to flood-related information.     

A management system for dirty data

A computer system for managing error prone and complex time-series data is described. The system processes statistics on the catches and fishing operations of fishermen operating in New Zealand waters. These statistics are used primarily for fisheries management and, for this purpose, many years of data may be required. Errors in the data can seriously compromise management and must be weeded out as far as possible. The system is organized around four groups of files: an input file in which data are held temporarily for error checking and correction; a master file which provides medium-term storage for the basic data; derived from this, several summary files tailored to the needs of different user groups; and, lastly, archival files in which the basic data are stored in the long term. The most significant feature of the system is providing effective and comprehensive error detection and correction, and in this is has proved most successful.     

A tool for mapping and spatio-temporal analysis of hydrological data

There is a need in water sciences for computational tools to integrate large spatially distributed datasets to provide insight into the spatial and temporal domains of the data while allowing visualization, analysis in the spatial and temporal dimensions, data metrics, and pattern recognition in the same application. Spatial and temporal variability of hydrological processes as well as the associated phenomena transport is better represented in high spatio-temporal resolution datasets. A conceptual data model and analysis tool, SPELLmap, was developed at the USDA Agricultural Research Service, Grazinglands Research Laboratory using the Delphi programming language to rapidly process, manipulate, analyze, and visualize large geo-located datasets. SPELLmap integrates the spatial and temporal domains of hydrological data to perform analyses in space and time while providing data metrics. SPELLmap has the capacity to represent three or four dimensional problems using a layer data structure. Three examples to illustrate SPELLmap functionalities were provided for the raster and raster-to-network domains. SPELLmap can be used for data interpolation, visualization, gridding, pattern recognition, and data metrics in integrated environmental modeling problems.     

AusPlots Rangelands field data collection and publication: Infrastructure for ecological monitoring

The TERN AusPlots Rangelands field data collection system has been developed to facilitate simple and efficient data collection by ecologists operating in the Australian outback. The infrastructure provides tooling for ‘clean’ data collection on mobile (tablet) devices, associated data storage in a cloud-based server infrastructure, facilities for data curation and management and interfaces with the Australian Ecological Knowledge and Observation System (ÆKOS) data repository for long-term data management and semantic enrichment. In this paper, we introduce the AusPlots Rangelands field data collection solution, providing a systems-level view and motivating its development through the discussion of key functional requirements. We provide an outline of the ÆKOS data repository and demonstrate that the combined system provides a unique end-to-end data collection, curation, archiving and publishing mechanism for ecological data.     

同义词库在公安大数据组织中的研究与应用

论文分析了公安大数据发展趋势、大数据全生命周期管理、同义词库研究现状,提出了一种数据元、同义词、字段关联方法,应用到上海市公安局数据标准管理系统,详细介绍了系统功能、整体架构设计,提出了系统设计的三个创新点。     

A research agenda for query processing in large-scale peer data management systems

Peer Data Management Systems (Pdms) are a novel, useful, but challenging paradigm for distributed data management and query processing. Conventional integrated information systems have a hierarchical structure with an integration component that manages a global schema and distributes queries against this schema to the underlying data sources. Pdms are a natural extension to this architecture by allowing each participating system (peer) to act both as a data source and as an integrator. Peers are interconnected by schema mappings, which guide the rewriting of queries between the heterogeneous schemas, and thus form a P2P (peer-to-peer)-like network.Despite several years of research, the development of efficient Pdms still holds many challenges. In this article we first survey the state of the art on peer data management: We classify Pdms by characteristics concerning their system model, their semantics, their query planning schemes, and their maintenance. Then we systematically examine open research directions in each of those areas. In particular, we observe that research results from both the domain of P2P systems and of conventional distributed data management can have an impact on the development of Pdms.     

A data cloning algorithm for computing maximum likelihood estimates in spatial generalized linear mixed models

Non-Gaussian spatial data are common in many sciences such as environmental sciences, biology and epidemiology. Spatial generalized linear mixed models (SGLMMs) are flexible models for modeling these types of data. Maximum likelihood estimation in SGLMMs is usually made cumbersome due to the high-dimensional intractable integrals involved in the likelihood function and therefore the most commonly used approach for estimating SGLMMs is based on the Bayesian approach. This paper proposes a computationally efficient strategy to fit SGLMMs based on the data cloning (DC) method suggested by Lele et al. (2007). This method uses Markov chain Monte Carlo simulations from an artificially constructed distribution to calculate the maximum likelihood estimates and their standard errors. In this paper, the DC method is adapted and generalized to estimate SGLMMs and some of its asymptotic properties are explored. Performance of the method is illustrated by a set of simulated binary and Poisson count data and also data about car accidents in Mashhad, Iran. The focus is inference in SGLMMs for small and medium data sets.     

A methodology and tool support for managing business rules in organisations

Business rules are evidently important for organisations as they describe how they are doing business. Their value has also been recognised within the information system (IS) domain, mostly because of their ability to make applications flexible and amenable to change. In this paper, we propose a methodology that helps business people and developers to keep business rules at the business level inline with the rules that are implemented at the system level. In contrast to several existing approaches that primarily focus on business rules in the scope of an application, our methodology addresses the entire IS of an organisation. The paper also describes requirements for a tool support that would be appropriate to support the methodology.     

Information ecology: open system environment for data, memories, and knowing

An information ecology provides a conceptual framework to consider data, the creation of knowledge, and the flow of information within a multidimensional context. This paper, reporting on a 1 year project to study the heterogeneity of information and its management within the Long Term Ecological Research (LTER) community, presents some manifestations of traditionally unreported ‘invisible work’ and associated elements of informal knowledge and unarticulated information. We draw from a range of ethnographic materials to understand ways in which data-information-knowledge are viewed within the community and consider some of the non-linear aspects of data-knowledge-information that relate to the development of a sustained, robust, persistent infrastructure for data collection in environmental science research. Taking data as the unit of study, the notion of long-term research and data holdings leads to consideration of types of memory and of knowledge important for design of cyberinfrastructures. Complexity, ambiguity, and nonlinearity are part of an information ecology and addressed today by exploring multiple types of knowledge, developing information system vocabularies, and recognizing the need for intermediation.     

Efficient selection, storage, and retrieval of irregularly distributed elevation data

The advantages and disadvantages of various digital terrain models are discussed briefly and a new model for triangulating a set of nonuniformly distributed three-dimensional surface observations is described. An algorithm for hierarchical subdivision of a set of data points into nested triangles is proposed. The algorithm selects a subset of the data points which reduce the maximum error between a piecewise linear approximation of the surface using only the selected points and the elevations of the points not selected. The data structure used is such that for any given degree of approximation (in the maximum-error sense) only the necessary points need to be stored. Furthermore, an efficient method is available to approximate the elevation of the surface at any point not included in the original data. The performance of the algorithm is demonstrated experimentally.     

Network traffic classification for data fusion: A survey

Traffic classification groups similar or related traffic data, which is one main stream technique of data fusion in the field of network management and security. With the rapid growth of network users and the emergence of new networking services, network traffic classification has attracted increasing attention. Many new traffic classification techniques have been developed and widely applied. However, the existing literature lacks a thorough survey to summarize, compare and analyze the recent advances of network traffic classification in order to deliver a holistic perspective. This paper carefully reviews existing network traffic classification methods from a new and comprehensive perspective by classifying them into five categories based on representative classification features, i.e., statistics-based classification, correlation-based classification, behavior-based classification, payload-based classification, and port-based classification. A series of criteria are proposed for the purpose of evaluating the performance of existing traffic classification methods. For each specified category, we analyze and discuss the details, advantages and disadvantages of its existing methods, and also present the traffic features commonly used. Summaries of investigation are offered for providing a holistic and specialized view on the state-of-art. For convenience, we also cover a discussion on the mostly used datasets and the traffic features adopted for traffic classification in the review. At the end, we identify a list of open issues and future directions in this research field.