Maximizing data locality in distributed systems

The effectiveness of a distributed system hinges on the manner in which tasks and data are assigned to the underlying system resources. Moreover, today's large-scale distributed systems must accommodate heterogeneity in both the offered load and in the makeup of the available storage and compute capacity. The ideal resource assignment must balance the utilization of the underlying system against the loss of locality incurred when individual tasks or data objects are fragmented among several servers. In this paper we describe this locality-maximizing placement problem and show that an optimal solution is NP-hard. We then describe a polynomial-time algorithm that generates a placement within an additive constant of two from optimal.     

Treating data privacy in distributed systems

This papers describes and is primarily concerned with the security data definition and management in a distributed data base of aggregated type, although the approach described may be applied to any distributed system architecture. A multi-level logical security architecture is presented reflecting the logical architecture of the distributed system. In particular, three security logical schemata are proposed: the network security schema, the external security schemata, and the intermediate security schemata. For each schema data models are introduced, allowing the definition and the management of security information. Mapping rules between the logical levels are discussed. Finally security mechanisms are analyzed.     

Materialisation and data partitioning algorithms for distributed RDF systems

Many RDF systems support reasoning with Datalog rules via materialisation, where all conclusions of RDF data and the rules are precomputed and explicitly stored in a preprocessing step. As the amount of RDF data used in applications keeps increasing, processing large datasets often requires distributing the data in a cluster of shared-nothing servers. While numerous distributed query answering techniques are known, distributed materialisation is less well understood. In this paper, we present several techniques that facilitate scalable materialisation in distributed RDF systems. First, we present a new distributed materialisation algorithm that aims to minimise communication and synchronisation in the cluster. Second, we present two new algorithms for partitioning RDF data, both of which aim to produce tightly connected partitions, but without loading complete datasets into memory. We evaluate our materialisation algorithm against two state-of-the-art distributed Datalog systems and show that our technique offers competitive performance, particularly when the rules are complex. Moreover, we analyse in depth the effects of data partitioning on reasoning performance and show that our techniques offer performance comparable or superior to the state of the art min-cut partitioning, but computing the partitions requires considerably less time and memory.     

Synchronization of replicated data in distributed systems

Replication is the key factor in improving the availability of data in distributed systems. Replicated data is stored redundantly at multiple sites so that it can be used by the user even when some of the copies are not available due to site failures. A major restriction in using replication is that replicated copies must behave like a single copy, i.e. mutual consistency as well as internal consistency must be preserved.Synchronization techniques based on the maintenance of multiple versions of data objects have been studied by many researchers in order to increase the degree of concurrency and to reduce the possibility of transaction rollback. Improved availability and increased degree of concurrency would result if multiversion concurrency control techniques can be used with replication control methods in distributed systems. This paper surveys synchronization methods for multiversion distributed systems with replicated data objects that have appeared in the literature. Different synchronization methods are grouped by the underlying mechanisms they use in ordering the operations, and their advantages and disadvantages are discussed. A theory that is used in analyzing the correctness of synchronization methods is reviewed, and a proving method based on the serializability theory is demonstrated using a simple synchronization scheme.     

分布式电网CPS 系统数据攻击下的状态估计

电力物理网络通过构建信息网络进行优化调控并构成信息物理融合系统, 实现大规模分布式系统的优化控制, 随之而来的问题是病毒、黑客入侵、拒绝服务等来自信息网络的威胁, 导致物理系统恶意破坏. 鉴于此, 以攻击可检测为前提, 建立攻击信号下的电力系统分布式动态模型, 设计动态状态估计器检测受攻击的信号, 并估计其原始信号. 最后通过3 机9 节点分布式电网系统仿真实验验证了所设计的状态估计器对于数据攻击检测的有效性.

     

Using execution trace data to improve distributed systems

One of the most challenging problems facing today's software engineer is to understand and modify distributed systems. One reason is that in actual use systems frequently behave differently than the developer intended. In order to cope with this challenge, we have developed a three‐step method to study the run‐time behavior of a distributed system. First, remote procedure calls are traced using CORBA interceptors. Next, the trace data is parsed to construct RPC call‐return sequences, and summary statistics are generated. Finally, a visualization tool is used to study the statistics and look for anomalous behavior. We have been using this method on a large distributed system (more than 500000 lines of code) with data collected during both system testing and operation at a customer's site. Despite the fact that the distributed system had been in operation for over three years, the method has uncovered system configuration and efficiency problems. Using these discoveries, the system support group has been able to improve product performance and their own product maintenance procedures. Copyright © 2002 John Wiley & Sons, Ltd.     

Data access in distributed data base management systems

A software organization which provides for data definition and manipulation in a distributed data base system is presented by describing the functions and interrelations of the component processes; with its methodology for access, the physical location of the data is transparent to the user program. The concepts of distributed data bases are discussed and current research is summarized as a means of establishing a method for the data placement and location mechanism. Procedures for the movement of data in a distributed data base system are presented, along with the data manipulation procedures, in terms of their performance and integrity effects. Enhancements to the mechanisms are suggested.     

An efficient multipath routing for distributed computing systems with data replication

In distributed computing environments, executing a program often requires the access of remote data files. An efficient data routing scheme is thus important for time-critical applications. To ensure a prior desired communication quality, we present a connection-oriented routing scheme, the multipath routing, which allows multiple routes to be established between the source and the destination. Based on the multipath routing scheme, the problem of finding a collection of routing paths for an application to minimize its data transmission time is addressed. Such a problem becomes a complex combinatorial one when the application accesses multiple replicated data sources. Since finding an optimal solution is computationally infeasible in practice, we thus propose a heuristic method to get a sub-optimal solution.     

Maintaining data temporal consistency in distributed real-time systems

Previous works on maintaining temporal consistency of real-time data objects mainly focuses on real-time database systems in which the transmission delays (jitters) of update jobs are simply ignored. However, this assumption does not hold in distributed real-time systems where the jitters of the update jobs can be large and change unpredictably with time. In this paper, we examine the design problems when the More-Less (ML) approach (Xiong and Ramamritham in Proc. of the IEEE real-time systems symposium 1999; IEEE Trans Comput 53:567?C583, 2004), known to be an efficient scheme for maintaining temporal consistency of real-time data objects, is applied in a distributed real-time system environment. We propose two new extensions based on ML, called Jitter-based More-Less (JB-ML) and Statistical Jitter-based More-Less (SJB-ML) to address the jitter problems. JB-ML assumes that in the system the jitter is a constant for each update task, and it provides a deterministic guarantee in temporal consistency of the real-time data objects. SJB-ML further relaxes this restriction and provides a statistical guarantee based on the given QoS requirements of the real-time data objects. We demonstrate through extensive simulation experiments that both JB-ML and SJB-ML are effective approaches and they significantly outperform ML in terms of improving schedulability.     

Reducing complexity of the distributed switch for parallel data processing systems

The paper aimed at demonstrating the feasibility of reducing complexity of the distributed switches interconnecting the units of the data processing systems. Simplification was obtained by the use of optical facilities: to switch n devices, it is possible to design the single-step switches of n log₂ n switching elements assuming any of the two possible states. Retroreflectors were proposed to simplify organization of the group connections. Methods for using the proposed devices in the switching modes that underlie many tasks of control of interaction of the data processing devices were presented.     

A high-performance computing method for data allocation in distributed database systems

Enhancing the performance of the DDBs (Distributed Database system) can be done by speeding up the computation of the data allocation, leading to higher speed allocation decisions and resulting in smaller data redundancy and shorter processing time. This paper deals with an integrated method for grouping the distributed sites into clusters and customizing the database fragments allocation to the clusters and their sites. We design a high speed clustering and allocating method to determine which fragments would be allocated to which cluster and site so as to maintain data availability and a constant systemic reliability, and evaluate the performance achieved by this method and demonstrate its efficiency by means of tabular and graphical representation. We tested our method over different network sites and found it reduces the data transferred between the sites during the execution time, minimizes the communication cost needed for processing applications, and handles the database queries and meets their future needs.     

Improving the flexibility for replicated data management in distributed database systems

This paper presents a novel scheme for implementing the flexible replica control protocol[13] in the distributed database systems. The scheme requires less nodes to be locked to perform the read(write) operations. This not only provides better performance, but also provides the system designer extra flexibility to implement the protocol.     

Authentication for distributed systems

A number of protocols used to authenticate users, hosts and processes are described. The three main types of authentication in a distributed computing system-message content authentication, message origin authentication, and general identity authentication-are explained. Authentication exchanges are identified, and paradigms of authentication protocols are presented. Authentication protocol failures are addressed, and an authentication framework is provided. As case studies, two authentication services, Kerberos and SPX, are examined     

Security for distributed systems

Growing reliance on computer systems means that organizations must ensure those systems are secure from external and internal interference. Government offices operate a very tight method of controlling access to files. Security measures can only work if taken seriously by all involved. Many current crimes would not occur if they were likely to be detected quickly.     

Generalized sampled data feedback control of distributed parameter systems

In the sampled-data control literature there are necessary conditions and sufficient conditions for stabilizability of distributed parameter systems by generalized sampled-data control. For finite-dimensional systems the necessary conditions are also known to be sufficient. We show that this equivalence extends to the infinite-dimensional case if the underlying semigroup is analytic. However, for general systems, the necessary conditions are not sufficient, nor are the sufficient conditions necessary. We prove this by a single example with a free parameter – one choice of parameter shows that the necessary conditions are too weak, and another choice shows that the sufficient conditions are too strong.     

Modified covariance intersection for data fusion in distributed nonhomogeneous monitoring systems network

Monitoring networks contain monitoring nodes that observe an area of interest to detect any possible existing object and estimate its states. Each node has characteristics such as probability of detection and clutter density that may have different values for distinct nodes in nonhomogeneous monitoring networks. This paper proposes a modified covariance intersection method for data fusion in such networks. It is derived by formulating a mixed game model between neighbor monitoring nodes as players and considering the inverse of the trace of fused covariance matrix as players' utility function. Monitoring nodes estimate the states of any possible existing object by applying joint target detection and tracking filter on their own observations. Processing nodes fuse the estimated states received from neighbor monitoring nodes by the proposed modified covariance intersection. It is validated by simulating target detection and tracking problem in 2 situations: 1 target and unknown number of targets.     

The Catalog Management Strategy of Distributed Data Base Systems

In this paper the catalog management strategy of the successfully integrating and running DDBMS C-POREL is summarized.The new catalog management strategy and its implementation scheme are based on the analysis of the catalog management methods of the pioneer DDBMS.The goal of the new strategy is to improve the system efficiency.Analysis and practice show that this strategy is successful.