Dataflow architectures and multithreading

Contrary to initial expectations, implementing dataflow computers has presented a monumental challenge. Now, however, multithreading offers a viable alternative for building hybrid architectures that exploit parallelism. The eventual success of dataflow computers will depend on their programmability. Traditionally, they've been programmed in languages such as Id and SISAL (Streams and Iterations in a Single Assignment Language) that use functional semantics. These languages reveal high levels of concurrency and translate on to dataflow machines and conventional parallel machines via the Threaded Abstract Machine (TAM). However, because their syntax and semantics differ from the imperative counterparts such as Fortran and C, they have been slow to gain acceptance in the programming community. An alternative is to explore the use of established imperative languages to program dataflow machines. However, the difficulty will be analyzing data dependencies and extracting parallelism from source code that contains side effects. Therefore, more research is still needed to develop compilers for conventional languages that can produce parallel code comparable to that of parallel functional languages     

Architecture-independent parallel computation

The major parallel architecture classes are considered: single-instruction multiple-data (SIMD) computers, tightly coupled multiple-instruction multiple-data (MIMD) computers, hypercuboid computers and constant-valence MIMD computers. An argument that the PRAM model is universal over tightly coupled and hypercube systems, but not over constant-valence-topology, loosely coupled-system is reviewed, showing precisely how the PRAM model is too powerful to permit broad universality. Ways in which a model of computation can be restricted to become universal over less powerful architectures are discussed. The Bird-Meertens formalism (R.S. Bird, 1989), is introduced and it is shown how it is used to express computations in a compact way. It is also shown that the Bird-Meertens formalism is universal over all four architecture classes and that nontrivial restrictions of functional programming languages exist that can be efficiently executed on disparate architectures. The use of the Bird-Meertens formalism as the basis for a programming language is discussed, and it is shown that it is expressive enough to be used for general programming. Other models and programming languages with architecture-independent properties are reviewed     

Bounded quantifications for iteration and concurrency in logic programming

The only means for repetition in most logic programming languages, including Prolog, is recursion. Definite iteration is introduced in logic programming languages through the bounded quantification construct. Firstly, it is claimed that this construct is often, though not always, more natural than recursion for expressing relations that involve repetition. In particular, programs involving arrays and similar data structures are significantly simplified. Secondly, it is argued that bounded quantifications should be efficiently implementable on sequential computers and have a high potential for running in parallel, particularly on computers supporting the SPMD model of computation. Bounded quantifications are compared with related constructs from other languages, including the definite loops of imperative languages and the array comprehensions of recent functional languages.     

Programming languages for fifth generation computers

Fifth Generation languages and computers, for use in the 1990's, may be based on significantly different concepts than traditional computing. There are four major categories of computing, each consisting of a programming style and corresponding computer. These are: procedural languages (and control flow computers); object-oriented languages (and actor computers); functional languages (and data flow or reduction computers); and logic languages (and logic computers). In the future any of these categories of languages may become “main-stream” programming styles. In this paper we briefly present these novel programming languages and examine their advantages and disadvantages for programming.     

Parallel context-sensitive compilation

The searching power of massively parallel associative computers is an under used and under investigated capability that can be used to facilitate software development. This paper describes the development of a context sensitive compiler for pattern-matching languages using that searching power. The described compiler was implemented on the STARAN parallel computer and the compiled OPS5 programs were also executed on the STARAN obtaining an estimated throughput of 6000 rules per second. The described compilation of production rules into equivalent procedural rules is completely data parallel, with the degree of parallelism depending on the number of tokens in the program being compiled. During any one step of the context-sensitive analysis, the entire program is processed in constant time.     

Evaluating the performance of a SISAL implementation of the abingdon cross image processing benchmark

There are many paradigms being promoted and explored for programming parallel computers, including modified sequential languages, new imperative languages and applicative languages. SISAL is an applicative language which has been designed by a consortium of industrial and research organizations for the specification and execution of parallel programs. It allows programs to be written with little concern for the structure of the underlying machine, thus the programmer is free to explore different ways of expressing the parallelism. A major problem with applicative languages has been their poor efficiency at handling large data structures. To counter this problem SISAL includes some advanced memory management techniques for reducing the amount of data copying that occurs. In this paper we discuss the implementation of some image processing benchmarks in SISAL and C to evaluate the effectiveness of the memory management code. In general, the SISAL program was easier to code than the C (augmented with the PARMACS macros) because we were not concerned with the parallel implementation details. We found that the SISAL performance was in general comparable to C, and that it could be brought in line with an efficient parallel C implementation by some programmer-specified code transformations.     

Up-to-Date International Standards of the Fortran Programming Language

Standardization of Fortran is considered. An overview of new features of modern Fortran standards and an outline of the future Fortran standard (Fortran 2000) are given. Fortran-based programming languages for parallel computers are discussed.     

Cellular processing tools for high-performance simulation

Cellular automata offer a powerful modeling approach for complex systems in which global behavior arises from the collective effect of many locally interacting, simple components. Several tools based on CA are providing meaningful results for real-world applications. Cellular automata represent an efficient paradigm for the computer solution of important problems in science and engineering. Moreover, the CA model lets researchers effectively use parallel computers to achieve scalable performance. As researchers use parallel computers to solve scientific problems, they will need problem representations (paradigms) for this class of computers. Abstract mathematical models that offer an implicitly parallel representation of problems better match those architectures, but could benefit from new high-level languages, environments, and techniques. The three should support all the development steps of computational science applications while hiding architectural details from users. Computational science is also an interdisciplinary field in which many areas converge, and developing applications in this field requires the cooperation of people from different domains. Modeling and simulation using parallel cellular methods helps researchers cooperate by offering both a way to code an algorithm and an integrated environment for developing software     

The use of combinators in translating a purely functional language to low-level data-flow graphs

FCL is a higher-order functional programming language which consolidates and extends a number of desirable features of existing languages. This paper describes the salient features of FCL and an algorithm for translation to highly parallel data flow graphs. The translation algorithm is based on a set of extended “combinators”. The relationship between functional programming languages and demand-driven or data-driven data flow architectures is established.     

Parallel pivoting algorithms for sparse symmetric matrices

In this paper it is investigated which pivots may be processed simultaneously when solving a set of linear equations. It is shown that for dense sets of equations all the pivots must necessarily be processed one at a time; only if the set is sufficiently sparse, some pivots may be processed simultaneously. We present parallel pivoting algorithms for MIMD computers with sufficiently many processors and a common memory. Moreover we present algorithms for MIMD computers with an arbitrary, but fixed number of processors. For both types of computers algorithms embodying an ordering strategy are given.     

The Hydra Parallel Programming System

The Hydra Parallel Programming System, a new parallel language extension to Java, and its supporting software are described. It is a fairly simple yet powerful language designed to address a number of areas that have not received much attention. One of these areas is the recompilation of parallel programs at runtime to allow a parallel program to adapt to the architecture it is executing on. The first version of this software system focuses on smaller Symmetric Multiprocessing and compatible architectures which are becoming more common. This particular class of machines has a great need for more options in the area of parallel programming among the vastly popular Java language programmers. Hydra programs will run as sequential Java on machines that do not have the parallel support or do not have an implemented Hydra runtime system without requirement of any modifications to the program. This paper describes the language, compares it with other languages (specifically with JOMP, an OpenMP implementation for Java), presents a brief discussion on compiling and executing Hydra programs, presents some sample benchmarks and their performance on three platforms, and concludes with a discussion of issues and future directions for Hydra. Copyright © 2007 John Wiley & Sons, Ltd.     

Computation with multiple CTCs of fixed length and width

We examine some variants of computation with closed timelike curves (CTCs), where various restrictions are imposed on the memory of the computer, and the information carrying capacity and range of the CTC. We give full characterizations of the classes of languages decided by polynomial time probabilistic and quantum computers that can send a single classical bit to their own past. We show that, given a time machine with constant negative delay, one can implement CTC-based computations without the need to know about the runtime beforehand. Chaining multiple instances of such fixed-length CTCs, the power of postselection can be endowed to deterministic computers, all languages in $\mathsf{NP} \cup \mathsf{coNP}$ can be decided with no error in worst-case polynomial time, and all Turing-decidable languages can be decided in constant expected time. We provide proofs of the following facts for weaker models: Augmenting probabilistic computers with a single CTC leads to an improvement in language recognition power. Quantum computers under these restrictions are more powerful than their classical counterparts. Some deterministic models assisted with multiple CTCs are more powerful than those with a single CTC.     

Measurement of the latency parameters of the Multi-BSP model: a multicore benchmarking approach

Computer benchmarking is a common method for measuring the parameters of a computational model. It helps to measure the parameters of any computer. With the emergence of multicore computers, the evaluation of computers was brought under consideration. Since these types of computers can be viewed and considered as parallel computers, the evaluation methods for parallel computers may be appropriate for multicore computers. However, because multicore architectures seriously focus on cache hierarchy, there is a need for new and different benchmarks to evaluate them correctly. To this end, this paper presents a method for measuring the parameters of one of the most famous multicore computational models, namely Multi-Bulk Synchronous Parallel (Multi-BSP). This method measures the hardware latency parameters of multicore computers, namely communication latency (g _i ) and synchronization latency (L _i ) for all levels of the cache memory hierarchy in a bottom-up manner. By determining the parameters, the performance of algorithms on multicore architectures can be evaluated as a sequence.     

User-Friendly Parallel Computations with Econometric Examples

This paper shows how a high-level matrix programming language may be used to perform Monte Carlo simulation, bootstrapping, estimation by maximum likelihood and GMM, and kernel regression in parallel on symmetric multiprocessor computers or clusters of workstations. The implementation of parallelization is done in a way such that an investigator may use the programs without any knowledge of parallel programming. A bootable CD that allows rapid creation of a cluster for parallel computing is introduced. Examples show that parallelization can lead to important reductions in computational time. Detailed discussion of how the Monte Carlo problem was parallelized is included as an example for learning to write parallel programs for Octave. JEL Classifications: C13; C14; C15; C63; C87     

Parallel programming for computer vision

Two Unix environments developed for programming parallel computers to handle image-processing and vision applications are described. Visx is a portable environment for the development of vision applications that has been used for many years on serial computers in research. Visx was adapted to run on a multiprocessor with modest parallelism by using functional decomposition and standard operating-system capabilities to exploit the parallel hardware. Paragon is a high-level environment for multiprocessor systems that has facilities for both functional decomposition and data partitioning. It provides primitives that will work efficiently on several parallel-processing systems. Paragon's primitives can be used to build special image-processing operations, allowing one's own programming environment to be grown naturally     

Memory requirements for parallel programs

Parallel execution is normally used to decrease the amount of time required to run a program. However, the parallel execution may require far more space than that required by the sequential execution. Worse yet, the parallel space requirement may be very much more difficult to predict than the sequential space requirement because there are more factors to consider. These include essentially nondeterministic factors that can influence scheduling, which in turn may dramatically influence space requirements. We survey some scheduling algorithms that attempt to place bounds on the amount of time and space used during parallel execution. We also outline a direction for future research. This direction takes us into the area of functional programming, where the declarative nature of the languages can help the programmer to produce correct parallel programs, a feat that can be difficult with procedural languages. Currently the high-level nature of functional languages can make it difficult for the programmer to understand the operational behavior of the program. We look at some of the problems in this area, with the goal of achieving a programming environment that supports correct, efficient parallel programs.     

Hierarchical structures and complexities of parallel isometric languages

The relationship between parallel isometric array languages and sequential isometric array languages is examined. Their hierarchical structures are investigated and a hierarchy is established by introducing parallel context-free array languages (PCFAL), derivation bounded array languages (DBAL), linear array languages (LAL), and extended regular afray languages (ERAL). It is interesting to find that some fundamental aspects that hold in one-dimensional string languages do not hold in their two-dimensional counterparts. Some parsing techniques are also explored. It is shown that while parallel parsing grammars may be simpler to write and parallel processing usually takes less time than sequential ones, the nature of parallel parsing is very complicated. Finally, several future research topics concerned with parallel isometric array languages including their complexities, hierarchical structures, and application to pattern recognition are discussed.     

Molecule: a language construct for layered development of parallelprograms

A new language construct, called molecule, is described for the efficient implementation of algorithms on parallel computers. A molecule can be considered a procedure associated with a molecule type. Each molecule type characterizes a particular computation mode (sequential, pipelining, array processing, dataflow, multiprocessing, etc.). Basic concepts of molecule are introduced with a procedural language, called PAL. A concrete example is presented to illustrate layered software development using PAL on a multicomputer (the iPSC). It is concluded that high-level languages, augmented with the molecule construct, offer application flexibility, user friendliness, and efficiency in implementing parallel programs     

Comparative evaluation of high-level real-time programming languages

Owing to the fast growing need for better means of building real-time systems, a number of representative languages used in real-time programming is surveyed. The evaluation focuses on seven languages which possess explicit real-time features. Based on a categorization of the latter, the seven languages are then compared with respect to their real-time capabilities. The strong points and the limitations of Ada and PEARL, the only high-level real-time languages readily applicable in industrial control environments, are covered in more detail. The evaluation reveals that none of the languages actually used in industry is genuinely real-time. Therefore, a number of new features is suggested for incorporation into existing or future languages and their run-time environments. These proposals are meant to advance the inadequate state of affairs, and also to reignite the discussion of this topic in the real-time community.