LALR(1, 1) parser generation for regular right part grammars

Summary Commonly used extensions to BNF can be modelled by the formalism of regular right part grammars. A method for building LR parsers for such grammars is given, which works by first constructing an LR(0) automaton and then augmenting it with readback machines constructed to recognize the reverse of the state sequences leading to a reduction. The state sequences which will be accepted by such readback machines are also the sequences which link reductions to their lookback states (states in which the parser may be re-started after the reduction), which are needed in order to compute LALR(1) lookahead sets using the algorithm devised recently by DeRemer and Pennello.An algorithm is presented which computes these lookback states using the structure of the LR(0) automaton, and it is shown how this can easily be extended to build readback machines at the same time.     

Methods for the automatic construction of error correcting parsers

Summary Methods for the automatic construction of error handling parsers are presented. The resulting parsers are capable of correcting all syntax errors by insertion and/or deletion of terminal symbols to the right of the error location. Thus, the output of the parser always corresponds to a syntactically valid program. This contributes significantly to the reliability and robustness of a compiler. The speed of parsing correct parts of a program is not affected by the presence of the error handling capability. The correction algorithm is easy to implement. Apart from the parsing tables only one character per parser state is required to control the correction process. The method is applicable to a wide class of stack automata including LL(k), LR(k), SLR(k), and LALR(k) parsers. It is shown that for LL(k) grammars error correction can be obtained as a byproduct of the canonical LL(k) parser generation. A similar result can be obtained for LR(k) grammars if the parser generator is slightly modified. The method has been successfully added to an LALR(1) parser generator.     

Foundations of Fast Communication via XML

Communication with XML often involves pre-agreed document types. In this paper, we propose an offline parser generation approach to enhance online processing performance for documents conforming to a given DTD. Our examination of DTDs and the languages they define demonstrates the existence of ambiguities. We present an algorithm that maps DTDs to deterministic context-free grammars defining the same languages. We prove the grammars to be LL(1) and LALR(1), making them suitable for standard parser generators. Our experiments show the superior performance of generated optimized parsers. Our results generalize from DTDs to XML schema specifications with certain restrictions, most notably the absence of namespaces, which exceed the scope of context-free grammars.     

LEFT TO RIGHT PARSING OF LEXICALIZED TREE-ADJOINING GRAMMARS1

This paper presents an algorithm (a parser) for analyzing sentences according to grammatical constraints expressed in the framework of lexicalized tree-adjoining grammar. For the current grammars of English, the algorithm behaves much better and requires much less time than its worst-case complexity. The main objective of this work is to design a practical parser whose average-case complexity is much superior to its worst case. Most of the previous methods always required the worst-case complexity. The algorithm can be used in two modes. As a recognizer it outputs whether the input sentence is grammatically correct or not. As a parser it outputs a detailed analysis of the grammatically correct sentences. As sentences are read from left to right, information about possible continuations of the sentence is computed. In this sense, the algorithm is called a predictive left to right parser. This feature reduces the average time required to process a given sentence. In the worst case, the parser requires an amount of time proportional to G²n⁶ for a sentence of n words and for a lexicalized tree-adjoining grammar of size G. The worst-case complexity is only reached with pathological (not naturally occurring) grammars and inputs.     

Yet another generation of LALR parsers for regular right part grammars

In this paper we introduce two methods for building LALR parsers for regular right part grammars (RRPGs). Both methods build a parser directly from a grammar, require no extra state or data structure, and can deal with all LALR RRPGs. The first method is quite simple. For almost all LALR RRPGs, including the majority of grammars with stacking conflicts, parsing actions are similar to those of LALR parsers for usual context free grammars. No extra action is required to recognize a handle in this case. For other LALR RRPGs, the right hand side of a production is checked to recognize a handle. The second method does not require checking of the right hand side of a production to recognize a handle. Instead, it records the number of conflicts in LR items and in the stack. Unlike previous methods, our method needs no extra data structure. Received: 23 September 1998 / 16 March 2001     

Upper bounds on the size of LR(k) parsers

It is shown that in many cases the trivial upper bound 2^{|G|k + 1} on the number of states of an LR(k) parser for a grammar G is too conservative. In particular, if G is not right-recursive, the canonical LR(k) parser for G has at most |G^k|G|·2^|G| states. Examples of grammars with large LR(k) parsers are given.     

A scheme for lr(k) parsing with error recovery part iii: error correction

The paper is the third in a series of three papers devoted to a detailed study of LR(k) parsing with error recovery and correction. A new class of syntax errors is introduced, called (k)-local parser defined errors, which suit better than the conventional minimum distance errors for characterization of error detection and recovery in LR(k) parsing. The question whether a given string has n k-local parser defined errors for some integer n is shown to be decidable. Using the formalization of LR(k) parsing and error recovery presented in the first and the second paper in the series it is shown that the canonical LR(k) parser of an LR(k) grammar always has an error recovering extension which is able to produce a correction for any terminal string containing only (k)-local parser defined errors.     

Semantic routines and LR(k) parsers

Summary Most applications of parsing require that the parser call semantic action routines while processing the input. For LR(k) parsers it is well known that a semantic action routine can be called when the end of a production is recognized. Often, however, it is desirable to call routines at other times.This paper presents fast algorithms that determine, for an LR(k) (or SLR(k)) grammar, which positions are suitable for calling routines. The algorithms are practical for use with LR(1) (SLR(1)) parser building programs, because the worst case running time is dominated by the time required to build the LR(1) (SLR(1)) parser. Applications of the algorithms to attribute grammars and automatic indentation are discussed.     

A practical general method for constructing LR(k) parsers

Summary The paper presents in detail the case for k=1 of a practical general method for constructing LR(k) parsers. For k=1 this method is of rival efficiency to the previous general algorithm described by the author in [21]. The method involves combining the states of an LR(k) parser as they are generated, reducing to a fraction, in the process, the number of configurations that need actually be evaluated, or for which space must be assigned — compared to such general methods as those of [1, 11, 12, 17]. The criteria of compatibility introduced for this purpose are such that the parser obtained is in practice identical in size to, or negligibly larger than, that obtained by resolving the inadequacies of an LR(o) parser (as is done for various subsets of the LR(k) grammars in [5, 8, 14, 20]).This paper is a development of one of the ideas proposed in Pager [16]. The work was supported by the National Science Foundation under Grant GJ-43362.     

Implementation issues in the development of the parsec parser

This paper describes the implementation of a constraint-based parser, PARSEC (Parallel ARchitecture SEntence Constrainer), which has the required flexibility that a user may easily construct a custom grammar and test it. Once the user designs grammar parameters, constraints, and a lexicon, our system checks them for consistency and creates a parser for the grammar. The parser has an X-windows interface that allows a user to view the state of a parse of a sentence, test new constraints, and dump the constraint network to a file. The parser has an option to perform the computationally expensive constraint propagation steps on the MasPar MP-1. Stream and socket communication was used to interface the MasPar constraint parser with a standard X-windows interface on our Sun Sparcstation. The design of our heterogeneous parser has benefitted from the use of object-oriented techniques. Without these techniques, it would have been more difficult to combine the processing power of the MasPar with a Sun Sparcstation. Also, these techniques allowed the parser to gracefully evolve from a system that operated on single sentences, to one capable of processing word graphs containing multiple sentences, consistent with speech processing. This system should provide an important component of a real-time speech understanding system.     

Eliminating unit productions from LR parsers

Summary In this paper we show how one can improve upon an algorithm by Aho and Ullman [3] for eliminating unit productions from an LR(k) parser so that the elimination concerned can be made in all cases, instead of only in the special case required by [3] where no two unit productions have the same left-hand side. In most practical grammars this special case does not in fact arise. Since the elimination of unit productions both reduces the size of the parser and increases its speed, it is of value to have a general method for achieving this objective.The algorithm provided eliminates from the parser all nonterminals that occur as left-hand sides of unit productions. This substantially contributes to the reduction in size obtained, and also provides a solution to an open problem by Aho and Ullman [3]. An application of the Algorithm to the parser construction method of Pager [19] is considered, and a method is provided for the use of default reductions and the elimination of final states in conjunction with the elimination of unit reductions. The sizes of the parsers obtained using the parser's algorithm are compared with those of Anderson, Eve, and Horning [4].This work was supported by the National Science Foundation under Grant GJ-43362. A shortened version of the paper was presented at the 2nd colloquium on Automata, Languages and Programming, University of Saarbrücken, July 1974     

Generalized deterministic left to right parsing

Summary A parser model is presented whose structure is a generalization of the well known LR(k) parsers. Various classes of this parser that would be both practical and efficient to use in a compiler are examined. Associated with these classes of parsers is a hierarchy of type-0 grammars, each grammatical class being defined in terms of the form and structure of derivations. In particular, parsers based on a class called deterministic regular parsable (DRP) grammars will detect any errors as soon as possible during a left to right scan of the input. LR(k) grammars are also DRP. Much research related to LR(k) grammars and parsing is also applicable to DRP grammars and their associated parsers.     

Llun—a high‐level debugger for generated parsers

Parser generators such as yacc have been used in a large number of applications, not just those that involve compiler writing. This has meant that these tools are being used increasingly by nonspecialist developers. A consequence of this is that good support is required for debugging a grammar and its generated parser(s). This paper describes Llun, a debugging tool that visualizes the operation of a generated parser at both a high‐level and a low‐level. Llun is superior to other parser visualization products by virtue of the high‐level facilities it offers. The paper describes some of the problems encountered using parser generators, outlines a visualization system which addresses a number of the problems and uses a taxonomy developed by Price to categorize the system. Copyright © 2001 John Wiley & Sons, Ltd.     

LR(0) grammars generated by LR(0) parsers

Summary Let be an LR(0) parser of a given LR(0) grammar G. Generally, does not only parse the words generated by G but also the words of some other LR(0) grammars different from G. In this paper we shall define a class of LR(0) parsers and shall present a characterization and a method for the construction of all LR(0) grammars which can be parsed by a given LR(0) parser.     

BRNGLR: a cubic Tomita-style GLR parsing algorithm

Tomita-style generalised LR (GLR) algorithms extend the standard LR algorithm to non-deterministic grammars by performing all possible choices of action. Cubic complexity is achieved if all rules are of length at most two. In this paper we shall show how to achieve cubic time bounds for all grammars by binarising the search performed whilst executing reduce actions in a GLR-style parser. We call the resulting algorithm Binary Right Nulled GLR (BRNGLR) parsing. The binarisation process generates run-time behaviour that is related to that shown by a parser which pre-processes its grammar or parse table into a binary form, but without the increase in table size and with a reduced run-time space overhead. BRNGLR parsers have worst-case cubic run time on all grammars, linear behaviour on LR(1) grammars and produce, in worst-case cubic time, a cubic size binary SPPF representation of all the derivations of a given sentence.     

Parallel <Emphasis Type="Italic">LL</Emphasis> parsing

A deterministic parallel LL parsing algorithm is presented. The algorithm is based on a transformation from a parsing problem to parallel reduction. First, a nondeterministic version of a parallel LL parser is introduced. Then, it is transformed into the deterministic version—the LLP parser. The deterministic LLP(q,k) parser uses two kinds of information to select the next operation — a lookahead string of length up to k symbols and a lookback string of length up to q symbols. Deterministic parsing is available for LLP grammars, a subclass of LL grammars. Since the presented deterministic and nondeterministic parallel parsers are both based on parallel reduction, they are suitable for most parallel architectures.     

Experimental evaluation of a tool for the verification and transformation of source code in event-driven systems

In event-driven systems, separating the reactive part of software (i.e., event-driven control) from the non-reactive part is a common design practice. The reactive part is typically structured according to the states and transitions of a system, whereas the non-reactive part is typically structured according to the concepts of the application domain (e.g., the services provided by the system). In such systems, the non-reactive part of software stimulates the reactive part with event calls. Whenever the non-reactive part is modified (e.g., during evolution), the existing event calls may become invalid, new event calls may become necessary, and the two parts of software may become incompatible. Manually finding and repairing these types of defects is a time-consuming and error-prone maintenance task. In this article, we present a solution that combines source code model checking and aspect-oriented programming techniques, to reduce the time spent by developers and to automatically find defects, while performing the maintenance task mentioned above. In addition, we present controlled experiments showing that the solution can reduce the time by 75%, and enable the prevention of one defect per 140 lines of source code.     

Taming the PL/I syntax

Ideally the syntactic part of a PL/I compiler would be generated directly from the semi-formal definition of ANSI Standard PL/I. A practical approach to this is described, using finite state machines and an LALR parser generator. The parser uses a method due to Aoe which avoids list searching. Adapted for this method the PL/I grammar has 841 states. The parse table generator exploits the freedom to renumber states in a way that improves on previous algorithms for compacting the tables. The parser tables occupy less than 4K bytes.     

The Grammar Deployment Kit — System Demonstration —

Grammar deployment is the process of turning a given grammar specification into a working parser. The Grammar Deployment Kit (for short, GDK) provides tool support in this process based on grammar engineering methods. We are mainly interested in the deployment of grammars for software renovation tools, that is, tools for software re- and reverse engineering. The current version of GDK is optimized for Cobol. We assume that grammar deployment starts from an initial grammar specification which is maybe still ambiguous or even incomplete. In practice, grammar deployment binds unaffordable human resources because of the unavailability of suitable grammar specifications, the diversity of parsing technology as well as the limitations of the technology, integration problems regarding the development of software renovation functionality, and the lack of tools and adherence to firm methods for grammar engineering. GDK helps to largely automate grammar deployment because tool support for grammar adaptation and parser generation is provided. We support different parsing technologies, among them btyacc, that is, yacc with backtracking. GDK is free software.