Debugging a Distributed Computing System

In this paper we discuss the issues involved in debugging a provide distributed computing system. We describe the major differences between debugging a distributed system and debugging a sequential program. We suggest a methodology for distributed debugging, and we propose various tools or aids.     

基于国产平台的智能跟踪调试技术

随着龙芯、飞腾、申威等国产处理器的诞生,目前涌现出了一批支持国产处理器的国产操作系统,如JARI-Works、中标麒麟等.但是,与国产软硬平台配套的调试工具发展却相对滞后,严重制约着国产平台下的软件调试效率.本文针对国产软硬平台,设计了一种基于Eclipse平台的智能调试方案.通过在Eclipse平台下集成面向国产平台的工具链和智能跟踪调试插件,自动启动并智能跟踪调试流程,从而实现调试功能的图形化和智能化,进而构建出一套界面友好、使用便捷、自主可控的集成开发环境.通过实验表明,该调试方案能够有效简化调试流程,提高国产平台下的软件调试效率.     

基于Eclipse平台的交叉调试方案

Eclipse平台提供了开发C/C++程序的插件CDT,但是针对嵌入式软件的交叉调试,CDT需要过多的用户参与.设计一种基于Eclipse平台的交叉调试方案,通过向Eclipse平台无缝集成交叉调试插件,对调试所需参数进行自动配置,充分利用Eclipse平台的扩展性,使用户可以直观的观察目标机信息,从而实现交叉调试对用户的透明化,达到自动化调试的目的.通过实验表明,该交叉调试方案能够简化用户的控制,提高交叉调试的效率.     

DENNET中降低调试器干扰的机制

DENNET是清华大学高性能计算所开发的一个机群系统并行程序调试环境．本文分析并行程序调试环境的现状和发展趋势；阐述调试环境对并行程序的干扰问题，指出降低乃至完全消除干扰是并行程序调试环境的一个重要研究方向，介绍了DENNET设计时采用的两种降低干扰的技术．     

High-level debugging of distributed systems: The behavioral abstraction approach

Most extant debugging aids force their users to think about errors in programs from a low-level, unit-at-a-time perspective. Such a perspective is inadequate for debugging large complex systems, particularly distributed systems. In this paper, we present a high-level approach to debugging that offers an alternative to the traditional techniques. We describe a language, edl, developed to support this high-level approach to debugging and outline a set of tools that has been constructed to effect this approach. The paper includes an example illustrating the approach and discusses a number of problems encountered while developing these debugging tools.     

Reversible debugging using program instrumentation

Reversible execution has not been fully exploited in symbolic debuggers. Debuggers that can undo instructions usually incur a significant performance penalty during a debugging session. We describe an efficient reversible debugging mechanism based on program instrumentation. The approach enables repetitive debugging sessions with selectable reversible routines and recording modes. Experimental results indicate that the execution penalty can be significantly reduced with moderate code growth     

Debugging other languages in APL

The advantages (and disadvantages) of on-line debugging have been discussed extensively in the profession. We propose to couple APL with the debugging process. Techniques for ‘translating’ other languages into APL are presented, and the trade-off between the cost of this translation process and the benefits to be gained by debugging in APL are discussed in terms of an actual PL/1 programming project.     

Debugging with dynamic slicing and backtracking

Programmers spend considerable time debugging code. Symbolic debuggers provide some help but the task remains complex and difficult. Other than breakpoints and tracing, these tools provide little high-level help. Programmers must perform many tasks manually that the tools could perform automatically, such as finding which statements in the program affect the value of an output variable for a given test case, and what was the value of a given variable when the control last reached a given program location. If debugging tools provided explicit support for these tasks, the debugging process could be automated to a significant extent. In this paper we present a debugging model, based on dynamic program slicing and execution backtracking techniques, that easily lends itself to automation. This model is based on experience with using these techniques to debug software. We also present a prototype debugging tool, SPYDER, that explicitly supports the proposed model, and with which we are performing further debugging research.     

Embedded software debugging using virtual filesystem abstractions

We present a scalable technique to simplify the software debugging process for embedded systems that is based on the use of distributed filesystem abstractions. In this technique, the principal building blocks within embedded systems comprising “system-on-chip” (SoC) devices export debugging interfaces realized with filesystem abstractions. These filesystem interfaces are composed in a manner consistent with the hardware hierarchy and provide a portable interface to concurrently debug software executing in the various processing elements within the embedded system. We illustrate application of the model to support the various software debugging requirements unique to SoCs, many of which traditional methods are ill-equipped to deal with. We also present an architecture that may be used to realize distributed filesystem abstractions within resource constrained embedded architectures.     

Debugging formal specifications: a practical approach using model-based diagnosis and counterstrategies

Creating a formal specification for a design is an error-prone process. At the same time, debugging incorrect specifications is difficult and time consuming. In this work, we propose a debugging method for formal specifications that does not require an implementation. We handle conflicts between a formal specification and the informal design intent using a simulation-based refinement loop, where we reduce the problem of debugging overconstrained specifications to that of debugging unrealizability. We show how model-based diagnosis can be applied to locate an error in an unrealizable specification. The diagnosis algorithm computes properties and signals that can be modified in such a way that the specification becomes realizable, thus pointing out potential error locations. In order to fix the specification, the user must understand the problem. We use counterstrategies to explain conflicts in the specification. Since counterstrategies may be large, we propose several ways to simplify them. First, we compute the counterstrategy not for the original specification but only for an unrealizable core. Second, we use a heuristic to search for a countertrace, i.e., a single input trace which necessarily leads to a specification violation. Finally, we present the countertrace or the counterstrategy as an interactive game against the user, and as a graph summarizing possible plays of this game. We introduce a user-friendly implementation of our debugging method and present experimental results for GR(1) specifications.     

Debugging Concurrent Software: Advances and Challenges

Concurrency debugging is an extremely important yet challenging problem that has been hampering developer productivity and software reliability in the multicore era. We have worked on this problem in the past eight years and have developed several effective methods and automated tools for helping developers debugging shared memory concurrent programs. This article discusses challenges in concurrency debugging and summarizes our research contributions in four important directions: concurrency bug reproduction, detection, understanding, and fixing. It also discusses other recent advances in tackling these challenges.     

TIDE: A Generic Debugging Framework — Tool Demonstration —

A language specific interactive debugger is one of the tools that we expect in any mature programming environment. We present applications of TIDE: a generic debugging framework that is related to the ASF+SDF Meta-Environment. TIDE can be applied to different levels of debugging that occur in language design.Firstly, TIDE was used to obtain a full-fledged debugger for language specifications based on term rewriting. Secondly, TIDE can be instantiated for any other programming language, including but not limited to domain specific languages that are defined and implemented using ASF+SDF.We demonstrate the common debugging interface, and indicate the amount of effort needed to instantiate new debuggers based on TIDE.     

An experiment in tool integration: The DDBG parallel and distributed debugger

This paper discusses the development of a debugging tool for parallel programs showing how the requirements posed by high-level tools for parallel program development have influenced the design of the debugging system since its early stages of development. We concentrate our attention upon the interfacing of the debugger with other tools of a parallel software engineering environment, namely a graphical programming language and a testing and debugging tool. This is illustrated with the results of our experimentation with the design and implementation of DDBG, a debugger for the PVM environment.     

Distributed debugging for mobile networks

Debuggers are an integral part, albeit often neglected, of the development of distributed applications. Ambient-oriented programming (AmOP) is a distributed paradigm for applications running on mobile ad hoc networks. In AmOP the complexity of programming in a distributed setting is married with the network fragility and open topology of mobile applications. To our knowledge, there is no debugging approach that tackles both these issues. In this paper we argue that a novel kind of distributed debugger that we term an ambient-oriented debugger, is required. We present REME-D (read as remedy), an online ambient-oriented debugger that integrates techniques from distributed debugging (event-based debugging, message breakpoints) and proposes facilities to deal with ad hoc, fragile networks – epidemic debugging, and support for frequent disconnections.     

Debugging methods in recursive structured FORTRAN

A version of structured FORTRAN named STAR is introduced. STAR allows recursive subroutines. Implementation problems such as parameter passing by value and local variable stacking are discussed. The major difficulty in the preprocessor approach is debugging. We claim that execution profiles are strong debugging aids, and demonstrate how to debug a STAR program at source language level with the clue in the profiles plus assertion statements. We make use of a quicksort program as an example.     

Guard: A Relative Debugger

A significant amount of software development is evolutionary, involving the modification of already existing programs. To a large extent, the modified programs produce the same results as the original program. This similarity between the original program and the development program is utilized by relative debugging. Relative debugging is a new concept that enables the user to compare the execution of two programs by specifying the expected correspondences between their states. A relative debugger concurrently executes the programs, verifies the correspondences, and reports any differences found. We describe our novel debugger, called Guard, and its relative debugging capabilities. Guard is implemented by using our library of debugging routines, called Dynascope, which provides debugging primitives in heterogeneous networked environments. To demonstrate the capacity of Guard for debugging in heterogeneous environments, we describe an experiment in which the execution of two programs is compared across Internet. The programs are written in different programming languages and executing on different computing platforms. © 1997 by John Wiley & Sons, Ltd.     

Adapting combinator and SECD machines to display snapshots of functional computations

A functional computation involves substituting for function applications in an expression until that expression is reduced to normal form. The views of computations presented by the sequence- and state-oriented debugging tools of imperative systems are inappropriate for use with functional computations. New debugging tools are needed to aid the development of functional programs, especially in the context of lazy evaluation. After surveying previously reported debugging tools, we discuss a new debugging tool. Its implementation involves changing the reduction rules of the machine. The new reduction rules are applied to an interrupted computation to give a snapshot of that computation in source-level terms. We have implemented tools to produce snapshots for eager SECD and lazy combinator reduction machines. Example snapshots are shown from each. The implementation for an eager SECD machine is relatively straightforward, so we confine discussion of this to a brief sketch. A more detailed account is given of the implementation for a lazy combinator reduction machine, as this offers one solution to well-known problems with debugging functional programs in the context of lazy evaluation and combinator code.     

Automated debugging based on a constraint model of the program and a test case

Debugging, i.e., fault localization, in case of a detected failure is a time consuming and intricate task. The automation or at least partial automation of debugging is therefore highly desired. In this paper, we discuss some of the most recent approaches for debugging namely spectrum-based, slicing-based, and model-based debugging. We focus on the latter, and introduce the underlying theory as well as discuss empirical results obtained from our implementation. The model-based approach we present in this paper relies on a constraint representation of a program that is equivalent to the original program in terms of the input-output behavior under some reasonable assumptions. By using constraints for representing programs and subsequently test cases we are able to state the debugging problem as a constraint satisfaction problem that can be effectively solved using a todays constraint solver. The given empirical results indicate that the approach can be used for debugging smaller programs in less than 1 s. Moreover, we briefly compare the three approaches and suggest a combination of them in order to improve the results and the overall necessary running time.     

Ahead of time static analysis for automatic generation of debugging interfaces to the Linux kernel

The Linux kernel does not export a stable, well-defined kernel interface, complicating the development of kernel-level services, such as device drivers and file systems. While there does exist a set of functions that are exported to external modules, this set of functions frequently changes, and the functions have implicit, ill-documented preconditions. No specific debugging support is provided. We present Diagnosys, an approach to automatically constructing a debugging interface for the Linux kernel. First, a designated kernel maintainer uses Diagnosys to identify constraints on the use of the exported functions. Based on this information, developers of kernel services can then use Diagnosys to generate a debugging interface specialized to their code. When a service including this interface is tested, it records information about potential problems. This information is preserved following a kernel crash or hang. Our experiments show that the generated debugging interface provides useful log information and incurs a low performance penalty.     

Tools and strategies for debugging distributed stream processing applications

Distributed data stream processing applications are often characterized by data flow graphs consisting of a large number of built‐in and user‐defined operators connected via streams. These flow graphs are typically deployed on a large set of nodes. The data processing is carried out on‐the‐fly, as tuples arrive at possibly very high rates, with minimum latency. It is well known that developing and debugging distributed, multi‐threaded, and asynchronous applications, such as stream processing applications, can be challenging. Thus, without domain‐specific debugging support, developers struggle when debugging distributed applications. In this paper, we describe tools and language support to support debugging distributed stream processing applications. Our key insight is to view debugging of stream processing applications from four different, but related, perspectives. First, debugging the semantics of the application involves verifying the operator‐level composition and inspecting the flows at the logical level. Second, debugging the user‐defined operators involves traditional source‐code debugging, but strongly tied to the stream‐level interactions. Third, debugging the deployment details of the application require understanding the runtime physical layout and configuration of the application. Fourth, debugging the performance of the application requires inspecting various performance metrics (such as communication rates, CPU utilization, etc.) associated with streams, operators, and nodes in the system. In light of this characterization, we developed several tools such as a debugger‐aware compiler and an associated stream debugger, composition and deployment visualizers, and performance visualizers, as well as language support, such as configuration knobs for logging and tracing, deployment configurations such as operator‐to‐process and process‐to‐node mappings, monitoring directives to inspect streams, and special sink adapters to intercept and dump streaming data to files and sockets, to name a few. We describe these tools in the context of Spade —a language for creating distributed stream processing applications, and System S —a distributed stream processing middleware under development at the IBM Watson Research Center. Published in 2009 by John Wiley & Sons, Ltd.