Exception handling in the spreadsheet paradigm

Exception handling is widely regarded as a necessity in programming languages today and almost every programming language currently used for professional software development supports some form of it. However, spreadsheet systems, which may be the most widely used type of “programming language” today in terms of number of users using it to create “programs” (spreadsheets), have traditionally had only extremely limited support for exception handling. Spreadsheet system users range from end users to professional programmers and this wide range suggests that an approach to exception handling for spreadsheet systems needs to be compatible with the equational reasoning model of spreadsheet formulas, yet feature expressive power comparable to that found in other programming languages. We present an approach to exception handling for spreadsheet system users that is aimed at this goal. Some of the features of the approach are new; others are not new, but their effects on the programming language properties of spreadsheet systems have not been discussed before in the literature. We explore these properties, offer our solutions to problems that arise with these properties, and compare the functionality of the approach with that of exception handling approaches in other languages     

Avoiding,finding and fixing spreadsheet errors – A survey of automated approaches for spreadsheet QA

Spreadsheet programs can be found everywhere in organizations and they are used for a variety of purposes, including financial calculations, planning, data aggregation and decision making tasks. A number of research surveys have however shown that such programs are particularly prone to errors. Some reasons for the error-proneness of spreadsheets are that spreadsheets are developed by end users and that standard software quality assurance processes are mostly not applied. Correspondingly, during the last two decades, researchers have proposed a number of techniques and automated tools aimed at supporting the end user in the development of error-free spreadsheets. In this paper, we provide a review of the research literature and develop a classification of automated spreadsheet quality assurance (QA) approaches, which range from spreadsheet visualization, static analysis and quality reports, over testing and support to model-based spreadsheet development. Based on this review, we outline possible opportunities for future work in the area of automated spreadsheet QA.     

All-Pairs: An Abstraction for Data-Intensive Computing on Campus Grids

Today, campus grids provide users with easy access to thousands of CPUs. However, it is not always easy for nonexpert users to harness these systems effectively. A large workload composed in what seems to be the obvious way by a naive user may accidentally abuse shared resources and achieve very poor performance. To address this problem, we argue that campus grids should provide end users with high-level abstractions that allow for the easy expression and efficient execution of data-intensive workloads. We present one example of an abstraction—All-Pairs—that fits the needs of several applications in biometrics, bioinformatics, and data mining. We demonstrate that an optimized All-Pairs abstraction is both easier to use than the underlying system, achieve performance orders of magnitude better than the obvious but naive approach, and is both faster and more efficient than a tuned conventional approach. This abstraction has been in production use for one year on a 500 CPU campus grid at the University of Notre Dame and has been used to carry out a groundbreaking analysis of biometric data.     

High-level debugging of distributed systems: The behavioral abstraction approach

Most extant debugging aids force their users to think about errors in programs from a low-level, unit-at-a-time perspective. Such a perspective is inadequate for debugging large complex systems, particularly distributed systems. In this paper, we present a high-level approach to debugging that offers an alternative to the traditional techniques. We describe a language, edl, developed to support this high-level approach to debugging and outline a set of tools that has been constructed to effect this approach. The paper includes an example illustrating the approach and discusses a number of problems encountered while developing these debugging tools.     

Spreadsheet quality assurance: a literature review

Spreadsheets are very common for information processing to support decision making by both professional developers and non-technical end users. Moreover, business intelligence and artificial intelligence are increasingly popular in the industry nowadays, where spreadsheets have been used as, or integrated into, intelligent or expert systems in various application domains. However, it has been repeatedly reported that faults often exist in operational spreadsheets, which could severely compromise the quality of conclusions and decisions based on the spreadsheets. With a view to systematically examining this problem via survey of existing work, we have conducted a comprehensive literature review on the quality issues and related techniques of spreadsheets over a 35.5-year period (from January 1987 to June 2022) for target journals and a 10.5-year period (from January 2012 to June 2022) for target conferences. Among other findings, two major ones are: (a) Spreadsheet quality is best addressed throughout the whole spreadsheet life cycle, rather than just focusing on a few specific stages of the life cycle. (b) Relatively more studies focus on spreadsheet testing and debugging (related to fault detection and removal) when compared with spreadsheet specification, modeling, and design (related to development). As prevention is better than cure, more research should be performed on the early stages of the spreadsheet life cycle. Enlightened by our comprehensive review, we have identified the major research gaps as well as highlighted key research directions for future work in the area.     

Model-based diagnosis of spreadsheet programs: a constraint-based debugging approach

Spreadsheet programs are probably the most successful example of end-user software development tools and are used for a variety of purposes. Like any type of software, they are prone to error, in particular as they are usually developed by non-programmers. While various techniques exist to support the developer in finding errors in procedural programs, the tool support for spreadsheet debugging is still limited. In this paper, we show how techniques from model-based diagnosis can be applied and extended for spreadsheet debugging by translating the relevant parts of a spreadsheet to a constraint satisfaction problem. We additionally propose both problem-specific and generalizable extensions to the classical diagnosis algorithms which help to detect potential problems in a spreadsheet based on user-provided test cases more efficiently. The proposed techniques were integrated into a modular framework for spreadsheet debugging and evaluated with respect to scalability based on a number of real-world and artificially created spreadsheets. An additional error detection exercise involving 24 subjects was performed to assess the general applicability of such advanced spreadsheet debugging techniques for end users.     

End-User Testing for the Lyee Methodology using the Screen Transition Paradigm and WYSIWYT

End-user specification of Lyee programs is one goal envisioned by the Lyee methodology. But with any software development effort comes the possibility of faults. Thus, providing end users a means to enter their own specifications is not enough; they must also be provided with the means to find faults in their specifications, in a manner that is appropriate not only for the end user's programming environment but also for his or her background. In this paper, we present an approach to solve this problem that marries two proven technologies for end users. One methodology for enabling end users to program is the screen transition paradigm. One useful visual testing methodology is ‘What you see is what you test (WYSIWYT)’. In this paper, we show that WYSIWYT test adequacy criteria can be used with the screen transition paradigm, and present a systematic translation from this paradigm to the formal model underlying WYSIWYT.     

Systematic evolution of model-based spreadsheet applications

Using spreadsheets is the preferred method to calculate, display or store anything that fits into a table-like structure. They are often used by end users to create applications, although they have one critical drawback—spreadsheets are very error-prone. Recent research has developed methods to reduce this error-proneness by introducing a new way of object-oriented modeling of spreadsheets before using them. These spreadsheet models, termed ClassSheets, are used to generate concrete spreadsheets on the instance level. By this approach sources of errors are reduced and spreadsheet applications become easier to understand.As usual for almost every other application, requirements on spreadsheets change due to the changing environment. Thus, the problem of evolution of spreadsheets arises. The update and evolution of spreadsheets is the uttermost source of errors that may have severe impact.In this paper, we will introduce a model-based approach to spreadsheet evolution by propagating updates on spreadsheet models (i.e. ClassSheets) to spreadsheets. To this end, update commands for the ClassSheet layer are automatically transformed to those for the spreadsheet layer. We describe spreadsheet model update propagation using a formal framework and present an integrated tool suite that allows the easy creation and safe update of spreadsheet models. The presented approach greatly contributes to the problem of software evolution and maintenance for spreadsheets and thus avoids many errors that might have severe impacts.     

Rule-induction and case-based reasoning: hybrid architecturesappear advantageous

Researchers have embraced a variety of machine learning (ML) techniques in their efforts to improve the quality of learning programs. The recent evolution of hybrid architectures for machine learning systems has resulted in several approaches that combine rule induction methods with case-based reasoning techniques to engender performance improvements over more traditional single-representation architectures. We briefly survey several major rule-induction and case-based reasoning ML systems. We then examine some interesting hybrid combinations of these systems and explain their strengths and weaknesses as learning systems. We present a balanced approach to constructing a hybrid architecture, along with arguments in favor of this balance and mechanisms for achieving a proper balance. Finally, we present some initial empirical results from testing our ideas and draw some conclusions based on those results     

Securing systems against external programs

Internet users routinely and often unknowingly download and run programs, such as Java applets; and some Web servers let users upload external programs and run them on the server. Although the practice of executing these external programs has the sanction of widespread use, its security implications haven't yet been systematically addressed. In the brief, dynamic history of the Internet, such a situation is not unusual. New communication mechanisms and computing paradigms are often implemented before the security issues they engender have been rigorously analyzed. Our goal is to address this problem in the subdomain of external programs by systematically outlining security issues and classifying current solutions. Our focus is solely on protecting a host from external programs. We do not address the problem of protecting the communication medium or protecting an external program from runtime systems. Furthermore, we do not address the problem of correctly identifying the source of an external program (authentication). We start our inquiry by reviewing the relevant models of computation, followed by an overview of the security problems associated with them. We then classify both the problems and the existing solutions using a resource-centric model that distinguishes problems associated with resource access from those associated with resource consumption. Finally, we classify solutions to each problem according to how and when they are applied     

Twinkling lights and nested loops: distributed problem solving and spreadsheet development

In contrast to the common view of spreadsheets as “single-user” programs, we have found that spreadsheets offer surprisingly strong support for cooperative development of a wide variety of applications. Ethnographic interviews with spreadsheet users showed that nearly all of the spreadsheets used in the work environments studied were the result of collaborative work by people with different levels of programming and domain expertise. We describe how spreadsheet users cooperate in developing, debugging and using spreadsheets. We examine the properties of spreadsheet software that enable cooperation, arguing that: (1) the division of the spreadsheet into two distinct programming layers permits effective distribution of computational tasks across users with different levels of programming skill; and (2) the spreadsheet's strong visual format for structuring and presenting data supports sharing of domain knowledge among co-workers.     

Performance measurement for parallel and distributed programs: astructured and automatic approach

Novel approaches are presented for designing performance measurement systems for parallel and distributed programs. The first approach involves unifying performance information into a single, regular structure that reflects the structure of programs under measurement. The authors define a hierarchical model for the execution of parallel and distributed programs as a framework for the performance measurement. A complete different levels of detail in the hierarchy. The second approach is based on the development of automatic guidance techniques that can direct users to the location of performance problems in the program. Guidance information from such techniques supplies facts about problems in the program and provides possible answers for the further improvement of program efficiency. A performance measurement system, called IPS, has been developed as a prototype of the authors' model and design. Some of the test results from IPS are also discussed     

Performance evaluation of manufacturing systems: a spreadsheet model

In this paper, we describe the spreadsheet modeling of manufacturing systems which are represented by multi-server and multi-product open networks. The spreadsheet software is characterized by easy data manipulation, on-screen numerical and visual feedback, fast calculation (what-if analysis), and its availability. So far, most spreadsheet models of this type have been used to solve static and deterministic problems in manufacturing systems, ignoring most of the uncertainties. We propose here that a spreadsheet can be implemented for capturing not only static features but also stochastic behaviour of manufacturing systems. The experiments show that the results from the spreadsheet model are reasonably close to those from the other existing approaches. The proposed spreadsheet model could be applied by an engineer to the modeling of manufacturing systems, especially for a first-cut design stage, utilizing his/her own spreadsheet software.     

An MHP framework to provide intelligent personalized recommendations about digital TV contents

Digital Television will bring a significant increase in the amount of channels and programs available to end users, with many more difficulties to find contents appealing to them among a myriad of irrelevant information. Thus, automatic content recommenders should receive special attention in the following years to improve their assistance to users. The current content recommenders have important deficiencies that hamper their wide acceptance. In this paper, we present a new approach for automatic content recommendation that significantly reduces those deficiencies. This approach, based on Semantic Web technologies, has been implemented in the AdVAnced Telematic search of Audiovisual contents by semantic Reasoning tool, a hybrid content recommender that makes extensive use of well‐known standards, such as Multimedia Home Platform, TV‐Anytime and OWL. Also, we have carried out an experimental evaluation, the results of which show that our proposal performs better than other existing approaches. Copyright © 2007 John Wiley & Sons, Ltd.     

Expert Systems and the "Myth" of Symbolic Reasoning

Elements of the artificial intelligence approach to expert systems offer great productivity advantages over traditional approaches to application systems development, even though the end result may be a program employing no AI techniques. These productivity advantages are the hidden truths behind the "myth" that symbolic reasoning programs are better than ordinary ones.     

Automated test generation using model checking: an industrial evaluation

In software development, testers often focus on functional testing to validate implemented programs against their specifications. In safety-critical software development, testers are also required to show that tests exercise, or cover, the structure and logic of the implementation. To achieve different types of logic coverage, various program artifacts such as decisions and conditions are required to be exercised during testing. Use of model checking for structural test generation has been proposed by several researchers. The limited application to models used in practice and the state space explosion can, however, impact model checking and hence the process of deriving tests for logic coverage. Thus, there is a need to validate these approaches against relevant industrial systems such that more knowledge is built on how to efficiently use them in practice. In this paper, we present a tool-supported approach to handle software written in the Function Block Diagram language such that logic coverage criteria can be formalized and used by a model checker to automatically generate tests. To this end, we conducted a study based on industrial use-case scenarios from Bombardier Transportation AB, showing how our toolbox CompleteTest can be applied to generate tests in software systems used in the safety-critical domain. To evaluate the approach, we applied the toolbox to 157 programs and found that it is efficient in terms of time required to generate tests that satisfy logic coverage and scales well for most of the programs.     

Socio-technical systems: From design methods to systems engineering

It is widely acknowledged that adopting a socio-technical approach to system development leads to systems that are more acceptable to end users and deliver better value to stakeholders. Despite this, such approaches are not widely practised. We analyse the reasons for this, highlighting some of the problems with the better known socio-technical design methods. Based on this analysis we propose a new pragmatic framework for socio-technical systems engineering (STSE) which builds on the (largely independent) research of groups investigating work design, information systems, computer-supported cooperative work, and cognitive systems engineering. STSE bridges the traditional gap between organisational change and system development using two main types of activity: sensitisation and awareness; and constructive engagement. From the framework, we identify an initial set of interdisciplinary research problems that address how to apply socio-technical approaches in a cost-effective way, and how to facilitate the integration of STSE with existing systems and software engineering approaches.     

Using interpolation to improve path planning: The Field D* algorithm

We present an interpolation‐based planning and replanning algorithm for generating low‐cost paths through uniform and nonuniform resolution grids. Most grid‐based path planners use discrete state transitions that artificially constrain an agent's motion to a small set of possible headings (e.g., 0, π/4, π/2, etc.). As a result, even “optimal” grid‐based planners produce unnatural, suboptimal paths. Our approach uses linear interpolation during planning to calculate accurate path cost estimates for arbitrary positions within each grid cell and produce paths with a range of continuous headings. Consequently, it is particularly well suited to planning low‐cost trajectories for mobile robots. In this paper, we introduce a version of the algorithm for uniform resolution grids and a version for nonuniform resolution grids. Together, these approaches address two of the most significant shortcomings of grid‐based path planning: the quality of the paths produced and the memory and computational requirements of planning over grids. We demonstrate our approaches on a number of example planning problems, compare them to related algorithms, and present several implementations on real robotic systems.     

The organic grid: self-organizing computation on a peer-to-peer network

Desktop grids have been used to perform some of the largest computations in the world and have the potential to grow by several more orders of magnitude. However, current approaches to utilizing desktop resources require either centralized servers or extensive knowledge of the underlying system, limiting their scalability. We propose a new design for desktop grids that relies on a self-organizing, fully decentralized approach to the organization of the computation. Our approach, called the organic grid, is a radical departure from current approaches and is modeled after the way complex biological systems organize themselves. Similar to current desktop grids, a large computational task is broken down into sufficiently small subtasks. Each subtask is encapsulated into a mobile agent, which is then released on the grid and discovers computational resources using autonomous behavior. In the process of "colonization" of available resources, the judicious design of the agent behavior produces the emergence of crucial properties of the computation that can be tailored to specific classes of applications. We demonstrate this concept with a reduced-scale proof-of-concept implementation that executes a data-intensive independent-task application on a set of heterogeneous, geographically distributed machines. We present a detailed exploration of the design space of our system and a performance evaluation of our implementation using metrics appropriate for assessing self-organizing desktop grids.     

Checking inside the black box: regression testing by comparing value spectra

Comparing behaviors of program versions has become an important task in software maintenance and regression testing. Black-box program outputs have been used to characterize program behaviors and they are compared over program versions in traditional regression testing. Program spectra have recently been proposed to characterize a program's behavior inside the black box. Comparing program spectra of program versions offers insights into the internal behavioral differences between versions. In this paper, we present a new class of program spectra, value spectra, that enriches the existing program spectra family. We compare the value spectra of a program's old version and new version to detect internal behavioral deviations in the new version. We use a deviation-propagation call tree to present the deviation details. Based on the deviation-propagation call tree, we propose two heuristics to locate deviation roots, which are program locations that trigger the behavioral deviations. We also use path spectra (previously proposed program spectra) to approximate the program states in value spectra. We then similarly compare path spectra to detect behavioral deviations and locate deviation roots in the new version. We have conducted an experiment on eight C programs to evaluate our spectra-comparison approach. The results show that both value-spectra-comparison and path-spectra-comparison approaches can effectively expose program behavioral differences between program versions even when their program outputs are the same, and our value-spectra-comparison approach reports deviation roots with high accuracy for most programs.