Test Case Generation as an AI Planning Problem

While Artificial Intelligence techniques have been applied to a variety of software engineering applications, the area of automated software testing remains largely unexplored. Yet, test cases for certain types of systems (e.g., those with command language interfaces and transaction based systems) are similar to plans. We have exploited this similarity by constructing an automated test case generator with an AI planning system at its core. We compared the functionality and output of two systems, one based on Software Engineering techniques and the other on planning, for a real application: the StorageTek robot tape library command language. From this, we showed that AI planning is a viable technique for test case generation and that the two approaches are complementary in their capabilities.     

Scalability issues with using FSMWeb to test web applications

Web applications are fast becoming more widespread, larger, more interactive, and more essential to the international use of computers. It is well understood that web applications must be highly dependable, and as a field we are just now beginning to understand how to model and test Web applications. One straightforward technique is to model Web applications as finite state machines. However, large numbers of input fields, input choices and the ability to enter values in any order combine to create a state space explosion problem. This paper evaluates a solution that uses constraints on the inputs to reduce the number of transitions, thus compressing the FSM. The paper presents an analysis of the potential savings of the compression technique and reports actual savings from two case studies.     

Building test cases and oracles to automate the testing of web database applications

Many organizations rely on web applications that use back-end databases to store important data. Testing such applications requires significant effort. Manual testing alone is often impractical, so testers also rely on automated testing techniques. However, current automated testing techniques may produce false positives (or false negatives) even in a perfectly working system because the outcome of a test case depends on the state of the database which changes over time as data is inserted and deleted. The Automatic Database Tester (AutoDBT) generates functional test cases that account for database updates. AutoDBT takes as input a model of the application and a set of testing criteria. The model consists of a state transition diagram that shows how users navigate pages, a data specification that captures how data flows, and an update specification that shows how the database is updated. AutoDBT generates guard queries to determine whether the database is in a state conducive to performing and evaluating tests. AutoDBT also generates partial oracles to help validate whether a back-end database is updated correctly during testing. This paper describes the design of AutoDBT, a prototype implementation, several experiments with the prototype, and four case studies.     

Demonstrating the effect of vertical and directional shear for resource mapping of wind power

The use of wind energy is growing around the world, and its growth is set to continue into the foreseeable future. Estimates of the wind speed and power are helpful to assess the potential of new sites for development and to facilitate electric grid integration studies. In the present paper, wind speed and power resource mapping analyses are performed. These resource mappings are produced on a 13 km, hourly model grid over the entire continental USA for the years of 2006–2014. The effects of the rotor equivalent wind speed (REWS) along with directional shear are investigated. The total dataset (wind speed and power) contains ≈152,000 model grid points, with each location containing ≈78,000 hourly time steps. The resource mapping and dataset are created from analysis fields, which are output from an advanced weather assimilation model. Two different methods were used to estimate the wind speed over the rotor swept area (with rotor diameter of 100 m). First, using a single wind speed at hub height (80 m) and, second, the REWS with directional shear. The demonstration study shows that in most locations the incorporation of the REWS reduces the average available wind power. In addition, the REWS technique estimates more wind power production at night and less production in the day compared with the hub height technique; potentially critical for siting new wind turbines and plants. However, the wind power estimate differences are dependent on seasonality, diurnal cycle and geographic location. More research is warranted into these effects to determine the level at which these features are observed at actual wind plants.© 2015 The Authors. Wind Energy published by John Wiley & Sons, Ltd.     

Testing Web applications by modeling with FSMs

Researchers and practitioners are still trying to find effective ways to model and test Web applications. This paper proposes a system-level testing technique that combines test generation based on finite state machines with constraints. We use a hierarchical approach to model potentially large Web applications. The approach builds hierarchies of Finite State Machines (FSMs) that model subsystems of the Web applications, and then generates test requirements as subsequences of states in the FSMs. These subsequences are then combined and refined to form complete executable tests. The constraints are used to select a reduced set of inputs with the goal of reducing the state space explosion otherwise inherent in using FSMs. The paper illustrates the technique with a running example of a Web-based course student information system and introduces a prototype implementation to support the technique.     

Assessing Project Success Using Subjective Evaluation Factors

Project evaluation is essential to understand and assess the key aspects of a project that make it either a success or failure. The latter is influenced by a large number of factors, and many times it is hard to measure them objectively. This paper addresses this by introducing a new method for identifying and assessing key project characteristics, which are crucial for a project's success. The method consists of a number of well-defined steps, which are described in detail. The method is applied to two case studies from different application domains and continents. It is concluded that patterns are possible to detect from the data sets. Further, the analysis of the two data sets shows that the proposed method using subjective factors is useful, since it provides an increased understanding, insight and assessment of which project factors might affect project success.     

Ethical Issues in Empirical Software Engineering: The Limits of Policy

Empirical studies in software engineering can involve a variety of organizations, each with their own set of policies and procedures geared at safeguarding the interests and responsibilities of the researchers, students, the collaborating company, the university, and possibly national funding agencies like the National Science Foundation and the National Institute of Health. Each of these organizations have differing goals for participating in these studies and bring widely different cultures and expectations to the table. While policies, procedures, contracts, and agreements set expectations, they by themselves cannot ensure ethical behavior. This position paper describes some of the common approaches to encourage ethical behavior and their limits for enforcing ethical behavior.