Comparison and evaluation of goal-oriented satisfaction analysis techniques

Goal-oriented requirements engineering (GORE) has been introduced as a means of modeling and understanding the motivations for system requirements. Using models to make goals explicit helps to avoid system failures due to implementing the wrong requirements or ignoring certain stakeholder needs. These models are unique when compared to other models used in system analysis in that their structure naturally lends itself to an analysis of goal satisfaction. Existing work claims that analysis using goal models can facilitate decision making over functional or design alternatives, using criteria in the model. Many different approaches to the analysis of goal-oriented requirements models have been proposed, including several procedures that analyze the satisfaction or denial of goals. These procedures make different choices in their interpretation of the goal model syntax, the methods to resolve conflicting or partial evidence, and in the way they represent satisfaction. This work uses three available tools implementing seven similar goal satisfaction analysis procedures to analyze three sample goal models. Results are reported and compared. The purpose of this comparison is to understand the ways in which procedural design choices affect analysis results, and how differences in analysis results could lead to different recommendations over alternatives in the model. Our comparison shows that different satisfaction analysis techniques for goal models can produce variable results, depending on the structure of the model. Comparison findings lead us to recommend the use of satisfaction analysis techniques for goal models as only heuristics for decision making. Our results emphasize investigation into the benefits of satisfaction analysis beyond decision making, namely improving model quality, increasing domain knowledge, and facilitating communication.