Using UML/MARTE to support performance tuning and stress testing in real-time systems

Real-time embedded systems (RTESs) operating in safety-critical domains have to satisfy strict performance requirements in terms of task deadlines, response time, and CPU usage. Two of the main factors affecting the satisfaction of these requirements are the configuration parameters regulating how the system interacts with hardware devices, and the external events triggering the system tasks. In particular, it is necessary to carefully tune the parameters in order to ensure a satisfactory trade-off between responsiveness and usage of computational resources, and also to stress test the system with worst-case inputs likely to violate the requirements. Performance tuning and stress testing are usually manual, time-consuming, and error-prone processes, because the system parameters and input values range in a large domain, and their impact over performance is hard to predict without executing the system. In this paper, we provide an approach, based on UML/MARTE, to support the generation of system configurations predicted to achieve a satisfactory trade-off between response time and CPU usage, and stress test cases that push the system tasks to violate their deadlines. First, we devise a conceptual model that specifies the abstractions required for analyzing task deadlines, response time, and CPU usage, and provide a mapping between these abstractions and UML/MARTE. Then, we prune the UML/MARTE metamodel to only contain a purpose-specific subset of entities needed to support performance tuning and stress testing. The pruned version is a supertype of UML/MARTE, which ensures that all instances of the pruned metamodel are also instances of UML/MARTE. Finally, we cast the generation of configurations and stress test cases as two constrained optimization problems (COPs) over our conceptual model. The input data for these COPs in automatically generated via a model-to-text (M2T) transformation from models specified in the pruned UML/MARTE metamodel to the Optimization Programming Language. We validate our approach in a safety-critical RTES from the maritime and energy domain, showing that (1) our conceptual model can be applied in an industrial setting with reasonable effort, and (2) the optimization problems effectively identify configurations predicted to minimize response time and CPU usage, and stress test cases that maximize deadline misses. Based on our experience, we highlight challenges and potential issues to be aware of when using UML/MARTE to support performance tuning and stress testing in an industrial context.