Empirically evaluating OCL and Java for specifying constraints on UML models

The Object Constraint Language (OCL) has been applied, along with UML models, for various purposes such as supporting model-based testing, code generation, and automated consistency checking of UML models. However, a lot of challenges have been raised in the literature regarding its applicability in industry such as extensive training, slow learning curve, and significant effort to use OCL due to lack of familiarity of practitioners. To confirm these challenges, empirical evidence is needed, which is severely lacking in the literature. To build such preliminary evidence, we report a controlled experiment that was designed to evaluate OCL by comparing it with Java; a programming language that has also been used to specify constraints on UML models. Results show that the participants using OCL perform as good as the participants working with Java in terms of three objective quality metrics (i.e., completeness, conformance and redundancy) and two subjective metrics (i.e., applicability and confidence level). In addition, the participants using OCL performed consistently well for all the constraints of varying complexity, while fluctuating results were obtained for the participants using Java for the same constraints. Based on the empirical evidence, we can conclude that it does not make much difference to use OCL or Java for specifying constraints on UML models. However, the participants working with OCL performed consistently well on specifying constraints of varying complexity suggesting that OCL can be used to model complicated constraints (commonly observed in industrial applications) with the same quality as for simpler constraints. Moreover, additional analyses on the constraints when using Java and OCL tools revealed that tools are needed to specify fully correct constraints that can be used to support automation.