OREX-J: towards a universal software framework for the experimental analysis of optimization algorithms

The Operations Research EXperiment Framework for Java (OREX-J) is an object-oriented software framework that helps users to design, implement and conduct computational experiments for the analysis of optimization algorithms. As it was designed in a generic way using object-oriented programming and design patterns, it is not limited to a specific class of optimization problems and algorithms. The purpose of the framework is to reduce the amount of manual labor required for conducting and evaluating computational experiments: OREX-J provides a generic, extensible data model for storing detailed data on an experimental design and its results. Those data can include algorithm parameters, test instance generator settings, the instances themselves, run-times, algorithm logs, solution properties, etc. All data are automatically saved in a relational database (MySQL, http://www.mysql.com/) by means of the object-relational mapping library Hibernate (http://www.hibernate.org/). This simplifies the task of analyzing computational results, as even complex analyses can be performed using comparatively simple Structured Query Language (SQL) queries. Also, OREX-J simplifies the comparison of algorithms developed by different researchers: Instead of integrating other researchers’ algorithms into proprietary test beds, researchers could use OREX-J as a common experiment framework. This paper describes the architecture and features of OREX-J and exemplifies its usage in a case study. OREX-J has already been used for experiments in three different areas: Algorithms and reformulations for mixed-integer programming models for dynamic lot-sizing with substitutions, a simulation-based optimization approach for a stochastic multi-location inventory control model, and an optimization model for software supplier selection and product portfolio planning.