The development of the HiPE system: design and experience report

The concurrent functional programming language Erlang has been designed to ease the development of large-scale distributed soft real-time control applications. So far, it has been used quite successfully in industry, both within Ericsson Telecom, where it was designed and developed, and by other companies. This declarative language success-story has taken place despite the fact that Erlang implementations are slow compared with implementations of other functional languages. Wanting to improve the performance aspects of publicly available Erlang implementations, which are based on emulators, we embarked on a project called HiPE (High-Performance Erlang) whose aim has been to develop an efficient just-in-time native code compiler for Erlang (called the HiPE system). Since its start in 1996, the system has gone through various (re-)design phases, partly due to implementation choices that did not turn out to be as promising as they appeared on paper, but mainly due to changes in Ericssons Erlang system upon which the HiPE system is built. In this article, we describe how the HiPE system was developed, what it currently looks like, and its current performance. We critically examine design decisions that we took, and the main lessons learnt from implementing them. Finally, we also report on our experiences from trying to keep up with the concurrent development of Ericssons base Erlang system. As such, this article both documents the HiPE system and can serve as possible guidance to anyone wishing to attempt a similar feat.