Business process management and automation has been the focus of intense research for a long time. Today, a plethora of process languages for specifying and implementing process models have evolved. Examples for such languages are established international standards, such as the Web Services Business Process Execution Language 2.0 or, more recently, the Business Process Model and Notation 2.0. Implementations of these standards which are able to execute models, so called process engines, differ in the quality of service they provide, e.g., in performance or usability, but also in the degree to which they actually implement a given standard. Selecting the “best” engine for a particular usage scenario is hard, as none of the existing process standards features an objective certification process to assess the quality of its implementations. To fill this gap, we present our work on process engine benchmarking. We discuss what has been achieved so far and point out future directions that deserve further investigation.