Business Process Management Systems (BPMS) aim to support the Business Process Management paradigm and to ease legacy application integration. Often, they rely on Service-oriented Architecture (SoA). However, do such systems really meet real-world requirements? This paper introduces and discusses a set of criteria which are important for BPMS and applies these criteria in comparing tools from three important vendors, namely IDS Scheer, Oracle and Intalio based on a case study.