In this paper, a metric for comparing different designs of variable stiffness actuators is introduced. For the formulation of this metric, we focus on the energy efficiency of the actuators. In particular, we propose a metric that is a measure of how much energy is used by the actuator for changing the output stiffness. In order to facilitate the analysis of the energy usage, we present a port-based modeling framework, from which design criteria are derived for the optimization of the metric. Finally, the metric is interpreted in a comparison between existing actuators.