This paper compares empirically four bagging-based ensemble classifiers, namely the ensemble adaptive neuro-fuzzy inference system (ANFIS), the ensemble support vector machine (SVM), the ensemble extreme learning machine (ELM) and the random forest. The comparison of these four ensemble classifiers is novel because it has not been reported in the existing literature. The classifiers are evaluated with thirteen binary class datasets and the empirical results show that the ensemble methods employed in the four ensemble classifiers boost the testing accuracy by 1–5% on average from their base classifiers. In addition, the testing accuracy can be improved by increasing the number of base classifiers. The empirical results also show that the bagging SVM is the most favorable ensemble classifier among them.