Email is a rapid and cheap communication medium for sending and receiving information where spam is becoming a nuisance for such communication. A good spam filtering cannot only be achieved by high performance accuracy but low false positive is also necessary. This paper presents a combining classifiers approach with committee selection mechanism where the main objective is to combine individual decisions of the good classifiers for utmost classification outcome in spam classification domain. In this context, three different classifiers have been selected i.e. "Boosted Bayesian", "Boosted Naïve Bayes and Support Vector Machine (SVM). For combining classifiers, boosted bayesian and boosted naïve bayes are chosen as members of committee and SVM is taken as the president. The member of committee have been selected from our previous study where we have identified boosting with adaboost improves the performance of probabilistic classifier. Results show the best results of novel combining classifier approach in compression with individual classifiers compared in terms of good performance accuracy and low false positives. In addition, greedy step wise feature search method is found to be good in this study.