With increasing connectivity between computers, the need to keep networks secure becomes more and more vital. Intrusion detection systems have become an essential component of network security to supplement existing defenses. This paper proposes a novel intrusion detection system, which combines the supervised classifiers and unsupervised clustering to detect intrusions. Decision tree, naive Bayes and Bayesian clustering are used at different levels. We also have made improvements to the Na'ive Bayes algorithm by choosing different attributes for different classes. The experiments demonstrate the effectiveness of the proposed approach, especially for U2R and R2L type attacks. The detection rate is significantly improved.