Decision rules for classification problem are requested to be open for public inspection to ensure the fairness. We may construct a more effective and useful classifier by combining the rules published by different organizations and companies with rules induced from our own dataset. However, few studies have devoted to the combination of the published rules with rules induced from individually owned dataset. In this paper, we propose a mixture model approach to constructing an effective binary classifier by combining the published rules and individually owned rules. We apply the idea of the mixture model to combine two score distributions over classes obtained from the published rules and individually owned rules. In order to obtain score distributions, we apply LERS algorithm to each of published and individually owned rulesets. LERS algorithm evaluates the score showing to what extent the object is in each class. We normalize the scores. EM algorithm is then used for optimizing the mixture ratios for constructing a classifier. Numerical experiments are conducted to examine the performance of the proposed approach. Our approach is compared with four conceivable methods in four real datasets. The effectiveness of the proposed approach is demonstrated by the numerical experiments.