It has been a bottleneck problem to establish a knowledge database in expert system of disease diagnosis. The quality of knowledge database has an impact not only on subsequent consequence but also on diagnostic precision. This paper proposes an improved method of rough sets, and then the method is applied into rheumatic mitral stenosis disease diagnosis. Abstracting rules based on rough set theory is used to analyze the diagnosed cases data and establish the fine-drawn relationship. The abstracting rules technique combined with doctors' clinical experience is also used to set up a relatively integrated diagnostic knowledge base and set a good groundwork for actual inference and explanation program. Our experimental results show that the method is efficient.