Theory of reliability and mathematical statistics provides us tools for monitoring the quality of a test. Using methodologies often covers estimation of validity, reliability and transparency. Of course quality tests depend on quality learning objectives and difficulty of questions. Such a study adds another dimension to perception of the causes of poor test results. Even though many authors warn about these criteria, these tools can provide useful information to the teacher. This paper presents the developed software for quality of test evaluation. The evaluation is carried out using real data sets describing test evaluation of the course Probability and Statistics at the Faculty of electrical engineering and informatics, University of Pardubice.