Face recognition is itself a very challenging task and it becomes more challenging when the input images have intra class variations and inter class similarities in a large scale. Yet the recognition accuracy can be improved in some extent by supporting the system with non-matched templates. Therefore a set of cohort images is used in this regard. But all the cohort templates of the initial cohort pool may not be relevant for each and every enrolled subject. So the main focus of this work is to select a subject specific and meaningful cohort subset. This paper proposes a cohort selection method called K-medoids Cohort Selection (KMCS) to select a reference set of non-matched templates which are almost appropriate to the respective subjects. Basically, all cohort scores of a subject are clustered first using K-medoids clustering. Afterward the cluster having more scattered members/scores from its medoid is selected as a cohort subset because this cluster is constituted with the cohorts carrying more discriminative features compared to others. The SIFT points and SURF points are extracted as facial feature. The experiments are conducted on FEI, ORL and Look-alike databases of face images. The matching scores between probe and query images are normalized using T-norm, Max-Min and Aggarwal (Max rule) cohort score normalization techniques before taking the final decision of acceptance or rejection. The results obtained from the experiments show the domination of the proposed system over the non-cohort face recognition system as well as random and Top 10 cohort selection methods. There is another comparative study between k-means and K-medoids clustering for cohort selection.