Migrating traditional access control into Role Based Access Control (RBAC) lightens us a practical way to improve the management efficiency while maintaining system security. However, since security and reliability of RBAC are mostly depending on the quality of roles, how to find proper roles is a challenging problem. However, few existing approaches have taken abnormal configurations in the original data into consideration, these abnormal configurations always lead to wrong roles and thus cause tremendous security risks. To address this problem, we propose a novel role mining framework considering user similarities and abnormal configurations, simultaneously. Specifically, we propose a novel spectral clustering algorithm based on a newly designed similarity function. Then an abnormal configuration hunting method is proposed to target potential abnormal assignments and give recommendations to correct these configurations based on clustering results. Experimental results show its performance over existing solutions.