In frequency domain blind source separation (FDBSS), separated frequency bin data in the same source must be grouped together before outputting the final result, which is the well-known permutation problem. Clustering techniques are broadly used in solving the permutation problem, however, some challenges still exist, for example, elongated datasets should be handled, and constraint from the background knowledge must be considered. Inspired by various successful applications of kernel and spectral clustering methods in machine learning and data mining community, we try to solve the permutation problem by these methods. In this paper, the weighted kernel k-means algorithm is modified according to the specific requirement of the permutation problem, and the spectral interpretation of the kernel approach is also investigated. In addition, we propose several kernel construction approaches to improving the permutation performance. Different experiments are carried out on a uniform platform, and show better performance of the proposed approach.