Sparse representation based on over-complete dictionary is a new signal representation theory. Recent activity in this field concentrated mainly on the study of sparse decomposition algorithm and dictionary design algorithm. In this paper, a novel dictionary design algorithm called K-LMS is proposed. It generalized the k-means clustering process, for adapting dictionaries to achieve sparse representation of signals. As regards to the image denoising, a new denoising method is introduced. With the application of image's sparse representations in over-complete dictionary, it reconstructs a simple threshold to realize image denoising. Experimental results demonstrate the effectiveness of the proposed method.