The key frame extraction from a video sequence is a crucial step for content-based video analysis, with key frames clients can summarize a long video and know about the content of the video. In this paper, we propose a novel scheme to extract key frames based on kernel locality preserving learning, for the purpose of video shot summarizing. Under the consistency assumption, we realize that the relationship between the frame feature space and the kernel high-dimensional (semantic) space is Local Linear Embedding, thus we represent the key frame by the linear combination of several neighboring frames and the key frame is corresponding to the center of the feature vectors (the map of kernel-mapping) in the high-dimensional (semantic) space. The experimental results demonstrate that the proposed scheme is efficient and effective.