In this paper, a fast CU depth decision algorithm based on support vector machine (SVM) is proposed to reduce the computational complexity of HEVC intra coding. It is systematic to develop the criterion of early CU splitting and termination by applying SVM. Appropriate features for training SVM models are extracted from spatial domain and pixel domain. Artificial neural network is used to analyze the impact of each extracted feature on CU size decision, and different weights are assigned to the output of SVMs. The experimental results show that the proposed fast algorithm provides 58.9% encoding time saving at most, and 46.5% time saving on average compared with HM 12.1.