As traveling wave fault location cannot be applied due to the low sampling rate in smart substation, a novel algorithm of transmission line fault location for smart substation is proposed in this paper. When there is a transmission line fault, the algorithm can select the best fault location method in the conventional single-ended frequency fault location methods, and then get a value that is the closest to the fault. The conventional frequency fault location methods have various sources of error, and fault location result is affected by power source, the opposite system impedance, transition resistance and other parameters. In the different fault case, every method has different location accuracy, namely the most accurate location method is always varying with varying parameters. In this paper, firstly we construct a large number of training samples of transmission line fault, secondly, the rough set theory is used to reduce training samples and find the intrinsic relationship between fault location and system parameters. When transmission line has a fault, this algorithm can search for a most accurate fault location method by k-Nearest Neighbor method and finally give the most credible fault distance. The results of ATP simulation an RTDS simulation shows that the algorithm can successfully keep away from the method of larger system error and select the optimal method and greatly improve the fault location accuracy.