Traditionally, multi-plane Support Vector Machines including twin support vector machine (TWSVM) and least squares twin support vector machine (LSTSVM) essentially fail to fully discover the local geometry inside the samples that may be important for classification performance and only preserve the global data structure. This motivates the rush towards new classifiers that can take advantage of underlying local data manifold. In this paper, we first indicate that both TWSVM and LSTSVM are essentially to solve two sub optimizations of the standard regularization method. Illuminated by several new-proposed geometrically motivated algorithms we then propose a graph learning algorithms based on LSTSVM, which is designed for classification and are constructed based on a new form of manifold regularization. Experimental evidence suggests that our methods are effective in performing classification task.