Motivated by the Vector of Locally Aggregated Descriptors (VLAD), we propose a new Histograms of Locally Aggregated Oriented Gradients descriptor (called HLAOG). In the Histograms of Oriented Gradients descriptor (HOG), the zero-order information of the gradients is captured. By contrast, in the HLAOG descriptor we accumulate the differences between gradient orientations and their nearest bin centers, which characterizes the distribution of the gradient orientations in regard to the bin centers. The HLAOG descriptor is demonstrated to be complementary to HOG in the experiments. Then, for setting the weights of the votes on different bins in a better way, we choose Gaussian function as the weighting method and present another new Gaussian Weighted Histograms of Oriented Gradients descriptor (called GWHOG) based on HOG. Evaluations on two public object recognition datasets (Caltech-101 and VOC2007) show that the combination of HOG and HLAOG outperforms HOG and the combination of HLAOG and GWHOG gets the best result.