Data representation plays an important role in a classifier's accuracy. A given dataset may lead to better results by simply applying a change of basis while keeping the original number of parameters. In this paper, Gabor Filter based image representation has been exploited for object classification. First, Gabor filter based convolution is computed for features extraction, then down-sampling is performed and features are normalized to zero mean and unit variance. This image representation having discriminative visual patterns is used for training of object classifier in Matlab Neural Toolbox. Performance of this proposed image representation is examined on two real world image datasets CIFAR and MNIST and results show that data representation using Gabor can provide good classification without increasing the number of trainable parameters. Finally, this approach is compared to different configurations of Convolutional Neural Network having trainable parameters to verify the validity of proposed image representation.