The process of assigning part of speech for every word in a given sentence according to the context is called as part of speech tagging. Part of speech tagging (POS tagging) plays an important role in the area of natural language processing (NLP) including applications such as speech recognition, speech synthesis, natural language parsing, information retrieval, multi words term extraction, word sense disambiguation and machine translation. This paper proposes an efficient and accurate POS tagging technique for Malayalam language using hybrid approach. We propose a Conditional Random Fields(CRF) based method integrated with Rule-Based method. We use SVM based method to compare the accuracy. The corpus both tagged and untagged used for training and testing the system is in the unicode format. The tagset developed by IIIT Hyderabad for Indian Languages is used. The system is tested for selected books of Bible and perform with an accuracy of 94%.