In cognitive radio networks, it is important to effectively use the under-utilized spectrum resources without affecting the primary users. In an underlay system, cognitive users are allowed to share the channel simultaneously with primary users with the restriction on interference level but not in an overlay system. In this paper, we consider a system where cognitive users can switch between overlay and underlay modes of operation in order to improve their throughput. The results, based on Markov chain analysis, are satisfactorily verified using Monte-Carlo simulation. It is shown that proper selection of transmission mode can provide greater improvement in throughput for cognitive users. If a primary user occupies the channel for a longer (shorter) period, then the system should allow the cognitive users to choose underlay (overlay) mode for throughput advantage. The exact strategy (or mode) switching thresholds can be found from our analysis and it depends primarily on the transition probabilities of the primary users and throughput ratio during underlay/overlay transmission.