Effective operation of cognitive radios (CRs) requires sensing the spectrum and dynamic adaptation of the available resources according to the sensed information. Although sensing and resource allocation are coupled, most existing designs optimize each of the tasks separately. This work optimizes them jointly for an underlay CR paradigm. The formulation considers that secondary users adapt their power and rate based on the available imperfect channel state information, while taking into account the cost associated with acquiring such an information. The objective of the optimization is twofold: maximize the (sum-rate) performance of the CR and protect the primary users through an average interference constraint. Designing the sensing in our underlay paradigm amounts to decide what channel/frequency slots are sensed at every time instant. Partial observability of the channel state (due to noisy and outdated information) calls for (Bayesian) sequential estimators to keep track of the interference channel gains, as well as for dynamic programming tools to design the optimal schemes. Together with the optimal schemes, a simple approximate solution is also developed.