We present a framework for robotic cognitive control endowed with adaptive mechanisms for attentional regulation and task execution. In cognitive psychology, cognitive control is the process that orchestrates executive and cognitive processes supporting adaptive responses and complex goal-directed behaviors. Similar mechanisms can be deployed in robotic systems in order to flexibly execute complex structured tasks. In this work, following a supervisory attentional system paradigm, we propose an approach that permits to learn how to exploit top-down and bottom-up attentional regulations to guide the execution of hierarchically structured tasks. We present the overall framework discussing its functioning in a mobile robot case study considering pick-carry-place tasks. In this setting, we show that the proposed system can be on-line trained by a user in order to execute incrementally complex activities.