Continuous and real-time learning is a difficult problem in robotics. This paper investigates how learning in the input layer of the cerebellum may successfully encode contextual knowledge in a representation useful for coordination and lifelong learning, and proposes that a sparsely distributed and statistically independent representation provides a valid criterion for the self-organizing classification and integration of context signals. This representation is beneficial for learning in the cerebellum by simplifying the credit assignment problem between what must be learned and the relevant signals in the current context for learning it and for life-long learning by reducing the destructive interference across tasks, while retaining the ability to generalize