In real sequences, one of the factors that most negatively affects the segmentation process result is the existence of scene noise. This impairs object segmentation which has to be corrected if we wish to have some minimum guarantees of success in the following tracking or classification stages. In this work we propose a generic knowledge-based model to improve the segmentation process. Specifically, the model uses a decomposition strategy in description levels to enable the feedback of information between adjacent levels. Finally, two case studies are proposed that instantiate the model proposed for detecting humans.