The present paper describes a novel adaptive cross correlation technique of face recognition using closed loop discriminator estimation for face detection. All possible variations in the human face can be obtained by scaling and three types of head roll in different planes. We show here that face recognition system comprises face detection and face verification. Feature selection schemes like eigenfaces, local binary pattern (LBP) and speeded up robust features are implemented to extract features with high discriminative power, thereby performing face verification. Experiments revealed that this method strikes a balance between accurate face recognition and identification with sufficient speed of convergence. The algorithm proposed by this paper allows tracking of face against a wide scale range with sufficient immunity against like camera vibrations, sensor errors, illumination level fluctuation etc.