Deep methods based on Convolutional Neural Networks serve as accurate facial points and body parts detectors. However, most methods do not provide a confidence score for the quality of the localization process. In real world applications, such a score could be invaluable. We, therefore, study the problem of estimating the success of the localization process during test time. Our method is based on mapping the network activation features to the area under the point-accuracy-curve. Our method greatly outperforms methods that were recently suggested, such as those which are based on the stability of the detection or on the norm of the representation layer.