We discuss how a binary detector can learn whether it is being subject to an oracle attack by resorting to a higher level of detection (metadetection). On a second step, assuming that the attacker is aware of the fact that the detector takes countermeasures, we investigate a possible way for him to react. Then, we study the interplay between the defender and the attacker when both of them try to do their best for pursuing their opposite goals. We focus our analysis on the metadetection of oracle attacks based on line search algorithms, as they are prevalent in the literature. In such scenario, we propose metadetectors, which work under very general conditions, that is, when the oracle is not exclusively fed with line search attacking queries, but only some of the malicious queries are made along the lines, whereas the others are done by mimicking the behavior of honest users. We theoretically evaluate the final achievable performance of these metadetectors, deriving conditions under which asymptotic powerful testing is possible. Experimental results show the power of metadetection for countering the line search attacks in both synthetic and real application scenarios.