Voice over IP (VoIP) telephony is vulnerable to a range of attacks, since its operation relies on the underlying IP network. The centralized design of the major VoIP signalling protocols such as the Session Initiation Protocol (SIP) makes the registration server a target for CPU-based denial of service (DoS) attacks. In this paper, we propose a history-based IP filtering layer to defeat these DoS attacks by blocking the SIP packets from previously unseen sources. Our empirical evaluation shows that our approach achieves significant improvement in CPU utilization under DoS attacks.