Face recognition represents an appealing solution for biometrics-based unobtrusive and flexible person authentication. However, most solutions proposed in the literature suffer from some significant drawbacks, such as high computational complexity, the need for a centralized biometrics database (which is not desirable due to widespread international provisions discouraging collections of sensitive personal data), and limited scalability on a large number of enrolled subjects. We propose a novel person authentication solution based on a cascade of face recognition and pattern matching algorithms that not only provides high reliability and robustness against impostors but also stores in a personal radio frequency identification (RFID) tag all the needed individual biometrics information of the user, who therefore always remains in control, and has the exclusive availability, of such sensitive data. This paper describes the proposed approach, called RFaceID, and discusses its performance in terms of the ratio between false acceptance rate and false rejection rate and in terms of authentication time when applied to the VidTIMIT, Extended Yale B, and Mobile Biometry (MOBIO) widely adopted face databases.