Ear and face based multimodal recognition could fully utilize their connection relationship, and implement recognizing people without their cooperation because of earpsilas special physiological location and structure. In this paper a kernel-based feature fusion algorithm is presented and applied to multimodal recognition based on fusing ear and face. With the algorithm, the associated feature vectors of ear and profile face are established for the following classification. The experimental results show that the method is efficient for feature fusion, and the multimodal recognition based on ear and profile face performs better than ear or profile face unimodal biometric recognition.