The accuracy of speech recognition from image data can be improved using a lip‐reading method, even in noisy environments. In related research, lip‐reading has not focused on jaw movements. In addition, few studies have focused on Japanese syllables. Owing to the relationship between Japanese vowels and jaw movements, such movements can improve the accuracy of syllable estimations. To improve the lip‐reading method, we studied the syllable identification method in Japanese using the features of both chin and lip movements. In addition to lip movements, the chin movement features are an improved F‐measure for a syllable identification method in Japanese. © 2022 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.