There has been steady effort to modelize or recognize human action in fields of computer visions or mechanical learning, which should lead to fruitful results. This study presents how to extract key postures that can explain human actions within video sequence. To detect key postures that can differentiate human actions significantly, we select key posture candidates using information entropy which is a global feature, and then during key posture matching using shape context, we can select critical key postures. The method proposed shows efficiency in the experimental results and will contribute to development of research by inferring human action through connection of key postures with respect to human action.