Along with the continuous development of science and technology, the multimedia retrieval is not only confined to the text retrieval. Content-based music retrieval technology has been widely used in modern society. The query by humming method based on the matching of user's humming fragments and music segments is a research hotspot in recent years. The establishment and the matching based on a large database is the research focuses of query by humming. This paper proposed a query by humming method based on the hierarchical matching algorithm. The hierarchical matching between humming fragments and melody features database by music phrases effectively improved the accuracy of query by humming based on a large database.