Carbohydrates are important biomolecules and promising novel drug candidates, but their structural complexity and heterogeneity hinder their study and application. Low molecular weight heparins (LMWHs) are widely used as anticoagulants in the clinic and are considered the most successful carbohydrate-based drugs. They consist of both natural structures inherited from parent heparin and modified structures derived from depolymerization reactions. Sophisticated analytical methods are in great demand to elucidate the fine structure of LMWHs. This review summarizes recent progress in mass spectrometry (MS) that facilitates the in-depth structural characterization of LMWHs. The strategies can be sorted into top-down approaches and bottom-up approaches. In top-down approaches, the intact oligosaccharides are analyzed directly by hyphenated MS techniques to reveal their distribution and composition. Bottom-up approaches provide complementary structural information by analyzing partially enzymatically digested fragments or exhaustively degraded disaccharide building blocks of LMWHs. Advances in the MS/MS sequencing of short oligosaccharides and bioinformatics tools are also reviewed. Multidimensional analysis by MS offers a rigorous inspection of LMWHs, which is indispensable for ensuring drug quality and expanding their applications.