For over a century, the origins and mechanisms underlying the diversification of the enormous temperate genus Carex (>2100 species; Cariceae, Cyperaceae) have remained largely speculative. Characteristics such as its diverse ecology, varied biogeography, and intriguing cytology have made Carex a powerful model for studying plant evolution, but its uncertain sister-group relationships hinder its use in studies that depend on accurate ancestral state estimates and biogeographic inferences. To identify the sister to Carex, we estimated the phylogeny of all genera in the Cariceae-Dulichieae-Scirpeae clade (CDS) using three plastid and two nuclear ribosomal markers. Ancestral state reconstructions of key characters were made, and a time-calibrated tree estimated. Carex is strongly supported as sister to the rare East Asian Sumatroscirpus, sole genus of a new tribe, Sumatroscirpeae trib. nov. Believed to be unique to Carex, the perigynium (prophyllar bract enclosing a flower) is in fact a synapomorphy shared with this small tribe (∼4 species) that appeared 36 Mya. We thus suggest the initial key innovation in the remarkable diversification of Carex is not the perigynium, but could be the release of mechanical constraints on perigynia through spikelet truncation, resulting in novel adaptive morphologies. Monoecy, chromosomal change, and rapid inflorescence development enabling phenological isolation may also be involved. The tiny tribe Sumatroscirpeae will provide unprecedented insights into the inflorescence homology, evolution, diversification, and biogeographic history of its sister-group Carex, one of the world’s most diverse plant lineages.