Thirty-seven populations of Norway spruce [Picea abies (L.) Karst.] across the Austrian Alps and Bohemian Massif were sampled to elucidate the geographical pattern of genetic differentiation. Three polymorphic mitochondrial DNA (mtDNA) loci were surveyed. Two or three alleles were detected at each locus, resulting in seven multilocus mtDNA haplotypes (A–F). Western populations proved to be monomorphic, whereas eastern and central Austrian populations were slightly to highly polymorphic, respectively. As revealed by spatial analysis of molecular variance and Monmonier’s analysis, the two main haplotypes A and B are not randomly distributed. Haplotype A was restricted to central and eastern Austria, whereas haplotype B occurred in all Austrian populations but was the only haplotype identified in western populations. This pattern may be explained by different glacial refugia located in the Dinaric Alps and the Carpathian mountains.