Alzheimer’s disease (AD) is characterized by pathological lesions such as amyloid-β (Aβ) plaques and cerebral amyloid angiopathy. Both these lesions consist mainly of aggregated Aβ protein and this aggregation is affected by macromolecules such as heparan sulfate (HS) proteoglycans. Previous studies demonstrated that HS enhances fibrillogenesis of Aβ and that this enhancement is dependent on the degree of sulfation of HS. In addition, it has been reported that these sulfation epitopes do not occur randomly but have a defined tissue distribution. Until now, the distribution of sulfation epitopes of HS has not yet been studied in human brain. We investigated whether a specific HS epitope is associated with Aβ plaques by performing immunohistochemistry on occipital neocortical and hippocampal tissue sections from AD patients using five HS epitope-specific phage display antibodies. Antibodies recognizing highly N-sulfated HS demonstrated the highest level of staining in both fibrillar Aβ plaques and non-fibrillar Aβ plaques, whereas antibodies recognizing HS regions with a lower degree of N-sulfate modifications were only immunoreactive with fibrillar Aβ plaques. Thus, our results suggest that a larger variety of HS epitopes is associated with fibrillar Aβ plaques, but the HS epitopes associated with non-fibrillar Aβ plaques seem to be more restricted, selectively consisting of highly N-sulfated epitopes.