A one-step reduction method was used for the preparation of stable graphitic carbon nitride-gold nanoparticles (g-C3N4-Au) nanocomposites from ultrathin g-C3N4 nanosheets and chloroauric acid by using NaBH4 as a reducing agent under ultrasonication. The nanocomposites were characterized by transmission electron microscopy, X-ray photoelectron spectroscopy, UV-Vis absorption and fluorescence spectroscopy etc. The results revealed that the gold nanoparticles (AuNPs) are uniformly formed on the g-C3N4 nanosheets. It is found that the peroxidase-like catalytic activity of this nanocomposite for the oxidation of 3,3′,5,5′-tetramethylbenzidine by H2O2 to form a blue-colored product is strongly enhanced in the presence of Hg(II). Based on this phenomenon, a sensitive “turn-on” colorimetric assay for Hg(II) was developed that works at physiological pH values. Under optimal conditions, the absorption signal at 652 nm increases linearly with Hg(II) concentration in the range from 5 to 500 nM. A detection limit as low as 3.0 nM was achieved. This assay has excellent selectivity over other metal ions. It was successfully applied to the determination of Hg(II) in real water samples. The method is cost-effective, rapid, and allows for visual detection.