Higher aspect ratio (height/width) AgVO3 nanobelts were fabricated via a hydrothermal route. The reaction system was simple and the length of the nanobelts could be adjusted by modulating reaction parameters. Also, with or without surfactant hexadecyl trimethyl ammonium bromide (CTAB), straight or curled nanobelts were obtained, respectively. With α-Aminopropionic acid as both surface modifying agents and reducing agents, Ag nanoparticles with size of 5–10 nm were produced uniformly on the surface of AgVO3 nanobelts. The formation of Ag nanoparticles created surface-enhanced Raman scattering (SERS) hot spots on the AgVO3 nanobelts. Moreover, the synergistic effect between Ag hotspots and AgVO3 semiconductor made the as-synthesized AgVO3/Ag nanocomposites exhibit higher SERS sensitivity and better signal reproducibility. Using the composite SERS substrate, 4-Mercaptobenzoic acid (4-MBA) with a concentration down to 10−8 M was identified. Moreover, such kind of combination prevented the aggregation of Ag nanoparticles, so the substrate could be stored for a long time, which was another merit of the substrate from the viewpoint of its application.