This paper presents a stereovision system to aid in navigating an Autonomous Surface Vehicle (ASV). The ASV is guided to follow a path mainly by GPS, while the stereovision system is designed for assisting it to avoid obstacles or follow a target floating on water surface. After calibration, the stereovision system can estimate the positions of floating objects founded simultaneously by both cameras, which can be used to plan the path such that the ASV can reach the target fast and avoid collision with other objects around. In fine sea state, when an obstacle is large enough and its color is different from the water surface obviously, the stereovision system can navigate the ASV to avoid it successfully. If the obstacle is small or difficult for the ASV to distinguish from the background, the operator in front of the console of the base station on a mother ship or on the shore can intervene in and guide the ASV remotely according to the couples of images from the two cameras.