In order for robots to effectively interact with people in close proximity, the systems must first be able to detect, track, and follow people. This paper describes results from the development of a mobile robot which will follow a single, unmarked pedestrian using vision. This work demonstrates an improvement over existing pedestrian following applications because (1) it uses sufficiently strong classifiers such that it does not need to adapt to any particular pedestrian, (2) uses only vision and does not rely on any laser range devices, (3) provides a single point benchmark for the level of performance required from a detector to achieve pedestrian following, and (4) its performance is characterized over several kilometers in both rainy and dry weather conditions. The system leverages Histograms of Oriented Gradients (HOG) features for pedestrian detection at over 8 Hz using video from a monochrome camera. The pedestrian’s heading is combined with distance from stereo depth data to yield a 3D estimate. A particle filter with some clutter rejection provides a continuous track, and a waypoint follow behavior servos the iRobot PackBot robot chassis to a desired location behind the pedestrian. The final system is able to detect, track, and follow a pedestrian over several kilometers in outdoor environments, demonstrating a level of performance not previously shown on a small unmanned ground vehicle.