<?Pub Dtl?>This paper discusses validation methods for underwater acoustic communication channel simulators, and validates direct and stochastic replay of underwater acoustic communication channels as implemented in a channel simulator called Mime. Direct replay filters an input signal directly with a measured time-varying impulse response, whereas stochastic replay filters an input signal with a synthetic impulse response consistent with the scattering function of the measured channel. The validation uses data from two sea experiments and a diverse selection of communication schemes. Good agreement is found between bit error rates and packet error rates of in situ transmissions and simulated transmissions. Long-term error statistics of in situ signaling are also reproduced in simulation when a single channel measurement is used to configure the simulator. In all except one comparison, the packet error rate in simulation is within 20% of the packet error rate measured on location. The implication is that this type of channel simulator can be employed to test new modulation schemes in a realistic fashion without going to sea, except for the initial data collection.