A novel anomaly detection algorithm is proposed to detect anomalies in trajectory data of autonomous underwater vehicles (AUVs). Compared to existing work, the proposed algorithm estimates vehicle speed (i.e., through-water speed) from trajectory data and detect abnormal motion with estimated vehicle speed by a threshold technique. The influence of ocean flow on AUVs is significant and must disturb AUV's trajectory. This paper presents an adaptive learning algorithm based on the framework of controlled Lagrangian particle, which leads to simultaneously estimating controlled speed and flow velocity so that estimated flow velocity is incorporated to prevent false alarms.