In this paper, we study the problem of anomaly detection with application to aviation systems. We proposed a framework for detecting precursors to aviation safety incidents due to human factors based on Hidden Semi-Markov Models (HSMM). We investigate HSMMs due to their inherent ability to model durations in addition to model latent state transitions based on the observed pilots actions. Empirical evaluation on synthetic data and flight simulator data show that HSMMs perform favorably compared to many other existing anomaly detection algorithms.