This article presents a statistical analysis of the Matrix Pencil method for estimating the mode and the amplitude of a single damped complex exponential. This study is based on a perturbation analysis of the mode and the amplitude, assuming a high signal-to-noise ratio. Closed-form expressions of the mean and variance of these perturbations are derived. It is shown that the estimates are unbiased and that the estimator can be tuned in order to obtain a minimal variance. The theoretical results are verified using Monte Carlo simulations.