The oxygen impurity is a common defect in tantalum nitrides. Experiments reveal that the concentration of oxygen impurity in Ta5N6 is as low as 0.2%, which is much lower than that in other tantalum nitrides such as Ta3N5 and Ta2N3. In this study, the density functional theory calculations were performed to unravel the cause of low oxygen impurity concentrations in Ta5N6. By calculating the elastic moduli, cohesive energies, defect formation energies and surface energies, we found that doping with high concentration oxygen impurities into Ta5N6 was thermodynamically and mechanically unfavorable. Electronic structure calculations revealed that the electron donated from oxygen impurity was localized, providing one proper reason for the difficulty of doping oxygen impurity into Ta5N6.