Non-covalently functionalized carbon nanotubes are more attractive for multifunction composites because they preserve nearly all the nanotubes’ intrinsic properties and enhance the electroconductivity of polymer composites. However, It is seldom reported that they make dramatic improvement in mechanical properties. In this paper we have successfully prepared a poly(vinyl alcohol) (PVA) nanocomposite with a non-covalently functionalized carbon nanotube (DOC-MWNTs) using a simple method, which achieve a significant enhancement in mechanical properties. The tensile modulus and tensile yield strength of the PVA composite film containing 5wt% DOC-MWNTs increased by 140% and 65%, respectively, comparing to the pure PVA film. FT-IR, TEM, SEM, and DSC were used to investigate the MWNTs and PVA/MWNTs nanocomposites. The results show that the separately dispersed DOC-MWNTs filler throughout the PVA matrix and the strong adhesion between the DOC-MWNTs filler and the PVA matrix are responsible for the significant reinforcement of the mechanical properties of the composite prepared.