A series of the surface‐functionalized nano‐SiO2/polybenzoxazine (PBOZ) composites was produced, and an attempt was made to improve the toughness of PBOZ material, without sacrificing other mechanical and thermal properties. A benzoxazine functional silane coupling agent was synthesized to modify the surface of nano‐SiO2 particles, which were then mixed with benzoxazine monomers to produce the nano‐SiO2‐PBOZ nanocomposites. The notched impact strength and the bending strength of the nano‐SiO2‐PBOZ nanocomposites increase 40% and 50%, respectively, only with the addition of 3 wt % nano‐SiO2. At the same load of nano‐SiO2, the nano‐SiO2‐PBOZ nanocomposites exhibit the highest storage modulus and glass‐transition temperature by dynamic viscoelastic analysis. Moreover, the thermal stability of the SiO2/PBOZ nanocomposites was enhanced, as explored by the thermogravimetric analysis. The 5% weight loss temperatures increased with the nano‐SiO2 content and were from 368°C (of the neat PBOZ) to 379°C or 405°C (of the neat PBOZ) to 426°C in air or nitrogen with additional 3 wt % nano‐SiO2. The weight residue of the same nanocomposite was as high as 50% in nitrogen at 800°C. © 2010 Wiley Periodicals, Inc. J Appl Polym Sci, 2011