Nanocomposite membranes based on the poly(vinyl alcohol) and hydrophilic polymer-grafted fumed silica nanoparticles were prepared via aqueous solution casting and cross-linking by glutardialdehyde of poly(vinyl alcohol) matrix containing various amounts (0–10 wt %) of the fumed silica nanoparticles (FSN). Modified nanoparticles were synthesized via surface-initiated redox polymerization of styrene sulfonic acid and 2-acrylamido-2-methyl-1-propane sulfonic acid from surface of the aminopropyl-functionalied FSNs. Nanocomposite membranes were then characterized by various techniques. Results showed that proton conductivity increases significantly by increasing amount of the modified FSNs in the nanocomposite membranes. The highest proton conductivity (17.89 mS/cm) was observed for poly(vinyl alcohol) membrane prepared in the presence of 10 wt % PAMPS-g-FSN nanoparticles. It was found that the presence of sulfonated polymer brushes onto the fumed silica nanoparticles enhances various properties such as proton conductivity and mechanical strength of the polymer electrolyte membranes.