Inorganic–organic hybrid proton exchange membranes were prepared via sol–gel reaction of N-(2-aminoethyl)-3-aminopropyl-trimethoxysilane (AAS) in a sulfuric or phosphoric acid aqueous solution. The chemical structures of these membranes are characterized by means of Fourier transform infrared (FTIR) and 29 Si cross polarization nuclear magnetic resonance ( 29 Si CP NMR). Those acid-doped membranes were stable at temperatures up to about 300°C and showed varied conductivities at different temperature ranges. Optical Microscopy and Scan Electron Microscopy (SEM) analysis revealed that the introduction of H 2 SO 4 led to the generation of nanoparticles in situ. Atomic force microscopy (AFM) and energy dispersive X-ray spectroscopy (EDS) results indicated that these nanoparticles were wrapped by the soft organic side chains. Root-mean-square (RMS) roughness measured by AFM demonstrated that lower water addition during synthesis led to rougher surface and higher conductivity of H 2 SO 4 -doped membrane, while the surface of H 3 PO 4 -doped membrane remained smooth and clean, and the conductivity did not show a significant change by varying water additions. All those results demonstrated that the higher conductivity of the H 2 SO 4 -doped membrane achieved contributed not only to the dissociation of the counter anion but also the morphology of the membrane. Finally, we proposed a potential mechanism for the proton conduction in such acid-doped membranes. This mechanism could possibly provide a method to construct effective proton channels in the cross-linked anhydrous proton exchange membranes.