Novel poly(methyl methacrylate) (PMMA) nanocomposites with high dielectric constant and low loss were successfully fabricated by using sandwich-structured polyaniline-graphene-polyaniline (rPANI-GS-rPANI) hybrids sheets as fillers. The hybrid sheets were obtained by two steps: aniline was first in situ polymerized on both surfaces of graphene oxide (GO) sheets and then the PANI-GS-PANI sheets were reduced by hydrazine. The fully reduced hybrid sheets were denoted as rPANI-GS-rPANI, which exhibit an insulator-conductor-insulator sandwich structure. It was found that, the rPANI-GS-rPANI based PMMA nanocomposites not only have high dielectric constant but also show a low loss. For instance, the dielectric constant and dielectric loss of the nanocomposite are around 28 and 0.05 at 1 MHz, respectively, when the volume ratio of GS to PMMA is 6 vol%. This study not only provides a new way to prepare polymer nanocomposites with high permittivity and low dielectric loss, but also expands the application space of graphene based fillers.