Rubbery materials are widely used in industrial applications and are often exposed to cyclic stress and strain conditions while in service. To ensure safety and reliability, quantifying the effect of loads on the life of rubbery material is an important but challenging task, due to the combination of geometric/material nonlinearities and loading conditions for extended time durations. In this work, a novel simulation approach based on nonlinear space-time finite element method (FEM) is presented with a goal to capture fatigue failure in rubbery material subjected to cyclic loads. It is established by integrating the time discontinuous Galerkin (TDG) formulation with nonlinear material constitutive laws. A continuum damage mechanics (CDM) model is introduced to account for the damage evolution and model parameters for synthetic rubber are calibrated based on experiment. The nonlinear space-time FEM coupled with CDM constitutive model shows good agreement with the fracture and low cycle fatigue test of notched rubber sheet specimen.