An ac photopyroelectric calorimeter has been designed and mounted to work at low temperatures (12 K to 77 K), where the sample is in a He atmosphere and with the detector region isolated from mechanical vibrations allowing the retrieval of high resolution measurements of thermal diffusivity and specific heat. The system has been used to study magnetic and ferroelectric phase transitions in RMnO3 (R = Sm, Tb, Dy). The high quality of the measurements has allowed study of the critical behavior of the transitions and extraction of their critical parameters. In the particular case of the antiferromagnetic-to-paramagnetic transition in SmMnO3, it belongs to the 3D-XY universality class while in the case of the lock-in, ferroelectric transition in TbMnO3, the results confirm that the transition is clearly second order.