Equilibrium molecular dynamics (EMD) simulations through Green–Kubo formula (GKF) have been widely used in the study of thermal conductivity of various materials. However, there exist controversial simulation results which have huge discrepancies with experimental ones in literatures. In this Letter, we demonstrate that the fluctuation in calculated thermal conductivity is due to the uncertainty in determination of the truncation time, which is related to the ensemble and size dependent phonon relaxation time. We thus propose a new scheme in the direct integration of heat current autocorrelation function (HCACF) and a nonzero correction in the double-exponential-fitting of HCACF to describe correctly the contribution to thermal conductivity from low frequency phonons. By using crystalline Silicon (Si) and Germanium (Ge) as examples, we demonstrate that our method can give rise to the values of thermal conductivity in an excellent agreement with experimental ones.