A non-equilibrium phase of SmCo 9.8 that does not exist in the conventional polycrystalline Sm–Co alloy system has been obtained in an ultrafine nanocrystalline alloy prepared by a developed method. The conditions for the formation and stability of the SmCo 9.8 phase were studied. It was found that the single-phase SmCo 9.8 exists stably only in nanocrystalline alloy with the grain size smaller than 10nm. With increasing the temperature and hence the coarsening of the grain structure, the destabilization of SmCo 9.8 phase occurs, leading to a phase constitution of the Sm 2 Co 17 and Co phases. The effects of the nanograin structure and the phase constitution on the magnetic performance of the nominal SmCo 9.8 alloys were demonstrated. Based on the preparation technique and detailed characterizations, the present study provides a new understanding of the stability of the high-Co Sm–Co phase and its evolution behavior, which may facilitate the alloys design according to the correlation of phase stability and magnetic performance.