We present measurements of the magnetization and specific heat of a CePtSn single crystal in the temperature range 1.7–15 K and in magnetic fields up to 14 T applied along the orthorhombic b axis. Possible microscopic mechanisms responsible for the observed magnetic phase transitions are discussed in the context of previously reported magnetoresistance and neutron-scattering data. Special emphasis is put on an irreversible field-induced transition around 4 T observed for temperatures T≤2 K. At this transition, found when sweeping the field up from the zero-field-cooled (ZFC) state, a tiny change in the magnetization is accompanied by a large negative GMR effect. Surprisingly, no transition is observed when sweeping the field back to zero, i.e. the ‘field-annealed’ phase (after the application of a sufficiently high field) persists even in zero field and a simple AF structure with a propagation vector q=(0,1/2,0) is proposed to characterize this phase. Clear evidence for this transition is observed also in temperature dependencies of the magnetization and specific heat in various magnetic fields. Based on these results, the completed magnetic-phase diagram of CePtSn in fields parallel to the b axis is presented.