A technique named Impacting Trailed Welding (ITW) was proposed, aimed at refining the grain size of the HAZ in multi-pass welding. The key idea of ITW is to obtain a large deformation in the HAZ during one weld pass, and get it recrystallized during the next weld pass. Theoretical analysis suggests that the deformed HAZ can get completely recrystallized and the degree of the successive grain growth is lower than the normal grain growth, so that the grains can be dramatically refined. The average grain size was reduced by a factor 2 through the application of the ITW technique, and remained close to the grain size of the base material. The results are consistent with the theoretical analysis.