Studies examining experts’ superiority within domain-specific structured pattern recall tasks have typically had athletes perform them at rest, which is far different from how they are executed in their sport. The aim of this study was to investigate whether performing these tasks under different physical exercise intensities influenced pattern recall results of experts, advanced and novices. In two experiments, 68 participants (experiment 1: n = 33; experiment 2: n = 35) were tested using a handball-specific pattern recall task both at rest and during physical exercise. Physical exercises of 60 % heart rate reserve (constant workload: experiment 1) and of 86.5–90 % HRmax (handball-specific interval load: experiment 2) were induced. Results of both experiments revealed significant group differences with experts recalling patterns more accurately than novices but no significant within-subject differences for the two conditions and no interaction between both factors. Our findings replicate prior research concerning perceptual-cognitive expertise in structured specific pattern recall tasks. However, the lack of intergroup differences between the two conditions or interactions was surprising, suggesting sport-specific pattern recall skill is robust to changes in exercise stimuli. Future work is needed to further examine the impact of “physiological specificity” on perceptual-cognitive expertise.