Given the critical importance of hand-eye coordination among psycho-motor skills, it is desirable to diagnose related developmental delays and disorders sooner rather than later. We argue that the assessment of hand-eye coordination skills and the monitoring of their progress can be performed by using a predefined set of video games as an assessment tool. Cameras can gather information on the player's motor performance with respect to visual stimuli presented by the game. The proposed computer vision algorithm detects the arm motions triggered by the game stimuli and computes the reaction time to every stimulus. We argue that the simplicity of the proposed algorithm is one of its strengths, as the algorithm can be embedded with minimal development effort in any game console or smart display. Moreover, the proposed approach is intended to become a baseline algorithm for monitoring the motor coordination of arm movements during game play.