We show how the theory of recursive matrices-bi-infinite matrices in which each row can be recursively computed from the previous one-can be used to formulate a version of the umbral calculus that is also suited for the study of polynomials p(x) taking integer values when the variable x is an integer. In this way, most results of the classical umbral calculus-such as expansion theorems and closed formulas-can be seen as immediate consequences of the two main properties of recursive matrices, namely, the Product Rule and the Double Recursion Theorem.