There is considerable dispute about the nature of infant memory. Using SEM models, we examined whether popular characterizations of the structure of adult memory, including the two‐process theory of recognition, are applicable in the infant and toddler years. The participants were a cohort of preterms and full‐terms assessed longitudinally – at 1, 2, and 3 years – on a battery containing tasks of immediate and delayed recognition, recall, and memory span (a measure of short‐term capacity). Results were in accord with adult models which assume that short‐ and long‐term memory are distinct, and that two processes – familiarity and recollection – underlie recognition memory, while one alone – recollection – supports recall. The finding that prematurity, which entails risk of hippocampal compromise, affected recollection, but not familiarity, accords well with adult findings that hippocampal damage selectively affects recollection. These findings reveal striking similarity between the structure and theoretical underpinnings of infant and adult memory.