Determining lifelong physical activity (PA) trajectories and their determinants is essential to promote a physically active lifestyle throughout the life‐course. We aimed to identify PA trajectories from childhood to midlife and their determinants in a longitudinal population‐based cohort. This study is a part of the Cardiovascular Risk in Young Finns Study. From 1980, a population‐based cohort (N = 3596; 1764 boys/1832 girls, age 3‐18 years) has been followed up for 31 years. PA indices were formed based on self‐reported data (between age 9‐49 years) on frequency, duration, and intensity of leisure (during childhood) or high‐intensity (at later age) PA and on sports club participation/competitions. PA trajectories were analyzed using group‐based trajectory modeling. Childhood (age 12 years), young adulthood (age 24 years), and early midlife (age 37 years) determinants were analyzed. Five PA trajectories were identified: persistently active (6.6%), decreasingly active (13.9%), increasingly active (13.5%), persistently low active (51.4%, reference group), persistently inactive (14.6%). In childhood, rural residential area (OR 0.45, 95% CI 0.21‐0.96) and high academic performance (OR 2.18; 95% CI 1.58‐3.00) associated with persistently active group. In early midlife, smoking (OR 1.66; 95% CI 1.07‐2.58) associated with persistently inactive group, regular alcohol drinking (OR 2.91; 95% CI 1.12‐7.55) with persistently active group and having children (OR 2.07; 95% CI 1.27‐3.38) with decreasingly active group. High adulthood education associated with both decreasingly (OR 1.87; 95% CI 1.05‐3.35) and increasingly (OR 2.09; 95% CI 1.19‐3.68) active groups. We identified five PA trajectories from childhood into midlife. Most prominent determinants were academic achievement, education, having children and health habits (i.e. smoking/alcohol use).