Abstract Objectives Genetic risks for cognitive decline are not modifiable; however their relative importance compared to modifiable factors is unclear. We used machine learning to evaluate modifiable and genetic risk factors for Alzheimer’s disease (AD), to predict cognitive decline. Methods Health and Retirement Study participants, aged 65–90 years, with DNA and >2 cognitive evaluations, were included (n = 7,142). Predictors included age, body mass index, gender, education, APOE ε4, cardiovascular, hypertension, diabetes, stroke, neighborhood socioeconomic status (NSES), and AD risk genes. Latent class trajectory analyses of cognitive scores determined the form and number of classes. Random Forests (RF) classification investigated predictors of cognitive trajectories. Performance metrics (accuracy, sensitivity, and specificity) were reported. Results Three classes were identified. Discriminating highest from lowest classes produced the best RF performance: accuracy = 78% (1.0%), sensitivity = 75% (1.0%), and specificity = 81% (1.0%). Top ranked predictors were education, age, gender, stroke, NSES, and diabetes, APOE ε4 carrier status, and body mass index (BMI). When discriminating high from medium classes, top predictors were education, age, gender, stroke, diabetes, NSES, and BMI. When discriminating medium from the low classes, education, NSES, age, diabetes, and stroke were top predictors. Discussion The combination of latent trajectories and RF classification techniques suggested that nongenetic factors contribute more to cognitive decline than genetic factors. Education was the most relevant predictor for discrimination.