The thermal evolution of nanoscale oxide inclusions in 316L stainless steel (SS) manufactured by laser powder bed fusion additive manufacturing (AM) was explored. The size, chemical composition, morphology, and distribution of the oxides were characterized as the function of heat treatment conditions. The study revealed the mechanistic driving force of the rapid oxide coarsening during recrystallization. Ostwald ripening governs oxide coarsening. The active grain boundary-oxide interaction at the early stage of recrystallization accelerated oxide coarsening via enhanced solute diffusion along grain boundaries. Pipe diffusion along dislocation cellular boundaries has a negligible contribution to oxide coarsening. At high temperatures (T > 1065 °C), although lattice diffusion primarily controlled the oxide growth, the contribution from the grain-boundary diffusion was necessary. The transformation from MnSiO3 to CrMn2O4 took place in the un-recrystallized grains but was not observed when recrystallization started. The interaction of grain boundary and oxides during recrystallization resulted in a high fraction of oxides accumulated at grain boundaries. While oxide coarsening does not significantly alter the toughness value, grain-boundary oxides promote microvoid formation and intergranular fracture under Charpy impact in the recrystallized AM 316L SS.