This study investigates wood density and anatomy of juvenile silver birch stems in Sweden, grown in mixed conifer stands. Our aim is to investigate if fertilization provides increased growth, as well as an eventual reduction in stem wood density. Measurements of basic density, ring width, cell wall thickness, and vessels are analyzed for 20 birch trees. Bark to pith radial sections are analyzed using a light microscope and the freeware ImageJ to compare treatments and ages. The results show that trees with fertilizer treatment have wider growth rings and thinner cell wall thickness compared to unfertilized trees. The fertilized trees also have a lower cambium age at the same height and the same diameter, and a slightly lower stem mean density (420 kg m−3) than the unfertilized stems (460 kg m−3). Fertilizer is a significant determinant of density and cell wall thickness in nonlinear models. The fertilized trees have increased growth and reached a fixed diameter earlier. The age difference between the trees likely explains some of the differences in cell wall thickness. This study supports the use of fertilizer as a silvicultural option for increasing the growth rate of silver birch for a relatively small reduction of wood density.