A series of Al-5 wt pct Si alloys with Yb additions (up to 6100 ppm) have been investigated using thermal analysis and multiscale microstructure characterization techniques. The addition of Yb was found to cause no modification effect to a fibrous morphology involving Si twinning; however, a refined plate-like eutectic structure was observed. The Al 2 Si 2 Yb phase was observed with Yb addition level of more than 1000 ppm. Within the eutectic Al and Si phases, the Al 2 Si 2 Yb phase was also found as a precipitation from the remained liquid. No Yb was detected in the α-Al matrix or plate-like Si particle, even with Yb addition up to 6100 ppm. The absence of Yb inside the eutectic Si particle may partly explain why no significant Si twinning was observed along {111} Si planes in the eutectic Si particle. In addition, the formation of the thermodynamic stable YbP phases is also proposed to deteriorate the potency of AlP phase in Al alloys. This investigation highlights to distinguish the modification associated with the ever present P in Al alloys. We define modification as a transition from faceted to fibrous morphology, while a reduction of the Si size is termed refinement.