Earthquakes that have occurred in the last twenty years in the Mediterranean area have had significant economic and social impacts. Most of the economic losses of reinforced concrete (RC) frames was due to nonstructural component damage, particularly masonry infills and partitions. Therefore, the seismic behaviour of masonry infills should be reliably characterized. The main goals of this study for a more reliable loss estimation for infilled RC frames are: (i) the analysis of the inter-story drift ratio (IDR) capacity at given damage states (DSs) with the aim to define drift-based fragility functions and (ii) analyse direct losses due to infill damage following seismic events. First, a database of experimental tests performed on 1-bay, 1-story scaled RC frames infilled with clay bricks or concrete blocks is collected. Drift-based fragility curves are obtained, which depend on the infill brick materials and properties. Then, the drift capacity threshold at each DS is correlated to the in-plane response of the infill panel to directly quantify the relationship that exists among them. The influence of openings on drift capacities is also evaluated. Then, seismic losses related to infills are computed, providing expected monetary losses depending on the infill typology. The required reparation activities and their costs are also listed. The bearing of each activity and cost at each DS is explicitly evaluated. Additionally, loss functions that directly depend on IDR demand are provided, thus fusing together the damage analysis and loss analysis. Finally, a simplified formulation for loss functions is proposed for a simple, practice-oriented loss calculation.