Unfairness plays an important role in economic decision making. This fMRI study sought to investigate how the loss and the gain contexts could modulate behavioral and brain responses to unfairness by focusing on participants' rejection behaviors during an Ultimatum Game paradigm. Participants were scanned while they were playing the Ultimatum Game as responders in both loss and gain contexts, i.e. receiving ¥50 as gains and paying for ¥50 as losses. At the behavioral level, lower fairness ratings and higher rejection rates were revealed for unfair losses than unfair gains. At the neural level, left dorsolateral prefrontal cortex, bilateral anterior insula, anterior cingulate cortex/anterior middle cingulate cortex and bilateral dorsal striatum were associated with rejection (vs. acceptance) in the loss context, but not in the gain context. Together, our data indicated that participants may experience more unfairness in UG and stronger desire to sanction social norm violations in the loss context than in the gain context, inducing more fairness-related neutral activities when rejecting (vs. accepting) unfair losses than unfair gains. These findings shed light on the significance of context (i.e. loss or gain) in fairness-related social decision-making processes. •Unfair losses had lower fairness ratings and higher rejection rates than unfair gains.•AI, ACC/aMCC, DS and DLPFC were engaged during unfair relative to fair offers.•Unfair rejection vs. unfair acceptance activated AI, aMCC, DS and left DLPFC.•Rejection of unfair losses (but not gains) activated AI, DS, ACC/aMCC and left DLPFC.