Abstract We investigate the impact of local environment on the galaxy stellar mass function (SMF) spanning a wide range of galaxy densities from the field up to dense cores of massive galaxy clusters. Data are drawn from a sample of eight fields from the Observations of Redshift Evolution in Large-Scale Environments (ORELSE) survey. Deep photometry allow us to select mass-complete samples of galaxies down to 109 M⊙. Taking advantage of >4000 secure spectroscopic redshifts from ORELSE and precise photometric redshifts, we construct three-dimensional density maps between 0.55 < z < 1.3 using a Voronoi tessellation approach. We find that the shape of the SMF depends strongly on local environment exhibited by a smooth, continual increase in the relative numbers of high- to low-mass galaxies towards denser environments. A straightforward implication is that local environment proportionally increases the efficiency of (a) destroying lower mass galaxies and/or (b) growth of higher mass galaxies. We also find a presence of this environmental dependence in the SMFs of star-forming and quiescent galaxies, although not quite as strongly for the quiescent subsample. To characterize the connection between the SMF of field galaxies and that of denser environments, we devise a simple semi-empirical model. The model begins with a sample of ≈106 galaxies at z start = 5 with stellar masses distributed according to the field. Simulated galaxies then evolve down to z final = 0.8 following empirical prescriptions for star-formation, quenching and galaxy–galaxy merging. We run the simulation multiple times, testing a variety of scenarios with differing overall amounts of merging. Our model suggests that a large number of mergers are required to reproduce the SMF in dense environments. Additionally, a large majority of these mergers would have to occur in intermediate density environments (e.g. galaxy groups).