For primordial perturbations, deviations from Gaussian statistics on the tail of the probability distribution can be associated with non-perturbative effects of inflation. In this paper, we present some particular examples in which the tail of the distribution becomes highly non-Gaussian although the statistics remains almost Gaussian in the perturbative regime. We begin with an extension of the ultra-slow-roll inflation that incorporates a transition process, where the inflaton climbs up a tiny potential step at the end of the non-attractor stage before it converges to the slow-roll attractor. Through this example, we identify the key role of the off-attractor behaviour for the upward-step transition, and then extend the analysis to another type of the transition with two slow-roll stages connected by a tiny step. We perform both the perturbative and non-perturbative analyses of primordial fluctuations generated around the step in detail, and show that the tiny but nontrivial transition may affect large perturbations in the tail of the distribution, while the perturbative non-Gaussianity remains small. Our result indicates that the non-Gaussian tails can have rich phenomenology which has been overlooked in conventional analyses. We also study the implications of this non-Gaussian tail for the formation of primordial black holes, and find that their mass fraction can be parametrically amplified by several orders of magnitudes in comparison with the case of the Gaussian distribution. Additionally, we also discuss a mechanism of primordial black holes formation for this upward step inflation model by trapping the inflaton in the bottom of the step.