Currently available uncertainty quantification (UQ) neural networks (NNs) are trained through the statistical error minimization. Therefore, NNs perform poorly for critical input patterns. Some input patterns have lower coverage probabilities than others. Such input dependent performance is evident in electricity price prediction where different input features are coming from heterogeneous monitoring sources. This paper proposes a prediction interval (PI) based UQ of the electricity price with the proposed partial adversarial training to achieve the input-domain independent performance. The proposed training consists of initial training, adversarial sample generations from critical samples and a final training with the combined datasets of critical samples and initial training samples. Critical situations are situations where prediction systems struggle to predict and make higher statistical errors. Multiple NNs are first trained with different initializations. Each uncovered sample to the NN pair in the training set generates an adversarial sample. The adversarial dataset is concatenated with the initial samples. The final NN training is performed with the combined dataset. The technique is visualized with rough sketches in both time and input domain. The feasibility and performance are examined on experimental electricity market price datasets.