Using the data samples of 102 million $\Upsilon(1S)$ and 158 million $\Upsilon(2S)$ events collected by the Belle detector, we search for a pentaquark state in the $pJ/\psi$ final state from $\Upsilon(1,2S)$ inclusive decays. Here, the charge-conjugate $\bar{p}J/\psi$ is included. We observe clear $pJ/\psi$ production in $\Upsilon(1,2S)$ decays and measure the branching fractions to be $\mathcal{B}\Upsilon(1S) \to pJ/\psi + anything = 4.27 \pm 0.16(stat.) \pm 0.20(syst.) \times 10^{-5}$ and $\mathcal{B}\Upsilon(2S) \to pJ/\psi + anything = 3.59 \pm 0.14(stat.) \pm 0.16(syst.) \times 10^{-5}$. We also measure the cross section of inclusive $pJ/\psi$ production in $e^+e^-$ annihilation to be $\sigma(e^+e^- \to pJ/\psi + anything) = 57.5 \pm 2.1 (stat.) \pm 2.5(syst.)$~fb at $\sqrt{s} = 10.52~\hbox{GeV}$ using an 89.5~fb$^{-1}$ continuum data sample. There is no significant $P_c(4312)^+$, $P_c(4440)^+$ or $P_c(4457)^+$ signal found in the $pJ/\psi$ final states in $\Upsilon(1,2S)$ inclusive decays. We determine the upper limits of $\mathcal{B}\Upsilon(1,2S)\to P_c^{+} + anything \cdot \mathcal{B}(P_c^{+}\to pJ/\psi)$ to be at the $10^{-6}$ level.