We study the machine learning techniques applied to the lattice gauge theory's critical behavior, particularly to the confinement/deconfinement phase transition in the SU(2) and SU(3) gauge theories. We find that the neural network, trained on lattice configurations of gauge fields at an unphysical value of the lattice parameters as an input, builds up a gauge-invariant function, and finds correlations with the target observable that is valid in the physical region of the parameter space. In particular, we show that the algorithm may be trained to build up the Polyakov loop which serves an order parameter of the deconfining phase transition. The machine learning techniques can thus be used as a numerical analog of the analytical continuation from easily accessible but physically uninteresting regions of the coupling space to the interesting but potentially not accessible regions.