•Novel connections between neural network architectures and HJ PDE viscosity solutions.•Exact solutions of certain high dimensional HJ PDEs using neural networks.•Grid-free methods for certain Hamilton-Jacobi Partial Differential Equations. We propose novel connections between several neural network architectures and viscosity solutions of some Hamilton–Jacobi (HJ) partial differential equations (PDEs) whose Hamiltonian is convex and only depends on the spatial gradient of the solution. To be specific, we prove that under certain assumptions, the two neural network architectures we proposed represent viscosity solutions to two sets of HJ PDEs with zero error. We also implement our proposed neural network architectures using Tensorflow and provide several examples and illustrations. Note that these neural network representations can avoid curve of dimensionality for certain HJ PDEs, since they do not involve neither grids nor discretization. Our results suggest that efficient dedicated hardware implementation for neural networks can be leveraged to evaluate viscosity solutions of certain HJ PDEs.