A graphics processing unit (GPU)-accelerated vector-form particle-element method, i.e., the finite particle method (FPM), is proposed for 3D elastoplastic contact of structures involving strong nonlinearities and computationally expensive contact calculations. A hexahedral FPM element with reduced integration and anti-hourglass is developed to model structural elastoplastic behaviors. The 3D space containing contact surfaces is decomposed into cubic cells and the contact search is performed between adjacent cells to improve search efficiency. A connected list data structure is used for storing contact particles to facilitate the parallel contact search procedure. The contact constraints are enforced by explicitly applying normal and tangential contact forces to the contact particles. The proposed method is fully accelerated by GPU-based parallel computing. After verification, the performance of the proposed method is compared with the serial finite element code Abaqus/Explicit by testing two large-scale contact examples. The maximum speedup of the proposed method over Abaqus/Explicit is approximately 80 for the overall computation and 340 for contact calculations. Therefore, the proposed method is shown to be effective and efficient.