Mobile edge computing (MEC) and device-to-device (D2D) offloading are two promising paradigms in the industrial Internet of Things (IIoT). In this article, we investigate task co-offloading, where computing-intensive industrial tasks can be offloaded to MEC servers via cellular links or nearby IIoT devices via D2D links. This co-offloading delivers small computation delay while avoiding network congestion. However, erratic movements, the selfish nature of devices and incomplete offloading information bring inherent challenges. Motivated by these, we propose a co-offloading framework, integrating migration cost and offloading willingness, in D2D-assisted MEC networks. Then, we investigate a learning-based task co-offloading algorithm, with the goal of minimal system cost (i.e., task delay and migration cost). The proposed algorithm enables IIoT devices to observe and learn the system cost from candidate edge nodes, thereby selecting the optimal edge node without requiring complete offloading information. Furthermore, we conduct simulations to verify the proposed co-offloading algorithm.