Remote-control technology is required in an unmanned vehicle such that it can replace humans for executing tasks in various extreme environments. In particular, a remote scene must be reconstructed using 3D meshes for enabling a user to remotely control an unmanned vehicle easily and intuitively. To this end, a large amount of multiple sensor data would be processed using various algorithms in real time. Considering the limited hardware specifications in extreme environments, it is difficult to implement 3D terrain reconstruction in high-quality and real time. This paper proposes the optimization of the architecture of a multiple-sensor-data pipeline. The improved performance resulting from the optimized architecture was analyzed through experimental comparison with a non-optimized system.