Communication efficiency and flocking stability are two basic requirements for the application of autonomous underwater vehicles (AUVs) in maritime transportation systems. Although they are closely related, most existing flocking approaches focus on the control techniques and ignore the influence of communication efficiency. This paper presents a joint design solution to the channel estimation and flocking control for multi-AUV-based maritime transportation systems, with the consideration of path loss, shadow and multipath fading channels. A value iteration-based reinforcement learning (RL) estimator is first designed to predict the channel quality of AUVs in positions that have not yet visited. With the predicted channel quality, we construct an integrated optimization problem for the co-design of communication and flocking strategies. Along with this, a value iteration-based RL flocking controller is developed to achieve the co-design of channel estimation and flocking control for AUVs. It is worth mentioning that, the value iteration-based RL estimator in this paper can avoid local optimal in traditional least-squares methods, and meanwhile the flocking controller in this paper can make a balance between flocking stability and communication efficiency for AUVs. Finally, simulation and experimental results reveal that the proposed approach in this paper has superior performances by comparing with the other works. As such, our approach is more useful for marine engineer to understand and explore the maritime transportation system from the communication and control view points.