The D-shaped coil is employed in the design of the hybrid toroidal magnet for 10 MJ high-temperature superconducting magnetic energy storage (HTS-SMES) to mitigate the huge bending moment on the coil. During the power compensation process of the SMES system, the time-varying currents induce ac losses within the magnet, leading to heating and temperature rise. Excessive temperature rise can result in the magnet experiencing a quench or even damage. Therefore, accurately calculating the ac losses generated by the magnet for HTS-SMES is crucial for the stable operation of the system. Generally, 2-D models cannot accurately simulate the complex geometric shapes of superconducting coils with nonaxisymmetric properties. This article applies the <inline-formula><tex-math notation="LaTeX">\bm {H}</tex-math></inline-formula>-formulation and <inline-formula><tex-math notation="LaTeX">T</tex-math></inline-formula>-<inline-formula><tex-math notation="LaTeX">A</tex-math></inline-formula> formulation and their homogeneous method to the 3-D finite element method (FEM) models of a D-shaped double-pancake coil (DPC). The effectiveness of the established 3-D FEM models has been verified through ac loss measurement experiments on a D-shaped HTS DPC at 77 K temperature. The application strategies of <inline-formula><tex-math notation="LaTeX">T</tex-math></inline-formula>-<inline-formula><tex-math notation="LaTeX">A</tex-math></inline-formula> full, <inline-formula><tex-math notation="LaTeX">T</tex-math></inline-formula>-<inline-formula><tex-math notation="LaTeX">A</tex-math></inline-formula> homogenization, and <inline-formula><tex-math notation="LaTeX">H</tex-math></inline-formula> homogenization in the 3-D FEM models have been comprehensively compared under different sizes of D-shaped HTS DPCs. Subsequently, the distribution characteristics of ac losses in a D-shaped DPC have been analyzed. The research findings reveal that the distribution of ac losses exhibits nonuniformity along the arc direction of the coil. The difference between the maximum and minimum values in this coil is approximately 15.1% in the <inline-formula><tex-math notation="LaTeX">\bm {H}</tex-math></inline-formula> homogeneous model and 14.0% in the <inline-formula><tex-math notation="LaTeX">T</tex-math></inline-formula>-<inline-formula><tex-math notation="LaTeX">A</tex-math></inline-formula> homogeneous model. This work holds significant guidance for the computational assessment of ac losses in the hybrid toroidal magnet for 10 MJ HTS-SMES.