Aragonite CaCO3 whisker (CW) could improve the mechanical properties of cementitious composites. The microstructure of calcite is different from that of aragonite, but only a few studies reported different effects in cementitious composites between incorporating aragonite and calcite. In order to fully use CW as high-performance, low-cost microfiber and clearly understand different mechanisms between aragonite and calcite CW, in this study, the heat treatment method was used to convert aragonite CW to calcite CW, and the effect of calcite and aragonite on the hydration and strength of CW reinforced cement paste (CWRC) was studied to bridge the gaps in this area. The aragonite and calcite CWs exhibited the overall physical dilution effect in Portland cement, significantly decreasing the hydration heat of cement and the total amount of non-evaporable water and Ca(OH)2. Calcite CW presents a more significant nucleation effect than aragonite CW. Compared to aragonite CWRC, the hydration of silicon phase in calcite CWRC is faster and the hydration heat release is higher; however, there is no significant difference in the hydration of aluminum phase. The Ca(OH)2 content in calcite CWRC was more than that in aragonite CWRC. More hydration products formed on the surface of calcite CW than that on aragonite CW, improving the bonding strength between calcite CW and cement matrix. Hence, both the compressive strength and flexural strength of calcite CWRC are higher than those of aragonite CWRC. The rheological test of fresh aragonite and calcite CWRC proved that both aragonite and calcite CWs significantly decreased the flow of cement paste, presenting viscosity effect. Compared to aragonite CW, calcite CW improved the flowability of fresh cement paste, indicating the lubricant effect of calcite CW. Aragonite and calcite CW present chemical effect similar to limestone powders, i.e., aluminum phase in Portland cement interacted with CO32- dissolved from CW to form mono- and semi- carbonate. The Ca(OH)2 orientation indexes of cement decreased by the introduction of CWs. Aragonite and calcite CWs can also be used as microfiber and filler in cementitious composites. These effects improved the micro-structure, thus enhanced the flexural and compressive strength of cement paste.