•Both treatments inhibited cancer proliferation in a time and dose dependent manner.•FxRF treatment were effective in inducing apoptosis in HepG2 cells than crude extract.•Treatments stimulated regulation in cell signalling, apoptotic and antioxidant genes. In this study, anti-proliferative effects of C. calcitrans extract and its fucoxanthin rich fraction (FxRF) were assessed on human liver HepG2 cancer cell line. Efficacy from each extract was determined by cytotoxicity assay, morphological observation, and cell cycle analysis. Mechanisms of action observed were evaluated using multiplex gene expression analysis. Results showed that CME and FxRF induced cytotoxicity to HepG2 cells in a dose and time-dependent manner. FxRF (IC50: 18.89 μg.mL−1) was found to be significantly more potent than CME (IC50: 87.5 μg.mL−1) (p < 0.05). Gene expression studies revealed that anti-proliferative effects in treated cells by C. calcitrans extracts were mediated partly through the modulation of numerous genes involved in cell signaling (AKT1, ERK1/2, JNK), apoptosis (BAX, BID, Bcl-2, APAF, CYCS) and oxidative stress (SOD1, SOD2, CAT). Overall, C. calcitrans extracts demonstrated effective intervention against HepG2 cancer cells where enhanced apoptotic activities were observed with increased fucoxanthin content.