The development of mouse embryos can be partially recapitulated by combining embryonic stem cells (ESCs), trophoblast stem cells (TS), and extra-embryonic endoderm (XEN) stem cells to generate embryo-like structures called ETX embryos. Although ETX embryos transcriptionally capture the mouse gastrula, their ability to recapitulate complex morphogenic events such as gastrulation is limited, possibly due to the limited potential of XEN cells. To address this, we generated ESCs transiently expressing transcription factor Gata4, which drives the extra-embryonic endoderm fate, and combined them with ESCs and TS cells to generate induced ETX embryos (iETX embryos). We show that iETX embryos establish a robust anterior signaling center that migrates unilaterally to break embryo symmetry. Furthermore, iETX embryos gastrulate generating embryonic and extra-embryonic mesoderm and definitive endoderm. Our findings reveal that replacement of XEN cells with ESCs transiently expressing Gata4 endows iETX embryos with greater developmental potential, thus enabling the study of the establishment of anterior-posterior patterning and gastrulation in an in vitro system. Display omitted •Stem cells generate mouse-embryo-like structures with improved potential•These structures undertake anterior visceral endoderm formation and gastrulation•Single-cell sequencing shows improved resemblance to mouse embryo Amadei et al. have generated stem-cell-based structures that resemble mouse post-implantation embryos and have the potential to form the anterior-posterior axis and undergo gastrulation in vitro. Single-cell sequencing shows gene-expression patterns similar to those of the natural embryo at a comparable stage of development.