Integrating conventional optics into compact nanostructured surfaces is the goal of flat optics. Despite the enormous progress in this technology, there are still critical challenges for real-world applications due to the limited operational efficiency in the visible region, on average lower than 60%, which originates from absorption losses in wavelength-thick (≈ 500 nm) structures. Another issue is the realization of on-demand optical components for controlling vectorial light at visible frequencies simultaneously in both reflection and transmission and with a predetermined wavefront shape. In this work, we developed an inverse design approach that allows the realization of highly efficient (up to 99%) ultrathin (down to 50 nm thick) optics for vectorial light control with broadband input-output responses in the visible and near-IR regions with a desired wavefront shape. The approach leverages suitably engineered semiconductor nanostructures, which behave as a neural network that can approximate a user-defined input-output function. Near-unity performance results from the ultrathin nature of these surfaces, which reduces absorption losses to near-negligible values. Experimentally, we discuss polarizing beam splitters, comparing their performance with the best results obtained from both direct and inverse design techniques, and new flat-optics components represented by dichroic mirrors and the basic unit of a flat-optics display that creates full colours by using only two subpixels, overcoming the limitations of conventional LCD/OLED technologies that require three subpixels for each composite colour. Our devices can be manufactured with a complementary metal-oxide-semiconductor (CMOS)-compatible process, making them scalable for mass production at low cost.