In this study, we consider Lie algebras that admit para-Kähler and hyper-para-Kähler structures. We provide new characterizations of these Lie algebras and develop many methods for building large classes of examples. Previously, Bai considered para-Kähler Lie algebras as left symmetric bialgebras. We reconsider this viewpoint and make improvements in order to obtain some new results. The study of para-Kähler and hyper-para-Kähler is intimately linked to the study of left symmetric algebras, particularly those that admit invariant symplectic forms. In this study, we provide many new classes of left symmetric algebras and complete descriptions of all the associative algebras that admit an invariant symplectic form. We also describe all four-dimensional hyper-para-Kähler Lie algebras.