In this survey article we discuss the origin, theory and applications of left-symmetric algebras (LSAs in short) in geometry in physics. Recently Connes, Kreimer and Kontsevich have introduced LSAs in mathematical physics (QFT and renormalization theory), where the name pre-Lie algebras is used quite often. Already Cayley wrote about such algebras more than hundred years ago. Indeed, LSAs arise in many different areas of mathematics and physics. We attempt to give a survey of the fields where LSAs play an important role. Furthermore we study the algebraic theory of LSAs such as structure theory, radical theory, cohomology theory and the classification of simple LSAs. We also discuss applications to faithful Lie algebra representations.