All-solid-state lithium ion batteries (ASSLBs) are considered next-generation devices for energy storage due to their advantages in safety and potentially high energy density. As the key component in ASSLBs, solid-state electrolytes (SSEs) with non-flammability and good adaptability to lithium metal anodes have attracted extensive attention in recent years. Among the current SSEs, composite solid-state electrolytes (CSSEs) with multiple phases have greater flexibility to customize and combine the advantages of single-phase electrolytes, which have been widely investigated recently and regarded as promising candidates for commercial ASSLBs. Based on existing investigations, herein, we present a comprehensive overview of the recent developments in CSSEs. Initially, we introduce the historical development from solid-state ionic conductors to CSSEs, and then summarize the fundamentals including mechanisms of lithium ion transport, key evaluation parameters, design principles, and key materials. Four main types of advanced structures for CSSEs are classified and highlighted according to the recent progress. Moreover, advanced characterization and computational simulation techniques including machine learning are reviewed for the first time, and the main challenges and perspectives of CSSEs are also provided for their future development. All-solid-state lithium ion batteries (ASSLBs) are considered next-generation devices for energy storage due to their advantages in safety and potentially high energy density.