The waves generated by faulting represent the primary threat posed by most large earthquakes. The effect of complex geological structures, such as sedimentary basins, on earthquake ground motion is a source of particular concern. We show that it is possible to extract reliable phase and amplitude response that includes the effects of complex structure for the elastodynamic Green's function from the ambient seismic field using deconvolution. We demonstrate the accuracy of the approach by predicting complex ground motion for a moderate (Mw = 4.6) earthquake in southern California as recorded in the Los Angeles Basin. This suggests a novel approach to seismic hazard analysis in which ground motion from hypothetical future earthquakes is simulated directly, without the need for modeling the detailed heterogeneity of the Earth's crust as an intermediate step.