We report on observations of Lyman break galaxies (LBGs) selected from the Great Observatories Origins Deep Survey at mean redshifts z ~ 4, 5, and 6 (B 435-, V 606-, and i 775-band dropouts, respectively), obtained with the red-sensitive FORS2 spectrograph at the ESO VLT. This program has yielded spectroscopic identifications for 114 galaxies (~60% of the targeted sample), of which 51 are at z ~ 4, 31 at z ~ 5, and 32 at z ~ 6. We demonstrate that the adopted selection criteria are effective, identifying galaxies at the expected redshift with minimal foreground contamination. Of the 10% interlopers, 83% turn out to be Galactic stars. Once selection effects are properly accounted for, the rest-frame ultraviolet (UV) spectra of the higher redshift LBGs appear to be similar to their counterparts at z ~ 3. As at z ~ 3, LBGs at z ~ 4 and z ~ 5 are observed with Lya both in emission and in absorption; when in absorption, strong interstellar lines are also observed in the spectra. The stacked spectra of Lya absorbers and emitters also show that the former have redder UV spectra and stronger but narrower interstellar lines, a fact also observed at z ~ 2 and 3. At z ~ 6, sensitivity issues bias our sample toward galaxies with Lya in emission; nevertheless, these spectra appear to be similar to their lower redshift counterparts. As in other studies at similar redshifts, we find clear evidence that brighter LBGs tend to have weaker Lya emission lines. At fixed rest-frame UV luminosity, the equivalent width of the Lya emission line is larger at higher redshifts. At all redshifts where the measurements can be reliably made, the redshift of the Lya emission line turns out to be larger than that of the interstellar absorption lines (ISLs), with a median velocity difference V ~ 400 km s-1 at z ~ 4 and 5, consistent with results at lower redshifts. This shows that powerful, large-scale winds are common at high redshift. In general, there is no strong correlation between the morphology of the UV light and the spectroscopic properties. However, galaxies with deep ISLs and strong Lya absorption appear to be more diffuse than galaxies with Lya in emission.