We present results from the Keck Baryonic Structure Survey (KBSS), a unique spectroscopic survey of the distant universe designed to explore the details of the connection between galaxies and intergalactic baryons within the same survey volumes, focusing particularly on scales from ~50 kpc to a few Mpc. The KBSS is optimized for the redshift range z ~ 2-3, combining S/N ~100 Keck/HIRES spectra of 15 of the brightest QSOs in the sky at z Asymptotically = to 2.5-2.9 with very densely sampled galaxy redshift surveys within a few arcmin of each QSO sightline. In this paper, we present quantitative results on the distribution, column density, kinematics, and absorber line widths of neutral hydrogen (H I) surrounding a subset of 886 KBSS star-forming galaxies with 2.0 <, ~ z <, ~ 2.8 and with projected distances < or =, slant3 physical Mpc from a QSO sightline. Using Voigt profile decompositions of the full Ly alpha forest region of all 15 QSO spectra, we compiled a catalog of ~6000 individual absorbers in the redshift range of interest, with 12 < or =, slant log (N sub(H i)) < or =, slant21. These are used to measure H I absorption statistics near the redshifts of foreground galaxies as a function of projected galactocentric distance from the QSO sightline and for randomly chosen locations in the intergalactic medium (IGM) within the survey volume. We find that N sub(H I) and the multiplicity of velocity-associated H I components increase rapidly with decreasing galactocentric impact parameter and as the systemic redshift of the galaxy is approached. The strongest H I absorbers within Asymptotically = to 100 physical kpc of galaxies have N sub(H I) ~3 orders of magnitude higher than those near random locations in the IGM. The circumgalactic zone of most significantly enhanced H I absorption is found within transverse distances of <, ~ 300 kpc and within + or -300 km s super(-1) of galaxy systemic redshifts. Taking this region as the defining bounds of the circumgalactic medium (CGM), nearly half of absorbers with log(N sub(H I)) > 15.5 are found within the CGM of galaxies meeting our photometric selection criteria, while their CGM occupy only 1.5% of the cosmic volume. The spatial covering fraction, multiplicity of absorption components, and characteristic N sub(H I) remain significantly elevated to transverse distances of ~2 physical Mpc from galaxies in our sample. Absorbers with N sub(H I) >10 super(14.5) cm super(-2) are tightly correlated with the positions of galaxies, while absorbers with lower N sub(H I) are correlated with galaxy positions only on >, ~Mpc scales. Redshift anisotropies on these larger scales indicate coherent infall toward galaxy locations, while on scales of ~100 physical kpc peculiar velocities of Delta v Asymptotically = to + or -260 km s super(-1) with respect to the galaxies are indicated. The median Doppler widths of individual absorbers within 1-3 r sub(vir) of galaxies are larger by Asymptotically = to 50% than randomly chosen absorbers of the same N sub(H I), suggesting higher gas temperatures and/or increased turbulence likely caused by some combination of accretion shocks and galactic winds around galaxies with M sub(halo) Asymptotically = to 10 super(12) M sub(middot in circle) at z ~ 2-3.