We investigate the impact of field-to-field variation, deriving from cosmic variance, in measured Lyman-$\alpha$ emitter (LAE) luminosity functions (LFs) and this variation's impact on inferences of the neutral fraction of the intergalactic medium (IGM) during reionization. We post-process a z=7 IGM simulation to populate the dark matter halos with LAEs. These LAEs have realistic UV magnitudes, Ly$\alpha$ fluxes, and Ly$\alpha$ line profiles. We calculate the attenuation of Ly$\alpha$ emission in universes with varying IGM neutral fraction, $\bar{\rm{x}}_{\rm{HI}}$. In a $\bar{\rm{x}}_{\rm{HI}}=0.3$ simulation, we perform 100 realizations of a mock 2 square degree survey with a redshift window $\Delta z = 0.5$ and flux limit $\rm{f}_{Ly\alpha}>1\times10^{-17}\:\rm{ergs}\:\: \rm{s}^{-1} \: \rm{cm}^{-2}$; such a survey is typical in depth and volume of the largest LAE surveys conducted today. For each realization, we compute the LAE LF and use it to recover the input $\bar{\rm{x}}_{\rm{HI}}$. Comparing the inferred values of $\bar{\rm{x}}_{\rm{HI}}$ across the ensemble of the surveys, we find that cosmic variance, deriving from large-scale structure and variation in the neutral gas along the sightline, imposes a floor in the uncertainty of $\Delta \bar{\rm{x}}_{\rm{HI}} \sim 0.2$ when $\bar{\rm{x}}_{\rm{HI}}$ $=0.3$. We explore mitigation strategies to decrease this uncertainty, such as increasing the volume, decreasing the flux limit, or probing the volume with many independent fields. Increasing the area and/or depth of the survey does not mitigate the uncertainty, but composing a survey with many independent fields is effective. This finding highlights the best strategy for LAE surveys aiming at constraining $\bar{\rm{x}}_{\rm{HI}}$ of the universe during reionization.