2,4-dichlorophenoxyacetic acid (2,4-D), a widely used herbicide, is a small organic chemical pollutant in the environment. To develop a nanobody-based immunoassay for monitoring trace levels of 2,4-D, a step-wise strategy for the generation of nanobodies highly specific against this small chemical was employed. Firstly, we synthesized three novel haptens mimicking 2,4-D and assessed their influence on the sensitivity and specificity of the existing antibody-based assay. Polyclonal antibodies (pAb) from rabbits showed good sensitivity and moderate specificity for 2,4-D, pAb from llama based on selected haptens showed similar performance when compared to those from rabbits. Secondly, nanobodies derived from llama were generated for 2,4-D by an effective procedure, including serum monitoring and one-step library construction. One nanobody, NB3-9, exhibited good sensitivity against 2,4-D (IC50 = 29.2 ng/mL) had better specificity than the rabbit pAb#1518, with no cross-reactivities against the 2,4-D analogs tested. Thirdly, one-step fluorescent enzyme immunoassay (FLEIA) for 2,4-D based on a nanobody-alkaline phosphatase (AP) fusion was developed with IC50 of 1.9 ng/mL and a linear range of 0.4–8.6 ng/mL. Environmental water samples were analyzed by FLEIA and LC-MS/MS for comparison, and the results were consistent between both methods. Therefore, the proposed step-wise strategy from hapten design to nanobody-AP fusion production was successfully conducted, and the resulting nanobody based FLEIA was demonstrated as a convenient tool to monitor 2,4-D residuals in the environment. Display omitted •Reliable step-wise workflow for anti-hapten nanobody discovery was presented.•Highly specific and sensitive nanobody against 2,4-D was obtained by optimized isolation procedure.•A sensitive one-step FLEIA for 2,4-D was developed based on nanobody-AP fusion protein.•The assay showed to be useful and applicable for monitoring 2,4-D pollution in environmental water.