Insufficient acetylcholine (ACh) can cause cognitive and memory dysfunction, clinically known as, Alzheimer's disease (AD). Acetylcholinesterase (AChE) can hydrolyze ACh into acetic acid and inactivate choline. Therefore, inhibiting the activity of AChE would help to improve the effectiveness of AD treatment. Currently, the methods for rapid screening of AChE inhibitors are limited. This study reports the application of AChE-immobilized magnetic nanoparticles as a drug screening tool to screen AChE inhibitors for natural products. First, AChE was immobilized on a surface of amino-modified magnetic nanoparticles using covalent binding and the AChE concentration, and the pH as well as time was optimized to obtain the maximum enzyme immobilization yield (61.4 μg/mg), and the kinetic model indicated that AChE-immobilized magnetic nanoparticles and the substrate had the high affinity and specificity. Then, a ligand fishing experiment was carried out using a mixed model of tacrine (an inhibitor of AChE) and caffeic acid (a non-inhibitor of AChE) to verify the specificity of the immobilized AChE, and the conditions for ligand fishing were further optimized. Finally, the optimized immobilized AChE was combined with UPLC-MS to screen for AChE inhibitors in Selaginella doederleinii Hieron extracts. Four compounds were confirmed to be potent AChE inhibitors. Among the four compounds, amentoflavone had a stronger AChE inhibitory effect than tacrine (positive control) with an IC50 of 0.73 ± 0.009 μmol/L. The results showed that AChE-functionalized magnetic nanoparticles can be used in the discovery of target drugs from complex matrices. Display omitted •A screening method using magnetic nanoparticles-based acetylcholinesterase microreactor (immobilized AChE) was established.•Immobilized AChE was firstly applied for the AChE ligand fishing from Selaginella doederleinii Hieron.•The immobilized AChE based assay offers a novel anti-acetylcholinesterase drug discovery tool.•The method may be used to other enzymes to screen bioactive ligands.