Mixed matrix membranes (MMMs) have long been considered as promising membrane types for industrial energy-intensive gas separation processes. Current MMMs are still facing grand challenges of poor filler dispersion and poor polymer-filler interfacial compatibility. The present study demonstrates that these challenges can be addressed by fabricating MMMs containing three-dimensional (3D) covalent organic framework (COF) fillers with ultrasmall size-selective pores. Two different polymer matrixes, including glassy 6FDA-DAM and rubbery Pebax, are explored to validate the effectiveness of 3D COF fillers in improving the membrane separation performance. The pure organic nature of COFs facilitates their high affinity with pure organic polymer matrixes, leading to good interfacial compatibility in the resultant MMMs. These porous COF-300 fillers can increase the membrane fractional free volume that enhances the membrane gas permeability. Besides, the ultrasmall pores of COF-300 fillers and the rigidified polymer chains at the filler surface can enhance the size discriminative processes, resulting in increased membrane gas pair selectivity. Moreover, the separation performance of COF-300 can be further improved by functionalizing it with polyethylenimine (PEI), enabling the design of advanced membranes suitable for industrial applications.