We investigated the use of whey protein isolate (WPI) as oleogelator in liquid oil. First, heat-set WPI hydrogels were prepared varying in microstructure and network density. Then, by applying a stepwise solvent exchange procedure via an intermediate solvent, full replacement of the internal aqueous phase within the protein matrix by sunflower oil was achieved. The solvent exchange procedure was performed by using either acetone or tetrahydrofuran (THF) as intermediate solvent. The oil inside the protein matrix was homogeneously distributed without any noticeable damage to the structure. Analyzing the weight change of the protein gel as a result of the solvent exchange shows that the oil holding capacity depends on the microstructure, the polarity of the intermediate solvent, and the kinetics of the solvent exchange. Depending on the gel microstructure and protein concentration of the preceding hydrogel, the oil content in the oleogels was found to be as high as 91 wt %. Oil holding capacity correlated well with the water holding capacity of the preceding hydrogel, and its Young’s modulus (stiffness). It was found that the oleogels, compared to the hydrogels, were much stiffer, as the Young’s modulus increased by 2 orders of magnitude and showed a lower strain at fracture. Our novel route of structuring oil by immobilizing liquid oil inside a biodegradable protein gel matrix with tunable mechanical properties could be relevant for developing novel materials, e.g., in pharmaceutical, nutraceutical, and food applications.