Rye ( Secale cereale L.) is an important genetic resource for improving the disease resistance of wheat. An increasing number of rye chromosome segments have been transferred into modern wheat cultivars via chromatin insertions. In this study, 185 recombinant inbred lines (RILs) derived from a cross between a wheat accession containing rye chromosomes 1RS and 3R and a wheat-breeding founder parent Chuanmai 42 from southwestern China were used to decipher the cytological and genetic effects of 1RS and 3R via fluorescence/genomic in situ hybridization and quantitative trait locus (QTL) analyses. Chromosome centromere breakage and fusion were detected in the RIL population. Additionally, the recombination of chromosomes 1BS and 3D from Chuanmai 42 was completely suppressed by 1RS and 3R in the RIL population. In contrast to chromosome 3D of Chuanmai 42, rye chromosome 3R was significantly associated with white seed coats and decreased yield-related traits, as revealed by QTL and single marker analyses, whereas it had no effect on stripe rust resistance. Rye chromosome 1RS did not affect yield-related traits and it increased the susceptibility of plants to stripe rust. Most of the detected QTLs that positively affected yield-related traits were from Chuanmai 42. The findings of this study suggest that the negative effects of rye-wheat substitutions or translocations, including the suppression of the pyramiding of favorable QTLs on paired wheat chromosomes from different parents and the transfer of disadvantageous alleles to filial generations, should be considered when selecting alien germplasm to enhance wheat-breeding founder parents or to breed new varieties.