On tests of verbal short-term memory, performance declines as a function of auditory distraction. The negative impact of to-be-ignored sound on serial recall is known as the irrelevant sound effect. It can occur with speech, sine tones, and music. Moreover, sound that changes acoustically from one token to the next (i.e., changing-state sound) is more disruptive to serial recall than repetitive, steady-state sound. We tested manipulations that resulted in changes in (higher levels of) perceptual organization for more complex tonal stimuli. Within a trial, the first two bars of a well-known melody were repeated (a) in the exact same manner, (b) with variations only in tempo, (c) with variations only in mode (e.g., Dorian or Phrygian), or (d) with variations in both tempo and mode. Participants serially recalled digits in each of the irrelevant sound conditions as well as in a silent control condition. In Experiment 1a, we tested non-music students and, to investigate whether musical expertise affected the findings, additionally tested students majoring in music in Experiment 1b. Across both samples, recall in the irrelevant sound conditions was significantly poorer than in the silent control condition, but only the tempo variation caused an additional harmful effect. The mode variation did not affect recall performance, in either music or non-music students. These findings indicate that, at least with music, changes are a matter of degree and not every additional variation impairs recall performance.