Shallow Diffusion For Fast Speech Is A Symptom