The real difference is the listening environment, IMO. The amount of background noise when "mobile" is enough to mask much of the finer difference, in my experience. Not to mention mental, visual, and aural distractions that are stealing attention from the music and it's reproduction.
You are absolutely correct in this but I will go on to say that only a small percentage of the population cares about these considerations. Audiophiles are an elite niche group.