When listening to music with normal speakers, I usually have the volume at a fairly "normal" level so that the sound only travels to the next room. I tend to turn the volume even lower for rock and metal.
With headphones, I usually have my volume set a two or three notches above the minimum level on my CD player, which is probably around 10-20%, so if I'm in a completely quiet setting, I can hear myself tapping my fingers on a tabletop.
Additionally, if I have to turn my music on higher than 5 notches, which is still too quiet to be heard by people standing next to me, I probably won't listen to anything. This makes it pretty difficult for me to listen to music on a bus and many other public settings.

Of course, this only applies to songs at "normal" volumes. I truly value my hearing, and I think that playing in a band makes it even more important for me not to listen to music at high volumes.
I have this friend who listens to his iPod at ridiculously high volumes. I can always hear what he's listening to from across the room. Well, whenever I talk to him, I have to yell.
How ya doing, buddy?