I don't think anyone--outside of maybe psychopaths--has a "hole in their soul." This kind of snooty judgmentalism isn't helpful.

The trick is not in denying yourself any and all "stuff." It's in deciding how much and what kind is meaningful to you.