Most parents want everything to cater to their family... you say. This irritates you.
However, you find it irritating when people call you selfish when things are catered for you.
Who are these people calling you selfish by the way? Who has said your selfish for not having kids for example.
This frustration with parents seem to come from a sense of guilt? Not that I agree you should feel that way. But all this talk of feeling selfish and being called selfish. I have never felt that way about any friends or family I know without kids. Never even crossed my mind.
If you're truly happy with your lifestyle choice, these things shouldn't bother you at all.