Ask YC: Why can't you down-mod people's responses to you?

I understand the potential for abuse here, but I feel like the pros outweigh the cons, and I highly doubt that adding this ability would have a "chilling effect" on disagreements. Quite frankly, most of us don't care about karma enough that we would withhold disagreements for fear of a 1-point reprisal.

I usually up-vote people who disagree with me, actually, if I find that they're contributing to the discussion in a thoughtful way. It's the HN way, no?

But occasionally I get a reply that really deserves it, like this one, which I cannot down-mod: http://news.ycombinator.com/item?id=457222