Supporting free speech doesn't mean that you support absolute free speech without any limitation. We all have a lot of rights, and exercising those rights can impinge on the rights of others. Why would freedom of speech be somehow special and trump every single other right?
I don't know in English, but in French we have a saying: the freedom of some begins where the freedom of others ends.