That is exactly free speech. A right to express an unsavoury view without the system punishing you.
Never gave you a right not to be ignored by your neighbours, or not banned from the bar, or not having others nudge their friends to make sure they notice and avoid, etc. Free speech almost always came with consequences until the internet age.