There are places where farming is either not feasible, less energy efficient, or where exclusively plant-based diets aren't feasible.
I guess by this logic, since Sahara exists, eating food is a luxury.
Your definition of luxury has transformed into a supreme being just so you can call vegetarianism/veganism a luxury.
If you live in the first world, eating plants over eating meat is not a luxury. It's the cheapest form of healthy sustaining food there is.