HN user
For people who like the simplicity of the rejection algorithm, it might be worth checking out Ziggurat sampling which is based on a similar idea, but shapes the sampling region to reduce rejections: https://en.m.wikipedia.org/wiki/Ziggurat_algorithm
To expand on the other answers, if you want a notion of angle between things then you need the distance to be defined by an inner product (dot product). This leads to the square. So, in some sense squared error generalized our geometric intuitions the best.
Depending on the project / time line I'd be interested to help. Is there a way to get in touch?