If they aren't interested enough to put up with racism, they shouldn't be here to begin with
I did not say "put up with", I said "deal with", which includes many options other than accepting what you're given. If your workplace discriminates on the basis of race, you could report them for violation of anti-discrimination laws, lobby for stronger laws, leave the workplace for a better one, etc.
It is their own responsibility to do something about it and fix the racist environment that they are in
Exactly what I just discussed, maybe in an ideal world such things shouldn't be your responsibility, but the only way to guarantee action is to take it. If someone pushes you into a puddle, you might be right to say that it shouldn't be your responsibility to remove yourself from that situation, as you did not put yourself into it, but I think most would agree that the right course of action is to deal with it yourself anyways.
I am not suggesting that the responsibility falls solely on women or minorities to deal with their own problems at all, but I am suggesting that they should take some agency of their own.