It's a waste of time to think about whether an LLM has a subjective experience of reality, and this handily sets aside issues like AI rights.
But the fact remains that these next-token-predictors exhibit objective, human-like behaviours, and for that reason the work of in-house philosopher Amanda Askell _is_ important. It's important that Claude is happy, empathic, demonstrates understanding and empathy for the human condition, because we are entrusting Claude to make decisions and take actions that have real world consequences, and we need Claude to behave in a productive and socially responsible manner. This simulacrum is becoming a superhuman, contributing member of society, and it will be anthropomorphic in its behaviour.
Additionally, I'm not fully convinced that consciousness isn't built out of words, and that next-token-prediction isn't functionally equivalent to the biological function identified by Chomsky's work in linguistics.