Ask HN: How does everyone keep track of user reactions to their LLM output?

For example: If you make a chat bot and let the user regenerate their responses; then there is an implicit preference ranking of the chatbot responses. Do any of you keep track of this?