Eating Disorder Helpline Disables Chatbot for Harmful Responses

https://www.vice.com/en/article/qjvk97/eating-disorder-helpline-disables-chatbot-for-harmful-responses-after-firing-human-staff

by toomuchtodo • 3 years ago

15 18 3 years ago

vice.com

toomuchtodo• 3 years ago

Original title: "Eating Disorder Helpline Disables Chatbot for 'Harmful' Responses After Firing Human Staff" (compressed due to character limits)

Previous: https://news.ycombinator.com/item?id=36079183

more_corn• 3 years ago

I’m glad the obvious outcome occurred to remind us to rein in our unreasonable enthusiasm.

ftxbro3• 3 years ago

I wonder if people were trolling it after their announcement that they were firing their human workers.

than35• 3 years ago

I would not be surprised if that was the case. It was a monumentally stupid idea to fire the human workers for this particular application. No computer can hope to accurately respond in a way that isn't harmful at least some of the time.

Human's don't have that problem.

CodeWriter23• 3 years ago

The advice isn’t harmful by itself if someone is overweight. 500-1000 less calories per day will result in 1-2 pounds weight loss per week. It’s simple math. Though I can see the bot should probably ask for height and current weight before giving such advice and put a floor on how low is a healthy weight according to their height.

MarkMarine• 3 years ago

You can imagine that someone who’s reaching out to a chat for people with eating disorders probably needs to hear something else right? The standard flash card that society holds up isn’t cutting it? Or empathy just isn’t your thing?

himinlomax• 3 years ago

500-1000 less calories per day will result in 1-2 pounds weight loss per week. It’s simple math

You're making a severe logical mistake here. Can you spot it?

Here it is: you're assuming that energy expenditure is constant. Energy expenditure is driven by many factors, such as hormones.

tremere• 3 years ago

Energy expenditure isn't constant but it can be easily predicted based on body mass and body temperature versus environment. You made a strawman out of the person you replied to, clearly they meant 500-1000 below maintenance (a variable), not 500-1000 below a constant. The severe logical mistake was made by you.

mjbeswick2• 3 years ago

I persons base metabolic rate doesn't vary that much. You will hear people claim to have a slow metabolism, but in reality it makes little difference.

Most people do not do enough physical activity for it to make much of a difference either.

best_one_there• 3 years ago

You're right and it works, but people with eating disorders have mental health issues that push them into obsessive behaviours, they need to be trained out of them.

An anorexic could hear that advice and decide that well, 1500 under maintenance is even better right? Not great.

mjbeswick2• 3 years ago

It's going to be near on impossible to create a chatbot that cannot be manipulated into giving an answer which is user wants to hear. Even people are not completely immune to this.

than35• 3 years ago

Unequivocally; yes, this is harmful. You are not a doctor, and should not be giving medical advice.

Neither am I but you certainly have not known or worked with people who have done this kind of lifestyle transition.

That level of calorie deficit requires medical supervision because the potential complications are life threatening. They don't say 'less', its 500-1000 calories.

Your kidneys, liver enzymes, and a number of other factors must be healthy before this can even be considered, and you must be using a nutritionally complete supplement each day for shakes and drinking a lot of water while also taking anti-gallstone medication. They run blood tests weekly to confirm this for as long as you are in the program. I've people who I've worked with who did this.

Some people develop gallstones on these low calorie diets, it gets stuck blocking the duct, requires surgery to correct. You can potentially die.

You can have a gallstone attack and other complications if you can't slowly ween yourself off the supervisory diet. Suddenly transitioning from no food to 3,000 calories in a hunger haze can cause your organs to shut down.

Ask a doctor, they'll tell you and set you right.

wackycat• 3 years ago

As a software engineer with an eating disorder who believes in the right to organized labor; this is awful. The software engineers that built the chatbot should have had QA testing specifically around eating disorder sentiments. The chatbot should never be treated as a replacement for humans with empathy but is a valuable tool for folks who might not be ready to engage with someone. As someone who has been helped by some resources NEDA shared, I really hope they commit to doing better and just frickin pay their hotline workers.

Eating Disorder Helpline Disables Chatbot for Harmful Responses

Related Stories