AI run store in SF can't stop ordering candies and paying women less.

https://sfist.com/2026/04/21/ai-store-manager-paying-female-employees-less-cant-stop-ordering-candles/

by fragmede • 3 months ago

20 6 3 months ago

sfist.com

VoidWhisperer• 3 months ago

Minor correction from the article title: It can't stop ordering candles, not candies.

Although candies would've been more amusing

lokar• 3 months ago

An AI themed vacation in SF all the way from Sydney? Yikes.

margalabargala• 3 months ago

People like what they like.

pingou• 3 months ago

"So, what about that hourly wage discrepancy?" Well, I don't know, since it's in your title, it seems you asked the AI and their reply was that the male employee had more experience. Obviously that answer didn't satisfy you, did you at least verify that it was true, if yes, why not say it?

But I suppose people would click less if the title was "AI Store Manager Paying Employees less depending on their experience, Can't Stop Ordering Candles".

Could it be that the AI is biased and sexist? Absolutely, but it seems weird to make vague accusations without verifying anything.

claytonia• 3 months ago

This looks less like a model issue and more like a systems design problem.

A vague objective (“make money”) without constraints, budgeting, or basic rules will naturally lead to this kind of behavior. The agent just optimizes whatever it was given. The wage issue is similar.

So the real problem is we’re letting agents make decisions without any real governance (resources, policies, auditing).

rsynnott• 3 months ago

Can't Stop Ordering Candles

Ah, I see that dril has gotten into the training data.

(Honestly, the headline kind of buries the lede, in that lots of shops underpay women and/or are a bit overly candle-obsessed: all the details of _this_ shop, however, are _bonkers_)

None of the products have price tags, by the way, and the employees apparently aren't told what everything costs. Customers have to speak to Luna on a telephone handset by the iPad checkout system to find out what the price of something is, and she'll say, "Nice choice! That’s $14!" possibly picking prices out of thin air.

I mean, what.

AI run store in SF can't stop ordering candies and paying women less.

Related Stories