Biases in Apple's Image Playground

https://www.giete.ma/blog/biases-in-apples-image-playground

by orf • 1 year ago

43 19 1 year ago

giete.ma

rafram• 1 year ago

How do you solve this without getting Gemini-style racially diverse Nazis?

Tinos• 1 year ago

I wonder if adding a simple "Describe yourself" feature along with the image to help guide the model would fix this?

Like if the model was told he is a white male + given the image

Majestic121• 1 year ago

Since it's taking a photo of the user as input, force match the ethnicity of the input photo and you're good.

david-gpu• 1 year ago

Yeah, I wondered the same.

Maybe attempt to predict and match the biases of the user prompting it? That may cause the least amount of friction.

Or refuse to show images where the input data proved to be strongly correlated with sensitive traits like gender or ethnicity.

I don't have a good solution.

thisismyswamp• 1 year ago

AI safety people worrying that basketball players don't have a perfectly balanced ethnical representation while mega corporations are trying to establish a monopoly on intelligence

gruez2• 1 year ago

AI safety people worrying [...]

"AI safety people" are hardly a monolith. This is probably the first time I've seen "racism" being cited as a "AI safety" issue in months.

iNic• 1 year ago

Please do not mistake "AI ethics" for "AI safety", where the latter is about "Studying and reducing the existential risks posed by advanced artificial intelligence" [1, 2].

[1] https://forum.effectivealtruism.org/topics/ai-safety [2]: https://www.safe.ai/work/statement-on-ai-risk

givinguflac• 1 year ago

“I could not replicate the above results with different photos, although I imagine that this will be possible with more effort.”

I have a feeling that the red background lighting in that image is what is causing confusion for the model.

That being said, I’m not surprised and I’m not sure there’s an obvious solution given current tech. I think apple is making the right choices here to “safely” or benignly provide a tiptoe into image generation for the public.

jbs789• 1 year ago

This was my gut reaction too.

I don’t know a tonne about the models but the white balance and lighting is quite unusual in this photo.

As well as the presence of another person with a darker skin tone.

So a person looking at the photo knows it’s a white dude but a machine has a harder time.

madeofpalk2• 1 year ago

The "obvious solution" is just a bigger model with better training data, right? Which seems to be against Apple's goals of trying to do this stuff with smaller models on device.

ericmason• 1 year ago

Seems like more of a bug than bias. The problem is in ignoring the appearance of the person in the first place. It's a statistical model, and of course there are more black rappers and white investment bankers. If it noticed that the person was white to begin with, and applied that trait, it wouldn't have to guess about the race at all.

prododev2• 1 year ago

It's a statistical model, and of course there are more black rappers and white investment bankers

Yes, this is what the author is pointing out - there's a statistical bias in the dataset that is showing in the results.

gruez2• 1 year ago

Is it a "statistical bias" if it reflects the underlying data? Is it "bias" to generate mostly male lumberjacks, even though most are male?

amelius• 1 year ago

This input

Honestly, the input doesn't seem very well chosen. It is a very low resolution picture of someone, with red eyes, in a circle, with a grey icon partially on top of it, and with somebody else in the picture, half outside the frame.

cranium• 1 year ago

I had one extensive exchange with ChatGPT to generate an image of a man and a woman working together on a leather crafting project. No matter the prompt, the man was systematically the one "working" and the woman being there to assist.

Bias correction in images feels a lot more primitive than in text.

miggol• 1 year ago

All this marketability tuning may some day result in models which are extremely finely attuned to our current societal norms and taboos.

At which point the models will stop reinforcing (racial/gender) biases and start reinforcing said taboos instead. I don't think anyone wants that either

slowmovintarget• 1 year ago

This is the real danger: Baking ideological bias into a utility function for the worst of all worlds.

An LLM can't be ethical because mercy cannot be computed through a distance function. Truth isn't weighted, and justice requires context windows so large that humans can barely manage it and often fail.

We can tune these things to be as inoffensive as possible and they will deliver a seeming that is impenetrable to the casual user, because that satisfies the utility function. That seeming will be worse than the loss of consensus reality we've experienced in the last few decades.

I don't find it frightening that LLMs spit out things that are wrong. I find it frightening that so many people are ready and willing to cede the details to these programs and abrogate critical thought.

madeofpalk2• 1 year ago

I don't know whether I would call it bias or not (and this is Apple's model falling for the same "poor means black" fallacy we've seen before), but I've found their image generation models to be incredible poor at matching the photo you provide it. I've done it with many different photos of myself and they all vary wildly and look nothing like myself.

It's comical how bad Apple's image generation models are.

Biases in Apple's Image Playground

Related Stories