Google created a fake pizza brand to test creative strategies for YouTube ads

https://techcrunch.com/2018/08/20/google-doctor-fork/

by CrazedGeek • 8 years ago

88 57 8 years ago

lalos2• 8 years ago

This concept is very creepy. This particular pizza example reported is not but who knows if they come clean on ALL their experiments. They have a lot of power in their hands and now they are running experiments on people without them even knowing? This can get out of hand just like ads were used to push certain narratives during elections all over the world in all platforms and just like Facebook was playing with people's emotions with different posts shown [0]. No regulation, no ethic panel, no external audit, no transparency and they still show it off innocently.

[0] https://www.theguardian.com/technology/2014/jun/30/facebook-...

ThomPete• 8 years ago

I am trying to understand the moral/ethical issue here. Care to elaborate? There might be something I am missing here cause I just don't see it. It's a genuine question.

anonymfus2• 8 years ago

These ads were lying.

Bluestrike2• 8 years ago

Google ran ads under a false brand name to test how viewers reacted to them. From what I can tell, the marketing experiment was fairly similar to standard A/B tests advertisers on ad design and copy or the kind of tests used by designers and developers for new site functionality. The biggest difference being that--and this is the possible ethical concern from the parent comment--the the brand wasn't real. Users had no way of knowing who the ad was produced by, or who was attempting to analyze their behavior.

They aren't totally analogous, but there are circumstances where obscured identity is tested: focus groups market research tests that ask consumers opinions about possible product lines and/or changes, political polling asking how potential voters would perceive an unnamed politician (such as might done when developing a response plan prior to a major scandal breaking...if they have time), etc. In those examples, however, the subject at least knows who is doing the research if not who commissioned it. And most importantly, they give consent.

The Facebook paper was a much more significant ethical concern, largely because it was (1) clearly a psychological study and was, in fact, published as such[0]; (2) had the explicit goal of eliciting changes in subject emotional responses and behavior in order to test the hypothesis; and (3) made no effort to comply with standard ethical practices for informed consent in human research. The paper got around the lack of informed consent because Facebook was a private company and was able to point to their Data Use Policy. That was enough of a fig leaf for Cornell's Institutional Review Board to accept because the Cornell professor was only given access to completed data after the experiment was independently run by Facebook. They accepted it in those circumstances, but I doubt anyone was very happy about it.

Google's experiment is quite tame in comparison, and there are obvious parallels to the sort of benign A/B testing marketers use. There's also--arguably--a pretty major difference in the type of experimentation going on, and it's not intended for academic publication. Also, YouTube terms of use presumably contain language that gives them enough cover to avoid any legal consequences.

Ethicists and university IRBs put a lot of thought into where the boundaries lie and how proposed experiments can cross them unexpectedly. Some of those boundaries have been learned about after some truly horrendous experiments in our past that nobody ever wants to even consider repeating. Others have been much less destructive, but nevertheless with their own flaws and problems.

Personally, I think Google ought to have at least labeled it as a research ad and put their name on it at the end or on the page users clicked through to. While this example was fairly benign with only relatively minor ethical concerns, I understand the parent comment's concerns. There's a lot of interesting questions that can be researched, and I'd imagine some pretty abusive ways in relevant data could be discovered.

In that context, the discussion's a good one to have for both the public and the companies themselves. Not because they're likely to jump into those abusive methods or because they'll repeat something along the lines of the Milgram experiment (or far, far worse). It's simply useful to understand the pitfalls beforehand. I'm not normally one for slippery slopes, but there are instances where considering them beforehand helps avoid the problematic outcomes altogether.

0. http://www.pnas.org/content/pnas/111/24/8788.full.pdf

Apocryphon• 8 years ago

There's definitely been creepy experiments done on Facebook and OkCupid when they play with peoples' emotions and social responses- especially on platforms they themselves control, but this specific example just seems like market research at its purest: seeing how people respond to a typical product. Agreed that such experiments should be published, or at least made to follow ethical guidelines.

rhizome4• 8 years ago

25 years ago I'd regularly (every month or two) get focus group invitations that paid $100+/hr. Along with informing the subjects, compensating participants is a rock-bottom ethical standard that Google apparently cannot meet.

norswap• 8 years ago

Now? Anyone's willing to spend ad $ has been able to do this for ages. It's basic A/B testing. It's impossible to prevent.

legitster• 8 years ago

1. It's hard to understand these conclusions without knowing what they used to measure the effectiveness. Clicks? Skips?

2. From the headline, I assumed that they used a fake brand to track the number of searches for it later. Perhaps to see how paid ads could influence organic search. It seems like that would have been a more interesting result.

ectospheno• 8 years ago

I read the article and I also can't figure out how they measured what they claim to have measured. The results seem made up in much the same way other advertising metrics and results seem made up.

alentodorov• 8 years ago

Whenever I used to run Youtube ads ($100k campaigns) the account manager at Google Ads informed us about an automated option of neasuring assisted and unassisted recall. People whom seen the ads were presented with one question surveys on Youtube. It was pretty interesting as you could see what would be a good reach and frequency for your ads.

iambenjones2• 8 years ago

Hey--This is Ben--I did the research. We used a survey-based tool called Brandlift that measures post-exposure lifts in recall and favorability against control audiences using a defined competitive set. The tool automatically separates out the control audiences and the various test cells so there is no cross contamination. There's detail on the tool here: https://www.thinkwithgoogle.com/products/brand-lift/ Happy to answer any other questions you have--glad you found it interesting...

marzell• 8 years ago

Is this even legal, given false advertising guidelines? They have provided an advertisement for products by a company that doesn't exist. I don't know the legal details, but I am questioning whether this is actually permitted in the U.S. if you go by the letter of the law.

mindfulhack• 8 years ago

My thoughts exactly.

I agree that companies should to have the freedom to be able to test markets to optimise their systems designs (free trade is important), but surely a multi-billion-dollar company can use a real business and spend a little extra dough to test their system without breaking the law. This seems lazy, and legally it should be investigated.

And this doesn't look good for their brand either, now that we live in an age of 'fake ads' swinging entire US elections.

oh_sigh2• 8 years ago

I am not an expert in this area, but none of the ads I watched seem to actually advertise anything. There is basically just a picture of a pizza and a saying like "The tangy sweetness of fresh mozzarella"

https://www.youtube.com/playlist?list=PLFfnLmpHEyLYLqHdyOmVE...

daeken• 8 years ago

For this to be false advertising, there would need to be an offer that's extended to customers. For instance, "$10 large pizza" would be an offer: give us $10 and we give you a large pizza. If they failed to follow through on that deal (by, say, not accepting money and not giving pizzas) then that would be false advertising. There's no such offer in these ads, or anything that could even be remotely construed as an offer.

This is not an advertisement for chocolate cake: https://tastesbetterfromscratch.com/wp-content/uploads/2016/...

modells• 8 years ago

Damn, I'm jonesing for a Celeste four-cheese microwave pizza really bad rn. Where might a lowly member of the public borrow magical 1970's microwave technology in the Mountain View area?

PS: Anyone have one of those huge Radarange RR-9 microwaves with that (capacitive?) touch-panel in the late 70's or early 80's? Yup, I'm ooooold.

sarcasmic• 8 years ago

This is not without precedent in the ad industry, although the level of subtlety varies a lot.

Billboards are a notable example, because the medium is in-house and there's almost always latent inventory. Adams Outdoor once advertised the fictitious, toilet humor brand 'Outhouse Springs', then years later the seemingly personality-enhancing wonder drug 'Reachemol'. Lamar's Milwaukee division advertised a cat doctor with a preference for chocolate who healed 'boo-boos with nom-noms'.

This is essentially a big A/B test, and unlike billboards, they can collect fine-grained data on how people interact with the ad.

trumped2• 8 years ago

looks like engineers or robots decided on the store name... "Doctor Fork"? for a pizza restaurant?

ronilan• 8 years ago

They ran an experiment.

Users reported that eating pizza with a fork was much preferred to eating it with chopsticks.

The Chinese version may differ though. They’ll test this later if the party allows.

thefounder• 8 years ago

Who eats pizza with chopsticks?

partiallypro• 8 years ago

I do wonder if some of the outcomes were dictated by people wondering about what this new brand was. It would be different if they were serving small populations, but they served it 20 Million times. I guess that is a good case study for small brands, but do the results stay the same when people recognize the brand?

sushid• 8 years ago

It seems like this might be the precursor to doing a real test (probably small/regional) with a real brand. If the response was extremely negative I'd think that they'd shelve it altogether.

fipple• 8 years ago

This is like the age-old "hack" of creating a landing page for a B2B startup that doesn't exist, to gauge demand and build a marketing list.

rhizome4• 8 years ago

That's the canonical definition of an MVP.

cookingrobot• 8 years ago

An MVP has just enough features to satisfy early customers. If all you've built is a landing page to gauge interest, that's useful for validating the idea, but that's not a product.

philwelch• 8 years ago

It's usually called a "fake door test". Fake door tests are neither viable nor products :)

lalos2• 8 years ago

Except a landing page clearly transmits that message, you are literally signing up for a reminder.

dmurray• 8 years ago

It's more like fine tuning the conversion process for that landing page. Google weren't trying to gauge the demand for pizza.

amelius• 8 years ago

Tl;dr: Google did a bunch of experiments to see how best to manipulate people. They used a fake pizza brand "Doctor Fork" for this purpose.

gukov• 8 years ago

If you're not paying, you're the prod... test subject.

jacquesm• 8 years ago

There's a Dutch IT company (Topicus) that has an actual Pizza brand, and they brew their own beer too!

drdeadringer• 8 years ago

This reminds me of a hotel that elected to advertise its own "Room Service Pizza" as a local commercial business. It went something like this...

A given guest checks in, unpacks, and finds themselves ready for dinner. They skim over the room service menu, considers Room Service Pizza, and then dials a nearby pizza chain for delivery. Reviews for Room Service Pizza were not good. At the time, "Dominoes Is Cardboard" among similar chain-pizza love was going on so it wasn't looking up for Room Service Pizza even though it was right there on site and arguably not that bad as the story goes.

The hotel tried "everything" but it would never work; recipes, surveys, fresh ingredients... "One Chain Pizza, Please" was always the result.

One day someone desperate and with authority had an idea. They had changed "everything else" by this point, so let's go "marketing". This person drums up a separate phone number, business name, menu, logo, uniforms, the whole bit for Room Service Pizza. Drops the flier menu into the "local restaurants" courtesy spread in their rooms. It's not "Room Service Pizza" anymore, it's "Tony's Tower Pizza" or whatever.

Orders pick up. Reviews go up. Deliveries for Chain Pizza start to drop a bit on a noticeable level. Every time the special phone line for "Room Service Pizza", aka "Tony's Tower Pizza", rang up they knew something had finally broke free for themselves.

... So, I guess my point is that this isn't the first time pizza and playing masquerade have come together for a business. Here, a hotel uses ads and "rebranding" to sell their own pizza vs Google using fake pizza to sell more ads. If pizza were a coin, these might be opposite sides of it.

dawnerd• 8 years ago

Universal Orlando has their own in house pizza that’s totally branded differently and they even throw flyers under the hotel room doors to make it look like a local chain. Thing is, it’s actually pretty decent pizza and the price isn’t too bad either. In their case I think most people have caught on to theme park resort pizza being awful so branding was essential.

nerdponx• 8 years ago

it’s actually pretty decent pizza and the price isn’t too bad either.

Most room service pizza is. Suggestion is a powerful force.

in_hindsight• 8 years ago

Is it the pizza they called during voice assistant demo? ;)

wiradikusuma• 8 years ago

The 1st and 2nd conclusions seem to contradict each other: one said put everything together while the other one said no?

tabtab• 8 years ago

"Totally fork news, believe me!"

Google created a fake pizza brand to test creative strategies for YouTube ads

Related Stories