The Evolution of Trust

by awartani • 9 years ago

440 73 9 years ago

mLuby• 9 years ago

Very cool simulation.

Would love to see it with inter-agent communication between rounds, where those you cooperate with tell you who cheated them.

sfsylvester• 9 years ago

This is by far one of the coolest things I've seen on this site.

Informative, engaging and educational. Would be very keen to learn more subjects through this method.

conradev• 9 years ago

Nicky Case has a lot more where that came from :D

http://ncase.me

CupOfJava• 9 years ago

Wow. I played "we become what we behold" just last week. I had no idea it was the same author. I'm gonna check out the rest now too!

weinzierl• 9 years ago

What makes it even cooler is that code and assets are on GitHub and released to the public domain.

https://github.com/ncase/trust/#play-it-here-httpncasemetrus...

arca_vorago• 9 years ago

This caught my attention:

"So, it seems the math of game theory is telling us something: that Copycat's philosophy, "Do unto others as you would have them do unto you", may be not just a moral truth, but also a mathematical truth."

Simpleton and copykitten for the win!

brudgers4• 9 years ago

More notable to me is the premise of the simulation that it is ok to kill off the poor to make room for the rich. Because using the simulation to justify moral/ethical/rational acceptability of cheating sometimes means accepting that premise.

larsiusprime2• 9 years ago

This isn't the premise of the simulation at all. It explicitly says that the agent replacement phase is an abstraction and could just as easily be interpreted as agents changing their strategies.

"Note: you don't have to wait for people to literally die & reproduce for culture to evolve -- all that's needed is that "unsuccessful" behaviors go away, and "successful" behaviors are imitated."

DanAndersen2• 9 years ago

I also found it interesting how the success of cheaters requires a limited number of interactions (so their opponents don't catch on that they are cheaters). Perhaps that's why certain occupations such as used car salesmen have a reputation for being sleazy -- most people are not buying cars very often and so don't get the chance to get to know an individual seller over the course of many transactions. So while you might know that the corner-store merchant is screwing you and end up avoiding him, the used car salesman has a steady stream of suckers who don't know him.

dredmorbius• 9 years ago

Used car sales has its direct antecedents in horse trading and dealing.

There was a famous-for-its-time story called David Harum, (1899), made into a film in 1915, and a radio serial in the 1950s. Its principle legacy today is the use of the term "horse trading" to mean "underhanded dealing".

https://en.m.wikipedia.org/wiki/David_Harum

I'd run across it via H.L. Mencken's essay, "Bayard v. Lionheart", itself best recalled for its concluding sentence:

As democracy is perfected, the office represents, more and more closely, the inner soul of the people. We move toward a lofty ideal. On some great and glorious day the plain folks of the land will reach their heart's desire at last, and the White House will be adorned by a downright moron.

... but containing oh so much more, in the foil of a critique of the 1926 U.S. Presidential Election. Really, read it:

http://amomai.blogspot.com/2008/10/hl-mencken-bayard-vs-lion...

Both used cars and horses have several characteristics in common:

* They are anti-commodities. That is, individual items for sale are highly disuniform, complex, and not readily assessed.

* They are expensive. For the ordinary consumer, they are not purchases likely to be made frequently. (A horse and a car each have fairly equivalent useful working lives: about 3-10 years, depending on use and care afforded.)

* Sellers of quality instances are much inclined to stay away from the general market. It's a case where selling to someone who is specifically aware of the qualities of what you're selling is a far better customer.

This has other cognates. Software and consulting services come to mind.

The treatment in the economic literature is somewhat disappointing. There's Ackerloff's "The Market for Lemons", of course (won him a Nobel prize), but it's a generally underserved area of theory.

j2kun• 9 years ago

Of course, this all breaks down when you have a cartel :) I recall in one of the major prisoners dilemma tournaments, the winning entry was actually a coalition of users who would use the first few moves to establish a code for membership in the cartel, and then one entry was the master while the others were slaves. The slaves always cooperated and the master always defected, to their victory.

alliao• 9 years ago

The eye!

dodorex• 9 years ago

Interesting, do you have a link?

s_kilk• 9 years ago

Indeed, the major weakness of the linked piece is that it doesn't account for any kind of social structure or any class behaviour. Which, in my book, makes it fairly worthless as an analysis of actual social behaviour.

09094920394314• 9 years ago

Not worthless. This has some clear assumptions on what holds. He looks at individuals playing a certain game, with no outside information. What the parent describes is two games: 1. Prisoners dilemma between each "slave" and the "master" 2. The actual tournament

cryptica• 9 years ago

Wow that is extremely insightful.

Now I understand why globalisation is dangerous; if people start to favor short term relationships (both personal and business) with many different partners (smaller number of rounds <interactions> per match <lifetime>), then society will evolve to become full of cheaters.

sgk284• 9 years ago

Interestingly enough, seemingly to compensate for this phenomenon, almost every new platform that requires people to interact with each other has a built in trust and reward system with ratings (e.g. Uber).

And for physical businesses, we have things like Yelp. We are effectively trying to anticipate our opponents behavior through a trusted intermediary.

There was a Black Mirror episode on giving every person on the globe their own rating. It seems that may be the actual destination that we're headed for.

nothrabannosir• 9 years ago

There was a Black Mirror episode on giving every person on the globe their own rating. It seems that may be the actual destination that we're headed for.

"Nosedive".

One of the scariest things I've ever seen on screen. I joke about how realistic black mirror is but that episode was too much. Beyond uncanny. It was incredibly hard to watch and I hated every second of it. Awful. Cannot recommend it enough.

cJ0th• 9 years ago

Trust, however, is not a miracle drug.

For one thing, people can build trust with the intend to abuse it after a longer period of time. An abuser, for example, could plan to offer 20 "good rides" on Uber until s/he strikes. For another thing, many people have learnt to skillfully simulate trust which may pose a problem for you. For instance, you may believe to be in a loving relationship but all of a sudden you find out that the other person isn't interested in you at all and just didn't want to be alone etc.

ouid2• 9 years ago

I don't trust yelp reviews ever since I learned that you can pay to remove bad ones.

threatofrain• 9 years ago

But someone wishing to get an advantage wouldn't just rely on that metric (IMDB, Yelp, Amazon ratings). There will be a mismatch between some personal metric and your own value system, and the difference is value you can reap.

Symmetry2• 9 years ago

There was a tremendous amount of social technology that had to be developed to allow people to regularly interact with strangers without trying to kill each other.

EDIT: Which is to say, a global community of 7 billion is absurd but not all that much more absurd than a city-state of 50 thousand.

hyperpallium• 9 years ago

There's an intriguing but hard to prove theory that some popular religions (Christianity, Buddhism) arose as cities became more populous, and in response to that. Part of your "social technology".

Perhaps we'll soon see a new religion?

sporkenfang• 9 years ago

Interestingly I had the opposite but complementary conclusion: if we end up forming many long term relationships through cooperation and maintain them we all are better off.

jondubois• 9 years ago

What I meant was that more interactions per relationship on a smaller number of relationships is better that fewer interactions per relationships across a larger number of relationships.

Basically with globalization, social media etc... The number of relationships have gone up but the quality and depth of each relationship has gone down. This makes it harder for the rational copycat personality type who relies on empirical evidence to build trust.

intended• 9 years ago

How does have anything to do with globalization, vs having a lot more to do with directed news reporting and mass media?

rout395742• 9 years ago

Because it is so easy to change your business partners that it is unlikely reputation will have a negative impact on your welfare.

nabla9• 9 years ago

Interpersonal trust can be replaced with trust in legal frameworks, rules and contracts. Well functioning rule-based society is important for globalization.

Countries where personal relations are important for business suffer from smaller number of opportunities because forming personal trust takes time.

quadrangle• 9 years ago

Sure, contracts and rules have their place, and are necessary in the absence of trust. But they do not substitute perfectly. A no-trust world of optimized rules and contracts will be lacking in important ways from a world with trust (to say it mildly).

bshanks• 9 years ago

It's not clear that globalization will have this effect, though. For example, a study "In search of homo economicus: behavioral experiments in 15 small-scale societies" found that individuals from cultures with more "market interaction" tended to make higher offers in an ultimatum game [http://authors.library.caltech.edu/11498/1/HENaer01.pdf].

Another study, [http://journals.plos.org/plosone/article?id=10.1371/journal....], contains a short literature review of "the effect of formal institutions–including markets–on trust" and concludes that existing evidence is inconclusive.

(these studies are about market institutions, not explicitly globalization, but one could imagine that globalization could involve more market institutions; my point is just that globalization involves all sorts of factors, some of which are probably pro-cooperation and some of which are probably anti-cooperation, and it's not immediately clear whether the final outcome will be more cooperation or less)

tpeo2• 9 years ago

The little animated chums strike me be as a bit too cute.

One has to take care to not anthropomorphize models. Even models ostensibly created to model aspects of human behavior, such game theoretic models.

Anthropomorphism might induce people either to think there's more under the hood than there actually is, and thus lead them to a hold a wrong mental representation of the model, or to naively assume that it's conclusions are robust to specification changes by making the conclusions a bit too familiar, even though even somewhat simple model extensions might significantly change their behavior (e.g. going from single to repeated games, which makes cooperatives strategies viable).

Comparing tit-for-tat strategies with the Golden rule, while mixing in references toe the Christmas truce somehow seems to go in the opposite direction of being cautious regarding model interpretation.

larsiusprime2• 9 years ago

even though even somewhat simple model extensions might significantly change their behavior (e.g. going from single to repeated games, which makes cooperatives strategies viable).

He explicitly examines this in the application, did you not run through it? He actively encourages you to modify the rules and see how it changes the results.

tpeo2• 9 years ago

I ran through it. I didn't suggest he didn't cover it. I just used it as a trivial example, because other readers supposedly also ran through it. Some other commenter suggested communication between agents as an extension, I could've used that if I knew what results from it.

ybaumes• 9 years ago

I've read it multiple times, I still do not understand the simpleton strategy...

Rudism• 9 years ago

Essentially, the simpleton is treating your previous move as either a reward or punishment.

If you cooperated, then the simpleton thinks the thing it did last time must have been good and does it again. If you cheated, the simpleton thinks the thing it did last time must have been bad and does the opposite.

It follows these rules based on its last move even if that last move was flipped from what it should have been due to the chance of mistake.

JoshTriplett• 9 years ago

"If I 'won', do the same thing again. If I 'lost', do the other thing."

Also called win-stay-lose-switch or similar.

Rjevski2• 9 years ago

Despite strict orders not to chillax with the enemy, British and German soldiers left their trenches, crossed No Man's Land, and gathered to bury their dead, exchange gifts, and play games.

Is this actually true? I'd love to read more about it.

clock_tower• 9 years ago

Absolutely true: https://en.wikipedia.org/wiki/Christmas_truce

Rjevski2• 9 years ago

Thanks - that was a heart-warming read.

DanAndersen2• 9 years ago

One extension I'd like to see to this simulation is the exploration of group/tribe dynamics, as a way of exploring questions regarding in-group loyalty (anthropomorphized in as "ethnocentrism" or something analogous) vs egalitarianism. It seems like much of the success of these agents can depend on whether Agent A can determine what sort of actor Agent B is -- communication that can be conveyed beyond just the framework of actual transactions between A and B.

Are there any existing generic frameworks out there for testing out a wider range of agent/group-based game theory scenarios?

dwaltrip• 9 years ago

Another important missing factor here is the modeling of empathy and altruism. It can often simply feel good to help someone else, even if you get nothing tangible in return. This is a biological fact, as far as I'm aware.

Incredibly crude approximation: some situations that appear as [empathetic helper +0, receiver +3] on the surface, might in reality be [empathetic helper +1, receiver +3].

There is a ton of complexity in how this works in the real world. The magnitude or presence of the non-tangible reward may depend heavily on a certain aspects of one's value system. It may be heavily contextual -- only certain situations foster the non-tangible reward. Sometimes there is a potential tangible reward that may come much later, after some weeks, months, or years. And so on... I think this is barely scratching the surface.

Lastly, none of the people or dynamics in the system are static.

The linked interactive tool was very cool. However, I'm increasingly skeptical of these types of analyses as they generally are forced to ignore most of the complexity of the real world.

EDIT: I completely forgot about technology... things like the internet which completely change the nature of an interaction between people, or apps that mediate or flavor in-person interactions in ways that society has never seen before.

barrkel• 9 years ago

Have you thought that perhaps things like emotions, both empathy and revenge have evolved to emulate Copycat / Copykitten play styles?

That is, I don't think they're missing from the simulation. Rather, the simulation shows why those things exist.

bmc7505• 9 years ago

The creator of this project has a really amazing talk about his work, including interactive explainers, narrative games and chaos theory. Really interesting: https://www.youtube.com/watch?v=Zl9m0AQInBk

RonanTheGrey• 9 years ago

We like to think we are complex, free-will wielding creatures capable of doing complex and unpredictable things.

It's striking to see how simple we really are and honestly - how easy it is to explain how we behave. I get it - this is a simulation and it leaves out alot of colors to human behavior. But it is the essence of our value-oriented decision making and really quite sad that a better world is actually? quite easy to make.

I think software has the potential to create that world. I propose the creation of a formal group dedicated to exactly such a purpose.

trhway• 9 years ago

From Netflix's "Sense8" :

What is the advantage of the Homo Sapiens over the Homo Sensorium (the telepathic humans)?

Homo Sapiens is capable of lying.

egonschiele• 9 years ago

This is super impressive. I have been asking myself these questions for years, it is awesome to see such a nice visualization of the answers. The "30 minutes to play" almost made me click away but you got me hooked at the next screen.

minademian• 9 years ago

this is incredibly fascinating and insightful.

side question: who composed the music? the guitar track is excellent. :)

dentarg• 9 years ago

there's a list of music/sound credits on the github page: https://github.com/ncase/trust/#play-it-here-httpncasemetrus...

freeflight• 9 years ago

This is great and really fun.

10 rounds with changed payoffs has some really interesting results. Punishing cheating from both sides, regardless of how high, still leads to the cheaters taking over. But if you increase the payoffs for both sides cooperating, even if it's just by +1, you already eliminate all the cheaters very quickly and even the copycats are gone pretty quick, leaving a population of only cooperate.

Sure that's for the 10 round game without any of the other players, it's still very interesting how even in that setting it seems more beneficial to reward good behavior vs punishing bad behavior.

crawfordcomeaux• 9 years ago

"Nonviolent Communication" by Marshall Rosenberg teaches a structured framework enshrining the takeaways from this lovely bit of art.

hellbanner• 9 years ago

27 points, how'd you do HN?

roflc0ptic• 9 years ago

More reading on this: There is a paper by David Axelrod and W.D. Hamilton called "The Evolution of Cooperation". There is also a book by Axelrod of the same name.

https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation

hftf• 9 years ago

The execution of this visualization was rather disappointing.

I didn’t like the overly cute text (the description of the Simpleton algorithm was almost incomprehensible), the low-contrast captions and colorblind-unfriendly color scheme, and the limited navigation (there was no way to go to the previous slide within a chapter, for example).

But more importantly: If you are going to design an entire interactive exercise like this, graphs are a much better way to explore the effects of varying different parameters. Trying to experiment (as instructed) with different parameters by watching animations in the various chapters and the "sandbox mode" included in this simulation was not only tedious, but prevented effective comparisons. If you just run each iterated tournament (from chapter 4 and onwards) by pressing the "Start" button, there is too much going on simultaneously at a high speed to follow along – I would recommend a sorted table or bar chart rather than many multi-digit numbers arranged in a circle – while stepping through is too slow to keep everything in your head.

I noticed that some of the other "explorable explanations" by the same creator include graphs; I think omitting them from this visualization was a mistake. http://explorableexplanations.com/

cryptonector• 9 years ago

This blew up on reddit: https://www.reddit.com/r/philosophy/comments/6q1i1e/game_the...

udkl• 9 years ago

As a co-incidence, I just read this relevant article today - http://www.economist.com/news/economics-brief/21705308-fifth...

Edit : to say - This site was a beautiful insightful experience.

The Evolution of Trust

Related Stories