The "overlearning the game" problem

http://andrewoneverything.com/the-overlearning-the-game-problem

by sendos • 15 years ago

362 137 15 years ago

andrewoneverything.com

kragen3• 15 years ago

Less Wrong talks about this a lot; their term for one variant of it is "superstimuli".

What's the difference between your "overlearning" and "hacking"? They sound like the same thing.

This phenomenon is the reason for the Wikipedia rule, "Ignore All Rules".

sendos5• 15 years ago

> What's the difference between your "overlearning" and "hacking"? They sound like the same thing.

They are similar, but I think one difference between "hacking" and "overlearning" is that a hacker can learn to subvert/circumvent a system quite quickly, and usually does so in a way that breaks the rules. Overlearning usually takes longer to achieve, and is a more insidious problem because it achieves its goals without actually breaking any rules.

kragen3• 15 years ago

It depends on what you think "the rules" are. Generally speaking, hacking happens in a framework that has unbreakable rules, whether those are rules of a game, of a CPU, or of an OS. A solution becomes a hack when it exploits those rules in a way that was previously thought impossible.

Of course, there may be other, breakable, rules that are broken — "You can't render 3-D in real time on a PC!" — but those aren't the relevant rules.

Many hacks are not quick at all to achieve.

philwelch6• 15 years ago

Ironically, if you actually do ignore all rules on Wikipedia the process and policy wonks will crucify you. In fact, WP process and policy is another great example--the original purpose was to write an encyclopedia, but now it's just used as a forum to bicker over the internet.

drhouse_md• 15 years ago

This article seems to be addressing 'the spirit' of a document or system.

Perhaps another way to look at the problem is to imagine creating an A.I. that you want to succeed at whatever system you present. In most cases, an A.I. will take the literal interpretation of the system and become a test-taker, an electable 'gotcha-game' politician or even an entity that finds it can maximize game theory to its own ends by complicating the rules of an existing system to the point of absurdity once it becomes powerful enough to modify and create rules.

So then how do we create systems resistant to beings that take everything literally? I suppose the only way is to reward certain outcomes as opposed to rewarding the direct product of the system itself.

Examples: After an election, have we elected someone who has met with a high degree of favorability in the electorate by the end of his term?

After having students become proficient test-takers, do they then become excellent doctors, lawyers, engineers, etc.. ?

In a game invented to be fun or fair, once overlearned, do they produce fun or fairness?

If not, then a new game needs to be created or the existing one might need to be extensively modified to produced the desired product. This is where the internet shines, where everyone is welcome to take an existing system and modify it to something better. The problem with politics, law, stock market, etc... is that they have become the only method adopted in real society (there is only one game in town).

If reality were allowed to adopt, incorporate or evolve from systems/games from virtual reality (i.e. internet) there might be some productive change. But first we would need to see the first step taking place, that being even flawed virtual systems are allowed to manifest in significant proportion within real life society.

Don't hate the game, hate the player. Every game that profits a winner will have its cheaters.

shubber• 15 years ago

I'm reminded of Mechanism Design: a Nobel prize winning theory of economics that starts with the supposition that agents in any system will exploit its rules to maximize their personal gain. The corollary, which I've been quite taken by, is that where we can influence the rules of the game, we should design them such that exploitation serves a social good.

wisty4• 15 years ago

It's related to Goodhart's Law:

Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.

This is often the result of attempting to overoptimize a system. You can optimize a race car to a huge degree, because you know exactly what you want it to do.

You can't optimize a schooling system, because you don't know exactly what you want it to do. A little noise is a good thing, because the you want a little wiggle room for teachers to sidestep the dictums of education czars, and students to sidestep the dictums of teachers.

The Greeks solved this quite a few years ago, with sortition. Under sortition (injecting noise into elections - http://en.wikipedia.org/wiki/Sortition), Bush and Gore would have been forced to pay "paper, scissors, rock" for the presidency. Under the US's more pure democracy, they would have been tempted to make all kind of Faustian bargains with sordid players to nail down the last 0.01% of votes.

Randomization means that the last percent is just not worth chasing, so players in a competition won't be tempted to bend the rules for a tiny advantage.

The same process could be used for tests. If you allocate places in desirable courses (say medicine) randomly to anyone above a certain score, the top students won't bother drilling as hard just to get the top score.

Stocks are the same - quants wouldn't sweat timing as much if their placement in order books was randomized. It would be more efficient to pay attention to fundamental value than momentary fluctuations if they weren't guaranteed to make large profits on the momentary fluctuations. Some would still work on timing, but not as many.

Patents are just bad policy badly implemented at the moment, not over-optimized.

tolmasky• 15 years ago

Do you have any links on this? I hadn't heard of it before and am quite interested in it now, but could not come up with anything after a quick Google search for "sortation". Thanks for sharing this!

wisty4• 15 years ago

I've edited in a link now.

The main disadvantage with sortation seems to be that it killed Socrates. Randomly selected jurors did quite a few nasty and arbitrary things. But an elected tyrant (especially one who was willing to do anything to stay popular) might be worse. It's hard to say that sortation is bad because it allowed mistakes - every political system allows mistakes.

Also, systematic slavery, misogyny, the occasional genocide and other discrimination against non-male-citizens (non-voters) was a problem, but that's not really related to sortation. Any democracy can have those problems. Fortunately, we seem to have progressed a little in the last 2000 years (despite the occasional step back).

sendos5• 15 years ago

Apparently, the expression is 'sortition' not 'sortation'. wisty has a link to the relevant Wikipedia article: http://en.wikipedia.org/wiki/Sortition, and a Google search for 'sortition' yields quite a few results.

apsurd• 15 years ago

Good comment. I smiled at the sheer simplicity of injecting randomness to mitigate the >95% in order to deter exploitation. Of course it would be an interesting experiment to conduct and compare results against.

Also regarding your racecar example; they've gotten so good at making them go fast they actually are required to have restrictor plates (http://en.wikipedia.org/wiki/Restrictor_plate) so this fits the example of exploiting a well-defined goal to the point it's no longer competitive/safe/fun.

I find that quite amusing and cool. (I don't follow the sport I just like watching science-type shows on tv)

tomjen32• 15 years ago

I don't think it is a good idea at all. If the political power boiled down to picking paper over scissors, the loosing side may very well ascribed no legitimacy to the outcome. That is almost certainly going to end far worse than actually counting all the votes (i am looking at you, supreme court).

The same problem happens with your exam example. Why is it fair that someone less competent than you got in? A better solution would be to let everyone over the cut off point in.

I can't disagree with your example of the stock market, so long as the trades were still executed with a few seconds.

wisty4• 15 years ago

If some guy gets into medicine because their parents did their homework, drilled them in SAT techniques, and hired a firm to fabricate a good admissions letter, are they really more competent than a kid who did their own homework, got a high SAT score because of their IQ and good study habits, and wrote an honest admissions letter?

Do people think our current politicians are legitimate? I'd rather see them be more willing to say what they think, and act with conviction than get a guy who is 0.1% more popular.

Yeah, it's a strawman. Will randomization discourage people from playing the game dishonestly? Or will people panic, and play the game harder, and give up on the honest component of what they are doing? I don't really know.

danssig2• 15 years ago

The losing side will always find some reason to claim the outcome is illegitimate and if you don't get in you will always be able to find someone you think you are better than.

rgoddard• 15 years ago

What might make sense would be a combination. Have some barrier to entrance to limit the people to choose from, but then select randomly from the remaining candidates. For example, to be considered for an elected position, you first have to receive %x of the vote, or maybe only select from the top 3. This way you help to ensure that the person being elected is representing some of the population, but hopefully eliminating some of the games.

SoftwareMaven• 15 years ago

It would just be recognizing the jitter that exists inherently in the system. You will always get a different number when counting the votes, so you can end up with a "50% voted for both" situation.

Why is flipping a coin worse than pretending the next count will be more correct? You kind of contradict your own point by implying the supreme court didn't supply legitimacy to the election. Nobody can blame a quarter for being stiffed with neocons on the heads side.

utoku• 15 years ago

I think the problem might be a deeper problem.

The writer of the article is asking generally about how a system's rules can be exploited to the point that the original intention of the system is no longer achievable. The sortition is also a system with rules, therefore it can be exploited. As soon as I saw the rock-paper-scissors, I remembered a competition some time ago where different bots played rock-paper-scissors against each other. Now the best bot exploited the reality of the seed number generators, compiled a database of sequences to find which seed the opponent was using, and so on.

So even sortition might be exploitable.

r00fus2• 15 years ago

> So even sortition might be exploitable.

But the idea is that it's not easy to exploit. If it is more than trivially difficult to leverage, then that is wasted time/effort that dilutes from the greater message.

ie, why waste effort on the "tie-breaker" message when you can just spend that effort avoiding it altogether and push yourself beyond the margin-of-error?

The key to sortition is that effort expended on gaming sortition is wasted and non-complementary effort compared to the original contest/competition... right now, getting that 0.1% by dirty tricks is equivalent to getting-out-the-vote of 0.1% more of the populace.

kondro• 15 years ago

Bring on the Culture!

tsycho• 15 years ago

Very interesting comment, hadn't heard of Sortition before. However, I see a few practical problems with this.

First, it makes it harder to fix any errors/cheating. You can ask for a reevaluation of exam papers or a recount of the ballot if you felt that something shady happened, but then what about the randomization phase. In the admissions example for instance, if after a re-eval, the composition of students in the >95 percentile changed, do you hold a new randomization to make it fair for the new kids who crossed the threshold? If you do that, is that fair to those who previously got selected in the previous admissions ballot, and weren't second time lucky? Would this lead to all the "losers" in the lottery to ask for re-evals just so that the lottery phase happens again? (given a large number of students, if there is any subjectivity in the evaluation, every re-eval is likely to change the composition, even if by a small amount)

Secondly, the losers in the lottery are going to feel cheated and angry, even if all evaluation was objective, and there was [provably] no cheating. I think most humans prefer an objective, unambiguous way of competing with each other, even when we think that the evaluation metric is flawed. So the unlucky competitors will feel angry post-lottery, even if everything was done in a fair manner.

Finally, it increases the chances of cheating/gaming the system at the randomization phase i.e. the administrator can game the lottery. And if/when the administrators do cheat, it is also easier for them to hide their cheating since the anomaly can be hidden within the random luck factor.

tomas• 15 years ago

The reevaluation issue could be solved if all students in the >95 percentile is assigned a random number (e.g between 0.0 and 1.0) and then the students with the highest numbers are selected.

If a re-eval happens, all students that were already assigned a random number since before will keep it, and new random numbers will only be given to the students that previously were below the threshold.

This will have the effect that the list of selected students will not change much even after a re-evaluation. Only the few with lowest numbers risk losing their place.

sophacles• 15 years ago

I don't think the core of the idea is literal randomness injection. Compare for a moment high precision measuring devices. Any decent measuring device comes with a manual describing the tolerances of the device, the error rate, the expected skew caused by error and so on. They may be highly accurate, and very precise, but they don't claim 100% accuracy. Yet, ideally, this should be a simple task -- the math says if we do X to make the device, there is only one solution and the device is perfect. We take pains with these devices to construct them perfectly, but errors still slip in - and we acknowledge it. To counter these errors we do statistical analysis on the machines' output, we scrutinize anomalous readings and discard or normalize data within the error range and otherwise account for errors.

Look now at the Bush/Gore election cited above, in light of this. A vote is simply a measurement of the people's desire for leader X or leader Y. This measurement is taken via a tool called the ballot. The ballot is for some reason taken as an ideal measuring device - for some reason we think it is a perfect tool of measurement. Yet, a huge chunk of the debate and contention of the scenario came out of problems with the ballot itself -- hanging chad, butterfly ballot confusion, and so on, are all measurement issues. At some point, we should have said "look this is just too close to call with our ballot measuring system, other measures or means should be used to determine the outcome".

The same is true of tests, they are an imperfect measuring system we are trying to treat as idea; so are many other systems you allude to.

The core idea here, is to stop treating our measurements as some sort of ideal measurement and instead look at alternate measures and methods once you get within the error range of the original measuring device -- at this point you are in the realm of "luck" anyway, because there are other unaccounted factors affecting the measurement anyway. Further, it has no more effect on cheating than treating an imperfect measure as ideal -- cheaters in both cases attempt to exploit unnoticable issues in the measurement. Statistical cheating detection already considers this sort of thing anyway, if the statsticians are any good.

As for the social situation you present -- this is a trickier problem, but I suspect a large portion of it could go away once proper understanding is given to people. A lot of the "unfair" complaints are essentially based around "the measure is perfect, so if I can tweak the value measured of my by a small bit, it has some fundamental implication". Essentially we need to train people on the difference between platonic ideals and the realities of the scenario. Then instead of complaining over obscure minutia, energy can be focused on either bettering the overall situation, or bettering the measurement.

As to your final point: ignoring the imperfectness and pretending it is perfect does not really change the gaming problem, in fact it adds to it, because when you treat the imperfect as perfect, you must have lots of complicated rules, which also allow for crazy gaming. Instead effort could be better spent on gaming detection or simple administrator investigation/oversight.

Greedy_Fools4• 15 years ago

Do you guys know that shooting theives stops them from stealing ?

Jun82• 15 years ago

Interesting. Somewhat akin to dithering, i.e. deliberately inserting noise in quantized signals to increase their perceived quality (http://en.wikipedia.org/wiki/Dither).

pixcavator• 15 years ago

>>Randomization means that the last percent is just not worth chasing...

True, but only in the case of complete randomization. In the case of randomization kicking in only when the election is close, one side will be chasing the last percent to get to randomization and the other to avoid it. Same thing...

mikeash2• 15 years ago

Maybe the randomization could be weighted by the individual percentages. For example, randomize if the candidates are within 1%, but give the higher-voted candidate an increased chance of winning weighted by how far apart they are. That way the difference kicks in slowly, reducing the incremental value of additional votes in a reasonably smooth fashion.

pkamb• 15 years ago

Over-optimization of race cars can be bad too. Take the recent popularity of vintage racing. People don't want to watch 50 similar-looking, generic, computer-designed, air-tunnel-tested, same-engine cars going around a round track.

What they want to see is unique and historical body styles for every company in the race, powerful cars vs. nimble cars, finicky engines, twisty turns and long straightaways, etc.

Could be an example in the original post, really. Over-optimization leads to a worse sport.

bluedanieru• 15 years ago

We had something on here many months ago about a voting system basically the same as what we have now, however instead of determining the winner by who got the most votes, the winner is determined by randomly choosing a single ballot.

learc832• 15 years ago

Couldn't they just require that you must win by at least 1% or there's a runoff, and achieve the same result?

drewcrawford2• 15 years ago

If you pay a man by the hour, he'll work a lot of hours. If you pay him by the brick, he'll lay a lot of bricks.

These "games" are basically the equivalent of counting lines of code or checkins. We're measuring poor proxies instead of the things we're actually interested in. The solution isn't an arms race to build bigger and better proxies, the solution is to measure real things instead of artificial ones.

Here's just one example of what I mean by "measure real things". Electing representatives every X years to decide the laws of the land was once upon a time the fairest and best way to have the voices of the masses heard. Today it is feasible to directly poll everybody about every issue, so we no longer need the proxy. If you say everyone cannot be educated about every issue, fine, I can "follow" PG's votes on wall street reform and grellas's votes on IP tort reform and Schneier's votes on TSA etc just by copying their votes on those issues into my ballot, a permission which I can revoke at any time or on a vote-by-vote basis, as easy as unfollowing them on VoteTwitter. This is better than the proxy of professional politicians deciding every issue with fixed terms.

sendos5• 15 years ago

I love the idea that electing representatives every X years to decide the laws of the land may not be the best way forward, and a new approach that takes into account modern technology may be a better way to go about this.

However, based on human nature, I think that given long enough time of your proposed system in place, some people will overlearn it and find a way to exploit it, e.g. to gather tons of votes for their favorite cause. We may even see 'vote consultants/strategists' that guarantee a huge increase in the votes for the cause you want to pursue.

roel_v6• 15 years ago

Of course, you already see this in indirect form in professional PR firms whose sole job it is to influence public opinion on a narrow issue, and in direct form in places where 'direct voting' is implemented in some form, like in California's referendum system for which on almost each issue specific lobbying groups have a major influence on the outcome.

I'd argue that referenda are more even susceptible to the problems described in the article than transparent representative democratic systems are; laymen are easy to influence, and groups of laymen even easier. In the end it depends on the population that is being ruled though - people get the rulers they deserve, and blaming the elected people while still participating in bribery, nepotism etc (while rational! classic prisoners dilemma!) is useless. But of course somebody needs to step up, like the Indian guy is doing now.

jacques_chester15• 15 years ago

The idea of directly polling everybody is not new -- that 'everyone' could vote on everything was at the core of Athenian government.

The US founders specifically rejected that system because it leads to majoritarianism. And it would do so again, if you let it.

drewcrawford2• 15 years ago

> The US founders specifically rejected that system because it leads to majoritarianism.

It would be wonderful if you could cite a few primary sources discussing a direct vs indirect legislative branch as leading to majoritarianism, whether a longitudinal study or quotation of the appropriate historical person.

You can still have separation of powers, judicial oversight, bills of rights, local governments, federalism, and all the checks we have today against majoritarianism within a direct legislative system. Direct legislature doesn't mean only one branch of government.

To put it another way, I don't believe the solution to the "tyranny of the majority" is to set up little tyrants to rule over the majority instead. Even if those were the only two options (and they're not), I would prefer majority rule to what we have.

jowiar• 15 years ago

Further along these lines, modern communication has caused the US political system to more closely approach direct democracy, causing problems along the lines of those foreseen.

Prisen• 15 years ago

The problem with this is that all issues regarding taxation/government spending are non-orthogonal, and can't be uncoupled easily (or at all).

erikb2• 15 years ago

It is just very hard to measuer real things. One program is not like the other. How do you want to measure success in a software project, where the result is actually not even clear to the customer?

danmaz74• 15 years ago

The real problem with measuring real things is that real things have infinite dimensions. You can only measure along a limited number of dimensions, and if you want to make a ranking you can only directly compare one (scalar) dimension at a time.

So in the end you will always get overoptimization over those dimensions you are measuring, to the detriment of the other dimensions. I don't think there is a better solution than continuously changing the dimensions of measurement.

pyrhho• 15 years ago

> Today it is feasible to directly poll everybody about every issue, so we no longer need the proxy. If you say everyone cannot be educated about every issue, fine, I can "follow" PG's votes on wall street reform and grellas's votes on IP tort reform [...]

You'd love Hyperarchy (http://blog.hyperarchy.com/2011/07/19/hyperarchy-democracy-f...)

adrianN• 15 years ago

This would not work in practice at all because it assumes that most people are sufficiently well educated, informed and interested to make a sound decision.

I for one would not want to spend much time on reading new law proposals and thinking about their exact meaning and implications, that's the job of a politician who is actually paid to do that.

redthrowaway2• 15 years ago

As you allude to, the problem with direct democracy is that people are stupid. Not all people about all things, but most people about most things. The realm of human knowledge is simply too vast for anyone to have a broad command of, so we specialize. This means that, for any given issue, the vast majority of people are ignorant about its specifics. Thus we cannot expect the masses to make wise decisions about most issues. The obvious counter-argument to this is education, that educating the public on the issues at hand would lead to better decisions. However, that would fall victim to the same demagoguery and corruption that we see infecting politics today.

By way of illustration, think of an issue that you care deeply about and are well-informed on. Now ask yourself if you would trust the public at large to make policy on it. For every pg, grellas, or Schneier, you'll have far more popular Becks, Palins, and Bachmanns. Direct democracy would be inherently susceptible to such demagogues, and as such cannot be seen to be any more reliable, or likely to produce wise policy, than the current system.

seats• 15 years ago

>> The realm of human knowledge is simply too vast for anyone to have a broad command of, so we specialize.

This statement could actually be used just as easily against voting for representatives as we do today. The real question is do you trust public at large to make a better or worse opinion than <name your representative> on some issue you are deeply informed on? In other words the comparison should be versus the existing system as a whole, not versus an expert on a single topic.

My perception of the demagogue is that their power comes from railing against the current authority. So while I can agree that it seems likely that a 'true democracy' would instantly prop up the Becks and Palins of the country, who do they blame when a majority of people vote that they are wrong? Or maybe even more extreme who do they blame when their views and proposals are voted for but still fail?

There is no longer a 'bad guy' to blame and arguments put forth by a demagogue will have to deal with two new realities-

1- They can no longer claim they are part of the majority opinion in an effort to gain further support. It'd be hard for Sarah Palin to keep up her 'regular joe' persona and reconcile it with being in major disagreement with a majority of the country

2- They can no longer claim obstruction by some authority steam rolling them in legislature. It would become pointless to argue about which pundit supported which policy or law in the past, because every individual would have their personal voting record.

acroyear2• 15 years ago

I like the idea of a Ward Republic (see Jefferson) coupled with a Delegate Democracy.

Greedy_Fools4• 15 years ago

It's not about people knowing a lot of things, it about people with a lot of things knowing. The fundamentals are what's lacking, the basic lies. "... ALL MEN are created equal" , lie from the very foundation, and why, why ,why can't they grasp "do until others as you would have them do unto you", the very heart of what they claim as their religion and proof of their legitimate claim to power ? It's only about holding key truths, not about vast sums of knowledge. People can do anything together, all the answers exist. It's the lack of power holding key truths that skews everything to the lack of progress,the lack of agreement and ability to find solutions. In short, we can fix it very quickly, the answers are actually simple and abundant, the people are deep and strong. BUT THEY WON'T LET US.

stretchwithme2• 15 years ago

We should at least be able to select who represents us. Think proportional representation.

I think its crazy that we can hire a lawyer or auto mechanic that must do what we ask, but we cannot be represented by someone that shares our perspective.

I don't think direct voting on every issue is the answer. Having the ability to pick our own delegate and change our selection any time is better. I really don't have the time to read every bill. But we can read people and share information with others that share our perspective.

Right now we must accept representation by whoever gets the most votes. And it doesn't even have to be a majority of the votes.

We also have a big cost shifting problem. People and states seek to shift costs to the larger country. That destroys any ability to weigh costs and benefits. Pushing such assessments down to as local a level as possible improves decision making.

uses• 15 years ago

I can see it now: "And make sure to VoteFollow us at #GlennBeck. Our research team carefully picks the right votes on all the issues."

learc832• 15 years ago

I'm in favor of ending the cap on representatives in the House. The House was never meant to be capped at 435.

Greedy_Fools4• 15 years ago

That would be an actual representative government. A good way,logical and efficient and there are others. None of which will ever happen because government cannot be representative, no one will give anything up and inequality cannot be represented. You know the saying, if voting changed anything they'd outlaw it. We don't need a better way to communicate individually with our governors, we need better minds holding power.

jacques_chester15• 15 years ago

Systems theorists say that "structure predicts behaviour". It's a bit trite, but also deep. Here's an example.

The US political system pretty much guarantees bad budgeting. Members of Congress are elected fairly independently. There is no party discipline, so each member will operate independently to maximise pork. This encourages horse-trading within and across party lines; nobody can be forced to give up something for a general good.

There's more: there's no incentive to balance the budget. The Executive's separation means that Congress does not need to concern itself with proper administration; it only doles out the cash. It has every incentive to ... maximise pork.

The dynamic behaviour of American government arises from the static structure. The drafters of the constitution drew on their knowledge of history and current affairs to try and avoid certain pitfalls. The US Constitution was state-of-the-art when it was written. It's less so now.

Countries where the Executive is formed out of the Legislative -- the Westminster system -- tend to have much stronger party discipline, because that discipline is required to pass budgets, enact legislation and to form the Executive. This tends to almost eliminate horse-trading, except between parties and independents. It's not perfect -- whole parties can engage in pork too -- but when policy emerges that benefits the many at the cost of a few, countries with party discipline will find it easier to adopt than those without.

Australia, which has the amongst the toughest party discipline in the democratic world, is also a reform leader. And I think a lot of that is explained by our constitutional arrangements.

groaner• 15 years ago

The cynic inside me would extend this problem to the entire economy or even human society itself. Any economic transaction is supposed to benefit both parties: the buyer offers an amount he is willing to pay for what he wants, the seller provides his goods or services at a price he is willing to accept. As PG might say, wealth is created in tandem with creating stuff that people want.

The problem is that some elements in society have become extremely effective at creating a perversion of "stuff people want" for their own benefit. They exploit loopholes in the system, whether by preying upon the poorly-informed, shifting costs onto externalities that we can't price properly, engaging in corruption, or producing items of questionable actual value but very attractive perceived value. They take advantage of our desire for easy answers to our problems with minimal effort expended.

Case in point: Ponzi schemes, coal power plants, Halliburton, the tobacco industry, Zynga, and even to an extent, religion

We've tried to control this with laws, education, and social norms, but ultimately it seems the invisible hand reigns supreme.

Joakal• 15 years ago

Hey USA, your first past the post system actually does tend towards dual party governments [0]. Compare it to preferential system where voters can pick several parties or vote for one party for the preferences listed.

For example, look at this: http://www.abc.net.au/elections/federal/2010/guide/deni.htm With USA's system, Australia Labor Party would have got re-elected.

There's still improvements to be made for democracy that doesn't need to involve such fancy technology yet [1].

[0] http://en.wikipedia.org/wiki/Duvergers_law

[1] http://techpp.com/2011/06/08/homomorphic-encryption/

loup-vaillant2• 15 years ago

I think "overlearning" is not a good term (edit: it is quite accurate, though), because it tend to suggest the worst solution of all: that we refrain from learning. I prefer "lost purposes"[1].

The primary purpose of a game is generally to Have Fun. This purpose is lost if you have "solved" the game. The stated purpose of patents is to foster innovation. However it doesn't work[2]. The purpose of schools is learning. However its methods are flawed [3,4]. And so on.

The trick is to know your goals, and then find out means to best achieve them. The author said:

> But, in real life, we need to keep "playing the game": we need to have elections, and protections for inventors, and laws that govern society, and a market where companies can raise money.

But the actual goals are different: We don't need election, we need a working democratic system (which may, or may not, mandate elections). We don't need protections for inventors, we need innovations. We don't need laws, we need a fair and working society. We don't need a market where companies can raise money, we need a working economy.

Well, I could attempt recurse further up until pure morality, but that would be intractable. But at least you get the idea. If something looks broken, think about its ultimate purpose before you try to fix it.

[1] http://lesswrong.com/lw/le/lost_purposes/

[2] http://www.dklevine.com/general/intellectual/against.htm

[3] http://vimeo.com/5513063 (Dr Tae)

[4] http://www.khanacademy.org/

jorleif• 15 years ago

Overlearning/Overfitting is a term from machine learning/statistics that means that a learning system fits its learning data so well that it does not generalize to new data. The examples given in the article seem like perfect examples of this, e.g. elected officials fit the election process very well, but do not "generalize" to leading a country.

loup-vaillant2• 15 years ago

Ah. I didn't see it as jargon. I recognize the term is accurate, but my main point still stands: the connotation is still there, and still does suggest we should learn less about those systems if we want to preserve their original purpose.

Worse, the end of the post suggest we have to preserve things like schools, markets and elections, even though we learned enough to spoil them. Therefore, the only thing left to sacrifice is learning itself.

That would really really sucks: I'm curious, and I always want to learn more. There are many things I would sacrifice before I accept to put hard limits on what I am allowed to learn.

acroyear2• 15 years ago

i prefer the term 'metagaming'.

donaq• 15 years ago

Regarding politics, it is interesting to note that the people have not, for their part, also overlearned the game of electing the best man for the job. I wonder why this asymmetry exists?

erikb2• 15 years ago

Many people see this from a macroeconomic or political point of view. So I think adding a more philosophical point of view might also be of value. Of course, because it is philosophy, I can only present my own point of view. There is no right or wrong.

For me, as a Zen student, the solution is to just accept that that it is, what it is. No system (shape, for Zen students) can be perfect. It is created to solve a problem and with time starts to fail badly at doing anything about the problem. Then another system is created by someone else and the cycle restarts. You might think for example that the stock market or democracy is a thing that doesn't change, because it exists longer then you live. But in the end it will change. We had different political systems before and we will develop different systems in the future.

So in the end the system is one of the most important things we have, because it gives us something to base our decisions and actions on and goals to strive for. But also the system is nothing, just an illusion we create for us, maybe based on how we understand the illusions other people created for themself.

The thing that is interesting to me personally is that every system itself is instable and will change or die in the end. But the life cycle of a system, what it does for us and doesn't, that all will always stay the same and even though we try to change that, we will never succeed. So while it changes a lot in one way it is totally unchangable in another. But that just as a side note.

resatori• 15 years ago

I think it is time we recognize that it is not the systems that shape our world but the individuals.

There is no optimal system, be it political, economic or whatever.

You can always find loopholes.

I think it is time we take responsibility for what we do - then there is no need for a better system.

The patent system is not responsible for people attacking each other - its the people.

DanielBMarkham2• 15 years ago

We run into this situation in programming quite a bit. The answer is refactoring: taking the principles and patterns that work, streamlining them, ditching the cruft that works for a few edge cases but mostly gets in the way, simplifying the underlying metaphor of the system, and re-asssembling.

The only way to do this with much larger systems, such as systems of governance, is revolution or exploring new lands. Personally I'm a bit concerned about revolution -- the assumption with refactoring is that the people refactoring understand what the "good" parts are and what the "bad" parts are. They also need to be able to generalize and simplify in order to keep the system cognitively approachable. In my experience, it's very easy to be angry-tear-down-the-system guy, very difficult to actually refactor. As the author points out, it's not that these systems are entirely useless. The hell of the thing is that the reasons for creating these systems are still very valid.

I remain convinced that programmers (hackers) have a lot to add to the discussion when talking about complex, brittle systems. After all, we spend a lot of time both working with them and fixing them. To me, programming and systems architecture is applied philosophy. Very cool stuff.

jacques_chester15• 15 years ago

Refactoring is what political types call 'reform'. Reformations are safer than Revolutions, because revolutions dissolve all civil structure. They allow the ruthless sociopath to prosper.

What programming has taught me is humility in the face of complexity. Refactoring only works because we can, at a stroke, reverse our changes; because we can place those changes in isolation and test the heck out of them.

But society, constitutions, laws: they are not software systems. A revolution can't be rolled back. A constitutional change is a hotfix applied to a running system. It's important to be careful.

You might find reading Burke on the French Revolution interesting; also Oakeshott on conservatism (the intellectual kind, not the god-n-guns kind).

DanielBMarkham2• 15 years ago

I think we completely agree. I did not mean to imply anything but humility was indicated here.

"Reform" is a big, fuzzy word. A lot more brittleness and complexity has been added under that rubric. Systems of governance eventually get "over learned", that is, people adapt to the system and use it in more ways than it was intended. The ability of hundreds of millions of people to adapt is much greater than any amount of complexity that we can create. Yet we persist in the idea that by continuing to "tweak" the system, somehow magic will come from it. To me this is where humility comes in. We can never create a perfect government. The best we can do is have one that balances stability and refactoring.

Election cycles and constitutional amendment processes were supposed to do this. What we've found, though, is that legal precedent, statutory code, and an expansion of pure democracy and the role of career politicians, given enough time, puts us exactly where we didn't want to go.

Interesting side note: overly complex and brittle systems that people execute actually give dictatorial powers to those responsible for executing them. That is, in a computer system, the complexity creates program crashes and dysfunction, because the computer has to treat each instruction cycle and piece of data the same. In a system executed by people, a subtle social goal takes over: things still appear to happen. The system still superficially looks like it is working somewhat -- those in authority just selectively apply and bend the rules depending on whatever their whims are. If you've ever worked in a large organization with too much process, you've easily observed this: most struggle under burdensome rules, never being able to get anything done. A few, however, work completely outside the system, given special permission. These folks are usually kept off the radar so as not to upset the troops. Complex people systems don't ever stop working, they just make things miserable on most everybody and give a few permission to do whatever they want.

You'd think that modern tyrannies would look like old Libya: one guy in a funny hat ruling totally. But that's not the way it works any more. It's not one guy, it's a distributed oligarchy, and it's not out in the open, it's obfuscated.

aconbere• 15 years ago

There was a book written in the 80's that covered the overarching philosophical discussion on this kind of thinking about games. The author broke games into two categories: finite and infinite.

Finite games like the one invented by the children described by the OP have end states, they have winners, and the goal within them are always framed from that perspective. While infinite games have no winners, have no end, and the goals are often framed around ensuring that the game never ends.

It's easy to think of finite games in our lives, they are everywhere in our society, and the OP does a good job of pointing out some of the less obvious cases. Infinite games are less obvious, the one that I found most illuminating was "language", a game where people actively collude to extend the game to the end of time, and has no winner.

Anyay, it's an interesting read, but a little bit mumbo-jumbo by the middle of the book. I would recommend it.

http://www.amazon.com/Finite-Infinite-Games-Vision-Possibili...

stretchwithme2• 15 years ago

Seems most of the things mentioned are an inevitable result of centralizing and expanding government power. Someone is being gamed alright.

Jun82• 15 years ago

Very interesting post!

However, I got hooked on thinking about the childhood game that he mentions briefly in the introduction. Asking more and more questions about an object and wanting all of them to be consistent is a good description of mathematics. According to Godel, both teams are bound to lose, because you cannot create a system of descriptions/properties about a system that are self-consistent (as their number increases), there will always be questions to a team whose answer will be inconsistent with the previous set of questions. The game is then to see which team can push the inevitable further.

IDEABOLT: It would be interesting to develop a program that plays this game. Each answer could be stored as an RDF statement in database.

One_adm12• 15 years ago

Great post. I face this everyday in my job at an "old media" company. There are policies, rules and constraints in place which served some purpose at some point in time, and haven't been challenged for years. Unlearning the rules of established games is almost as important as evolving them. Everything should be questioned and challenged to find the root of the point of the rule/law/restriction and if it doesn't make sense any longer, throw it out.

useflyer• 15 years ago

Have you noticed that yesterday every comment was concise, and now that applications are open, every comment is elaborate, long-winded, and footnoted?

Greedy_Fools4• 15 years ago

People are predators and that's what predators do, never stop searching for what they need. Predatory behavior is the problem, and there is little will in successful predators to outlaw there own behavior.

thewisedude• 15 years ago

Doesn't "Exploiting the System" capture the same idea? I think basically what people are doing here is understanding the core weakness that is intrinsic to the system(which probably is not perfect) and using(abusing) it to their advantage.

The "overlearning the game" problem

Related Stories