The Data/Human Goal Gap

http://ken.arneson.name/2016/06/the-data-human-goal-gap

by NaOH • 10 years ago

45 11 10 years ago

ken.arneson.name

pm• 10 years ago

I imagine this is a corollary of the phrase, "you manage what you measure."

Usually the metrics we measure are an indirect substitute for what we actually want to measure, e.g., lines of code as a measure of programmer productivity. What inevitably happens is that behaviour is optimised for the metric, and not what the metric was intending to measure. It's really just gaming the system.

yummyfajitas• 10 years ago

Tl;dr if you don't know what your goals are, you won't achieve them. Shocking!

Though in half his examples (education, politics, baseball), the complaint actually seems to be "other people don't share my goals".

What this has to do with measurement, I have no idea. Would this problem occur less if we non-quantitatively tried to achieve the wrong goal?

xivzgrev• 10 years ago

the complaint actually seems to be "other people don't share my goals

I guess. But i think what he is driving at is, in data driven world, we can lose sight of bigger picture ie in quest for test scores we can unreasonably stress kids out, in quest for profits we can shit on employees and the environment, etc.

Just taking the last point, people generally like parks and nature. What person would say "oh yes id love to pollute my closest river?"

but then suppose you ran a factory on a river, and it would cost your company 90% of profits to dispose of your factory's waste in an environmentally sound way, and you are measured on how much profits you bring in. Or you can dump in the river for free.

Would you choose to dump your waste into the river? I think most people would.

prawn• 10 years ago

I'd like to think that most people wouldn't. I'd rather run no business at all.

gipp• 10 years ago

He's right, it _does_ already have a name. A few, in fact.

https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Campbell%27s_law https://en.wikipedia.org/wiki/Reification_(fallacy)

projektfu• 10 years ago

I think Goodhart's law is closest to what he is getting at. There is also the aphorism of Deming, that the most important figures for managing your business are unknown and unknowable. We want to measure successful learning. Instead, we measure test scores. Whatever figures actually measure successful learning are unknown and likely unknowable. "Management", in this case the parents, teachers and administration, are focused on test scores and other marks of achievement. College admissions departments have their own measures that may include all of these things. But, in the end, hitting high values for these targets may not be reaching the actual goal. It's a good way of looking at an age-old problem.

rajanchandi• 10 years ago

One solution to this problem is making "flipped classroom" a mandatory at all the schools.i.e. students go thru lectures/videos at home and do homework collaboratively in the Classroom. The Real world is more collaborative compared to our schools. Flipped Classroom is proven to increase scores as well. Question: What efforts are you willing to put in making this happen?

dyarosla• 10 years ago

I haven't seen any sources showing that flipped classrooms were proven to increase scores- what studies are you referring to?

Edit: grammar

douche• 10 years ago

The Real world is more collaborative compared to our schools

People keep saying this. I must be living in a different shard of the real world than those people. Collaboration is nice, but at the end of the day, the work has to actually be done, not just talked about.

jeffbr13• 10 years ago

Distinguishing between work requiring a heads-down approach and work that can be sped up by collaboration (shared understanding, parallelisable tasks) needs to be learned and school's a good a place to get started on that as any.

michaelfeathers• 10 years ago

I used to call them 'Prosthetic Goals' http://michaelfeathers.typepad.com/michael_feathers_blog/200...

The Data/Human Goal Gap

Related Stories