The Excel Depression

http://www.nytimes.com/2013/04/19/opinion/krugman-the-excel-depression.html?smid=tw-NytimesKrugman&seid=auto&_r=0

by nate_martin • 13 years ago

95 68 13 years ago

FelixP• 13 years ago

This is a bit misleading since the Excel error in question merely overstated the magnitude of the impact of high debt, not the direction.

podperson5• 13 years ago

Yes, the direction was created by ignoring swathes of contrary data and arbitrarily weighting the data that was included. The paper was unworthy of publication in a credible journal. Just correcting the latter eliminates their result, while the former would probably suggest the reverse effect.

Lazare• 13 years ago

I'm not sure where you're getting this, but that's absolutely untrue.

As luck would have it, their coding errors mostly cancelled each other out; fixing them changed their conclusion from growth of -0.1% in high-debt countries to 0.2%; an insignificant change.

As for the weighting chosen, Reinhart and Rogoff chose a fairly standard weighting which - as it turns out - has some flaws. In particular it makes their conclusion highly sensitive to the experience of NZ (which had terrible growth during the brief period it had high debt), which is - admittedly - problematic. But their chief critic suggests an unusual weighting which is subject to the precise same flaws except in reverse. By weighting each year equally, Herndon's conclusions are dominated by the results of Greece (which ran debts for year after year will little result). Than sensitivity is also problematic, and the inherent correlation of debt loads in sequential years raises questions about the meaningfulness of their results.

Professional economists are now debating the question of the best weighting, but if anything Reinhart and Rogoff seem to have the better argument so far.

In short: Their errors did not eliminate their result. Even changing weighting did not eliminate the result (much less reverse it - that would mean more debt leads to higher growth; a simply absurd result!). And in any case, the method they chose has strong arguments in its favour.

Or for real TL;DR: Everything you said was wrong.

teyc• 13 years ago

The figure was negative 0.1 percent (contraction) vs 2.2 percent (growth). Furthermore, Reinhart and Rogoff were careful to not link correlation with causation. Neither should we.

pseut• 13 years ago

They weren't universally careful -- they fudged the difference in other settings.

mbetter3• 13 years ago

"Impact" implies causation. Low growth, for any reason, would lead to a higher ratio of debt to GDP. The original (erroneous) results of this study suggested that there was a point at which the relationship between the two changed.

olivier1664• 13 years ago

The excel error: Coding Error. As Herndon-Ash-Pollin puts it: "A coding error in the RR working spreadsheet entirely excludes five countries, Australia, Austria, Belgium, Canada, and Denmark, from the analysis. [Reinhart-Rogoff] averaged cells in lines 30 to 44 instead of lines 30 to 49.

http://www.nextnewdeal.net/rortybomb/researchers-finally-rep...

_account• 13 years ago

Why not the 'Arithmetic Error Depression' in place of 'Excel'?

To accurate/concise for Krugman? Must be half a dozen articles about this I've seen now and all have titles suggesting Excel made the error.

podperson5• 13 years ago

Again, it's not arithmetic that's the cause.

rjempson2• 13 years ago

Nothing sucks in readers like a bit of Microsoft Schadenfreude. The may as well have implied the electricity powering the computers was to blame.

Anechoic• 13 years ago

To accurate/concise for Krugman?

Chances are Krugman didn't write the headline.

gtani• 13 years ago

I thought it was kind of well known: don't use Excel for floating point math, statistical analysis, anything moderately involved that would have to be reproducible and auditable

http://www.pages.drexel.edu/~bdm25/excel2007.pdf

http://support.microsoft.com/kb/78113

http://en.wikipedia.org/wiki/Numeric_precision_in_Microsoft_...

podperson5• 13 years ago

The errors in question has nothing to do with inherent flaws in Excel (beyond, perhaps, the difficulty of reviewing Excel code). The problems were in decisions made to exclude data, arbitrary "analysis" with no basis in statistics or common sense (19 - IIRC - years of data from England were given the same weight as one year of data from New Zealand), and an egregious formula error that simply ignored several rows of data.

mbetter3• 13 years ago

I would suggest that in a situation like this, "the difficulty of reviewing Excel code" is even overstated. The direct dependencies of any cell are discoverable with trivial effort.

tadfisher• 13 years ago

To be fair, Excel's concept of a weighted average has no basis in statistics or common sense, as I have seen in a couple of published works.

Here's a tip: if your results are reproducible only in Excel, something is terribly wrong.

Toenex• 13 years ago

beyond, perhaps, the difficulty of reviewing Excel code

This has always been one of my concerns when Excel is used much beyond 'electronic squared paper'. The interwoven data and code nature of the tool makes testing and verification more difficult - certainly not impossible but going against the grain of the tool. Excel is used by a diverse range of people, many of whom have limited software skills and could do with a tool that enforced the right behaviour.

Also I think this shows the need for openness when people make such important claims based on data. Particularly in this case where all the data and calculations could easily be zipped up and put on a website.

achal• 13 years ago

From what I can tell, the actual issue in this case was that they simply summed the wrong thing, rather than ran into some floating point error.

Also, I haven't gone through the first paper, but that second link just seems to show how they generally conform to IEEE 754, except for three cases (div/0 errors instead of infinity, NaN errors immediately instead of allowing further computations, and [this one is a bit more important] they don't implement denormalized numbers). I'm not sure why that would mean Excel is terrible at floating point math.

jabbernotty• 13 years ago

It is not well known. I'm guessing that what you are saying isn't really the reality, based on what little I know of the oil and gas industry.

nandemo• 13 years ago

The financial industry uses a lot of Excel too.

klipt• 13 years ago

I don't know if it's still the case, but when I was an undergrad a professor pointed out that Excel's Gaussian tail probabilities disagreed with everything else (Mathematica, Matlab, R, etc.) for large values.

mbetter3• 13 years ago

Your citations don't even remotely suggest your conclusion.

tlb• 13 years ago

GDP is in $/yr, while debt is in $. Thus, debt/GDP has units of time. 90% really means 10.8 months.

One of the reasons the idea was sticky is that 90% seemed like a significant number, almost all the way to 100%. Where there's nothing so obviously magical about 10 or 12 months.

Someone• 13 years ago

For an analogy, look at individual households. Many of them have over a year of debt, aka over 100%. Getting a mortgage of 3 or 4 times one's yearly income isn't that hard.

Now, people will argue that, for many, the value of their house will be higher. That is true, but the same applies to governments. Infrastructure such as roads has value.

Traditionally, in many countries, the state also owned airports, water pipes, and electrical wiring that brought in money. That, I think, is where the real problem lies for some countries: they have sold the stuff that brings in money without bringing down their debt.

gd12• 13 years ago

GDP isn't your income, tax receipts are. Tax receipts as a percentage of GDP are about ~25%, so 100% debt/GDP is 4 years worth of debt, not 1.

humanrebar• 13 years ago

If the U.S. national debt was a mortgage, it would be a mortgage with a term that spans centuries with payments that don't even cover the interest.

"Getting a mortgage of 3 or 4 times one's yearly income..." One of lessons of the last ten years is that people, businesses, and governments can very easily get loans they have no ability to repay. And too often, the borrowers have to default. This is bad for the borrower.

rjempson2• 13 years ago

Nice headline, but I can't see what this has to do with Excel.

riffraff• 13 years ago

among the issues in the original analysis there appear to be a messed up excel formula (they excluded some rows from the calculation)

kfk2• 13 years ago

Please, bear in mind that Krugman has a very opinionated view against austerity (he seems to be going the Keynes way these days) and this article is not about Excel. He just wants to show how right he is and how sloppy the austerity guys are.

The way to compete against Excel is actually learning to love the tool (it took me 2 years, but I almost can say I love it... sometimes).

EDIT Since I am on HN, I always try to have a serious discussion about Excel and business software. That's why I always try to tie this kind of topic to my general idea: that we need to understand very well the type of work people do in Excel before proposing any solution.

But... if we want to talk about austerity, let's talk.

What is the real problem with anti-austerity measures today? We could accept a GDP/Debt ratio of 200, 300, 1000, infinity, and let people live their lives, right? Well, hell no. For 1 simple reason: interests. And I am not talking about the obvious problem with interests (that you have to pay them), but the bit that everybody forgets: they are a bet against the future that growth will be high enough to pay them. And that's the problem. We have a low growth rate expectancy (we western countries). And that's why serious economists (meaning: not Krugman) are telling us that increasing our debt exposure is the path to future ruin.

Now, you will say, but anti-austerity financed with higher taxes will bring growth! If that were true, we would have had serious growth the last 50 years (a period within which tax ratio on GDP increased pretty much everywere). The truth is that historically public spending has brought much less growth (if any) than private spending. Meaning: for each $ that we take away from privates to the public we are deliberately reducing present and future growth. And why public spending brings less (if any) growth? Well, simply because the market is so complex that we need multiple agents. Giving too many resources to 1 agent (the State) is like betting most of our money on 1 black jack game, does it make sense? No. You need to spread risk and resources, you need to let people make decisions and you need to let people keep most of what they produce.

dnc• 13 years ago

I have the same impression that the Excel error is just abused so he can reiterate his attack against austerity proponents. The paper in question just fails to provide evidence that 90% is a tipping point of debt to GDP ratio. It does not provide evidence that there is no such a point whatsoever.

igravious3• 13 years ago

"" some correlation between high debt and slow growth, with no indication of which is causing which, but no sign at all of that 90 percent “threshold. ""

We aren't told how high high is. We don't now which direction it goes in. This debt ratio thing needs to be knocked on the head. Everybody without the correct information will assume that 100% = very very bad whereas it only means a year's GDP. I say only but you get my drift. If the consensus tomorrow was to base the debt ratio off of 3 years GDP then 90% (~10 months) would go down to 30% wouldn't it? 30% doesn't sound that bad! Hey we've only got a 30% debt ratio, let's party!

woodpanel• 13 years ago

He is basically doing the same thing He accuses r&r for: using a chosen snippet of reality, adding his conclusions as facts to support his broader world view.

igravious3• 13 years ago

"" Finally, Ms. Reinhart and Mr. Rogoff allowed researchers at the University of Massachusetts to look at their original spreadsheet — and the mystery of the irreproducible results was solved. First, they omitted some data; second, they used unusual and highly questionable statistical procedures; and finally, yes, they made an Excel coding error. Correct these oddities and errors, and you get what other researchers have found: some correlation between high debt and slow growth, with no indication of which is causing which, but no sign at all of that 90 percent “threshold.” ""

Krugman has a very opinionated -->> "other researchers have found"

not about Excel -->> "finally, yes, they made an Excel coding error"

way to compete against Excel is actually learning to love the tool -->> Irrelevant to the topic at hand really. What might be more relevant is news like this[1]: suicides and murders are dramatically up in Greece in the wake of the austerity measures. Choice quote, "Suicide and murder rates climbed from 2007 to 2009 particularly among men, and unusual outbreaks of malaria, West Nile virus and HIV took clinicians by surprise, said the findings in the American Journal of Public Health." Austerity measures in Europe are having very real and tangible consequences, the vagaries of Excel-wrangling are neither here nor there. Granted the NY Times article title is arguably link-baity and sensationalist. One wonders do R/R worry about how their shoddy research has been parlayed into needless suffering?

[1] http://www.thejournal.ie/suicides-murders-rise-as-greek-aust...

pc862• 13 years ago

So you're saying that austerity measures are causing malaria and HIV? Really?

Correlation != causation as I'm sure everyone knows.

fulafel5• 13 years ago

He does have a very polemical writing style but of the New Keynesian vs Chicago school (or saltvater vs freshwater) camps his side does come out of the post-2008 crisis looking better, if you look at the predictions.

Pro-austerity camp have been on pretty weak ice recently, with the IMF admitting the big multiplier and European states (eg. Britain) doing so poorly with austerity. And now this.

And yeah, he's not really blaming Excel, just ridiculing the programming error lightly. Hw could be saying Word if it turned out some key wording had been autocorrected to an unintended meaning.

Edit: your amended discussion about austerity goes ad-hominem against Krugman ("not serious"?, he's got a Nobel, and is one of the most respected & prolific guys in the field) and is based on counterfactual claims (absence of "serious growth in last 50 years", see eg. http://en.wikipedia.org/wiki/List_of_regions_by_past_GDP_%28...)

gd12• 13 years ago

Exactly what kind of "austerity" do you think is happening in Britain? Government spending has gone up. Saying something doesn't make it true.

JPKab2• 13 years ago

The Keynesians were never wrong. The issue is that the rich countries, especially the US, have never followed a Keynesian policy of minimal to no deficit during periods of private sector growth, They only follow Keynesian philosophy when they are in recession and they want to run high deficits then. Rather than acknowledge this, the austerity people scream out that Keynes is flawed, when in reality its the democratic tendency towards giving people low taxes and lots of services that is the problem.

salvadors• 13 years ago

OTOH, Latvia and Estonia have both done very well with austerity.

wazoox• 13 years ago

The problem is the nature of the creditors. Basically, creditors are proxies for "the rich and the powerful", and the social programs are "benefits for the poor and the powerless". If we simply decided to call for a jubilee and a nullification of all national debts what would happen? The "rich and powerful" would be instantly much less rich and powerful, and everybody else would probably be a lot better.

jbooth2• 13 years ago

The view against austerity was mainstream until very recently. Counter-cyclical spending was a bipartisan point of agreement from the 40s until the year 2009. You never heard anyone accuse each other of being "keynesian" or "hayekian", it was just econ 101 that counter-cyclical policy made sense.

Now that we're out of the woods on the economy, we should in fact be trying fix the debt problem -- but without hurting the economy too much in the process. Shrinking the deficit while shrinking GDP doesn't really do much for you.

MaysonL• 13 years ago

And of course the US budget deficit is falling faster at present than it ever has.

pasbesoin• 13 years ago

Excel is a great tool. Having used it professionally for a number of years, in the midst of other financial professionals, I can say with a somewhat well-qualified opinion that the majority of people using it should not be. Or rather, they should be treated as if they were using a Word processor, which is what Excel essentially is for them: A tablified "Word processor" with a built-in calculator.

Who knows what they typed in, nor how they tied it together? For work of any complexity -- no, really, not infrequently even for the apparently simplest of work -- if the result is of any importance, you had better get the original workbook file and check it for yourself.

Excel can be used, very well and very efficiently, for enormously complex work. But... you have to know what you're doing. And that takes not only time but significant will and effort to achieve -- something that even many professionals do not have -- have not demonstrated, in my experience.

Construct calculations in ways that create robustness. Build in cross checks and alternate paths of calculation to double check your work. Audit your own work and perform manual calculations to verify. Essentially, you've built a software product and you need to test it.

You want to know what the results of "programming for the masses" initiatives might look like when applied to society? Well, Excel is one (if not necessarily the best) extant example. It's not all roses.

P.S. We insist that grade school students "show their work" when doing math. How should this not apply to professional economists?

james10713• 13 years ago

The reason is that the paper was not peer reviewed.

gordaco• 13 years ago

"So the Reinhart-Rogoff fiasco needs to be seen in the broader context of austerity mania: the obviously intense desire of policy makers, politicians and pundits across the Western world to turn their backs on the unemployed and instead use the economic crisis as an excuse to slash social programs."

Amen. After reading any piece of news, you'd think that the only utility economists have is to (rather poorly) justify decisions that are actually fully ideological in nature, using numbers whose origin is uncertain.

YokoZar• 13 years ago

I think Krugman himself has noted that economists are referenced most on things they disagree on (such as the relation between deficits and growth), and referenced least on things they agree on (such as the effects of rent control).

temphn• 13 years ago

Question: has Paul Krugman made a practice of releasing his raw data? Can anyone point me to the database of tab-delimited text files behind the econometric papers he's written, or to any systematic attempts to replicate them? If not, should be fun to try to get a dataset out of him. Most published macroeconomics research won't stand up to attempts to replicate it (whether "pro" or "anti" austerity), if the great replication experiment in psychology is any guide:

http://psychfiledrawer.org

Basic kinds of corrections like weighting nations by population will often invalidate macroeconomic correlations.

pessimizer• 13 years ago

Couldn't you answer this question yourself by using the Internet? A combination of Google and other, more academic services might give you a list of papers, and by reading them, you might find datasets that are mentioned, but do not seem to be public. Then you could return to HN to ask a more specific question, or email Krugman yourself for the answers you seek. That should be fun, too.

jbooth2• 13 years ago

Fair enough, but that's way less effective as rhetoric. How do we know Krugman didn't falsify his results and murder people?

vacri• 13 years ago

I don't really understand how this paper could be claimed to have 'destroyed the economies of the world' when it's in the aftermath of the GFC. 'Not made it better', sure, but bad fiscal practices causing deep financial problems were there before 2010.

pekk• 13 years ago

That argument would be based on the policies that the paper served to justify.

The Excel Depression

Related Stories