http://therealnews.com/...emid=74&jumival=10099

well not exactly, but they certainly pwnd a the research paper:P

everybody gets an A+ this semester?

R&R made mistakes that masters students shouldn't be making, let alone professors. They have no defense for their actions.

Well, aside from the billions they have probably been paid under the table by Koch and other elite austerity supporters.

But aside from that money, they're very ashamed.

Wow...that is a serious debunking....makes you wonder what other flawed studies have a huge impact on policies...

*cough* climate change *cough*

It is common sense that money going to fund interest payments on the public debt is completely unproductive and harms growth. There may also be a tipping point where up to a certain point harm to growth is limited but then leads to economic disaster. You cannot argue that debt did not harm the Greek economy.

@Klown: the point is that under a sustained low interest rate environmnet there appears to be little impact around a 90% debt:gdp ratio and the the benefits of borrowing might outweigh the loss due to interest rate payments (to foreign companies).
Greece's debt:GDP ratio isn't anywhere near 90%.

No one is going to argue that a huge debt:gdp ratio is either sustainable nor a good idea.

The real point here is that one should be a lot more careful with data (or at worst one should not manipulate data) to achieve a desired result.

think the real point is that Everybody Makes Mistakes. take the time to check their math, they might've typo'd or are getting completely senile.

That was no mistake...

^

Just from what they did with the averaging, you would have to assume that either they sucked massive fluff to get their Ph.D.s because they were total morons and would have trouble getting a job at McDonalds, or they did that on purpose to further their agenda.

'Sugar Association's intent to use science to defeat critics uncovered by dentist'

http://www.cbc.ca/...f-vp-crowe-big-sugar.html

HERNDON: There was actually a few errors. The one that's received a lot of the media attention has been the spreadsheet error where they average 15 out of the 20 countries instead of 20 out of the 20 countries. Additionally, we found that there was kind of an unconventional way to make the average. Instead of looking at all 96 observations for their over-90 category and then taking the average with those 96 observations, they first averaged by country and then averaged the country averages. This has the effect, though, of making countries with very little experience over 90 weighted higher than countries who have spent, for example, two decades over 90





My jaw hit the ground when i read this, lol

Anyone who has worked in acedamia would know that something published is first subject to pier review. Having studied with statistiticians (professors/grad students), they will say (anecdotally) that many scientific articles are riddles with statistical errors due to lack of expertese in that subject from the reviewers, although this is improving.
However for economics, that type of thing is very much part of their field so those types of errors are somewhat less excusable and should probably have been caught at the pier review level.

The moral is, if you are going to base important decisions on something, it better be something that has been reasonably well scrutinized. A single error is understandable, it happens to everyone. A paper riddled with mathematical errors from a field that uses a lot of mathematics is quite something else.

@Darkna: To be fair, in my line of work, that method of averaging does make sense in some situations. However to me it doesn't make sense in that one.

The perils of averaging averages is something you learn approximately the middle of the second week of every undergrad's first course in statistics. However, if R&R did their math right, they wouldn't achieve a result that would get them paid by the austerity advocates. To that end, I think these "errors" were a feature, not a bug.

@tellarion:
certain corporations are extremely guilty of doing this, especially when it comes to product safety and some pharmaceuticals.
Other industries are the other extreme (insurance industry when it comes to vehicle and product safety) but unfortunately can't share their findings because they would get sued into oblivion.

And martian: Even peer reviewed magazines can get in the trap of not performing a gage R&R on every submission. Unless it's Journal of the Royal Statistical Society SerB, my guess would actually be whoever did the "peer reviewing" got caught wanting to get their scotch early, checking only the conclusions -- and assuming the data worked.

Averaging averages is very common in education data too, especially K-12... i.e. averaging student performance data at a class level, then averaging the classes in that grade, then averaging the grades for school, then averaging the schools for the district, then averaging districts to get state performance then averaging states to get some "national" performance number...

It doesn't always happen that way, but quality data is very hard to find in K-12. Partially because of the protection of student individual records, often the raw data isn't available to higher levels and the "averages" may be all they have to work with.

...and martian... you were making me shudder each time you said "pier review"... We don't need the local dock-workers union reviewing our economic studies...

pier/peer.. pfft. I like to peer at the peer on the pier as they appear?:P (why does appear have an a when "peer" doesn't?, pear is something quite tasty:P )

competing currencies ftw

anybody who has ever reviewed anything knows that the data and analyses are never replicated by the reviewer, so these errors will never ever be caught at the peer review level...it's up to people to replicate afterward and for the authors to post their data online, but unfortunately academia is structured such that nobody has any incentive to do either of those things.

also, my entire dissertation is on the perils of averages and why/how we should estimate individual effects :P

@highrock: I think it depends on the field and the journal.
In my field they almost always are. If it cannot be replicated it's not considered valid.
For my job, it's acutally a *requirement* of some of my work both in terms of professional guidance and in certain instances, a matter of regulation.
Even when I say "simulated" there are cases where that must be exactly replicable (ie use the same random seed to get the same results).

Accounting is even worse for this and there are days when I hate all accountants.. but that's off topic:P

I feel thos conversation leads into Academic Integrity. We are starting to see death of independant research by Universitites and more research funded by corporations wanting a specific outcome..

AS always, follow the money.

I couldn't agree more. All kinds of flawed studies trying to show climate change isn't real.

-Fooglmog
Guy with no clue.

@martian

ah i didn't realize that, but i can say for sure in economics (since i am in a related field), replication is definitely not done during peer review at all. reviewers do not get access to data and most journals don't even require the data to be made available ever.

more like all kinds of flawed studies period.

are you in fenance?

on a related note, you also get cases where the results are replicable and valid but obviously the result of data mining and picking and choosing variables. big problem in fields like empirical corporate finance.

as for reinhard and rogoff, they maintain their results are still valid, havent followed this too closely due to vacation / general lack of knowledge about whats being published in macro. some of the discussion on ejmr: http://www.econjobrumors.com/...-who-caught-out-the-profs http://www.econjobrumors.com/...einhart-coding-error-ouch

actually the methodology and data manipulation is generally scrutinized during peer review for most of the more reputable economics journals.

It is common for subsequent articles to be published that debunk the methodology/results of previous publications. This is generally how the academic process works (for all fields I'm pretty sure?).

So I don't know why it is considered such a big deal that this particular article/result has been debunked/scrutinized through testing in subsequent studies. This is very much the status quo.

the reason why it got so much attention is significant public policy was made on the basis on this paper being correct and there is political capital to be gained by this happening.

:P

well sure, but the fact that governments can "hit the wall" with debt issues only stands to reason and we have actually seen cases of this in the past (and a few close calls, including Canada in the early 1990's).

As such "austerity" is still reasonable given this and given all the other negative implications of running structural deficits (unless all of us consider it "morally just" to economically enslave our descendants by forcing them to pay off the debts we run up to live beyond our means? Because I don't consider that moral at all).

The only real implication of debunking the original result is that they cannot say with any sort of certainty exactly WHAT level of indebtedness, on average, causes a government to "hit the wall". <-- this was the major implication of the Reinhard and Rogoff paper.

It is highly likely this "threshold level of debt" that causes governments to hit the wall varies and is highly dependant on the specifics on the economy involved and more importantly, the market's expectations/perceptions regarding the government/economy in question's ability to manage and service said debt.

err really? i dont think so? the identification strategy is talked about but its not very common to check whether the data is real or whether the results can be reproduced from the data.

i learned about weighted averages just by checking the data. it was like Hey man, this crap don't add up, what the heck did i do wrong this time?

no statistics classes for me. i could've been an accountant, if i couldn't program the computer to do that boring repetitious stuff.

I have never tried to get anything published, but I can provide a personal example that demonstrates my point:

Last summer one of my co-workers, whom I also went to grad school with, begged me to complete a quick turnaround request for him at work because he was busy preparing additional information on how a particular figure was calculated in a paper that he and his thesis superviser were trying to publish together.

That information request came from the person reviewing the paper for the journal.

Thus such scrutiny/inspection does occur. It very likely depends on the meticulousness of the individual conducting the review, but this is the case for any field and in every field you get lazy reviewers who don't do their due diligence and you get keener reviewers who do. But you can't make a blanket statement that it doesn't occur, because it very much does.

Also I can't recall reading any empirical paper ever where it didn't clearly lay out the methodology and the steps taken to achieve the results.

They do this specifically for the purpose of allowing people to replicate the methodology later, either to scrutinize it or do apply the approach to a different question/different date etc.

The only thing supporting austerity is actually a morality play.. there's zero examples of its long-term success if data isn't cherry-picked. That quadruple-dip recession in England shows austerity there just isn't working out.. Spain's unemployment is 60%.. This is the new version of the old morality play that was quoted by Hoover at the beginning of the Great Depression -- "liquidate labor, liquidate stocks, liquidate farmers" etc.

If the theory is Conventional Wisdom? It sucks. If theory doesn't explain reality.. time for the theory to go. The Euro is an ongoing object lesson in precisely what happens when idiots attempt to shoehorn erroneous economic theory into the real world: The unfounded theory gets smacked upside the head with a clue-by-4.



Not true at all. Default happens all the time, and doesn't even prevent the next issuance of debt. There's more than zero examples of countries defaulting and still getting their next issuance purchased.


Why is unverifiable assertion enough to believe that austerity is good.. but at the same time it's just so tough to believe that a country could default and still get its next issuance purchased?

Here's the link to the paper debunking R&R:
http://www.peri.umass.edu/...aa6388b1/publication/566/
The fundamental conclusion of the people who corrected R&R's work for them? no difference in growth rates between countries with low debt/GDP and those with high.

Unless the fundamental goal of the people implementing austerity as a policy is to rape that country's natural resources for the benefit of a few hundred people in financial firms in other countries [aka treat nations as colonies] then austerity is a failed policy.

Until you post counter-factuals you have proven nothing. To claim that austerity isn't working because hey, those who have done it are still in bad shape, is not rational. You have to show through simulations achieved using reasonable assumptions that under a scenario where austerity wasn't taken, things would be better.

Without the counter-factual you have proven nothing other than that recessions are bad and are hard to recover from, well no fluff.



Where did I ever claim it leads to non-issuance of debt? Since when did "hitting the wall" mean that? You are making stuff up now... and it is quite ridiculous, almost as ridiculous as your implied claim that defaulting on debt is somehow not as bad as what we are seeing in Spain, yeah... ok then, those currency crises coupled with banking crises are a lot easier to handle than just a banking crisis....

As for who has defaulted before, wikipedia has a list list of them for you, looks like at least a couple hundred cases to me.
http://en.wikipedia.org/...ist_of_sovereign_defaults

wow, i read half of this and now my head hurts

i clicked the thread hoping the word austerity would be explained, boy was i wrong!

it's just a side effect of socialism.

fluffing loser freak show!!!! HAHAHHA WHAT A LOSER FREAAKAZOID!!!!


FREAK

RR 2010 clearly explain their methodology and data sources (all public sources afaik)... so no, I dont think it's common. I agree with you that it's not unheard of and you definitely get a lot of papers that replicate past studies with different results, but normally as long as the methodology as explained is sound it doesn't get scrutinized that much. Journals will ask for detail on certain figures etc. especially ones that look fishy but when you clearly explain what data you use and what methodology you use with that data, generally it just gets accepted that your results are correct (methodology/identificaiton strategy does get questioned like I said earlier)