www.newscientist.com/article.ns?id=dn7915&feedId=online-news_rss091

many of the ones that are wrong are probably nonsense.  does this include studies that are done with agendas such as a certain university doing a study for someone who gives them large sums on money?

the places that do research for as seen on tv, and scientists and organizations that do studies with agendas or flaky evidence to begin with?

Quoted: So the paper that claims 50% of papers is wrong is probably wrong? Not surprising, most of these researchers have a bias against his Noodly Goodness anyway.They are just trumping up studies to deny his plate place as the true creator of trees, mountains, midges (I meant midgets, but I guess midges works also) and other stuff. www.venganza.org/images/jollyfishth.jpg

I am wearing my FSM t-shirt right now.  

Umm....that's the PURPOSE of publishing scientific papers:  so that peer review can disprove the ones that are wrong.  Something the anti-science creationist types never seem to appreciate.

Actually, the point of the article is both CORRECT and IDIOTIC.


It is correct, because it is true that many studies are not properly designed, use incorrect or inappropriate statistical analysis, samples that are too small, etc.  

However, it is also worth pointing out that there are great differences in the QUALITY of journals.  If I have a very strong empirical paper in my field, I will send it to the top journals, and it may get published.  If I know the paper has some flaws, I will send it to the lesser quality journals.  Every field has a range of journal quality, but everyone knows what the "good" journals are.


However, the point is also compeltely IDIOTIC because scientifica studies are almost never definitively "right" or "wrong" - in that the statisitcal analytical methods used are inherently probabilisitc.  When we test a hypothesis, we do so with an a priori level of probability (of type 1 and type 2 error).  This (alpha) level is usually set at 0.05 - in the social sciences at least - meaning that if I "find" something, I found it with a 95% chance of it being a "real" effect, and a 5% chance of it being a statistical artifact.  By the same token, if I fail to find the effect, it could actually be "real", but I was just unlucky and fell into the 5% crack.  This actually becomes a more complicatied discussion (beyond statistical type 1 and 2 error), since the POWER of analysis varies with sample size and analysis type, which affects error rates, and the ability to detect various effect sizes.

Right now, for example - I am finishing up a study that involves 140 subjects, but in 33 groups.  Since the analysis is at the team level, my analysis has quite low power, meaning that it is difficult to detect "real" effects.  One of my analyses is looking for interactions between two constructs, and this makes the analysis more difficult, because it eats up degrees of freedom, further limiting my abililty to find the effect, even if it is "real".  However, with a sample of 330 groups, if the effect is "really" there, I could easily find it.  (By the same token, if this was an individual-level analysis, not a team-level one, there would be a lot more statistical power available).


But absolutely, there is a lot of CRAP research out there, and given the incentives of the tenure-system, there is probably a fair amount of grey-area unethical reporting of results.

The need for citeable fodder for grants, the need to pad one's C.V. for the tenure committee and the requirement of a certain number of peer-reviewed manuscripts for grad students leads to alot of papers published based on the LPU (least publishable unit).