“Truth is mighty and will prevail. There is nothing the matter with this, except that it ain’t so.”
– Mark Twain

Professor Damodarn has written up a great post about academia.  He readily admits that academia is not the “truth seeking” profession that is idealized in movies and television (and indeed in academia!).  Instead, he gives six points about the realities of academia (which I must say that I broadly agree with after being around the block once or twice).  I paraphrase here (for brevity):

1. Research what will get you published, not necessarily what is interesting.
2. Bias is inherent in any human endeavor and drives not only the research direction, but also the how they experiments are setup and analyzed.
3. Brand matters.  Who you are, your school, your mentor, your research topic can significantly affect your chances of publication.
4. Every discipline has an “established” view.  If you follow it, your paper will have an easier time being published.  The inverse is also true.
5. Peer review is widely variable and sometimes biased.
6. Data abuse happens.

Sounds pretty bleak, doesn’t it?  It’s almost as if you shouldn’t blindly trust any single paper that has come out of academia.  And that’s exactly what you should be doing!  Sort of.  Although he gives such a cynical outlook, he still strongly believes in peer review and empirical testing.  And being the pragmatic, straight talking wise man that he is, Professor Damodaran also provides some advice for laymen:

1. Don’t assume that academics don’t have an agenda and don’t play politics. They do.
2. Don’t let “research findings” sway you too much – for every conclusive result in one direction, there is almost always just as conclusive a result in the opposite one.
3. Just because something has been published does not make it the truth. Conversely, the failure to publish does not mean that a paper is unworthy.
4. Develop your own vision of the world before you start reading papers in an area. Take what you find to be interesting and provocative and abandon the fluff (and there is plenty in the typical published paper).
5. Learn statistics. It is amazing how much of what you see reported as the truth fails the “standard error” test.

All sage advice.  One important point in defense of academics though is that most work is incremental.  Break-throughs don’t happen that often, especially not every publication.  I suspect that almost every “break-through” we read about in the news is only one step in a promising new direction.  And only that — a step.  No one paper is going to have the exact cure for Alzheimer’s disease or produce cold fusion.  Most likely the results will have to be reproduced, built upon and engineered (I couldn’t leave this part out) before we realize the impact and value of the publication.  And that’s the beauty of it.  Millions of researchers are taking millions of steps in different directions trying to expand human knowledge.  Most of it will be insignificant in the grand scheme of things, but a few will change the world forever.  And of course along the way, there will always be people who cheat, as well as people who do great work, but (scientific) progress is made.  Instead of being cynical I like to think about it more like the Carpenters: “we’ve only just begun”.