Wednesday, 8 July 2009

The title "Bad science and bad research"

on Jane's, How Publishing Really Works, jumped out at me.
As a child I took a little time to work out what "7 out of 10 dentists" as opposed to "7 out of every 10 dentists" really meant. Once I had sussed it out I started to look for similar examples. There were plenty. I will not bore you with them.
Then I discovered statistics in a different way. I was expected to use the damn things at university. I was taught about 'bell curves' and 'chi-square' and ANOVA and then ANCOVA and all sorts of other fancy arithmetical playthings beloved of psychologists to try and show that something they have thought of can be 'proven' correct.
I nearly failed my doctoral thesis because I made a small change to the way in which a 'standard' diagnostic test was applied. In doing so I succeeded (quite by accident) in bringing down the entire house of cards relating to that body of work. I was not popular.
I can remember sitting in the basement canteen of a certain psychology department and, behind me, hearing the professor of statistics and the department's statistical expert discussing a problem and the words, "If you use 'x' test rather than 'y' then you will get a significant result."
I nearly quit university at that point. Perhaps I should have.
Since then I have tended to treat all 'research' with some scepticism. It makes life uncomfortable. There is no certainty any more. When there is a 'breakthrough' I merely wonder what is going on and hope that there might be a grain of truth in it rather than a new way to sell another drug that has side effects worse than the disease. When the polls of 484 voters show massive support for one political party over another I wonder how many people will be caught up in 'well I might as well vote for them. They are going to win anyway" mode.
We should teach logic in schools. We should train the young to apply it to what they hear and read. In about fifty years from now we might have a few people who could really think for themselves. Research suggests that changing beliefs and opinions will be much harder to do.

5 comments:

Jane Smith said...

"We should teach logic in schools. We should train the young to apply it to what they hear and read. In about fifty years from now we might have a few people who could really think for themselves."

Yes. Absolutely. Beautifully written, Cat.

catdownunder said...

Thankyou Jane. You were the inspiration for my rant.

john said...

My father taught me to 'believe half of what you see and nothing of what you hear'. I tell my children that the most important word in the English language is 'why'. My daugher's response of "Why, Daddy?", far from hoisting me with my own petard, led to an interesting discussion which supported the claim.

Children must be encouraged in the home to simply question things, mustn't they?

catdownunder said...

Which is why I think that family mealtimes, among other things, are so important! We asked so many questions at mealtimes that I sometimes wonder anything got eaten. :-)

john said...

I'll second that. "Give me a child till he is seven, and I'll give you the man". Someone said that - Jesuits I think. Hard to disagree. Or to exaggerate the importance of the family in all this.