Which evidence do you trust?

Claim: [GROUP] cannot be trusted; their data is tainted by [TAINT].

I’ve templated this argument because I’ve seen it used a lot.

Not all evidence is trustworthy. The question is: what evidence do you trust? The easiest case is empirical evidence: if you’ve tested something firsthand, that is obviously very compelling (though not *always* accurate – scientists have been fooled by stage magicians, of course).

Once you accept evidence from other sources, you have to trust that the source is not lying and has not been deceived. Things to support this include lack of conflict of interest, track record, and peer review.

A conflict of interest generally means that the study is compromised by being paid for in such a way that rewards a particular outcome and penalises other outcomes, but it can also be the result of a moral stance that has a particular desirable outcome. In general, the scientific method tries to overcome this by ensuring at least that funding is not dependent on getting particular results, but mostly through peer review.

Track record needs to be examined closely to be of any use – previous fraud or repeated/recent poor quality results are the best indications of poor data, but certainly don’t *prove* that any current results are incorrect. Again, peer review is vital.

Peer review can make up for some deficiencies in conflict of interest and track record. The scientific method relies heavily on peer review, and (in my opinion) rightly so. However, it is not a guarantee of quality data. The peers themselves must not have a conflict of interest and must have a good track record.

I will almost always accept peer-reviewed scientific data as valid, and the more a study has withstood scrutiny, the more confident I am of it.

What other standards of evidence do you think should be acceptable? I suppose the lower the stakes, the more I will accept anecdotal data.

2 Responses a “Which evidence do you trust?”

  1. Marco Parigi Says:

    As with the definition of “lying” – I believe what is most important is what is happening inside the author’s head. It doesn’t matter whether they quote actual facts by actual scientists, what matters is their *Intent*. If they intend the reader to believe something that doesn’t have a sound basis in reality, they are a liar as bad as any other and should be ignored. It doesn’t matter how noble the end is, a scientist should stand up and say that there is motive, opportunity and circumstantial evidence that they are doing this, regardless if the conclusion is in agreement with what they believe.

    To me there is considerable evidence that these three things are in abundance in Al Gores articles and especially the inconvenient truth “documentary”. This is not a minor detail about style, but about separating the science from political activism. Indeed, I would say that also about the “Fielding” link you have also.

  2. Kate Says:

    You’ll have seen this article in NS earlier this year – it’s highly relevant (and explains so many of my interactions online over the years!).


Leave a comment!

  • Pages

  • Archives

  • Categories