Which evidence do you trust?
Claim: [GROUP] cannot be trusted; their data is tainted by [TAINT].
I’ve templated this argument because I’ve seen it used a lot.
Not all evidence is trustworthy. The question is: what evidence do you trust? The easiest case is empirical evidence: if you’ve tested something firsthand, that is obviously very compelling (though not *always* accurate – scientists have been fooled by stage magicians, of course).
Once you accept evidence from other sources, you have to trust that the source is not lying and has not been deceived. Things to support this include lack of conflict of interest, track record, and peer review.
A conflict of interest generally means that the study is compromised by being paid for in such a way that rewards a particular outcome and penalises other outcomes, but it can also be the result of a moral stance that has a particular desirable outcome. In general, the scientific method tries to overcome this by ensuring at least that funding is not dependent on getting particular results, but mostly through peer review.
Track record needs to be examined closely to be of any use – previous fraud or repeated/recent poor quality results are the best indications of poor data, but certainly don’t *prove* that any current results are incorrect. Again, peer review is vital.
Peer review can make up for some deficiencies in conflict of interest and track record. The scientific method relies heavily on peer review, and (in my opinion) rightly so. However, it is not a guarantee of quality data. The peers themselves must not have a conflict of interest and must have a good track record.
I will almost always accept peer-reviewed scientific data as valid, and the more a study has withstood scrutiny, the more confident I am of it.
What other standards of evidence do you think should be acceptable? I suppose the lower the stakes, the more I will accept anecdotal data.