I read a couple of troubling articles today about some forensic techniques that were used by the FBI that were used as evidence in criminal cases and were sold by the FBI as reliable techniques, but as it turns out were not the case. Those articles can be found here, and here.
In a horrible fit of madness I looked at some of the comments, and of course there are plenty of conspiracy people there, but what was more interesting is how many people thought that this was confirmation that scientific consensus doesn’t mean anything, or how science is unreliable, and many of these people were clearly conservatives who are climate change deniers. It annoys me to see science and logic so misunderstood, so I thought I just write down a few thoughts.
First of all it’s important to remember that one case of science being misused is not evidence that all science can’t be trusted.
Second, this is not a case of science being misused. The science was correct the entire time, it’s the FBI that lied about the science. Whether it was the forensics people themselves who misrepresented the science or lead investigator in charge of the case is unclear, but it was actually objective scientific investigation that showed the corruption of the FBI. There were actually peer-reviewed publications that demonstrated the lack of reliability of these techniques, just as there are 1000’s and 1000’s of peer-reviewed journal articles that establish the truth of human-induced climate change and this is much different than someone having the truth in a journal article, but then lying about it in terms of how that knowledge is applied.
Third, you could call what the FBI is doing bad science, but you can also see how easily that bad science was uncovered by the recent investigation. However when it comes to climate change, even their own scientists agree about the evidence for human-induced climate change. The party just refuses to listen.
More careful investigation of this exposure of the FBI’s techniques is not an indictment of science, but rather something that reveals it’s value at uncovering bias.
It is a common message from atheists that ultimately faith and science are incompatible. Science forms conclusions based on evidence, and faith forms conclusions despite evidence. In various debates I’ve had with people of ‘faith’ it occurs to me that there might be a slight problem with this statement. While it’s true that people who hold strong religious beliefs often do not bother trying to explain evidence that is contradictory to their views. Sometimes they will simply rationalize contradictory evidence away as not being accurate, or say some blanket statement “well science doesn’t know everything” or “science isn’t always right”. These always seem like strange arguments of course. Science doesn’t claim to know everything, and while it is true that science isn’t always right, but in proving a current hypothesis or theory wrong the bonus is that you end up with something that is more correct than what you had before. And of course it’s true to say that faith isn’t always right either. If faith was unfailing in the results it provides, I would certainly be willing to submit that perhaps scientific investigation wasn’t always the way to go.
But I would like to look just beyond religion and talk more about belief in general. People who hold strong beliefs whether it is about religion, their country, or even their sports team (okay being a little glib there about sports) all share something in common. Not only do they ignore evidence to the contrary, they also seem to have a different idea than most scientists about what evidence actually is.
As a scientist it feels fairly obvious to me what I should count as evidence and what I should not. In the so called “hard” sciences this is fairly easy to grasp. As an atmospheric scientist I have often dealt with vast quantities of data, measured from some unfeeling
instrument. It’s unlikely to have any personal bias. Of course instruments are not perfect and they have errors. As a scientist I always need to be aware of the errors in the evidence. Then once I have those hopefully close to perfect measurements I must then analyze the data. My interpretation is thus subjective, perhaps to my own biases. I of course try to minimize my biases by being well aware of the body of knowledge that surrounds the particular problem I am trying to solve. I try to be aware of conflicting perspectives and points of view from previous scientist to help me keep as open a mind as possible about what my results might mean. But in the end, even in a field that is steeped in physical equations, I might present results that are biased. Most likely I have just made a mistake, but bias is also a mistake. 🙂
So any one study can be biased, but ultimately my mistakes will be revealed by others who are using my conclusions as truth to their study. If I am wrong, their study will fail, and someone will say, perhaps my conclusions were incorrect thus their experiment started with a faulty premise. An important part of science is verification and repeatability. When there is a lot of disagreement amongst researchers, this probably means we don’t understand the problem yet very well.
In the social sciences, disagreements are more frequent, because here data and valid evidence are harder to obtain because so much depends on effective sampling and dealing with imperfect forms of subject material (namely us!). For instance let’s say I wanted to see how humans felt about death, and I only sampled people in the U.S. who were of a Christian faith. Meanwhile somewhere in India somebody wanted to perform the same study, but only sampled Hindus. Any conclusion either of us made about how humans feel about death is incorrect. Because really all we’ve addressed is how Hindus feel about death, or how Christians feel about death. At best maybe we could draw some conclusions about how Americans or Indians felt about death. Someone looking at our studies would say, well there are a few similarities in the our study so perhaps together you are one step closer to answering your original question, but for the most part you have only illuminated cultural difference between Indians and Americans on how they view death. And if we were responsible scientists we would at least admit at the end of our study that our sampling was biased and thus we can only make limited claims about what humans think about death.
So ultimately any good scientist, regardless of the field, is aware of the errors and uncertainty of their data, and the scope to which their evidence is able to support their
hypothesis. And it is through the body of evidence in a particular field that we can turn hypotheses into theories and be confident in our conclusions. We have a peer-review process to evaluate our application of the scientific method, and we have the process of verification and repeatability to strengthen the findings of any one scientist.
What one should gather from this is that science is actually really hard. It takes a long time to learn how to interpret results accurately, be familiar with the types of errors that can be made, and to understand to what degree of certainty you can attribute to your research. Nevertheless no other system has found to work better.
In my next post, I will get back to my original idea, which is what many seem to view as evidence and how much certainty they associate with that evidence.