Forensic science often hits the headlines for the wrong reasons. It’s interesting that what makes a headline is often a very different kind of story to the often unseen story of the underlying and systemic challenges that forensic science is facing. We need to be considering these unseen challenges that arguably have an even greater impact than the stories behind the headlines. We need to address interpreting what forensic science evidence means reliably transparently and reproducibly.
Misinterpreted evidence doesn’t get air time because it’s a difficult story to tell. We are used to seeing committed scientists with cutting edge technology following the evidence and closing cases in barely 40 minutes (give or take the ad breaks). In real life, the technological developments have been incredible and we can identify and classify forensic materials more accurately and quicker than ever before. That is all good and we need to keep those developments and innovations happening to make sure that forensic science is equipped for the future challenges that are just around the corner. But we need to be doing that at the same time as addressing misinterpreted evidence.
Misinterpreted evidence is often difficult to identify, difficult to demonstrate, and difficult to quantify. It’s often not tangible, and acknowledgement of it is reserved for the few isolated cases where miscarriages of justice have been shown (often many years later) to have happened. We have seen that there is a ‘dark (or hidden) figure of crime’, that refers to the amount of crime that is unreported or undiscovered and so does not make it into the official crime statistics. In forensic science we are facing a ‘dark figure of misinterpreted evidence’.
A study of ours that came out earlier this year demonstrated for the first time the number of cases at the Court of Appeal where ‘criminal evidence’ was misinterpreted in the original trial. Of the nearly 1000 cases between 2010-2016, misinterpretation could be identified in 22% of the cases. But these are only the cases where the misinterpretation was identified and the appeal case was successful. We suggest in that paper that the cases that were identified in this study are only ‘the tip of the iceberg’. We are left with a ‘dark figure’ of cases where we don’t know if the evidence was misinterpreted, and we don’t know how many past, current and future cases this may affect. At the moment, this study offers a glimpse at the tip of an iceberg, but the true size and scale of that iceberg of misinterpreted evidence is under the water line, and it isn’t being acknowledged because we can’t really see it.
This month a study was published that aimed to examine the sources of variability in the interpretation of mixed DNA profiles (something that is far more common now because of the increased capabilities and sensitivities of the DNA technology). One part of the study asked 108 different labs in the US to consider a series of mixed DNA profiles. One of the profiles was a complex 4 person mixture that was provided to the lab along with 3 reference samples. Two of the reference samples came from people who were included in the 4 person mixture, but the third reference sample had nothing to do with the mixed profile. The study found that 74 out of the 108 labs incorrectly included the third reference sample (the person who wasn’t in the mixture) as a contributor to the mixed sample. Only 7 labs rightly excluded that person from the mixture. But even within those 7 labs there were different reasons given for the accurate exclusion. The authors concluded from their findings that interpretation of complex samples is variable across different labs. Misinterpreted evidence is therefore happening, it’s often unseen, and it’s even happening within the gold standard field of DNA analysis.
We need to prioritise understanding this ‘dark figure’ of misinterpreted evidence. To do that we need to create an environment where research that is addressing this question is valued and enabled. We also need to be looking outside of the traditional fields in forensic science. There are insights that we need from economics, medicine, health economics, psychology, behavioural science, and we need to be engaging with other industries, especially in the creative space. We need to be truly innovative and creative. We need to re-think forensic science and how we go about addressing the challenge of misinterpretation. We need to look more broadly and more widely than anyone has before. The challenge is a big one, but as we’ve seen throughout history, the greatest challenges produce the greatest ingenuity.
