Home Overview Press Room Blog Publications For Students about us

All That Glitters Isn’t Gold

by Osagie K. Obasogie and Troy DusterThe Hastings Center Report
October 12th, 2011

The increasing use of DNA evidence has revolutionized criminal investigations. Over the past several years, DNA forensics—once thought to be a less reliable identifier than other forensic techniques, such as latent fingerprinting—have now become the evidentiary gold standard in criminal prosecutions. At the same time, non-DNA-based forensic techniques that have incarcerated thousands are coming under fire.

The policy implications of this shifting dynamic—what Michael Lynch and colleagues call an “inversion of credibility”[1]—can be most clearly seen in the National Research Council’s 2009 report, Strengthening Forensic Science in the United States: A Path Forward. Conducted at Congress’s request by a highly esteemed committee, this report—over three hundred pages—assesses the current state of forensic science.

The committee found remarkable shortcomings in what they call the forensic science knowledge base, noting that the scientific theories and methods used to substantiate many forensic claims frequently cannot withstand close scrutiny. They found an alarmingly “wide variability in capacity, oversight, staffing,
certification, and accreditation.”[2] For example, lack of transparency, susceptibility to bias, and questionable methodologies for friction ridge analyses (analyses of the prints left by fingers, palms, or soles) make it difficult for two analysts to come to the same conclusion.[3] The report’s sobering message is that many forensic applications simply lack scientific rigor despite their routine use in legal proceedings.

Although the committee acknowledges that DNA forensics are not always perfect, the report and its recommendations are framed by an implied yet powerful claim: non-DNA forensic techniques should live up to the gold standard created by DNA typing. But this framing has its own serious drawbacks that obscure much deeper issues concerning both technical matters related to the scientific validity of extending basic DNA identification techniques to novel applications and the ethical, legal, and social implications of DNA forensics’ expanding uses.

DNA Exceptionalism?

The NRC report is replete with both general and specific declarations that frame the accuracy and reliability of DNA typing as the new standard for forensic
investigations. At the broad end of the spectrum, the committee notes “DNA typing is now universally recognized as the standard against which many other
forensic individualization techniques are judged.”[4] This sentiment also shapes the analysis of specific techniques. For example, they note that “overall, the process for toolmark and firearms comparison lacks the specificity of the protocols for, say, 13 STR [short tandem repeat] DNAanalysis.” [5]

Few seriously doubt DNA typing’s high reliability in determining whether any two isolated samples match. Yet DNA typing is only one of many ways in which DNA analyses are used in forensic investigations. For over two decades, state and federal governments have been collecting convicted felons’ genetic profiles and depositing them into databases in order to be able to identify repeat offenders who leave biological samples at crime scenes.

DNA databases give rise to techniques beyond mere DNA typing that expand criminal investigations’ scope and impact. This repository’s growth—the federal database itself is almost at eight million profiles [6]—has given rise to three related techniques: cold hits, partial matches, and familial searches.

“Cold hits” occur when investigators are able to match unknown biological materials left at a crime scene with a known database profile. Partial matches occur when investigators identify a suspect using fewer than thirteen loci—the standard number of chromosome sites where, if identical, a “match” between two profiles can be declared. Familial searches work from the premise that relatives share many identical loci. While the number of shared markers between an unknown suspect and a database hit might not incriminate the person with the known profile, it can and has pointed to a relative, who is then the subject of a criminal investigation.

Many assume that these database-oriented techniques have the same precision as typing two individual samples. But what often gets obscured—as it does in the NRC report—is that these newer uses of DNA forensics share many of the same shortcomings that the NRC identifies with non-DNA forensic techniques.

Partial Matches and the Scientific Knowledge Base

DNA databases gain much of their authority from the oft-repeated claim that the chance that two profiles will randomly match—even partially—is only one in several million. It has been argued that a nine-locus match can uniquely
identify perpetrators,[7] and individuals have been convicted on such partial-match evidence.

However, increasing evidence suggests that the way scientists have calculated the probability for random matches may not be accurate. Data obtained from the Arizona state DNA database showed that out of 65,493 profiles, 122 pairs matched at nine loci, twenty pairs matched at ten, and two pairs (siblings) matched at eleven and twelve loci.[8] Findings from the Illinois state database yielded similar results: 903 pairs matched at nine or more loci out of a total of more than 200,000 profiles. Data from the Maryland database, with 30,000 profiles, was also surprising: thirty-two pairs matched at nine loci, and three matched at all thirteen.[9]

How could so many profiles randomly match at so many loci? No one knows for sure, which is why scientists and legal scholars are calling for more access to research government databases.[10] The FBI and several states have thus far refused to comply.

Establishing Standards

The NRC report is particularly concerned with forensic science’s lack of standardization; the field has few established protocols on how to accurately describe the significance of such evidence. While this a demonstrable problem
with non-DNA forensics, it is also true for DNA evidence related to database searches.

For example, with “cold-hit” evidence, where investigators run an unknown sample against thousands of database profiles in search of a “hit,” any match is only as significant as the statistical probability that it might be coincidental. Yet there is no agreed-upon standard, much less a gold standard, for how to calculate this statistic or how to present this evidence in court.

The seemingly compelling one-in-a-million statistic that we often hear associated with cold-hit matches uses a population figure as a referent—the
likelihood that an unknown profile matches a suspect (identified for other reasons) purely by coincidence. However, cold-hit matches that occur within databases have a substantially higher probability of being coincidental; when searching through large databases with millions of profiles (like the federal database), there are millions of chances for coincidental matches.[11] Transposing the statistical significance of the former approach to the latter
is what is often referred to as the prosecutor’s fallacy and has been known to impact determinations of individuals’ guilt or innocence.[12]

The substantially diminished probabilities stemming from cold-hit database searches that take database size into account more accurately reflect the statistical limitations of this approach. This has led an NRC committee[13] and an FBI advisory board[14] to recommend making these database limitations part of the calculation. Yet neither recommendation has been widely implemented by authorities. While the committee briefly points to these prior recommendations, they do not fully discuss how the absence of consistently enforced standards, procedures, or oversight can lead courts to receive misleading information that can undermine DNA forensics’ credibility as much as a failure to standardize non-DNA forensics affects its integrity.

Questions of Justice

While DNA typing itself is far from infallible, database-oriented DNA forensics raise a profound series of second-order questions for entire groups, in contrast to the NRC report’s singular focus on individuals. Much of this stems from the unique ways in which the criminal justice system interacts with blacks and Latinos, particularly through policies regarding DNA databases. For example, the overpolicing of minority communities, along with related injustices, has led to these groups’ dramatic overrepresentation in prisons; 30 percent of black males will be convicted of a felony at least once compared to 5 percent of whites, and an adult black male is eight times more likely to be incarcerated.[15] Aggressive public policies encouraging sample collections for almost any contact with law enforcement is leading to an alarming statistic: although blacks represent only 13 percent of the population, they make up an estimated 40 percent of federal DNA database profiles.[16]

Given the disparate composition of DNA databases, techniques like familial searching raise significant questions regarding systemic bias. Leveraging the shared genetic variants and short tandem repeat lengths between relatives to
find suspects will have a much larger impact on blacks and Latinos. What does it mean for government to turn people with existing stored profiles into “genetic informants” on their relatives without their knowledge or consent and with few safeguards to prevent wrongful convictions from errant cold hits? What are the ethical, political, and legal implications of placing a population under a lifetime of genetic surveillance in which each DNA fragment shared with a banked
relative is screened against future crime scene evidence? And is it just for these significant civil liberties concerns to disproportionately fall upon groups already unfairly burdened by injustices linked to what we know to be selective patterns of law enforcement?[17]

Worth Its Weight in Gold?

In addition to these concerns, several other technical issues and questionable practices with unresolved social and ethical implications also plague DNA forensics. On the technical side, contaminated samples can lead to false positives and false negatives, clerical errors can lead to incorrectly logged samples and poor data entry, and crime labs can misinterpret old, small, or mixed samples from multiple individuals. These and other technical issues are linked to broader social and ethical issues, such as varying practices regarding the destruction of samples after law enforcement has analyzed them, the propriety of using forensic samples for nonforensic purposes like research, the ethics of surreptitious sample acquisition by police, and a host of other privacy issues connected to the general idea of the government storing sensitive genetic information indefinitely.

Public policy regarding the use and expansion of DNA
forensics is evolving as quickly as the technologies themselves. For example, a California law that went into effect in 2009 allows authorities to take and retain DNA samples from individuals merely arrested—not charged or convicted—
for felonies.[18] And New York enacted a law in late 2009 allowing authorities to use DNA database partial matches to identify suspects.[19] This highlights a current trend, whereby law is being used to radically expand DNA databases to include larger, unsuspecting portions of the population.

The NRC report and its recommendations represent an important first step to putting the scientific method’s rigor into forensics so that justice can prevail. If we are to take the report as seriously as it deserves, then the critiques it raises for non-DNA forensic applications must also be applied to the growing spectrum of DNA forensics. Like our prior uncritical acceptance of latent fingerprint technology, the new proverbial gold standard might, on closer inspection, have
far more tarnish then we have been led to believe.

1. M. Lynch, S.A. Cole, R. McNally, and K. Jordan, Truth Machine:
The Contentious History of DNA Fingerprinting (Chicago, Ill.: University
of Chicago Press, 2008).
2. National Research Council, Committee on Identifying the Needs
of the Forensic Sciences Community, Strengthening Forensic Science in
the United States: A Path Forward (Washington, D.C.: National Academies
Press, 2009), 14.
3. Ibid., 136-45.
4. Ibid., 130.
5. Ibid., 155.
6. Federal Bureau of Investigation, http://www.fbi.gov/about-us/
7. W.C. Thompson, “The Potential for Error in Forensic DNA
Testing (and How That Complicates the Use of DNA Databases for
Criminal Identification),” August 12, 2008, paper for the Council for
Responsible Genetics’ national conference, Forensic DNA Databases
and Race: Issues, Abuses and Actions, available at http://www.councilforresponsiblegenetics.
8. J. Jefferson, “Cold Hits Meet Cold Facts: Are DNA Matches Infallible?”
Transcript Magazine 40, no. 1 (2008): 29-33.
9. L. Geddes, “Unreliable Evidence? Time to Open Up DNA Databases,”
New Scientist no. 2742 (January 6, 2010): 8-9, http://www.
10. D.E. Krane et al., “Time for DNA Disclosure,” Science 326
(2009): 1631-32.
11. Thompson, “The Potential for Error in Forensic DNA Testing.”
The NRC report briefly cites this issue in a footnote (chapter 3, footnote
12. J. Felch and M. Dolan, “DNA Matches Aren’t Always a Lock,”
Los Angeles Times, May 3, 2008.
13. National Research Council, The Evaluation of Forensic DNA Evidence.
Commission on DNA Forensic Science: An Update (Washington,
D.C.: National Academies Press, 1996).
14. FBI DNA Advisory Board, “Statistical and Population Genetics
Issues Affecting the Evaluation of the Frequency of Occurrence of
DNA Profiles Calculated From Pertinent Population Databases,” Forensic
Science Communications 2, no. 3 (2000), http://www2.fbi.gov/
15. D.H. Kaye and M.E. Smith, “DNA Identification Databases:
Legality, Legitimacy, and the Case for Population-Wide Coverage,”
Wisconsin Law Review 413 (2003): 413-59.
16. H.T. Greely et al., “Family Ties: The Use of DNA Offender Databases
to Catch Offender’s Kin,” Journal of Law, Medicine, and Ethics
34, no. 248 (2007): 248-62.
17. New York State Assembly Committees on Codes and on Corrections,
Testimony of Harry G. Levine Regarding Pending and Proposed
Legislation to Collect DNA from All People Convicted of a Misdemeanor
in New York State, and Also Regarding New York City’s Epidemic
of Marijuana Possession Arrests, Albany, N.Y., May 31, 2007, http://
18. The DNA Fingerprint, Unsolved Crime and Innocence Protection
Act (November 2004), California Proposition 69, available at
19. J.W. Peters, “New Rule Allows Use of Partial DNA Matches,”
New York Times, January 24, 2010.

This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of biotechnology and public policy issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information go to: http://www.copyright.gov/title17/92chap1.html#107. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.


home | overview | blog | publications| about us | donate | newsletter | press room | privacy policy

CGS • 1122 University Ave, Suite 100, Berkeley, CA 94702 • • (p) 1.510.665.7760 • (F) 1.510.665.8760