empirical legal research notes

Empirical Legal Research Resources

Statistics and Datasets
Journals and Law Reviews
Data Analytics Help and Resources at Stanford
Current Awareness and Training Resources
Bulk Download Availability

Ask us anything!

Law librarians and library staff are available to help you find your way around the library and its legal materials, provide research support, and assist you in locating materials that are not in our library. You can find the latest information about our hours and services on the library homepage . There are several ways to contact us:

[email protected]
Recommend a Resource

In Person : The Borrowing Services Desk and Reference Office are on the main floor of the law library. Online and in-person appointments with the reference staff can be made from the library homepage .

Borrowing Services Desk: (650) 723-2477 Reference Desk: (650) 725-0800

This guide introduces selective resources and tools for conducting empirical legal research. These resources include:

Statistics and Datasets: Resources available that collect statistical information
Journals and law reviews: Scholarly publications focused on empirical legal research
Treatises: Texts explaining and incorporating empirical legal research
Data Analytics Help and Resources available at Stanford
Bulk Download Availability: Resources that allow of mass download of information

For other research guides on this topic, see:

Georgetown Law Library-Statistics and Empirical Legal Studies Research Guide
The Maloney Library - Empirical Research
The University of Chicago Library - Empirical Legal Research: Data Sources and Repositories
Stanford Graduate School of Business Library - Data and Decisions
Next: Statistics and Datasets >>
Last Updated: Dec 15, 2023 1:06 PM
URL: https://guides.law.stanford.edu/empiricallegalresearch

Stanford Home
Maps & Directions
Search Stanford
Emergency Info
Terms of Use
Non-Discrimination
Accessibility

Harvard Empirical Legal Studies Series

5005 Wasserstein Hall (WCC) 1585 Massachusetts Avenue Cambridge, MA02138

Contact the Graduate Program

The Harvard Empirical Legal Studies (HELS) Series explores a range of empirical methods, both qualitative and quantitative, and their application in legal scholarship in different areas of the law. It is a platform for engaging with current empirical research, hearing from leading scholars working in a variety of fields, and developing ideas and empirical projects.

HELS is open to all students and scholars with an interest in empirical research. No prior background in empirical legal research is necessary. If you would like to join HELS and receive information about our sessions, please subscribe to our mailing list by completing the HELS mailing list form .

If you have any questions, do not hesitate to contact the current HELS coordinator, Tiran Bajgiran.

All times are provided in U.S. Eastern Time (UTC/GMT-0400).

Spring 2024 Sessions

Empire and the shaping of american constitutional law.

Aziz Rana, BC Law

Monday, Mar. 25, 12:15 PM Lewis 202

This talk will explore how US imperial practice has influenced the methods and boundaries of American constitutional study.

Historical Approaches to Neoliberal Legality

Quinn Slobodian, Boston University

Thursday, Mar. 28, 12:15 PM Lewis 202

Fall 2023 Sessions

On critical quantitative methods.

Hendrik Theine , WU, Vienna/Univ. of Pennsylvania Monday, Nov. 6, 12:30 PM Lewis 202

Economic inequality is a profound challenge in the United States. Both income and wealth inequality increased remarkably since the 1980s. This growing concentration of economic inequality creates real-world political and societal problems which are increasingly reflected by social science scholarship. Among those detriments is for instance the increasing economic and political power of the super-rich. The research at hand takes a new radical look at media discourses of economic inequality over four decades in various elite US newspapers by way of quantitative critical discourse analysis. It shows that up until recently, there was minimal media coverage of economic inequality, but interest has steadily increased since then. Initially, the focus was primarily on income inequality, but over time, it has expanded to encompass broader issues of inequality. Notably, the discourse on economic inequality is significantly influenced by party politics and elections. The study also highlights certain limitations in the discourse. Critiques of inequality tend to remain at a general level, discussing concepts like capitalist and racial inequality. There is relatively less focus on policy-related discussions, such as tax reform, or discussions centered around specific actors, like the wealthy and their charitable contributions.

Spring 2023 Sessions

How to conduct qualitative empirical legal scholarship.

Jessica Silbey , Professor of Law at Boston University Yanakakis Faculty Research Scholar

Friday, March 31, 12:30 PM WCC 3034

This session explores the benefits and some limitations of qualitative research methods to study intellectual property law. It compares quantitative research methods and the economic analysis of law in the same field as other kinds of empirical inquiry that are helpful in collaboration but limited in isolation. Creativity and innovation, the practices intellectual property law purports to regulate, are not amenable to quantification without identifying qualitative variables. The lessons from this session apply across fields of legal research.

Fall 2022 Sessions

How to read quantitative empirical legal scholarship.

Holger Spamann , Lawrence R. Grove Professor of Law

Friday, September 13, 12:30 PM WCC 3007

As legal scholars, what tools do we need to read critically and engage productively with quantitative empirical scholarship? In the first session of the 2022-2023 Harvard Empirical Legal Studies Series, Harvard Law School Professor Holger Spamann will compare and discuss different quantitative studies. This session will be a first approximation to be able to understand and eventually produce empirical legal scholarship. All students and scholars interested in empirical research are welcome and encouraged to attend.

How do People Learn from Not Being Caught? An Experimental Investigation of a “Non-Occurrence Bias”

Tom Zur , John M. Olin Fellow and SJD candidate, HLS

Friday, November 4, 2:00 PM WCC 3007

The law and economics literature on specific deterrence has long theorized that offenders rationally learn from being caught and sanctioned. This paper presents evidence from a randomized controlled trial showing that offenders learn differently when not being caught as compared to being caught, which we call a “non-occurrence bias.” This implies that the socially optimal level of investment in law enforcement should be lower than stipulated by rational choice theory, even on grounds of deterrence alone.

Empirical Legal Research: Using Data and Methodology to Craft a Research Agenda

Florencia Marotta-Wurgler , NYU Boxer Family Professor of Law Faculty Director, NYU Law in Buenos Aires

Monday, November 14, 12:30 PM Lewis 202

Using a series of examples, this discussion will focus on strategies to conduct empirical legal research and develop a robust research agenda. Topics will include creating a data set and leveraging to answer unexplored questions, developing meaningful methodologies to address legal questions, building on existing work to develop a robust research agenda, and engaging the process of automation and scaling up to develop large scale data sets using machine learning approaches.

Resources for Empirical Research

HLS Library Empirical Research Service
Harvard Institute for Quantitative Social Research (IQSS)
Harvard Committee on the Use of Human Subjects
Qualtrics Harvard
Harvard Kennedy School Behavioral Insights Group

Past HELS Sessions

Holger Spamann (Lawrence R. Grove Professor of Law) – How to Read Quantitative Empirical Legal Scholarship?

Katerina Linos (Professor of Law at UC Berkeley School of Law) – Qualitative Methods for Law Review Writing

Aziza Ahmed (Professor of Law at UC Irvine School of Law) – Risk and Rage: How Feminists Transformed the Law and Science of AIDS

Amy Kapczynski and Yochai Benkler –(Professor of Law at Yale; Professor of Law at Harvard) Law & Political Economy and the Question of Method

Jessica Silbey – (Boston University School of Law) Ethnography in Legal Scholarship

Roberto Tallarita – (Lecturer on Law, and Associate Director of the Program on Corporate Governance at Harvard) The Limits of Portfolio Primacy

Susan S. Silbey – (Leon and Anne Goldberg Professor of Humanities, Sociology and Anthropology at MIT) HELS with Susan Silbey: Analyzing Ethnographic Data and Producting New Theory

Cass R. Sunstein (University Professor at Harvard) – Optimal Sludge? The Price of Program Integrity

Scott L. Cummings (Professor of Legal Ethics and Professor of Law at UCLA School of Law) – The Making of Public Interest Lawyers

Elliot Ash (Assistant Professor of Law, Economics, and Data Science at ETH Zürich) – Gender Attitudes in the Judiciary: Evidence from U.S. Circuit Courts

Kathleen Thelen (Ford Professor of Political Science at MIT) – Employer Organization in the United States: Historical Legacies and the Long Shadow of the American Courts

Omer Kimhi (Associate Professor at Haifa University Law School) – Caught In a Circle of Debt – Consumer Bankruptcy Discharge and Its Aftereffects

Suresh Naidu (Professor in Economics and International and Public Affairs, Columbia School of International and Public Affairs) – Ideas Have Consequences: The Impact of Law and Economics on American Justice

Vardit Ravitsky (Full Professor at the Bioethics Program, School of Public Health, University of Montreal) – Empirical Bioethics: The Example of Research on Prenatal Testing

Johnnie Lotesta (Postdoctoral Democracy Fellow at the Ash Center for Democratic Governance and Innovation at the Harvard Kennedy School) – Opinion Crafting and the Making of U.S. Labor Law in the States

David Hagmann (Harvard Kennedy School) – The Agent-Selection Dilemma in Distributive Bargaining

Cass R. Sunstein (Harvard Law School) – Rear Visibility and Some Problems for Economic Analysis (with Particular Reference to Experience Goods)

Talia Gillis (Ph.D. Candidate and S.J.D. Candidate, Harvard Business School and Graduate School of Arts and Sciences and Harvard Law School) – False Dreams of Algorithmic Fairness: The Case of Credit Pricing

Tzachi Raz (Ph.D. Candidate in Economics at Harvard University) – There’s No Such Thing as Free Land: The Homestead Act and Economic Development

Crystal Yang (Harvard Law School) – Fear and the Safety Net: Evidence from Secure Communities

Adaner Usmani (Harvard Sociology) – The Origins of Mass Incarceration

Jim Greiner (Harvard Law School) – Randomized Control Trials in the Legal Profession

Talia Shiff (Postdoctoral Fellow, Weatherhead Center for International Affairs and Department of Sociology, Harvard University) – Legal Standards and Moral Worth in Frontline Decision-Making: Evaluations of Victimization in US Asylum Determinations

Francesca Gino (Harvard Business School) – Rebel Talent

Joscha Legewie (Department of Sociology, Harvard University) – The Effects of Policing on Educational Outcomes and Health of Minority Youth

Ryan D. Enos (Department of Government, Harvard University) – The Space Between Us: Social Geography and Politics

Katerina Linos (Berkeley Law, University of California) – How Technology Transforms Refugee Law

Roie Hauser (Visiting Researcher at the Program on Corporate Governance, Harvard Law School) – Term Length and the Role of Independent Directors in Acquisitions

Anina Schwarzenbach (Fellow, National Security Program, the Belfer Center for Science and International Affairs, Harvard Kennedy School) – A Challenge to Legitimacy: Effects of Stop-and-Search Police Contacts on Young People’s Relations with the Police

Cass R. Sunstein (Harvard Law School) – Willingness to Pay to Use Facebook, Twitter, Youtube, Instagram, Snapchat, and More: A National Survey

Netta Barak-Corren (Hebrew University of Jerusalem) – The War Within

James Greiner & Holger Spamann (Harvard Law School) – Panel: Why Does the Legal Profession Resist Rigorous Empiricism?

Mila Versteeg (University of Virginia School of Law) (with Adam Chilton) – Do Constitutional Rights Make a Difference?

Susan S. Silbey (MIT Department of Anthropology) (with Patricia Ewick) – The Common Place of Law

Holger Spamann (Harvard Law School) – Empirical Legal Studies: What They Are and How NOT to Do Them

Arevik Avedian (Harvard Law School) – How to Read an Empirical Paper in Law

James Greiner (Harvard Law School) – Randomized Experiments in the Law

Robert MacCoun (Stanford Law School) – Coping with Rapidly Changing Standards and Practices in the Empirical Sciences (including ELS)

Mario Small (Harvard Department of Sociology) – Qualitative Research in the Big Data Era

Adam Chilton (University of Chicago Law School) – Trade Openness and Antitrust Law

Jennifer Lerner (Harvard Kennedy School and Department of Psychology) – Anger in Legal Decision Making

Sarah Dryden-Peterson (Harvard Graduate School of Education) – Respect, Reciprocity, and Relationships in Interview-Based Research

Charles Wang (Harvard Business School) – Natural Experiments and Court Rulings

Guhan Subramanian (Harvard Law School) – Determining Fair Value

James Greiner (Harvard Law School) – Randomized Control Trials and the Impact of Legal Aid

Maya Sen (Harvard Kennedy School) – The Political Ideologies of Law Clerks and their Judges

Daria Roithmayr (University of Southern California Law School) – The Dynamics of Police Violence

Crystal Yang (Harvard Law School) – Empiricism in the Service of Criminal Law and Theory

Oren Bar-Gill (Harvard Law School) – Is Empirical Legal Studies Changing Law and Economics?

Elizabeth Linos (Harvard Kennedy School; VP, Head of Research and Evaluation, North America, Behavioral Insights Team) – Behavioral Law and Economics in Action: BIT, BIG, and the policymaking of choice architecture

Meira Levinson (Harvard School of Education) – Justice in Schools: Qualitative Sociological Research and Normative Ethics in Schools

Howell Jackson (HLS) – Cost-Benefit Analysis

Michael Heise (Cornell Law School) – Quantitative Research in Law: An Introductory Workshop

Susan Silbey (MIT) – Interviews: An Introductory Workshop

Kevin Quinn (UC Berkeley) – Quantifying Judicial Decisions

Holger Spamman (Harvard Law School) – Comparative Empirical Research

James Greiner (Harvard Law School) – Randomized Controlled Trials in the Research of Legal Problems

Michael Heise (Cornell Law School) – Quantitative Research in Law

James Greiner (Harvard Law School) – A Typology of Empirical Methods in Law

David Wilkins (Harvard Law School) – Mixed Methods Work and the Legal Profession

Tom Tyler (Yale Law School) – Fairness and Policing

Modal Gallery

Gallery block modal gallery.

The Oxford Handbook of Empirical Legal Research

Academics (Law)

Research output : Book/Report › Book

The early years of the first decade of the twenty-first century saw the emergence and rapid development of a movement that labeled itself "Empirical Legal Studies" (ELS). This book acknowledges the diversity of empirical investigation of law, legal systems, and other legal phenomena. In particular, there are at least three approaches and research groupings that predate the contemporary ELS movement, which may be respectively identified as socio-legal/law and society (an interdisciplinary movement with strong roots in sociology but including scholars from a wide range of traditional disciplines including law), empirically oriented law-and-economics, and judicial behavior/politics. This book also explores three key dimensions of policing: order management, crime management, and security management. Finally, it concludes by identifying some emerging trends in the organization and conduct of police work as policing organizations seek to reconfigure their capacities and capabilities to meet new challenges. The phrase "empirical legal research" in the title, The Oxford Handbook of Empirical Legal Research, is designed both to reflect and to celebrate the healthy pluralism of empirical approaches to the study of law and legal phenomena.

Bibliographical note

Els movement
Empirical investigation
Empirical legal studies
Legal systems

Publisher link

10.1093/oxfordhb/9780199542475.001.0001

Fingerprint

Law Keyphrases 100%
Empirical Legal Research Keyphrases 100%
Oxford Handbook Keyphrases 100%
Research Social Sciences 100%
Guides Social Sciences 100%
Legal Phenomenon Keyphrases 66%
Empirical Legal Studies Keyphrases 66%
Organizations Social Sciences 40%

T1 - The Oxford Handbook of Empirical Legal Research

AU - Cane, Peter

AU - Kritzer, Herbert M.

PY - 2012/9/18

Y1 - 2012/9/18

N2 - The early years of the first decade of the twenty-first century saw the emergence and rapid development of a movement that labeled itself "Empirical Legal Studies" (ELS). This book acknowledges the diversity of empirical investigation of law, legal systems, and other legal phenomena. In particular, there are at least three approaches and research groupings that predate the contemporary ELS movement, which may be respectively identified as socio-legal/law and society (an interdisciplinary movement with strong roots in sociology but including scholars from a wide range of traditional disciplines including law), empirically oriented law-and-economics, and judicial behavior/politics. This book also explores three key dimensions of policing: order management, crime management, and security management. Finally, it concludes by identifying some emerging trends in the organization and conduct of police work as policing organizations seek to reconfigure their capacities and capabilities to meet new challenges. The phrase "empirical legal research" in the title, The Oxford Handbook of Empirical Legal Research, is designed both to reflect and to celebrate the healthy pluralism of empirical approaches to the study of law and legal phenomena.

AB - The early years of the first decade of the twenty-first century saw the emergence and rapid development of a movement that labeled itself "Empirical Legal Studies" (ELS). This book acknowledges the diversity of empirical investigation of law, legal systems, and other legal phenomena. In particular, there are at least three approaches and research groupings that predate the contemporary ELS movement, which may be respectively identified as socio-legal/law and society (an interdisciplinary movement with strong roots in sociology but including scholars from a wide range of traditional disciplines including law), empirically oriented law-and-economics, and judicial behavior/politics. This book also explores three key dimensions of policing: order management, crime management, and security management. Finally, it concludes by identifying some emerging trends in the organization and conduct of police work as policing organizations seek to reconfigure their capacities and capabilities to meet new challenges. The phrase "empirical legal research" in the title, The Oxford Handbook of Empirical Legal Research, is designed both to reflect and to celebrate the healthy pluralism of empirical approaches to the study of law and legal phenomena.

KW - Els movement

KW - Empirical investigation

KW - Empirical legal studies

KW - Legal systems

KW - Policing

UR - http://www.scopus.com/inward/record.url?scp=84924243369&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84924243369&partnerID=8YFLogxK

U2 - 10.1093/oxfordhb/9780199542475.001.0001

DO - 10.1093/oxfordhb/9780199542475.001.0001

AN - SCOPUS:84924243369

SN - 9780199542475

BT - The Oxford Handbook of Empirical Legal Research

PB - Oxford University Press

CY - Oxford, England

Library electronic resources outage May 29th and 30th

Between 9:00 PM EST on Saturday, May 29th and 9:00 PM EST on Sunday, May 30th users will not be able to access resources through the Law Library’s Catalog, the Law Library’s Database List, the Law Library’s Frequently Used Databases List, or the Law Library’s Research Guides. Users can still access databases that require an individual user account (ex. Westlaw, LexisNexis, and Bloomberg Law), or databases listed on the Main Library’s A-Z Database List.

Georgetown Law Library
Research Process

Statistics and Empirical Legal Studies Research Guide

Data collection.

Getting Started
Research Design
Articles on Empirical Methodology in Law
Selected Books on Empirical Legal Research Methodology
Selected Books on Empirical Methodology in Other Disciplines
Searching for Empirical Studies on a Particular Topic
Research Data Archives and Specialized Search Engines
Courts and the Judiciary
Crime and Criminal Justice
Demographics
Economics, Finance and Trade
Elections and Public Opinion
Foreign and International Data
Government Enforcement, Regulation, and Spending
Health and Well-being
Statistics & Probability in Law
Data Analysis Tools

Key to Icons

Georgetown only
On Bloomberg
More Info (hover)
Preeminent Treatise

Data collection involves identifying the "population" about which you hope to make inferences, then figuring out what data you need based on your research questions, theories, and the observable implications of those theories. Sometimes you will need to collect or generate data from scratch, while other times you may be able to reuse existing data. If you cannot reuse existing data, you will need to make a plan for collecting or generating new data in an unbiased manner.

Identifying the Target Population

Before you collect any data, you should define your target population. The target population, or population of interest, is the population about which we want to make inferences. The population consists of all the units (people, cases, countries - whatever we are studying) about which we would collect data if our resources were unlimited, and depends on our research question. Epstein & Martin 64-65 (2014). A population may be delimited by geographic factors (e.g., people in Texas), social factors (e.g., law students) and timeframe factors (e.g., all the Texas law students who took at least one class between 2012 and 2018), among others. Ryan 155 (2015). Defining your target population is essential to planning your data collection process, because you need to collect data from a representative sample of the population about which you want to make inferences.

Locating, Gathering or Generating Data

Once you define your target population, you should be able to determine what data you need based on that and the measures you adopted during the research design process. Epstein & Martin 65 (2014). Before you attempt to gather or generate your own data, you should look into whether someone else has already produced data that you can reuse. See the Existing Datasets and Statistics portion of this guide for more information. If you decide to rely on existing data, you should first be sure you understand the process that was used to generate or gather it, including what population it comes from. Epstein & Martin 69. If you have checked for existing data sources and determined that they either do not exist or are of inadequate quality, you will have to gather or generate your own data.

In empirical legal research, there are four ways researchers commonly generate their own data: performing experiments, surveying, observing, and analyzing text. Epstein & Martin 70 (2014). True experiments are rare, especially in legal research. A true experiment requires both the random selection of units from the studied population, and the random assignment of those units to either a control group or a treatment group, something that is difficult to do in a legal setting. Id . at 71. Surveys are more common, and involve asking people about their attitudes, opinions, or behavior. However, people selected for a survey may not always respond truthfully, accurately, or at all. Id . at 74-75. Observation involves watching your units of study as they engage in real-world activities. Some observational studies will involve subjects who know they are being observed and may alter their behavior accordingly. All observational studies require researchers to document the observed behavior, possibly introducing bias into the process, as when the researcher notes behavior that supports his or her hypothesis and fails to note behavior that doesn't. Id . at 80-81. Analyzing text can involve the simple extraction of facts (such as the race of a suspect described in a criminal probable cause affidavit) or the more complex and subjective extraction of "sentiment" (such as determining the ideology of different judges by analyzing their opinions). Id . at 81-82.

Analyzing legal texts such as judicial opinions and statutes is an extremely common activity in empirical legal research, and therefore deserves more attention here than the other three methods of gathering or generating data. Text analysis is one example of the broader technique known as "content analysis," and involves "collect[ing] a set of documents, such as judicial opinions on a particular subject, and systematically read[ing] them, recording consistent features of each and drawing inferences about their use and meaning." Hall & Wright 64 (2008). The process of "recording consistent features" of each text is also known as coding, and allows for translating properties of the text into variables that are suitable for systematic analysis. Epstein & Martin 95 (2014). For much more information about the empirical analysis of legal texts, see Hall & Wright (2008) and Epstein & Martin ch. 5 (2014).

Because different types of data collection are subject to different types of measurement error, it may be useful to collect data using more than one method. Epstein & King 102 (2002).

Deciding How Much Data to Collect

Collect as much data as you can given the available resources. It is better to collect too much data than too little. Id . For example, if you are analyzing the outcome of federal appellate cases and your research question requires you to know which president appointed each deciding judge but you are uncertain whether you need to know the judges' ages, you should probably err on the side of recording their ages. The same process that allows you to determine the appointing presidents (reviewing judicial biographies) may also allow you to determine their ages. If you don't record all potentially relevant information the first time through and decide that you need it later, you will end up duplicating efforts. If you record data that you don't end up using for the current project, maybe it will be useful for some later project.

Another factor in deciding how much data to collect is how much uncertainty you are willing (or able) to tolerate in your research results. In general, the more data your conclusions are based on, the more certain those conclusions are. If you are reporting inferential statistics, you should include the margin of error, which is a measure of uncertainty about the statistics. Mathematically, a larger sample size results in a smaller margin of error. Epstein & Martin 85 (2014).

Avoiding Bias in the Selection of Observations

It is often not possible to study all members of a target population. In such situations, you will want to use a method of selecting observations that won't bias your sample either for or against your theory. Id . at 86-87. If you have the resources to collect data from a fairly large sample, then the least biased way to select the sample is through random probability sampling. "A random probability sample is a sample in which each element in the total population has a known (and preferably the same) probability of being selected." Id . at 87. If you do not have the resources to conduct a large sample, other selection mechanisms may be better for avoiding bias.

A great deal more information about sampling techniques is available in Johnnie Daniel, Sampling Essentials: Practical Guidelines for Making Sampling Choices (2012) .

Documenting Collection Methods

Regardless of which methods are used to collect data, the collection process should be thoroughly documented. Documentation is adequate when it provides enough detail so that other researchers could replicate the data collection process independently. Epstein & King 38 (2002). For example, consider the statement that jurors were "randomly sampled" for interviews. Id . at 39. Based on this statement alone, could you repeat the original author's method of selecting a group of jurors to interview? Documenting your data collection methods makes your ultimate conclusions more credible to other researchers, because it permits them to evaluate your methods for flaws and biases.

<< Previous: Research Design
Next: Resources on Empirical Research Methodology >>
© Georgetown University Law Library. These guides may be used for educational purposes, as long as proper credit is given. These guides may not be sold. Any comments, suggestions, or requests to republish or adapt a guide should be submitted using the Research Guides Comments form . Proper credit includes the statement: Written by, or adapted from, Georgetown Law Library (current as of .....).
Last Updated: Apr 15, 2024 3:30 PM
URL: https://guides.ll.georgetown.edu/EmpiricalLegalStudies

1 Historical Roots of Empirical Legal Studies

Empirical Legal Studies is the product of several histories. Mid-nineteenth century U.S. legal instruction and practice paved the way for law as an academic discipline. Late nineteenth century development of social science departments gave law faculty access to university infrastructure and interdisciplinary colleagues. Early twentieth century experiments in law and social science paved the way for research subfields such as law and economics, empirical legal studies, new legal realism, and law and social science. Many refer to these recent traditions collectively, as Empirical Legal Studies, empirical legal studies, or ELS, though this labelling is a source of debate.

Understanding these histories is important for three reasons. First, the histories point us to primary source research material such as the Litchfield notebooks, which were written by law students at one of the nation’s early law schools and digitized for research by twenty-first century law librarians (Eiseman, Bagnall, Kellett, & Lam, 2016). Second, the histories explain the dominance of certain social scientific disciplines and methods in past and current empirical legal studies. Finally, the histories of empirical legal studies enable us to predict the future of the field.

Chapter learning objectives

Understand the early history of U.S. legal practice and instruction
Analyze the interdisciplinary roots of empirical legal research
Evaluate the influence of historical practices on current legal research and teaching
Create historical research aids

Abbreviations and specialized terms

American Association of Law Libraries (AALL), Annual Law and Economics Review , Annual Review of Law and Social Science , case law, Conference on Empirical Legal Studies (CELS), clerk, content analysis, de facto , de jure , empirical legal studies, Inns of Court, IPUMS, Jurimetrics , Journal of Empirical Legal Studies (JELS), Law and Human Behavior , law and social science, Law & Society Review , Law Library Journal , legal realism, natural sciences, new legal realism, pragmatism, qualitative coding, Society for Empirical Legal Studies (SELS), social sciences, statutes

Early Legal Education and U.S. Law

United States legal instruction developed slowly. The colonial colleges offered no practical training in law (Klafter, 1993). So, wealthy young men went to Britain, attended Inns of Court, and applied to become English barristers (Lucas, 1974; Klafter, 1993). As trainees, they observed court proceedings and read barristers’ books and legal notes (Lucas, 1974). This pathway to practice was reserved for affluent, well-connected families. Following the U.S. revolution, these students began to train closer to home.

By the late 1700s, most new attorneys were self-taught practitioners that studied state statutes, or laws created by legislatures, in preparation for admission to the state bar (Klafter, 1993). Many served as apprentices, or law clerks, to practicing attorneys and judges. Clerkships were often unstructured, menial, and inadequate (Klafter, 1993). To fill the gap, prominent attorneys and judges began to launch private training programs.

Beginning in the late 1700s in Connecticut and surrounding states, aspiring attorneys could enroll in private classes (Forgeus, 1939). Many of these classes took place in law offices. The Litchfield law school, founded by Tapping Reeve and known for decades as Judge Reeve’s School, is widely considered the earliest formal training program, though a number of others opened soon after (Forgeus, 1939, 1942).

The early law schools typically had classes of a dozen or fewer students. Judge Reeve’s school was especially popular, though, and had 50 students around 1810 (Forgeus, 1942). Law students often hailed from prominent families, attended Ivy League undergraduate institutions, and went on to notable positions in government and industry (Forgeus, 1939, 1942, 1946). They kept notebooks that they recited from and corrected to master their assigned subjects.

By the early 1800s, the private training programs began developing formal curricula. A student notebook, circa 1811-13, from Sylvester Gilbert’s law school in Hebron, Connecticut lists a program of study focused on common legal issues such as mortgages, legal writing (e.g., pleadings), and legal practice in Connecticut state (Forgeus, 1942; Figure 1.1).

1.1 Edited excerpt of Gilbert curriculum (Forgeus, 1942, p. 201)

Gilbert and other instructors utilized legal treatises, such as Kirby, Root, and Day’s collection of Connecticut cases or Swift’s treatise on Evidence (Fernandez, 2012; Forgeus, 1939, 1942). Still, their lectures focused on English cases (Forgeus, 1942) and the old country principles the instructors knew best (Klafter, 1993). As a result, the private training programs offered a regressive legal education that failed to fully prepare students in emerging U.S. laws and legal customs (Klafter, 1993).

1.2 Law Librarian Spotlight: Elizabeth Forgeus

From the mid-1930s to late 1940s, Elizabeth Forgeus published numerous articles on law school history. Rich with passages and lists extracted from student letters and notebooks, her pithy articles humanized the early law schools. She also assisted with longer research works on the history of Yale Law School and legal practice in New England. Serving as Assistant Law Librarian at the Yale Law School for more than a decade, Ms. Forgeus was also active in the American Association of Law Libraries (AALL). Past issues of AALL’s flagship publication, Law Library Journal , list Ms. Forgeus as a member of the Committee on Cooperation with Latin-American Law Libraries and the Advisory Committee on Education for Law Librarianship—Editorial Board on Monographs. Her work demonstrated the value of reference librarianship, archival collecting, international outreach, and educational advocacy in law libraries.

By the mid-nineteenth century, private law programs gave way to academic law schools. The College of William and Mary and Transylvania College offered the first vocational legal instruction (Klafter, 1993). Harvard, the University of Maryland, Washington and Lee, Yale and others followed (Klafter, 1993). Despite the growing number of academic law programs, university legal instruction remained in flux for decades, in part due to a dearth of textbooks.

In 1870, Christopher Columbus Langdell joined the Harvard Law faculty. The next year, he published a Contracts casebook for use in his classes. In the preface, Langdell explained that most law students accessed books through the library and had no lasting copies of the cases they read (1871). He felt that reported cases were the best evidence of how the law was developing and that students needed to rigorously analyze case law to learn the principles and doctrines, or science as he called it, of U.S. law (1871).

Langdell’s approach was treatise-based and deductive. This view of legal education—and legal practice—was a hallmark of the era known as the classical period, or era of legal formalism (Hackney, 2014). Langdell’s Contracts casebook encouraged students to deduce general rules from existing cases. This case law method is still the most common form of teaching in first-year law school courses.

Langdell’s casebook is an artifact of a larger debate over legal formalism in the early twentieth century. Legal formalism posited law as apolitical or a priori to social concerns (Hackney, 2014). Believing that law should exist outside of messy social situations, many experts recommended contracts as a way to deal with complex social issues such as unfair working conditions and environmental pollution (Hackney, 2014). The Lochner case is now taught as quintessential formalism.

1.3 Legal Touchstone: Lochner v. New York

In 1905, the Supreme Court issued its opinion in Lochner v. New York , finding that a New York state employment law was unconstitutional. The law prohibited bakeries from allowing their employees to work more than 60 hours a week or 10 hours a day, with some exceptions. In a five-justice majority opinion written by Justice Peckham, the Supreme Court found that the state statute interfered with employees’ contract rights under the 14th Amendment to the U.S. Constitution. Justice Harlan wrote a dissent, which was joined by Justices White and Day, that discussed workers’ health and argued that the government could legislate to protect the health of citizens. Justice Holmes also dissented and argued that Lochner was “decided upon an economic theory [laissez-faire] which a large part of the country [did] not entertain” (p. 75) The Lochner majority opinion represented the formalist position that the law is generally set apart from concerns of health and economics (Hackney, 2014). The dissents reflected a growing trend toward viewing the law as embedded in social relations.

Today, scholars and practitioners actively debate the law’s relation to society (Suchman & Mertz, 2010). In fact, empirical legal studies can be understood in opposition to formalism because ELS explores how the law can cause or ameliorate social problems. But despite the widespread acceptance of empirical legal studies as a valid research field, the empirical approach has been slow to infuse law school curriculum. First year law students still largely dissect cases presented in Langdell-style casebooks; they do not study how real people interpret liability waivers or rental contracts, etc. So, law school instruction focuses on formal rules while empirical legal studies research explores law in social context.

Research that examines how the law treats various classes of plaintiffs or improves the function of government, etc. requires more than a social orientation. It requires social science theories and research methodologies. It is no surprise, then, that empirical legal research followed the rise of modern social science departments in the United States.

Development of U.S. Social Science Departments

Prior to the 1840s, U.S. colleges generally taught a classics curriculum—Latin, Greek, mathematics, literature, and rhetoric—derived from leading Western European universities (Geiger, 2000; Thomas, 2015). By the mid-1800s, this standard curriculum began to vary across institutions and regions of the country (Geiger, 2000). Institutions such as the Lander College for women in South Carolina featured the classical A.B. degree into the twentieth century, whereas colleges throughout the state of Ohio began moving away from the curriculum decades earlier (Bondurant, 1909; Geiger, 2000). Overall, the traditional classics degree declined nationally as multipurpose colleges began to take shape (Geiger, 2000). Multipurpose colleges enrolled a more diverse student body, including women and lower class students, and responded to “emerging markets for practical, vocational skills” (Geiger, 2000, p. 148).

1.4 Understanding early U.S. college admissions trends through scholarly literature and government data

The multipurpose colleges were more diverse than the colleges that preceded them, but their student bodies were still largely White and upper class. Most nineteenth century colleges did not admit Black students and many had de jure (by policy or law) or de facto (in practice) admissions policies that limited the attendance of Asian, Latino, Jewish, and Native American students. Empirical researchers can explore scholarly literature on college admissions via a literature search. For instance, the following search string should produce useful results in EBSCOhost: ((college OR university) AND admiss* AND (historic* OR “nineteenth century”) AND (race OR gender OR Black OR “African American” OR Asian OR Hispanic OR Jewish OR Latino OR “Native American”)). For demographic context, the U.S. Census Bureau maintains historical publication series. The Minnesota Population Center lists the Census questionnaires from 1850 forward on its IPUMS, or Census microdata, site at https://usa.ipums.org/usa/voliii/tEnumForm.shtml. The Census questionnaires offer a glimpse into the evolving language of racial and ethnic identity—and exclusion—in the U.S.

In 1862 and 1890, Congress passed the Morrill Acts (Geiger, 2000). The Morrill Acts gave federal lands to states for the creation of local colleges focused on “agriculture and the mechanic arts” (Morrill Act of 1862). The new laws furthered an existing system of publicly-supported colleges—including tax-supported private colleges like Harvard and Yale (Thomas, 2015)—but emphasized the vocational role of land-grant institutions. As a result, U.S. colleges became more attuned to local industries and interests than their Western European counterparts (Thomas, 2015). But though U.S. colleges were becoming more pragmatically focused, they were not yet incubators of scientific discovery or rigorous social science research.

A number of factors slowed the development of scientific and social scientific research in U.S. colleges. Most U.S. colleges had “serious religious commitments,” and formalistic theological instruction permeated the curricula until well after the Civil War (Veysey, 1965). For instance, more than 40% of Yale University’s 1900 budget supported the Theological Department (Treasurer of Yale University, 1900; 1.5).

Medical, Law & Music combined were 15% of the budget; Art School was 8%; Forest School was 9%; Sheffield Scientific School was 27%; and Theological Department was 41%

1.5 Yale University departmental budget allotments, 1900

Nineteenth century U.S. colleges, unlike their Scottish counterparts, had no track record of scientific excellence, and no community of science researchers or educators (Ben-David, 1992). When reformers at Cornell and Harvard wanted to increase scientific research and teaching in the late 1800s, they struggled to find competent teachers and a place in the undergraduate curriculum for such courses (Ben-David, 1992). As a result, reformers shifted their attention to the creation of graduate schools at existing colleges.

Once graduate schools began producing rigorously trained academics, those scholars were able to modernize both research and undergraduate instruction in the sciences and social sciences (Ben-David, 1992). At that point, the U.S. system again diverged from much of Western Europe, where “first-rate scholars were not willing to teach courses in a program of general education” (Ben-David, 1992, p. 83). In the U.S., top-tier researchers conducted research, employed graduate students, and taught undergraduate students in university departments dedicated to the natural and social sciences. Their research built upon the work of European and U.S. scholars.

European pioneers of the social sciences included Adam Smith, Marie Jean Antoine Condorcet, and Johann Gottfried Herder (Ross, 1993). In the U.S., notable figures included Harvard’s William James and the University of Chicago’s John Dewey (Buxton, 1984; Thayer, 1982), not to be confused with Columbia University’s library science educator, Melvil Dewey. James and John Dewey led the development of pragmatism, a U.S.-born philosophical tradition that emphasized the value of grounded theories of people and the social world (Misak, 2013; Stanford Encyclopedia of Philosophy). Dewey’s view of pragmatism, like much early U.S. social science, hewed to natural science methods (Misak, 2013; Ross, 1993). According to historian Dorothy Ross, the U.S. social sciences were generally less heterogeneous and more derivative of the natural sciences than their European counterparts (1993). Throughout the twentieth century, Economics increasingly dominated the U.S. social sciences (Haskell, 1977; Siegfried, 2008). As evidence: the American Social Science Association, founded in late 1800s, now operates as a subset of the American Economic Association (see Haskell, 1977; Siegfried, 2008). The unique trajectory of the U.S. social sciences as empirical, experimental, and economics-dominated set the stage for the empirical legal research movements of the twentieth century.

1.6 Notes from the Desk of Sarah E. Ryan: My empirical research on the ‘facebook’

In 2012, Yale Law School hired me as the nation’s first dedicated empirical legal research librarian (Miguel-Stearns & Ryan, 2014; Ryan & Miguel-Sterns, 2014). So, I had to build the new service area. My biggest issue was that I did not know where my patrons—especially the students—were starting from. Were they continental philosophers? Economists? Mechanical engineers? I mulled the idea as I began to take appointments. To prepare for one reference meeting, I looked up the patron in our print “facebook,” which contained a picture of each student, faculty, and staff member as well as students’ prior degrees. It dawned on me that the “facebook” was a data source and that mining it would be an interesting content analysis, or coding, project. For the “facebook” project, my chief aim was efficiency, so I adopted a binary (0/1) code for social science degree (i.e., either the student had a social science degree or didn’t). However, I quickly saw that colleges award a dizzying array of degrees, many of which are interdisciplinary. Further, I realized that I had never given much thought to what divides the humanities from the social sciences and the social sciences from the natural sciences. So, I reviewed the websites of prominent universities and kept a tally of what they counted as social sciences (Ryan, 2013). I then used a spreadsheet as my coding instrument and followed a simple rule: “students with degrees in six key social science fields would receive a one rather than a zero in the degree columns” (Ryan, 2013). The project yielded a useful statistic: more than 70% of Yale law students held a social science degree (Ryan, 2013). The empirical research was a fun way get to know my patrons, organize my work, and earn a first publication in law librarianship.

Legal Realism: The First Attempt to Merge Legal Research And Social Sciences

As the social sciences gained prominence in academia, legal scholarship began to move beyond formalism. While formalists championed the scientific application of context-neutral law, a new group of thinkers challenged the deductive, apolitical view of legal practice and justice. The legal realists of the early twentieth century adopted a contextual vision of the law and legal training (Hackney, 2016). In an effort to predict how judges and bureaucrats would enact law, for instance, realists studied their decision-making (Schlegal, 1980). Realists grounded this work in logic, philosophy, psychoanalysis, and scientific management theory (Cohen, 1923), and engaged with theorists outside of the law school. Underhill Moore, a realist concerned with how bankers assess risk (Moore & Hope, 1929), was encouraged to pursue his law-in-action research by leading social scientists of the day, including John Dewey (Schlegal, 1980). By the 1930s, dozens of scholars were conducting realist studies (Llewellyn, 1930; Hull, 1987). John Hopkins University launched an Institute of Law to foster realist research (John Hopkins University, 1929). But before long, the realists faced challenges from across their universities.

The realist movement coalesced around the New Deal and boasted prominent legal scholars (Suchman & Mertz, 2010). But by the mid-twentieth century, the movement had all but collapsed. Social scientists criticized legal realist work as crude and devoid of guiding theory or disciplinary tradition (Schlegal). Many traditional legal scholars dismissed the realists wholesale without appreciating their aims or methods (Schlegal). From all sides, the critiques understated differences within the realist movement—e.g., scholars focused more on reform rather than social science and vice versa—and between realism and formalism (Schlegal). Most importantly, the realists had neither an established place in the university nor in the legal profession (Schlegal). Without institutional support, the realists were unable to sustain their endeavor (Schlegal; Suchman & Mertz).

But even if the realists failed in their objectives, their approach to the law was a break with tradition that had lasting impacts on legal education, practice, and research. Realists like Underhill Moore, Charles Clark, William Douglas, Karl Llewellyn, and Louis Brandeis,

secured a place for social science in the law (Suchman & Mertz). As a result, realism paved the way for Empirical Legal Studies, the Law and Society movement, and other late 20th century legal subfields.

Contemporary Empirical Legal Studies

Today, the history of legal realism is itself a cottage industry. Some monographs focus on Roscoe Pound, others on Underhill Moore, and still others on Harold Lasswell and legal realists at Yale Law School. As professor Jack M. Balkin explains, “the choice of who is enshrined in the [realism] canon turns very much on the points one wants to make about the history of American law and legal education” (1998, p. 200). Similarly, the story of post-World War II empirical legal studies is contested and political.

Following World War II, law and social science projects gained steam again, most notably at the University of Chicago and Yale Law School. At Chicago, economist Ronald Coase led groundbreaking econometric studies of property law and transaction costs, and fostered a law and economics movement that survives to this day (Coase, 1993; Landes & Posner, 1993; Ho & Rubin, 2011). At Yale Law School, Underhill Moore was soon joined by Eugene Rostow, and then Boris Bittker and Guido Calebresi. Guido Calabresi produced empirical-theoretical treatises on strict liability that challenged formalism and neoclassical economics (Hackney, 2014). Calabresi’s progressive policy focus still informs descriptive and normative empirical scholarship. Elsewhere, social scientists such as Fred Kort used empirical methods to study judicial behavior and other legal topics (see Kort, 1957); their work is now seen as foundational to the field even if it was underappreciated by law professors at the time (Hall & Wright, 2008). The topical and methodological heterogeneity evidenced by Coase, Calebresi, and Kort’s work leads some scholars to view modern law and social science research as a continuation of legal realism (i.e., New Legal Realism; Suchman & Mertz, 2010). Fortunately, the broad field encompassing empirical legal studies, new legal realism, and law and social science has become more rigorous than legal realism ever was.

Until the late twentieth century, empirical legal research was an eclectic and undisciplined field. Content analysis was an early empirical research method, for instance, but most legal empiricists developed their analytic practices organically and without reference to methodological advances in fields such as Linguistics or Political Science (Hall & Wright, 2008; Suchman & Mertz, 2010). For instance, Richard A. Posner “read every published accident opinion of an American appellate court (state or federal, final or intermediate) issued in the first quarter of 1875, 1885, 1895, and 1905 . . . .” for his seminal article, A Theory of Negligence , but did not develop a codebook or utilize a content analysis instrument or employ a neutral coder to systematize and validate his findings (Posner, 1972; Hall & Wright).

Noting the need for methodological advancement, research training, and routine scholarly communication among empiricists, Theodore (Ted) Eisenberg of Cornell Law School and others founded the Society for Empirical Legal Studies (SELS), Journal of Empirical Legal Studies (JELS), and Conference on Empirical Legal Studies (CELS). Today, empirical legal researchers can present their early work at conferences featuring artificial intelligence and law, psychology and law, text analysis and law, and more. This organizing work was one of the factors in the rapid growth of empirical legal studies from the 1990s to the 2010s (Heise, 2011). So to was the development of specialized law journals. Today, law and social science scholars can publish their findings in a range of empirical research journals, including: Annual Law and Economics Review , Annual Review of Law and Social Science , Jurimetrics , Law and Human Behavior , Law & Society Review , and more. As a result, ELS offers many opportunities and resists simple classification. The field of several histories continues to branch out in many directions.

Reflection Questions

Prior to reading this chapter, what was your impression of how U.S. legal instruction developed? Has that changed? If so, how? If not, which facts or ideas in this chapter reinforced your existing beliefs?
Do you agree or disagree with the chapter’s premise that Economics has long dominated the social sciences in the U.S.? Explain. If you were educated in a non-U.S. country, which perspectives seem to dominate the social sciences in that country(ies)?
Pragmatism played a key role in the development of the U.S. social sciences. How might pragmatism affect legal researchers’ choice of research subjects or methods today?
How might you display historical information in this chapter visually (e.g., in presentation slides)?

Balkin, J. (1998). John Henry Schlegel, American Legal Realism and Empirical Social Science, Chapel Hill: The University of North Carolina Press, 1995 [book review]. Law and History Review , 16 (1), 199-201.

Ben-David, J. (1992). Centers of learning: Britain, France, German, United States . Routledge.

Bondurant, B. C. (1909). The status of the classics in the South. The Classical Journal, 5 (2), 59-67.

Buxton, M. (1984). The influence of William James on John Dewey’s early work. Journal of the History of Ideas, 45 (3), 451-463.

Coase, R. H. (1993). Law and economics at Chicago. Journal of Law and Economics, 36 (1), 239-254.

Cohen, M. R. (1923). On the logic of fiction. The Journal of Philosophy , 20 (18), 477-488.

Eiseman, J., Bagnall, W., Kellett, C. & Lam, C. (2016). Litchfield unbound: Unlocking legal history with metadata, digitization, and digital tools. Law and History Review , 34 (4), 831-855.

Fernandez, A. (2012). Tapping Reeve, coverture and America’s first legal treatise. In A.Fernandez & M. D. Dubber (Eds.), Law books in action: Essays on the Anglo-American legal treatise (pp. 63-81). Hart Publishing.

Forgeus, E. (1939). Letters concerning some Litchfield Law School notebooks. Law Library Journal, 32 (5), 201-05.

Forgeus, E. (1942). An early Connecticut law school: Sylvester Gilbert’s school at Hebron. Law Library Journal, 35 (4), 200-03.

Forgeus, E. (1946). Sylvester Gilbert’s law school at Hebron, Connecticut: The students. Law Library Journal, 39 (2), 49-52.

Geiger, R. L. (2000). The era of the multipurpose colleges in American higher education. In R. Geiger (Ed.), The American college in the nineteenth century (pp. 127-152). Vanderbilt University Press.

Hackney, J.R., Jr. (2014). Guido Calabresi and the construction of contemporary American legal theory. Law and Contemporary Problems, 77 (1), 45-64.

Hall, M. A., & Wright, R. F. (2008). Systematic content analysis of judicial opinions. California Law Review, 96 (1), 63-122.

Haskell, J. L. (1977). The emergence of professional social science: The American Social Science Association and the nineteenth-century crisis of authority . University of Illinois Press.

Heise, M. (2011). An empirical analysis of empirical legal scholarship production, 1990-2009. University of Illinois Law Review, 2011 (5), 1739-1752.

Ho, D. E., & Rubin, D. B. (2011). Credible causal inference for empirical legal studies. Annual Review of Law and Social Science , 7 , 17-40.

Hull, N. E. (1987). Some realism about the Llewellyn-Pound exchange over realism: The newly uncovered private correspondence, 1927-1931. Wisconsin Law Review, 1987 (6), 921-969.

John Hopkins University. (1929). The story of the Institute of Law at the John Hopkins University . Norman T.A. Munder & Co.

Lochner v. New York , 198 U.S. 45 (1905).

Klafter, C. E. (1993). The influence of vocational law schools on the origins of American legal thought, 1779-1829. The American Journal of Legal History, 37 (3), 307-31.

Kort, F. (1957). Predicting Supreme Court decisions mathematically: A quantitative analysis of the “right to counsel” cases. American Political Science Review , 51 (1), 1-12.

Landes, W. M., & Posner, R. A. (1993). The influence of economics on law: A quantitative study. The journal of law and economics , 36 (1, Part 2), 385-424.

Langdell, C. C. (1871). A selection of cases on the law of contracts: With references and citations . Harvard Law School.

Llewellyn, K. N. (1930). A realistic jurisprudence –The next step. Columbia Law Review, 30 (4), 431-465.

Miguel-Stearns, T., & Ryan, S. E. (2014). The empirical research law librarian. Part 1: Making the case and filling the role. Trends in Law Library Management and Technology, 24 , 1-6.

Misak, C. (2013). Rorty, pragmatism, and analytic philosophy. Humanities, 2 (3), 369-83.

Moore, U., & Hope, T. (1929). An institutional approach to the law of commercial banking. Yale Law Journal, 38 (6), 703-19.

Morrill Act of 1862. (2012). Codified at 7 U.S.C. § 301 et seq.

Posner, R. A. (1972). A theory of negligence. The Journal of Legal Studies , 1 (1), 29-96.

Ross, D. (1993). An historian’s view of American social science. Journal of the History of the Behavioral Sciences, 29 (2), 99-112.

Ryan, S. E. (2013, June). Digging in to patron data: What I learned from the law school “facebook.” AALL Spectrum , 20-21.

Ryan, S. E., & Miguel-Stearns, T. (2014). The empirical research law librarian Part 2: Developing the role. Trends in Law Library Management and Technology, 24 , 7-12.

Schlegel, J. H. (1980). American legal realism and empirical social science: The singular case of Underhill Moore. Buffalo Law Review, 29 (2), 195-323.

Siegfried, J. J. (2008). History of the meetings of the Allied Social Science Associations since World War II. The American Journal of Economics and Sociology, 67 (5), 973-84.

Stanford Encyclopedia of Philosophy. Pragmatism . https://plato.stanford.edu/entries/pragmatism/

Suchman, M.C. & Mertz, E. (2010). Toward a new legal empiricism: Empirical legal studies and new legal realism. Annual Review of Law and Social Science, 6 , 555-79.

Thayer, H. S. (Ed.). (1982). Pragmatism, the classic writings: Charles Sanders Peirce, William James, Clarence Irving Lewis, John Dewey, George Herbert Mead . Hackett Publishing.

Thomas, G. (2015). The founders and the idea of a national university: Constituting the American mind . Cambridge University Press.

Treasurer of Yale University. (1900). Report of the treasurer of Yale University . Yale University.

Veysey, L. R. (1965). The emergence of the American university . University of Chicago Press.

Empirical Legal Research Services by University of North Texas is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License , except where otherwise noted.

Share This Book

Sign up to get instant access to your free e-book

You have subscribed successfully!

Some error occured

Please fill all the required * fields.

We respect your privacy. We will never spam you.

An Empirical Study in Legal Methodology

Abstract: This empirical study delves into the realm of legal methodology, aiming to scrutinize the effectiveness of various approaches in legal research and analysis. The study employs a mixed-methods approach, combining qualitative and quantitative techniques to provide a comprehensive understanding of the subject matter. Through surveys, case studies, and data analysis, this research seeks to uncover insights that can enhance legal scholarship and practice. By evaluating the strengths and weaknesses of different methodologies, the study aims to contribute to the refinement of legal research techniques and the development of more robust analytical frameworks.

Introduction: Legal methodology plays a pivotal role in shaping the foundations of law and jurisprudence. It encompasses a diverse range of approaches, from doctrinal analysis to socio-legal research, each offering unique insights into the complexities of legal systems. However, despite the abundance of methodologies, questions persist regarding their efficacy and suitability in addressing contemporary legal challenges. This empirical study seeks to address this gap by systematically examining the strengths and limitations of various methodologies employed in legal research.

Research Objectives:

To identify and categorize different methodologies commonly used in legal research.
To assess the effectiveness of these methodologies in generating reliable and insightful legal analysis.
To explore the practical implications of different methodologies in addressing real-world legal issues.
To provide recommendations for enhancing the rigor and relevance of legal research methodologies.

Methodology: This study adopts a mixed-methods research design, combining qualitative and quantitative approaches to gather comprehensive data. The research methodology comprises the following components:

Literature Review: A thorough review of existing literature on legal methodology to establish a theoretical framework and identify key methodologies.
Survey: Conducting surveys among legal scholars, practitioners, and students to gather insights into the prevalence and perceived effectiveness of different methodologies.
Case Studies: Analyzing case studies to examine how various methodologies are applied in real-world legal contexts and their impact on legal outcomes.
Data Analysis: Employing statistical analysis techniques to quantify the strengths and weaknesses of different methodologies based on empirical data.
Comparative Analysis: Comparing the findings from different research methods to identify convergences and divergences, thereby enhancing the robustness of the study.

Results and Discussion: The findings of this empirical study reveal a nuanced understanding of legal methodology and its practical implications. While traditional doctrinal approaches remain prevalent, there is growing recognition of the value of interdisciplinary and empirical methods in enriching legal analysis. The survey results indicate a diverse range of perspectives among legal scholars and practitioners regarding the efficacy of different methodologies, underscoring the need for methodological pluralism in legal research. Case studies highlight the contextual factors that influence the choice of methodology and its impact on legal outcomes. Statistical analysis provides quantitative insights into the strengths and limitations of various approaches, facilitating evidence-based decision-making in legal scholarship and practice.

Conclusion and Implications: This empirical study contributes to the ongoing discourse on legal methodology by offering empirical evidence and insights into the effectiveness of different approaches. By acknowledging the plurality of methodologies and their respective contributions to legal scholarship, this research advocates for a more inclusive and interdisciplinary approach to legal research. The study’s findings have practical implications for legal scholars, practitioners, and policymakers, guiding them in adopting appropriate methodologies to address complex legal issues effectively. Moreover, by promoting methodological reflexivity and innovation, this research aims to foster continuous improvement in legal research practices and enhance the relevance and impact of legal scholarship in the contemporary socio-legal landscape.

There are no comments yet

28 Evidence Law

Gary Edmond is Professor of Law in the School of Law and Centre for Interdisciplinary Studies of Law at the University of New South Wales.

David Hamer is Associate Professor in Evidence and Proof at the Sydney Law School, University of Sydney.

Published: 18 September 2012
Cite Icon Cite
Permissions Icon Permissions

This article reviews contemporary response to several contrasting strands of recent empirical work. It begins with discussing the scope and rationale of evidence law. Experimental studies on eyewitness memory and testimony illustrate the potential value of empirical studies to the practice of investigations, prosecutions, and appeals. This article discusses several lines of empirical inquiry employing diverse methodologies, experiments, surveys, and approaches and reviews their limitations, and implications and significance for the understanding and practice of law. Many of the contributions from empirical legal studies are provisional and their precise value for practice is uncertain or ambiguous, but they raise important issues worthy of serious consideration. By identifying problems with eyewitness evidence and the limitations of many of the forensic sciences, and in many other ways, empirical and experimental studies have substantially outpaced legal experience. Indifference to empirical legal study is likely to reduce the social legitimacy of legal institutions.

Introduction 652

Evidence Law in the “ Rationalist Tradition” 653

Scope of evidence law 653

Rationale of evidence law 654

Diverse Empirical Studies 656

The exclusion and admission of hearsay 656

Expert opinion evidence 661

Subjective probability and human inference 666

Discussion 669

Abstraction from trial environment 669

Specificity of conditions, generalizability of conclusions 670

Factual uncertainty and the benchmark problem 671

Competing goals and values 672

Bigger pictures 673

Rationalist and Empiricist Legacies 674

I. Introduction

“E vidence law,” as a “subject suitable to be treated as a unified field of regulation and of study,” is a product of the Anglo-American adversarial tradition (Damaška, 1997 : 109). Continental civil law systems have fewer rules of evidence, and these tend to be associated with specific bodies of substantive law. While Anglo-American evidence law is broadly accepted as a relatively discrete domain, it is generally viewed as “labyrinthine” and “dishevelled,” “l argely ununified and scattered, existing for disparate and sometimes conflicting reasons … a mixture of astonishing judicial achievements and sterile, inconvenient disasters” (Damaška, 1997 : 10–11; Heydon, 1984 : 3).

There are many unanswered questions about evidence law, concerning its historical development, its various purposes, and the interrelation between its diverse components and other areas of legal and non-legal practice. Most, if not all, invite empirical inquiry. And yet, notwithstanding that evidence law is itself concerned with evidence and proof and is generally supposed to have developed in a “rationalist tradition” (Twining, 2006 : 75), legal practice and evidence scholarship have been remarkably indifferent, and even hostile, to empirical study and evidence-based proposals for reform. Political expediency and the experience of legal practitioners, especially the judiciary, appear to be more important than scholarly attempts to understand evidence law in terms of its legal and social context(s).

Rather than providing a comprehensive account of empirical studies pertaining to evidence law, our goal here is to present a brief review and contemporary response to several contrasting strands of recent empirical work. We begin by setting the scene with an outline of the scope and rationale of evidence law.

II. Evidence Law in the “ Rationalist Tradition”

A. scope of evidence law.

To varying degrees evidence law regulates the admissibility and presentation of evidence and fact-finding at trial. The trial is a highly formalized system of empirical enquiry, operating within a prescribed procedural framework. In a jury trial, the jury is responsible for fact-finding while the judge determines the applicable law. In the absence of a jury the judge is responsible for legal and factual decisions. Jury trials are becoming increasingly rare, though a notional bifurcation between the tribunal of law and tribunal of fact remains.

In the adversarial tradition the parties identify the areas of factual dispute, and decide what evidence to make available to the court. At trial the plaintiff or prosecutor presents her case first, and then, if there is a case to answer, the defendant makes his case. Most evidence takes the form of witness responses to questions directed by the parties, though the traditional preference for oral evidence is in decline, as documentary evidence and “paper trials” become more prevalent, most conspicuously in civil litigation without juries. Those few disputes which are not abandoned or negotiated are generally litigated through a single continuous trial. While jury decisions, incorporating factual reasoning and the application of law to facts, are left unexplained, in many jurisdictions judges provide reasons for interlocutory decisions and their verdicts.

The trial judge should only admit evidence that is relevant to a fact in issue and survives exclusionary rules, such as those pertaining to hearsay, opinion, bad character, propensity, and so on. Evidence that is otherwise admissible may nevertheless be excluded as a matter of judicial discretion, for example, due to the risk of unfair prejudice to the defendant, or the public policy against obtaining evidence in certain ways (such as through physical coercion). Litigants, witnesses, and interested parties may also be able to prevent the admission of evidence by the exercise of legal professional privilege, the privilege against self-incrimination, and public interest immunity.

Once all of the evidence has been tendered, the judge will instruct the jury as to how it can be used. In a judge-only trial interlocutory decisions and instructions often form part of the written judgment. While evidence law is concerned primarily with admissibility rather than weight, some types of evidence—such as eyewitness identification and the testimony of prison informants—give rise to special concerns and the fact-finder will be directed to exercise special caution. The trial judge also instructs the jury on the burden of proof. Generally, the state (or prosecution) must prove a defendant's guilt beyond reasonable doubt, and a civil plaintiff must prove liability on the balance of probabilities (or preponderance of evidence).

Appeals are frequently focused on the (mis)application of rules of evidence (and practice), usually the admission of inadmissible evidence and/or the exclusion of admissible evidence. Appeals on matters of fact are more restricted, though possible where the evidence does not support the verdict or some new (or “fresh”) evidence emerges after the verdict or appeal.

B. Rationale of evidence law

Evidence law, as the foregoing discussion indicates, consists of a mass of rules and exceptions, many of which confer choices, discretions or require particular actions. Whether they can be coherently accommodated within a single theoretical framework is open to question. Most scholars have viewed evidence law as being concerned with “rational methods of determining questions of fact,” in which a central goal is to “maximize accuracy in fact-determination”(Twining, 2006 : 76). Jurists have debated how accuracy is best achieved, and the relative weight to be assigned to sometimes competing goals—such as efficiency, due process, and accuracy—though, these debates have been conducted almost entirely within this shared rationalist tradition.

One major ongoing debate concerns freedom of proof (Twining, 2006 : 43–4;Stein, 2005 : 107–16). Factual inquiries in other spheres of endeavor are not governed by formal rules, so why are legal disputes governed by rules of evidence? In the Anglo-American legal tradition evidence law (along with the choices and discretions it confers) now exerts far greater influence on practice than in most other legal traditions (Damaška 1997 : 19–20; cf. Gluckman, 1955 ), routinely excluding evidence on various grounds. As early as the turn of the nineteenth century Jeremy Bentham ( 1843 : VII, 24) forcefully opposed this approach: “Evidence is the basis of justice: to exclude evidence is to exclude justice.”

Various justifications have been offered for Anglo-American exclusionary rules. One is provided by the “best evidence” rationale (e.g., Gilbert, and Nance)—encouraging the parties to uncover and provide stronger alternatives to the excluded evidence. Exceptions to exclusionary rules are frequently grounded in necessity—particularly the absence of alternative evidence. But this reveals a potential weakness with handing responsibility for gathering and presenting evidence to the parties. Understandably, parties consider utility as well as reliability. Why then are parties afforded so much control? This freedom reflects the Anglo-American perception of the trial, not only as a vehicle for the pursuit of truth and justice, but also as a means of socially legitimate conflict resolution (Damaška, 1997 : 110–11; Tyler, 1990 ).

Another common explanation for exclusionary rules focuses on the position of the jury as fact-finder (e.g., Thayer, 1898 and Wigmore, 1940 ). Evidence is excluded because of the danger that the jury will improperly value it. This, of course, immediately raises supplementary concerns. If lay juries cannot be trusted with the evaluation of evidence, then why should they be used as fact-finders? Persistence with the jury might suggest it serves several functions, including a celebrated form of participatory democracy and a means to disseminate social norms (Damaška, 1997 : 29). However, the jury's ability to handle complex evidence and the effects of popular beliefs and culture (exemplified in high-rating television shows such as “CSI: Crime Scene Investigation”) remain controversial (Cole and Dioso-Villa, 2009 ).

Both the jury's role and the scope of exclusionary rules have tended to diminish over time. The general trend has been in the direction of free proof. However, in recent years several scholars have resisted this trend, either questioning the attribution of some putatively proper value to the evidence (e.g., Edmond) or seeking to demonstrate the importance of values other than accuracy (Ho, 2008 ;Stein, 2005 : 133).

III. Diverse Empirical Studies

Empirical research is being undertaken in many areas of evidence law. Perhaps the most influential body of work has been directed toward eyewitness evidence (e.g., identification evidence), in part, perhaps, because it involves a neat factual issue with few normative complications (Park and Saks, 2006 : 960, 973).

Experimental studies on eyewitness memory and testimony illustrate the potential value of empirical studies to the practice of investigations, prosecutions and appeals. Of scholarly interest for more than a century, from the 1970s psychologists, such as Elizabeth Loftus and Gary Wells, began to conduct systematic experiments on the ability of people to remember things, particularly past events and the identity of persons of interest. Subsequent research focused on how the conditions of observation and investigative processes might influence (and compromise) memory and on ways of improving investigative procedures so as to minimize risks of contamination, suggestion, and displacement.

This research has exerted a positive influence on the ways police, trial and appellate courts respond to identification evidence, although the substantial and convergent results of numerous experimental studies have not been fully implemented (Park and Saks, 2006 : 960–4). Most jurisdictions routinely allow investigators and others to subvert protections designed to enhance the reliability of eyewitness testimony, and retain practices and directions that are antiquated in terms of empirical research and mainstream academic consensus (Wells and Quinlivan, 2009 ). Further, in many jurisdictions, psychologists are not permitted to explain to fact-finders the general problems with identification evidence or known problems with particular practices or specific circumstances. And yet research on wrongful convictions demonstrates that even in sexual assault cases, where the eyewitness often has a relatively good opportunity to observe the offender at close proximity, mistaken eyewitness identification is notorious (see Figure 3 ).

The reluctance to engage with empirical legal studies is an issue to which we will return. In the remainder of this section we report on research in three domains more fraught than eyewitness memory, namely hearsay, expert evidence, and probabilistic reasoning. The studies discussed below illustrate great diversity in empirical research styles, the types of scholars involved, the mix with theory and data, the socio-political implications, and the scope the research creates for intervention and change.

A. The exclusion and admission of hearsay

Ordinarily, witnesses give sworn testimony about their observations, and are available for cross-examination. The hearsay witness, however, testifies as to what someone else (the “declarant”) has said about her observations. Relative to ordinary evidence, hearsay evidence involves at least one additional step, making the fact-finder more remote from the event under consideration (Figure 1 ). The traditional bases for exclusion are that the declarant's out-of-court statement is not under oath and the declarant is unavailable for cross-examination.

Thompson and Pathak's 1999 : 457) “hearsay chain”

Damaška ( 1997 : 1), a comparativist, describes the hearsay rule as “so bizarre [as to] occupy one of the most forbidding corners of the entire Anglo-American legal structure.” The stringency of the historical rule has been tempered by numerous exceptions, but these have added complexity to legal practice. The modern rule, as Rakos and Landsman ( 1992 : 668) report, “remains an amalgam of concerns about juror competence, cross-examination, and fairness.”

Most of the empirical work on hearsay has been carried out by psychologists. Following Hugo Munsterberg's, On the Witness Stand (1908),Hutchins and Slesinger (1928) drew upon the psychological literature to question the rule's scientific legitimacy. However, with few exceptions, only in the last two decades have experimental psychologists and lawyers approached hearsay rules and evidence in ways that transcend historical and doctrinal approaches or attempts to make extant psychological knowledge accessible to a legal audience (McGough, 1999 : 487). Here, we review recent experimental research relating to mock juror assessment of hearsay evidence and related work focused on the hearsay evidence of children.

1. Mock juror responses to hearsay evidence

The first experimental studies endeavored to determine whether jurors overvalue hearsay evidence. In the early 1990s, Rakos and Landsman systematically manipulated the strength of hearsay testimony across versions of a trial transcript pertaining to a prosecution for theft and compared the mock juror responses. They concluded that the “mere introduction of hearsay testimony may not disproportionately influence juror decisions” (Rakos and Landsman, 1992 : 664).

Miene, Park, and Borgida drew similar conclusions. They compared responses to combinations of circumstantial, hearsay and eyewitness testimony using a video simulation of a theft trial. The eyewitness and hearsay witness provided virtually identical evidence, and participants in the hearsay condition received cautionary instructions. These researchers found that participants in the hearsay condition were less likely to produce a guilty verdict and rated the hearsay testimony as less important, influential, and reliable than those responding to eyewitness evidence. This led the investigators to question one of the major rationales for the exclusionary rule: “the data from this study suggests that hearsay as a form of testimony is not overvalued by jurors, as some legal scholars have suggested” (Miene et al., 1992 : 699).

Kovera, Park, and Penrod also expressed confidence in juror abilities, at least in relation to hearsay. Their mock jurors were “more sceptical of the value and reliability of hearsay testimony than of eyewitness testimony” and, further, gave more weight to hearsay testimony when there was a short delay (one day) as opposed to a longer delay (one week) (Kovera et al., 1992 : 719). However, these mock jurors, also exposed to eyewitness identification evidence, were apparently insensitive to its limitations, particularly the corrosive effects of delay.

In contrast, Paglia and Schuller 1998 formed a less favorable view of juror reasoning. Participants in their experiments used hearsay evidence in ways that were inconsistent with judicial instructions included in the audio recording of a mock trial. Regardless of their form and timing, cautionary instructions about prosecution hearsay evidence exerted little discernible impact on the decisions. In an earlier study, focused on exculpatory hearsay provided through an expert witness, Schuller ( 1995 : 359) found that the participants had difficulty ignoring hearsay evidence even when instructed to do so.

2. Persuasiveness and reliability of children's statements

A prominent strand of hearsay research focuses on out-of-court declarations by children. This type of hearsay, common where sexual assault is alleged, is of interest for a number of reasons. Early reports may be valuable as the memory of children is especially vulnerable to influence and degradation. Concerns also arise about child complainants being traumatized by testifying in court in the presence of the alleged perpetrator (Buck et al., 2004 ). More broadly, there is widespread social concern about pedophilia, but relatively low rates of complaint, prosecution, and conviction. In this environment, many jurisdictions have made special provision to admit the out-of-court statements of children, and researchers have sought to understand their potential effects (McGough, 1999 ).

Initially research focused on the believability of hearsay evidence and whether jurors might convict in cases of alleged sexual assault where the child complainant does not testify. In an experiment using a fictional summary of a child sexual assault trial, Golding, Sanchez, and Sego found that the child complainant's testimony was considered more believable than the child's complaint presented as hearsay. Nevertheless, the hearsay evidence seemed to influence “conviction” decisions and the authors concluded that “it may not be necessary for the alleged victim to testify on her own behalf for the defendant to be judged culpable” (Golding et al., 1997 : 318).

Subsequent research considered the identity and status of the hearsay witness. Using a “highly realistic” video of a sexual assault trial, Ross, Lindsay, and Marsil concluded that the persuasiveness of child testimony in hearsay form depended upon the identity of the hearsay witness. Apart from one condition, where the hearsay witness was the mother of the complainant—embroiled in a “heated divorce” with the alleged perpetrator—the child's testimony was “significantly less likely to produce ‘guilty votes’ ” than the evidence of the mother, the child's doctor and teacher (Ross et al., 1999 : 450–1). Studying the effects of the ages of the complainant and the hearsay witness, Golding, Alexander, and Stewart (1999) found that mock jurors split along gender lines, with women generally more likely to accept the hearsay evidence of assault.

So far, the studies in this subsection have focused on the persuasiveness of hearsay evidence. Another strand is concerned with reliability and the competence of jurors. Pathak and Thompson 1999 sought to address a limitation with the studies by Rakos and Landsman 1992 , Miene et al. ( 1992 ), and Kovera et al. ( 1992 ). These earlier studies had concluded that “hearsay is unlikely to be overvalued,” yet they did not include an “objective or normative standard against which to compare [mock] jurors' evaluations.” Pathak and Thompson sought to test “people's inferences about the reliability of hearsay evidence in circumstances that allowed the actual reliability of the evidence to be objectively verified” (1999: 373). They contrived a situation where they covertly controlled a (child) witness's experiences of a mock janitor's behavior and elicited an account through questioning.

Following a complaint, children are usually interviewed by social workers, police, or other professionals. Pathak and Thompson's experiments considered how hearsay is evaluated depending on whether the child is interviewed in a suggestive or neutral manner. They concluded that mock jurors failed to take sufficient account of suggestive questioning:

Although the videotaped “hearsay witnesses” commented on the suggestiveness of the interrogations with the child, the “jurors” did not realize that the suggestive interrogations had a greater influence on children's reports than the neutral interrogations (Pathak and Thompson, 1999 : 381).

Where there is no video or audio recording, the hearsay testimony of the interviewer, along with any notes, may be the only “record” of such exchanges. As Warren and Woodall 1999 : 356) explain, “to properly evaluate a child's statements presented through hearsay, jurors and fact finders need to hear not only what the child said (the gist of the interview), but how it was said (a verbatim account including specific questions and answers).” They found that while interviewers recalled the gist of interviews accurately, they incorrectly recounted the use of open-ended questions even when they made extensive use of specific and leading questions. Further, “[e]ven immediately after an interview, important content was omitted from hearsay accounts, and the majority of the verbatim information (specific wording and content of questions and answers) was lost” (Warren and Woodall, 1999 : 369). Their conclusion: “asking adults to recreate the structure of their conversations or interviews with children after the fact (i.e. during courtroom testimony) is risky” (ibid: 365).

These preliminary results prompted further investigation. Warren, Nunez, Keeney, Buck, and Smith compared the impact on mock jurors of: (1) a video of an interview with a child; (2) the interviewer providing a verbatim account of the interview; and (3) the interviewer providing the gist of the interaction. Participants rated the credibility of the adult gist witness “higher than that of the verbatim witness or child witness, and the verbatim witness was rated as significantly more credible than the child” (Warren et al., 2002 : 852; cf. Golding et al., 1997 ). Gist evidence was perceived as less suggestive, more spontaneous and more open-ended. Counterintuitively, the greater the displacement of the interview from the testimony, the more persuasive it seems to have been (Warren et al., 2002 : 850–1). Results such as these led Buck, Warren, and Brigham to suggest that the use of video or a transcript would provide “a better compromise between protecting the child and the rights of the defendant than the use of testimony by a hearsay witness” (Buck et al., 2004 : 618–20).

Finally, one of the most recent studies examined “the veracity of children's accurate, unintentionally false, or intentionally false eyewitness reports” (Goodman et al., 2006 : 368). Like the study by Pathak and Thompson, it was undertaken in circumstances where the child's experience it was controlled. Young children reported being touched on the stomach, nose, or neck by a “defendant.” Some children who had not been touched during the play session were instructed to falsely claim that they had been. Comparisons were drawn between mock juror responses to live testimony, video of forensic interviews with a social worker, and the social worker testifying about what the child had said during the interview.

The conclusions might be considered disconcerting:

First, this study demonstrates, quite provocatively, that children coached to lie can maintain that lie in the face of repeated questions. In fact, children who were instructed to “fool” the interviewer (and the others in the mock trial) were often more consistent in their claims than children who really had been touched. … Second, our results indicate that adults, when faced with the task of determining whether unauthorized touching of a child occurred, were poor at distinguishing whether a particular child was lying or telling the truth. Furthermore, adults' abilities were neither helped nor hindered, for the most part, by seeing the child live or on videotape, or by hearing a social worker recount what the child said. … Third, [mock] jurors relied on predictable aspects of the children's accounts when making judgments about the veracity of the allegations … jurors tend to use witness consistency as an indicator of accurate statements. However, … ironically it was the liars—not the truth-tellers—who were more consistent, particularly in the two hearsay conditions (Goodman et al., 2006 : 390–1).

B. Expert opinion evidence

Opinion evidence is also subject to exclusion. As far as practically possible, witnesses should describe their sensory perceptions in concrete factual terms, without the overlay of interpretation or opinion. A very important exception to this exclusionary orientation is opinion evidence provided by experts. Where, by reason of “specialized knowledge,” a witness can provide a relevant opinion that is beyond the ken of the average juror, that opinion may be admissible.

Until quite recently the most important empirical work on experts was primarily qualitative or historical (e.g., Jasanoff, Jones, and Golan). Case studies, and a few surveys, examined the roles of expert evidence in public inquiries (e.g., Wynne), litigation clusters (e.g., Schuck, Green, and Sanders); miscarriages of justice (e.g., Nobles and Schiff; Dwyer, Neufeld, and Scheck; and Gross); civil litigation (e.g., Shuman, Champagne, and Whittaker); and civil justice procedures such as court-appointed experts and concurrent evidence (e.g., Cecil and Willging, and Edmond). Scholars, such as Monahan and Walker, were influential in documenting the legal uses of social scientific evidence. Empirical research, particularly quantitative work, has increased in recent years as long-standing concerns about partisanship, expense, comprehension and reliability have become more prominent, particularly in response to social and legal developments in the United States.

1. Admissibility decision-making in the United States

One important strand of empirical research has focused on the impact of the U.S. Supreme Court's Daubert v. Merrell Dow Pharmaceuticals, Inc . (1993) decision. Daubert was an appeal over the admissibility standard for expert evidence under the Federal Rules of Evidence (1975). There, the Court explained that scientific evidence must be both “relevant and reliable” and emphasized the trial judge's gate-keeping responsibility. The majority provided four criteria to help trial judges determine the reliability of scientific evidence. The criteria are whether the theory or technique: (1) has been tested (referring to Karl Popper's notion of “falsifiability”); (2) has been published and/or peer-reviewed; (3) has a known or potential rate of error; and (4) is “generally accepted” in the relevant specialist community. The last of these was drawn from Frye v. United States (1923).

Frequency with which reliability was addressed and evidence was found unreliable.

Reprinted with permission from Dixon and Gill ( 2002 : 273).

In the aftermath of Daubert , Dixon and Gill examined 399 decisions issued in civil proceedings between 1980 and 1999. Starting in the early 1990s the number of challenges to the reliability and admissibility of expert evidence began to rise (Figure 2 ). They concluded that the “standards for reliability have tightened” and judges “have become more watchful gatekeepers” against expert evidence proffered by plaintiffs.

The number of successful challenges began to decline after 1996–1997 which Dixon and Gill thought might be attributable to changes in the behavior of lawyers and parties as they gradually “tailored the evidence … to the new standards” (Dixon and Gill, 2002 : 299).

These findings were generally consistent with the results of investigations by Krafka et al. ( 2002 ). These scholars surveyed federal district court judges in 1991 (responses = 335) and 1998 (responses = 303) about their most recent experience with expert evidence in a civil case. More judges excluded expert evidence in 1998 (41 of cases) than in 1991 (25%). There was also an increased use of pre-trial admissibility hearings (that became known as Daubert hearings). Just over half of the judges reported using pre-trial hearings in 1991 whereas more than three-quarters reported using Daubert hearings in 1998 (p. 327 ).

Interestingly, Krafka et al. found that the actual Daubert criteria did not seem to play an important role in these developments:

Judges who excluded testimony in the recent survey did so most often because it was not relevant, the witness was not qualified, or the testimony would not have assisted the trier of fact. These reasons are similar to the reasons most frequently cited by judges in 1991, and they do not reflect the factors cited in Daubert (Krafka et al., 2002 : 330).

Further insights into the impact of Daubert were provided by Cheng and Yoon 2005 . They examined the rate at which defendants removed cases from state to federal courts in tort and product liability suits—which are usually dependent on expert evidence. Daubert is binding on all federal courts, but only some state courts. A higher rate of removal from non- Daubert states (e.g., Frye jurisdictions) than from Daubert states would suggest that defendants thought they had a greater chance of excluding plaintiffs' expert evidence under Daubert . Cheng and Yoon ( 2005 : 503) found that the difference was not statistically significant, and inferred that “debates about the practical merits and drawbacks of adopting a Frye versus a Daubert standard are largely superfluous.” This was not to deny that Daubert had an impact, but its exclusionary influence seems to extend to non- Daubert jurisdictions: “[T]he power of the Supreme Court's decision was not so much in its formal doctrinal test, but rather in its ability to create greater awareness of the problems of junk science” (Cheng and Yoon, 2005 : 503; Harris 2008 ).

2. The forensic “sciences ”

Most of the admissibility studies have focused on civil litigation. However, a study of criminal appellate decisions between 1988 and 1998 suggests that Daubert may have exerted a more limited impact on criminal proceedings. Groscup et al. ( 2002 ) observed more “discussion” of expert evidence in judgments after Daubert , but no change in the proportion of evidence excluded in criminal proceedings. These researchers also noted the “mysterious … lack of discussion” of the Daubert criteria, particularly “falsifiability, peer review, and error rates” (p. 353 ).

The “exclusionary ethos” associated with civil proceedings does not seem to have been applied, and certainly not with the same level of rigor, to forensic science evidence produced and relied upon by the state. Studies of commonly admitted forensic science techniques—including fingerprint, voice, image, bite mark, hair, and footprint comparisons, document examination, blood spatter analysis and so on—reveal that: most have not been validated; error rates are often unknown; “expert” witnesses are often poorly educated (relative to research scientists); systemic bias and exaggeration are ubiquitous; and limitations and problems, even if known, are not always disclosed or explained during criminal proceedings (Saks and Koehler, 2005 ).

A recent review of institutionalized forensic sciences by the U.S. National Academy of Sciences (NAS) lamented that:

With the exception of nuclear DNA analysis … no forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source (NAS, 2009 : 5). Figure 3 Open in new tab Download slide Analysis of incriminating evidence in DNA exoneration cases Reprinted with permission from Saks and Faigman ( 2008 : 172)

Reviews, such as those associated with Innocence Projects (Figure 3 ), suggest that mistaken or misleading forensic science evidence is a feature in many wrongful convictions (Garrett and Neufeld, 2009 ; Findley, 2008 ).

Several psychologists, lawyers, and scientists, responding to these disturbing revelations, have characterized the identification sciences as the “nonscience forensic sciences” (Saks and Faigman, 2008 : 149).

3. Surveys of judicial understanding of “science”

The focus on admissibility standards in recent decades has also generated interest in judicial understanding of the Daubert criteria. One explanation for the ostensible lack of engagement is provided by Gatowski, Dobbin, and their colleagues.

Gatowski et al. (2001) surveyed hundreds of judges about the Daubert criteria. They asked questions designed to elicit information about judicial understanding of falsification (“Falsif.”), error rates (“ER”), peer review and publication (“PR/ Pub.”) and general acceptance (“GA”). The results, presented in Figure 4 , led them to conclude:

[M]ost judges have a questionable level of understanding with respect to the basic concepts of science, or of most Daubert guidelines and their underlying scientific meaning, with the concepts of falsifiability and error rate particularly problematic for many judges (Dobbin et al., 2007 : 13).

Such results are often used to ground reforms, particularly proposals based on judicial re-education, although their implications for fact-finders are less clear.

“Understanding × Guideline”

Reprinted from Gatowski et al. ( 2001 : 445) with kind permission from Springer Science+Business Media.

4. (Reflexive) interventions in the “fingerprint wars ”

One interesting aspect of the controversy over the forensic sciences (and judicial scientific literacy) emerged out of recent contributions from post-Kuhnian Science & Technology Studies (STS). STS scholars form part of a scholarly tradition that values qualitative empirical investigation of what scientists, doctors and engineers actually do. On the basis of laboratory (and other workplace and institutional) studies they have concluded that scientists do not adhere, in any simple way, to a prescriptive scientific method doctrine (such as falsification) and are not consistently constrained by universal norms (such as disinterestedness and skepticism).

STS has much to say about law-science interactions (e.g., Jasanoff, 2008 ). Simon Cole's work on fingerprints is of particular interest in the present context. Cole has produced impressive scholarly accounts of problems with individualization, validation, and the social organization of fingerprint examiners. He has also appeared as an expert witness, contesting the admissibility and probative value of fingerprint evidence. Through a commentary on his participation in a Daubert hearing, Cole and Michael Lynch explored some of the dilemmas encountered when an STS scholar is confronted with a challenge to the admissibility of his own expert evidence (Lynch and Cole, 2005 ).

In People v. Hyatt (2001) Cole had to carefully negotiate questions about his expertise and interventions. Did it matter, for example, that he was not a fingerprint examiner? Was he a scientist or something else (such as historian, sociologist, or meta-expert)? What were the practical and theoretical difficulties involved in demarcating between genuine science and fingerprint evidence? Was it appropriate for an STS scholar to embrace the Daubert criteria in order to impugn fingerprint evidence? What are the practical limits of epistemic radicalism? And, could tacit knowledge and experience (central to many STS analyses) be used to excuse limitations with fingerprint identification? Resolving these and other dilemmas raised a series of tricky practical and theoretical issues.

Cole, whose opinion evidence has been admitted in other cases, approached his performance as an expert witness pragmatically. Nevertheless, the judge in Hyatt excluded his opinion evidence as “junk science” while admitting the impugned fingerprint evidence. This pejorative rejection provided ammunitionfor Cole's detractors in the continuing controversy over the reliability of fingerprint evidence. Discussing his ongoing “participation” more recently,Cole 2009 questioned the generalizability of particular experiences in court, stressed the protracted nature of the “campaign” to improve fingerprint evidence and placed his activities in the context of broader criticisms of the forensic sciences. Reflexively, and defensively, Cole also explained that exclusion and not “winning” in a Daubert hearing (or trial) cannot simply be equated with “failure.” The exclusion of his opinions in Hyatt , for example, may have actually discredited the judiciary and the institutionalized forensic sciences, thereby consolidating support and allies in the ongoing campaign (e.g., NAS, 2009 ). Cole's ( 2009 : 135) work demonstrates the difficulty of intervening to effect legal change and confirms that judgments about interventions can be just as “difficult and ambiguous” as the interventions themselves.

C. Subjective probability and human inference

The studies discussed in sub-sections A and B concern the operation of exclusionary rules. However, once evidence is admitted, fact-finders are largely unconstrained as to how it is used. This raises empirical and normative questions about how human inference operates and its accuracy.

Considerable research has been undertaken in this area over the last few decades. In experiments conducted by Daniel Kahneman, Amos Tversky, and others (e.g., Gilovich et al., 2002 ) the probabilistic reasoning of participants was found to depart from that prescribed by Bayes's Theorem—a rule of probability theory which provides a means by which a prior probability assessment can be updated to take account of the impact of additional evidence. Investigators suggested that, rather than reasoning probabilistically, humans may employ a limited number of inference mechanisms or “heuristics,” which sometimes produce a “bias.” Gerd Gigerenzer and others responded to the negative implications of the “heuristics and biases” (H&B) studies by emphasizing the ecological rationality of heuristics. For Gigerenzer and his colleagues, heuristics operate effectively—often more effectively than the “normative” Bayesian methods of the H&B researchers—in the environments in which they are typically employed (Gigerenzer and Selten, 2001 ; for legal implications, see Saks and Kidd, 1980 ; Gigerenzer and Engel, 2006 ).

Another significant contribution is Pennington and Hastie's “story model” of juror decision-making (1992). This descriptive model encompasses the juror's various tasks at trial, from the hearing of evidence right through to the selection of a verdict. According to the fact-finding component of the model, the juror settles upon a version of facts by organizing the evidence into one or more stories. The acceptability of a particular story is governed by a set of certainty principles, such as coverage, coherence and uniqueness.

Here we focus upon a strand of research informed by the H&B endeavors and connected with the story model. This work, inaugurated by Gary Wells 1992 , investigates the relationship between fact-finders' subjective probability assessments and liability verdicts.

Wells's experiments all took a similar form. In the first experiment, for example, participants were given the following information. A woman is suing the Blue Bus Company (BBC) for having caused the death of her dog. It was killed by the reckless driving of a bus driver. The woman is color blind. Only two bus companies use that road. BBC runs 80 of the buses on the road and the Grey Bus Company (GBC) runs the other 20. Wells's second experiment contained the same basic information except that the volume-of-traffic data was replaced with the evidence of a weigh-station attendant. He logged a blue bus passing along the road just before the accident, but his log entries are only 80 accurate. In both the volume-of-traffic and weigh-station-attendant versions, most participants said there was an 80 probability that the dog was hit by a blue bus. This would appear to satisfy the civil standard of proof requiring a “preponderance of probabilities.” However, whereas a clear majority of participants would hold BBC “liable” in the weigh-station-attendant version (67.1 ), very few participants would on the basis of the volume-of-traffic data (8.2).

Wells conducted further experiments with slight variations in order to understand why a high subjective probability was viewed as an insufficient basis for liability in certain situations. He tested the hypothesis that fact-finders have a preference for causally strong evidence by replacing volume-of-traffic data with accident statistics. The bus company that is involved in more accidents may, for example, have poorer drivers, which may be the explanation for the occurrence of this particular accident. However, with this evidence most participants were still not prepared to make a finding of liability. The hypothesis that jurors were concerned with distributional fairness was also rejected A verdict based upon the volume-of-traffic data would blame all the accidents on the company that happened to run the majority of buses along the route. Instead, the participants were given forensic evidence that tire tracks on the dog matched 8 out of 10 of BBC's buses, but only 2 out of 10 of GBC's buses. Utilizing this kind of evidence, a company's liability would tend to be in proportion with the company's involvement in accidents. Participants arrived at a “correct” probability assessment but remained reluctant to assign liability.

Wells's final experiment again involved matching tire tracks. On this occasion, the forensic witness testified that the technique is 80 reliable, and expressed his belief that the dog was run over by a BBC bus. In this tire-track-belief version most participants were prepared to attribute liability to BBC. Wells explained these results in terms of a preference for “bidirectional” evidence: “in order for evidence to have a significant impact on people's verdict preferences, one's hypothetical belief about the ultimate fact must affect one's belief about the evidence” (1992: 746). Only in the weigh-station-attendant and tire-track-belief variations would the fact that a GBC bus hit the dog invalidate the evidence. The evidence in the other variations, although making it more likely that a BBC bus hit the dog, is consistent with it having been a GBC bus.

Two groups of investigators subsequently confirmed the “Wells effect” but questioned Wells' fact-to-evidence hypothesis. Niedermeier et al. ( 1999 : 534) suggested that it attributed to participants a “rather complex process” for dealing with the evidence. Sykes and Johnson ( 1999 : 201) suggested that Wells's hypothesis “constitutes more of an analytical description of the phenomenon than a theoretical account of why it occurs.” Both groups sought an alternative and simpler explanation for the “Wells effect.” Experiments led to the rejection of an obvious possibility-that participants would prefer to base a finding of BBC's liability on the professed belief of a witness rather than statistical data (Sykes and Johnson, 1999 : 204, 208; Niedermeier et al., 1999 : 536).

Ultimately, both groups advanced similar explanations for the “Wells effect.” Niedermeier et al. suggested that participants may be less prepared to hold BBC liable where they can “more easily think of or about an alternative scenario in which [BBC] is not liable” (1999: 537). Sykes and Johnson contended that participants would be less prepared to believe an event where it is “relatively easy … to imagine an alternative … and to regard this alternative as a plausible scenario” (Sykes and Johnson, 1999 : 202). The evidence in all versions supports an 80 probability of BBC's liability but the evidence in the tire-track, accident-statistics, and volume-of-traffic versions expressly refers to the possibility of GBC liability, and therefore allows the creation of a scenario where a GBC bus hit the dog.

While the investigators identified a connection between their hypothesis and the “story model” of Pennington and Hastie, it should be noted that the present hypothesis is concerned with the ease or difficulty of imagining the “alternative scenario” or “counterfactual” rather than the liability scenario itself (Sykes and Johnson, 1999 : 205; Niedermeier et al., 1999 : 540 fn 5). For Sykes and Johnson, the decisive factor is “the difficulty of mutating [the liability scenario] into the alternative possibility involving the grey bus” (Sykes and Johnson, 1999 : 210).

One additional point is worth mentioning. Wells suggested that his experiments revealed a situation where the participants' subjective probabilities were “statistically correct” (1992: 739), unlike those in the H&B research, but participants were reluctant to find liability on this basis. Sykes and Johnson questioned this interpretation, explaining that “there may be a difference between calculated probability estimations based on participants' understanding of the rules of probability, and measures affecting participants' ‘gut feeling’, or subjective sense of the likelihood of an event” (1999: 201). To test this, Sykes and Johnson asked participants to record both the “probability” that the bus was blue, and “how certain they felt … [based] on intuition or their ‘gut feeling’ ” (p. 204 ). They found that, unlike “probability,” “[s]ubjective certainty was … significantly affected by evidence form, and … was a significant mediator of liability” (p. 209 ). This suggests a stronger link with the H&B work: the divergence between the subjective certainty and statistical probability might be viewed as a “bias.”

IV. Discussion

Having briefly reviewed several lines of empirical inquiry employing diverse methodologies—experiments, surveys, quantitative and qualitative approaches—we now discuss some of their limitations, and their implications and significance for the understanding and practice of law.

A. Abstraction from trial environment

Research on juror reasoning relied upon experiments that were, in various ways, abstracted from the trial environment, thereby raising issues of ecological validity. Do real jurors, for example, behave like the “mock jurors” of the experiments? And, do the experiments resemble real world trials. Much of the information in the hearsay and human inference studies was presented via written summaries, audio and video recordings or, at best, via live actors. (An exception is Goodman et al.'s (2006) experiments where, in one variation, children reported their actual experiences to mock jurors.) The presentation of evidence in such forms is questionable, and particularly problematic in relation to hearsay, because the rule is directed toward providing fact-finders with first-hand accounts. In some of the studies, the “non-hearsay evidence” was, strictly speaking, hearsay, while the “hearsay evidence” was actually hearsay upon hearsay.

A related problem is that the participants tended to be provided with a single piece of evidence in isolation. The subjective probability studies were, in part, concerned with the impact of naked statistical evidence on fact-finders. However, it is questionable whether any piece of evidence can be truly naked. As Sykes and Johnson observed, “our study does not assess whether the effects for our manipulation may be attenuated by other factors that exist in the information-rich forum of an actual trial” (1999: 211). And, here we should not overlook procedures and strategy as well as other evidence.

Many of the hearsay studies compared mock-juror responses to hearsay evidence with their responses to the observer/declarant's testimony with essentially the same content. But this misses another significant epistemic effect of the hearsay rule. By requiring the actual observer of the events to testify (rather than someone who merely heard about them), it may be possible to obtain further detail, particularly through cross-examination. This additional testimony may be relevant both to the events in issue and the declarant's credibility. As far as the comparison between hearsay evidence and declarant testimony is concerned, this additional testimony may confer a greater epistemic advantage to the fact-finder than the declarant's oath or demeanor.

B. Specificity of conditions, generalizability of conclusions

An empirical experiment inevitably involves a quite specific set of conditions raising a question as to the generalizability of any conclusions from that experiment. Of course, variations can be introduced through further experiments. Across the hearsay studies, for example, variations included: different ages of declarants and hearsay witnesses; different types of relationships between declarants and hearsay witnesses; differences in the status and experience of the hearsay witness; differences in whether mock jurors reached decisions individually or in groups; differences in the gender of mock jurors; differences in the nature of the experience leading to the declaration; different types of “crime”—whether theft, innocent touching, or sexual assault; differences in procedure—whether the hearsay witness (or declarant) was cross-examined; whether there was other evidence; and whether the evidence was presented by the state or the defense. However, it is not obvious that the experimental results are sufficiently consistent, coherent, or robust to provide a solid basis for drawing generalizations and conclusions justifying specific hearsay reforms with application to real world litigation.

Even where the studies do seem to identify potential problems with current rules and assumptions, as with the suggestibility of children or the recollection of interviewing techniques, questions remain about the precise implications for practice. Should we, for example, insist on recorded interviews with children at the earliest opportunity? If so, what happens when interviews are not recorded? Should we allow expert witnesses (e.g., the psychologists) to explain potential problems to fact-finders at trial? If so, should they be required to adhere strictly to the results of published studies or allowed to extrapolate? Should Daubert play a role? Would recourse to experimental psychologists as expert witnesses make any difference? And, how should the lengthening of trials, the added costs, and greater complexity in evidence (and judicial directions) be factored in?

Significantly, the “high degree of convergence” in empirical research as to the dangers of eyewitness testimony (Park and Saks, 2006 : 960) may be exceptional, as may be the relative clarity of the procedural prescriptions flowing from this research. Empirical legal studies often present as many questions as answers. And, proposals for law reform—such as additional judicial education—are often simplistic or naïve in political, sociological, and institutional terms.

C. Factual uncertainty and the benchmark problem

Evidence law is an institutionalized (and socially contingent) response to the inherent difficulty of arriving at an accurate version of past events. It is rare that the factual accuracy of inferences can be assessed, let alone the extent to which accuracy is advanced by particular evidentiary principles. Not insignificantly, DNA profiling, particularly when it provides the basis for exonerations, has enabled some verdicts to be benchmarked against the accused's actual innocence. Regrettably, interventions by Innocence Projects give criminal justice systems few grounds for complacency. Many wrongful convictions had, prior to DNA-based exoneration, survived multiple appeals and re-trials.

The benchmarking issue also arises for empirical researchers seeking to assess the epistemic value of an evidentiary rule, type of evidence or verdict. A number of the studies sought to draw normative conclusions about the accuracy with which human fact-finders handled particular types of evidence. The earlier hearsay studies concluded that mock jurors appropriately discounted hearsay evidence. Yet, in most of the experiments the “accuracy” of the hearsay evidence was unknown and unknowable—there was no benchmark, as the ultimate factual question was fictitious. Some investigators sought to respond to this issue by basing their experiment on real events. One study, for example, began with the staging of the actual event—an adult touching (or not touching) a child (Goodman et al., 2006 ). For obvious reasons, there are limits to the degree of realism that can be introduced to such experiments.

Some of the accounts of human inference have looked to Bayesian probability theory rather than objective fact as a benchmark. The H&B researchers, in particular, labeled departure from the calculus of probability, and Bayes's Theorem in particular, as a “bias.” Wells and others, in conducting their subjective probability experiments, were more concerned with developing a descriptive model. However, their work, revealing a similar departure, also implies that human reasoning in conditions of uncertainty is suboptimal. Perhaps there is a role for evidence law to intervene and address empirically-identified biases (Saks and Kidd, 1980 ).

An immediate difficulty with this proposal is that the Bayesian norm is highly contentious. Several legal commentators agree that “naked statistical evidence” provides an inadequate basis for liability. For them, a high probability figure must be supported by sufficient weight, detail and/or resilience (Stein, 2005 : 120; Ho, 2008 : 166). If a high base-rate probability is sufficient, what incentive is there to seek more specific evidence? And yet, subject to resource considerations, the enterprise of juridical proof is founded on the assumption that the more evidence the better. This preference underlies Benthamite criticisms of the exclusionary rules and, sometimes, their justifications. Recall that the hearsay rule can be rationalized on the basis that it increases the quality and quantity of evidence available to the fact-finder.

As noted in Section II, in recent decades Bentham's free-proof position has come to predominate, albeit indirectly. Daubert's tightening of admissibility requirements for expert evidence in civil proceedings represents something of an anomaly shaped by institutional pressures and socio-economic impressions and beliefs (more below).

D. Competing goals and values

A further issue arising from the empirical studies is the tendency to focus predominantly on the goal of factual accuracy. Prescriptions derived from the studies may provide benefits in this particular area, but these need to be weighed against potential costs elsewhere. Factual accuracy may be the primary goal of evidence law, but it is certainly not the only goal (e.g., Twining, 2006 : 76; Ho, 2008 : 339). Rules dealing with the dangers of eyewitness evidence may fall fairly squarely under the accuracy heading, but principle may implicate other goals. Depriving the fact-finder of probative evidence, via the hearsay rule may, on balance, be preferable to denying the opposing party the right to confront an accuser. The remoteness and lack of detail of hearsay or naked statistical evidence may make verdicts reliant upon them unfocused and impersonal, and hinder the effective expression of norms. A verdict based upon inaccessible or incomprehensible expert opinion may present similar problems.

It may be that the primacy of factual accuracy is such that many of these arguments can ultimately be dismissed. Our point, however, is that they cannot be ignored or disregarded altogether because concern about factual accuracy underpins only some of the goals, values and assumptions shaping rules of evidence and legal practice.

E. Bigger pictures

Empirical research and attendant legal reform should be sensitive to broader socio-political considerations and theory. STS perspectives, for example, may help to expand the focus from rules, processes and simplistic images of expertise to interests, institutions and social processes.

The majority in Daubert , for instance, supported its first criterion with reference to the work of Popper and Carl Hempel. The juxtaposition of these two irreconcilable philosophical accounts, and their eclectic combination with other, more sociological criteria (e.g., peer review and general acceptance), not only renders the Supreme Court's philosophical understanding open to doubt, but raises questions about the epistemological legitimacy of its admissibility jurisprudence (Haack, 2001 ). Yet, in their survey of judicial understanding of Daubert , Gatowski et al. based their assessments on folk versions of falsifiability and abstract questions rather than studying what judges in situ actually do. In Figure 4 the lines between understanding, misunderstanding and uncertainty are not merely blurred but conceptually suspect. Judicial responses to a survey instrument were assessed against idealized, and philosophically inflected, representations of science rather than empirical studies of actual scientific practice (see also Freckelton et al., 1999 ; cf. Edmond, 2005 ).

STS perspectives encourage us to ask: If Daubert does not represent a neutral vision of science, why were the particular criteria favored? And, are the criteria well suited to determining the admissibility of expert evidence in civil and criminal proceedings? Answers to such questions might help us to understand why Daubert and “gate-keeping” seem to have achieved such symbolic significance even though their conceptualizations of science and expertise appear simplistic, philosophically flawed and under-utilized.

One way to interpret the Supreme Court's admissibility jurisprudence is as a response to perceived problems with civil litigation, especially litigation “explosions,” “junk science,” excessive damages awards, and resulting deleterious economic effects (Haltom and McCann, 2004 ). Vigorous gate-keeping is a less overtly political intervention than changes to substantive tort or product liability laws and policies. Daubert provided judges with a means of regulating civil claims deemed “unworthy,” ideologically as much as factually, while publicly affirming their commitment to factual accuracy, the Seventh Amendment and the rule of law (Finley, 1999 ).

This more speculative reading of Daubert also brings trends in the criminal sphere into sharper relief. Trial and appellate judges have been relatively unresponsive to defense attempts to impugn incriminating scientific evidence. Expert evidence favoring the compensation of plaintiffs has been held to higher standards than forensic science evidence implicating defendants in criminal activities. This seems to reflect, directly and/or indirectly, socio-economic, and ideological concerns about excessive litigation as well, as the perceived need for more severe crime control.

There is obvious value in trying to integrate quantitative studies, and even surveys, into a more hermeneutic synthesis. These kinds of meta-analyses, while tentative and controvertible, generate new understandings and testable theories as well as questions about current practice and reform. They suggest, for example, that improving judicial scientific literacy might not be particularly effective. Confronted with a civil justice system purportedly “in crisis,” U.S. judges would probably have operationalized any admissibility standard more aggressively. If we reflect on admissibility trends (recall Figure 2 ) we find that in civil cases federal judges began to exclude expert evidence more proactively before Daubert , and practice in Frye jurisdictions was almost indistinguishable.

More critical approaches to expertise help to release scholars from slavish adherence to the descriptively dubious Daubert criteria, as well as polemical concepts like “junk science.” STS-inflected approaches allow for admissibility criteria to be indexed to the kinds of principles, values, and outcomes to which particular institutions and societies aspire. If, for example, we claim to value the presumption of innocence, fairness, and factual accuracy, then we should be more interested in the reliability of forensic science techniques. Rather than disguising our policy-political preferences in terms of appeals to purportedly proper definitions of science, demonizing the evidence of opponents as “junk science” or invoking long-standing practice (as with fingerprint evidence), it might be preferable to formulate strategic models of science and expertise based on explicit policy preferences and principle.

V. Rationalist and Empiricist Legacies

Most of those operating in the rationalist tradition have invoked or exploited the experiential and pragmatic nature of evidence and evidence law, though without much engagement or interest in empirical study. This is almost as true of legal scholars as it is of legal practitioners and judges (Park, 2003 ). Years spent in legal practice (or scholarship), so it is thought, gives practitioners and judges privileged exposure to both the real world and human nature .

While it might be fair to say that many of the contributions from empirical legal studies are provisional and their precise value for practice is uncertain or ambiguous, it would be equally unfair to suggest that they did not raise important issues worthy of serious consideration (Park and Saks, 2006 ). Moreover, by identifying problems with eyewitness evidence and the limitations of many of the forensic sciences, and in many other ways, empirical and experimental studies have substantially outpaced and repeatedly embarrassed legal experience . Nevertheless, in most jurisdictions judges have responded even to mature research traditions at best superficially and, at worst, with disdain shored up by complacent reference to collective experience or legal exceptionalism.

The obdurate indifference of lawyers, judges and policy-makers to empirical research on evidence law seems misguided (Leiter and Allen, 2001 ). It is difficult to know how to promote more principled and empirically calibrated approaches to evidence and proof. One response might be to encourage the most attentive and influential legal practitioners to participate in qualitative and quantitative forms of inquiry. The complexity and diversity of legal practice, along with its relative inaccessibility, makes multi-disciplinary investigation with research teams composed of empirical scholars and theorists, as well as lawyers and judges, a potentially productive, if practically and methodologically awkward, means of facilitating relevant real world research with direct bearing on practice (Edmond et al., 2009 ). It might also serve to remind us that law reform and empirical research should be related and ongoing.

Sir Francis Bacon (1561–1626), an early proponent of experimental natural philosophy and sometime Lord Chancellor of England, reputedly died from pneumonia after endeavoring to preserve the flesh of fowls with snow. Most of those involved with evidence law, particularly our lawyers, judges, and evidence scholars, seem to have taken more from Bacon's empirically induced fate than from his experimentally oriented philosophy. Perhaps ironically, commitment to legal experience places modern Anglo-American judges in a position similar to the Aristotelian schoolmen Bacon railed against. Unabated, indifference to empirical legal study is likely to reduce the social legitimacy of our legal institutions and undermine the fairness and accuracy of their rules, processes, and results.

Bentham, J. ( 1843 ). The Works of Jeremy Bentham , J. Bowring (ed.), 11 vols, Edinburgh: Simpkin, Marshall, & Co.

Google Scholar

Google Preview

Buck, J., Warren, A., and Brigham, J. ( 2004 ). “ When does quality count? Perceptions of Hearsay Testimony about Child Sexual Abuse Interviews, ” Law & Human Behavior 28: 599–621.

Cheng, E. and Yoon, A. ( 2005 ). “Does Frye or Daubert Matter? A Study of Scientific Admissibility Standards,” Virginia Law Review 91: 471–513.

Cole, S. ( 2009 ). “ A Cautionary Tale about Cautionary Tales about Intervention, ” Organization 16: 121–141.

Cole, S. and Dioso-Villa, R. ( 2009 ). “ Investigating the ‘CSI Effect’: Media and Litigation Crisis in Criminal Law, ” Stanford Law Review 61: 1335–1373.

Cranor, C. ( 2006 ). Toxic torts: Science, law and the possibility of justice , Cambridge: Cambridge University Press.

Damaška, M. ( 1997 ). Evidence Law Adrift , New Haven: Yale University Press.

Dixon, L. and Gill, B. ( 2002 ). “Changes in the Standards for Admitting Expert Evidence in Federal Civil Cases since the Daubert Decision,” Psychology, Public Policy & Law 8: 251–308.

Dobbin, S., Gatowski, S., Eyre, R., Dahir, V., Merlino, M., and Richardson, J. ( 2007 ). “ Federal and State Trial Judges on the Proffer and Presentation of Expert Evidence, ” The Justice System Journal 28: 1–15.

Edmond, G. ( 2005 ). “ Judging Surveys: Experts, Empirical Evidence and Law Reform, ” Federal Law Review 33: 95–139.

Edmond, G., Biber, K., Kemp, R., and Porter, G. ( 2009 ). “ Law's Looking Glass: Expert Identification Evidence Derived from Photographic and Video Images, ” Current Issues in Criminal Justice 20: 337–377.

Findley, K. ( 2008 ). “ Innocents at Risk: Adversary Imbalance, Forensic Science, and the Search for Truth, ” Seton Hall Law Review 38: 893–973.

Finley, L. ( 1999 ). “ Guarding the Gate to the Courthouse: How Trial Judges are Using their Evidentiary Screening Role to Remake Tort Causation Rules, ” DePaul University Law Review 49: 335–376.

Freckelton, I., Reddy, P., and Selby, H. ( 1999 ). Australian Judicial Perspectives on Expert Evidence: An Empirical Study , Carlton, VIC: AIJA.

Garrett, B. and Neufeld, P. ( 2009 ). “ Invalid Forensic Science Testimony and Wrongful Convictions, ” Virginia Law Review 95: 1–97.

Gatowski, S., Dobbin, S., Richardson, J., Ginsburg, G., Merlino, M., and Dahir, V. ( 2001 ). “Asking the Gatekeepers: A National Survey of Judges on Judging Expert Evidence in a post- Daubert World,” Law & Human Behavior 25: 433–458.

Gigerenzer, G. and Engel, C. (eds.) ( 2006 ). Heuristics and the Law , Cambridge, MA: MIT Press.

Gigerenzer, G. and Selten, R. (eds.) ( 2001 ). Bounded Rationality: The Adaptive Toolbox , Cambridge, MA: MIT Press.

Gilovich, T., Griffin, D., and Kahneman, D. (eds.) ( 2002 ). Heuristics and Biases: The Psychology of Intuitive Judgment , Cambridge: Cambridge University Press.

Gluckman, M. ( 1955 ). The judicial process among the Barotse of Northern Rhodesia , Manchester: University Press for the Rhodes Livingston Institute.

Golding, J., Alexander, M., and Stewart, T. ( 1999 ). “ The Effect of Hearsay Witness Age in a Child Sexual Assault Trial, ” Psychology, Public Policy & Law 5: 420–438.

Golding, J., Sanchez, R., and Sego, S. ( 1997 ). “ The Believability of Hearsay Testimony in a Child Sexual Assault Trial, ” Law & Human Behavior 21: 299–325.

Goodman, G., Myers, J., Qin, J., Quas, J., Castelli, P., Redlich, A., and Rogers, L. ( 2006 ). “ Hearsay versus children's testimony: Effects of truthful and deceptive statements on juror's decisions, ” Law & Human Behavior 30: 363–401.

Groscup, J., Penrod, S., Studebaker, C., and Huss, M. ( 2002 ). “The Effects of Daubert on the admissibility of expert testimony in state and federal criminal cases,” Psychology, Public Policy & Law 8: 339–372.

Haack, S. ( 2001 ). “ An Epistemologist in the Bramble-Bush: At the Supreme Court with Mr. Joiner, ” Journal of Health Politics, Policy & Law 26: 217–248.

Haltom, W. and McCann, M.. ( 2004 ) Distorting the law: Politics, Media and the litigation crisis , Chicago: University of Chicago Press.

Harris, R.C. ( 2008 ). Black Robes, White Coats: The Puzzle of Judicial Policy-making and Scientific Evidence , New Jersey: Rutgers University Press.

Heydon, J.D. ( 1984 ). Evidence: Cases and Materials , (2nd edn.), London: Butterworths.

Ho, H.L. ( 2008 ). A Philosophy of Evidence Law: Justice in the Search for Truth , Oxford: Oxford University Press.

Hutchins, R. and Sleisinger, D. ( 1928 ). “ Some observations on the law of evidence, ” Columbia Law Review 28: 432–440.

Jasanoff, S. ( 2008 ). “Making Order: Law and Science in Action,” in E. Hackett et al (eds.), The handbook of science and technology studies , (3rd edn.), Cambridge MA: MIT Press,761–786.

Johnson, M., Krafka, C., and Cecil, J. ( 2000 ). Expert testimony in federal civil trials: A preliminary analysis , Washington, DC: Federal Judicial Center.

Kahneman, D., Slovic, P., and Tversky, A. (eds.) ( 1982 ). Judgment under uncertainty: Heuristics and biases , Cambridge: Cambridge University Press.

Kovera, M., Park, R., and Penrod, S. ( 1992 ). “ Jurors' Perceptions of Eyewitness and Hearsay Evidence, ” Minnesota Law Review 76: 702–722.

Krafka, C., Dunn, M., Johnson, M., Cecil, J., and Miletich, D. ( 2002 ). “ Judge and Attorney Experiences, Practices, and Concerns Regarding Expert Testimony in Federal Civil Trials, ” Psychology, Public Policy & Law 8: 309–332.

Leiter, B. and Allen, R. ( 2001 ). “ Naturalized Epistemology and the Law of Evidence, ” Virginia Law Review 87: 1491–1550.

Lynch, M. and Cole, S. ( 2005 ). “ Science and Technology Studies on Trial: Dilemmas of Expertise, ” Social Studies of Science 35: 269–311.

McGough, L. ( 1999 ). “ Hearing and Believing Hearsay, ” Psychology, Public Policy & Law 5: 485–498.

Miene, P., Park, R., and Borgida, E. ( 1992 ). “ Juror Decision Making and the Evaluation of Hearsay Evidence, ” Minnesota Law Review 76: 83–701.

Munsterberg, H. ( 1908 ). On the Witness Stand. Essays on Psychology and Crime , New York: Doubleday.

NAS ( 2009 ). Strengthening the Forensic Sciences in the United States: A Path Forward , Washington, DC: The National Academies Press.

Niedermeier, K.E., Kerr, N.L., and Messé, L.A., ( 1999 ). “ Jurors' Use of Naked Statistical Evidence: Exploring Bases and Implications of the Wells Effect, ” Journal of Personality and Social Psychology 76: 533–542.

Paglia, A. and Schuller, R. ( 1998 ). “ Jurors' Use of Hearsay Evidence: The Effects of Type and Timing of Instructions, ” Law & Human Behavior 22: 501–518.

Park, R. ( 2003 ). “ Visions of Applying the Scientific Method to the Hearsay Rule, ” Michigan State Law Review 1149–1174.

Park, R. and Saks, M. ( 2006 ). “ Evidence Scholarship Reconsidered: Results of the Interdisciplinary Turn, ” Boston College Law Review 46: 949–1031.

Pathak, M., and Thompson, W. ( 1999 ). “ From Child to Witness to Jury: Effects of Suggestion on the Transmission and Evaluation of Hearsay, ” Psychology, Public Policy & Law 5: 372–387.

Pennington, N. and Hastie, R. ( 1992 ). “ Explaining the Evidence: Tests of the Story Model for Juror Decision Making, ” Journal of Personality and Social Psychology 62: 189–206.

Rakos, R. and Landsman, S. ( 1992 ). “ Researching the Hearsay Rule: Emerging Findings, General Issues, and Future Directions, ” Minnesota Law Review 76: 655–682.

Ross, D., Lindsay, R., and Marsil, D. ( 1999 ). “ The Impact of Hearsay Testimony on Conviction Rates in Trials of Child Sexual Abuse: Toward Balancing the Rights of Defendants and Child Witnesses, ” Psychology, Public Policy & Law 5: 439–455.

Saks, M. and Faigman, D. ( 2008 ). “ Failed Forensics: how forensic science lost its way and how it might yet find it, ” Annual Review of Law & Social Science 4: 149–171.

Saks, M. and Kidd, R.F. ( 1980 ). “ Human Information Processing and Adjudication: Trial by Heuristics, ” Law and Society Review 15: 123–160.

Saks, M. and Koehler, J. ( 2005 ). “The Coming Paradigm Shift in Forensic Identification Science, ” Science 309: 892–895.

Schuller, R. ( 1995 ). “ Expert Evidence and Hearsay: The Influence of ‘Secondhand’ Information on Jurors' Decisions, ” Law & Human Behavior 19: 345–362.

Stein, A. ( 2005 ). Foundations of Evidence Law , Oxford: Oxford University Press.

Sykes, D.L. and Johnson, J.T. ( 1999 ). “ Probabilistic Evidence Versus the Representation of an Event: The Curious Case of Mrs. Prob's Dog, ” Basic and Applied Social Psychology 21: 199–212.

Thayer, J.B. ( 1898 ). A Preliminary Treatise on Evidence at Common Law , Boston: Little Brown.

Thompson, W. and Pathak, M. ( 1999 ). “ Empirical Study of Hearsay Rules: Bridging the Gap Between Psychology and Law, ” Psychology, Public Policy & Law 5: 456–472.

Twining, W. ( 2006 ). Rethinking Evidence: Exploratory Essays , (2nd edn.), Cambridge: Cambridge University Press.

Tyler, T.R. ( 1990 ). Why people obey the law: Procedural Justice, Legitimacy, and Compliance , New Haven: Yale University Press.

Warren, A., Nunez, N., Keeney, J., Buck, J., and Smith, B. ( 2002 ). “ The Believability of Children and their Interviewers' Hearsay Testimony: When less is more, ” Journal of Applied Psychology 87: 846–857.

Warren, A.R. and Woodall, C.E. ( 1999 ). “ The Reliability of Hearsay Testimony: How well do interviewers recall their interviews with children?, ” Psychology, Public Policy, and Law 5: 355–371.

Wells, G.L. ( 1992 ). “ Naked Statistical Evidence of Liability, ” Journal of Personality and Social Psychology 62: 739–752.

Wells, G.L. and Quinlivan, D.S. ( 2009 ). “ Suggestive Eyewitness Identification Procedures and the Supreme Court's Reliability Test in Light of Eyewitness Science: 30 years later, ” Law and Human Behavior 33: 1–24.

Wigmore, J.H. ( 1940 ). A Treatise on the System of Evidence in Trials at Common Law (3rd edn.), Boston: Little Brown.

About Oxford Academic
Publish journals with us
University press partners
What we publish
New features
Open access
Institutional account management
Rights and permissions
Get help with access
Accessibility
Advertising
Media enquiries
Oxford University Press
Oxford Languages
University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

Cookie settings
Cookie policy
Privacy policy
Legal notice

This Feature Is Available To Subscribers Only

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

COMMENTS

Introduction
The phrase "empirical legal research" in the title, The Oxford Handbook of Empirical Legal Research, is designed both to reflect and to celebrate the healthy pluralism of empirical approaches to the study of law and legal phenomena. American legal realists were, perhaps, the first to appreciate the value and importance of, and to promote ...
42 Empirical Legal Research and Policy-making
Notes. Notes. List of Abbreviations Notes. Notes. Contributors and Editors Notes. Notes. Introduction Notes. Notes. Expand Part I Surveying EmpiricaL Legal Research 2 ... Empirical legal research (ELR) seeks to understand and explain how law works in the real world. Empirical research on law has become a recognized part of the social science ...
Empirical Legal Research: Nature, Features, and Expanding Horizons
This chapter discusses the use and merits of empirical legal research as an evidence-based method of conducting research that systematically carr. ... Epstein and King note: 6. What makes research empirical is that it is based on observations of the world, in other words, data, which is just a term for facts about the world. These facts may be ...
Statistics and Empirical Legal Studies Research Guide
Lee Epstein & Andrew D. Martin, An Introduction to Empirical Legal Research (2014) Mark A. Hall & Ronald F. Wright, Systematic Content Analysis of Judicial Opinions, 96 Calif. L. Rev. 63 (2008). Sarah E. Ryan, Teaching Empirical Legal Research Study Design: Topics & Resources, 23 Perspectives 152 (2015)
Teaching Empirical Legal Research
Empirical legal research can be taught as a specialized legal research course. The course can take one of three formats: 1) readings in empirical legal research, 2) basic empirical legal research skills, and 3) study design. The first two formats are the subject of this book.
The Oxford Handbook of Empirical Legal Research
The empirical study of law, legal systems and legal institutions is widely viewed as one of the most exciting and important intellectual developments in the modern history of legal research. Motivated by a conviction that legal phenomena can and should be understood not only in normative terms but also as social practices of political, economic and ethical significance, empirical legal ...
LibGuides: Empirical Legal Research Resources: Overview
Overview. This guide introduces selective resources and tools for conducting empirical legal research. These resources include: Statistics and Datasets: Resources available that collect statistical information. Journals and law reviews: Scholarly publications focused on empirical legal research. Treatises: Texts explaining and incorporating ...
Advanced introduction to empirical legal research (Elgar advanced
Herbert K. Kritzer's Advanced Introduction to Empirical Research Methods describes the defining features, history and purpose of Empirical Legal Research (ELR). The book adds to the series of Elgar...
Harvard Empirical Legal Studies Series
Overview. The Harvard Empirical Legal Studies (HELS) Series explores a range of empirical methods, both qualitative and quantitative, and their application in legal scholarship in different areas of the law.It is a platform for engaging with current empirical research, hearing from leading scholars working in a variety of fields, and developing ideas and empirical projects.
Empirical Legal Studies: Research Design
2021-2022 Autumn. Empirical Legal Studies: Research Design (7510): Empirical legal studies have become trendy in the U.S. and are now spreading to law faculties in other countries as well. The popular image of an empirical study is that it involves sophisticated statistical analysis of quantitative data. Often the author of the study starts ...
The Oxford Handbook of Empirical Legal Research
The phrase "empirical legal research" in the title, The Oxford Handbook of Empirical Legal Research, is designed both to reflect and to celebrate the healthy pluralism of empirical approaches to the study of law and legal phenomena. KW - Els movement. KW - Empirical investigation. KW - Empirical legal studies. KW - Legal systems. KW - Policing
38 Quantitative Approaches to Empirical Legal Research
T he title of this Chapter seems too wordy. Why call it doing "empirical legal research," and not simply doing "empirical research"? After all, regardless of whether empirical researchers are addressing a legal question or any other, they follow the same rules—the rules that enable them to draw inferences from the data they have collected (Epstein and King, 2002; King et al., 1994).
Statistics and Empirical Legal Studies Research Guide
In empirical legal research, there are four ways researchers commonly generate their own data: performing experiments, surveying, observing, and analyzing text. ... as when the researcher notes behavior that supports his or her hypothesis and fails to note behavior that doesn't. Id. at 80-81. Analyzing text can involve the simple extraction of ...
Empirical research in law
Undertaking empirical research in law can be a daunting task, one for which current undergraduate and postgraduate legal education does not provide a great deal of preparation. Yet the ability to undertake such research is valuable and, some suggest, in demand. Many areas of law, its operation and effects, can be usefully informed by empirical ...
Historical Roots of Empirical Legal Studies
1.6 Notes from the Desk of Sarah E. Ryan: My empirical research on the 'facebook' ... Until the late twentieth century, empirical legal research was an eclectic and undisciplined field. Content analysis was an early empirical research method, for instance, but most legal empiricists developed their analytic practices organically and without ...
PDF RESEARCH METHODOLOGY LEGAL RESEARCH
3. Legal Research. 4. Purpose of Legal Research. 4.1 Ascertaining the Law. 4.2 Highlighting Ambiguities and Gaps. 4.3 Determining Coherence, Stability and Consistency. 4.4 Social Auditing of Law. 4.5 Suggesting reforms. 5. Broad Nature of Legal Research 5.1 Quantitative Legal Research. 5.2 Qualitative Legal Research. 6. Kinds of Legal Research
41 Legal Theory and Empirical Research
L egal theory and empirical research into law are conducted independently of each other, each asking its own questions, using its own methods, and drawing its own conclusions. The aim of this Chapter is to examine the links between them and whether each could be of more use to the other than is now the case. Legal theory takes different forms, appears under different names, and varies as to ...
(Pdf) Legal Research Methodology: an Overview
Abstract:-. Research methodology is the process for direct approach through mixed types of research. techniques. The research approach supports the researcher to come across the research result ...
(PDF) Empirical research in law
Abstract. Undertaking empirical research in law can be a daunting task, one for which current undergraduate and postgraduate legal education does not provide a great deal of preparation. Yet the ...
An Empirical Study in Legal Methodology
Abstract: This empirical study delves into the realm of legal methodology, aiming to scrutinize the effectiveness of various approaches in legal research and analysis. The study employs a mixed-methods approach, combining qualitative and quantitative techniques to provide a comprehensive understanding of the subject matter.
An Introduction to Empirical Legal Research
This paper presents a meta-analysis of multiple Regression Analysis methods and its applications to data collection, interpretation, and visualization in the context of large data sets. 1. Some Preliminaries PART I: DESIGNING RESEARCH 2. Questions, Theories, Observable Implications 3. Measurement PART II: COLLECTING AND CODING DATA 4. Collecting Data 5. Coding Data PART III: ANALYZING DATA 6 ...
Evidence Law
A. Scope of evidence law. To varying degrees evidence law regulates the admissibility and presentation of evidence and fact-finding at trial. The trial is a highly formalized system of empirical enquiry, operating within a prescribed procedural framework. In a jury trial, the jury is responsible for fact-finding while the judge determines the ...
All you need to know about legal research
Conceptual legal research: Empirical legal research: Meaning: It is related to an abstract theory or idea. Generally used by philosophers and thinkers either to develop or re-interpret the existing concept. It is done by relying upon experience or observation alone, often without due regard for any system or theory.