meaning of empirical economic research

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case NPS+ Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research

Empirical Research: Definition, Methods, Types and Examples

Content Index

Empirical research: Definition

Empirical research: origin, quantitative research methods, qualitative research methods, steps for conducting empirical research, empirical research methodology cycle, advantages of empirical research, disadvantages of empirical research, why is there a need for empirical research.

Empirical research is defined as any research where conclusions of the study is strictly drawn from concretely empirical evidence, and therefore “verifiable” evidence.

This empirical evidence can be gathered using quantitative market research and  qualitative market research  methods.

For example: A research is being conducted to find out if listening to happy music in the workplace while working may promote creativity? An experiment is conducted by using a music website survey on a set of audience who are exposed to happy music and another set who are not listening to music at all, and the subjects are then observed. The results derived from such a research will give empirical evidence if it does promote creativity or not.

LEARN ABOUT: Behavioral Research

You must have heard the quote” I will not believe it unless I see it”. This came from the ancient empiricists, a fundamental understanding that powered the emergence of medieval science during the renaissance period and laid the foundation of modern science, as we know it today. The word itself has its roots in greek. It is derived from the greek word empeirikos which means “experienced”.

In today’s world, the word empirical refers to collection of data using evidence that is collected through observation or experience or by using calibrated scientific instruments. All of the above origins have one thing in common which is dependence of observation and experiments to collect data and test them to come up with conclusions.

LEARN ABOUT: Causal Research

Types and methodologies of empirical research

Empirical research can be conducted and analysed using qualitative or quantitative methods.

• Quantitative research : Quantitative research methods are used to gather information through numerical data. It is used to quantify opinions, behaviors or other defined variables . These are predetermined and are in a more structured format. Some of the commonly used methods are survey, longitudinal studies, polls, etc
• Qualitative research:   Qualitative research methods are used to gather non numerical data.  It is used to find meanings, opinions, or the underlying reasons from its subjects. These methods are unstructured or semi structured. The sample size for such a research is usually small and it is a conversational type of method to provide more insight or in-depth information about the problem Some of the most popular forms of methods are focus groups, experiments, interviews, etc.

Data collected from these will need to be analysed. Empirical evidence can also be analysed either quantitatively and qualitatively. Using this, the researcher can answer empirical questions which have to be clearly defined and answerable with the findings he has got. The type of research design used will vary depending on the field in which it is going to be used. Many of them might choose to do a collective research involving quantitative and qualitative method to better answer questions which cannot be studied in a laboratory setting.

LEARN ABOUT: Qualitative Research Questions and Questionnaires

Quantitative research methods aid in analyzing the empirical evidence gathered. By using these a researcher can find out if his hypothesis is supported or not.

• Survey research: Survey research generally involves a large audience to collect a large amount of data. This is a quantitative method having a predetermined set of closed questions which are pretty easy to answer. Because of the simplicity of such a method, high responses are achieved. It is one of the most commonly used methods for all kinds of research in today’s world.

Previously, surveys were taken face to face only with maybe a recorder. However, with advancement in technology and for ease, new mediums such as emails , or social media have emerged.

For example: Depletion of energy resources is a growing concern and hence there is a need for awareness about renewable energy. According to recent studies, fossil fuels still account for around 80% of energy consumption in the United States. Even though there is a rise in the use of green energy every year, there are certain parameters because of which the general population is still not opting for green energy. In order to understand why, a survey can be conducted to gather opinions of the general population about green energy and the factors that influence their choice of switching to renewable energy. Such a survey can help institutions or governing bodies to promote appropriate awareness and incentive schemes to push the use of greener energy.

Learn more: Renewable Energy Survey Template Descriptive Research vs Correlational Research

• Experimental research: In experimental research , an experiment is set up and a hypothesis is tested by creating a situation in which one of the variable is manipulated. This is also used to check cause and effect. It is tested to see what happens to the independent variable if the other one is removed or altered. The process for such a method is usually proposing a hypothesis, experimenting on it, analyzing the findings and reporting the findings to understand if it supports the theory or not.

For example: A particular product company is trying to find what is the reason for them to not be able to capture the market. So the organisation makes changes in each one of the processes like manufacturing, marketing, sales and operations. Through the experiment they understand that sales training directly impacts the market coverage for their product. If the person is trained well, then the product will have better coverage.

• Correlational research: Correlational research is used to find relation between two set of variables . Regression analysis is generally used to predict outcomes of such a method. It can be positive, negative or neutral correlation.

LEARN ABOUT: Level of Analysis

For example: Higher educated individuals will get higher paying jobs. This means higher education enables the individual to high paying job and less education will lead to lower paying jobs.

• Longitudinal study: Longitudinal study is used to understand the traits or behavior of a subject under observation after repeatedly testing the subject over a period of time. Data collected from such a method can be qualitative or quantitative in nature.

For example: A research to find out benefits of exercise. The target is asked to exercise everyday for a particular period of time and the results show higher endurance, stamina, and muscle growth. This supports the fact that exercise benefits an individual body.

• Cross sectional: Cross sectional study is an observational type of method, in which a set of audience is observed at a given point in time. In this type, the set of people are chosen in a fashion which depicts similarity in all the variables except the one which is being researched. This type does not enable the researcher to establish a cause and effect relationship as it is not observed for a continuous time period. It is majorly used by healthcare sector or the retail industry.

For example: A medical study to find the prevalence of under-nutrition disorders in kids of a given population. This will involve looking at a wide range of parameters like age, ethnicity, location, incomes  and social backgrounds. If a significant number of kids coming from poor families show under-nutrition disorders, the researcher can further investigate into it. Usually a cross sectional study is followed by a longitudinal study to find out the exact reason.

• Causal-Comparative research : This method is based on comparison. It is mainly used to find out cause-effect relationship between two variables or even multiple variables.

For example: A researcher measured the productivity of employees in a company which gave breaks to the employees during work and compared that to the employees of the company which did not give breaks at all.

LEARN ABOUT: Action Research

Some research questions need to be analysed qualitatively, as quantitative methods are not applicable there. In many cases, in-depth information is needed or a researcher may need to observe a target audience behavior, hence the results needed are in a descriptive analysis form. Qualitative research results will be descriptive rather than predictive. It enables the researcher to build or support theories for future potential quantitative research. In such a situation qualitative research methods are used to derive a conclusion to support the theory or hypothesis being studied.

LEARN ABOUT: Qualitative Interview

• Case study: Case study method is used to find more information through carefully analyzing existing cases. It is very often used for business research or to gather empirical evidence for investigation purpose. It is a method to investigate a problem within its real life context through existing cases. The researcher has to carefully analyse making sure the parameter and variables in the existing case are the same as to the case that is being investigated. Using the findings from the case study, conclusions can be drawn regarding the topic that is being studied.

For example: A report mentioning the solution provided by a company to its client. The challenges they faced during initiation and deployment, the findings of the case and solutions they offered for the problems. Such case studies are used by most companies as it forms an empirical evidence for the company to promote in order to get more business.

• Observational method:   Observational method is a process to observe and gather data from its target. Since it is a qualitative method it is time consuming and very personal. It can be said that observational research method is a part of ethnographic research which is also used to gather empirical evidence. This is usually a qualitative form of research, however in some cases it can be quantitative as well depending on what is being studied.

For example: setting up a research to observe a particular animal in the rain-forests of amazon. Such a research usually take a lot of time as observation has to be done for a set amount of time to study patterns or behavior of the subject. Another example used widely nowadays is to observe people shopping in a mall to figure out buying behavior of consumers.

• One-on-one interview: Such a method is purely qualitative and one of the most widely used. The reason being it enables a researcher get precise meaningful data if the right questions are asked. It is a conversational method where in-depth data can be gathered depending on where the conversation leads.

For example: A one-on-one interview with the finance minister to gather data on financial policies of the country and its implications on the public.

• Focus groups: Focus groups are used when a researcher wants to find answers to why, what and how questions. A small group is generally chosen for such a method and it is not necessary to interact with the group in person. A moderator is generally needed in case the group is being addressed in person. This is widely used by product companies to collect data about their brands and the product.

For example: A mobile phone manufacturer wanting to have a feedback on the dimensions of one of their models which is yet to be launched. Such studies help the company meet the demand of the customer and position their model appropriately in the market.

• Text analysis: Text analysis method is a little new compared to the other types. Such a method is used to analyse social life by going through images or words used by the individual. In today’s world, with social media playing a major part of everyone’s life, such a method enables the research to follow the pattern that relates to his study.

For example: A lot of companies ask for feedback from the customer in detail mentioning how satisfied are they with their customer support team. Such data enables the researcher to take appropriate decisions to make their support team better.

Sometimes a combination of the methods is also needed for some questions that cannot be answered using only one type of method especially when a researcher needs to gain a complete understanding of complex subject matter.

We recently published a blog that talks about examples of qualitative data in education ; why don’t you check it out for more ideas?

Since empirical research is based on observation and capturing experiences, it is important to plan the steps to conduct the experiment and how to analyse it. This will enable the researcher to resolve problems or obstacles which can occur during the experiment.

Step #1: Define the purpose of the research

This is the step where the researcher has to answer questions like what exactly do I want to find out? What is the problem statement? Are there any issues in terms of the availability of knowledge, data, time or resources. Will this research be more beneficial than what it will cost.

Before going ahead, a researcher has to clearly define his purpose for the research and set up a plan to carry out further tasks.

Step #2 : Supporting theories and relevant literature

The researcher needs to find out if there are theories which can be linked to his research problem . He has to figure out if any theory can help him support his findings. All kind of relevant literature will help the researcher to find if there are others who have researched this before, or what are the problems faced during this research. The researcher will also have to set up assumptions and also find out if there is any history regarding his research problem

Step #3: Creation of Hypothesis and measurement

Before beginning the actual research he needs to provide himself a working hypothesis or guess what will be the probable result. Researcher has to set up variables, decide the environment for the research and find out how can he relate between the variables.

Researcher will also need to define the units of measurements, tolerable degree for errors, and find out if the measurement chosen will be acceptable by others.

Step #4: Methodology, research design and data collection

In this step, the researcher has to define a strategy for conducting his research. He has to set up experiments to collect data which will enable him to propose the hypothesis. The researcher will decide whether he will need experimental or non experimental method for conducting the research. The type of research design will vary depending on the field in which the research is being conducted. Last but not the least, the researcher will have to find out parameters that will affect the validity of the research design. Data collection will need to be done by choosing appropriate samples depending on the research question. To carry out the research, he can use one of the many sampling techniques. Once data collection is complete, researcher will have empirical data which needs to be analysed.

LEARN ABOUT: Best Data Collection Tools

Step #5: Data Analysis and result

Data analysis can be done in two ways, qualitatively and quantitatively. Researcher will need to find out what qualitative method or quantitative method will be needed or will he need a combination of both. Depending on the unit of analysis of his data, he will know if his hypothesis is supported or rejected. Analyzing this data is the most important part to support his hypothesis.

Step #6: Conclusion

A report will need to be made with the findings of the research. The researcher can give the theories and literature that support his research. He can make suggestions or recommendations for further research on his topic.

A.D. de Groot, a famous dutch psychologist and a chess expert conducted some of the most notable experiments using chess in the 1940’s. During his study, he came up with a cycle which is consistent and now widely used to conduct empirical research. It consists of 5 phases with each phase being as important as the next one. The empirical cycle captures the process of coming up with hypothesis about how certain subjects work or behave and then testing these hypothesis against empirical data in a systematic and rigorous approach. It can be said that it characterizes the deductive approach to science. Following is the empirical cycle.

• Observation: At this phase an idea is sparked for proposing a hypothesis. During this phase empirical data is gathered using observation. For example: a particular species of flower bloom in a different color only during a specific season.
• Induction: Inductive reasoning is then carried out to form a general conclusion from the data gathered through observation. For example: As stated above it is observed that the species of flower blooms in a different color during a specific season. A researcher may ask a question “does the temperature in the season cause the color change in the flower?” He can assume that is the case, however it is a mere conjecture and hence an experiment needs to be set up to support this hypothesis. So he tags a few set of flowers kept at a different temperature and observes if they still change the color?
• Deduction: This phase helps the researcher to deduce a conclusion out of his experiment. This has to be based on logic and rationality to come up with specific unbiased results.For example: In the experiment, if the tagged flowers in a different temperature environment do not change the color then it can be concluded that temperature plays a role in changing the color of the bloom.
• Testing: This phase involves the researcher to return to empirical methods to put his hypothesis to the test. The researcher now needs to make sense of his data and hence needs to use statistical analysis plans to determine the temperature and bloom color relationship. If the researcher finds out that most flowers bloom a different color when exposed to the certain temperature and the others do not when the temperature is different, he has found support to his hypothesis. Please note this not proof but just a support to his hypothesis.
• Evaluation: This phase is generally forgotten by most but is an important one to keep gaining knowledge. During this phase the researcher puts forth the data he has collected, the support argument and his conclusion. The researcher also states the limitations for the experiment and his hypothesis and suggests tips for others to pick it up and continue a more in-depth research for others in the future. LEARN MORE: Population vs Sample

LEARN MORE: Population vs Sample

There is a reason why empirical research is one of the most widely used method. There are a few advantages associated with it. Following are a few of them.

• It is used to authenticate traditional research through various experiments and observations.
• This research methodology makes the research being conducted more competent and authentic.
• It enables a researcher understand the dynamic changes that can happen and change his strategy accordingly.
• The level of control in such a research is high so the researcher can control multiple variables.
• It plays a vital role in increasing internal validity .

Even though empirical research makes the research more competent and authentic, it does have a few disadvantages. Following are a few of them.

• Such a research needs patience as it can be very time consuming. The researcher has to collect data from multiple sources and the parameters involved are quite a few, which will lead to a time consuming research.
• Most of the time, a researcher will need to conduct research at different locations or in different environments, this can lead to an expensive affair.
• There are a few rules in which experiments can be performed and hence permissions are needed. Many a times, it is very difficult to get certain permissions to carry out different methods of this research.
• Collection of data can be a problem sometimes, as it has to be collected from a variety of sources through different methods.

LEARN ABOUT:  Social Communication Questionnaire

Empirical research is important in today’s world because most people believe in something only that they can see, hear or experience. It is used to validate multiple hypothesis and increase human knowledge and continue doing it to keep advancing in various fields.

For example: Pharmaceutical companies use empirical research to try out a specific drug on controlled groups or random groups to study the effect and cause. This way, they prove certain theories they had proposed for the specific drug. Such research is very important as sometimes it can lead to finding a cure for a disease that has existed for many years. It is useful in science and many other fields like history, social sciences, business, etc.

LEARN ABOUT: 12 Best Tools for Researchers

With the advancement in today’s world, empirical research has become critical and a norm in many fields to support their hypothesis and gain more knowledge. The methods mentioned above are very useful for carrying out such research. However, a number of new methods will keep coming up as the nature of new investigative questions keeps getting unique or changing.

Create a single source of real data with a built-for-insights platform. Store past data, add nuggets of insights, and import research data from various sources into a CRM for insights. Build on ever-growing research with a real-time dashboard in a unified research management platform to turn insights into knowledge.

LEARN MORE         FREE TRIAL

MORE LIKE THIS

Raked Weighting: A Key Tool for Accurate Survey Results

May 31, 2024

Top 8 Data Trends to Understand the Future of Data

May 30, 2024

Top 12 Interactive Presentation Software to Engage Your User

May 29, 2024

Trend Report: Guide for Market Dynamics & Strategic Analysis

Other categories.

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Uncategorized
• Video Learning Series
• What’s Coming Up
• Workforce Intelligence

What is Empirical Research? Definition, Methods, Examples

Appinio Research · 09.02.2024 · 36min read

Ever wondered how we gather the facts, unveil hidden truths, and make informed decisions in a world filled with questions? Empirical research holds the key.

In this guide, we'll delve deep into the art and science of empirical research, unraveling its methods, mysteries, and manifold applications. From defining the core principles to mastering data analysis and reporting findings, we're here to equip you with the knowledge and tools to navigate the empirical landscape.

What is Empirical Research?

Empirical research is the cornerstone of scientific inquiry, providing a systematic and structured approach to investigating the world around us. It is the process of gathering and analyzing empirical or observable data to test hypotheses, answer research questions, or gain insights into various phenomena. This form of research relies on evidence derived from direct observation or experimentation, allowing researchers to draw conclusions based on real-world data rather than purely theoretical or speculative reasoning.

Characteristics of Empirical Research

Empirical research is characterized by several key features:

• Observation and Measurement : It involves the systematic observation or measurement of variables, events, or behaviors.
• Data Collection : Researchers collect data through various methods, such as surveys, experiments, observations, or interviews.
• Testable Hypotheses : Empirical research often starts with testable hypotheses that are evaluated using collected data.
• Quantitative or Qualitative Data : Data can be quantitative (numerical) or qualitative (non-numerical), depending on the research design.
• Statistical Analysis : Quantitative data often undergo statistical analysis to determine patterns , relationships, or significance.
• Objectivity and Replicability : Empirical research strives for objectivity, minimizing researcher bias . It should be replicable, allowing other researchers to conduct the same study to verify results.
• Conclusions and Generalizations : Empirical research generates findings based on data and aims to make generalizations about larger populations or phenomena.

Importance of Empirical Research

Empirical research plays a pivotal role in advancing knowledge across various disciplines. Its importance extends to academia, industry, and society as a whole. Here are several reasons why empirical research is essential:

• Evidence-Based Knowledge : Empirical research provides a solid foundation of evidence-based knowledge. It enables us to test hypotheses, confirm or refute theories, and build a robust understanding of the world.
• Scientific Progress : In the scientific community, empirical research fuels progress by expanding the boundaries of existing knowledge. It contributes to the development of theories and the formulation of new research questions.
• Problem Solving : Empirical research is instrumental in addressing real-world problems and challenges. It offers insights and data-driven solutions to complex issues in fields like healthcare, economics, and environmental science.
• Informed Decision-Making : In policymaking, business, and healthcare, empirical research informs decision-makers by providing data-driven insights. It guides strategies, investments, and policies for optimal outcomes.
• Quality Assurance : Empirical research is essential for quality assurance and validation in various industries, including pharmaceuticals, manufacturing, and technology. It ensures that products and processes meet established standards.
• Continuous Improvement : Businesses and organizations use empirical research to evaluate performance, customer satisfaction, and product effectiveness. This data-driven approach fosters continuous improvement and innovation.
• Human Advancement : Empirical research in fields like medicine and psychology contributes to the betterment of human health and well-being. It leads to medical breakthroughs, improved therapies, and enhanced psychological interventions.
• Critical Thinking and Problem Solving : Engaging in empirical research fosters critical thinking skills, problem-solving abilities, and a deep appreciation for evidence-based decision-making.

Empirical research empowers us to explore, understand, and improve the world around us. It forms the bedrock of scientific inquiry and drives progress in countless domains, shaping our understanding of both the natural and social sciences.

How to Conduct Empirical Research?

So, you've decided to dive into the world of empirical research. Let's begin by exploring the crucial steps involved in getting started with your research project.

1. Select a Research Topic

Selecting the right research topic is the cornerstone of a successful empirical study. It's essential to choose a topic that not only piques your interest but also aligns with your research goals and objectives. Here's how to go about it:

• Identify Your Interests : Start by reflecting on your passions and interests. What topics fascinate you the most? Your enthusiasm will be your driving force throughout the research process.
• Brainstorm Ideas : Engage in brainstorming sessions to generate potential research topics. Consider the questions you've always wanted to answer or the issues that intrigue you.
• Relevance and Significance : Assess the relevance and significance of your chosen topic. Does it contribute to existing knowledge? Is it a pressing issue in your field of study or the broader community?
• Feasibility : Evaluate the feasibility of your research topic. Do you have access to the necessary resources, data, and participants (if applicable)?

2. Formulate Research Questions

Once you've narrowed down your research topic, the next step is to formulate clear and precise research questions . These questions will guide your entire research process and shape your study's direction. To create effective research questions:

• Specificity : Ensure that your research questions are specific and focused. Vague or overly broad questions can lead to inconclusive results.
• Relevance : Your research questions should directly relate to your chosen topic. They should address gaps in knowledge or contribute to solving a particular problem.
• Testability : Ensure that your questions are testable through empirical methods. You should be able to gather data and analyze it to answer these questions.
• Avoid Bias : Craft your questions in a way that avoids leading or biased language. Maintain neutrality to uphold the integrity of your research.

3. Review Existing Literature

Before you embark on your empirical research journey, it's essential to immerse yourself in the existing body of literature related to your chosen topic. This step, often referred to as a literature review, serves several purposes:

• Contextualization : Understand the historical context and current state of research in your field. What have previous studies found, and what questions remain unanswered?
• Identifying Gaps : Identify gaps or areas where existing research falls short. These gaps will help you formulate meaningful research questions and hypotheses.
• Theory Development : If your study is theoretical, consider how existing theories apply to your topic. If it's empirical, understand how previous studies have approached data collection and analysis.
• Methodological Insights : Learn from the methodologies employed in previous research. What methods were successful, and what challenges did researchers face?

4. Define Variables

Variables are fundamental components of empirical research. They are the factors or characteristics that can change or be manipulated during your study. Properly defining and categorizing variables is crucial for the clarity and validity of your research. Here's what you need to know:

• Independent Variables : These are the variables that you, as the researcher, manipulate or control. They are the "cause" in cause-and-effect relationships.
• Dependent Variables : Dependent variables are the outcomes or responses that you measure or observe. They are the "effect" influenced by changes in independent variables.
• Operational Definitions : To ensure consistency and clarity, provide operational definitions for your variables. Specify how you will measure or manipulate each variable.
• Control Variables : In some studies, controlling for other variables that may influence your dependent variable is essential. These are known as control variables.

Understanding these foundational aspects of empirical research will set a solid foundation for the rest of your journey. Now that you've grasped the essentials of getting started, let's delve deeper into the intricacies of research design.

Empirical Research Design

Now that you've selected your research topic, formulated research questions, and defined your variables, it's time to delve into the heart of your empirical research journey – research design . This pivotal step determines how you will collect data and what methods you'll employ to answer your research questions. Let's explore the various facets of research design in detail.

Types of Empirical Research

Empirical research can take on several forms, each with its own unique approach and methodologies. Understanding the different types of empirical research will help you choose the most suitable design for your study. Here are some common types:

• Experimental Research : In this type, researchers manipulate one or more independent variables to observe their impact on dependent variables. It's highly controlled and often conducted in a laboratory setting.
• Observational Research : Observational research involves the systematic observation of subjects or phenomena without intervention. Researchers are passive observers, documenting behaviors, events, or patterns.
• Survey Research : Surveys are used to collect data through structured questionnaires or interviews. This method is efficient for gathering information from a large number of participants.
• Case Study Research : Case studies focus on in-depth exploration of one or a few cases. Researchers gather detailed information through various sources such as interviews, documents, and observations.
• Qualitative Research : Qualitative research aims to understand behaviors, experiences, and opinions in depth. It often involves open-ended questions, interviews, and thematic analysis.
• Quantitative Research : Quantitative research collects numerical data and relies on statistical analysis to draw conclusions. It involves structured questionnaires, experiments, and surveys.

Your choice of research type should align with your research questions and objectives. Experimental research, for example, is ideal for testing cause-and-effect relationships, while qualitative research is more suitable for exploring complex phenomena.

Experimental Design

Experimental research is a systematic approach to studying causal relationships. It's characterized by the manipulation of one or more independent variables while controlling for other factors. Here are some key aspects of experimental design:

• Control and Experimental Groups : Participants are randomly assigned to either a control group or an experimental group. The independent variable is manipulated for the experimental group but not for the control group.
• Randomization : Randomization is crucial to eliminate bias in group assignment. It ensures that each participant has an equal chance of being in either group.
• Hypothesis Testing : Experimental research often involves hypothesis testing. Researchers formulate hypotheses about the expected effects of the independent variable and use statistical analysis to test these hypotheses.

Observational Design

Observational research entails careful and systematic observation of subjects or phenomena. It's advantageous when you want to understand natural behaviors or events. Key aspects of observational design include:

• Participant Observation : Researchers immerse themselves in the environment they are studying. They become part of the group being observed, allowing for a deep understanding of behaviors.
• Non-Participant Observation : In non-participant observation, researchers remain separate from the subjects. They observe and document behaviors without direct involvement.
• Data Collection Methods : Observational research can involve various data collection methods, such as field notes, video recordings, photographs, or coding of observed behaviors.

Survey Design

Surveys are a popular choice for collecting data from a large number of participants. Effective survey design is essential to ensure the validity and reliability of your data. Consider the following:

• Questionnaire Design : Create clear and concise questions that are easy for participants to understand. Avoid leading or biased questions.
• Sampling Methods : Decide on the appropriate sampling method for your study, whether it's random, stratified, or convenience sampling.
• Data Collection Tools : Choose the right tools for data collection, whether it's paper surveys, online questionnaires, or face-to-face interviews.

Case Study Design

Case studies are an in-depth exploration of one or a few cases to gain a deep understanding of a particular phenomenon. Key aspects of case study design include:

• Single Case vs. Multiple Case Studies : Decide whether you'll focus on a single case or multiple cases. Single case studies are intensive and allow for detailed examination, while multiple case studies provide comparative insights.
• Data Collection Methods : Gather data through interviews, observations, document analysis, or a combination of these methods.

Qualitative vs. Quantitative Research

In empirical research, you'll often encounter the distinction between qualitative and quantitative research . Here's a closer look at these two approaches:

• Qualitative Research : Qualitative research seeks an in-depth understanding of human behavior, experiences, and perspectives. It involves open-ended questions, interviews, and the analysis of textual or narrative data. Qualitative research is exploratory and often used when the research question is complex and requires a nuanced understanding.
• Quantitative Research : Quantitative research collects numerical data and employs statistical analysis to draw conclusions. It involves structured questionnaires, experiments, and surveys. Quantitative research is ideal for testing hypotheses and establishing cause-and-effect relationships.

Understanding the various research design options is crucial in determining the most appropriate approach for your study. Your choice should align with your research questions, objectives, and the nature of the phenomenon you're investigating.

Data Collection for Empirical Research

Now that you've established your research design, it's time to roll up your sleeves and collect the data that will fuel your empirical research. Effective data collection is essential for obtaining accurate and reliable results.

Sampling Methods

Sampling methods are critical in empirical research, as they determine the subset of individuals or elements from your target population that you will study. Here are some standard sampling methods:

• Random Sampling : Random sampling ensures that every member of the population has an equal chance of being selected. It minimizes bias and is often used in quantitative research.
• Stratified Sampling : Stratified sampling involves dividing the population into subgroups or strata based on specific characteristics (e.g., age, gender, location). Samples are then randomly selected from each stratum, ensuring representation of all subgroups.
• Convenience Sampling : Convenience sampling involves selecting participants who are readily available or easily accessible. While it's convenient, it may introduce bias and limit the generalizability of results.
• Snowball Sampling : Snowball sampling is instrumental when studying hard-to-reach or hidden populations. One participant leads you to another, creating a "snowball" effect. This method is common in qualitative research.
• Purposive Sampling : In purposive sampling, researchers deliberately select participants who meet specific criteria relevant to their research questions. It's often used in qualitative studies to gather in-depth information.

The choice of sampling method depends on the nature of your research, available resources, and the degree of precision required. It's crucial to carefully consider your sampling strategy to ensure that your sample accurately represents your target population.

Data Collection Instruments

Data collection instruments are the tools you use to gather information from your participants or sources. These instruments should be designed to capture the data you need accurately. Here are some popular data collection instruments:

• Questionnaires : Questionnaires consist of structured questions with predefined response options. When designing questionnaires, consider the clarity of questions, the order of questions, and the response format (e.g., Likert scale , multiple-choice).
• Interviews : Interviews involve direct communication between the researcher and participants. They can be structured (with predetermined questions) or unstructured (open-ended). Effective interviews require active listening and probing for deeper insights.
• Observations : Observations entail systematically and objectively recording behaviors, events, or phenomena. Researchers must establish clear criteria for what to observe, how to record observations, and when to observe.
• Surveys : Surveys are a common data collection instrument for quantitative research. They can be administered through various means, including online surveys, paper surveys, and telephone surveys.
• Documents and Archives : In some cases, data may be collected from existing documents, records, or archives. Ensure that the sources are reliable, relevant, and properly documented.

To streamline your process and gather insights with precision and efficiency, consider leveraging innovative tools like Appinio . With Appinio's intuitive platform, you can harness the power of real-time consumer data to inform your research decisions effectively. Whether you're conducting surveys, interviews, or observations, Appinio empowers you to define your target audience, collect data from diverse demographics, and analyze results seamlessly.

By incorporating Appinio into your data collection toolkit, you can unlock a world of possibilities and elevate the impact of your empirical research. Ready to revolutionize your approach to data collection?

Book a Demo

Data Collection Procedures

Data collection procedures outline the step-by-step process for gathering data. These procedures should be meticulously planned and executed to maintain the integrity of your research.

• Training : If you have a research team, ensure that they are trained in data collection methods and protocols. Consistency in data collection is crucial.
• Pilot Testing : Before launching your data collection, conduct a pilot test with a small group to identify any potential problems with your instruments or procedures. Make necessary adjustments based on feedback.
• Data Recording : Establish a systematic method for recording data. This may include timestamps, codes, or identifiers for each data point.
• Data Security : Safeguard the confidentiality and security of collected data. Ensure that only authorized individuals have access to the data.
• Data Storage : Properly organize and store your data in a secure location, whether in physical or digital form. Back up data to prevent loss.

Ethical Considerations

Ethical considerations are paramount in empirical research, as they ensure the well-being and rights of participants are protected.

• Informed Consent : Obtain informed consent from participants, providing clear information about the research purpose, procedures, risks, and their right to withdraw at any time.
• Privacy and Confidentiality : Protect the privacy and confidentiality of participants. Ensure that data is anonymized and sensitive information is kept confidential.
• Beneficence : Ensure that your research benefits participants and society while minimizing harm. Consider the potential risks and benefits of your study.
• Honesty and Integrity : Conduct research with honesty and integrity. Report findings accurately and transparently, even if they are not what you expected.
• Respect for Participants : Treat participants with respect, dignity, and sensitivity to cultural differences. Avoid any form of coercion or manipulation.
• Institutional Review Board (IRB) : If required, seek approval from an IRB or ethics committee before conducting your research, particularly when working with human participants.

Adhering to ethical guidelines is not only essential for the ethical conduct of research but also crucial for the credibility and validity of your study. Ethical research practices build trust between researchers and participants and contribute to the advancement of knowledge with integrity.

With a solid understanding of data collection, including sampling methods, instruments, procedures, and ethical considerations, you are now well-equipped to gather the data needed to answer your research questions.

Empirical Research Data Analysis

Now comes the exciting phase of data analysis, where the raw data you've diligently collected starts to yield insights and answers to your research questions. We will explore the various aspects of data analysis, from preparing your data to drawing meaningful conclusions through statistics and visualization.

Data Preparation

Data preparation is the crucial first step in data analysis. It involves cleaning, organizing, and transforming your raw data into a format that is ready for analysis. Effective data preparation ensures the accuracy and reliability of your results.

• Data Cleaning : Identify and rectify errors, missing values, and inconsistencies in your dataset. This may involve correcting typos, removing outliers, and imputing missing data.
• Data Coding : Assign numerical values or codes to categorical variables to make them suitable for statistical analysis. For example, converting "Yes" and "No" to 1 and 0.
• Data Transformation : Transform variables as needed to meet the assumptions of the statistical tests you plan to use. Common transformations include logarithmic or square root transformations.
• Data Integration : If your data comes from multiple sources, integrate it into a unified dataset, ensuring that variables match and align.
• Data Documentation : Maintain clear documentation of all data preparation steps, as well as the rationale behind each decision. This transparency is essential for replicability.

Effective data preparation lays the foundation for accurate and meaningful analysis. It allows you to trust the results that will follow in the subsequent stages.

Descriptive Statistics

Descriptive statistics help you summarize and make sense of your data by providing a clear overview of its key characteristics. These statistics are essential for understanding the central tendencies, variability, and distribution of your variables. Descriptive statistics include:

• Measures of Central Tendency : These include the mean (average), median (middle value), and mode (most frequent value). They help you understand the typical or central value of your data.
• Measures of Dispersion : Measures like the range, variance, and standard deviation provide insights into the spread or variability of your data points.
• Frequency Distributions : Creating frequency distributions or histograms allows you to visualize the distribution of your data across different values or categories.

Descriptive statistics provide the initial insights needed to understand your data's basic characteristics, which can inform further analysis.

Inferential Statistics

Inferential statistics take your analysis to the next level by allowing you to make inferences or predictions about a larger population based on your sample data. These methods help you test hypotheses and draw meaningful conclusions. Key concepts in inferential statistics include:

• Hypothesis Testing : Hypothesis tests (e.g., t-tests, chi-squared tests) help you determine whether observed differences or associations in your data are statistically significant or occurred by chance.
• Confidence Intervals : Confidence intervals provide a range within which population parameters (e.g., population mean) are likely to fall based on your sample data.
• Regression Analysis : Regression models (linear, logistic, etc.) help you explore relationships between variables and make predictions.
• Analysis of Variance (ANOVA) : ANOVA tests are used to compare means between multiple groups, allowing you to assess whether differences are statistically significant.

Inferential statistics are powerful tools for drawing conclusions from your data and assessing the generalizability of your findings to the broader population.

Qualitative Data Analysis

Qualitative data analysis is employed when working with non-numerical data, such as text, interviews, or open-ended survey responses. It focuses on understanding the underlying themes, patterns, and meanings within qualitative data. Qualitative analysis techniques include:

• Thematic Analysis : Identifying and analyzing recurring themes or patterns within textual data.
• Content Analysis : Categorizing and coding qualitative data to extract meaningful insights.
• Grounded Theory : Developing theories or frameworks based on emergent themes from the data.
• Narrative Analysis : Examining the structure and content of narratives to uncover meaning.

Qualitative data analysis provides a rich and nuanced understanding of complex phenomena and human experiences.

Data Visualization

Data visualization is the art of representing data graphically to make complex information more understandable and accessible. Effective data visualization can reveal patterns, trends, and outliers in your data. Common types of data visualization include:

• Bar Charts and Histograms : Used to display the distribution of categorical data or discrete data .
• Line Charts : Ideal for showing trends and changes in data over time.
• Scatter Plots : Visualize relationships and correlations between two variables.
• Pie Charts : Display the composition of a whole in terms of its parts.
• Heatmaps : Depict patterns and relationships in multidimensional data through color-coding.
• Box Plots : Provide a summary of the data distribution, including outliers.
• Interactive Dashboards : Create dynamic visualizations that allow users to explore data interactively.

Data visualization not only enhances your understanding of the data but also serves as a powerful communication tool to convey your findings to others.

As you embark on the data analysis phase of your empirical research, remember that the specific methods and techniques you choose will depend on your research questions, data type, and objectives. Effective data analysis transforms raw data into valuable insights, bringing you closer to the answers you seek.

How to Report Empirical Research Results?

At this stage, you get to share your empirical research findings with the world. Effective reporting and presentation of your results are crucial for communicating your research's impact and insights.

1. Write the Research Paper

Writing a research paper is the culmination of your empirical research journey. It's where you synthesize your findings, provide context, and contribute to the body of knowledge in your field.

• Title and Abstract : Craft a clear and concise title that reflects your research's essence. The abstract should provide a brief summary of your research objectives, methods, findings, and implications.
• Introduction : In the introduction, introduce your research topic, state your research questions or hypotheses, and explain the significance of your study. Provide context by discussing relevant literature.
• Methods : Describe your research design, data collection methods, and sampling procedures. Be precise and transparent, allowing readers to understand how you conducted your study.
• Results : Present your findings in a clear and organized manner. Use tables, graphs, and statistical analyses to support your results. Avoid interpreting your findings in this section; focus on the presentation of raw data.
• Discussion : Interpret your findings and discuss their implications. Relate your results to your research questions and the existing literature. Address any limitations of your study and suggest avenues for future research.
• Conclusion : Summarize the key points of your research and its significance. Restate your main findings and their implications.
• References : Cite all sources used in your research following a specific citation style (e.g., APA, MLA, Chicago). Ensure accuracy and consistency in your citations.
• Appendices : Include any supplementary material, such as questionnaires, data coding sheets, or additional analyses, in the appendices.

Writing a research paper is a skill that improves with practice. Ensure clarity, coherence, and conciseness in your writing to make your research accessible to a broader audience.

2. Create Visuals and Tables

Visuals and tables are powerful tools for presenting complex data in an accessible and understandable manner.

• Clarity : Ensure that your visuals and tables are clear and easy to interpret. Use descriptive titles and labels.
• Consistency : Maintain consistency in formatting, such as font size and style, across all visuals and tables.
• Appropriateness : Choose the most suitable visual representation for your data. Bar charts, line graphs, and scatter plots work well for different types of data.
• Simplicity : Avoid clutter and unnecessary details. Focus on conveying the main points.
• Accessibility : Make sure your visuals and tables are accessible to a broad audience, including those with visual impairments.
• Captions : Include informative captions that explain the significance of each visual or table.

Compelling visuals and tables enhance the reader's understanding of your research and can be the key to conveying complex information efficiently.

3. Interpret Findings

Interpreting your findings is where you bridge the gap between data and meaning. It's your opportunity to provide context, discuss implications, and offer insights. When interpreting your findings:

• Relate to Research Questions : Discuss how your findings directly address your research questions or hypotheses.
• Compare with Literature : Analyze how your results align with or deviate from previous research in your field. What insights can you draw from these comparisons?
• Discuss Limitations : Be transparent about the limitations of your study. Address any constraints, biases, or potential sources of error.
• Practical Implications : Explore the real-world implications of your findings. How can they be applied or inform decision-making?
• Future Research Directions : Suggest areas for future research based on the gaps or unanswered questions that emerged from your study.

Interpreting findings goes beyond simply presenting data; it's about weaving a narrative that helps readers grasp the significance of your research in the broader context.

With your research paper written, structured, and enriched with visuals, and your findings expertly interpreted, you are now prepared to communicate your research effectively. Sharing your insights and contributing to the body of knowledge in your field is a significant accomplishment in empirical research.

Examples of Empirical Research

To solidify your understanding of empirical research, let's delve into some real-world examples across different fields. These examples will illustrate how empirical research is applied to gather data, analyze findings, and draw conclusions.

Social Sciences

In the realm of social sciences, consider a sociological study exploring the impact of socioeconomic status on educational attainment. Researchers gather data from a diverse group of individuals, including their family backgrounds, income levels, and academic achievements.

Through statistical analysis, they can identify correlations and trends, revealing whether individuals from lower socioeconomic backgrounds are less likely to attain higher levels of education. This empirical research helps shed light on societal inequalities and informs policymakers on potential interventions to address disparities in educational access.

Environmental Science

Environmental scientists often employ empirical research to assess the effects of environmental changes. For instance, researchers studying the impact of climate change on wildlife might collect data on animal populations, weather patterns, and habitat conditions over an extended period.

By analyzing this empirical data, they can identify correlations between climate fluctuations and changes in wildlife behavior, migration patterns, or population sizes. This empirical research is crucial for understanding the ecological consequences of climate change and informing conservation efforts.

Business and Economics

In the business world, empirical research is essential for making data-driven decisions. Consider a market research study conducted by a business seeking to launch a new product. They collect data through surveys , focus groups , and consumer behavior analysis.

By examining this empirical data, the company can gauge consumer preferences, demand, and potential market size. Empirical research in business helps guide product development, pricing strategies, and marketing campaigns, increasing the likelihood of a successful product launch.

Psychological studies frequently rely on empirical research to understand human behavior and cognition. For instance, a psychologist interested in examining the impact of stress on memory might design an experiment. Participants are exposed to stress-inducing situations, and their memory performance is assessed through various tasks.

By analyzing the data collected, the psychologist can determine whether stress has a significant effect on memory recall. This empirical research contributes to our understanding of the complex interplay between psychological factors and cognitive processes.

These examples highlight the versatility and applicability of empirical research across diverse fields. Whether in medicine, social sciences, environmental science, business, or psychology, empirical research serves as a fundamental tool for gaining insights, testing hypotheses, and driving advancements in knowledge and practice.

Conclusion for Empirical Research

Empirical research is a powerful tool for gaining insights, testing hypotheses, and making informed decisions. By following the steps outlined in this guide, you've learned how to select research topics, collect data, analyze findings, and effectively communicate your research to the world. Remember, empirical research is a journey of discovery, and each step you take brings you closer to a deeper understanding of the world around you. Whether you're a scientist, a student, or someone curious about the process, the principles of empirical research empower you to explore, learn, and contribute to the ever-expanding realm of knowledge.

How to Collect Data for Empirical Research?

Introducing Appinio , the real-time market research platform revolutionizing how companies gather consumer insights for their empirical research endeavors. With Appinio, you can conduct your own market research in minutes, gaining valuable data to fuel your data-driven decisions.

Appinio is more than just a market research platform; it's a catalyst for transforming the way you approach empirical research, making it exciting, intuitive, and seamlessly integrated into your decision-making process.

Here's why Appinio is the go-to solution for empirical research:

• From Questions to Insights in Minutes : With Appinio's streamlined process, you can go from formulating your research questions to obtaining actionable insights in a matter of minutes, saving you time and effort.
• Intuitive Platform for Everyone : No need for a PhD in research; Appinio's platform is designed to be intuitive and user-friendly, ensuring that anyone can navigate and utilize it effectively.
• Rapid Response Times : With an average field time of under 23 minutes for 1,000 respondents, Appinio delivers rapid results, allowing you to gather data swiftly and efficiently.
• Global Reach with Targeted Precision : With access to over 90 countries and the ability to define target groups based on 1200+ characteristics, Appinio empowers you to reach your desired audience with precision and ease.

Get free access to the platform!

Join the loop 💌

Be the first to hear about new updates, product news, and data insights. We'll send it all straight to your inbox.

Get the latest market research news straight to your inbox! 💌

Wait, there's more

30.05.2024 | 29min read

Pareto Analysis: Definition, Pareto Chart, Examples

28.05.2024 | 32min read

What is Systematic Sampling? Definition, Types, Examples

16.05.2024 | 30min read

Time Series Analysis: Definition, Types, Techniques, Examples

• University of Memphis Libraries
• Research Guides

Empirical Research: Defining, Identifying, & Finding

Defining empirical research, what is empirical research, quantitative or qualitative.

• Introduction
• Database Tools
• Search Terms
• Image Descriptions

Calfee & Chambliss (2005)  (UofM login required) describe empirical research as a "systematic approach for answering certain types of questions."  Those questions are answered "[t]hrough the collection of evidence under carefully defined and replicable conditions" (p. 43). 

The evidence collected during empirical research is often referred to as "data." 

Characteristics of Empirical Research

Emerald Publishing's guide to conducting empirical research identifies a number of common elements to empirical research: 

• A  research question , which will determine research objectives.
• A particular and planned  design  for the research, which will depend on the question and which will find ways of answering it with appropriate use of resources.
• The gathering of  primary data , which is then analysed.
• A particular  methodology  for collecting and analysing the data, such as an experiment or survey.
• The limitation of the data to a particular group, area or time scale, known as a sample [emphasis added]: for example, a specific number of employees of a particular company type, or all users of a library over a given time scale. The sample should be somehow representative of a wider population.
• The ability to  recreate  the study and test the results. This is known as  reliability .
• The ability to  generalize  from the findings to a larger sample and to other situations.

If you see these elements in a research article, you can feel confident that you have found empirical research. Emerald's guide goes into more detail on each element. 

Empirical research methodologies can be described as quantitative, qualitative, or a mix of both (usually called mixed-methods).

Ruane (2016)  (UofM login required) gets at the basic differences in approach between quantitative and qualitative research:

• Quantitative research  -- an approach to documenting reality that relies heavily on numbers both for the measurement of variables and for data analysis (p. 33).
• Qualitative research  -- an approach to documenting reality that relies on words and images as the primary data source (p. 33).

Both quantitative and qualitative methods are empirical . If you can recognize that a research study is quantitative or qualitative study, then you have also recognized that it is empirical study. 

Below are information on the characteristics of quantitative and qualitative research. This video from Scribbr also offers a good overall introduction to the two approaches to research methodology: 

Characteristics of Quantitative Research 

Researchers test hypotheses, or theories, based in assumptions about causality, i.e. we expect variable X to cause variable Y. Variables have to be controlled as much as possible to ensure validity. The results explain the relationship between the variables. Measures are based in pre-defined instruments.

Examples: experimental or quasi-experimental design, pretest & post-test, survey or questionnaire with closed-ended questions. Studies that identify factors that influence an outcomes, the utility of an intervention, or understanding predictors of outcomes. 

Characteristics of Qualitative Research

Researchers explore “meaning individuals or groups ascribe to social or human problems (Creswell & Creswell, 2018, p3).” Questions and procedures emerge rather than being prescribed. Complexity, nuance, and individual meaning are valued. Research is both inductive and deductive. Data sources are multiple and varied, i.e. interviews, observations, documents, photographs, etc. The researcher is a key instrument and must be reflective of their background, culture, and experiences as influential of the research.

Examples: open question interviews and surveys, focus groups, case studies, grounded theory, ethnography, discourse analysis, narrative, phenomenology, participatory action research.

Calfee, R. C. & Chambliss, M. (2005). The design of empirical research. In J. Flood, D. Lapp, J. R. Squire, & J. Jensen (Eds.),  Methods of research on teaching the English language arts: The methodology chapters from the handbook of research on teaching the English language arts (pp. 43-78). Routledge.  http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=nlebk&AN=125955&site=eds-live&scope=site .

Creswell, J. W., & Creswell, J. D. (2018).  Research design: Qualitative, quantitative, and mixed methods approaches  (5th ed.). Thousand Oaks: Sage.

How to... conduct empirical research . (n.d.). Emerald Publishing.  https://www.emeraldgrouppublishing.com/how-to/research-methods/conduct-empirical-research .

Scribbr. (2019). Quantitative vs. qualitative: The differences explained  [video]. YouTube.  https://www.youtube.com/watch?v=a-XtVF7Bofg .

Ruane, J. M. (2016).  Introducing social research methods : Essentials for getting the edge . Wiley-Blackwell.  http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=nlebk&AN=1107215&site=eds-live&scope=site .  

• << Previous: Home
• Next: Identifying Empirical Research >>
• Last Updated: Apr 2, 2024 11:25 AM
• URL: https://libguides.memphis.edu/empirical-research

• Architecture and Design
• Asian and Pacific Studies
• Business and Economics
• Classical and Ancient Near Eastern Studies
• Computer Sciences
• Cultural Studies
• Engineering
• General Interest
• Geosciences
• Industrial Chemistry
• Islamic and Middle Eastern Studies
• Jewish Studies
• Library and Information Science, Book Studies
• Life Sciences
• Linguistics and Semiotics
• Literary Studies
• Materials Sciences
• Mathematics
• Social Sciences
• Sports and Recreation
• Theology and Religion
• Publish your article
• The role of authors
• Promoting your article
• Abstracting & indexing
• Publishing Ethics
• Why publish with De Gruyter
• How to publish with De Gruyter
• Our book series
• Our subject areas
• Your digital product at De Gruyter
• Contribute to our reference works
• Product information
• Tools & resources
• Product Information
• Promotional Materials
• Orders and Inquiries
• FAQ for Library Suppliers and Book Sellers
• Repository Policy
• Free access policy
• Open Access agreements
• Database portals
• For Authors
• Customer service
• People + Culture
• Journal Management
• How to join us
• Working at De Gruyter
• Mission & Vision
• De Gruyter Foundation
• De Gruyter Ebound
• Our Responsibility
• Partner publishers

Your purchase has been completed. Your documents are now available to view.

Methods Used in Economic Research: An Empirical Study of Trends and Levels

The methods used in economic research are analyzed on a sample of all 3,415 regular research papers published in 10 general interest journals every 5th year from 1997 to 2017. The papers are classified into three main groups by method: theory, experiments, and empirics. The theory and empirics groups are almost equally large. Most empiric papers use the classical method, which derives an operational model from theory and runs regressions. The number of papers published increases by 3.3% p.a. Two trends are highly significant: The fraction of theoretical papers has fallen by 26 pp (percentage points), while the fraction of papers using the classical method has increased by 15 pp. Economic theory predicts that such papers exaggerate, and the papers that have been analyzed by meta-analysis confirm the prediction. It is discussed if other methods have smaller problems.

1 Introduction

This paper studies the pattern in the research methods in economics by a sample of 3,415 regular papers published in the years 1997, 2002, 2007, 2012, and 2017 in 10 journals. The analysis builds on the beliefs that truth exists, but it is difficult to find, and that all the methods listed in the next paragraph have problems as discussed in Sections 2 and 4. Hereby I do not imply that all – or even most – papers have these problems, but we rarely know how serious it is when we read a paper. A key aspect of the problem is that a “perfect” study is very demanding and requires far too much space to report, especially if the paper looks for usable results. Thus, each paper is just one look at an aspect of the problem analyzed. Only when many studies using different methods reach a joint finding, we can trust that it is true.

Section 2 discusses the classification of papers by method into three main categories: (M1) Theory , with three subgroups: (M1.1) economic theory, (M1.2) statistical methods, and (M1.3) surveys. (M2) Experiments , with two subgroups: (M2.1) lab experiments and (M2.2) natural experiments. (M3) Empirics , with three subgroups: (M3.1) descriptive, (M3.2) classical empirics, and (M3.3) newer empirics. More than 90% of the papers are easy to classify, but a stochastic element enters in the classification of the rest. Thus, the study has some – hopefully random – measurement errors.

Section 3 discusses the sample of journals chosen. The choice has been limited by the following main criteria: It should be good journals below the top ten A-journals, i.e., my article covers B-journals, which are the journals where most research economists publish. It should be general interest journals, and the journals should be so different that it is likely that patterns that generalize across these journals apply to more (most?) journals. The Appendix gives some crude counts of researchers, departments, and journals. It assesses that there are about 150 B-level journals, but less than half meet the criteria, so I have selected about 15% of the possible ones. This is the most problematic element in the study. If the reader accepts my choice, the paper tells an interesting story about economic research.

All B-level journals try hard to have a serious refereeing process. If our selection is representative, the 150 journals have increased the annual number of papers published from about 7,500 in 1997 to about 14,000 papers in 2017, giving about 200,000 papers for the period. Thus, the B-level dominates our science. Our sample is about 6% for the years covered, but less than 2% of all papers published in B-journals in the period. However, it is a larger fraction of the papers in general interest journals.

It is impossible for anyone to read more than a small fraction of this flood of papers. Consequently, researchers compete for space in journals and for attention from the readers, as measured in the form of citations. It should be uncontroversial that papers that hold a clear message are easier to publish and get more citations. Thus, an element of sales promotion may enter papers in the form of exaggeration , which is a joint problem for all eight methods. This is in accordance with economic theory that predicts that rational researchers report exaggerated results; see Paldam ( 2016 , 2018 ). For empirical papers, meta-methods exist to summarize the results from many papers, notably papers using regressions. Section 4.4 reports that meta-studies find that exaggeration is common.

The empirical literature surveying the use of research methods is quite small, as I have found two articles only: Hamermesh ( 2013 ) covers 748 articles in 6 years a decade apart studies in three A-journals using a slightly different classification of methods, [1] while my study covers B-journals. Angrist, Azoulay, Ellison, Hill, and Lu ( 2017 ) use a machine-learning classification of 134,000 papers in 80 journals to look at the three main methods. My study subdivide the three categories into eight. The machine-learning algorithm is only sketched, so the paper is difficult to replicate, but it is surely a major effort. A key result in both articles is the strong decrease of theory in economic publications. This finding is confirmed, and it is shown that the corresponding increase in empirical articles is concentrated on the classical method.

I have tried to explain what I have done, so that everything is easy to replicate, in full or for one journal or one year. The coding of each article is available at least for the next five years. I should add that I have been in economic research for half a century. Some of the assessments in the paper will reflect my observations/experience during this period (indicated as my assessments). This especially applies to the judgements expressed in Section 4.

2 The eight categories

Table 1 reports that the annual number of papers in the ten journals has increased 1.9 times, or by 3.3% per year. The Appendix gives the full counts per category, journal, and year. By looking at data over two decades, I study how economic research develops. The increase in the production of papers is caused by two factors: The increase in the number of researchers. The increasing importance of publications for the careers of researchers.

The 3,415 papers

2.1 (M1) Theory: subgroups (M1.1) to (M1.3)

Table 2 lists the groups and main numbers discussed in the rest of the paper. Section 2.1 discusses (M1) theory. Section 2.2 covers (M2) experimental methods, while Section 2.3 looks at (M3) empirical methods using statistical inference from data.

The 3,415 papers – fractions in percent

The change of the fractions from 1997 to 2017 in percentage points

Note: Section 3.4 tests if the pattern observed in Table 3 is statistically significant. The Appendix reports the full data.

2.1.1 (M1.1) Economic theory

Papers are where the main content is the development of a theoretical model. The ideal theory paper presents a (simple) new model that recasts the way we look at something important. Such papers are rare and obtain large numbers of citations. Most theoretical papers present variants of known models and obtain few citations.

In a few papers, the analysis is verbal, but more than 95% rely on mathematics, though the technical level differs. Theory papers may start by a descriptive introduction giving the stylized fact the model explains, but the bulk of the paper is the formal analysis, building a model and deriving proofs of some propositions from the model. It is often demonstrated how the model works by a set of simulations, including a calibration made to look realistic. However, the calibrations differ greatly by the efforts made to reach realism. Often, the simulations are in lieu of an analytical solution or just an illustration suggesting the magnitudes of the results reached.

Theoretical papers suffer from the problem known as T-hacking , [2] where the able author by a careful selection of assumptions can tailor the theory to give the results desired. Thus, the proofs made from the model may represent the ability and preferences of the researcher rather than the properties of the economy.

2.1.2 (M1.2) Statistical method

Papers reporting new estimators and tests are published in a handful of specialized journals in econometrics and mathematical statistics – such journals are not included. In our general interest journals, some papers compare estimators on actual data sets. If the demonstration of a methodological improvement is the main feature of the paper, it belongs to (M1.2), but if the economic interpretation is the main point of the paper, it belongs to (M3.2) or (M3.3). [3]

Some papers, including a special issue of Empirical Economics (vol. 53–1), deal with forecasting models. Such models normally have a weak relation to economic theory. They are sometimes justified precisely because of their eclectic nature. They are classified as either (M1.2) or (M3.1), depending upon the focus. It appears that different methods work better on different data sets, and perhaps a trade-off exists between the user-friendliness of the model and the improvement reached.

2.1.3 (M1.3) Surveys

When the literature in a certain field becomes substantial, it normally presents a motley picture with an amazing variation, especially when different schools exist in the field. Thus, a survey is needed, and our sample contains 68 survey articles. They are of two types, where the second type is still rare:

2.1.3.1 (M1.3.1) Assessed surveys

Here, the author reads the papers and assesses what the most reliable results are. Such assessments require judgement that is often quite difficult to distinguish from priors, even for the author of the survey.

2.1.3.2 (M1.3.2) Meta-studies

They are quantitative surveys of estimates of parameters claimed to be the same. Over the two decades from 1997 to 2017, about 500 meta-studies have been made in economics. Our sample includes five, which is 0.15%. [4] Meta-analysis has two levels: The basic level collects and codes the estimates and studies their distribution. This is a rather objective exercise where results seem to replicate rather well. [5] The second level analyzes the variation between the results. This is less objective. The papers analyzed by meta-studies are empirical studies using method (M3.2), though a few use estimates from (M3.1) and (M3.3).

2.2 (M2) Experimental methods: subgroups (M2.1) and (M2.2)

Experiments are of three distinct types, where the last two are rare, so they are lumped together. They are taking place in real life.

2.2.1 (M2.1) Lab experiments

The sample had 1.9% papers using this method in 1997, and it has expanded to 9.7% in 2017. It is a technique that is much easier to apply to micro- than to macroeconomics, so it has spread unequally in the 10 journals, and many experiments are reported in a couple of special journals that are not included in our sample.

Most of these experiments take place in a laboratory, where the subjects communicate with a computer, giving a controlled, but artificial, environment. [6] A number of subjects are told a (more or less abstract) story and paid to react in either of a number of possible ways. A great deal of ingenuity has gone into the construction of such experiments and in the methods used to analyze the results. Lab experiments do allow studies of behavior that are hard to analyze in any other way, and they frequently show sides of human behavior that are difficult to rationalize by economic theory. It appears that such demonstration is a strong argument for the publication of a study.

However, everything is artificial – even the payment. In some cases, the stories told are so elaborate and abstract that framing must be a substantial risk; [7] see Levitt and List ( 2007 ) for a lucid summary, and Bergh and Wichardt ( 2018 ) for a striking example. In addition, experiments cost money, which limits the number of subjects. It is also worth pointing to the difference between expressive and real behavior. It is typically much cheaper for the subject to “express” nice behavior in a lab than to be nice in the real world.

(M2.2) Event studies are studies of real world experiments. They are of two types:

(M2.2.1) Field experiments analyze cases where some people get a certain treatment and others do not. The “gold standard” for such experiments is double blind random sampling, where everything (but the result!) is preannounced; see Christensen and Miguel ( 2018 ). Experiments with humans require permission from the relevant authorities, and the experiment takes time too. In the process, things may happen that compromise the strict rules of the standard. [8] Controlled experiments are expensive, as they require a team of researchers. Our sample of papers contains no study that fulfills the gold standard requirements, but there are a few less stringent studies of real life experiments.

(M2.2.2) Natural experiments take advantage of a discontinuity in the environment, i.e., the period before and after an (unpredicted) change of a law, an earthquake, etc. Methods have been developed to find the effect of the discontinuity. Often, such studies look like (M3.2) classical studies with many controls that may or may not belong. Thus, the problems discussed under (M3.2) will also apply.

2.3 (M3) Empirical methods: subgroups (M3.1) to (M3.3)

The remaining methods are studies making inference from “real” data, which are data samples where the researcher chooses the sample, but has no control over the data generating process.

(M3.1) Descriptive studies are deductive. The researcher describes the data aiming at finding structures that tell a story, which can be interpreted. The findings may call for a formal test. If one clean test follows from the description, [9] the paper is classified under (M3.1). If a more elaborate regression analysis is used, it is classified as (M3.2). Descriptive studies often contain a great deal of theory.

Some descriptive studies present a new data set developed by the author to analyze a debated issue. In these cases, it is often possible to make a clean test, so to the extent that biases sneak in, they are hidden in the details of the assessments made when the data are compiled.

(M3.2) Classical empirics has three steps: It starts by a theory, which is developed into an operational model. Then it presents the data set, and finally it runs regressions.

The significance levels of the t -ratios on the coefficient estimated assume that the regression is the first meeting of the estimation model and the data. We all know that this is rarely the case; see also point (m1) in Section 4.4. In practice, the classical method is often just a presentation technique. The great virtue of the method is that it can be applied to real problems outside academia. The relevance comes with a price: The method is quite flexible as many choices have to be made, and they often give different results. Preferences and interests, as discussed in Sections 4.3 and 4.4 below, notably as point (m2), may affect these choices.

(M3.3) Newer empirics . Partly as a reaction to the problems of (M3.2), the last 3–4 decades have seen a whole set of newer empirical techniques. [10] They include different types of VARs, Bayesian techniques, causality/co-integration tests, Kalman Filters, hazard functions, etc. I have found 162 (or 4.7%) papers where these techniques are the main ones used. The fraction was highest in 1997. Since then it has varied, but with no trend.

I think that the main reason for the lack of success for the new empirics is that it is quite bulky to report a careful set of co-integration tests or VARs, and they often show results that are far from useful in the sense that they are unclear and difficult to interpret. With some introduction and discussion, there is not much space left in the article. Therefore, we are dealing with a cookbook that makes for rather dull dishes, which are difficult to sell in the market.

Note the contrast between (M3.2) and (M3.3): (M3.2) makes it possible to write papers that are too good, while (M3.3) often makes them too dull. This contributes to explain why (M3.2) is getting (even) more popular and the lack of success of (M3.3), but then, it is arguable that it is more dangerous to act on exaggerated results than on results that are weak.

3 The 10 journals

The 10 journals chosen are: (J1) Can [Canadian Journal of Economics], (J2) Emp [Empirical Economics], (J3) EER [European Economic Review], (J4) EJPE [European Journal of Political Economy], (J5) JEBO [Journal of Economic Behavior & Organization], (J6) Inter [Journal of International Economics], (J7) Macro [Journal of Macroeconomics], (J8) Kyklos, (J9) PuCh [Public Choice], and (J10) SJE [Scandinavian Journal of Economics].

Section 3.1 discusses the choice of journals, while Section 3.2 considers how journals deal with the pressure for publication. Section 3.3 shows the marked difference in publication profile of the journals, and Section 3.4 tests if the trends in methods are significant.

3.1 The selection of journals

They should be general interest journals – methodological journals are excluded. By general interest, I mean that they bring papers where an executive summary may interest policymakers and people in general. (ii) They should be journals in English (the Canadian Journal includes one paper in French), which are open to researchers from all countries, so that the majority of the authors are from outside the country of the journal. [11] (iii) They should be sufficiently different so that it is likely that patterns, which apply to these journals, tell a believable story about economic research. Note that (i) and (iii) require some compromises, as is evident in the choice of (J2), (J6), (J7), and (J8) ( Table 4 ).

The 10 journals covered

Note. Growth is the average annual growth from 1997 to 2017 in the number of papers published.

Methodological journals are excluded, as they are not interesting to outsiders. However, new methods are developed to be used in general interest journals. From studies of citations, we know that useful methodological papers are highly cited. If they remain unused, we presume that it is because they are useless, though, of course, there may be a long lag.

The choice of journals may contain some subjectivity, but I think that they are sufficiently diverse so that patterns that generalize across these journals will also generalize across a broader range of good journals.

The papers included are the regular research articles. Consequently, I exclude short notes to other papers and book reviews, [12] except for a few article-long discussions of controversial books.

3.2 Creating space in journals

As mentioned in the introduction, the annual production of research papers in economics has now reached about 1,000 papers in top journals, and about 14,000 papers in the group of good journals. [13] The production has grown with 3.3% per year, and thus it has doubled the last twenty years. The hard-working researcher will read less than 100 papers a year. I know of no signs that this number is increasing. Thus, the upward trend in publication must be due to the large increase in the importance of publications for the careers of researchers, which has greatly increased the production of papers. There has also been a large increase in the number of researches, but as citations are increasingly skewed toward the top journals (see Heckman & Moktan, 2018 ), it has not increased demand for papers correspondingly. The pressures from the supply side have caused journals to look for ways to create space.

Book reviews have dropped to less than 1/3. Perhaps, it also indicates that economists read fewer books than they used to. Journals have increasingly come to use smaller fonts and larger pages, allowing more words per page. The journals from North-Holland Elsevier have managed to cram almost two old pages into one new one. [14] This makes it easier to publish papers, while they become harder to read.

Many journals have changed their numbering system for the annual issues, making it less transparent how much they publish. Only three – Canadian Economic Journal, Kyklos, and Scandinavian Journal of Economics – have kept the schedule of publishing one volume of four issues per year. It gives about 40 papers per year. Public Choice has a (fairly) consistent system with four volumes of two double issues per year – this gives about 100 papers. The remaining journals have changed their numbering system and increased the number of papers published per year – often dramatically.

Thus, I assess the wave of publications is caused by the increased supply of papers and not to the demand for reading material. Consequently, the study confirms and updates the observation by Temple ( 1918 , p. 242): “… as the world gets older the more people are inclined to write but the less they are inclined to read.”

3.3 How different are the journals?

The appendix reports the counts for each year and journal of the research methods. From these counts, a set of χ 2 -scores is calculated for the three main groups of methods – they are reported in Table 5 . It gives the χ 2 -test comparing the profile of each journal to the one of the other nine journals taken to be the theoretical distribution.

The methodological profile of the journals –  χ 2 -scores for main groups

Note: The χ 2 -scores are calculated relative to all other journals. The sign (+) or (−) indicates if the journal has too many or too few papers relatively in the category. The P -values for the χ 2 (3)-test always reject that the journal has the same methodological profile as the other nine journals.

The test rejects that the distribution is the same as the average for any of the journals. The closest to the average is the EJPE and Public Choice. The two most deviating scores are for the most micro-oriented journal JEBO, which brings many experimental papers, and of course, Empirical Economics, which brings many empirical papers.

3.4 Trends in the use of the methods

Table 3 already gave an impression of the main trends in the methods preferred by economists. I now test if these impressions are statistically significant. The tests have to be tailored to disregard three differences between the journals: their methodological profiles, the number of papers they publish, and the trend in the number. Table 6 reports a set of distribution free tests, which overcome these differences. The tests are done on the shares of each research method for each journal. As the data cover five years, it gives 10 pairs of years to compare. [15] The three trend-scores in the []-brackets count how often the shares go up, down, or stay the same in the 10 cases. This is the count done for a Kendall rank correlation comparing the five shares with a positive trend (such as 1, 2, 3, 4, and 5).

Trend-scores and tests for the eight subgroups of methods across the 10 journals

Note: The three trend-scores in each [ I 1 , I 2 , I 3 ]-bracket are a Kendall-count over all 10 combinations of years. I 1 counts how often the share goes up. I 2 counts when the share goes down, and I 3 counts the number of ties. Most ties occur when there are no observations either year. Thus, I 1 + I 2 + I 3 = 10. The tests are two-sided binominal tests disregarding the zeroes. The test results in bold are significant at the 5% level.

The first set of trend-scores for (M1.1) and (J1) is [1, 9, 0]. It means that 1 of the 10 share-pairs increases, while nine decrease and no ties are found. The two-sided binominal test is 2%, so it is unlikely to happen. Nine of the ten journals in the (M1.1)-column have a majority of falling shares. The important point is that the counts in one column can be added – as is done in the all-row; this gives a powerful trend test that disregards differences between journals and the number of papers published. ( Table A1 )

Four of the trend-tests are significant: The fall in theoretical papers and the rise in classical papers. There is also a rise in the share of stat method and event studies. It is surprising that there is no trend in the number of experimental studies, but see Table A2 (in Appendix).

4 An attempt to interpret the pattern found

The development in the methods pursued by researchers in economics is a reaction to the demand and supply forces on the market for economic papers. As already argued, it seems that a key factor is the increasing production of papers.

The shares add to 100, so the decline of one method means that the others rise. Section 4.1 looks at the biggest change – the reduction in theory papers. Section 4.2 discusses the rise in two new categories. Section 4.3 considers the large increase in the classical method, while Section 4.4 looks at what we know about that method from meta-analysis.

4.1 The decline of theory: economics suffers from theory fatigue [16]

The papers in economic theory have dropped from 59.5 to 33.6% – this is the largest change for any of the eight subgroups. [17] It is highly significant in the trend test. I attribute this drop to theory fatigue.

As mentioned in Section 2.1, the ideal theory paper presents a (simple) new model that recasts the way we look at something important. However, most theory papers are less exciting: They start from the standard model and argue that a well-known conclusion reached from the model hinges upon a debatable assumption – if it changes, so does the conclusion. Such papers are useful. From a literature on one main model, the profession learns its strengths and weaknesses. It appears that no generally accepted method exists to summarize this knowledge in a systematic way, though many thoughtful summaries have appeared.

I think that there is a deeper problem explaining theory fatigue. It is that many theoretical papers are quite unconvincing. Granted that the calculations are done right, believability hinges on the realism of the assumptions at the start and of the results presented at the end. In order for a model to convince, it should (at least) demonstrate the realism of either the assumptions or the outcome. [18] If both ends appear to hang in the air, it becomes a game giving little new knowledge about the world, however skillfully played.

The theory fatigue has caused a demand for simulations demonstrating that the models can mimic something in the world. Kydland and Prescott pioneered calibration methods (see their 1991 ). Calibrations may be carefully done, but it often appears like a numerical solution of a model that is too complex to allow an analytical solution.

4.2 Two examples of waves: one that is still rising and another that is fizzling out

When a new method of gaining insights in the economy first appears, it is surrounded by doubts, but it also promises a high marginal productivity of knowledge. Gradually the doubts subside, and many researchers enter the field. After some time this will cause the marginal productivity of the method to fall, and it becomes less interesting. The eight methods include two newer ones: Lab experiments and newer stats. [19]

It is not surprising that papers with lab experiments are increasing, though it did take a long time: The seminal paper presenting the technique was Smith ( 1962 ), but only a handful of papers are from the 1960s. Charles Plott organized the first experimental lab 10 years later – this created a new standard for experiments, but required an investment in a lab and some staff. Labs became more common in the 1990s as PCs got cheaper and software was developed to handle experiments, but only 1.9% of the papers in the 10 journals reported lab experiments in 1997. This has now increased to 9.7%, so the wave is still rising. The trend in experiments is concentrated in a few journals, so the trend test in Table 6 is insignificant, but it is significant in the Appendix Table A2 , where it is done on the sum of articles irrespective of the journal.

In addition to the rising share of lab experiment papers in some journals, the journal Experimental Economics was started in 1998, where it published 281 pages in three issues. In 2017, it had reached 1,006 pages in four issues, [20] which is an annual increase of 6.5%.

Compared with the success of experimental economics, the motley category of newer empirics has had a more modest success, as the fraction of papers in the 5 years are 5.8, 5.2, 3.5, 5.4, and 4.2, which has no trend. Newer stats also require investment, but mainly in human capital. [21] Some of the papers using the classical methodology contain a table with Dickey-Fuller tests or some eigenvalues of the data matrix, but they are normally peripheral to the analysis. A couple of papers use Kalman filters, and a dozen papers use Bayesian VARs. However, it is clear that the newer empirics have made little headway into our sample of general interest journals.

4.3 The steady rise of the classical method: flexibility rewarded

The typical classical paper provides estimates of a key effect that decision-makers outside academia want to know. This makes the paper policy relevant right from the start, and in many cases, it is possible to write a one page executive summary to the said decision-makers.

The three-step convention (see Section 2.3) is often followed rather loosely. The estimation model is nearly always much simpler than the theory. Thus, while the model can be derived from a theory, the reverse does not apply. Sometimes, the model seems to follow straight from common sense, and if the link from the theory to the model is thin, it begs the question: Is the theory really necessary? In such cases, it is hard to be convinced that the tests “confirm” the theory, but then, of course, tests only say that the data do not reject the theory.

The classical method is often only a presentation devise. Think of a researcher who has reached a nice publishable result through a long and tortuous path, including some failed attempts to find such results. It is not possible to describe that path within the severely limited space of an article. In addition, such a presentation would be rather dull to read, and none of us likes to talk about wasted efforts that in hindsight seem a bit silly. Here, the classical method becomes a convenient presentation device.

The biggest source of variation in the results is the choice of control/modifier variables. All datasets presumably contain some general and some special information, where the latter depends on the circumstances prevailing when the data were compiled. The regression should be controlled for these circumstances in order to reach the general result. Such ceteris paribus controls are not part of the theory, so many possible controls may be added. The ones chosen for publication often appear to be the ones delivering the “right” results by the priors of the researcher. The justification for their inclusion is often thin, and if two-stage regressions are used, the first stage instruments often have an even thinner justification.

Thus, the classical method is rather malleable to the preferences and interests of researchers and sponsors. This means that some papers using the classical technique are not what they pretend, as already pointed out by Leamer ( 1983 ), see also Paldam ( 2018 ) for new references and theory. The fact that data mining is tempting suggests that it is often possible to reach smashing results, making the paper nice to read. This may be precisely why it is cited.

Many papers using the classical method throw in some bits of exotic statistics technique to demonstrate the robustness of the result and the ability of the researcher. This presumably helps to generate credibility.

4.4 Knowledge about classical papers reached from meta-studies

Individual studies using the classical method often look better than they are, and thus they are more uncertain than they appear, but we may think of the value of convergence for large N s (number of observations) as the truth. The exaggeration is largest in the beginning of a new literature, but gradually it becomes smaller. Thus, the classical method does generate truth when the effect searched for has been studied from many sides. The word research does mean that the search has to be repeated! It is highly risky to trust a few papers only.

Meta-analysis has found other results such as: Results in top journals do not stand out. It is necessary to look at many journals, as many papers on the same effect are needed. Little of the large variation between results is due to the choice of estimators.

A similar development should occur also for experimental economics. Experiments fall in families: A large number cover prisoner’s dilemma games, but there are also many studies of dictator games, auction games, etc. Surveys summarizing what we have learned about these games seem highly needed. Assessed summaries of old experiments are common, notably in introductions to papers reporting new ones. It should be possible to extract the knowledge reached by sets of related lab experiments in a quantitative way, by some sort of meta-technique, but this has barely started. The first pioneering meta-studies of lab experiments do find the usual wide variation of results from seemingly closely related experiments. [25] A recent large-scale replicability study by Camerer et al. ( 2018 ) finds that published experiments in the high quality journal Nature and Science exaggerate by a factor two just like regression studies using the classical method.

5 Conclusion

The study presents evidence that over the last 20 years economic research has moved away from theory towards empirical work using the classical method.

From the eighties onward, there has been a steady stream of papers pointing out that the classical method suffers from excess flexibility. It does deliver relevant results, but they tend to be too good. [26] While, increasingly, we know the size of the problems of the classical method, systematic knowledge about the problems of the other methods is weaker. It is possible that the problems are smaller, but we do not know.

Therefore, it is clear that obtaining solid knowledge about the size of an important effect requires a great deal of papers analyzing many aspects of the effect and a careful quantitative survey. It is a well-known principle in the harder sciences that results need repeated independent replication to be truly trustworthy. In economics, this is only accepted in principle.

The classical method of empirical research is gradually winning, and this is a fine development: It does give answers to important policy questions. These answers are highly variable and often exaggerated, but through the efforts of many competing researchers, solid knowledge will gradually emerge.

Home page: http://www.martin.paldam.dk

Acknowledgments

The paper has been presented at the 2018 MAER-Net Colloquium in Melbourne, the Kiel Aarhus workshop in 2018, and at the European Public Choice 2019 Meeting in Jerusalem. I am grateful for all comments, especially from Chris Doucouliagos, Eelke de Jong, and Bob Reed. In addition, I thank the referees for constructive advice.

Conflict of interest: Author states no conflict of interest.

Appendix: Two tables and some assessments of the size of the profession

The text needs some numbers to assess the representativity of the results reached. These numbers just need to be orders of magnitude. I use the standard three-level classification in A, B, and C of researchers, departments, and journals. The connections between the three categories are dynamic and rely on complex sorting mechanisms. In an international setting, it matters that researchers have preferences for countries, notably their own. The relation between the three categories has a stochastic element.

The World of Learning organization reports on 36,000 universities, colleges, and other institutes of tertiary education and research. Many of these institutions are mainly engaged in undergraduate teaching, and some are quite modest. If half of these institutions have a program in economics, with a staff of at least five, the total stock of academic economists is 100,000, of which most are at the C-level.

The A-level of about 500 tenured researchers working at the top ten universities (mainly) publishes in the top 10 journals that bring less than 1,000 papers per year; [27] see Heckman and Moktan (2020). They (mainly) cite each other, but they greatly influence other researchers. [28] The B-level consists of about 15–20,000 researchers who work at 4–500 research universities, with graduate programs and ambitions to publish. They (mainly) publish in the next level of about 150 journals. [29] In addition, there are at least another 1,000 institutions that strive to move up in the hierarchy.

The counts for each of the 10 journals

Counts, shares, and changes for all ten journals for subgroups

Note: The trend-scores are calculated as in Table 6 . Compared to the results in Table 6 , the results are similar, but the power is less than before. However, note that the results in Column (M2.1) dealing with experiments are stronger in Table A2 . This has to do with the way missing observations are treated in the test.

Angrist, J. , Azoulay, P. , Ellison, G. , Hill, R. , & Lu, S. F. (2017). Economic research evolves: Fields and styles. American Economic Review (Papers & Proceedings), 107, 293–297. 10.1257/aer.p20171117 Search in Google Scholar

Bergh, A. , & Wichardt, P. C. (2018). Mine, ours or yours? Unintended framing effects in dictator games (INF Working Papers, No 1205). Research Institute of Industrial Econ, Stockholm. München: CESifo. 10.2139/ssrn.3208589 Search in Google Scholar

Brodeur, A. , Cook, N. , & Heyes, A. (2020). Methods matter: p-Hacking and publication bias in causal analysis in economics. American Economic Review, 110(11), 3634–3660. 10.1257/aer.20190687 Search in Google Scholar

Camerer, C. F. , Dreber, A. , Holzmaster, F. , Ho, T.-H. , Huber, J. , Johannesson, M. , … Wu, H. (27 August 2018). Nature Human Behaviour. https://www.nature.com/articles/M2.11562-018-0399-z Search in Google Scholar

Card, D. , & DellaVigna, A. (2013). Nine facts about top journals in economics. Journal of Economic Literature, 51, 144–161 10.3386/w18665 Search in Google Scholar

Christensen, G. , & Miguel, E. (2018). Transparency, reproducibility, and the credibility of economics research. Journal of Economic Literature, 56, 920–980 10.3386/w22989 Search in Google Scholar

Doucouliagos, H. , Paldam, M. , & Stanley, T. D. (2018). Skating on thin evidence: Implications for public policy. European Journal of Political Economy, 54, 16–25 10.1016/j.ejpoleco.2018.03.004 Search in Google Scholar

Engel, C. (2011). Dictator games: A meta study. Experimental Economics, 14, 583–610 10.1007/s10683-011-9283-7 Search in Google Scholar

Fiala, L. , & Suentes, S. (2017). Transparency and cooperation in repeated dilemma games: A meta study. Experimental Economics, 20, 755–771 10.1007/s10683-017-9517-4 Search in Google Scholar

Friedman, M. (1953). Essays in positive economics. Chicago: University of Chicago Press. Search in Google Scholar

Hamermesh, D. (2013). Six decades of top economics publishing: Who and how? Journal of Economic Literature, 51, 162–172 10.3386/w18635 Search in Google Scholar

Heckman, J. J. , & Moktan, S. (2018). Publishing and promotion in economics: The tyranny of the top five. Journal of Economic Literature, 51, 419–470 10.3386/w25093 Search in Google Scholar

Ioannidis, J. P. A. , Stanley, T. D. , & Doucouliagos, H. (2017). The power of bias in economics research. Economic Journal, 127, F236–F265 10.1111/ecoj.12461 Search in Google Scholar

Johansen, S. , & Juselius, K. (1990). Maximum likelihood estimation and inference on cointegration – with application to the demand for money. Oxford Bulletin of Economics and Statistics, 52, 169–210 10.1111/j.1468-0084.1990.mp52002003.x Search in Google Scholar

Justman, M. (2018). Randomized controlled trials informing public policy: Lessons from the project STAR and class size reduction. European Journal of Political Economy, 54, 167–174 10.1016/j.ejpoleco.2018.04.005 Search in Google Scholar

Kydland, F. , & Prescott, E. C. (1991). The econometrics of the general equilibrium approach to business cycles. Scandinavian Journal of Economics, 93, 161–178 10.2307/3440324 Search in Google Scholar

Leamer, E. E. (1983). Let’s take the con out of econometrics. American Economic Review, 73, 31–43 Search in Google Scholar

Levitt, S. D. , & List, J. A. (2007). On the generalizability of lab behaviour to the field. Canadian Journal of Economics, 40, 347–370 10.1111/j.1365-2966.2007.00412.x Search in Google Scholar

Paldam, M. (April 14th 2015). Meta-analysis in a nutshell: Techniques and general findings. Economics. The Open-Access, Open-Assessment E-Journal, 9, 1–4 10.5018/economics-ejournal.ja.2015-11 Search in Google Scholar

Paldam, M. (2016). Simulating an empirical paper by the rational economist. Empirical Economics, 50, 1383–1407 10.1007/s00181-015-0971-6 Search in Google Scholar

Paldam, M. (2018). A model of the representative economist, as researcher and policy advisor. European Journal of Political Economy, 54, 6–15 10.1016/j.ejpoleco.2018.03.005 Search in Google Scholar

Smith, V. (1962). An experimental study of competitive market behavior. Journal of Political Economy, 70, 111–137 10.1017/CBO9780511528354.003 Search in Google Scholar

Stanley, T. D. , & Doucouliagos, H. (2012). Meta-regression analysis in economics and business. Abingdon: Routledge. 10.4324/9780203111710 Search in Google Scholar

Temple, C. L. (1918). Native races and their rulers; sketches and studies of official life and administrative problems in Nigeria. Cape Town: Argus Search in Google Scholar

© 2021 Martin Paldam, published by De Gruyter

This work is licensed under the Creative Commons Attribution 4.0 International License.

• X / Twitter

Supplementary Materials

• Supplementary material

Please login or register with De Gruyter to order this product.

Journal and Issue

Articles in the same issue.

Researching and writing for Economics students

6 economic theory, modeling, and connecting this to empirical work, 6.1 (from theory to) empirical work, 6.2 doing economic modelling and theory.

Economic models usually involve one or more of:

Individual (or firm or other actor) optimization… maximization of a utility, profit or welfare function (over time, perhaps involving uncertainty) subject to one or more constraints

Aggregation of the above and consideration of an ‘equilibrium’ outcome, usually involving an equilibrium price

Game theoretic (or principle-agent/mechanism design) treatment of a strategic interaction between multiple ‘actors’, considering the equilibrium (or other reasonable predicted outcome) and the ‘comparative statics; of this as key factors (’parameters’) are varied

Doing economic theory

You may have heard that ‘you should not do a theory dissertation as an undergraduate.’ That is the case that it is difficult to make a substantial contribution to pure economic theory. This work is highly mathematical. The models stemming from our basic framework and beyond have been explored in great depth and sophistication. The frontiers have been pushed very far in terms of ‘what are the implications of our standard assumptions and in what ways can they be relaxed’.

However, this does not mean that you should not do theory and modeling in your undergraduate dissertation. You should try to incorporating and adapting existing models (remember Economic theory=models=maths… approximately). You can also make this the main focus of your dissertation.

Remember, you are not writing this dissertation to advance the frontiers of Economics. In large part, you are trying to show your understanding and ability to apply techniques to specific questions.

If you can explain and adapt a rigorous theoretical model to apply to particular case, yielding intuition, you have done well. (For example… )

Even if you’re not doing a “theory paper”, specifying a model will benefit your paper in several ways:

Fixing your ideas and arguments precisely and demonstrating internal consistency

Motivating your (empirical) analysis

E.g., why might we expect an impact of education on income? Why might the private returns exceed the social returns? What are the channels by which education could yield personal and social gains (or losses)?

Providing structure and ‘restrictions’ for your empirical analysis

Connecting to the Economic literature and incorporating the general insights of the field

Helping you consider and estimation ‘policy implications’ of your results

Building an economic model

Posing your hypothesis as an empirical test

Writing an empirical/econometric model

6.3 Economic theory and empirical research: writing about your work

Explain the limitations of your analysis to the reader, and what the next step would be. Perhaps you are aware there is an advanced estimation technique, or a larger data set, that could better answer your thesis question. However, this might be “too difficult” considering your abilities and resources. If you can explain this, do so.

If you’re doing a theory paper (also useful in an empirical paper) try to explicitly state and clearly explain a formal economic model, using mathematical notation.

If you’re doing an empirical paper clearly explain and describe your data, techniques, and results. Explain the econometrics behind your techniques as clearly as you can.

6.4 Empirical work: techniques and econometrics

Techniques . Understand what techniques others have used to answer your question, what technique you are using and why, and the arguments for each technique. Understand the limitations of each technique, previous papers, and of your own work.

Show you understand economic theory and the connection between theory and econometrics and empirical work. Understand the difference between these, and what each can do.

Use of techniques: use the tools you can handle, understand, and explain. Try to use the right techniques, but also try to limit yourself to techniques you can explain, at least in general terms.

Justify the techniques you use; don’t merely hide behind the rationalisation that “other authors did it”. If other authors jumped off the Brooklyn bridge, would you jump?

Know your limits. Set reasonable goals for your dissertation, and do not claim to have achieved more than you have done.

This website uses cookies.

By clicking the "Accept" button or continuing to browse our site, you agree to first-party and session-only cookies being stored on your device to enhance site navigation and analyze site performance and traffic. For more information on our use of cookies, please see our Privacy Policy .

• Research Highlights

An empirical turn in economics research

• Featured Chart
• June 26, 2017

A table of results in an issue of the American Economic Review.

Gian Romagnoli

Over the past few decades, economists have increasingly been cited in the press and sought by Congress to give testimony on the issues of the day. This could be due in part to the increasingly empirical nature of economics research.

Aided by internet connections that allow datasets to be assembled from disparate sources and cheap computing power to crunch the numbers, economists are more and more often turning to real-world data to complement and test theoretical models.

This trend was documented in a 2013 article from the Journal of Economic Literature that showed, in a sample of 748 academic journal articles in top economics journals, that empirical work has become much more common since the 1960s.

In the spirit of empirical inquiry, the authors of a study appearing in the May issue of the American Economic Review: Papers & Proceedings used machine learning techniques to expand this analysis to a much larger set of 135,000 papers published across 80 academic journals cited frequently in the American Economic Review .

Figure 4  from Angrist et al. (2017)

Sorting hundreds of thousands of papers into “theoretical” and “empirical” piles by hand would be prohibitive, so authors Joshua Angrist , Pierre Azoulay , Glenn Ellison , Ryan Hill, and Susan Feng Lu use latent Dirichlet allocation and logistic ridge regression to analyze the wording of titles and abstracts and assign each paper to a category.

Based on a smaller group of five thousand papers classified by research assistants, the algorithm learns what keywords are associated with empirical work and theoretical work and then can quickly classifies thousands of other papers that weren’t reviewed directly by the researchers.

The figure above shows the prevalence of empirical work as determined by the authors’ model has been rising across fields since 1980. The authors note that the empirical turn is not a result of certain more empirical fields overtaking other more theoretical ones, but instead every field becoming more empirically-minded.

Related Articles

Economic research evolves: fields and styles.

We examine the evolution of economics research using a machine-learning-based classification of publications into fields and styles. The changing fiel...

• Search Search Please fill out this field.

What Is Econometrics?

Understanding econometrics.

• Limitations
• Econometrics FAQs

The Bottom Line

• Corporate Finance
• Financial Analysis

Econometrics: Definition, Models, and Methods

Adam Hayes, Ph.D., CFA, is a financial writer with 15+ years Wall Street experience as a derivatives trader. Besides his extensive derivative trading expertise, Adam is an expert in economics and behavioral finance. Adam received his master's in economics from The New School for Social Research and his Ph.D. from the University of Wisconsin-Madison in sociology. He is a CFA charterholder as well as holding FINRA Series 7, 55 & 63 licenses. He currently researches and teaches economic sociology and the social studies of finance at the Hebrew University in Jerusalem.

Econometrics is the use of statistical and mathematical models to develop theories or test existing hypotheses in economics and to forecast future trends from historical data. It subjects real-world data to statistical trials and then compares the results against the theory being tested.

Depending on whether you are interested in testing an existing theory or in using existing data to develop a new hypothesis, econometrics can be subdivided into two major categories: theoretical and applied. Those who routinely engage in this practice are commonly known as econometricians.

Key Takeaways

• Econometrics is the use of statistical methods to develop theories or test existing hypotheses in economics or finance.
• Econometrics relies on techniques such as regression models and null hypothesis testing.
• Econometrics can also be used to try to forecast future economic or financial trends.
• As with other statistical tools, econometricians should be careful not to infer a causal relationship from statistical correlation.
• Some economists have criticized the field of econometrics for prioritizing statistical models over economic reasoning.

Investopedia / Michela Buttignol

Econometrics analyzes data using statistical methods in order to test or develop economic theory. These methods rely on statistical inferences to quantify and analyze economic theories by leveraging tools such as frequency distributions , probability, and probability distributions , statistical inference, correlation analysis, simple and multiple regression analysis, simultaneous equations models, and time series methods.

Econometrics was pioneered by Lawrence Klein , Ragnar Frisch, and Simon Kuznets . All three won the Nobel Prize in economics for their contributions. Today, it is used regularly among academics as well as practitioners such as Wall Street traders and analysts.

An example of the application of econometrics is to study the income effect using observable data. An economist may hypothesize that as a person increases their income, their spending will also increase.

If the data show that such an association is present, a regression analysis can then be conducted to understand the strength of the relationship between income and consumption and whether or not that relationship is statistically significant—that is, it appears to be unlikely that it is due to chance alone.

Methods of Econometrics

The first step to econometric methodology is to obtain and analyze a set of data and define a specific hypothesis that explains the nature and shape of the set. This data may be, for example, the historical prices for a stock index, observations collected from a survey of consumer finances, or unemployment and inflation rates in different countries.

If you are interested in the relationship between the annual price change of the S&P 500 and the unemployment rate, you'd collect both sets of data. Then, you might test the idea that higher unemployment leads to lower stock market prices. In this example, stock market price would be the dependent variable and the unemployment rate is the independent or explanatory variable.

The most common relationship is linear, meaning that any change in the explanatory variable will have a positive correlation with the dependent variable. This relationship could be explored with a simple regression model, which amounts to generating a best-fit line between the two sets of data and then testing to see how far each data point is, on average, from that line.

Note that you can have several explanatory variables in your analysis—for example, changes to GDP and inflation in addition to unemployment in explaining stock market prices. When more than one explanatory variable is used, it is referred to as multiple linear regression . This is the most commonly used tool in econometrics.

Some economists, including John Maynard Keynes , have criticized econometricians for their over-reliance on statistical correlations in lieu of economic thinking.

Different Regression Models

There are several different regression models that are optimized depending on the nature of the data being analyzed and the type of question being asked. The most common example is the ordinary least squares (OLS) regression, which can be conducted on several types of cross-sectional or time-series data. If you're interested in a binary (yes-no) outcome—for instance, how likely you are to be fired from a job based on your productivity—you might use a logistic regression or a probit model. Today, econometricians have hundreds of models at their disposal.

Econometrics is now conducted using statistical analysis software packages designed for these purposes, such as STATA, SPSS, or R. These software packages can also easily test for statistical significance to determine the likelihood that correlations might arise by chance. R-squared , t-tests ,  p-values , and null-hypothesis testing are all methods used by econometricians to evaluate the validity of their model results.

Limitations of Econometrics

Econometrics is sometimes criticized for relying too heavily on the interpretation of raw data without linking it to established economic theory or looking for causal mechanisms. It is crucial that the findings revealed in the data are able to be adequately explained by a theory, even if that means developing your own theory of the underlying processes.

Regression analysis also does not prove causation, and just because two data sets show an association, it may be spurious. For example, drowning deaths in swimming pools increase with GDP. Does a growing economy cause people to drown? This is unlikely, but perhaps more people buy pools when the economy is booming. Econometrics is largely concerned with correlation analysis, and it is important to remember that correlation does not equal causation.

What Are Estimators in Econometrics?

An estimator is a statistic that is used to estimate some fact or measurement about a larger population. Estimators are frequently used in situations where it is not practical to measure the entire population. For example, it is not possible to measure the exact employment rate at any specific time, but it is possible to estimate unemployment based on a randomly-chosen sample of the population.

What Is Autocorrelation in Econometrics?

Autocorrelation measures the relationships between a single variable at different time periods. For this reason, it is sometimes called lagged correlation or serial correlation, since it is used to measure how the past value of a certain variable might predict future values of the same variable. Autocorrelation is a useful tool for traders, especially in technical analysis.

What Is Endogeneity in Econometrics?

An endogenous variable is a variable that is influenced by changes in another variable. Due to the complexity of economic systems, it is difficult to determine all of the subtle relationships between different factors, and some variables may be partially endogenous and partially exogenous. In econometric studies, the researchers must be careful to account for the possibility that the error term may be partially correlated with other variables.

Econometrics is a popular discipline that integrates statistical tools and modeling for economic data, and it is frequently used by policymakers to forecast the result of policy changes. Like with other statistical tools, there are many possibilities for error when econometric tools are used carelessly. Econometricians must be careful to justify their conclusions with sound reasoning as well as statistical inferences.

The Nobel Prize. " Simon Kuznets ."

The Nobel Prize. " Ragnar Frisch ."

The Nobel Prize. " Lawrence R. Klein ."

Statistics How To. " Endogenous Variable and Exogenous Variable ."

• Terms of Service
• Editorial Policy
• Privacy Policy
• Your Privacy Choices

• Policy & Society

Peter Wilschke ’24, political science and economics, publishes empirical research as the sole author in the State and Local Government Review journal

Published: May 23, 2024

By: Catalina Sofia Dansberger Duque

Peter Wilschke ’s weeks leading up to graduation have been filled with unexpected excitement. His article “Political Drivers of State Fiscal Cyclicality” has been accepted for publication in the State and Local Government Review , the official journal of the Section on Intergovernmental Administration and Management of the American Society for Public Administration. The journal shares the latest research on state and local governments and the intergovernmental dimensions of public-sector activity.

“I’m exploring whether and how political factors—political polarization, turnover, and electoral competition—work to explain why some U.S. states tend to spend more in good times and less in bad times, unlike the federal government,” says Wilschke, who presented the paper at UMBC’s 2024 Undergraduate Research and Creative Achievement Day. 

During a recent conference, editors of State and Local Government Review shared with Eric Stokan , associate professor of political science and Wilschke’s mentor, that the acceptance rate of articles is about 14 percent. “That’s for the entire field of scholars,” says Stokan. “Beyond being methodologically sophisticated, several at our Midwest Political Science Association conference also noted how truly important Peter’s work is for the field and practice.”

Pursuing a research idea

A couple of years ago Wilschke attended UMBC’s Center for Social Science Scholarship’s Mullen Lecture given by Carlos A. Vegh, the Fred H. Sanderson professor of international economics at Johns Hopkins University. Vegh discussed his research on how fiscal policy is conducted over the business cycle in both developing and developed countries. Wilschke wasn’t expecting the talk would set in motion a two-year research project. “It sparked an urge in me to find out more. I knew there was more to the story,” says Wilschke.

This was Wilschke’s first time leading a research project. “I didn’t quite know what I was getting myself into,” says Wilschke. He shared the idea with Carolyn Forestiere , professor of political science and his professor for research methods in political science, and she connected him with Stokan. 

Stokan says he enjoyed working with Wilschke weekly over two semesters to help him think through all aspects of the work. He taught Wilschke R, a programming language for statistical computing and data visualization. Stokan also suggested pieces of literature on the topic, which helped Wilschke think through the operationalization of his variables and analytic specifications, and framing policy. “Each session he would come back far ahead of where I thought he would be. I would consistently say, ‘Okay, your next step should be X,’ and he would have done X, Y, and Z,” says Stokan. “It was amazing. He learned R more quickly than anyone I have ever met.” Wilschke adds, “It was very helpful to have a mentor who had gone through this process multiple times before.”

After several drafts, Wilschke wanted to publish it in the Pi Sigma Alpha—the national political science honor society—undergraduate research journal. Stokan encouraged Wilschke to share the paper with Forestiere, the faculty advisor for Pi Sigma Alpha. “Eric and I encouraged Peter to submit it to a professional journal,” says Forestiere. “We figured that it would be a learning experience even if it was rejected.” They were all delighted when the journal asked Wilschke to revise and resubmit. “We were ecstatic when it was finally accepted for publication!”

Preparing for the unexpected

During the same time, Wilschke was interning at The Hilltop Institute at UMBC, a nonpartisan research organization at UMBC dedicated to improving the health and wellbeing of people and communities. He worked for Morgan Henderson , principal data scientist and affiliate professor of economics, and Morgane Mouslim , policy analyst, on a research project funded by the National Science Foundation on hospital pricing transparency. His two-year internship entailed helping to organize data collected from hundreds of hospitals and writing a news brief, “The Impact of Market Concentration on Hospital Pricing” and presenting it at URCAD 2024. 

“Peter is an excellent researcher who helped our hospital price transparency project significantly over the past two years,” says Henderson. “The quality of his work is top-notch—we predict that he’ll go far.”

Wilschke did not plan on contributing empirical research to the field of political science as an undergraduate student. Looking back he is grateful for the classes and opportunities UMBC afforded him that prepared him to follow a hunch. Wilschke said that he felt that some students don’t look forward to statistical analysis or research methods classes in economics or political science because they’ve heard the classes can be difficult and may appear irrelevant at the moment.

“Once you take these courses, your world is kind of open to how empirical research is actually conducted in those fields. Without these classes I would not have known where to start,” says Wilschke. He advises students to approach empirical work as a combination of two things. “You have to care about your research question to push through all the time and hard work needed,” says Wilschke, “and put to work all the research skills you’ve learned to answer a question that needs to be answered, that policymakers can use to improve people’s lives.” This summer, Wilschke will work as a research assistant at the Federal Reserve Board of Governors in Washington, D.C.

Tags: CAHSS , CAHSS_research , Economics , PoliticalScience , Research

Related Posts

• Science & Tech

UMBC statistician selected to work with Addis Ababa University in Ethiopia

• Class of 2024

Pam Voulalas ’24—From molecular neuropharmacologist to classical music composer

• Campus Life

PEZ—The Sweetest Hobby

Want more umbc news, get top stories delivered to your inbox..

Sign up for our weekly UMBC Top Stories email :

Share a story idea and learn more about the news team.

Search UMBC Search

• Accreditation
• Consumer Information
• Equal Opportunity
• Privacy PDF Download
• Web Accessibility

Search UMBC.edu

• Search Menu
• Sign in through your institution
• Advance articles
• Author Guidelines
• Submission Site
• Book Reviews
• Open Access
• About Journal of International Economic Law
• Editorial Board
• Advertising and Corporate Services
• Journals Career Network
• Self-Archiving Policy
• Dispatch Dates
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Introduction, the role of the scholar/practitioner in investment arbitration, research design and empirical materials, citation of legal scholarship in icsid arbitration, the development of a definition of an ‘investment’ in icsid arbitration, the definition of an ‘investment’ in the scholarship of arbitrators, investment arbitration as a dialectical relation between theory and practice, the influence of legal scholars on the development of international investment law.

William Hamilton Byrne, Assistant Professor, MOBILE Center of Excellence for Global Mobility Law, Faculty of Law, University of Copenhagen, Copenhagen 2300, Denmark. Tel: +35222626; Email: [email protected] . This research was funded by the Danish National Research Foundation Grant No DNRF169. I would like to thank Henrik Palmer Olsen, Mikael Rask Madsen, Lucía López Zurita, and commentators at workshops in Miami and Copenhagen, as well as the anonymous reviewers and editors, for helpful comments.

• Article contents
• Figures & tables
• Supplementary Data

William Hamilton Byrne, The influence of legal scholars on the development of international investment law, Journal of International Economic Law , 2024;, jgae014, https://doi.org/10.1093/jiel/jgae014

• Permissions Icon Permissions

International investment arbitration is inhabited by actors who theorize the law as scholars and decide on it when arbitrating, which raises critical questions concerning their normative role within the legal system. This article explores empirically the influence of legal scholars on the development of inter-national investment law through a combined legal-empirical and sociological approach designed to capture the multidirectional nature of influence. It relies on a novel database of all citations to legal scholarship in International Centre for the Settlement of Disputes (ICSID) case law, which is analysed quantitatively and qualitatively and contextualized alongside interviews with 16 leading arbitrators and lawyers. The article firstly introduces the arbitrator/academic as an agent of international law, and then details the methodology. It then turns to provide an overview of citations to legal scholarship in ICSID case law, before considering why legal actors find it useful. The following parts show how legal scholarship influenced the development of the definition of an ‘investment’ in ICSID arbitration through an analysis of legal decisions and scholarship authored by arbitrators. The final part goes into conversation with the actors to further unravel how they manage competing and complementary obligations towards theory and practice, and the article concludes by reflecting on the ethical implications of their law-making function.

It is often observed that much of the legal scholarship written on international investment arbitration is authored by persons who practice in it. This came ahead as a potential conflict of interest in Urbaserv Argentina , where Argentina brought a challenge before an International Centre for the Settlement of Disputes (ICSID) Tribunal to decide whether arbitrator McLachlan QC should be disqualified, as he had described the Maffezini decision under issue as ‘heretical’ in his leading textbook. 1 The Tribunal denied the challenge, holding that ‘one of the main qualities of an academic is the ability to change his/her opinion as required in light of the current state of academic knowledge’ and that ‘a legal scholar who becomes an ICSID arbitrator does not lose his/her capacity of being a scholar that conveys academic opinions’. It further noted that if it were to allow the challenge, any arbitrators who had expressed opinion on the aspect of ICSID arbitration could potentially be subject to disqualification, thereby leading to ‘paralysis’ of the system.

Whilst switching roles as part of an ‘invisible college’ has certainly been not uncommon within the international legal profession, it has drawn suspicion in arbitration as part of a wider debate about ‘double hatting’ amongst arbitrators. 2 In a legal order that is highly decentralized, arbitrators have come to play a significant role in the development of international investment law through the affirmation of a body of legal decisions. 3 This gives rise to legitimacy issues, particularly in respect of the financial incentives that are offered for arbitrating on investment tribunals. 4 However, the scholar/practitioner also faces difficult questions on how to manage the balance between theory and practice, as their role is to not only decide legal disputes but also contribute to the law’s cohesive formulation. This further indicates that these actors are also uniquely placed to develop international investment law as a system of legal principles.

Despite being such a lynchpin, the role of the arbitrator/academic in the development of international investment law has received only cursory attention in the previous literature. Sociological analyses have provided insight into the field of international investment lawyers 5 and identified power brokers in an elite group of arbitrators, lawyers, secretaries, and expert witnesses. 6 Quantitative studies have mapped the publication culture of arbitration scholars and provided data on the frequency of citation of scholarship by investment tribunals. 7 This literature provides compelling observations on the nature of the profession and its use of legal materials but only in pieces of a broader puzzle; the central role is played by legal scholars in making international investment law through their integration in international arbitration.

This article empirically explores the influence of legal scholars on the development of international investment law. It takes this as both a legal-empirical and sociological issue. It is a legal-empirical issue because influence on legal decision-making occurs at a discursive level through legal texts, which must be assessed on empirical terms to detect when scholarship influences law, but also, and crucially, how this process works. 8 It is a sociological issue because influence is not a simple causal relation, because it is affected by the mind of the agent and social environment in which they make their decision, which in turn helps to explain how scholarship shapes as part of a legal community. 9 This article adopts a multimodal approach to unveil a more encompassing understanding of the symbiotic relationship between scholars and practitioners that makes investment law and arbitration as a legal system.

To this end, the article takes up a sociolegal methodology that analyses the practice of scholars and practitioners in investment arbitration through three principal data sources: firstly, a quantitative analysis of citations to legal scholarship in ICSID awards and decisions on jurisdiction rendered by ICSID Tribunals from 1988 to the end of 2019 (376 Awards, 112 Orders, and 53 separate opinions); second, a qualitative analysis of ICSID case law concerning the development of the legal definition of an ‘investment’ that was analysed alongside legal scholarship authored by arbitrators sitting on these cases; and thirdly, semistructured interviews with 16 leading investment arbitrators and lawyers conducted under conditions of anonymity via Skype from 20 April to 2 May 2020. These data points were not considered in isolation but were put into dialogue to derive an overall, legal-empirical and sociological synthesis on the relationship between theory and practice in investment arbitration.

The article proceeds as follows. The first section introduces the scholar/practitioner as an actor of investment law performing a dual role of law application and knowledge production. The section after that details the methodology. The following section then undertake a quantitative analysis of legal schol-arship cited by ICSID Tribunals, and thereafter qualitatively assess the interaction between case law and legal scholarship in the establishment the definition of an ‘investment’ under the ICSID Convention. The final section unravels the nature of a social field as com-posed of theoreticians and practitioners through the interviews, and the article concludes by summarizing the findings and reflecting on the ethical implications of investment arbitration as a praxis.

This part firstly sets the theoretical frame for analysis by introducing the role of the arbitrator/academic as an agent for legal development in international investment arbitration. As practitioner, their primary task is to apply the law, that is, to decide legal disputes between parties on the basis of applicable law. 10 As scholars, their primary task is produce knowledge on what is or should be the law by considering legal events as part of a normative order. 11 These roles must be further unpacked before considering how they interact in a legal system.

The arbitrator is firstly engaged in a process of law application when undertaking the process of arbitrating because they apply law to factual disputes in order to find a legal resolution. This function is outlined in positivist legal philosophy. For HLA Hart, judges are ‘law-applying authorities’ entrusted with preserving a legal system. 12 For Kelsen, law application is what makes law extend from general principles to specific meanings. 13 It of course remains open to question whether judges merely apply the law or develop it, and this tension is reflected in diverging views on the normative role of the investment law arbitrator. 14 Investments tribunals are in principle ‘sovereign and may retain … a different solution for resolving the same problem … (but they) are also free to adopt the same solution’. 15 However, the growth of jurisprudence in recent decades has generated the impression that it is a relatively coherent body of law 16 that arguably operates through a system that is akin to legal precedent. 17

The academic is secondly engaged in an exercise of knowledge production. Academics are the ‘teachers’ of international law—they are the principal authors of the ‘teachings of the most highly qualified publicists’, 18 they frequently instruct university courses on the subject, and are also often asked to provide forms of counsel. 19 Their scholarship is an exercise in knowledge production because whether alone or in conjunction, this work provides a basis for understanding the law as legal doctrine. 20 This ‘doctrine’ can also be conceived as a ‘theory’ that supports the ‘practice’ of law in a co-constitutive relationship. 21 From this perspective, the ‘theory’ and ‘practice’ of investment law could be regarded as ‘two sides of the same coin’, and its usefulness is not restricted to litigation. 22 The academic plays no ‘formal’ role in the developing investment law but their work may otherwise be influential, as it provides a source of background knowledge for the understanding the law through its function of theorization.

However, these social categories are not neatly bounded, as investment arbitrators often wear two hats, one for theory and one for practice, and one could expect some degree of cross-fertilization between their occupations. Sociological analysis has explored the foundations of investment arbitration as forged in competition between academics and technocrats, 23 whilst recent studies indicate that public and private international law scholars are now amongst the most oft-appointed investment arbitrators. 24 Quantitative studies of citations have revealed that legal scholarship is the second most cited ‘source of interpretative argument’ in investment arbitration, 25 where it is typically cited in relation to substantive issues (compared to procedural issues, where references to arbitral decisions are more common). 26 Yet, whilst each of these approaches has merits, they are not well attuned for analysing ‘legal development’.

This article thus proceeds with a multimodal approach that combines elements of legal-empirical and sociological analyses. It focuses principally on the role of legal scholarship within law but develops further insights into the sociolegal function of the arbitrator/academic because it takes these two research objects as intimately related. This choice is deliberately intended to track the integrative relationship between the ‘theory’ of arbitrators and academics and the ‘practice’ of case law and the social field by putting aspects of their interrelations into action.

The first stage arises from quantitative analysis of all citations to legal scholarship in all ICSID awards and decisions on jurisdiction from 1988 to the end of 2019 (376 ICSID Awards and 112 ICSID Orders on Jurisdiction, and 53 separate or dissenting opinions). 27 This is of course is not the ‘complete’ dataset, because as much as half of ICSID awards and decisions are confidential and unpublished. A total of 94,510 footnotes—where citations almost exclusively appear—were extracted from these documents via computational methods (and for 95 documents where automation was not possible, by hand) and then compiled in an Excel spreadsheet. 28 The footnotes were then read manually to identify citations to legal scholarship using identifying points of textbooks, monographs, journal articles, chapters in edited works, blog posts, and texts produced by institutions such as the International Law Commission (ILC). 29 Expert opinions were not included in this dataset because they do not theorize law in the same way as scholarship but provide a form of testimony. 30 Citations to legal scholarship were found in 1267 footnotes. Citations were coded with reference to the author of the scholarship and the area of law that it appeared to address as indicated by the title (eg ‘damages’ is likely about ‘damages’). This method is imprecise and thus results on citation by subject matter can be only indicative. 31

The second stage arose from a qualitative analysis of influence of legal scholarship on a line of legal decisions. This was undertaken on the assumption that citation alone cannot be equated with influence, 32 as citations arise for different purposes, such as to express agreement, disagreement, or to contrast analyses. 33 As it would be near impossible to examine every citation to scholarship, a case study was chosen on an issue where citation seemed to be common—the legal definition of an ‘investment’. These cases were read using techniques of content analysis that examined how cited sources influenced the Tribunal’s reasoning and how arbitrators themselves develop a legal position (in contrast to the doctrinal exercise of finding a ‘correct’ decision). 34 This data was then compared with all legal scholarships authored by arbitrators sitting on these decisions to see how they approached the issue in their writings and whether this indicated that they posited a particular theory of what the law should be.

The final stage arose from 16 semi-structured qualitative interviews conducted with investment arbitrators and lawyers under conditions of anonymity by the author alone via Skype in the period of 20 April–2 May 2020. The interviews sought to ascertain the broader influence of scholars and the role of theorizing on law within the social field beyond what is visible from texts of legal decisions. 35 The interviews were obtained by ‘cold calling’ 100 requests to persons listed in the International Arbitration Institute directory of members. Of the interviewees, 12 had acted as arbitrator in ICSID arbitration and four as lawyer only. Most of the interviewees were pre-eminent actors in the field. For each interview, a guide was prepared on the basis of a common framework, and interviewees were asked questions on their background, extent of practice, how they use legal sources, and how they see their different roles in arbitration. Having outlined these preliminary steps, we can now turn to the analysis.

The first step of quantitative analysis was thus undertaken to obtain a broad overview of the extent of citation to scholarship in ICSID case law across points of analysis concerning frequency of citation over time, subject matter of scholarship, and identity of cited scholars. This provides an entry point for considering how legal scholarship is cited ICSID case law, how it is integrated into legal decisions, and then ultimately why legal actors find it useful.

Of the 541 analysed documents, legal scholarship was cited in almost every ICSID award, decision, or separate opinion, arising in approximately 1.3 per cent of all footnotes. 36 Whilst this does not seem like a high citation rate, a large portion of any judgment will inevitably comprise a large number of footnotes documenting facts and sources of law. 37 Approximately one-third of all citations arose in the Tribunal’s rehearsing of pleadings, thus reflecting the party driven nature of arbitration, but these citations were usually not repeated later in the documents. This also shows—and as the interviews confirm—that arbitrators mostly turn to legal scholarship on their own accord. Citation rates to legal scholarship have remained relatively consistent over time—1.8 per cent of all footnotes in ICSID cases in 1988 to 1 per cent in 2019—in a period when the typical length of judgments has considerably expanded (so there are more footnotes per case overall). This also suggests that arbitrators continue to turn to legal scholarship despite the rise past of past arbitral decisions as a source of authority, which qualifies the assumption—and as the case study further supports—that the influence of legal scholarship will decline as legal precedent grows. 38

The data were then analysed to ascertain the subject matter to which citations to legal scholarship referred. As noted earlier, this coding was undertaken with reference to the title of the work and the data were calculated as a percentage within all citations to scholarship (100 per cent).

These data points should not be taken as absolute values but rather to give a guide to the relative share of citations across subject matter. For instance, ‘jurisdiction’ is a very broad concept that encapsulated issues of consent and denunciation, whilst ‘denial of justice’ is narrower, thus creating distortions in the data. State responsibility is something of an outlier because the ILC’s work has become the placeholder on this topic and it was easy to idenitfy. 39 Damages were also stark which may have made the results on this part disproportionately higher. Vice versa, in some instances it was difficult to infer what a citation referred to, which raised residual categories. ICSID Tribunals also sometimes cite ‘non-legal scholarship’, such as economic analyses.

Nevertheless, some clear implications can be drawn from the data. It firstly clearly shows that legal scholarship to cited all major conceptual categories of international investment law, and its influence is not limited to contested subjects (such as the legal definition of an ‘investment’). 40 The dispersion furthermore coheres with the results of a study on international arbitral precedent which found a similar distribution across conceptual categories; issues of ‘expropriation’ amount to roughly 20 per cent of all citations to precedent, damages are 14 per cent, denial of justice is 3 per cent, and so on. 41 This indicates that legal scholarship likely holds some continuing and dynamic relation with case law. It could be anticipated that precisely because law’s scope is limited, legal scholarship helps it to adapt to new circumstances and conflicts.

A more sociological account offers explanations for this relationship between legal decisions and scholarship which is simply not possible to ascertain from quantitative data. The interviews overwhelmingly showed that practitioners turn to scholarship to understand the law, as one interviewee aptly noted, ‘(i)f I don’t understand an issue then I would generally turn to the main treatises on investment arbitration’. 42 Legal scholarship offers a means to apply the law to new facts, ‘(some) cases … you know them by heart … (s)ometimes the scholarship is very helpful ….’ 43 This also gives a sense of form to the messy array of case law: 44

I will first go to the main textbooks … I won’t just go (straight) to Schreuer’s ( Commentary ,) I will go to Dolzer and Schreuer, I will go to the (inaudible) book on international arbitration. … Then I will turn to doctrine (academic articles) and then I will turn to case law. I am obviously not going to read every case … It’s a pyramid, you start off with the textbooks, then the articles, and then the case law…

Legal scholarship thus structures their understanding but also provides a means to argue. Another similarly suggested that s/he would consult scholarship and case law ‘at the same time’, 45 comparing texts to find a medial point, which challenges the view described above that the use or influence of scholarship declines as precedent grows. In any given case, both will usually be present, but one source may be treated as more authoritative. What is ultimately cited in a legal decision will thus not necessarily reflect the true influence of legal scholarship.

The data was then analysed to find the most cited authors in ICSID case law by reference to the frequency of authors’ names arising in the data set of citations to scholarship.

These statistics may be unsurprising for an investment lawyer or scholar. Schreuer’s (co-authored) Commentary is often described as ‘being the book on that particular subject’. 46 One arbitrator’s comments further reveal this as social role of providing argumentative authority: 47

Its a weird thing. The one totally exceptional thing is called Christoph Schreuer, who’s Commentary on the ICSID convention is actually treated as the Bible. That specific book has a weight far beyond the average of investment literature, and in international law generally. There is no other book in public international law which has so much authority … Even Crawford’s edition of Brownlie’s Principles of Public International Law … is not quoted as much.

However, other textbooks are also frequently cited in ICSID case law, including Douglas ( The International Law of Investment Claims ) and Newcombe and Paradell ( Law and Practice of Investment Treaties ). Crawford also accrued a high number of citations, but overwhelmingly for his role in leading the finalization of the ILC’s Articles on State Responsibility. This was distinct from UNCTAD ‘Pink Series’, which is produced by scholars but usually cited as UNCTAD as the author. One arbitrator explained the unique role played by Crawford: 48

I would also refer to the ILC Articles – it is an intermediate source to me, it is more than the doctrine in terms of weight, but it is not (scholarship). Because it’s a commentary and because it’s James Crawford, he would be rather like a scholar …. (but) in this context, it gives it more weight than if it was in a book.

A number of authors were frequently cited for their work on specific areas of the law. Thus, Ripinsky was mostly cited for Damages in International Investment Law , whilst Paulsson was usually cited for Denial of Justice in International Law . There is a large number of citations to international law textbooks as well as monographs such as Cheng’s General Principles of Law as Applied by International Courts . Their citation rate was also high, which seems odd; why would an actor cite Cheng instead of Schreuer, if Schreuer wrote the Bible? The interviews indicated that this was largely because the authority that they associated with specific works arose from its function as providing a source of legal knowledge: 49

This has partly to do with experience, but also the degree of seriousness of the underlying research that their literature is based on. So, is it comprehensive, is their analytical depth, is there a theoretical background, is there an extending of research into the broader sources, the broader framework. And there is some very valuable scholarship, and there is scholarship that has very little value, just on an academic assessment, but one also has (to make that assessment) in the case, on the quality of the scholarship.

The interviewees further indicated that this assessment was mostly not affected by the extent of an actor’s practice in the law, as one arbitrator noted: ‘you can mark a difference between true scholars and people trying to advertise themselves as scholars.’ 50 Indeed, of Langford, Behn, and Lie’s analysis of the ‘power brokers’ (measured by appointments as arbitrator, lawyer, witness, etc.) only Crawford, Paulsson, and Gaillard appear as most cited authors. 51 One interviewee, when asked if practical work of the author was relevant in this respect, stated 52

…. there is a reason why certain scholars dominate. Their ideas are good, their ideas resonate in practice, and for anyone coming in to the field … if you look at Schreuer, he is the foremost scholar on the ICSID Convention, and he’s been a scholar presently first, he wrote the Commentary when he had hardly any practical involvement, and it became the most cited source … because it is a very well thoroughly researched source.

In effect, they mostly turn to scholarship for what a doctrinal scholar would call ‘quality’ of the scholarship and a sociologist might refer to as the ‘expertise’ of the scholar, 53 and not necessarily because citing a specific author enhances the persuasive power of an argument. 54 Of course, the relation between knowledge and power can be cumulative, as one lawyer noted on what s/he finds authoritative, ‘people like Schreuer, Gaillard, they have really the foresight of what should be a ruling’. 55 Vice versa, a high citation rate may also result from the force of habit as legal actors rely on familiar authors due to time constraints. 56 But these actors seem mostly drawn to scholarship for its hermeneutic resonance; the theory to the practice, or the legal knowledge that supports the tasks of lawyering and arbitrating. This indication is further underlined by the absence of scholarship that is more critical of investment arbitration in the citations, as a field of literature that has blossomed over the last decade. 57

The analysis has ultimately revealed how doctrinal legal scholarship is thoroughly embedded in the normative architecture of international investment law. It is cited relatively frequently across all subject areas, and persistently across time despite the rise of arbitral precedent. The interviews indicate that investment arbitrators and lawyers seem to be mostly drawn to scholarship because it helps them to understand new issues or to apply the law within a wider doctrinal framework. Citation can thus be only one indication of the influence of legal scholarship—indeed, it is likely always present in legal decisions in some capacity but often buried within legal arguments it also appears to play an integral sociological role of providing a source of background knowledge for legal decision makers. However, the mere presence of a citation cannot itself be equated with influence on legal developments, which ultimately requires a more qualitative assessment of the role of scholarship in arbitration.

This quantitative analysis further revealed that citations to scholarship associated with the legal definition of an ‘investment’ were especially common. This provision is ‘central to the Convention ’ 58 but it remains undefined, and thus debates over its meaning are often ‘more lengthy and hard fought than any eventually ensuing merits phase’. 59 Moreover, this is an area where legal scholarship has been regarded as particularly influential, on the assumption that Schreuer virtually invented the prevailing test. 60 A closer analysis of legal decisions reveals that the definition has emerged in a dialogue between scholars and arbitrators, whereby arbitrators draw on legal knowledge to promote legal change, and thereby develop international investment law in arbitral practice.

The foundation of a definition of an ‘investment’ between law and scholarship

In the first key case, Fedax N.V.v The Republic of Venezuela , arbitrators Vicuña, Heth, and Owen commenced by stating that because the term had been broadly understood in previous decisions it had never been a ‘major source of contention’. 61 The Tribunal followed previous awards holding that an ‘investment’ must be derived from the consent of the parties, and agreed with ‘distinguished commentators’ that this implies a broad interpretation, further rejecting Venezuela’s submission that it was identical to the meaning found in a dictionary of economic terms. 62 The Tribunal then catalogued a number of findings of an ‘investment’ in ICSID decisions, before turning to an early article by Schreuer 63 to articulate a central premise; ‘(t)he basic features of an investment (are) … a certain duration, a certain regularity of profit and return, assumption of risk, a substantial commitment and a significance for the host State’s development.’ 64 Legal scholarship here thus provided a crucial role of systematizing emerging legal principles to develop a legal test for an actionable ‘investment’ in ICSID arbitration.

Saliniv Morocco , arbitrated by Briner, Cremades, and Fadlallah, followed closely from Fedax and subsequently became the leading case. 65 Here, the Tribunal firstly affirmed that ‘ICSID case law and legal authors agree that the investment requirement must be respected as an objective condition of the jurisdiction of the Centre (cf E. Gaillard, in JDI 1999…)’. 66 Having surveyed a number of different types of investment, the Tribunal held that ‘(t)he doctrine generally considers that investment infers: contributions, a certain duration of performance of the contract and a participation in the risks of the transaction (cf commentary by E. Gaillard).’ 67 The Tribunal then further added to this formulation, ‘on the Convention’s preamble, one may add the contribution to the economic development of the host State’. 68 The Tribunal thus upheld the Fedax criteria but excluded the requirement of ‘profit and return’, and substituted the element of a ‘commitment’ for a ‘contribution’ to ‘economic development’.

Scheuer was only cited in this instance on the requirement of ‘duration’ and not for the test itself, and a closer reading of the text suggests that the Tribunal’s influences were diverse. In 1982, Delaume had written that the ICSID Convention definition of an ‘investment’ required some contribution to the host State’s economic development. 69 Furthermore, in CSOB , which was decided prior to Salini , the Tribunal had outlined basis of this element for finding an investment in the preamble of the ICSID Convention . 70 In Salini , the Tribunal cited Delaume’s article on the issue of the formal requirements for consent but did not refer to CSOB at any stage. 71 The Tribunal also contrasted its analyses (as ‘C.f’) at multiple stages with a journal article by Gaillard ‘(J.D.I’), which was in fact a case note on the Fedax decision expressing some disagreement with the criteria. The decision shows how the influence of legal scholarship can sometimes be implicit, emerging as an intertextual dialogue between scholars and arbitrators.

Consolidation and contestation of an ‘investment’ in the arbitral practice

Subsequent cases seemed to suggest a consolidation of the legal definition of an ‘investment’. Joy Mining v Egypt , constituted by Vicuña, Craig, and Weeramantry, upheld Salini as an objective set of jurisdictional requirements by citing as its authority the newly released ‘C Schreuer, The ICSID Convention: A Commentary (2001)’. 72 Shortly after, Schreuer himself arbitrated on the meaning of an ‘investment’ alongside Kaufmann-Kohler and Otton in Saipem v Bangladesh , where the Tribunal held that ‘it will apply the well-known … “Salini test”’, but cited the Salini decision rather than Schreuer’s scholarship, thus reinforcing the precedential weight of the test. 73 In Jan de Nulv v Egypt , arbitrators Kaufmann-Kohler, Mayer, and Stern applied the ‘Salini test’ but added that these elements are ‘interrelated and … will normally depend on the circumstances of each case’, further citing the expert testimony ‘statement of Prof. Schreuer’. 74 The Bayandir Tribunal adopted a similar formulation but cited Schreuer’s scholarship to reaffirm a separate basis of the Salini criteria in legal scholarship. 75 In these cases, scholarship and precedent were cited almost interchangeably and this reciprocal process of confirmation likely enhanced the doctrinal authoritativeness of the Salini test.

At the same time, fissures began to emerge in its theoretical tenets. In L.E.S.I.v Algeria , Gaillard, the author of the text cited as Cf in Salini held alongside Faures and Tercier, that contribution to economic development was not essential to satisfy the test. 76 In Biwater Gauff arbitrators Born, Landau, and Hanotaiu cited Schreuer’s Commentary to hold that the criteria should not be strictly applied because they do not appear in the Convention . 77 Subsequent Tribunals split on these questions. Hwang, the sole arbitrator in Malaysia Historical Salvors , sought to reconcile the emerging positions that saw Salini as either a set of ‘jurisdictional requirements’ or ‘typical characteristics’ and found that ‘the consensus of legal authors and ISCID case law holds that Article 25 imposes a double-barrelled’ test; one looking to the consent of the parties, the other to objective requirements of the Centre. 78 A majority of two former ICJ Judges (Schwebel and Tomka) annulled this decision to hold that these criteria were ‘typical characteristics’, by citing scholarship that was critical of arbitrator Hwang’s decision. 79 Judge Shahabuddeen in dissent shed light on the nature of the conflict: 80

What this case hinges on is a perception of the objectives of ICSID: Was the jurisdiction of ICSID meant to be solely dependent on the will of the parties? Or, was it meant to be dependent on the will of the parties subject to conformity with the overriding objectives of ICSID as a body concerned with the economic development of the host State? … The cleavage marks a titanic struggle between ideas, and correspondingly between capital exporting countries and capital importing ones … (If) the subjectivist view … continues to prosper, ICSID may well become just another arbitration institution, competing with a range of others ….

For the Judge, the Convention upheld ‘what Schreuer … calls the “outer limits”’ of an investment’, and a contribution to development was ‘the only possible objective meaning’. 81 The definition of an ICSID ‘investment’ was thus increasingly coming to be contested in legal scholarship and this commentary provided the fuel for subsequent legal developments. Each side of arbitrators seemed to hold a particular theory on the purpose of the definition for ICSID arbitration, and this was coming to be manifested in case law dealing with an ‘investment’.

The purpose of an ‘investment’ for ICSID arbitration

As case law developed it began to encompass a more diverse range of perspectives. In Phoenixv Czech Republic arbitrators Stern, Bucher, and Fernández-Armesto cited Paulwelyn to hold that a State ‘cannot contract out of the system of international law’, and from Lauterpacht to propose that ‘good faith’ was a requisite for finding a protected investment. 82 Yet, in Saba Fakesv Turkey , a Tribunal consisting of arbitrators van Houtte, Lévy and Gaillard cited a new edition of Schreuer’s Commentary on the ICSID Convention (2009) to hold that neither good faith nor a contribution to a host State’s economic development were necessary. 83 These examples show how legal scholarship can support expanding the normative scope of law, but also maintain consistency in legal decision-making across a clear line of principle.

In Ambiente , a Tribunal composed of Simma, Böckstiegel, and Bernárdez sought to revise the issue as an exercise in treaty interpretation, suggesting that the object and purpose of the ICSID Convention could be broad or restrictive. 84 The Tribunal turned to the subsidiary sources to resolve this quandary, and found that an objective interpretation was advocated in many sources. 85 It then tackled ‘the relevance of the so-called Salini test’: 86

In the Commentary’s first edition, Professor Schreuer identified the five criteria enumerated above and characterized them as “typical” to “most of the operations” … In the second edition of the Commentary , Professor Schreuer comments on the rise of the Salini test in these words: “The development in practice from a descriptive list of typical features towards a set of mandatory legal requirements is unfortunate. The First Edition of this Commentary cannot serve as authority for this development.”

The Tribunal sided with Schreuer in this respect, ‘the criteria assembled may still prove useful, provided that they are treated as guidelines’. 87 To recall, this is also similar to the position that Schreuer himself adopted in Sapiemv Bangladesh . Constant affirmation of the test in practice and scholarship thus appeared to have established a doctrine of flexible application. Yet, subsequent decisions still cast doubt on this. In Phillip Morrisv Uruguay , deciding shortly after, Bernardini, Crawford, and Born expressed that the ‘controversy regarding the term “investment” shown by various arbitral decisions and doctrinal writings … is far from settled … there is no such a “ jurisprudence constante ” with respect to acceptance of the Salini test’. 88 Nevertheless, the Salini test has since continued to prevail as a kind of operative precedent. 89

The influence of legal scholarship on legal developments in ICSID arbitration

This case study reveals how legal scholarship influences legal development on different levels. Legal scholarship as cited in ICSID arbitration firstly provided a means to establish a legal principle (eg Fedax, Salini ), and then it supported contestation (eg Historical Salvors, Saba Fakes, Ambiente ), but also normative theorizing on the purpose of this form of dispute settlement (eg Phoenix ). The relationship between case law and legal scholarship functioned like a hermeneutic circle, as legal decisions generated commentary which recursively fed back into legal decisions. Some of the decisions (eg Salini ) also showed that this influence can be implicit. Influence is thus not simply one directional (ie ‘scholarship made law’) 90 but an ongoing interaction that supports the development of legal norms. In this way, legal scholarship also supported the emergence of a legal system, and a means to encounter its engagement with new issues. This example is not exceptional but part of broader interpretative operation that gives meaning to legal provisions. 91

The analysis has further indicated that arbitrators themselves hold theoretical positions (eg Vicuña in Fedax and Joy Mining , or Gailliard in L.E.S.I and Saba Fakes ) which they develop part of a social group, and these can also influence the direction of legal developments. As scholars have rightly noted, ‘achieving investment protection before ICSID … might depend on the doctrinal inclinations of the arbitral tribunal’. 92 The lack of final authority in ICSID arbitration elevates the role of arbitrators and scholars in deciding what the law is, which in practice suggests that respect for settled law requires a commitment to certain principles. 93 Yet, the analysis might also suggest that these ‘theories’ are little more than a disguise for preferences, 94 which raises the spectre of this form of ‘double hatting’ as a kind of normative enterprise conducted by the arbitrator/academic to secure appointments. 95 Further insight into this dynamic can be drawn taking a closer look at the scholarship produced by arbitrators.

All of the legal scholarship authored by arbitrators sitting on this line of cases was thus analysed to see how they approached the issue of an ‘investment’ in their writings. This was undertaken to ascertain how arbitrators use their scholarship to pursue normative points, and the way that their collective work gives shape to a scheme of legal meaning as legal doctrine.

The normativity in doctrinal statements on international investment law

The arbitrators surveyed for this part came from a wide field of experience. There were public international lawyers (Simma, Weeramantry, Buergenthal, Tomka, Shahabuddeen, Crawford) whilst others were more known for expertise in commercial arbitration (Born, Böckstiegel, Mayer, Lévy, van Houtte, Craig, Bernandini, and Vicuña) and some who often transverse fields (Fernández-Armesto, Tercier, Bucher). Others published sparsely (Briner, Landau) and there were also judges of national courts (Otton and Owen). Yet, there was a clear tendency for arbitrators more directly involved in ICSID arbitration to write on the topic. Four of the arbitrators here have also been regarded as ‘power brokers’ of international arbitration. 96

The literature analysed for this part firstly reveals the significant capacity for doctrinal legal scholarship to express a normative position within the confines of doctrinal ‘objectivity’. Thus, in The ICSID Convention: A Commentary , Schreuer (et al) adopt a classic case-based approach that presents case law as evolving across lines of legal principle, but are nevertheless clear on the point that ‘(a) rigid list of criteria is not likely to facilitate the task of tribunals’. 97 Dolzer and Schreuer’s Principles of International Investment Law is more analytical in tone, but ultimately upholds that ‘possible factual settings and existing case law appears to indicate that a combination of the flexible versions of the two approaches’. 98 This scholarship bears the notes of the classic doctrinal systemic methodology, but it also expresses a view at crucial moments. Schreuer (et al) thus appear to hold a particular theory of what the law should be, which shapes the way they approach the task of systematizing legal decisions.

Other arbitrators similarly used their scholarship to express a normative position on the subject. Fadlallah, who arbitrated on Salini , thought that ‘necessary cumulative criteria’ is a ‘dogmatic and formalistic’ and ‘does not seem to be compatible with the specific language of the Convention ’. 99 Schwebel, who sat in the Majority on the Annulment in Malaysia Historical Salvors also used his writings to actively defend his position; ‘in the case of the negotiation of the ICSID Convention , the record is complete … (and) sustains the holding on annulment’. 100 Hanotiau, who arbitrated on Biwater Gauff , offered a more pragmatic response by calling for precise drafting of investment treaties. 101 Stern, who arbitrated on Phoenix and Jan de Nul , wrote that Salini must be supplemented with a wider meaning; ‘I am definitely in favour of … the “outer limits” approach … A good example of the teleological test is found in the Phoenix case, which would result in a more balanced system of arbitration.’ 102 These articles reveal that arbitrators use their scholarship for many different reasons: to express legal points, to call for reform of the system, or even to connect the law to its social consequences.

The doctrine as a normative structure of international investment law

The analysis further reveals that whilst arbitrators have strong opinions on which way the law should go, they generally work to express these through the terms of the practice. Gaillard’s writings are instructive here. In his first article on the subject he cited D. Carreau, T. Flory, and P. Juillard, Droit International Economique (1990) to propose that the criteria of an ‘investment’ should be elements of ‘contribution, duration, and a certain risk to the investor’. 103 Yet, he would later write Salini ‘was an important milestone’ but ‘one may hope that these diverging trends will be harmonized in a manner consistent with … the Convention’s drafters’. 104 He thus appears to hold a normative position that commands respect for the terms of what has been decided as a principle, as greater than the preference of any single actor.

From a more sociological angle, it can be seen how agents express normative evaluations which collectively create a structure of legal knowledge, and this doctrine in turn constrains their freedom, but cannot determine them as they are free thinking individuals. 105 The nature of this dynamic further becomes evident in an article by Hwang, the sole arbitrator in Malaysian Historical Salvors , who concluded his article on the subject by stating ‘(t)he point is not to defend my Award, but to point out that, in the light of the conflicting jurisprudence, the question of what is an investment still remains an open one in the future—even for me’. 106 In other words, the arbitrator/academic can always change their opinions, and these two hats are often worn simultaneously—one for deciding on the law, and one for theorizing it. Whether and how these actors wear one of these hats will always be something at their own liberty.

These texts thus show a tendency for some arbitrators to use their scholarship to pursue normative points, although the vast majority did not write on the topic, which suggests that this phenomenon is not as widespread as often supposed. However, certain actors more invested in the practice appeared to articular a theory, from the perspective of pedigree (Gaillard), system (Schreuer), or teleology (Stern) and the balance that emerges from these approaches is likely not coincidental (ie they are legal interpretative positions which direct legal change—text, context, evolutionary). 107 This suggests that the ability of arbitrators to theorize their practice, and the dialectic between legal positions that emerges from their interactions, supports change and stability in the legal system. The sociological nature of this dynamic, whereby the theories of arbitrators and academics drive legal developments within a field of practice, becomes further clear from the interviews.

The interviews undertaken for this study explored the legal knowledge that enlivens investment arbitration through questions on the process of legal decision-making and how actors see their different roles in the system. The interviews reveal that ICSID arbitration operates through a close relationship between theory and practice, and whilst the pull of the market is omnipresent, its actors are grounded with a sense of responsibility to the legal system.

A field of practice socialized by theory and bound by a sense of obligation

The interviews revealed that cultivating a theoretical element was integral to the process of socialization into the field, and ultimately to being a good practitioner. The common view that investment arbitration was a ‘practitioner’s field’, 108 only developed by doing it or somehow divorced of theoretical content was thus a myth. One arbitrator reflected as such: 109

How you go about handling a procedure, directing or presiding … is something that one learns on the job, but this is not sufficient, you need a lot of theoretical and conceptual knowledge, without which you can’t really seriously assess and adjudicate a case on the law, and I think investment treaty arbitration … there is a lot of public international law, but also a public law aspect … which is necessary in order to do justice to the case.

This continuity was generally traced from law degrees to doctorates, to teaching and to publishing, and ultimately, being a competent lawyer; ‘there is a huge cross fertilization. You need theoretical knowledge to implement in practice, and then you fertilize your theoretical knowledge by the practice’. 110 This learning curve was generally spoken of as a process of mutual consolidation; ‘you learn through the theory and you learn it actually by doing it.’ 111 This integration between theory and practice is ultimately what makes them professionals.

The process of socialization also affects how they approach legal sources, and in particular, the difficult question of arbitral precedent. The interviewees were widely aware that there are ‘notorious issues where there is great divergence’ in the law 112 and this is systemic: 113

(Arbitrators) quote from previous decisions to show or give the impression that there is a consistent body of case law …. But probably on the most important issues of investment law, there is simply no consistency. Fair and equitable treatment, legitimate expectations, MFN dispute resolution provisions ….

Yet, many arbitrators feel a sense of obligation to precedent ‘… I try to be consistent with previous decisions’; 114 or indeed, ‘some consistency is intrinsic in one’s decision making’. 115 This gives effect to shared understandings; ‘most arbitrators have a great sense for building up a system of case law … arbitrators feel … they are part of a system, and are working on creating a system of precedent.’ 116 Some connected this to the public functions of the arbitrator: 117

A lot of cases need to be decided not just on the specifics of the case but also the underlying principle, and is that first of all a principle that is good law, and second of all is acceptable to the international community at large. We are not catering just to the two parties before the Tribunals, or worse the party that appointed you – that’s not compatible with the public function of the arbitrator.

Of course, this view was not universal. One arbitrator sought to underline that ‘arbitration is by its nature … a private activity … arbitrators would not be there unless the parties agreed to it … I sit there as a private individual’. 118 However, most of the interviewees stressed that ICSID arbitration was special; ‘(it) has a different mind-set (to commercial arbitration)’; 119 ‘you realize the public dimension of the case, a public in front of the case…there are consequences for public policy’. 120 Moreover, and as the previous case study showed, this positions can be complementary in infusing a diversity of opinion in investment arbitration system. 121

Co-operation, conflict, and the pull of market

It is now almost a common knowledge that ICSID arbitration is a market, and furthermore ‘it is a really closed market, with law firms and practitioners having their hand on the leverage of new matters, arbitrators that are extremely busy and very relevant’. 122 Theory can thus also be strategic: ‘since the coherence in the case law is superficial, there is a big competitive advantage, in having a theoretical knowledge. There is nothing as practical as good theory … it helps you grasp arguments.’ 123 This might also shape the arguments that are delivered, as some actors noted; ‘you try to take into account who your arbitrators are, whether they are scholars’, 124 or ‘knowing what their thinking is and arguing on it’, 125 ‘they have their own “school of thought” … you may even refrain from providing points because you know that it will be counter-productive’. 126 Yet, the arbitrator is always free to change their opinion: 127

They may try to gauge our suitability on the basis of prior writings, and as to whether that would be favorable for their position – some of the cases, after I see the legal arguments, I can see why a certain party appointed me and found it was favorable for their position – sometimes these things matter and sometimes they don’t matter because at the end I may take a completely different route on a point of law.

The pull of the market also shapes the knowledge that is produced on investment arbitration: 128

I think a lot of people try to position themselves or do position themselves strategically in terms of the scholarship that they do. But of course not everybody. And it’s often also not the real scholars or the real professors who have that attitude … I think there is also a great confusion on legal writings in investment arbitration, there are many practitioners writing on things, and I think that they are principally, strategic. I think often that think they are producing scholarship, but it is probably more professional legal writing. And so one also has to have some sort of idea of what passes as scholarship and what does not.

In other words, ‘you can mark a difference between true scholars and people trying to advertise themselves as scholars’. 129 The data thus suggest that arbitrators reflexively assess the state of their practice and respond to it. They are capable of distinguishing what is legal advocacy and what is legal knowledge, and use this understanding to enrich their approach for cognizing the international investment law and their process of developing it within investment arbitration.

However, whilst this reciprocal relation between theory and practice generates shared understandings, it can also silence certain narratives. The dominance of key players within the field shapes this, as one arbitrator lamented: ‘(t)here are many, many theoretical insights which are of immense value and which do not reach the courtroom unfortunately, due to lack of time, or probably also a lack of interest. …To read, and actually think, (to) truly understand, it takes time.’ 130 Others reflected that the increasing influence of magic circle law firms in investment arbitration may further accelerate this process towards the consolidation of certain logics within investment arbitration. 131 However, the scholar/practitioner is also not immune from this criticism, as one arbitrator noted: ‘there is rhetoric because it helps to legitimatize the system, to say that there is no problem … it also legitimatize the arbitration community which of course has its own professional interests.’ 132 Investment arbitrators are both enabled and constrained by the social field which makes them professionals. Their theories can support expanding the normative scope of the law, but only within the terms of what their discourse deems valuable.

International investment law as a mutual constitution of theory and practice

The interviewees nevertheless unanimously sought to underline the symbiotic function of theory and practice play in their work. One arbitrator was unequivocal in this respect: ‘this field has always been a mixture of theory and practice. You cannot be at the forefront of investment arbitration without having theoretical scope or background.’ 133 Another elaborated: 134

There is a mutual influence between theory and practice. My scholarship certainly gets enriched by the fact that I see practical cases, but also cases raise issues which scholarship hasn’t thought of, hasn’t been addressed, so there is certainly a cross fertilization in that direction. And in the other direction to the extent that one can, one is part of a collective body of arbitrators, one also tries as a law professor to see a case not only as an individual instance but also as part of a broader system, to see how that case might actually contribute to international investment law and international law more broadly. In addition, one tries to practice what one has preached as a scholar … I take an aspect that I advocate seriously also in practice and also, I try not to commit the errors that I as a scholar have identified in the past.

For some arbitrators, it would seem that the extent of practice and extent of duty was correlative: ‘I felt at this stage writing, about commas and the interpretation of a provision … was not what I was interested in now. I am interested in the functioning of the system … there are many younger scholars … who know it better than I do in terms of theory; I have the privilege of also knowing it from the inside’. 135 One arbitrator further expressed that this duty was connected to their ‘public function’: ‘(w)e are not catering just to the two parties before the Tribunals, or worse the party that appointed you – that’s not compatible with the public function of the arbitrator. We are (independent and impartial) organs of international law … we speak the law not in the name of the parties’. 136 In other words, and as a senior arbitrator stated, ‘(t)he role of the ICSID arbitrator is for the creation and development of international law’. 137

This article has empirically explored the influence of legal scholars on the development of international investment law. It has sought to achieve a synthesis between legal-empirical and sociological methodologies to examine the role of legal scholarship and scholars practicing in arbitration giving content to legal norms through knowledge production and law application. It has found that the development of investment law in ICSID arbitration operates through an interdependent, dynamic, and recursive relationship between theory and practice.

The quantitative analysis of citations to legal scholarship revealed how legal scholarship is thoroughly embedded in international investment law, as it is frequently used across all subject areas, and predominantly for its role in helping actors to understand and apply the law. The analysis thus suggests that legal scholarship provides a sense of system to case law by not only providing to a means to argue within the terms of the law but also acting as a source of background knowledge. Simply put, legal scholarship provides the theory that supports the practice of investment law.

The qualitative analysis then showed how legal scholarship influenced the development of the definition of an ‘investment’ under the ICSID Convention in the nature of a hermeneutic circle, as legal decisions generated commentary which recursively fed back into legal decisions. It further revealed how legal developments are not necessarily one directional as legal scholarship provides a means for contesting normative points and thereby helps to manage change and stability in the legal system as law encounters new issues and conflicts.

The final parts then demonstrated that arbitrators themselves are theoretical subjects; they express visions of the law’s pedigree, system, and teleology, and they carry through these theories in their actions with a sense of fidelity towards the law’s ultimate purpose. However, this dynamic interaction between social actors that makes law in practice and upholds its values also works to disguise agent’s motivations and silence certain narratives. At a time when investment arbitration faces sustained legitimacy crises, this remains important to keep in mind.

The arbitrator/academic is engaged in a praxis , the type of action that arises precisely at the intersection of theory and practice, where persons in public mutually constitute an object that is not detachable from them. 138   Praxis in this Aristotelian sense is ‘doing’ the law through scholarship and arbitrating, where theoria (thinking) and poiesis (making) are so completely enmeshed that their relation is confounding. Yet, and as this article has hoped to show, practice without theory is like a wheel without an axil, without which there can be no future path.

This research was funded by Danish National Research Foundation Grant DNRF169.

Urbaser SA and Consorcio de Aguas Bilbao Bizkaia, Bilbao Biskaia Ur Partzuergoa v Argentine Republi c, Decision on Claimants’ Proposal to Disqualify Professor Campbell McLachlan, ICSID Case No ARB/07/26, 12 August 2010.

Although the role of the scholar is usually excluded from these analyses, see eg Nassib G Ziadé, ‘How Many Hats Can a Player Wear: Arbitrator, Counsel and Expert?’ (2009) 24 ICSID Rev 49; Malcolm Langford, Daniel Behn and Runar Lie, ‘The Revolving Door in International Investment Arbitration’ (2017) 20(2) J Intl Econ L 1.

Of course, much of the development of international investment law occurs at the treaty level, see especially Runar Lie, ‘Treaty Influencers: A Computational Analysis of the Development of International Investment Law’ (2023) 26 J Intl Econ L 500; but this article is concerned with that which occurs at the level of legal practice. See also recently, Runar Lie, ‘The Influencers of International Investment Law: A Computational Study of ISDS Actors “Changing Behavior”’ (2023) 23 Ger Law J 350.

See generally Daniel Behn, Malcolm Langford and L Létourneau-Tremblay, ‘Empirical Perspectives on Investment Arbitration: What Do We Know? Does It Matter?’ (2020) 21 J World Invest Trade 188.

Yves Dezalay and Bryant Garth, Dealing in Virtue: International Commercial Arbitration and the Construction of a Transnational Legal Order (University of Chicago Press, Chicago, IL 1996); Emmanuel Gaillard, ‘Sociology of International Arbitration’ (2015) 31 Arb Intl 1.

See Langford, Behn, Lie (n 2); Sergio Puig, ‘Social Capital in the Arbitration Market’ (2014) 25 Eur J Int Law 387.

Niccolò Ridi and Thomas Schultz, ‘Empirically Mapping Investment Arbitration Scholarship: Networks, Authorities, and the Research Front’ in Katia Fach Gomez (ed), European Yearbook of International Economic Law: Private Actors in International Investment Law (Springer, Switzerland 2020) 210; See also Niccolò Ridi and Thomas Schultz, ‘Arbitration Literature’ in Thomas Schultz and Federico Ortino (eds), The Oxford Handbook of International Arbitration (Oxford University Press, Oxford 2020) 1; Ole Kristian Fauchald, ‘The Legal Reasoning of ICSID Tribunals – An Empirical Analysis’ (2008) 19 Eur J Int Law 301.

On citation as not equalling to causation, see Sandesh Sivakumaran, ‘The Influence of Teachings of Publicists on the Development of International Law’ (2017) 66 Int’l & Comp L Q 1, 20.

On the need to account for sociological processes in an assessment of influence of scholars, see Jörg Kammerhofer, ‘Law-Making by Scholars’ in Catherine Brölmann and Yannick Radi (eds), Research Handbook on the Theory and Practice of International Law-Making (Edward Elgar, Cheltenham 2015) 305.

On the role of the arbitrator here, see Andrea K Bjorklund and L Vanhonnaeker, ‘Applicable Law in International Investment Arbitration’ in CL Lim (ed), The Cambridge Companion to International Arbitration (Cambridge University Press, Cambridge 2021).

See further William Hamilton Byrne and Henrik Palmer Olsen, ‘Doctrinal Legal Science: A Science of its Own?’ (forthcoming, 2024) Can J L Juris.

HLA Hart, The Concept of Law (3rd edn, OUP, Oxford 2012).

H Kelsen, Pure Theory of Law (2nd edn, University of California Press, Berkeley 1967) 225.

See eg Anthea Roberts, ‘Clash of Paradigms: Actors and Analogies Shaping the Investment Treaty System’ (2017) 107 Am J Int’l L 45.

AES Corpv Argentina , ARB/02/17, Decision on Jurisdiction, 26 April 2005 [30].

See eg Stephan Schill, The Multilateralization of International Investment Law (Oxford University Press, Oxford 2010). See also Lie (n 3) on the increasing systemic approach of arbitrators.

, cf Gabrielle Kaufmann-Kohler, ‘Arbitral Precedent: Dream, Necessity or Excuse’ (2007) 23 Arbitr Int 357 on the divergence of more sociological views on this topic.

Statute of the International Court of Justice Art 38(1)(d).

See eg Manfred Lachs, Teachers and Teaching of International Law (The Hague Academy, Recueil des Cours 1976).

See generally Andrea Bianchi, ‘Knowledge Production in International Law: Forces and Processes’ in Andrea Bianchi and Moshe Hirsch (eds), International Law’s Invisible Frames: Social Cognition and Knowledge Production in International Legal Processes (Oxford University Press, Oxford 2021) 155.

Gleider Hernández, ‘The Responsibility of the International Legal Academic Situating the Grammarian Within the “Invisible College”’ in Jean d’Aspremont and others (eds), International Law as a Profession (Cambridge University Press, Cambridge 2017) 160.

Laurence Boisson de Chazournes, ‘Theory and Practice: Two Sides of the Same Coin’ in Jeffrey Dunoff and Mark A Pollack (eds), International Legal Theory: Foundations and Frontiers (Cambridge University Press, Cambridge 2020) 345.

See Dezalay and Garth (n 5).

See Puig (n 5) and also Sara Dezalay, ‘Professionals of International Justice: From the Shadow of State Diplomacy to the Pull of the Market for Commercial Arbitration’ in Jean d’Aspremont and others (eds), International Law as a Profession (CUP, Cambridge 2017) 311; and Stephan Schill, ‘W(h)ither Fragmentation? On the Literature and Sociology of International Investment Law’ (2011) 22 Eur J Int Law 875, arguing that this diversification is now reflected in scholarship produced on investment arbitration.

Fauchald (n 8) 351; cf Ridi and Schultz (n 8) 229–30.

Fauchald (n 8) 351.

The database used was ITA Law < https://www.italaw.com/search/site > accessed 9 May 2024.

I thank Nicolai Nyströmer, data specialist at iCourts, for assistance with this extraction.

Note that it is disputed whether documents produced by institutions such as the ILC are legal scholarship. This analysis treats this work as scholarship because it espouses legal doctrine in a similar manner to work of a single author, notwithstanding its textual form. See similarly, David Caron, ‘The ILC Articles on State Responsibility: The Paradoxical Relationship between Authority and Form’ (2017) 96 Am J Int’l L 857. See also Jorg Kammerhofer, ‘Review of Sondre Torp Helmersen, The Application of Teachings by the International Court of Justice’ (2022) 33 Eur J Int Law 315, 318 calling the exclusion of such literature from a similar analysis ‘hyper-formality’.

cf Fauchald (n 8) 351; see also Andrew Newcombe, ‘The Strange Case of Expert Legal Opinions in Investment Treaty Arbitrations’ Kluwer Arbitration Blog, < http://arbitrationblog.kluwerarbitration.com/2010/03/18/the-strange-case-of-expert-legal-opinions-in-investment-treaty-arbitrations/ > accessed 16 February 2024, noting that ‘many of the expert legal opinions … are used by counsel as legal submission in everything but name’.

Whilst no inter-coder reliability tests were applied, the results on most cited authors also cohere with Ridi and Schultz’s (n 8) findings.

Ridi and Schultz (n 8) 216; Sivakumaran (n 8) 19.

Sivakumaran (n 8) 23.

On the nature of this method, see Mark Hall and Ronald Wright, ‘Systematic Content Analysis of Judicial Opinions’ (2008) 96 Calif Law Rev 63.

For remarks on influence as an edifice of knowledge, see Byrne and Olsen (n 11).

See similarly Fauchauld (n 8) 351.

It is also a higher rate than other international courts and tribunals; see William Hamilton Byrne, Legal Scholarship at Work: An Empirical Analysis of the Use of Theory in the Practice of International Courts and Tribunals (PhD thesis on file at the University of Copenhagen Faculty of Law); cf Sondre Torp Helmersen, The Application of Teachings by the International Court of Justice (Cambridge University Press, Cambridge 2021) finding that the ICJ almost never cites legal scholarship in majority judgments and decisions.

For various iterations on this theme, see Michael Wood, ‘Teachings of the Most Highly Qualified Publicists’, Max Planck Encyclopedia of Public International Law < https://opil.ouplaw.com/display/10.1093/law:epil/9780199231690/law-9780199231690-e1480 > accessed 9 April 2023, see also Ridi and Schultz (n 8).

As anticipated by Caron (n 29).

cf Carsten Stahn and Eric de Brabandere, ‘The Future of International Legal Scholarship: Some Thoughts on ‘Practice’, ‘Growth’, and ‘Dissemination’ (2014) 27 Leiden J Int Law 1, 5.

Alec Stone Sweet and Florian Grisel, The Evolution of International Arbitration: Judicialization, Governance, Legitimacy (Oxford University Press, Oxford 2017) 154 (also including non-ICSID arbitration).

Interview with ICSID Arbitrator 10, 2 May 2020.

Interview with Lawyer 3, 28 April 2020.

Interview with Lawyer 4, 28 April 2020.

Sivakumaran (n 8) 16.

Interview with ICSID Arbitrator 8, 27 April 2020.

Interview with ICSID Arbitrator 10, 1 May 2020.

Interview with ICSID Arbitrator 6, 23 April 2020.

Interview with ICSID Arbitrator 5, 22 April 2020.

Langford, Behn and Lie (n 2) 313.

See also Helmersen (n 37) 93–103, finding that citations to scholarship by the International Court of Justice are driven predominantly by the perceived ‘quality’ and ‘expertise’ of the scholar.

On the strategic use of citations to scholarship, see eg Robert J Hume, ‘Strategic-Instrument Theory and the Use of Non-Authoritative Sources by Federal Judges: Explaining References to Law Review Articles’ (2010) 31 Justice Syst J 291; cf Ingo Venzke, How Interpretation makes International Law: On Semantic Change and Normative Twists (Cambridge University Press, Cambridge 2013) 67 for whom scholars generally do not exercise ‘semantic authority’ unless they are attached to an institutional authority such as the ILC.

Interview with Lawyer 1, 20 April 2020.

I further explore this in William Hamilton Byrne, ‘Is Critique Part of the Practice of International Law?’ London Review of International Law (forthcoming 2024).

Christoph Schreuer and others, The ICSID Convention: a Commentary , (2nd edn, Cambridge University Press, Cambridge 2009) 116.

Emmanuel Gaillard, ‘Introduction’ in Yas Banifatemi (eds), Jurisdiction in Investment Treaty Arbitration (Juris Net LLC IAI Series No 8, New York, NY 2018) 1.

Sivakumaran (n 8) 34–35.

Fedax N.V.v The Republic of Venezuela , Decision of the Tribunal on Objections to Jurisdiction, ICSID Case No ARB/96/3, 11 July 1997 (16).

ibid 22–23.

As found in (and cited as) ‘ ICSID Review – Foreign Investment Law Journal , Vol. 11, 1996, 316’.

Salini Costruttori S.p.A. and Italstrade S.p.A.v Kingdom of Morocco , Decision on Jurisdiction, ICSID Case No ARB/00/4, 31 July 2001 (hereinafter, ‘ Salini ’).

As cited Emmanuel Gaillard, ‘Identify or Define? Reflections on the Concept of Evolution in ICSID Practice’ in Christina Binder and others (eds), International Investment Law for the Twenty-First Century: Essays in Honour of Christoph Schreuer (Oxford University Press, Oxford 2009) 404, 406.

Ceskoslovenska Obchodni Banka, A.S.v The Slovak Republic , Decision of the Tribunal on Objections to Jurisdiction, 24 May 1999, ICSID Case No ARB/97/4 (64).

Salini (27) (52).

Joy Mining Machinery Limitedv Arab Republic of Egypt , Award on Jurisdiction, ICSID Case No ARB/03/11, 6 August 2004 (50).

Saipem S.p.A.v The People’s Republic of Bangladesh , Decision on Jurisdiction and Recommendation on Provisional Measures, ICSID Case No ARB/05/07, 21 March 2007 (111).

Jan de Nul N.V. and Dredging International N.V.v Arab Republic of Egypt , 16 June 2006, ICSID Case No ARB/04/13 (91) (99–102).

Bayindir Insaat Turizm Ticaret Ve Sanayi A.S.v Islamic Republic of Pakistan , Decision on Jurisdiction, ICSID Case No ARB/03/29, 14 November 2005.

L.E.S.I. S.p.A. and ASTALDI S.p.A.v République Algérienne Démocratique et Populaire , ICSID Case No ARB/05/3, 12 July 2006 (13).

Biwater Gauffv United Republic of Tanzania , Award, ICSID Case No ARB/05/22, 34 July 2008 (312).

Malaysian Historical Salvors, SDN, BHDv The Government of Malaysia , Award on Jurisdiction ICSID Case No ARB/05/10, 17 May 2007 (44).

Malaysian Historical Salvors, SDN, BHDv The Government of Malaysia , Decision on Annulment, ICSID Case No ARB/05/10, 16 April 2009 (78).

Malaysian Historical Salvors , SDN, BHD v The Government of Malaysia, Annulment, Dissenting Opinion of Judge Mohamed Shahabuddeen, ICSID Case No ARB/05/10, 16 April 2009 (8).

Phoenix Action, Ltdv The Czech Republic , Award, ICSID Case No ARB/06/5, 15 April 2009 (77, 107).

Saba Fakesv Republic of Turkey , Award, ICSID Case No ARB/07/20, 14 July 2010 (101–102).

Ambiente Ufficio S.p.A. and othersv Argentine Republic , Decision on Jurisdiction and Admissibility, 8 February 2013, ICSID Case No ARB/08/(464).

ibid 468, 470–472.

Philip Morris Brands Sàrl, Philip Morris Products S.A. and Abal Hermanos S.A.v Oriental Republic of Uruguay , Decision on Jurisdiction, ICSID Case No ARB/10/7, 2 July 2014 (196, 206).

For a recent analysis, see eg Michael Waibel, ‘Subject Matter Jurisdiction: The Notion of Investment’ (2021) 19 ICSID Rep 1.

On this fallacy, see especially Kammerhofer (n 9).

This example is part of a broader analysis that examined the influence of legal scholarship across conceptual categories of international investment law, see Byrne (n 37).

Velimir Zivkovic, ‘Recognition of Contracts as Investments in International Investment Arbitration’ (2012) 5 Eur J Leg Stud 174, 150.

See also Hernández (n 19) on the structural characteristics of international law, and on the uniqueness of investment law in this context as defined by broad-based norms, Stahn and de Brabandere (n 41) 5.

See also eg Hume (n 53).

For a similar analysis on the activation of ICSID dispute settlement in the 1990s, see Gus van Harten, The Trouble with Foreign Investment Protection (Oxford University Press, Oxford 2020).

Langford, Behn and Lee (n 2).

Schreuer and others (n 58) 133–34.

Rudolf Dolzer and Christoph Schreuer, Principles of International Investment Law (Oxford University Press, Oxford 2012) 76.

Ibrahim Fadlallah, ‘La Notion d’investissement: vers une Restriction à la Compétence du CIRDI’ in Gerald Aksen and Robert Briner (eds), Liber Amicorum Robert Briner (ICC Publishers, Paris 2005) 260 as cited in Gaillard (n 68) 409.

Stephen M Schwebel, Justice in International Law: Selected Further Writings (Cambridge University Press, Cambridge 2011) 286.

Bernard Hanotiau, ‘Are Bilateral Investment Treaties and Free Trade Agreements Drafted with Sufficient Clarity to Give Guidance to Tribunals? (2018) 5 Am U Bus 313, 320.

Brigitte Stern, ‘Contours of the Notion of Protected Investment’ (2009) 24 ICSID Rev 534.

As cited in Gaillard (n 68) 405.

ibid 403, 416.

The sociological foundation of this dynamic is further detailed in Byrne (n 58).

Michael Hwang and Jennifer Fong, ‘Definition of “Investment”—A Voice from the Eye of the Storm’ (2011) 1 Asian J Int Law 99, 100, 129.

The notable exception here was Kaufmann-Kohler who does not appear to have published on the subject, but nevertheless has written on other foundational issues of the law, see Kaufmann-Kohler (n 28).

Thomas Schultz, ‘Arbitration as an iPhone, or Why Conduct Academic Research in Arbitration?’ (2011) 2 J Intl Dispute Settl 279, 282.

Interview with ICSID Arbitrator 3, 21 April 2020.

Interview with ICSID Arbitrator 9, 28 April 2020.

Interview with ICSID Arbitrator 1, 20 April 2020.

cf Roberts (n 25).

Interview with ICSID Arbitrator 11, 2 May 2020.

On this tradition, see Joseph Dunne, Back to the Rough Ground: Practical Judgment and the Lure of Technique (University of Notre Dame Press, Indiana 1993).

Author notes

Email alerts, citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1464-3758
• Print ISSN 1369-3034
• Copyright © 2024 Oxford University Press
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

The University of Chicago The Law School

Jacob goldin awarded 2024 donald m. ephraim prize in law and economics.

The second annual Donald M. Ephraim Prize in Law and Economics has been awarded to Jacob Goldin, Richard M. Lipton Professor of Tax Law at the University of Chicago Law School.

The prize, established last year by the University of Chicago through the generous support of Donald M. Ephraim, ’54, recognizes an early-career scholar in the area of law and economics whose work has advanced the state of knowledge in the field and whose intellectual impact has the potential to reach the legal academy, legal profession, and beyond. The inaugural winner was Megan T. Stevenson, Associate Professor of Law and Associate Professor of Economics at the University of Virginia.

Trained as a lawyer and economist, Professor Goldin primarily focuses his scholarship on US tax policy affecting low-income households. His research interests also include health policy, tax administration, and the application of behavioral economics to policy design. Ephraim was pleased to see the Prize go to Goldin: “I am delighted that our selection committee chose Professor Goldin, a prolific scholar whose work is already impacting multiple domains of law and policy.”

William H.J. Hubbard, Deputy Dean and Harry N. Wyatt Professor of Law, chaired the selection committee and felt the decision was compelled by Goldin’s body of work. “Jacob has made major contributions to law and economics. His work is incredibly wide-ranging, studying topics from the child tax credit, to bail in criminal prosecutions, to health insurance coverage. He has made both theoretical and empirical contributions, and he has published in economics journals, law and econ journals, and law reviews. Most of all, Jacob’s work has had influence within and beyond the academy. His recent work on racial disparities in tax audits, for example, points the way toward improving IRS auditing processes to reduce racial disparities while maintaining the effectiveness of audits in identifying underreporting.”

In response to his selection, Goldin said, “I’m honored and grateful to receive this prize. There is so much exciting work being done in law and economics these days, it’s a great community to be a part of.”

The Prize includes a cash award (currently $53,000), as well as the opportunity to present research at the Ephraim Prize Lecture at the Law School during the following academic year.

The impact of environmental regulation and economic expectations on crop-livestock integration among hog farmers: a field study from China

• Research Article
• Published: 01 June 2024

Cite this article

• Jing Cao 1 ,
• Jiapeng Xu 1 ,
• Huimin Cao 1 ,
• Fangfang Wang 1 ,
• Zhenyu Yan   ORCID: orcid.org/0000-0002-3154-9763 1 &
• Taimoor Muhammad 1  

Decoupling of crop-livestock systems increases the risks of pollution, waste of nutrient resources, and biodiversity loss. Crop-livestock integration (CLI) is an effective solution to these problems, and motivating farmers to adopt CLI is the key. Many countries have implemented environmental regulations (ER) aiming to influence farmers’ CLI adoption decisions. Based on a field study of 316 hog farmers from Shaanxi Province of China, this paper applies the triple-hurdle model to empirically examine the impacts of economic expectations (EE) and ER on CLI adoption decisions. It also verifies the income effect of CLI. The results are as follows: 90.5% of farmers are willing to adopt CLI, but the adoption rate is only 40.8% and the average integration degree is only 0.236; CLI not been widely popularized. EE and ER promote farmers’ CLI adoption significantly, while the impact of interaction between EE and ER on CLI adoption differs. IER weakens the positive impact of EE on farmers’ CLI integration degree, which has a “crowding out effect.” GER negatively moderates the impact of EE on farmers’ adoption willingness of CLI. CER strengthens the positive effect of EE on farmers’ adoption behavior and CLI integration degree. CLI increases the farmers’ income. These results contribute to our understanding of the mechanisms of CLI adoption decisions and sustainable policy optimization for green agricultural development.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Data availability

All data needed to evaluate the conclusions in the paper are present in the paper.

When testing for sample selection errors, IMR was not significant. Therefore, only the test results of IMR are explained here, and the overall regression results are not reported (Models 1–8).

Asai M, Moraine M, Ryschawy J et al (2018) Critical factors for crop-livestock integration beyond the farm level: a cross-analysis of worldwide case studies. Land Use Policy 73:184–194. https://doi.org/10.1016/j.landusepol.2017.12.010

Article   Google Scholar  

Bai Z, Ma L, Jin S et al (2016) Nitrogen, phosphorus, and potassium flows through the manure management chain in China. Environ Sci Technol 50:13409–13418. https://doi.org/10.1021/acs.est.6b03348

Article   CAS   Google Scholar  

Bai Z, Ma W, Ma L et al (2018) China’s livestock transition: driving forces, impacts, and consequences. Sci Adv 4:eaar8534. https://doi.org/10.1126/sciadv.aar8534

Bai Z, Fan X, Jin X et al (2022) Relocate 10 billion livestock to reduce harmful nitrogen pollution exposure for 90% of China’s population. Nat Food 3:152–160. https://doi.org/10.1038/s43016-021-00453-z

Bao W, Wu Y, Bao H (2024) Transaction costs, crop-livestock integration participation, and income effects in China. Front Sustain Food Syst 7:1247770. https://doi.org/10.3389/fsufs.2023.1247770

Barnes AP, Willock J, Hall C, Toma L (2009) Farmer perspectives and practices regarding water pollution control programmes in Scotland. Agric Water Manag 96:1715–1722. https://doi.org/10.1016/j.agwat.2009.07.002

Belay DG, Jensen JD (2020) ‘The scarlet letters’: information disclosure and self-regulation: evidence from antibiotic use in Denmark. J Environ Econ Manag 104:102385. https://doi.org/10.1016/j.jeem.2020.102385

Bell LW, Moore AD (2012) Integrated crop–livestock systems in Australian agriculture: trends, drivers and implications. Agric Syst 111:1–12. https://doi.org/10.1016/j.agsy.2012.04.003

Branca G, Perelli C (2020) ‘Clearing the air’: common drivers of climate-smart smallholder food production in Eastern and Southern Africa. J Clean Prod 270:121900. https://doi.org/10.1016/j.jclepro.2020.121900

Burke WJ, Myers RJ, Jayne TS (2015) A triple-hurdle model of production and market participation in Kenya’s dairy market. Am J Agric Econ 97:1227–1246. https://doi.org/10.1093/ajae/aav009

Cao D, Li H, Wang G, Huang T (2017) Identifying and contextualising the motivations for BIM implementation in construction projects: an empirical study in China. Int J Proj Manag 35:658–669. https://doi.org/10.1016/j.ijproman.2016.02.002

Carrer MJ, Maia AG, de Mello BrandãoVinholis M, de Souza Filho HM (2020) Assessing the effectiveness of rural credit policy on the adoption of integrated crop-livestock systems in Brazil. Land Use Policy 92:104468. https://doi.org/10.1016/j.landusepol.2020.104468

Chen Q, Xu Q, Yu X (2023) Triple-hurdle model analysis of aquaculture farmers’ multi-stage willingness to participate in green and healthy aquaculture actions in China: based on ecological cognition and environmental regulation perspectives. Front Sustain Food Syst 7:1211392. https://doi.org/10.3389/fsufs.2023.1211392

Cheng P, Li J, Zhang H, Cheng G (2023) Sustainable management behavior of farmland shelterbelt of farmers in ecologically fragile areas: empirical evidence from Xinjiang, China. Sustainability 15:2011. https://doi.org/10.3390/su15032011

Cortner O, Garrett RD, Valentim JF et al (2019) Perceptions of integrated crop-livestock systems for sustainable intensification in the Brazilian Amazon. Land Use Policy 82:841–853. https://doi.org/10.1016/j.landusepol.2019.01.006

Cui G, Liu Z (2022) The impact of environmental regulations and social norms on farmers’ chemical fertilizer reduction behaviors: an investigation of citrus farmers in southern China. Sustainability 14:8157. https://doi.org/10.3390/su14138157

de Groot JIM, Steg L (2009) Mean or green: which values can promote stable pro-environmental behavior? Conserv Lett 2:61–66. https://doi.org/10.1111/j.1755-263X.2009.00048.x

Dessart FJ, Barreiro-Hurlé J, Van Bavel R (2019) Behavioural factors affecting the adoption of sustainable farming practices: a policy-oriented review. Eur Rev Agric Econ 46:417–471. https://doi.org/10.1093/erae/jbz019

Dos Reis JC, Kamoi MYT, Latorraca D et al (2020) Assessing the economic viability of integrated crop−livestock systems in Mato Grosso, Brazil. Renew Agric Food Syst 35:631–642. https://doi.org/10.1017/S1742170519000280

dos Reis JC, Rodrigues GS, de Barros I et al (2021) Integrated crop-livestock systems: a sustainable land-use alternative for food production in the Brazilian Cerrado and Amazon. J Clean Prod 283:124580. https://doi.org/10.1016/j.jclepro.2020.124580

Du S, Liu J, Fu Z (2021a) The impact of village rules and formal environmental regulations on farmers’ cleaner production behavior: new evidence from China. Int J Environ Res Public Health 18:7311. https://doi.org/10.3390/ijerph18147311

Du S, Luo X, Huang Y et al (2021b) Risk perception, specialized agricultural services and rice farmers’ adoption behavior of biological pesticide technology. Resour Environ Yangtze Basin 30:1768–1779

Google Scholar  

Finger R, Lehmann N (2012) Policy reforms to promote efficient and sustainable water use in Swiss agriculture. Water Policy 14:887–901. https://doi.org/10.2166/wp.2012.152

Garrett R, Niles M, Gil J et al (2017) Policies for reintegrating crop and livestock systems: a comparative analysis. Sustainability 9:473. https://doi.org/10.3390/su9030473

Garrett R, Ryschawy J, Bell L et al (2020) Drivers of decoupling and recoupling of crop and livestock systems at farm and territorial scales. Ecol Soc 25. https://doi.org/10.5751/ES-11412-250124

Gil JDB, Garrett R, Berger T (2016) Determinants of crop-livestock integration in Brazil: evidence from the household and regional levels. Land Use Policy 59:557–568. https://doi.org/10.1016/j.landusepol.2016.09.022

Gil JDB, Garrett RD, Rotz A et al (2018) Tradeoffs in the quest for climate smart agricultural intensification in Mato Grosso. Brazil Environ Res Lett 13:064025. https://doi.org/10.1088/1748-9326/aac4d1

Goodhue RE, Mohapatra S, Rausser GC (2010) Interactions between incentive instruments: contracts and quality in processing tomatoes. Am J Agric Econ 92:1283–1293. https://doi.org/10.1093/ajae/aaq061

Gu B (2022) Recoupling livestock and crops. Nat Food 3:102–103. https://doi.org/10.1038/s43016-022-00466-2

Guo Q, Li H, Li S, Nan L (2021) Research on farmers’ pro-environmental behavior from the perspective of paradox existing between behavior and willingness: taking the organic fertilizers application as an example. Resour Environ Yangtze Basin 30:212–224

Guo Z, Chen X, Zhang Y (2022) Impact of environmental regulation perception on farmers’ agricultural green production technology adoption: a new perspective of social capital. Technol Soc 71:102085. https://doi.org/10.1016/j.techsoc.2022.102085

Guo Q (2021) The fading relationship between agriculture and animal husbandry and its reconstruction. Chin Rural Econ 22–35. https://kns.cnki.net/kcms/detail/11.1262.F.20211011.0858.004.html

Hamazakaza P, Kabwe G, Kuntashula E et al (2022) Adoption of sustainable agriculture intensification in maize-based farming systems of Katete District in Zambia. Land 11:880. https://doi.org/10.3390/land11060880

Han Z, Han C, Shi Z et al (2023) Rebuilding the crop-livestock integration system in China ——based on the perspective of circular economy. J Clean Prod 393:136347. https://doi.org/10.1016/j.jclepro.2023.136347

Harper JK, Roth GW, Garalejić B, Škrbić N (2018) Programs to promote adoption of conservation tillage: a Serbian case study. Land Use Policy 78:295–302. https://doi.org/10.1016/j.landusepol.2018.06.028

Ji C, Jin S, Wang H, Ye C (2019) Estimating effects of cooperative membership on farmers’ safe production behaviors: evidence from pig sector in China. Food Policy 83:231–245. https://doi.org/10.1016/j.foodpol.2019.01.007

Jin S, Zhang B, Wu B et al (2020) Decoupling livestock and crop production at the household level in China. Nat Sustain 4:48–55. https://doi.org/10.1038/s41893-020-00596-0

Kaczan DJ, Swallow BM, Adamowicz WL (Vic) (2019) Forest conservation policy and motivational crowding: experimental evidence from Tanzania. Ecol Econ 156:444–453. https://doi.org/10.1016/j.ecolecon.2016.07.002

Kelifa A (2023) Review of Tobit, Heckman and double hurdle econometric models: supported with evidences from the studies conducted in Ethiopia. SN Bus Econ 3:104. https://doi.org/10.1007/s43546-023-00478-5

Khed VD, Krishna VV (2023) Agency and time poverty: linking decision-making powers and leisure time of male and female farmers of Central India. World Dev Perspect 29:100484. https://doi.org/10.1016/j.wdp.2022.100484

Knight J, Deng Q, Li S (2011) The puzzle of migrant labour shortage and rural labour surplus in China, China. Econ Rev 22:585–600. https://doi.org/10.1016/j.chieco.2011.01.006

Li Y, Wang B (2022) Environmental motivation or economic motivation? Explaining individuals’ intention to carry reusable bags for shopping in China. Front Psychol 13:972748. https://doi.org/10.3389/fpsyg.2022.972748

Li F, Ren J, Wimmer S et al (2020) Incentive mechanism for promoting farmers to plant green manure in China. J Clean Prod 267:122197. https://doi.org/10.1016/j.jclepro.2020.122197

Li Y, Sun Z, Accatino F et al (2021) Comparing specialised crop and integrated crop-livestock systems in China with a multi-criteria approach using the emergy method. J Clean Prod 314:127974. https://doi.org/10.1016/j.jclepro.2021.127974

Liu Z, Sun J, Zhu W, Qu Y (2021) Exploring impacts of perceived value and government regulation on farmers’ willingness to adopt wheat straw incorporation in China. Land 10:1051. https://doi.org/10.3390/land10101051

Lu Y, Tan Y, Wang H (2022) Impact of environmental regulation on green technology adoption by farmers microscopic investigation evidence from pig breeding in China. Front Environ Sci 10:885933. https://doi.org/10.3389/fenvs.2022.885933

Ma W, Abdulai A (2016) Does cooperative membership improve household welfare? Evidence from apple farmers in China. Food Policy 58:94–102. https://doi.org/10.1016/j.foodpol.2015.12.002

Ma L, Bai Z, Ma W et al (2019) Exploring future food provision scenarios for China. Environ Sci Technol 53:1385–1393. https://doi.org/10.1021/acs.est.8b04375

Martin G, Moraine M, Ryschawy J et al (2016) Crop–livestock integration beyond the farm level: a review. Agron Sustain Dev 36:53. https://doi.org/10.1007/s13593-016-0390-x

Mather DL, Jayne TS (2018) Fertilizer subsidies and the role of targeting in crowding out: evidence from Kenya. Food Secur 10:397–417. https://doi.org/10.1007/s12571-018-0773-8

Meraner M, Finger R (2019) Risk perceptions, preferences and management strategies: evidence from a case study using German livestock farmers. J Risk Res 22:110–135. https://doi.org/10.1080/13669877.2017.1351476

Midler E, Pascual U, Drucker AG et al (2015) Unraveling the effects of payments for ecosystem services on motivations for collective action. Ecol Econ 120:394–405. https://doi.org/10.1016/j.ecolecon.2015.04.006

Missiame A, Nyikal RA, Irungu P (2021) What is the impact of rural bank credit access on the technical efficiency of smallholder cassava farmers in Ghana? An endogenous switching regression analysis. Heliyon 7:e07102. https://doi.org/10.1016/j.heliyon.2021.e07102

Moraine M, Duru M, Nicholas P et al (2014) Farming system design for innovative crop-livestock integration in Europe. Anim Int J Anim Biosci 8:1204–1217. https://doi.org/10.1017/S1751731114001189

Niles MT, Garrett RD, Walsh D (2017) Ecological and economic benefits of integrating sheep into viticulture production. Agron Sustain Dev 38:1. https://doi.org/10.1007/s13593-017-0478-y

Pelletier LG, Tuson KM, Haddad NK (1997) Client motivation for therapy scale: a measure of intrinsic motivation, extrinsic motivation, and amotivation for therapy. J Pers Assess 68:414–435. https://doi.org/10.1207/s15327752jpa6802_11

Peterson CA, Deiss L, Gaudin ACM (2020) Commercial integrated crop-livestock systems achieve comparable crop yields to specialized production systems: a meta-analysis. PLoS One 15:e0231840. https://doi.org/10.1371/journal.pone.0231840

Poeschl M, Ward S, Owende P (2012) Environmental impacts of biogas deployment – part II: life cycle assessment of multiple production and utilization pathways. J Clean Prod 24:184–201. https://doi.org/10.1016/j.jclepro.2011.10.030

Prokopy LS, Floress K, Arbuckle JG et al (2019) Adoption of agricultural conservation practices in the United States: evidence from 35 years of quantitative literature. J Soil Water Conserv 74:520–534. https://doi.org/10.2489/jswc.74.5.520

Regan JT, Marton S, Barrantes O et al (2017) Does the recoupling of dairy and crop production via cooperation between farms generate environmental benefits? A case-study approach in Europe. Eur J Agron 82:342–356. https://doi.org/10.1016/j.eja.2016.08.005

Ryschawy J, Grillot M, Charmeau A et al (2022) A participatory approach based on the serious game Dynamix to co-design scenarios of crop-livestock integration among farms. Agric Syst 201:103414. https://doi.org/10.1016/j.agsy.2022.103414

Schut AGT, Cooledge EC, Moraine M et al (2021) Reintegration of crop-livestock systems in Europe: an overview. Front Agric Sci Eng 8:111. https://doi.org/10.15302/J-FASE-2020373

Shi H, Sui D, Wu H, Zhao M (2018a) The influence of social capital on farmers’ participation in watershed ecological management behavior: evidence from Heihe Basin. Chin Rural Econ 34–45. https://kns.cnki.net/kcms/detail/11.1262.F.20180201.1801.008.html

Shi Y, Yao L, Zhao M (2018b) The effect of social capital on herds men’s participation willingness in grassland community governance: an analysis based on triple-hurdle model. China Rural Surv 35–50. https://kns.cnki.net/kcms/detail/11.3586.F.20180507.1530.006.html

Shi ZH, Zhang H (2021) Research on social norms, environmental regulations and farmers’ choice of fertilization behavior. Chin J Agric Resour Reg Plan 42:51–61. https://doi.org/10.7621/cjarrp.1005-9121.20211107

Skaalsveen K, Ingram J, Urquhart J (2020) The role of farmers’ social networks in the implementation of no-till farming practices. Agric Syst 181:102824. https://doi.org/10.1016/j.agsy.2020.102824

Sneessens I, Veysset P, Benoit M et al (2016) Direct and indirect impacts of crop–livestock organization on mixed crop–livestock systems sustainability: a model-based study. Animal 10:1911–1922. https://doi.org/10.1017/S1751731116000720

Tan K, Cai G, Du Z et al (2023) Emergy synthesis of decoupling and recoupling crop-livestock systems under unified system boundary and modified indices. Sci Total Environ 877:162880. https://doi.org/10.1016/j.scitotenv.2023.162880

Trujillo-Barrera A, Pennings JME, Hofenk D (2016) Understanding producers’ motives for adopting sustainable practices: the role of expected rewards, risk perception and risk tolerance. Eur Rev Agric Econ 43:359–382. https://doi.org/10.1093/erae/jbv038

Wang X, Tan S (2020) Cost-efficiency analysis of rice-crayfish integrated land operation mode based on non-homogeneous DEA. China Land Sci 34:56–63. https://doi.org/10.11994/zgtdkx.20191225.105326

Wang TX, Teng CG, Zhang ZH (2020) Informal social support, environmental regulation and farmers’ film recycling behavior. J Arid Land Resour Environ 34:109–115. https://doi.org/10.13448/j.cnki.jalre.2020.218

Wang L, Song B, Wu J, Wang L (2022) An empirical study on the influence of environmental regulation and farmers’ cognition on the application behavior of organic fertilizer: based on the survey data of 741 farmers in YE county, PINGDINGSHAN city, Henan province. J Henan Univ Technol Soc Sci 38:57–67. https://doi.org/10.16433/j.cnki.cn41-1379.2022.03.014

Willett W, Rockström J, Loken B et al (2019) Food in the Anthropocene: the EAT–Lancet Commission on healthy diets from sustainable food systems. Lancet 393:447–492. https://doi.org/10.1016/S0140-6736(18)31788-4

Yang C, Liang X, Xue Y et al (2024) Can government regulation weak the gap between green production intention and behavior? Based on the perspective of farmers’ perceptions. J Clean Prod 434:139743. https://doi.org/10.1016/j.jclepro.2023.139743

Yang X, Qi Z (2022) The impact of expected return and technology subsidy on farmers’ adoption of agroecological technology: taking rice–crayfish co–culture technology as an example. J Huazhong Agric Univ Sci Ed 89–100. https://doi.org/10.13300/j.cnki.hnwkxb.2022.05.010

Zhang J, Chen M, Huang C, Lai Z (2022) Labor endowment, cultivated land fragmentation, and ecological farming adoption strategies among farmers in Jiangxi Province, China. Land 11:679. https://doi.org/10.3390/land11050679

Zhao J, Liu L, Qi J, Dong J (2022) Study on the influence of environmental regulation on the environmentally friendly behavior of farmers in China. Front Environ Sci 10:1009151. https://doi.org/10.3389/fenvs.2022.1009151

Zhao Y, Liu L, Zhao J (2023) Perceived benefits, environmental regulation and farmers ’ waste recycling behavior-taking duck farmers as an example. World Agric 98–110. https://doi.org/10.13856/j.cn11-1097/s.2023.04.009

Zhu X, Cai J (2016) The influences of perceived values and capability approach on farmers willingness to exit rural residential land and its intergenerational difference. China Land Sci 30:64–72. https://doi.org/10.11994/zgtdkx.20161024.124548

Download references

This work was supported by Key Think Tank Research Project of Shaanxi Province on “Social Sciences Helping County Economies Develop in High Quality” (2023ZD0662); Humanities and Social Sciences Project of Fundamental Research Funds for the Central Universities in 2023 (452023307); Soft Science Research Program of Shaanxi Province (2022KRM032; 2023-CX-RKX-103); and Social Science Foundation of Shaanxi Province (2021D058; 2022D022). The authors would like to thank the anonymous referees for their helpful suggestions and corrections on the earlier draft of our paper.

Author information

Authors and affiliations.

College of Economics and Management, Northwest A&F University, No. 3 Taicheng Road, Yangling, 712100, Shaanxi, People’s Republic of China

Jing Cao, Jiapeng Xu, Huimin Cao, Fangfang Wang, Zhenyu Yan & Taimoor Muhammad

You can also search for this author in PubMed   Google Scholar

Contributions

Jing Cao: methodology; writing—original draft; writing—review and editing.

Jiapeng Xu: conceptualization, supervision, project administration, article modification.

Huimin Cao: data, research methods.

Fangfang Wang: data, research methods.

Zhenyu Yan: conceptualization, supervision, project administration, article modification.

Taimoor Muhammad: data, research methods.

Corresponding author

Correspondence to Zhenyu Yan .

Ethics declarations

Ethics approval.

Not applicable.

Consent to participate

Consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Responsible Editor: Baojing Gu

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cao, J., Xu, J., Cao, H. et al. The impact of environmental regulation and economic expectations on crop-livestock integration among hog farmers: a field study from China. Environ Sci Pollut Res (2024). https://doi.org/10.1007/s11356-024-33616-z

Download citation

Received : 03 January 2024

Accepted : 05 May 2024

Published : 01 June 2024

DOI : https://doi.org/10.1007/s11356-024-33616-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Crop-livestock integration
• Economic expectations
• Environmental regulation
• Triple-hurdle model
• Pig farmers
• Find a journal
• Publish with us
• Track your research