types of findings in research

• Privacy Policy

Buy Me a Coffee

Home » Research Findings – Types Examples and Writing Guide

Research Findings – Types Examples and Writing Guide

Table of Contents

Research Findings

Definition:

Research findings refer to the results obtained from a study or investigation conducted through a systematic and scientific approach. These findings are the outcomes of the data analysis, interpretation, and evaluation carried out during the research process.

Types of Research Findings

There are two main types of research findings:

Qualitative Findings

Qualitative research is an exploratory research method used to understand the complexities of human behavior and experiences. Qualitative findings are non-numerical and descriptive data that describe the meaning and interpretation of the data collected. Examples of qualitative findings include quotes from participants, themes that emerge from the data, and descriptions of experiences and phenomena.

Quantitative Findings

Quantitative research is a research method that uses numerical data and statistical analysis to measure and quantify a phenomenon or behavior. Quantitative findings include numerical data such as mean, median, and mode, as well as statistical analyses such as t-tests, ANOVA, and regression analysis. These findings are often presented in tables, graphs, or charts.

Both qualitative and quantitative findings are important in research and can provide different insights into a research question or problem. Combining both types of findings can provide a more comprehensive understanding of a phenomenon and improve the validity and reliability of research results.

Parts of Research Findings

Research findings typically consist of several parts, including:

• Introduction: This section provides an overview of the research topic and the purpose of the study.
• Literature Review: This section summarizes previous research studies and findings that are relevant to the current study.
• Methodology : This section describes the research design, methods, and procedures used in the study, including details on the sample, data collection, and data analysis.
• Results : This section presents the findings of the study, including statistical analyses and data visualizations.
• Discussion : This section interprets the results and explains what they mean in relation to the research question(s) and hypotheses. It may also compare and contrast the current findings with previous research studies and explore any implications or limitations of the study.
• Conclusion : This section provides a summary of the key findings and the main conclusions of the study.
• Recommendations: This section suggests areas for further research and potential applications or implications of the study’s findings.

How to Write Research Findings

Writing research findings requires careful planning and attention to detail. Here are some general steps to follow when writing research findings:

• Organize your findings: Before you begin writing, it’s essential to organize your findings logically. Consider creating an outline or a flowchart that outlines the main points you want to make and how they relate to one another.
• Use clear and concise language : When presenting your findings, be sure to use clear and concise language that is easy to understand. Avoid using jargon or technical terms unless they are necessary to convey your meaning.
• Use visual aids : Visual aids such as tables, charts, and graphs can be helpful in presenting your findings. Be sure to label and title your visual aids clearly, and make sure they are easy to read.
• Use headings and subheadings: Using headings and subheadings can help organize your findings and make them easier to read. Make sure your headings and subheadings are clear and descriptive.
• Interpret your findings : When presenting your findings, it’s important to provide some interpretation of what the results mean. This can include discussing how your findings relate to the existing literature, identifying any limitations of your study, and suggesting areas for future research.
• Be precise and accurate : When presenting your findings, be sure to use precise and accurate language. Avoid making generalizations or overstatements and be careful not to misrepresent your data.
• Edit and revise: Once you have written your research findings, be sure to edit and revise them carefully. Check for grammar and spelling errors, make sure your formatting is consistent, and ensure that your writing is clear and concise.

Research Findings Example

Following is a Research Findings Example sample for students:

Title: The Effects of Exercise on Mental Health

Sample : 500 participants, both men and women, between the ages of 18-45.

Methodology : Participants were divided into two groups. The first group engaged in 30 minutes of moderate intensity exercise five times a week for eight weeks. The second group did not exercise during the study period. Participants in both groups completed a questionnaire that assessed their mental health before and after the study period.

Findings : The group that engaged in regular exercise reported a significant improvement in mental health compared to the control group. Specifically, they reported lower levels of anxiety and depression, improved mood, and increased self-esteem.

Conclusion : Regular exercise can have a positive impact on mental health and may be an effective intervention for individuals experiencing symptoms of anxiety or depression.

Applications of Research Findings

Research findings can be applied in various fields to improve processes, products, services, and outcomes. Here are some examples:

• Healthcare : Research findings in medicine and healthcare can be applied to improve patient outcomes, reduce morbidity and mortality rates, and develop new treatments for various diseases.
• Education : Research findings in education can be used to develop effective teaching methods, improve learning outcomes, and design new educational programs.
• Technology : Research findings in technology can be applied to develop new products, improve existing products, and enhance user experiences.
• Business : Research findings in business can be applied to develop new strategies, improve operations, and increase profitability.
• Public Policy: Research findings can be used to inform public policy decisions on issues such as environmental protection, social welfare, and economic development.
• Social Sciences: Research findings in social sciences can be used to improve understanding of human behavior and social phenomena, inform public policy decisions, and develop interventions to address social issues.
• Agriculture: Research findings in agriculture can be applied to improve crop yields, develop new farming techniques, and enhance food security.
• Sports : Research findings in sports can be applied to improve athlete performance, reduce injuries, and develop new training programs.

When to use Research Findings

Research findings can be used in a variety of situations, depending on the context and the purpose. Here are some examples of when research findings may be useful:

• Decision-making : Research findings can be used to inform decisions in various fields, such as business, education, healthcare, and public policy. For example, a business may use market research findings to make decisions about new product development or marketing strategies.
• Problem-solving : Research findings can be used to solve problems or challenges in various fields, such as healthcare, engineering, and social sciences. For example, medical researchers may use findings from clinical trials to develop new treatments for diseases.
• Policy development : Research findings can be used to inform the development of policies in various fields, such as environmental protection, social welfare, and economic development. For example, policymakers may use research findings to develop policies aimed at reducing greenhouse gas emissions.
• Program evaluation: Research findings can be used to evaluate the effectiveness of programs or interventions in various fields, such as education, healthcare, and social services. For example, educational researchers may use findings from evaluations of educational programs to improve teaching and learning outcomes.
• Innovation: Research findings can be used to inspire or guide innovation in various fields, such as technology and engineering. For example, engineers may use research findings on materials science to develop new and innovative products.

Purpose of Research Findings

The purpose of research findings is to contribute to the knowledge and understanding of a particular topic or issue. Research findings are the result of a systematic and rigorous investigation of a research question or hypothesis, using appropriate research methods and techniques.

The main purposes of research findings are:

• To generate new knowledge : Research findings contribute to the body of knowledge on a particular topic, by adding new information, insights, and understanding to the existing knowledge base.
• To test hypotheses or theories : Research findings can be used to test hypotheses or theories that have been proposed in a particular field or discipline. This helps to determine the validity and reliability of the hypotheses or theories, and to refine or develop new ones.
• To inform practice: Research findings can be used to inform practice in various fields, such as healthcare, education, and business. By identifying best practices and evidence-based interventions, research findings can help practitioners to make informed decisions and improve outcomes.
• To identify gaps in knowledge: Research findings can help to identify gaps in knowledge and understanding of a particular topic, which can then be addressed by further research.
• To contribute to policy development: Research findings can be used to inform policy development in various fields, such as environmental protection, social welfare, and economic development. By providing evidence-based recommendations, research findings can help policymakers to develop effective policies that address societal challenges.

Characteristics of Research Findings

Research findings have several key characteristics that distinguish them from other types of information or knowledge. Here are some of the main characteristics of research findings:

• Objective : Research findings are based on a systematic and rigorous investigation of a research question or hypothesis, using appropriate research methods and techniques. As such, they are generally considered to be more objective and reliable than other types of information.
• Empirical : Research findings are based on empirical evidence, which means that they are derived from observations or measurements of the real world. This gives them a high degree of credibility and validity.
• Generalizable : Research findings are often intended to be generalizable to a larger population or context beyond the specific study. This means that the findings can be applied to other situations or populations with similar characteristics.
• Transparent : Research findings are typically reported in a transparent manner, with a clear description of the research methods and data analysis techniques used. This allows others to assess the credibility and reliability of the findings.
• Peer-reviewed: Research findings are often subject to a rigorous peer-review process, in which experts in the field review the research methods, data analysis, and conclusions of the study. This helps to ensure the validity and reliability of the findings.
• Reproducible : Research findings are often designed to be reproducible, meaning that other researchers can replicate the study using the same methods and obtain similar results. This helps to ensure the validity and reliability of the findings.

Advantages of Research Findings

Research findings have many advantages, which make them valuable sources of knowledge and information. Here are some of the main advantages of research findings:

• Evidence-based: Research findings are based on empirical evidence, which means that they are grounded in data and observations from the real world. This makes them a reliable and credible source of information.
• Inform decision-making: Research findings can be used to inform decision-making in various fields, such as healthcare, education, and business. By identifying best practices and evidence-based interventions, research findings can help practitioners and policymakers to make informed decisions and improve outcomes.
• Identify gaps in knowledge: Research findings can help to identify gaps in knowledge and understanding of a particular topic, which can then be addressed by further research. This contributes to the ongoing development of knowledge in various fields.
• Improve outcomes : Research findings can be used to develop and implement evidence-based practices and interventions, which have been shown to improve outcomes in various fields, such as healthcare, education, and social services.
• Foster innovation: Research findings can inspire or guide innovation in various fields, such as technology and engineering. By providing new information and understanding of a particular topic, research findings can stimulate new ideas and approaches to problem-solving.
• Enhance credibility: Research findings are generally considered to be more credible and reliable than other types of information, as they are based on rigorous research methods and are subject to peer-review processes.

Limitations of Research Findings

While research findings have many advantages, they also have some limitations. Here are some of the main limitations of research findings:

• Limited scope: Research findings are typically based on a particular study or set of studies, which may have a limited scope or focus. This means that they may not be applicable to other contexts or populations.
• Potential for bias : Research findings can be influenced by various sources of bias, such as researcher bias, selection bias, or measurement bias. This can affect the validity and reliability of the findings.
• Ethical considerations: Research findings can raise ethical considerations, particularly in studies involving human subjects. Researchers must ensure that their studies are conducted in an ethical and responsible manner, with appropriate measures to protect the welfare and privacy of participants.
• Time and resource constraints : Research studies can be time-consuming and require significant resources, which can limit the number and scope of studies that are conducted. This can lead to gaps in knowledge or a lack of research on certain topics.
• Complexity: Some research findings can be complex and difficult to interpret, particularly in fields such as science or medicine. This can make it challenging for practitioners and policymakers to apply the findings to their work.
• Lack of generalizability : While research findings are intended to be generalizable to larger populations or contexts, there may be factors that limit their generalizability. For example, cultural or environmental factors may influence how a particular intervention or treatment works in different populations or contexts.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Data Collection – Methods Types and Examples

Delimitations in Research – Types, Examples and...

Research Process – Steps, Examples and Tips

Research Design – Types, Methods and Examples

Institutional Review Board – Application Sample...

Evaluating Research – Process, Examples and...

• Affiliate Program

• UNITED STATES
• 台灣 (TAIWAN)
• TÜRKIYE (TURKEY)
• Academic Editing Services
• - Research Paper
• - Journal Manuscript
• - Dissertation
• - College & University Assignments
• Admissions Editing Services
• - Application Essay
• - Personal Statement
• - Recommendation Letter
• - Cover Letter
• - CV/Resume
• Business Editing Services
• - Business Documents
• - Report & Brochure
• - Website & Blog
• Writer Editing Services
• - Script & Screenplay
• Our Editors
• Client Reviews
• Editing & Proofreading Prices
• Wordvice Points
• Partner Discount
• Plagiarism Checker
• APA Citation Generator
• MLA Citation Generator
• Chicago Citation Generator
• Vancouver Citation Generator
• - APA Style
• - MLA Style
• - Chicago Style
• - Vancouver Style
• Writing & Editing Guide
• Academic Resources
• Admissions Resources

How to Write the Results/Findings Section in Research

What is the research paper Results section and what does it do?

The Results section of a scientific research paper represents the core findings of a study derived from the methods applied to gather and analyze information. It presents these findings in a logical sequence without bias or interpretation from the author, setting up the reader for later interpretation and evaluation in the Discussion section. A major purpose of the Results section is to break down the data into sentences that show its significance to the research question(s).

The Results section appears third in the section sequence in most scientific papers. It follows the presentation of the Methods and Materials and is presented before the Discussion section —although the Results and Discussion are presented together in many journals. This section answers the basic question “What did you find in your research?”

What is included in the Results section?

The Results section should include the findings of your study and ONLY the findings of your study. The findings include:

• Data presented in tables, charts, graphs, and other figures (may be placed into the text or on separate pages at the end of the manuscript)
• A contextual analysis of this data explaining its meaning in sentence form
• All data that corresponds to the central research question(s)
• All secondary findings (secondary outcomes, subgroup analyses, etc.)

If the scope of the study is broad, or if you studied a variety of variables, or if the methodology used yields a wide range of different results, the author should present only those results that are most relevant to the research question stated in the Introduction section .

As a general rule, any information that does not present the direct findings or outcome of the study should be left out of this section. Unless the journal requests that authors combine the Results and Discussion sections, explanations and interpretations should be omitted from the Results.

How are the results organized?

The best way to organize your Results section is “logically.” One logical and clear method of organizing research results is to provide them alongside the research questions—within each research question, present the type of data that addresses that research question.

Let’s look at an example. Your research question is based on a survey among patients who were treated at a hospital and received postoperative care. Let’s say your first research question is:

“What do hospital patients over age 55 think about postoperative care?”

This can actually be represented as a heading within your Results section, though it might be presented as a statement rather than a question:

Attitudes towards postoperative care in patients over the age of 55

Now present the results that address this specific research question first. In this case, perhaps a table illustrating data from a survey. Likert items can be included in this example. Tables can also present standard deviations, probabilities, correlation matrices, etc.

Following this, present a content analysis, in words, of one end of the spectrum of the survey or data table. In our example case, start with the POSITIVE survey responses regarding postoperative care, using descriptive phrases. For example:

“Sixty-five percent of patients over 55 responded positively to the question “ Are you satisfied with your hospital’s postoperative care ?” (Fig. 2)

Include other results such as subcategory analyses. The amount of textual description used will depend on how much interpretation of tables and figures is necessary and how many examples the reader needs in order to understand the significance of your research findings.

Next, present a content analysis of another part of the spectrum of the same research question, perhaps the NEGATIVE or NEUTRAL responses to the survey. For instance:

  “As Figure 1 shows, 15 out of 60 patients in Group A responded negatively to Question 2.”

After you have assessed the data in one figure and explained it sufficiently, move on to your next research question. For example:

  “How does patient satisfaction correspond to in-hospital improvements made to postoperative care?”

This kind of data may be presented through a figure or set of figures (for instance, a paired T-test table).

Explain the data you present, here in a table, with a concise content analysis:

“The p-value for the comparison between the before and after groups of patients was .03% (Fig. 2), indicating that the greater the dissatisfaction among patients, the more frequent the improvements that were made to postoperative care.”

Let’s examine another example of a Results section from a study on plant tolerance to heavy metal stress . In the Introduction section, the aims of the study are presented as “determining the physiological and morphological responses of Allium cepa L. towards increased cadmium toxicity” and “evaluating its potential to accumulate the metal and its associated environmental consequences.” The Results section presents data showing how these aims are achieved in tables alongside a content analysis, beginning with an overview of the findings:

“Cadmium caused inhibition of root and leave elongation, with increasing effects at higher exposure doses (Fig. 1a-c).”

The figure containing this data is cited in parentheses. Note that this author has combined three graphs into one single figure. Separating the data into separate graphs focusing on specific aspects makes it easier for the reader to assess the findings, and consolidating this information into one figure saves space and makes it easy to locate the most relevant results.

Following this overall summary, the relevant data in the tables is broken down into greater detail in text form in the Results section.

• “Results on the bio-accumulation of cadmium were found to be the highest (17.5 mg kgG1) in the bulb, when the concentration of cadmium in the solution was 1×10G2 M and lowest (0.11 mg kgG1) in the leaves when the concentration was 1×10G3 M.”

Captioning and Referencing Tables and Figures

Tables and figures are central components of your Results section and you need to carefully think about the most effective way to use graphs and tables to present your findings . Therefore, it is crucial to know how to write strong figure captions and to refer to them within the text of the Results section.

The most important advice one can give here as well as throughout the paper is to check the requirements and standards of the journal to which you are submitting your work. Every journal has its own design and layout standards, which you can find in the author instructions on the target journal’s website. Perusing a journal’s published articles will also give you an idea of the proper number, size, and complexity of your figures.

Regardless of which format you use, the figures should be placed in the order they are referenced in the Results section and be as clear and easy to understand as possible. If there are multiple variables being considered (within one or more research questions), it can be a good idea to split these up into separate figures. Subsequently, these can be referenced and analyzed under separate headings and paragraphs in the text.

To create a caption, consider the research question being asked and change it into a phrase. For instance, if one question is “Which color did participants choose?”, the caption might be “Color choice by participant group.” Or in our last research paper example, where the question was “What is the concentration of cadmium in different parts of the onion after 14 days?” the caption reads:

 “Fig. 1(a-c): Mean concentration of Cd determined in (a) bulbs, (b) leaves, and (c) roots of onions after a 14-day period.”

Steps for Composing the Results Section

Because each study is unique, there is no one-size-fits-all approach when it comes to designing a strategy for structuring and writing the section of a research paper where findings are presented. The content and layout of this section will be determined by the specific area of research, the design of the study and its particular methodologies, and the guidelines of the target journal and its editors. However, the following steps can be used to compose the results of most scientific research studies and are essential for researchers who are new to preparing a manuscript for publication or who need a reminder of how to construct the Results section.

Step 1 : Consult the guidelines or instructions that the target journal or publisher provides authors and read research papers it has published, especially those with similar topics, methods, or results to your study.

• The guidelines will generally outline specific requirements for the results or findings section, and the published articles will provide sound examples of successful approaches.
• Note length limitations on restrictions on content. For instance, while many journals require the Results and Discussion sections to be separate, others do not—qualitative research papers often include results and interpretations in the same section (“Results and Discussion”).
• Reading the aims and scope in the journal’s “ guide for authors ” section and understanding the interests of its readers will be invaluable in preparing to write the Results section.

Step 2 : Consider your research results in relation to the journal’s requirements and catalogue your results.

• Focus on experimental results and other findings that are especially relevant to your research questions and objectives and include them even if they are unexpected or do not support your ideas and hypotheses.
• Catalogue your findings—use subheadings to streamline and clarify your report. This will help you avoid excessive and peripheral details as you write and also help your reader understand and remember your findings. Create appendices that might interest specialists but prove too long or distracting for other readers.
• Decide how you will structure of your results. You might match the order of the research questions and hypotheses to your results, or you could arrange them according to the order presented in the Methods section. A chronological order or even a hierarchy of importance or meaningful grouping of main themes or categories might prove effective. Consider your audience, evidence, and most importantly, the objectives of your research when choosing a structure for presenting your findings.

Step 3 : Design figures and tables to present and illustrate your data.

• Tables and figures should be numbered according to the order in which they are mentioned in the main text of the paper.
• Information in figures should be relatively self-explanatory (with the aid of captions), and their design should include all definitions and other information necessary for readers to understand the findings without reading all of the text.
• Use tables and figures as a focal point to tell a clear and informative story about your research and avoid repeating information. But remember that while figures clarify and enhance the text, they cannot replace it.

Step 4 : Draft your Results section using the findings and figures you have organized.

• The goal is to communicate this complex information as clearly and precisely as possible; precise and compact phrases and sentences are most effective.
• In the opening paragraph of this section, restate your research questions or aims to focus the reader’s attention to what the results are trying to show. It is also a good idea to summarize key findings at the end of this section to create a logical transition to the interpretation and discussion that follows.
• Try to write in the past tense and the active voice to relay the findings since the research has already been done and the agent is usually clear. This will ensure that your explanations are also clear and logical.
• Make sure that any specialized terminology or abbreviation you have used here has been defined and clarified in the  Introduction section .

Step 5 : Review your draft; edit and revise until it reports results exactly as you would like to have them reported to your readers.

• Double-check the accuracy and consistency of all the data, as well as all of the visual elements included.
• Read your draft aloud to catch language errors (grammar, spelling, and mechanics), awkward phrases, and missing transitions.
• Ensure that your results are presented in the best order to focus on objectives and prepare readers for interpretations, valuations, and recommendations in the Discussion section . Look back over the paper’s Introduction and background while anticipating the Discussion and Conclusion sections to ensure that the presentation of your results is consistent and effective.
• Consider seeking additional guidance on your paper. Find additional readers to look over your Results section and see if it can be improved in any way. Peers, professors, or qualified experts can provide valuable insights.

One excellent option is to use a professional English proofreading and editing service  such as Wordvice, including our paper editing service . With hundreds of qualified editors from dozens of scientific fields, Wordvice has helped thousands of authors revise their manuscripts and get accepted into their target journals. Read more about the  proofreading and editing process  before proceeding with getting academic editing services and manuscript editing services for your manuscript.

As the representation of your study’s data output, the Results section presents the core information in your research paper. By writing with clarity and conciseness and by highlighting and explaining the crucial findings of their study, authors increase the impact and effectiveness of their research manuscripts.

For more articles and videos on writing your research manuscript, visit Wordvice’s Resources page.

Wordvice Resources

• How to Write a Research Paper Introduction 
• Which Verb Tenses to Use in a Research Paper
• How to Write an Abstract for a Research Paper
• How to Write a Research Paper Title
• Useful Phrases for Academic Writing
• Common Transition Terms in Academic Papers
• Active and Passive Voice in Research Papers
• 100+ Verbs That Will Make Your Research Writing Amazing
• Tips for Paraphrasing in Research Papers

Community Blog

Keep up-to-date on postgraduate related issues with our quick reads written by students, postdocs, professors and industry leaders.

Types of Research – Explained with Examples

• By DiscoverPhDs
• October 2, 2020

Types of Research

Research is about using established methods to investigate a problem or question in detail with the aim of generating new knowledge about it.

It is a vital tool for scientific advancement because it allows researchers to prove or refute hypotheses based on clearly defined parameters, environments and assumptions. Due to this, it enables us to confidently contribute to knowledge as it allows research to be verified and replicated.

Knowing the types of research and what each of them focuses on will allow you to better plan your project, utilises the most appropriate methodologies and techniques and better communicate your findings to other researchers and supervisors.

Classification of Types of Research

There are various types of research that are classified according to their objective, depth of study, analysed data, time required to study the phenomenon and other factors. It’s important to note that a research project will not be limited to one type of research, but will likely use several.

According to its Purpose

Theoretical research.

Theoretical research, also referred to as pure or basic research, focuses on generating knowledge , regardless of its practical application. Here, data collection is used to generate new general concepts for a better understanding of a particular field or to answer a theoretical research question.

Results of this kind are usually oriented towards the formulation of theories and are usually based on documentary analysis, the development of mathematical formulas and the reflection of high-level researchers.

Applied Research

Here, the goal is to find strategies that can be used to address a specific research problem. Applied research draws on theory to generate practical scientific knowledge, and its use is very common in STEM fields such as engineering, computer science and medicine.

This type of research is subdivided into two types:

• Technological applied research : looks towards improving efficiency in a particular productive sector through the improvement of processes or machinery related to said productive processes.
• Scientific applied research : has predictive purposes. Through this type of research design, we can measure certain variables to predict behaviours useful to the goods and services sector, such as consumption patterns and viability of commercial projects.

According to your Depth of Scope

Exploratory research.

Exploratory research is used for the preliminary investigation of a subject that is not yet well understood or sufficiently researched. It serves to establish a frame of reference and a hypothesis from which an in-depth study can be developed that will enable conclusive results to be generated.

Because exploratory research is based on the study of little-studied phenomena, it relies less on theory and more on the collection of data to identify patterns that explain these phenomena.

Descriptive Research

The primary objective of descriptive research is to define the characteristics of a particular phenomenon without necessarily investigating the causes that produce it.

In this type of research, the researcher must take particular care not to intervene in the observed object or phenomenon, as its behaviour may change if an external factor is involved.

Explanatory Research

Explanatory research is the most common type of research method and is responsible for establishing cause-and-effect relationships that allow generalisations to be extended to similar realities. It is closely related to descriptive research, although it provides additional information about the observed object and its interactions with the environment.

Correlational Research

The purpose of this type of scientific research is to identify the relationship between two or more variables. A correlational study aims to determine whether a variable changes, how much the other elements of the observed system change.

According to the Type of Data Used

Qualitative research.

Qualitative methods are often used in the social sciences to collect, compare and interpret information, has a linguistic-semiotic basis and is used in techniques such as discourse analysis, interviews, surveys, records and participant observations.

In order to use statistical methods to validate their results, the observations collected must be evaluated numerically. Qualitative research, however, tends to be subjective, since not all data can be fully controlled. Therefore, this type of research design is better suited to extracting meaning from an event or phenomenon (the ‘why’) than its cause (the ‘how’).

Quantitative Research

Quantitative research study delves into a phenomena through quantitative data collection and using mathematical, statistical and computer-aided tools to measure them . This allows generalised conclusions to be projected over time.

According to the Degree of Manipulation of Variables

Experimental research.

It is about designing or replicating a phenomenon whose variables are manipulated under strictly controlled conditions in order to identify or discover its effect on another independent variable or object. The phenomenon to be studied is measured through study and control groups, and according to the guidelines of the scientific method.

Non-Experimental Research

Also known as an observational study, it focuses on the analysis of a phenomenon in its natural context. As such, the researcher does not intervene directly, but limits their involvement to measuring the variables required for the study. Due to its observational nature, it is often used in descriptive research.

Quasi-Experimental Research

It controls only some variables of the phenomenon under investigation and is therefore not entirely experimental. In this case, the study and the focus group cannot be randomly selected, but are chosen from existing groups or populations . This is to ensure the collected data is relevant and that the knowledge, perspectives and opinions of the population can be incorporated into the study.

According to the Type of Inference

Deductive investigation.

In this type of research, reality is explained by general laws that point to certain conclusions; conclusions are expected to be part of the premise of the research problem and considered correct if the premise is valid and the inductive method is applied correctly.

Inductive Research

In this type of research, knowledge is generated from an observation to achieve a generalisation. It is based on the collection of specific data to develop new theories.

Hypothetical-Deductive Investigation

It is based on observing reality to make a hypothesis, then use deduction to obtain a conclusion and finally verify or reject it through experience.

According to the Time in Which it is Carried Out

Longitudinal study (also referred to as diachronic research).

It is the monitoring of the same event, individual or group over a defined period of time. It aims to track changes in a number of variables and see how they evolve over time. It is often used in medical, psychological and social areas .

Cross-Sectional Study (also referred to as Synchronous Research)

Cross-sectional research design is used to observe phenomena, an individual or a group of research subjects at a given time.

According to The Sources of Information

Primary research.

This fundamental research type is defined by the fact that the data is collected directly from the source, that is, it consists of primary, first-hand information.

Secondary research

Unlike primary research, secondary research is developed with information from secondary sources, which are generally based on scientific literature and other documents compiled by another researcher.

According to How the Data is Obtained

Documentary (cabinet).

Documentary research, or secondary sources, is based on a systematic review of existing sources of information on a particular subject. This type of scientific research is commonly used when undertaking literature reviews or producing a case study.

Field research study involves the direct collection of information at the location where the observed phenomenon occurs.

From Laboratory

Laboratory research is carried out in a controlled environment in order to isolate a dependent variable and establish its relationship with other variables through scientific methods.

Mixed-Method: Documentary, Field and/or Laboratory

Mixed research methodologies combine results from both secondary (documentary) sources and primary sources through field or laboratory research.

Thinking about applying to a PhD? Then don’t miss out on these 4 tips on how to best prepare your application.

Considering whether to do an MBA or a PhD? If so, find out what their differences are, and more importantly, which one is better suited for you.

Are you always finding yourself working on sections of your research tasks right up until your deadlines? Are you still finding yourself distracted the moment

Join thousands of other students and stay up to date with the latest PhD programmes, funding opportunities and advice.

Browse PhDs Now

The term research instrument refers to any tool that you may use to collect, measure and analyse research data.

In Finland, all new PhD holders are given a traditional Doctoral Hat and Doctoral Sword during a Conferment Ceremony, symbolising the freedom of research.

Dr Manning gained his PhD in Chemical Engineering from the University of Sheffield in 2019. He is now a postdoc researcher studying molecular simulations on nanomaterials at the University of Bath.

Dr Roberts gained her PhD from Duke University in 2014 in the field of biomedical engineering. She now runs her own business named Personal Finance for PhDs.

Join Thousands of Students

• Anxiety Disorder
• Bipolar Disorder
• Schizophrenia
• Adjustment Disorder
• Agoraphobia
• Antisocial Personality Disorder
• Borderline Personality Disorder
• Childhood ADHD
• Dissociative Identity Disorder
• Narcissistic Personality Disorder
• Oppositional Defiant Disorder
• Panic Attack
• Postpartum Depression
• Schizoaffective Disorder
• Seasonal Affective Disorder
• Sex Addiction
• Social Anxiety
• Specific Phobias
• Teenage Depression
• Black Mental Health
• Emotional Health
• Sex & Relationships
• Understanding Therapy
• Workplace Mental Health
• My Life with OCD
• Caregivers Chronicles
• Empathy at Work
• Sex, Love & All of the Above
• Parent Central
• Mindful Moment
• Mental Health News
• Live Town Hall: Mental Health in Focus
• Inside Mental Health
• Inside Schizophrenia
• Inside Bipolar
• ADHD Symptoms Quiz
• Anxiety Symptoms Quiz
• Autism Quiz: Family & Friends
• Autism Symptoms Quiz
• Bipolar Disorder Quiz
• Borderline Personality Test
• Childhood ADHD Quiz
• Depression Symptoms Quiz
• Eating Disorder Quiz
• Narcissim Symptoms Test
• OCD Symptoms Quiz
• Psychopathy Test
• PTSD Symptoms Quiz
• Schizophrenia Quiz
• Attachment Style Quiz
• Career Test
• Do I Need Therapy Quiz?
• Domestic Violence Screening Quiz
• Emotional Type Quiz
• Loneliness Quiz
• Parenting Style Quiz
• Personality Test
• Relationship Quiz
• Stress Test
• What's Your Sleep Like?
• Find Support
• Suicide Prevention
• Drugs & Medications
• Find a Therapist

Research 101: Understanding Research Studies

One of the secrets of science is to understand the language of science, and science’s primary language is the research study . Research studies allow scientists to communicate with one another and share results of their work. There are many different kinds of research and many varying fields of research. And although journals were designed to help professionals communicate such research findings with one another, many times professionals in one field don’t significantly interact with (or are even aware of) researchers in a different field than themselves (e.g., a neuropsychologist may not keep up on the same research findings as a neurologist). This article reviews the major types of research done in the social, behavioral and brain sciences and provides some guideposts to better evaluate the context in which to place new research.

Types of Research

The basis of a scientific research study follows a common pattern:

• Define the question
• Gather information and resources
• Form hypotheses
• Perform an experiment and collect data
• Analyze the data
• Interpret the data and draw conclusions
• Publish results in a peer-reviewed journal

While there are dozens of types of research, most research done falls into one of five categories: clinical case studies; small, non-randomized studies or surveys; large, randomized clinical studies; literature reviews; and meta-analytic studies. Studies can also occur in widely varying fields, from psychology, pharmacology and sociology (what I’ll call “behavioral and treatment studies”), to genetics and brain scans (what I’ll call “organic studies”) to animal studies. Some fields contribute results that are more instantly relevant, while others’ results may help researchers develop new tests or treatments decades from now.

Clinical Case Studies

A clinical case study involves reporting on a single case (or series of cases) that the researcher or clinician has tracked over a period of some significant time (usually months or even years). Many times, such case studies emphasize a narrative or more subjective approach, but may also include objective measures. For instance, a researcher might publish a case study about the positive effects of cognitive-behavioral psychotherapy for a person with depression. The researcher measured the client’s level of depression with an objective measure such as the Beck Depression Inventory, but also describes in detail the client’s progress with specific cognitive-behavioral techniques , such as doing regular “homework” or keeping a journal of one’s thoughts.

The clinical case study is a very good research design for generating and testing hypotheses that may be used in larger studies. It is also a very good manner for disseminating the effectiveness of specific or novel techniques for individuals, or for those that may have a fairly uncommon set of diagnoses. However, generally a clinical case study’s results are not able to be generalized to a broader population. A case study is therefore of limited value to the general population.

Small Studies and Survey Research

There’s no specific criteria that differentiates a “small study” from a “large study,” but I place any non-randomized study in this category, as well as pretty much all survey research. Small studies are generally conducted on student populations (because students are often required to be a research subject for their university psychology classes), involve less than 80 to 100 participants or subjects, and often lack at least one of the core, important research components most often found in larger studies. This component can be the lack of true randomization of subjects, a lack of heterogeneity (e.g., no diversity in the population being studied), or a lack of a control group (or a relevant control group, e.g. a placebo control).

Most survey research also falls into this category, because it also lacks one of these core research components. For instance, a lot of survey research asks participants to identify themselves as having a particular problem, and if they do, then they fill out the survey. While this will almost guarantee the researchers interesting results, it’s also not very generalizable.

The upshot is that while these studies often provide interesting insights and information that can be used for future research, people shouldn’t read too much into these research findings. They are important data points in our overall understanding of the subject. When you take 10 or 20 of these data points and string them together, they should provide a fairly clear and consistent picture about the topic. If the results don’t provide such a clear picture, then there is likely more work to be done in the subject area before meaningful conclusions can be made. Literature reviews and meta-analyses (discussed below) help professionals and individuals better understand such findings over time.

Large, Randomized Studies

Large, randomized studies that draw from diverse populations and include relevant, appropriate control groups are considered the “gold standard” in research. So why aren’t they done more often? Such large studies, often done at multiple geographic locations, are very expensive to run because they include dozens of researchers, research assistants, statisticians, and other professionals as well as hundreds, and sometimes thousands, of subjects or participants. But the findings from such research are robust and can be generalized to others far more easily, so their value to research is important.

Large studies are not immune to problems found in other kinds of research. It’s just that the problems tend to have a much smaller effect, if there are any, since the number of subjects is so large and mixed (heterogeneous). When properly designed and using accepted statistical analyses, large research studies provide both individuals and professionals with solid findings that they can act upon.

Literature Reviews

A literature review is pretty much what it describes. Virtually all peer-reviewed, published research includes what might be called a “mini literature review” in its introduction. In this section of a study, the researchers review previous studies to put the current study into some context. “Research X found 123, Research Y found 456, so we hope to find 789.”

Sometimes, however, the number of studies in a particular area of study is so large and covers so many results that it’s difficult to understand exactly what our understanding is at the moment. To help give researchers a better understanding and context for future research, a literature review may be conducted and published as its own “study.” This will basically be a comprehensive, large-scale review of all studies in a particular area published within the past 10 or 20 years. The review will describe the research efforts, expand on specific findings, and may draw some general conclusions that can be gleaned from such a global review. These reviews are usually fairly subjective and are mainly for other professionals. Their use to the general public is limited and they almost never produce new findings of interest.

Meta-Analytic Studies

A meta-analysis is similar to a literature review in that it seeks to examine all previous research in a very specific topic area. However, unlike a literature review, a meta-analytic study takes the review one important step further – it actually pulls together all of the previous study’s data and analyzes it with additional statistics to draw global conclusions about the data. Why bother? Because so much research is published in many fields that it’s virtually impossible for an individual to draw any solid conclusions from the research without such a global review that pulls together all that data and statistically analyzes it for trends and solid findings.

The key to meta-analytic studies is to understand that researchers can alter the results of such a review by being particular (or not very particular) about the kinds of studies they include in their review. If, for instance, the researchers decide to include non-randomized studies in their review, they will often get different findings than if they hadn’t included them. Sometimes researchers will require certain statistical procedures to have been performed in order for the study to be included, or certain data thresholds to be met (e.g., we’ll only examine studies that had more than 50 subjects). Depending upon what criteria researchers choose to include in their meta-analysis, it will effect the results of the meta-analysis.

Meta-analytic studies, when done properly, are important contributions to our scientific knowledge and understanding. When a meta-analysis is published, it generally acts as a new foundation for other studies to build upon. It also synthesizes a great deal of previous knowledge into a more digestable Chunk of Knowledge for everyone.

Three General Categories of Research

While we’ve discussed the five general types of research in behavioral and mental health, there are also three other categories to consider.

Behavioral & Treatment Studies

Behavioral or treatment studies examine specific behaviors, treatments or therapies and see how they work on people. In psychology and sociology, most research conducted is of this nature. Such research provides direct insights into human behavior or therapeutic techniques that may be of value for treating a specific kind of disorder. This kind of research also helps us better understand a specific health or mental health concern, and how it manifests itself in a certain group of people (e.g., teenagers versus adults). This is the most “actionable” type of research – research that professionals and individuals can take action based upon its findings.

Organic Studies

Research that examines brain structures, neurochemical reactions via PET or other brain imaging techniques, gene research, or research that examines other organic structures in a human body falls under this category. In most cases, such research helps further our understanding of the human body and how it functions, but doesn’t provide immediate insight or help in dealing with a problem today, or suggest new treatments that will be readily available. For instance, researchers often publish findings about how a particular gene may be correlated with a specific disorder. While such findings may eventually lead to some sort of medical test being developed for the disorder, it may be a decade or two before a finding of this nature translates into an actual test or new treatment method.

While such research is vitally important to our eventual better understanding of how our brains and bodies function, research in this category tends not to have much importance today for people dealing with a mental disorder or mental health problem.

Animal Studies

Research is sometimes conducted on an animal to better understand how a specific organ system (such as the brain) reacts to changes, or how an animal’s behavior may be altered by specific social or environmental changes. Animal research, mostly on rats, in the 1950’s and 1960’s focused on studying animal behavior which, in psychology, led to the field of behaviorism and behavior therapy . More recently, the focus of animal studies has been on their biological makeup, to examine certain brain structures and genes related to health or mental health issues.

While certain animals have organ systems that may be very similar to human organ systems, results from animal studies are not automatically generalizable to humans. Animal studies are therefore of limited value to the general population. Research news based upon an animal study generally means any possible significant treatments from such a study are at least a decade or more away from being introduced. In many cases, no specific treatments are developed from animal studies, instead they are used to better understand how a human organ system functions or reacts to a change.

Research in the social sciences and in pharmacology is important because it helps us not only better understand human behavior (both normal and dysfunctional behavior), but also to find more effective and less time-consuming treatments to help with a person is suffering from an emotional or mental health issue.

The best kind of research – large-scale, randomized studies – are also the most rare because of their cost and the amount of resources needed to undertake them. Smaller-scale studies also contribute important data points along the way, inbetween the larger studies, while meta-analyses and literature reviews helps us gain a more global perspective and understanding of our knowledge so far.

While animal research and studies into the brain’s structures and genes are important to contributing to our overall better understanding of how our brains and bodies function, behavioral and treatment research provide concrete data that can generally be used immediately to help people improve their lives.

Last medically reviewed on May 17, 2016

Read this next

JK Rowling broke out medical science to defend her controversial stance on trans women. Responding to twitter

When we're feeling stressed, studies show that one of the best things we can do is to help someone else. Try these simple suggestions for being of…

Even before the nationwide lockdowns, there were far too many people in the U.S. with not enough to eat. The p

I suspect that when most people think about single parents, they think about single mothers. And, yes, single

In the latest example of tone-deaf celeb activism, several celebs including Julianne Moore, Sara Paulson, and

No good comes from constantly refreshing your newsfeed for the latest on COVID-19. Here are 10 things you can do that aren't related to coronavirus.

On Facebook a few days ago, a friend posted that there was no toilet paper anywhere in the town where I live.

If someone lacks food and water, we know the body will suffer. But what about when they lack a sense of belong

Are you an ABA service provider (a BCBA, BCaBA, or other clinician providing ABA services)? Does part of your

To trust someone you love is important. Otherwise, you’ll be forever doubting that person, creating serious

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case NPS+ Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research

What is Research: Definition, Methods, Types & Examples

The search for knowledge is closely linked to the object of study; that is, to the reconstruction of the facts that will provide an explanation to an observed event and that at first sight can be considered as a problem. It is very human to seek answers and satisfy our curiosity. Let’s talk about research.

Content Index

What is Research?

What are the characteristics of research.

• Comparative analysis chart

Qualitative methods

Quantitative methods, 8 tips for conducting accurate research.

Research is the careful consideration of study regarding a particular concern or research problem using scientific methods. According to the American sociologist Earl Robert Babbie, “research is a systematic inquiry to describe, explain, predict, and control the observed phenomenon. It involves inductive and deductive methods.”

Inductive methods analyze an observed event, while deductive methods verify the observed event. Inductive approaches are associated with qualitative research , and deductive methods are more commonly associated with quantitative analysis .

Research is conducted with a purpose to:

• Identify potential and new customers
• Understand existing customers
• Set pragmatic goals
• Develop productive market strategies
• Address business challenges
• Put together a business expansion plan
• Identify new business opportunities
• Good research follows a systematic approach to capture accurate data. Researchers need to practice ethics and a code of conduct while making observations or drawing conclusions.
• The analysis is based on logical reasoning and involves both inductive and deductive methods.
• Real-time data and knowledge is derived from actual observations in natural settings.
• There is an in-depth analysis of all data collected so that there are no anomalies associated with it.
• It creates a path for generating new questions. Existing data helps create more research opportunities.
• It is analytical and uses all the available data so that there is no ambiguity in inference.
• Accuracy is one of the most critical aspects of research. The information must be accurate and correct. For example, laboratories provide a controlled environment to collect data. Accuracy is measured in the instruments used, the calibrations of instruments or tools, and the experiment’s final result.

What is the purpose of research?

There are three main purposes:

• Exploratory: As the name suggests, researchers conduct exploratory studies to explore a group of questions. The answers and analytics may not offer a conclusion to the perceived problem. It is undertaken to handle new problem areas that haven’t been explored before. This exploratory data analysis process lays the foundation for more conclusive data collection and analysis.

LEARN ABOUT: Descriptive Analysis

• Descriptive: It focuses on expanding knowledge on current issues through a process of data collection. Descriptive research describe the behavior of a sample population. Only one variable is required to conduct the study. The three primary purposes of descriptive studies are describing, explaining, and validating the findings. For example, a study conducted to know if top-level management leaders in the 21st century possess the moral right to receive a considerable sum of money from the company profit.

LEARN ABOUT: Best Data Collection Tools

• Explanatory: Causal research or explanatory research is conducted to understand the impact of specific changes in existing standard procedures. Running experiments is the most popular form. For example, a study that is conducted to understand the effect of rebranding on customer loyalty.

Here is a comparative analysis chart for a better understanding:

It begins by asking the right questions and choosing an appropriate method to investigate the problem. After collecting answers to your questions, you can analyze the findings or observations to draw reasonable conclusions.

When it comes to customers and market studies, the more thorough your questions, the better the analysis. You get essential insights into brand perception and product needs by thoroughly collecting customer data through surveys and questionnaires . You can use this data to make smart decisions about your marketing strategies to position your business effectively.

To make sense of your study and get insights faster, it helps to use a research repository as a single source of truth in your organization and manage your research data in one centralized data repository .

Types of research methods and Examples

Research methods are broadly classified as Qualitative and Quantitative .

Both methods have distinctive properties and data collection methods.

Qualitative research is a method that collects data using conversational methods, usually open-ended questions . The responses collected are essentially non-numerical. This method helps a researcher understand what participants think and why they think in a particular way.

Types of qualitative methods include:

• One-to-one Interview
• Focus Groups
• Ethnographic studies
• Text Analysis

Quantitative methods deal with numbers and measurable forms . It uses a systematic way of investigating events or data. It answers questions to justify relationships with measurable variables to either explain, predict, or control a phenomenon.

Types of quantitative methods include:

• Survey research
• Descriptive research
• Correlational research

LEARN MORE: Descriptive Research vs Correlational Research

Remember, it is only valuable and useful when it is valid, accurate, and reliable. Incorrect results can lead to customer churn and a decrease in sales.

It is essential to ensure that your data is:

• Valid – founded, logical, rigorous, and impartial.
• Accurate – free of errors and including required details.
• Reliable – other people who investigate in the same way can produce similar results.
• Timely – current and collected within an appropriate time frame.
• Complete – includes all the data you need to support your business decisions.

Gather insights

• Identify the main trends and issues, opportunities, and problems you observe. Write a sentence describing each one.
• Keep track of the frequency with which each of the main findings appears.
• Make a list of your findings from the most common to the least common.
• Evaluate a list of the strengths, weaknesses, opportunities, and threats identified in a SWOT analysis .
• Prepare conclusions and recommendations about your study.
• Act on your strategies
• Look for gaps in the information, and consider doing additional inquiry if necessary
• Plan to review the results and consider efficient methods to analyze and interpret results.

Review your goals before making any conclusions about your study. Remember how the process you have completed and the data you have gathered help answer your questions. Ask yourself if what your analysis revealed facilitates the identification of your conclusions and recommendations.

LEARN MORE ABOUT OUR SOFTWARE         FREE TRIAL

MORE LIKE THIS

Employee Engagement App: Top 11 For Workforce Improvement 

Apr 10, 2024

Top 15 Employee Evaluation Software to Enhance Performance

Event Feedback Software: Top 11 Best in 2024

Apr 9, 2024

Top 10 Free Market Research Tools to Boost Your Business

Other categories.

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Uncategorized
• Video Learning Series
• What’s Coming Up
• Workforce Intelligence

• Resources Home 🏠
• Try SciSpace Copilot
• Search research papers
• Add Copilot Extension
• Try AI Detector
• Try Paraphraser
• Try Citation Generator
• April Papers
• June Papers
• July Papers

A Comprehensive Guide to Methodology in Research

Table of Contents

Research methodology plays a crucial role in any study or investigation. It provides the framework for collecting, analyzing, and interpreting data, ensuring that the research is reliable, valid, and credible. Understanding the importance of research methodology is essential for conducting rigorous and meaningful research.

In this article, we'll explore the various aspects of research methodology, from its types to best practices, ensuring you have the knowledge needed to conduct impactful research.

What is Research Methodology?

Research methodology refers to the system of procedures, techniques, and tools used to carry out a research study. It encompasses the overall approach, including the research design, data collection methods, data analysis techniques, and the interpretation of findings.

Research methodology plays a crucial role in the field of research, as it sets the foundation for any study. It provides researchers with a structured framework to ensure that their investigations are conducted in a systematic and organized manner. By following a well-defined methodology, researchers can ensure that their findings are reliable, valid, and meaningful.

When defining research methodology, one of the first steps is to identify the research problem. This involves clearly understanding the issue or topic that the study aims to address. By defining the research problem, researchers can narrow down their focus and determine the specific objectives they want to achieve through their study.

How to Define Research Methodology

Once the research problem is identified, researchers move on to defining the research questions. These questions serve as a guide for the study, helping researchers to gather relevant information and analyze it effectively. The research questions should be clear, concise, and aligned with the overall goals of the study.

After defining the research questions, researchers need to determine how data will be collected and analyzed. This involves selecting appropriate data collection methods, such as surveys, interviews, observations, or experiments. The choice of data collection methods depends on various factors, including the nature of the research problem, the target population, and the available resources.

Once the data is collected, researchers need to analyze it using appropriate data analysis techniques. This may involve statistical analysis, qualitative analysis, or a combination of both, depending on the nature of the data and the research questions. The analysis of data helps researchers to draw meaningful conclusions and make informed decisions based on their findings.

Role of Methodology in Research

Methodology plays a crucial role in research, as it ensures that the study is conducted in a systematic and organized manner. It provides a clear roadmap for researchers to follow, ensuring that the research objectives are met effectively. By following a well-defined methodology, researchers can minimize bias, errors, and inconsistencies in their study, thus enhancing the reliability and validity of their findings.

In addition to providing a structured approach, research methodology also helps in establishing the reliability and validity of the study. Reliability refers to the consistency and stability of the research findings, while validity refers to the accuracy and truthfulness of the findings. By using appropriate research methods and techniques, researchers can ensure that their study produces reliable and valid results, which can be used to make informed decisions and contribute to the existing body of knowledge.

Steps in Choosing the Right Research Methodology

Choosing the appropriate research methodology for your study is a critical step in ensuring the success of your research. Let's explore some steps to help you select the right research methodology:

Identifying the Research Problem

The first step in choosing the right research methodology is to clearly identify and define the research problem. Understanding the research problem will help you determine which methodology will best address your research questions and objectives.

Identifying the research problem involves a thorough examination of the existing literature in your field of study. This step allows you to gain a comprehensive understanding of the current state of knowledge and identify any gaps that your research can fill. By identifying the research problem, you can ensure that your study contributes to the existing body of knowledge and addresses a significant research gap.

Once you have identified the research problem, you need to consider the scope of your study. Are you focusing on a specific population, geographic area, or time frame? Understanding the scope of your research will help you determine the appropriate research methodology to use.

Reviewing Previous Research

Before finalizing the research methodology, it is essential to review previous research conducted in the field. This will allow you to identify gaps, determine the most effective methodologies used in similar studies, and build upon existing knowledge.

Reviewing previous research involves conducting a systematic review of relevant literature. This process includes searching for and analyzing published studies, articles, and reports that are related to your research topic. By reviewing previous research, you can gain insights into the strengths and limitations of different methodologies and make informed decisions about which approach to adopt.

During the review process, it is important to critically evaluate the quality and reliability of the existing research. Consider factors such as the sample size, research design, data collection methods, and statistical analysis techniques used in previous studies. This evaluation will help you determine the most appropriate research methodology for your own study.

Formulating Research Questions

Once the research problem is identified, formulate specific and relevant research questions. These questions will guide your methodology selection process by helping you determine what type of data you need to collect and how to analyze it.

Formulating research questions involves breaking down the research problem into smaller, more manageable components. These questions should be clear, concise, and measurable. They should also align with the objectives of your study and provide a framework for data collection and analysis.

When formulating research questions, consider the different types of data that can be collected, such as qualitative or quantitative data. Depending on the nature of your research questions, you may need to employ different data collection methods, such as interviews, surveys, observations, or experiments. By carefully formulating research questions, you can ensure that your chosen methodology will enable you to collect the necessary data to answer your research questions effectively.

Implementing the Research Methodology

After choosing the appropriate research methodology, it is time to implement it. This stage involves collecting data using various techniques and analyzing the gathered information. Let's explore two crucial aspects of implementing the research methodology:

Data Collection Techniques

Data collection techniques depend on the chosen research methodology. They can include surveys, interviews, observations, experiments, or document analysis. Selecting the most suitable data collection techniques will ensure accurate and relevant data for your study.

Data Analysis Methods

Data analysis is a critical part of the research process. It involves interpreting and making sense of the collected data to draw meaningful conclusions. Depending on the research methodology, data analysis methods can include statistical analysis, content analysis, thematic analysis, or grounded theory.

Ensuring the Validity and Reliability of Your Research

In order to ensure the validity and reliability of your research findings, it is important to address these two key aspects:

Understanding Validity in Research

Validity refers to the accuracy and soundness of a research study. It is crucial to ensure that the research methods used effectively measure what they intend to measure. Researchers can enhance validity by using proper sampling techniques, carefully designing research instruments, and ensuring accurate data collection.

Ensuring Reliability in Your Study

Reliability refers to the consistency and stability of the research results. It is important to ensure that the research methods and instruments used yield consistent and reproducible results. Researchers can enhance reliability by using standardized procedures, ensuring inter-rater reliability, and conducting pilot studies.

A comprehensive understanding of research methodology is essential for conducting high-quality research. By selecting the right research methodology, researchers can ensure that their studies are rigorous, reliable, and valid. It is crucial to follow the steps in choosing the appropriate methodology, implement the chosen methodology effectively, and address validity and reliability concerns throughout the research process. By doing so, researchers can contribute valuable insights and advances in their respective fields.

You might also like

AI for Meta Analysis — A Comprehensive Guide

How To Write An Argumentative Essay

Beyond Google Scholar: Why SciSpace is the best alternative

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 24, Issue 2
• Five tips for developing useful literature summary tables for writing review articles
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0003-0157-5319 Ahtisham Younas 1 , 2 ,
• http://orcid.org/0000-0002-7839-8130 Parveen Ali 3 , 4
• 1 Memorial University of Newfoundland , St John's , Newfoundland , Canada
• 2 Swat College of Nursing , Pakistan
• 3 School of Nursing and Midwifery , University of Sheffield , Sheffield , South Yorkshire , UK
• 4 Sheffield University Interpersonal Violence Research Group , Sheffield University , Sheffield , UK
• Correspondence to Ahtisham Younas, Memorial University of Newfoundland, St John's, NL A1C 5C4, Canada; ay6133{at}mun.ca

https://doi.org/10.1136/ebnurs-2021-103417

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Literature reviews offer a critical synthesis of empirical and theoretical literature to assess the strength of evidence, develop guidelines for practice and policymaking, and identify areas for future research. 1 It is often essential and usually the first task in any research endeavour, particularly in masters or doctoral level education. For effective data extraction and rigorous synthesis in reviews, the use of literature summary tables is of utmost importance. A literature summary table provides a synopsis of an included article. It succinctly presents its purpose, methods, findings and other relevant information pertinent to the review. The aim of developing these literature summary tables is to provide the reader with the information at one glance. Since there are multiple types of reviews (eg, systematic, integrative, scoping, critical and mixed methods) with distinct purposes and techniques, 2 there could be various approaches for developing literature summary tables making it a complex task specialty for the novice researchers or reviewers. Here, we offer five tips for authors of the review articles, relevant to all types of reviews, for creating useful and relevant literature summary tables. We also provide examples from our published reviews to illustrate how useful literature summary tables can be developed and what sort of information should be provided.

Tip 1: provide detailed information about frameworks and methods

• Download figure
• Open in new tab
• Download powerpoint

Tabular literature summaries from a scoping review. Source: Rasheed et al . 3

The provision of information about conceptual and theoretical frameworks and methods is useful for several reasons. First, in quantitative (reviews synthesising the results of quantitative studies) and mixed reviews (reviews synthesising the results of both qualitative and quantitative studies to address a mixed review question), it allows the readers to assess the congruence of the core findings and methods with the adapted framework and tested assumptions. In qualitative reviews (reviews synthesising results of qualitative studies), this information is beneficial for readers to recognise the underlying philosophical and paradigmatic stance of the authors of the included articles. For example, imagine the authors of an article, included in a review, used phenomenological inquiry for their research. In that case, the review authors and the readers of the review need to know what kind of (transcendental or hermeneutic) philosophical stance guided the inquiry. Review authors should, therefore, include the philosophical stance in their literature summary for the particular article. Second, information about frameworks and methods enables review authors and readers to judge the quality of the research, which allows for discerning the strengths and limitations of the article. For example, if authors of an included article intended to develop a new scale and test its psychometric properties. To achieve this aim, they used a convenience sample of 150 participants and performed exploratory (EFA) and confirmatory factor analysis (CFA) on the same sample. Such an approach would indicate a flawed methodology because EFA and CFA should not be conducted on the same sample. The review authors must include this information in their summary table. Omitting this information from a summary could lead to the inclusion of a flawed article in the review, thereby jeopardising the review’s rigour.

Tip 2: include strengths and limitations for each article

Critical appraisal of individual articles included in a review is crucial for increasing the rigour of the review. Despite using various templates for critical appraisal, authors often do not provide detailed information about each reviewed article’s strengths and limitations. Merely noting the quality score based on standardised critical appraisal templates is not adequate because the readers should be able to identify the reasons for assigning a weak or moderate rating. Many recent critical appraisal checklists (eg, Mixed Methods Appraisal Tool) discourage review authors from assigning a quality score and recommend noting the main strengths and limitations of included studies. It is also vital that methodological and conceptual limitations and strengths of the articles included in the review are provided because not all review articles include empirical research papers. Rather some review synthesises the theoretical aspects of articles. Providing information about conceptual limitations is also important for readers to judge the quality of foundations of the research. For example, if you included a mixed-methods study in the review, reporting the methodological and conceptual limitations about ‘integration’ is critical for evaluating the study’s strength. Suppose the authors only collected qualitative and quantitative data and did not state the intent and timing of integration. In that case, the strength of the study is weak. Integration only occurred at the levels of data collection. However, integration may not have occurred at the analysis, interpretation and reporting levels.

Tip 3: write conceptual contribution of each reviewed article

While reading and evaluating review papers, we have observed that many review authors only provide core results of the article included in a review and do not explain the conceptual contribution offered by the included article. We refer to conceptual contribution as a description of how the article’s key results contribute towards the development of potential codes, themes or subthemes, or emerging patterns that are reported as the review findings. For example, the authors of a review article noted that one of the research articles included in their review demonstrated the usefulness of case studies and reflective logs as strategies for fostering compassion in nursing students. The conceptual contribution of this research article could be that experiential learning is one way to teach compassion to nursing students, as supported by case studies and reflective logs. This conceptual contribution of the article should be mentioned in the literature summary table. Delineating each reviewed article’s conceptual contribution is particularly beneficial in qualitative reviews, mixed-methods reviews, and critical reviews that often focus on developing models and describing or explaining various phenomena. Figure 2 offers an example of a literature summary table. 4

Tabular literature summaries from a critical review. Source: Younas and Maddigan. 4

Tip 4: compose potential themes from each article during summary writing

While developing literature summary tables, many authors use themes or subthemes reported in the given articles as the key results of their own review. Such an approach prevents the review authors from understanding the article’s conceptual contribution, developing rigorous synthesis and drawing reasonable interpretations of results from an individual article. Ultimately, it affects the generation of novel review findings. For example, one of the articles about women’s healthcare-seeking behaviours in developing countries reported a theme ‘social-cultural determinants of health as precursors of delays’. Instead of using this theme as one of the review findings, the reviewers should read and interpret beyond the given description in an article, compare and contrast themes, findings from one article with findings and themes from another article to find similarities and differences and to understand and explain bigger picture for their readers. Therefore, while developing literature summary tables, think twice before using the predeveloped themes. Including your themes in the summary tables (see figure 1 ) demonstrates to the readers that a robust method of data extraction and synthesis has been followed.

Tip 5: create your personalised template for literature summaries

Often templates are available for data extraction and development of literature summary tables. The available templates may be in the form of a table, chart or a structured framework that extracts some essential information about every article. The commonly used information may include authors, purpose, methods, key results and quality scores. While extracting all relevant information is important, such templates should be tailored to meet the needs of the individuals’ review. For example, for a review about the effectiveness of healthcare interventions, a literature summary table must include information about the intervention, its type, content timing, duration, setting, effectiveness, negative consequences, and receivers and implementers’ experiences of its usage. Similarly, literature summary tables for articles included in a meta-synthesis must include information about the participants’ characteristics, research context and conceptual contribution of each reviewed article so as to help the reader make an informed decision about the usefulness or lack of usefulness of the individual article in the review and the whole review.

In conclusion, narrative or systematic reviews are almost always conducted as a part of any educational project (thesis or dissertation) or academic or clinical research. Literature reviews are the foundation of research on a given topic. Robust and high-quality reviews play an instrumental role in guiding research, practice and policymaking. However, the quality of reviews is also contingent on rigorous data extraction and synthesis, which require developing literature summaries. We have outlined five tips that could enhance the quality of the data extraction and synthesis process by developing useful literature summaries.

• Aromataris E ,
• Rasheed SP ,

Twitter @Ahtisham04, @parveenazamali

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Patient consent for publication Not required.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF:

Exploring Types of Research Methods: A Comprehensive Guide

Grasping the concept of research method is essential for anyone engaged in research or assessing the outcomes of studies. Whether you're an academic student, a dedicated researcher, or just inquisitive about the world, a thorough understanding of the diverse research methods will assist you in sifting through the extensive array of information at your disposal.

Our detailed guide will walk you through the types of research design, including qualitative and quantitative approaches, as well as descriptive, correlational, experimental, and mixed methods research. We will also touch upon the different types of research methodology, ensuring a comprehensive understanding of the various types of methods in research.

This article will also highlight the pivotal factors to consider when crafting a study and the inherent strengths and limitations of different type of research methods.Whether you're embarking on your own research project or looking to enhance your critical thinking skills Armed with the research methods definition, this guide will equip you with the essential knowledge to make well-informed decisions and formulate significant conclusions in the field of research.

Qualitative vs. Quantitative Research Methods

Qualitative and quantitative research methods represent two fundamentally different approaches to data collection and analysis. Qualitative observation delves into non-numerical data, while quantitative observation involves the scrutiny of data that is numerical and quantifiable.

Qualitative Research:

• Involves gathering and interpreting non-numerical data, such as text, video, photographs, or audio recordings
• Uses sources like interviews, focus groups, documents, personal accounts, cultural records, and observation
• Unstructured or semi-structured format
• Open-ended questions
• Comprehensive perspective on individuals' experiences
• Comparison of participants' feedback and input
• Focus on answering the "why" behind a phenomenon, correlation, or behavior
• Ethnography, for instance, seeks to gain insights into phenomena, groups, or experiences that cannot be objectively measured or quantified, offering a deep dive into the cultural fabric of a community.
• This method is used to understand how an individual subjectively perceives and imparts meaning to their social reality, often revealing underlying bias that can influence the interpretation of social phenomena.
• Data analysis techniques include content analysis, grounded theory, thematic analysis, or discourse analysis

Quantitative Research:

• Focuses on numerical or measurable data
• Uses sources such as experiments, questionnaires, surveys, and database reports
• Multiple-choice format
• Countable answers (e.g., "yes" or "no")
• Numerical analysis
• Statistical picture of a trend or connection
• To define research methods, one must focus on answering the 'what' or 'how' in relation to a particular phenomenon, correlation, or behavior. This foundational approach is crucial in the realm of empirical inquiry.
• Provides precise causal explanations that can be measured and communicated mathematically
• The objectives of scientific inquiry often include hypothesis testing to examine causal relationships between variables, making accurate predictions, and generalizing findings to broader populations.
• Aims to establish general laws of behavior and phenomenon across different settings/contexts
• Used to test a theory and ultimately support or reject it
• Empirical research in psychology utilizes examples of quantitative data such as standardized psychological assessments, neuroimaging data, and clinical outcome measures to inform its findings.
• Data analysis techniques include descriptive and inferential statistics

When selecting research methodology types, it's important to consider various factors such as the study's primary goal, the nature of the research questions and conceptual framework, the variables involved, the context of the study, ethical issues, and whether the focus is on individuals or groups, or on comparing groups and understanding their relationships.

Research design methods play a pivotal role in determining the appropriateness of qualitative methods for studies involving individuals or groups, while quantitative methods are often chosen for studies aimed at comparing groups or deciphering the relationship between variables.

Descriptive Research

Descriptive research is a methodological approach that aims to accurately and systematically depict a population, situation, or phenomenon. It adeptly addresses 'what', 'where', 'when', and 'how' questions, although it steers clear of exploring 'why'. Employing a descriptive research design means observing and documenting variables without exerting control or manipulation, which is particularly beneficial when exploring new topics or problems to identify characteristics, frequencies, trends, and categories.

Descriptive research methods include:

• Surveys: Survey research is a powerful tool that enables researchers to collect extensive data sets, which can then be meticulously analyzed to uncover frequencies, averages, and emerging patterns.
• Observations: Utilizing observation allows researchers to collect data on behaviors and phenomena, ensuring the gathered information is not tainted by the honesty or accuracy of respondents.
• Case studies: Case study research delves into detailed data to pinpoint the unique characteristics of a narrowly defined subject, providing in-depth insights.

Descriptive research can be conducted in different ways:

• Cross-sectional : Observing a population at a single point in time.
• Longitudinal : Following a population over a period of time.
• Surveys or interviews : When the researcher interacts with the participant.
• Observational studies or data collection using existing records : When the researcher does not interact with the participant.

Advantages of descriptive research include:

• Varied data collection methods
• A natural environment for respondents
• Quick and cheap data collection
• A holistic understanding of the research topic

Limitations of  descriptive research studies:

• They cannot establish cause and effect relationships.
• The reliability and validity of survey responses can be compromised if respondents are not truthful or tend to provide socially desirable answers.
• The choice and wording of questions on a questionnaire may influence the descriptive findings.

Correlational Research

Correlational research, a non-experimental method, delves into the dynamics between two variables, focusing on the strength and direction of their relationship without manipulating any factors, which is pivotal in understanding associations rather than causality.

Researchers may choose correlational research in the following situations

• When manipulating the independent variable is impractical, impossible, or unethical
• When exploring non-causal relationships between variables
• When testing new measurement tools

In correlational research, the correlation coefficient is measured, which can range from -1 to +1, indicating the relationship's direction and strength. A comprehensive meta-analysis can further elucidate these types of correlations.

• Positive correlation: Both variables change in the same direction
• Negative correlation: Variables change in opposite directions
• Zero correlation: No relationship exists between the variables

Data collection methods for correlational research include

• Naturalistic observation
• Archival research or secondary data

Analytical research methods, such as correlation or regression analyses, are employed to analyze correlational data, with the former yielding a coefficient that clarifies the relationship's intensity and direction, and the latter forecasting the impact of variable changes.

Experimental Research

Experimental research, a methodical scientific approach, manipulates variables to observe their effects and is indispensable for establishing cause-and-effect relationships and making informed decisions in the face of inadequate data.

• Pre-experimental research design : Includes One-shot Case Study Research Design, One-group Pretest-posttest Research Design, and Static-group Comparison.
• True experimental research design Statistical analysis, a cornerstone in testing hypotheses, is pivotal in research for its accuracy in proving or disproving a hypothesis. It's uniquely capable of establishing a cause-effect relationship within a group, making it an indispensable tool for researchers.
• Quasi-experimental design : Similar to an experimental design but assigns participants to groups non-randomly.

Experimental research is essential for various fields, such as:

• Developing new drugs and medical treatments
• Understanding human behavior in psychology
• Improving educational outcomes
• Identifying opportunities for businesses and organizations

To conduct experimental research effectively, researchers must consider three key factors:

• A Control Group and an Experimental Group
• A variable that can be manipulated by the researcher
• Random distribution of participants

Experimental research, whether conducted in laboratory settings with high control variables and internal validity or in field settings that boast both internal and external validity, presents a spectrum of advantages and challenges. Researchers must navigate potential threats to internal validity, including history, maturation, testing, instrumentation, mortality, and regression threats.

Mixed Methods Research

Mixed methods research, an approach that synergizes the rigor of quantitative and qualitative research methods, capitalizes on the strengths of each to provide a comprehensive analysis. This integration, which can occur during data collection, analysis, or presentation of results, is a hallmark of mixed methods research designs.

• Convergent design
• Explanatory sequential design
• Exploratory sequential design
• Embedded design

The practice of triangulation in mixed methods research enhances the integration of quantitative and qualitative data, offering multiple perspectives and a more comprehensive understanding. It also allows for a deeper explanation of statistical results, as exemplified by the EQUALITY study's exploratory sequential design for patient-centered data collection.

In mixed methods research, the intricate research design and methodology combine qualitative and quantitative data collection and analysis. This purposeful mixing of methods and data integration at strategic stages of the research process can reveal relationships between complex layers of research questions, although it demands significant resources and specialized training.

• Explanatory
• Exploratory
• Nested (embedded) designs

Mixed methods research, characterized by its diverse research design and methods, integrates quantitative and qualitative approaches within a single study. Grounded in positivism and interpretivism, it provides a multifaceted understanding of research topics, despite the challenges of mastering both methodologies and collaborating with multidisciplinary teams.

In sum, a thorough grasp of the various research methodologies is crucial for conducting robust research and critically assessing others' findings. From qualitative to quantitative, descriptive, correlational, experimental, and mixed methods research, each approach offers distinct strengths and limitations, guiding researchers to the most suitable methods for effective data collection and analysis.

As we navigate the vast landscape of information available, understanding what are research methods empowers us to make informed decisions, draw meaningful conclusions, and contribute to the advancement of knowledge across various fields. Embracing the diversity of research methods, whether you're a student, researcher, or simply curious, will enhance your critical thinking skills and enable you to uncover valuable insights that shape our understanding of the world.

What are the seven most commonly used research methods? The seven most commonly used research methods are:

• Observation / Participant Observation
• Focus Groups
• Experiments
• Secondary Data Analysis / Archival Study
• Mixed Methods (a combination of some of the above)

What does comprehensive research methodology entail?

Comprehensive research methodology involves conducting a thorough and exhaustive investigation on a specific topic, subject, or issue. This approach is characterized by the meticulous collection, analysis, and evaluation of a wide array of information, data, and sources, with the objective of achieving a deep and comprehensive understanding of the subject matter.

What are the three primary methods to investigate a specific research question?

To investigate a specific research question, you can use:

• Quantitative methods for measuring something or testing a hypothesis.
• Qualitative methods for exploring ideas, thoughts, and meanings.
• Secondary data analysis for examining a large volume of readily-available data.

What does exploration mean in the context of research methodology?

Exploration in research methodology signifies a research approach that aims to delve into questions that have not been extensively explored before. Exploratory research, often qualitative and primary in nature, is focused on uncovering new insights and understanding. Nonetheless, it can also adopt a quantitative stance, particularly when it involves analyzing a large sample size, to further the scope of exploratory research.

Sign up for more like this.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Can J Hosp Pharm
• v.68(3); May-Jun 2015

Qualitative Research: Data Collection, Analysis, and Management

Introduction.

In an earlier paper, 1 we presented an introduction to using qualitative research methods in pharmacy practice. In this article, we review some principles of the collection, analysis, and management of qualitative data to help pharmacists interested in doing research in their practice to continue their learning in this area. Qualitative research can help researchers to access the thoughts and feelings of research participants, which can enable development of an understanding of the meaning that people ascribe to their experiences. Whereas quantitative research methods can be used to determine how many people undertake particular behaviours, qualitative methods can help researchers to understand how and why such behaviours take place. Within the context of pharmacy practice research, qualitative approaches have been used to examine a diverse array of topics, including the perceptions of key stakeholders regarding prescribing by pharmacists and the postgraduation employment experiences of young pharmacists (see “Further Reading” section at the end of this article).

In the previous paper, 1 we outlined 3 commonly used methodologies: ethnography 2 , grounded theory 3 , and phenomenology. 4 Briefly, ethnography involves researchers using direct observation to study participants in their “real life” environment, sometimes over extended periods. Grounded theory and its later modified versions (e.g., Strauss and Corbin 5 ) use face-to-face interviews and interactions such as focus groups to explore a particular research phenomenon and may help in clarifying a less-well-understood problem, situation, or context. Phenomenology shares some features with grounded theory (such as an exploration of participants’ behaviour) and uses similar techniques to collect data, but it focuses on understanding how human beings experience their world. It gives researchers the opportunity to put themselves in another person’s shoes and to understand the subjective experiences of participants. 6 Some researchers use qualitative methodologies but adopt a different standpoint, and an example of this appears in the work of Thurston and others, 7 discussed later in this paper.

Qualitative work requires reflection on the part of researchers, both before and during the research process, as a way of providing context and understanding for readers. When being reflexive, researchers should not try to simply ignore or avoid their own biases (as this would likely be impossible); instead, reflexivity requires researchers to reflect upon and clearly articulate their position and subjectivities (world view, perspectives, biases), so that readers can better understand the filters through which questions were asked, data were gathered and analyzed, and findings were reported. From this perspective, bias and subjectivity are not inherently negative but they are unavoidable; as a result, it is best that they be articulated up-front in a manner that is clear and coherent for readers.

THE PARTICIPANT’S VIEWPOINT

What qualitative study seeks to convey is why people have thoughts and feelings that might affect the way they behave. Such study may occur in any number of contexts, but here, we focus on pharmacy practice and the way people behave with regard to medicines use (e.g., to understand patients’ reasons for nonadherence with medication therapy or to explore physicians’ resistance to pharmacists’ clinical suggestions). As we suggested in our earlier article, 1 an important point about qualitative research is that there is no attempt to generalize the findings to a wider population. Qualitative research is used to gain insights into people’s feelings and thoughts, which may provide the basis for a future stand-alone qualitative study or may help researchers to map out survey instruments for use in a quantitative study. It is also possible to use different types of research in the same study, an approach known as “mixed methods” research, and further reading on this topic may be found at the end of this paper.

The role of the researcher in qualitative research is to attempt to access the thoughts and feelings of study participants. This is not an easy task, as it involves asking people to talk about things that may be very personal to them. Sometimes the experiences being explored are fresh in the participant’s mind, whereas on other occasions reliving past experiences may be difficult. However the data are being collected, a primary responsibility of the researcher is to safeguard participants and their data. Mechanisms for such safeguarding must be clearly articulated to participants and must be approved by a relevant research ethics review board before the research begins. Researchers and practitioners new to qualitative research should seek advice from an experienced qualitative researcher before embarking on their project.

DATA COLLECTION

Whatever philosophical standpoint the researcher is taking and whatever the data collection method (e.g., focus group, one-to-one interviews), the process will involve the generation of large amounts of data. In addition to the variety of study methodologies available, there are also different ways of making a record of what is said and done during an interview or focus group, such as taking handwritten notes or video-recording. If the researcher is audio- or video-recording data collection, then the recordings must be transcribed verbatim before data analysis can begin. As a rough guide, it can take an experienced researcher/transcriber 8 hours to transcribe one 45-minute audio-recorded interview, a process than will generate 20–30 pages of written dialogue.

Many researchers will also maintain a folder of “field notes” to complement audio-taped interviews. Field notes allow the researcher to maintain and comment upon impressions, environmental contexts, behaviours, and nonverbal cues that may not be adequately captured through the audio-recording; they are typically handwritten in a small notebook at the same time the interview takes place. Field notes can provide important context to the interpretation of audio-taped data and can help remind the researcher of situational factors that may be important during data analysis. Such notes need not be formal, but they should be maintained and secured in a similar manner to audio tapes and transcripts, as they contain sensitive information and are relevant to the research. For more information about collecting qualitative data, please see the “Further Reading” section at the end of this paper.

DATA ANALYSIS AND MANAGEMENT

If, as suggested earlier, doing qualitative research is about putting oneself in another person’s shoes and seeing the world from that person’s perspective, the most important part of data analysis and management is to be true to the participants. It is their voices that the researcher is trying to hear, so that they can be interpreted and reported on for others to read and learn from. To illustrate this point, consider the anonymized transcript excerpt presented in Appendix 1 , which is taken from a research interview conducted by one of the authors (J.S.). We refer to this excerpt throughout the remainder of this paper to illustrate how data can be managed, analyzed, and presented.

Interpretation of Data

Interpretation of the data will depend on the theoretical standpoint taken by researchers. For example, the title of the research report by Thurston and others, 7 “Discordant indigenous and provider frames explain challenges in improving access to arthritis care: a qualitative study using constructivist grounded theory,” indicates at least 2 theoretical standpoints. The first is the culture of the indigenous population of Canada and the place of this population in society, and the second is the social constructivist theory used in the constructivist grounded theory method. With regard to the first standpoint, it can be surmised that, to have decided to conduct the research, the researchers must have felt that there was anecdotal evidence of differences in access to arthritis care for patients from indigenous and non-indigenous backgrounds. With regard to the second standpoint, it can be surmised that the researchers used social constructivist theory because it assumes that behaviour is socially constructed; in other words, people do things because of the expectations of those in their personal world or in the wider society in which they live. (Please see the “Further Reading” section for resources providing more information about social constructivist theory and reflexivity.) Thus, these 2 standpoints (and there may have been others relevant to the research of Thurston and others 7 ) will have affected the way in which these researchers interpreted the experiences of the indigenous population participants and those providing their care. Another standpoint is feminist standpoint theory which, among other things, focuses on marginalized groups in society. Such theories are helpful to researchers, as they enable us to think about things from a different perspective. Being aware of the standpoints you are taking in your own research is one of the foundations of qualitative work. Without such awareness, it is easy to slip into interpreting other people’s narratives from your own viewpoint, rather than that of the participants.

To analyze the example in Appendix 1 , we will adopt a phenomenological approach because we want to understand how the participant experienced the illness and we want to try to see the experience from that person’s perspective. It is important for the researcher to reflect upon and articulate his or her starting point for such analysis; for example, in the example, the coder could reflect upon her own experience as a female of a majority ethnocultural group who has lived within middle class and upper middle class settings. This personal history therefore forms the filter through which the data will be examined. This filter does not diminish the quality or significance of the analysis, since every researcher has his or her own filters; however, by explicitly stating and acknowledging what these filters are, the researcher makes it easer for readers to contextualize the work.

Transcribing and Checking

For the purposes of this paper it is assumed that interviews or focus groups have been audio-recorded. As mentioned above, transcribing is an arduous process, even for the most experienced transcribers, but it must be done to convert the spoken word to the written word to facilitate analysis. For anyone new to conducting qualitative research, it is beneficial to transcribe at least one interview and one focus group. It is only by doing this that researchers realize how difficult the task is, and this realization affects their expectations when asking others to transcribe. If the research project has sufficient funding, then a professional transcriber can be hired to do the work. If this is the case, then it is a good idea to sit down with the transcriber, if possible, and talk through the research and what the participants were talking about. This background knowledge for the transcriber is especially important in research in which people are using jargon or medical terms (as in pharmacy practice). Involving your transcriber in this way makes the work both easier and more rewarding, as he or she will feel part of the team. Transcription editing software is also available, but it is expensive. For example, ELAN (more formally known as EUDICO Linguistic Annotator, developed at the Technical University of Berlin) 8 is a tool that can help keep data organized by linking media and data files (particularly valuable if, for example, video-taping of interviews is complemented by transcriptions). It can also be helpful in searching complex data sets. Products such as ELAN do not actually automatically transcribe interviews or complete analyses, and they do require some time and effort to learn; nonetheless, for some research applications, it may be a valuable to consider such software tools.

All audio recordings should be transcribed verbatim, regardless of how intelligible the transcript may be when it is read back. Lines of text should be numbered. Once the transcription is complete, the researcher should read it while listening to the recording and do the following: correct any spelling or other errors; anonymize the transcript so that the participant cannot be identified from anything that is said (e.g., names, places, significant events); insert notations for pauses, laughter, looks of discomfort; insert any punctuation, such as commas and full stops (periods) (see Appendix 1 for examples of inserted punctuation), and include any other contextual information that might have affected the participant (e.g., temperature or comfort of the room).

Dealing with the transcription of a focus group is slightly more difficult, as multiple voices are involved. One way of transcribing such data is to “tag” each voice (e.g., Voice A, Voice B). In addition, the focus group will usually have 2 facilitators, whose respective roles will help in making sense of the data. While one facilitator guides participants through the topic, the other can make notes about context and group dynamics. More information about group dynamics and focus groups can be found in resources listed in the “Further Reading” section.

Reading between the Lines

During the process outlined above, the researcher can begin to get a feel for the participant’s experience of the phenomenon in question and can start to think about things that could be pursued in subsequent interviews or focus groups (if appropriate). In this way, one participant’s narrative informs the next, and the researcher can continue to interview until nothing new is being heard or, as it says in the text books, “saturation is reached”. While continuing with the processes of coding and theming (described in the next 2 sections), it is important to consider not just what the person is saying but also what they are not saying. For example, is a lengthy pause an indication that the participant is finding the subject difficult, or is the person simply deciding what to say? The aim of the whole process from data collection to presentation is to tell the participants’ stories using exemplars from their own narratives, thus grounding the research findings in the participants’ lived experiences.

Smith 9 suggested a qualitative research method known as interpretative phenomenological analysis, which has 2 basic tenets: first, that it is rooted in phenomenology, attempting to understand the meaning that individuals ascribe to their lived experiences, and second, that the researcher must attempt to interpret this meaning in the context of the research. That the researcher has some knowledge and expertise in the subject of the research means that he or she can have considerable scope in interpreting the participant’s experiences. Larkin and others 10 discussed the importance of not just providing a description of what participants say. Rather, interpretative phenomenological analysis is about getting underneath what a person is saying to try to truly understand the world from his or her perspective.

Once all of the research interviews have been transcribed and checked, it is time to begin coding. Field notes compiled during an interview can be a useful complementary source of information to facilitate this process, as the gap in time between an interview, transcribing, and coding can result in memory bias regarding nonverbal or environmental context issues that may affect interpretation of data.

Coding refers to the identification of topics, issues, similarities, and differences that are revealed through the participants’ narratives and interpreted by the researcher. This process enables the researcher to begin to understand the world from each participant’s perspective. Coding can be done by hand on a hard copy of the transcript, by making notes in the margin or by highlighting and naming sections of text. More commonly, researchers use qualitative research software (e.g., NVivo, QSR International Pty Ltd; www.qsrinternational.com/products_nvivo.aspx ) to help manage their transcriptions. It is advised that researchers undertake a formal course in the use of such software or seek supervision from a researcher experienced in these tools.

Returning to Appendix 1 and reading from lines 8–11, a code for this section might be “diagnosis of mental health condition”, but this would just be a description of what the participant is talking about at that point. If we read a little more deeply, we can ask ourselves how the participant might have come to feel that the doctor assumed he or she was aware of the diagnosis or indeed that they had only just been told the diagnosis. There are a number of pauses in the narrative that might suggest the participant is finding it difficult to recall that experience. Later in the text, the participant says “nobody asked me any questions about my life” (line 19). This could be coded simply as “health care professionals’ consultation skills”, but that would not reflect how the participant must have felt never to be asked anything about his or her personal life, about the participant as a human being. At the end of this excerpt, the participant just trails off, recalling that no-one showed any interest, which makes for very moving reading. For practitioners in pharmacy, it might also be pertinent to explore the participant’s experience of akathisia and why this was left untreated for 20 years.

One of the questions that arises about qualitative research relates to the reliability of the interpretation and representation of the participants’ narratives. There are no statistical tests that can be used to check reliability and validity as there are in quantitative research. However, work by Lincoln and Guba 11 suggests that there are other ways to “establish confidence in the ‘truth’ of the findings” (p. 218). They call this confidence “trustworthiness” and suggest that there are 4 criteria of trustworthiness: credibility (confidence in the “truth” of the findings), transferability (showing that the findings have applicability in other contexts), dependability (showing that the findings are consistent and could be repeated), and confirmability (the extent to which the findings of a study are shaped by the respondents and not researcher bias, motivation, or interest).

One way of establishing the “credibility” of the coding is to ask another researcher to code the same transcript and then to discuss any similarities and differences in the 2 resulting sets of codes. This simple act can result in revisions to the codes and can help to clarify and confirm the research findings.

Theming refers to the drawing together of codes from one or more transcripts to present the findings of qualitative research in a coherent and meaningful way. For example, there may be examples across participants’ narratives of the way in which they were treated in hospital, such as “not being listened to” or “lack of interest in personal experiences” (see Appendix 1 ). These may be drawn together as a theme running through the narratives that could be named “the patient’s experience of hospital care”. The importance of going through this process is that at its conclusion, it will be possible to present the data from the interviews using quotations from the individual transcripts to illustrate the source of the researchers’ interpretations. Thus, when the findings are organized for presentation, each theme can become the heading of a section in the report or presentation. Underneath each theme will be the codes, examples from the transcripts, and the researcher’s own interpretation of what the themes mean. Implications for real life (e.g., the treatment of people with chronic mental health problems) should also be given.

DATA SYNTHESIS

In this final section of this paper, we describe some ways of drawing together or “synthesizing” research findings to represent, as faithfully as possible, the meaning that participants ascribe to their life experiences. This synthesis is the aim of the final stage of qualitative research. For most readers, the synthesis of data presented by the researcher is of crucial significance—this is usually where “the story” of the participants can be distilled, summarized, and told in a manner that is both respectful to those participants and meaningful to readers. There are a number of ways in which researchers can synthesize and present their findings, but any conclusions drawn by the researchers must be supported by direct quotations from the participants. In this way, it is made clear to the reader that the themes under discussion have emerged from the participants’ interviews and not the mind of the researcher. The work of Latif and others 12 gives an example of how qualitative research findings might be presented.

Planning and Writing the Report

As has been suggested above, if researchers code and theme their material appropriately, they will naturally find the headings for sections of their report. Qualitative researchers tend to report “findings” rather than “results”, as the latter term typically implies that the data have come from a quantitative source. The final presentation of the research will usually be in the form of a report or a paper and so should follow accepted academic guidelines. In particular, the article should begin with an introduction, including a literature review and rationale for the research. There should be a section on the chosen methodology and a brief discussion about why qualitative methodology was most appropriate for the study question and why one particular methodology (e.g., interpretative phenomenological analysis rather than grounded theory) was selected to guide the research. The method itself should then be described, including ethics approval, choice of participants, mode of recruitment, and method of data collection (e.g., semistructured interviews or focus groups), followed by the research findings, which will be the main body of the report or paper. The findings should be written as if a story is being told; as such, it is not necessary to have a lengthy discussion section at the end. This is because much of the discussion will take place around the participants’ quotes, such that all that is needed to close the report or paper is a summary, limitations of the research, and the implications that the research has for practice. As stated earlier, it is not the intention of qualitative research to allow the findings to be generalized, and therefore this is not, in itself, a limitation.

Planning out the way that findings are to be presented is helpful. It is useful to insert the headings of the sections (the themes) and then make a note of the codes that exemplify the thoughts and feelings of your participants. It is generally advisable to put in the quotations that you want to use for each theme, using each quotation only once. After all this is done, the telling of the story can begin as you give your voice to the experiences of the participants, writing around their quotations. Do not be afraid to draw assumptions from the participants’ narratives, as this is necessary to give an in-depth account of the phenomena in question. Discuss these assumptions, drawing on your participants’ words to support you as you move from one code to another and from one theme to the next. Finally, as appropriate, it is possible to include examples from literature or policy documents that add support for your findings. As an exercise, you may wish to code and theme the sample excerpt in Appendix 1 and tell the participant’s story in your own way. Further reading about “doing” qualitative research can be found at the end of this paper.

CONCLUSIONS

Qualitative research can help researchers to access the thoughts and feelings of research participants, which can enable development of an understanding of the meaning that people ascribe to their experiences. It can be used in pharmacy practice research to explore how patients feel about their health and their treatment. Qualitative research has been used by pharmacists to explore a variety of questions and problems (see the “Further Reading” section for examples). An understanding of these issues can help pharmacists and other health care professionals to tailor health care to match the individual needs of patients and to develop a concordant relationship. Doing qualitative research is not easy and may require a complete rethink of how research is conducted, particularly for researchers who are more familiar with quantitative approaches. There are many ways of conducting qualitative research, and this paper has covered some of the practical issues regarding data collection, analysis, and management. Further reading around the subject will be essential to truly understand this method of accessing peoples’ thoughts and feelings to enable researchers to tell participants’ stories.

Appendix 1. Excerpt from a sample transcript

The participant (age late 50s) had suffered from a chronic mental health illness for 30 years. The participant had become a “revolving door patient,” someone who is frequently in and out of hospital. As the participant talked about past experiences, the researcher asked:

• What was treatment like 30 years ago?
• Umm—well it was pretty much they could do what they wanted with you because I was put into the er, the er kind of system er, I was just on
• endless section threes.
• Really…
• But what I didn’t realize until later was that if you haven’t actually posed a threat to someone or yourself they can’t really do that but I didn’t know
• that. So wh-when I first went into hospital they put me on the forensic ward ’cause they said, “We don’t think you’ll stay here we think you’ll just
• run-run away.” So they put me then onto the acute admissions ward and – er – I can remember one of the first things I recall when I got onto that
• ward was sitting down with a er a Dr XXX. He had a book this thick [gestures] and on each page it was like three questions and he went through
• all these questions and I answered all these questions. So we’re there for I don’t maybe two hours doing all that and he asked me he said “well
• when did somebody tell you then that you have schizophrenia” I said “well nobody’s told me that” so he seemed very surprised but nobody had
• actually [pause] whe-when I first went up there under police escort erm the senior kind of consultants people I’d been to where I was staying and
• ermm so er [pause] I . . . the, I can remember the very first night that I was there and given this injection in this muscle here [gestures] and just
• having dreadful side effects the next day I woke up [pause]
• . . . and I suffered that akathesia I swear to you, every minute of every day for about 20 years.
• Oh how awful.
• And that side of it just makes life impossible so the care on the wards [pause] umm I don’t know it’s kind of, it’s kind of hard to put into words
• [pause]. Because I’m not saying they were sort of like not friendly or interested but then nobody ever seemed to want to talk about your life [pause]
• nobody asked me any questions about my life. The only questions that came into was they asked me if I’d be a volunteer for these student exams
• and things and I said “yeah” so all the questions were like “oh what jobs have you done,” er about your relationships and things and er but
• nobody actually sat down and had a talk and showed some interest in you as a person you were just there basically [pause] um labelled and you
• know there was there was [pause] but umm [pause] yeah . . .

This article is the 10th in the CJHP Research Primer Series, an initiative of the CJHP Editorial Board and the CSHP Research Committee. The planned 2-year series is intended to appeal to relatively inexperienced researchers, with the goal of building research capacity among practising pharmacists. The articles, presenting simple but rigorous guidance to encourage and support novice researchers, are being solicited from authors with appropriate expertise.

Previous articles in this series:

Bond CM. The research jigsaw: how to get started. Can J Hosp Pharm . 2014;67(1):28–30.

Tully MP. Research: articulating questions, generating hypotheses, and choosing study designs. Can J Hosp Pharm . 2014;67(1):31–4.

Loewen P. Ethical issues in pharmacy practice research: an introductory guide. Can J Hosp Pharm. 2014;67(2):133–7.

Tsuyuki RT. Designing pharmacy practice research trials. Can J Hosp Pharm . 2014;67(3):226–9.

Bresee LC. An introduction to developing surveys for pharmacy practice research. Can J Hosp Pharm . 2014;67(4):286–91.

Gamble JM. An introduction to the fundamentals of cohort and case–control studies. Can J Hosp Pharm . 2014;67(5):366–72.

Austin Z, Sutton J. Qualitative research: getting started. C an J Hosp Pharm . 2014;67(6):436–40.

Houle S. An introduction to the fundamentals of randomized controlled trials in pharmacy research. Can J Hosp Pharm . 2014; 68(1):28–32.

Charrois TL. Systematic reviews: What do you need to know to get started? Can J Hosp Pharm . 2014;68(2):144–8.

Competing interests: None declared.

Further Reading

Examples of qualitative research in pharmacy practice.

• Farrell B, Pottie K, Woodend K, Yao V, Dolovich L, Kennie N, et al. Shifts in expectations: evaluating physicians’ perceptions as pharmacists integrated into family practice. J Interprof Care. 2010; 24 (1):80–9. [ PubMed ] [ Google Scholar ]
• Gregory P, Austin Z. Postgraduation employment experiences of new pharmacists in Ontario in 2012–2013. Can Pharm J. 2014; 147 (5):290–9. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Marks PZ, Jennnings B, Farrell B, Kennie-Kaulbach N, Jorgenson D, Pearson-Sharpe J, et al. “I gained a skill and a change in attitude”: a case study describing how an online continuing professional education course for pharmacists supported achievement of its transfer to practice outcomes. Can J Univ Contin Educ. 2014; 40 (2):1–18. [ Google Scholar ]
• Nair KM, Dolovich L, Brazil K, Raina P. It’s all about relationships: a qualitative study of health researchers’ perspectives on interdisciplinary research. BMC Health Serv Res. 2008; 8 :110. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Pojskic N, MacKeigan L, Boon H, Austin Z. Initial perceptions of key stakeholders in Ontario regarding independent prescriptive authority for pharmacists. Res Soc Adm Pharm. 2014; 10 (2):341–54. [ PubMed ] [ Google Scholar ]

Qualitative Research in General

• Breakwell GM, Hammond S, Fife-Schaw C. Research methods in psychology. Thousand Oaks (CA): Sage Publications; 1995. [ Google Scholar ]
• Given LM. 100 questions (and answers) about qualitative research. Thousand Oaks (CA): Sage Publications; 2015. [ Google Scholar ]
• Miles B, Huberman AM. Qualitative data analysis. Thousand Oaks (CA): Sage Publications; 2009. [ Google Scholar ]
• Patton M. Qualitative research and evaluation methods. Thousand Oaks (CA): Sage Publications; 2002. [ Google Scholar ]
• Willig C. Introducing qualitative research in psychology. Buckingham (UK): Open University Press; 2001. [ Google Scholar ]

Group Dynamics in Focus Groups

• Farnsworth J, Boon B. Analysing group dynamics within the focus group. Qual Res. 2010; 10 (5):605–24. [ Google Scholar ]

Social Constructivism

• Social constructivism. Berkeley (CA): University of California, Berkeley, Berkeley Graduate Division, Graduate Student Instruction Teaching & Resource Center; [cited 2015 June 4]. Available from: http://gsi.berkeley.edu/gsi-guide-contents/learning-theory-research/social-constructivism/ [ Google Scholar ]

Mixed Methods

• Creswell J. Research design: qualitative, quantitative, and mixed methods approaches. Thousand Oaks (CA): Sage Publications; 2009. [ Google Scholar ]

Collecting Qualitative Data

• Arksey H, Knight P. Interviewing for social scientists: an introductory resource with examples. Thousand Oaks (CA): Sage Publications; 1999. [ Google Scholar ]
• Guest G, Namey EE, Mitchel ML. Collecting qualitative data: a field manual for applied research. Thousand Oaks (CA): Sage Publications; 2013. [ Google Scholar ]

Constructivist Grounded Theory

• Charmaz K. Grounded theory: objectivist and constructivist methods. In: Denzin N, Lincoln Y, editors. Handbook of qualitative research. 2nd ed. Thousand Oaks (CA): Sage Publications; 2000. pp. 509–35. [ Google Scholar ]

• Chamberlain University Library
• Chamberlain Library Core

Finding Types of Research

• Evidence-Based Research

On This Guide

About this guide, understand evidence-based practice, identify research study types.

• Quantitative Studies
• Qualitative Studies
• Meta-Analysis
• Systematic Reviews
• Randomized Controlled Trials
• Observational Studies
• Literature Reviews
• Finding Research Tools This link opens in a new window

Throughout your schooling, you may need to find different types of evidence and research to support your course work. This guide provides a high-level overview of evidence-based practice as well as the different types of research and study designs. Each page of this guide offers an overview and search tips for finding articles that fit that study design.

Note! If you need help finding a specific type of study, visit the  Get Research Help guide  to contact the librarians.

What is Evidence-Based Practice?

One of the requirements for your coursework is to find articles that support evidence-based practice. But what exactly is evidence-based practice? Evidence-based practice is a method that uses relevant and current evidence to plan, implement and evaluate patient care. This definition is included in the video below, which explains all the steps of evidence-based practice in greater detail.

• Video - Evidence-based practice: What it is and what it is not. Medcom (Producer), & Cobb, D. (Director). (2017). Evidence-based practice: What it is and what it is not [Streaming Video]. United States of America: Producer. Retrieved from Alexander Street Press Nursing Education Collection

Quantitative and Qualitative Studies

Research is broken down into two different types: quantitative and qualitative. Quantitative studies are all about measurement. They will report statistics of things that can be physically measured like blood pressure, weight and oxygen saturation. Qualitative studies, on the other hand, are about people's experiences and how they feel about something. This type of information cannot be measured using statistics. Both of these types of studies report original research and are considered single studies. Watch the video below for more information.

Study Designs

Some research study types that you will encounter include:

• Case-Control Studies
• Cohort Studies
• Cross-Sectional Studies

Studies that Synthesize Other Studies

Sometimes, a research study will look at the results of many studies and look for trends and draw conclusions. These types of studies include:

• Meta Analyses

Tip! How do you determine the research article's study type or level of evidence? First, look at the article abstract. Most of the time the abstract will have a methodology section, which should tell you what type of study design the researchers are using. If it is not in the abstract, look for the methodology section of the article. It should tell you all about what type of study the researcher is doing and the steps they used to carry out the study.

Read the book below to learn how to read a clinical paper, including the types of study designs you will encounter.

• Search Website
• Library Tech Support
• Services for Colleagues

Chamberlain College of Nursing is owned and operated by Chamberlain University LLC. In certain states, Chamberlain operates as Chamberlain College of Nursing pending state authorization for Chamberlain University.

Wiz Research finds architecture risks that may compromise AI-as-a-Service providers and consequently risk customer data; works with Hugging Face on mitigations

Wiz researchers discovered architecture risks that may compromise AI-as-a-Service providers and put customer data at risk. Wiz and Hugging Face worked together to mitigate the issue.

The world has never seen a piece of technology adopted at the pace of AI. As more organizations worldwide adopt AI-as-a-Service (a.k.a. “AI cloud”) the industry must recognize the possible risks in this shared infrastructure that holds sensitive data and enforce mature regulation and security practices that are similar to those enforced on public cloud service providers.  

When we move fast, we break things. In recent months, Wiz Research partnered with AI-as-a-Service companies to uncover common security risks that may impact the industry and subsequently put users’ data and models at risk. In our State of AI in the Cloud report , we show that AI services are already present in more than 70% of cloud environments, showcasing how critical the impact of those findings are. 

In this blog we outline our joint work with Hugging Face, one of the best-known AI-as-a-Service providers. Hugging Face has undergone a meteoric rise and grown at an unprecedented rate to meet swelling demand. What we found not only presented an opportunity for Hugging Face to strengthen the platform’s security (which they did); it also carries broader takeaways that apply to many AI systems and AI as-a-service platforms. 

AI models require strong GPU to run, which is often outsourced to AI service providers. In Hugging Face, this service is called Hugging Face Inference API. For ease of understanding, this can be compared, at a high level, to consuming cloud infrastructure from AWS/GCP/Azure to run your applications and code. Wiz Research was able to compromise the service running the custom models by uploading our own malicious model and leveraging container escape techniques to break out from our tenant and compromise the entire service. This means Wiz research could gain cross-tenant access to other customers' models stored and run in Hugging Face.  

We believe those findings are not unique to Hugging Face and represent challenges of tenant separation that many AI-as-a-Service companies will face, considering the model in which they run customer code and handle large amounts of data while growing faster than any industry before. We in the security community should partner closely with those companies to ensure safe infrastructure and guardrails are put in place without hindering this rapid (and truly incredible) growth. 

We want to thank the Hugging Face team for their collaboration and partnership. They have published their own blog post in response to our research, detailing the events and outcomes from their perspective.  

About Hugging Face 

Hugging Face stands out as the de facto open and collaborative platform for AI builders with a mission to democratize good Machine Learning. It provides users with the necessary infrastructure to host, train, and collaborate on AI model development within their teams. In addition to these capabilities, Hugging Face also serves as one of the most popular hubs where users can explore and utilize AI models developed by the AI community, discover and employ datasets, and experiment with demos. As part of its mission, Hugging Face feels a responsibility to keep up to date with AI/ML risks .  

Being a pivotal player in the broader AI development ecosystem, Hugging Face has also become an attractive target for adversaries. If a malicious actor were to compromise Hugging Face's platform, they could potentially gain access to private AI models, datasets, and critical applications, leading to widespread damage and potential supply chain risk. 

What did we find?   

Malicious models represent a major risk to AI systems, especially for AI-as-a-service providers because potential attackers may leverage these models to perform cross-tenant attacks. The potential impact is devastating, as attackers may be able to access the millions of private AI models and apps stored within AI-as-a-service providers. Wiz found two critical risks present in Hugging Face’s environment that we could have taken advantage of:  

Shared Inference infrastructure takeover risk – AI Inference is the process of using an already-trained model to generate predictions for a given input. Our research found that inference infrastructure often runs untrusted, potentially malicious models that use the “pickle” format. A malicious pickle-serialized model could contain a remote code execution payload, potentially granting the attacker escalated privileges and cross-tenant access to other customers' models.  

Shared CI/CD takeover risk – compiling malicious AI applications also represents a major risk as attackers may attempt to take over the CI/CD pipeline itself and perform a supply chain attack. A malicious AI app could have done so after taking over the CI/CD cluster.  

Different types of AI/ML applications 

When thinking about security for AI/ML, it is important to distinguish between different types of applications and scopes. An average application that uses AI/ML would consist of the following components: 

Model : The AI models that are being used (i.e. LLaMA, Bert, Whisper, etc.). 

Application : The application code that passes inputs to the AI model and makes use of the predictions it creates. 

Inference Infrastructure : The infrastructure that allows execution of the AI model — being “on edge” (like Transformers.js ) or via API or Inference-as-a-Service (like Hugging Face’s Inference Endpoints ). 

Potential adversaries can choose to attack each of the above components via different methods. For instance, to attack the AI model specifically, attackers can use certain inputs that would cause the model to produce false predictions (like adversarial.js ). To attack the application that utilizes AI/ML, attackers can use an input that produces a prediction that is correct — but is being used unsafely within the application (for instance, producing a prediction that would cause an SQL injection to the database, since the application would consider the output prediction of the model to be a trusted input).  

Finally, it is also possible to attack the inference infrastructure by utilizing a specially crafted, pickle-serialized malicious model. It is very common to treat AI models as black-box and to utilize other publicly available AI models. Currently, there are very few tools that can be used to examine the integrity of a given model and verify that it is indeed not malicious (such as Pickle Scanning by Hugging Face) — so developers and engineers must be very careful deciding where to download the models from. Using an untrusted AI model could introduce integrity and security risks to your application and is equivalent to including untrusted code within your application. 

In this blog post, we will demonstrate how to gain access to Hugging Face’s infrastructure with a special handcrafted serialization exploit, and detail what can be done to minimize the risk. 

The AI security questions and findings 

The Wiz research team is highly focused on isolation vulnerabilities in cloud environments. When we saw the rise of AI-as-a-service companies, we were concerned about the potential implications of a malicious actor leveraging them to gain privileged cross-tenant access, since AI models in practice are actually code. By design, AI-as-a-service providers build a shared compute service for their customers, which triggers an immediate question: is the AI model running in an isolated environment? Is it isolated enough?  

In this research, we focused on three key offerings of the platform: 

Inference API – which allows the community to browse and experiment with available models on the hub, without having to install required dependencies locally. Instead, users can interact with and “preview” these models via a modal on the platform, which is powered by Inference API. 

Inference Endpoints – which is a fully managed offering by Hugging Face that lets users easily deploy AI models on dedicated infrastructure for production purposes (i.e. Inference-as-a-Service). 

Spaces – which offers a simple way to host AI/ML applications, for the purpose of showcasing AI models or working collaboratively on developing AI-powered applications. 

Researching Hugging Face Inference API and inference endpoints 

When we, as attackers, examined the Inference offerings of Hugging Face (both Inference API and Inference Endpoints), we realized that any user could upload their own model. Behind the scenes, Hugging Face will dedicate resources, with the dependencies and infrastructure required for users to be able to interact with it and obtain predictions. 

This raised an interesting question: could we, as users of the platform, upload a specially crafted model – one could call it malicious – that would let us execute arbitrary code in that interface? And if we did manage to execute code inside Inference API, what would we find there? 

Uploading a Malicious Model to the Hub 

Hugging Face’s platform supports various AI model formats. By performing a quick search on Hugging Face, we can see that two formats are more prominent than others: PyTorch ( Pickle ) and Safetensors . 

It is relatively well-known that Python’s Pickle format is unsafe, and that it is possible to achieve remote code execution upon deserialization of untrusted data when using this format. This is even mentioned in Pickle’s own documentation :  

Because Pickle is an unsafe format, Hugging Face performs some analysis ( Pickle Scanning and Malware Scanning ) on Pickle files uploaded to their platform, and even highlights those they deem to be dangerous. 

Hugging Face will still let the user infer the uploaded Pickle-based model on the platform’s infrastructure, even when deemed dangerous. Because the community still uses PyTorch pickle, Hugging Face needs to support it. 

As researchers, we wanted to find out what would happen if we uploaded a malicious Pickle-based model to Hugging Face and interacted with it using the Inference API feature. Would our malicious code be executed? Would it run in a sandboxed environment? Do our models share the same infrastructure as those of other Hugging Face users? (In other words, is Inference API a multi-tenant service?) 

Let’s find out. 

Remote code execution via specially crafted Pickle file 

Without going into too much detail, we can state that it is relatively straightforward to craft a PyTorch (Pickle) model that will execute arbitrary code upon loading. To achieve remote code execution, we simply cloned a legitimate model ( gpt2 ), which already includes all of the necessary files that instruct Hugging Face on how this model should be run (i.e. config.json). We then modified it in a way that would run our reverse-shell upon loading. Next, we uploaded our hand-crafted model to Hugging Face as a private model and attempted to interact with it using the Inference API feature — and voila, we got our reverse shell! 

For convenience, instead of invoking a reverse shell every time we needed to check something, we chose to craft a version of our malicious model that could function like a shell. By hooking a couple functions in Hugging Face's python code, which manages the model's inference result (following the Pickle-deserialization remote code execution stage), we achieved shell-like functionality. The results are the following: 

Amazon EKS privilege escalation via IMDS 

After executing code inside Hugging Face Inference API and receiving our reverse shell, we started exploring the environment where we were running. It was quickly noticeable to us that we were running inside a Pod in a Kubernetes cluster hosted on Amazon EKS.  

In the past year, we encountered Amazon EKS multiple times during our research into service provider security issues. In fact, we have encountered Amazon EKS enough times to prompt us to create a playbook outlining what to look for when we see an EKS cluster (some of these key takeaways are documented in the 2023 Kubernetes Security report ). 

Following our “playbook” of common EKS misconfigurations (or insecure defaults) and how to identify them, we noticed that we could query the node’s IMDS (169.254.169.254) from within the pod where we were running. Since we could query the node’s IMDS and obtain its identity, we could also obtain the role of a Node inside the EKS cluster by abusing the aws eks get-token command . This is a fairly common misconfiguration (/ insecure default) in Amazon EKS. It is popular enough that we have included this exact trick in our EKS Cluster Games CTF (Challenge #4) even prior to doing this research. A small caveat with this method is that, in order to generate a valid token for the Kubernetes cluster, we must supply the correct cluster name to the aws eks get-token command. We tried guessing the correct cluster name a couple of times with no luck (based on the name of our AWS role), but eventually noticed that our AWS role also had permissions to call DescribeInstances (a default configuration), which revealed the name of the cluster via a tag attached to nodes’ compute. 

Using the aws eks get-token command and the IAM identity from the IMDS, we generated a valid Kubernetes token with the role of a Node. 

Now that we have the role of a node inside the Amazon EKS cluster, we have more privileges, and we can use them to explore the environment even further.  

One of the things we did was to list information about the Pod where we were running via kubectl get pods/$(hostname) , and then view the secrets that are associated with our pod. We were able to prove that by obtaining secrets (using kubectl get secrets ) it was possible to perform lateral movement within the EKS cluster.  

Potential impact and mitigations 

The secrets we obtained could have had a significant impact on the platform if they were in the hands of a malicious actor. Secrets within shared environments may often lead to cross-tenant access and sensitive data leakage.  

To mitigate this issue, we recommend enabling IMDSv2 with Hop Limit to prevent pods from accessing the IMDS and obtaining the role of a node within the cluster. 

Researching Hugging Face Spaces 

As we mentioned, Spaces is a different service in Hugging Face that allows users to host their AI-powered application on Hugging Face’s infrastructure for the purpose of collaborative development and showcasing the application to the public. Conveniently, all Hugging Face requires from the user in order to run their application on the Hugging Face Spaces service is a Dockerfile. 

Remote Code Execution via a specially crafted Dockerfile  

We began our engagement by providing a Dockerfile that executes a malicious payload via the CMD instruction, which specifies what program to run once the docker container is started. After gaining code execution and exploring the environment for a while, we found it to be quite restrictive and isolated. Subsequently, we decided to use the RUN instruction instead of the CMD instruction, enabling us to execute code in the build process and potentially encounter a different environment. 

Network isolation issue – write access to centralized container registry 

After executing code in the building process of our image, we used the netstat command to examine network connections made from our machine. One connection was to an internal container registry to which our built layers were pushed. This makes sense. An image was built, and it should be stored somewhere — this is a perfect application for a container registry. However, this container registry did not serve only us; it also served more of Hugging Face’s customers. Due to insufficient scoping, it was possible to pull and push (thus overwrite) all the images that were available on that container registry. 

In the wrong hands, the ability to write to the internal container registry could have significant implications for the platform's integrity and lead to supply chain attacks on customers’ spaces. To mitigate such issues, we recommend enforcing authentication even for internal container registries and, in general, limiting access to them. 

This research demonstrates that utilizing untrusted AI models (especially Pickle-based ones) could result in serious security consequences. Furthermore, if you intend to let users utilize untrusted AI models in your environment, it is extremely important to ensure that they are running in a sandboxed environment — since you could unknowingly be giving them the ability to execute arbitrary code on your infrastructure. The pace of AI adoption is unprecedented and enables great innovation. However, organizations should ensure that they have visibility and governance of the entire AI stack being used and carefully analyze all risks, including usage of malicious models, exposure of training data, sensitive data in training, vulnerabilities in AI SDKs, exposure of AI services, and other toxic risk combinations that may exploited by attackers. 

This research also highlights the value of collaboration between security researchers and platform developers. Collaboration of this type aids in gaining a deeper understanding of the risks associated with the platform, and ultimately enhances its security posture . 

Hugging Face has recently implemented Wiz CSPM and vulnerability scanning to proactively identify and mitigate some of the toxic risk combinations found here. In addition, Hugging Face is also currently going through its annual penetration test to ensure identified items have been sufficiently mitigated. 

Stay in touch!

Hi there! We are Sagi Tzadik ( @sagitz_ ), Shir Tamari ( @shirtamari ), Nir Ohfeld ( @nirohfeld ), Ronen Shustin ( @ronenshh ) and Hillai Ben-Sasson ( @hillai ) from the Wiz Research Team ( @wiz_io ). We are a group of veteran white-hat hackers with a single goal: to make the cloud a safer place for everyone. We primarily focus on finding new attack vectors in the cloud and uncovering isolation issues in cloud vendors and service providers. We would love to hear from you! Feel free to contact us on X (Twitter) or via email: [email protected] .

Continue reading

Defense in depth: XZ Utils

We explore assessment, prevention, and detection strategies for protecting your organization from the XZ Utils vulnerability.

Wizards of security, casting spells on themselves for ultimate digital security

Wiz practices what it preaches. Let’s look at how the security team at Wiz uses the power of the Wiz platform to monitor all its cloud-based infrastructure and services.  

Top security talks from KubeCon Europe 2024

KubeCon Europe is the largest open source community conference in Europe with hundreds of talks, many of them about security. All the sessions are available online; in this blog, we’ll discuss our favorites.

Ready to see Wiz in action?

“Best User Experience I have ever seen, provides full visibility to cloud workloads.”

“Wiz provides a single pane of glass to see what is going on in our cloud environments.”

“We know that if Wiz identifies something as critical, it actually is.”

This paper is in the following e-collection/theme issue:

Published on 10.4.2024 in Vol 26 (2024)

Effectiveness of a Web-Based Individual Coping and Alcohol Intervention Program for Children of Parents With Alcohol Use Problems: Randomized Controlled Trial

Authors of this article:

Original Paper

• Håkan Wall 1 , PhD   ; 
• Helena Hansson 2 , PhD   ; 
• Ulla Zetterlind 3 , PhD   ; 
• Pia Kvillemo 1 , PhD   ; 
• Tobias H Elgán 1 , PhD  

1 Stockholm Prevents Alcohol and Drug Problems, Centre for Psychiatry Research, Department of Clinical Neuroscience, Karolinska Institutet, & Stockholm Health Care Services, Stockholm, Sweden

2 School of Social Work, Faculty of Social Sciences, Lund University, Lund, Sweden

3 Clinical Health Promotion Centre, Department of Health Sciences, Lund University, Lund, Sweden

Corresponding Author:

Tobias H Elgán, PhD

Stockholm Prevents Alcohol and Drug Problems, Centre for Psychiatry Research

Department of Clinical Neuroscience

Karolinska Institutet, & Stockholm Health Care Services

Norra Stationsgatan 69

Stockholm, 11364

Phone: 46 700011003

Email: [email protected]

Background: Children whose parents have alcohol use problems are at an increased risk of several negative consequences, such as poor school performance, an earlier onset of substance use, and poor mental health. Many would benefit from support programs, but the figures reveal that only a small proportion is reached by existing support. Digital interventions can provide readily accessible support and potentially reach a large number of children. Research on digital interventions aimed at this target group is scarce. We have developed a novel digital therapist-assisted self-management intervention targeting adolescents whose parents had alcohol use problems. This program aims to strengthen coping behaviors, improve mental health, and decrease alcohol consumption in adolescents.

Objective: This study aims to examine the effectiveness of a novel web-based therapist-assisted self-management intervention for adolescents whose parents have alcohol use problems.

Methods: Participants were recruited on the internet from social media and websites containing health-related information about adolescents. Possible participants were screened using the short version of the Children of Alcoholics Screening Test-6. Eligible participants were randomly allocated to either the intervention group (n=101) or the waitlist control group (n=103), and they were unblinded to the condition. The assessments, all self-assessed, consisted of a baseline and 2 follow-ups after 2 and 6 months. The primary outcome was the Coping With Parents Abuse Questionnaire (CPAQ), and secondary outcomes were the Center for Epidemiological Studies Depression Scale, Alcohol Use Disorders Identification Test (AUDIT-C), and Ladder of Life (LoL).

Results: For the primary outcome, CPAQ, a small but inconclusive treatment effect was observed (Cohen d =–0.05 at both follow-up time points). The intervention group scored 38% and 46% lower than the control group on the continuous part of the AUDIT-C at the 2- and 6-month follow-up, respectively. All other between-group comparisons were inconclusive at either follow-up time point. Adherence was low, as only 24% (24/101) of the participants in the intervention group completed the intervention.

Conclusions: The findings were inconclusive for the primary outcome but demonstrate that a digital therapist-assisted self-management intervention may contribute to a reduction in alcohol consumption. These results highlight the potential for digital interventions to reach a vulnerable, hard-to-reach group of adolescents but underscore the need to develop more engaging support interventions to increase adherence.

Trial Registration: ISRCTN Registry ISRCTN41545712; https://www.isrctn.com/ISRCTN41545712?q=ISRCTN41545712

International Registered Report Identifier (IRRID): RR2-10.1186/1471-2458-12-35

Introduction

Children who grow up with parents who have substance use problems or disorders face extraordinary challenges. Approximately 20% of all children have parents with alcohol problems [ 1 - 5 ], while approximately 5% have parents with alcohol use disorders [ 4 , 6 , 7 ]. Children growing up with parental substance abuse are at an increased risk of several negative outcomes, such as psychiatric morbidity [ 8 - 12 ]; poor intellectual, cognitive, and academic achievement [ 13 - 15 ]; domestic physical abuse [ 16 ]; and early drinking onset and the development of substance use problems [ 9 , 17 , 18 ]. Thus, children exposed to parental substance abuse comprise a target group for selective interventions and prevention strategies [ 19 - 22 ].

In Sweden, municipalities account for most of the support offered to these children. An annual survey by the junior association of the Swedish branch of Movendi International (ie, an international temperance movement) reported that 97% of all municipalities provided support resources [ 23 ]. However, estimates from the same survey showed that approximately 2% of the children in the target group received support. Hence, an overwhelming majority never receives support, mainly because of difficulties in identifying and attracting them to intervention programs [ 22 , 24 ].

The internet has become an appealing way to reach and support a large number of people [ 25 , 26 ]. Web-based interventions seem particularly attractive to adolescents, as they generally use digital technology and social media. Furthermore, research has shown that adolescents regard the internet as inviting because it is a readily accessible, anonymous way of seeking help [ 27 ]. Web-based interventions can reduce the stigma associated with face-to-face consultations in health care settings [ 28 ], and young people appreciate the flexibility of completing web-based sessions to fit their own schedules [ 29 ]. The positive effects of web-based interventions have been detected across a broad range of conditions. A recent review by Hedman-Lagerlöf et al [ 30 ] concluded that therapist-supported internet-based cognitive behavioral therapy for adults yielded similar effects as face-to-face therapy. To date, most web-based interventions have been designed for adults. Although the number of web-based interventions targeting children or adolescents is increasing [ 25 , 31 - 33 ], the number of digital interventions aimed at children of substance-abusing parents is still scarce [ 22 , 34 - 38 ]. Those described in the literature, however, all have in common that they are quite extensive, with a duration over several weeks, and a brief digital intervention could complement these more extended interventions. For instance, our research group initiated a study on a web-based group chat for 15- to 25-year-old individuals who have parents with mental illness or substance use problems [ 35 ]. The duration of the program is 8 weeks, and it is a translated version of a program from the Netherlands [ 34 ], which has been shown to have inconclusive treatment effects [ 39 ]. In Sweden, 2 other programs with inconclusive treatment effects have been tested that target significant others and their children [ 37 , 38 ]. Finally, a digital intervention developed in Australia for 18- to 25-year-old individuals with parents with mental illness or substance use disorder [ 36 ] was tested in a pilot study demonstrating positive findings [ 40 ].

To meet the need for a brief, web-based intervention that targets adolescents having parents with alcohol problems and build on the evidence base of digital interventions targeting this vulnerable group, we developed a novel internet-delivered therapist-assisted self-management intervention called “Alcohol and Coping.” Our program originated from a manual-based face-to-face intervention called the “Individual Coping and Alcohol Intervention Program” (ICAIP) [ 41 , 42 ]. Previous studies on both the ICAIP, which aimed at college students having parents with alcohol problems, and a coping skills intervention program, which aimed at spouses of partners with alcohol dependency [ 43 ], have demonstrated positive effects regarding decreased alcohol consumption and improved mental health and coping behaviors [ 41 - 44 ]. Furthermore, the results from these studies underscore the importance of improving coping skills [ 42 , 44 ]. Among college students, those who received a combination of coping skills and an alcohol intervention program had better long-term outcomes [ 42 ].

The aim of this study was to test the effectiveness of Alcohol and Coping among a sample of adolescents aged 15-19 years with at least 1 parent with alcohol use problems. We hypothesized that the intervention group would be superior to the control group in improving coping skills. Secondary research questions concerned the participants’ improvement in (1) depression, (2) alcohol consumption, and (3) quality of life.

This study was a parallel-group randomized controlled trial in which participants were randomized to either the intervention or waitlist control group in a 1:1 allocation ratio. The trial design is illustrated in Figure 1 .

Recruitment and Screening

The participants were recruited from August 2012 to December 2013 through advertisements on social media (Facebook). The advertisements targeted individuals aged 15-19 years with Facebook accounts. Participants were recruited on the internet through advertisements on websites containing health-related information about adolescents. The advertisements included the text, “Do your parents drink too much? Participate in a study.” The advertisement contained an invitation to perform a web-based, self-assessed screening procedure. In addition to questions about age and sex, participants were screened for having parents with alcohol problems using the short version of the Children of Alcoholics Screening Test-6 (CAST-6), developed from a 30-item original version [ 45 ]. The CAST-6 is a 6-item true-false measure designed to assess whether participants perceive their parents’ alcohol consumption to be problematic. The CAST-6 has demonstrated high internal consistency ( r =0.92-0.94), test-retest reliability ( r =0.94), and high validity as compared to the 30-item version ( r =0.93) using the recommended threshold score of 3 or higher [ 45 , 46 ]. We previously translated the CAST-6 into Swedish and validated the translated version among 1450 adolescents, showing good internal consistency (α=.88), excellent test-retest reliability (intraclass correlation coefficient=0.93), and loading into 1 latent factor [ 47 ]. Additional inclusion criteria included having access to a computer and the internet and being sufficiently fluent in Swedish. Participants were excluded from the study and were referred to appropriate care if there were indications of either suicidal or self-inflicted harmful behaviors. Individuals eligible for inclusion received further information about the study and were asked to provide consent to participate by providing an email address.

Data Collection and Measures

All assessments were administered through email invitations containing a hyperlink to the web-based self-reported assessments. Up to 3 reminders were sent through email at 5, 10, and 15 days after the first invitation. A baseline assessment (t 0 ) was collected before randomization, and follow-up assessments were conducted at 2 and 6 months (t 1 and t 2 , respectively) after the initial assessment.

Participants were asked for age, sex, whether they lived with a parent (mother and father, mother or father, mother or father and stepparent, or alternate between mother and father), where their parents were born (Sweden or a Nordic country excluding Sweden or outside of the Nordic countries), parental status (employed, student, on parental leave, or unemployed), and any previous or present participation in support activities for children having parents with alcohol use problems. The primary outcome was coping, measured using the Coping With Parents Abuse Questionnaire (CPAQ) based on the Coping Behavior Scale developed by Orford et al [ 48 ]. Secondary outcomes were the Center for Epidemiological Studies Depression Scale (CES-DC) [ 49 ], the 3-question Alcohol Use Disorders Identification Test (AUDIT-C) [ 50 ], and the Ladder of Life (LoL), which measures the overall quality of life by asking about the participants’ past, present, and future ratings of their overall life satisfaction [ 50 ]. CPAQ has been shown to be reliable [ 41 , 42 ]. For this study, this scale was factor-analyzed to reduce the number of questions from 37 to 20. The resulting scale measures 6 coping typologies (discord, emotion, control, relationship, avoidance, and taking specific action) using a 4-point Likert scale, with a threshold score above 50 points (out of 80) indicating dysfunctional coping behavior. The CES-DC measures depressive symptoms during the past week using a 4-point Likert scale, where a higher total score indicates more depressive symptoms [ 49 ]. A cutoff score of ≥16 indicates symptoms of moderate depression, while a score of ≥30 indicates symptoms of severe depression [ 51 , 52 ]. The scale measures 4 dimensions of depression: depressed mood, tiredness, inability to concentrate, and feelings of being outside and lonely, and has positively stated items [ 52 ]. Additionally, this scale is a general measure of childhood psychopathology [ 53 ] and has been demonstrated to be reliable and valid among Swedish adolescents [ 52 ]. Alcohol consumption was measured using a modified AUDIT-C, which assesses the frequency of drinking, quantity consumed on a typical occasion, and frequency of heavy episodic drinking (ie, binge drinking) [ 50 ] using a 30-day perspective (as opposed to the original 12-month perspective). These questions have previously been translated into Swedish [ 54 ], and a score of ≥4 and ≥5 points for women and men, respectively, was used as a cutoff for risky drinking. This scale has been demonstrated to be reliable and valid for Swedish adolescents [ 55 ]. Furthermore, 2 questions were added concerning whether the participants had ever consumed alcohol to the point of intoxication and their age at the onset of drinking and intoxication. The original version of the LoL was designed for adults and asked the respondents to reflect on their, present, and future life status from a 5-year perspective on a 10-point Visual Analogue Scale representing life status from “worst” to “best” possible life imaginable [ 56 ]. A modified version for children, using a time frame of 1 year, has been used previously in Sweden [ 57 ] and was used in this study.

Randomization

After completing the baseline assessment, each participant was allocated to either the intervention or the control group. An external researcher generated an unrestricted random allocation sequence using random allocation software [ 58 ]. Neither the participants nor the researchers involved in the study were blinded to group allocation.

Based on the order in which participants were included in the study, they were allocated to 1 of the 2 study groups and informed of their allocation by email. Additionally, those who were randomized to the intervention group received a hyperlink to the Alcohol and Coping program, whereas the control group participants received information that they would gain access to Alcohol and Coping after the last follow-up assessment (ie, the waitlist control group). All participants were informed about other information and support available through web pages, notably drugsmart [ 59 ], which contains general information and facts about alcohol and drugs, in addition to more specific information about having substance-abusing parents. Telephone numbers and contact information for other organizations and primary health care facilities were also provided.

The Intervention

As noted previously, Alcohol and Coping is derived from the aforementioned manual-based face-to-face ICAIP intervention program [ 41 , 42 ]. The ICAIP consists of a combination of an alcohol intervention program, which is based on the short version of the Brief Alcohol Screening and Intervention for College Students program [ 60 ], and a coping intervention program developed for the purpose of the ICAIP [ 41 , 42 ]. Like the original ICAIP intervention, Alcohol and Coping builds on psychoeducational principles and includes components such as film-based lectures, various exercises, and both automated and therapist-assisted feedback. Briefly, once the participants logged into the Alcohol and Coping platform, they were introduced to the program, which followed the pattern of a board game ( Figure 2 ). Following the introduction, participants took part in 3 film-based lectures (between 8 and 15 minutes each, Figure 3 ) concerning alcohol problems within the family. The respective lectures included information about (1) dependency in general as well as the genetic and environmental risks for developing dependency, (2) family patterns and how the family adapts to the one having alcohol problems, and (3) attitudes toward alcohol and how they influence drinking and the physiological effects of alcohol. After completing the lectures, the participants were asked to answer 2 questions about their own alcohol consumption (ie, how often they drink and how often they drink to intoxication), followed by an automatic feedback message that depended on their answers. It was then suggested that the participants log out of the intervention for a 1- to 2-day break. The reason for this break was to give the participants a chance to digest all information and impressions. When they logged back into the intervention, they were asked to answer 20 questions about their coping strategies, which were also followed by automatic feedback. This feedback comprised a library covering all the prewritten feedback messages, each of which was tailored to the participants’ specific answers. The participants then participated in a 5-minute–long film-based lecture on emotion and problem-focused coping in relation to family alcohol problems ( Figure 3 ). This was followed by 4 exercises where the participants read through vignette-like stories from 4 fictional persons describing their everyday lives related to coping and alcohol problems in the family. The stories are presented by film-based introductions that are each 1-2 minutes long. Participants were then requested to respond to each story by describing how the fictive person could have coped with their situation. As a final exercise, participants were asked to reflect on their own family situation and how they cope with situations. The participants then had to take a break for a few days.

During the break, a therapist composed individual feedback that covered reflections and confirmation of the participant’s exercises and answers to questions and included suggestions on well-suited coping strategies. Additionally, the therapist encouraged the participants to talk to others in their surroundings, such as friends, teachers, or coaches, and seek further support elsewhere, such as from municipal social services, youth health care centers, or other organizations. Finally, the therapist reflected on the participants’ alcohol consumption patterns and reminded them of increased genetic and environmental risks. Those who revealed patterns of risky alcohol use were encouraged to look at 2 additional film-based lectures with more information about alcohol and intoxication (4 minutes) and alcohol use and dependency (5 minutes). Participants received this feedback once they logged back into the program, but they also had the opportunity to receive feedback through email. The total estimated effective time for completing the program was about 1 hour, but as described above, there was 1 required break when the individualized feedback was written. To keep track of the dose each participant received, each of the 15 components in the program ( Figure 1 ) is equal to completing 6.7% (1/15) of the program in total.

Sample Size

The trial was designed to detect a medium or large effect size corresponding to a standardized mean difference (Cohen d >0.5) [ 61 ]. An a priori calculation of the estimated sample size, using the software G*Power (G*Power Team) [ 62 ], revealed that a total of 128 participants (64 in each group) were required to enroll in the trial (power=0.80; α=.05; 2-tailed). However, to account for an estimated attrition rate of approximately 30% [ 34 ], it was necessary to enroll a minimum of 128/(1 – 0.3) = 183 participants in the trial. After a total of 204 individuals had been recruited and randomized into 2 study arms, recruitment was ended.

Statistical Analysis

Data were analyzed according to the intention-to-treat (ITT) principle, and all randomized participants were included, irrespective of whether they participated in the trial. The 4 research variables were depression (CES-DC), coping (CPAQ), alcohol use (AUDIT-C), and life status (LoL).

Data analysis consisted of comparing outcome measurements at t 1 and t 2 . The baseline measurement t 0 value was added as an adjustment variable in all models. The resulting data from CPAQ, CES-DC, and LoL were normally distributed and analyzed using linear mixed models. The resulting AUDIT-C scores were nonnormally distributed, with an excess of 0 values, and were analyzed using a 2-part model for longitudinal data. This model is sufficiently flexible to account for numerous 0 reports. This was achieved by combining a logistic generalized linear mixed model (GLMM) for the 0 parts and a skewed continuous GLMM for the non-0 alcohol consumption parts. R-package brms (Bayesian regression models using Stan; R Foundation for Statistical Computing) [ 63 ], a higher-level interface for the probabilistic programming language Stan [ 64 ], and a custom brms family for a marginalized 2-part lognormal distribution were used to fit the model [ 65 ]. The logistic part of the model represents the subject-specific effects on the odds of reporting no drinking. The continuous part was modeled using a gamma GLMM with a log link. The exponentiated treatment effect represents the subject-specific ratio of the total AUDIT-C scores between the treatment and waitlist control groups for those who reported drinking during the specific follow-up period.

Handling of Missing Data

GLMMs include all available data and provide unbiased ITT estimates under the assumption that data are missing at random, meaning that the missing data can be explained by existing data. However, it is impossible to determine whether the data are missing at random or whether the missing data are due to unobserved factors [ 66 ]. Therefore, we also assumed that data were not missing at random, and subsequent sensitivity analyses were performed [ 66 ]. We used the pattern mixture method, which assumes not missing at random, to compare those who completed the follow-up at 6 months (t 2 ) with those who did not (but completed the 2-month follow-up). The overall effect of this model is a combination of the effects of each subgroup. We also tested the robustness of the results by performing ANCOVAs at the 2-month follow-up, both using complete cases and with missing values imputed using multilevel multiple imputation.

The effect of the program was estimated using Cohen d , where a value of approximately 0.2 indicates a small effect size and values of approximately 0.5 and 0.8 indicate medium and large effect sizes, respectively [ 61 ].

Ethical Considerations

All procedures were performed in accordance with the ethical standards of the institutional or national research committees, the 1964 Helsinki Declaration and its later amendments, and comparable ethical standards. Informed consent was obtained from all the participants included in the study. This study was approved by the Swedish Ethical Review Authority (formerly the Regional Ethical Review Board in Stockholm, No. 2011/1648-31/5).

To enhance the response rates, participants received a cinema gift certificate corresponding to approximately EUR 11 (US $12) as compensation for completing each assessment. If a participant completed all assessments, an additional gift certificate was provided. The participants could subsequently receive 4 cinema gift certificates totaling EUR 44 (US $48).

The trial profile is depicted in Figure 1 and reveals that 2722 individuals who were aged between 15 and 19 years performed the screening procedure. A total of 1448 individuals did not fulfill the inclusion criteria and were excluded, leaving 1274 eligible participants. Another 1070 individuals were excluded because they did not provide informed consent or complete the baseline assessment, leaving 204 participants who were allocated to 1 of the 2 study groups. A total of 140 (69%) and 131 (64%) participants completed t 1 and t 2 assessments, respectively. Of the participants in the intervention group (n=101), 63% (n=64) registered an account on the Alcohol and Coping website, 35% (n=35) completed the alcohol intervention section, and 24% (n=24) completed both the alcohol and coping intervention sections.

Sample Characteristics

The mean age of the sample was 17.0 (SD 1.23) years, and the vast majority were female, with both parents born in Sweden and currently working ( Table 1 ). Approximately one-third of the participants reported living with both parents. The mean score on the CAST-6 was 5.33 (SD 0.87) out of a total of 6, and the majority of the sample (147/204, 72.1%) perceived their father to have alcohol problems. Approximately 12% (25/204) had never consumed alcohol, whereas approximately 70% (144/204) had consumed alcohol at a level of intoxication. The mean age at onset was 13.7 (SD 2.07) years and the age at first intoxication was 14.8 (SD 1.56) years. The proportion of participants with symptoms of at least moderate depression was 77.5% (158/204), of whom 55.1% (87/158) had symptoms of severe depression and 42.6% (87/204) had symptoms of dysfunctional coping behaviors. The percentage of participants who consumed alcohol at a risky level was 39.7% (81/204). Table 1 provides complete information regarding the study sample.

a Significance levels calculated by Pearson chi-square statistics for categorical variables and 2-tailed t tests for continuous variables.

Treatment Effects

For the primary outcome, coping behavior (CPAQ), we found a small but inconclusive treatment effect in favor of treatment at both 2 (t 1 ) and 6 (t 2 ) months (Cohen d =–0.05 at both t 1 and t 2 ). For the secondary outcome, alcohol use (AUDIT-C), we found a treatment effect in that the intervention group scored 38% less than the control group on the continuous part (ie, drinking when it occurred) at t 1 and 46% less at t 2 . Regarding depression (CES-DC) and life status (LoL), all between-group comparisons of treatment effects were inconclusive at both follow-up time points ( Table 2 ).

a CPAQ: Coping With Parents Abuse Questionnaire.

b CES-DC: Center for Epidemiological Studies Depression Scale.

c LoL: Ladder of Life.

d AUDIT-C: Alcohol Use Disorders Identification Test.

e N/A: not applicable.

Missing Data

In contrast to the ITT analyses, the sensitivity analyses showed that the treatment group, averaged over the levels of dropout, scored higher (ie, a negative effect) on the main outcome, coping behavior (CPAQ), at t 1 (2.44; P =.20). However, the results remain inconclusive.

Dose-Response Effects

We did not find any evidence for greater involvement in the program being linked to improved outcomes with regard to coping behavior.

We did not find any support for the primary hypothesis: the intervention was not superior to the control condition with regard to coping behavior. Inconclusive results with small effect sizes were observed at both follow-up time points. However, for the secondary outcomes, we found that those in the intervention group who drank alcohol drank approximately 40%-50% less than those in the control group at both follow-ups. These results corroborate previous findings on the precursor face-to-face ICAIP intervention program, demonstrating that participants who received a combined alcohol and coping intervention reported superior outcomes with regard to alcohol-related outcomes compared to participants in the other 2 study arms, who received only a coping or alcohol intervention [ 41 , 42 ]. In contrast to this study, Hansson et al [ 42 ] found that all groups improved their coping skills, although the between-group comparisons were inconclusive and the improvements were maintained over time. These differences could be explained by the different settings in which the precursor program was provided (ie, face-to-face to young adults in a university setting), whereas this study targeted young people (15-19 years of age) through a web-based digital intervention. Additionally, the poor adherence in this study may explain the absence of primary results favoring the intervention group. In a recent study, parents without alcohol problems were recruited to participate in a randomized trial evaluating the web-based SPARE (Supportive Parenting and Reinforcement) program to improve children’s mental health and reduce coparents’ alcohol use. In line with our study, the authors did not find the primary outcome of the SPARE program to be superior to that of the active control group (which received written psychoeducation); however, both groups reported decreased coparental alcohol consumption [ 38 ].

Considering that approximately 3600 children in 2022 participated in various forms of support provided by Swedish municipalities [ 23 ], our recruitment activities reached a large number of eligible individuals, pointing to the potential of finding these children on these platforms. There were unexpectedly high levels of depression among the participants in this study. Although the intervention did not target depressive symptoms per se , there was a trend for the intervention group to have decreased depression levels compared to the control group. A large proportion of participants had symptoms of severe depression, which may have aggravated their capacity for improvement at follow-up [ 28 , 67 ]. Targeting dysfunctional coping patterns could affect an individual’s perceived mental health, and studies have shown that healthy coping strategies positively affect depression and anxiety in a positive way [ 68 ]. Using dysfunctional coping strategies, such as negative self-talk and alcohol consumption, can lead to depressive symptoms [ 69 ]. Targeting these symptoms in the context of healthy and unhealthy coping strategies may be a viable route to fostering appropriate coping strategies that work in the long run. Given that the young people who were reached by the intervention in this study displayed high levels of depression, future interventions for this group should include programs targeting depressive symptoms.

Almost 37% (37/101) of the intervention group did not log into the intervention at all, and only 24% (24/101) of the intervention group participants completed all parts of the program. The fact that a high proportion of the participants had symptoms of severe depression could explain the low adherence. Another reason could be that the initial film-based lectures were too long to maintain the participants’ attention, as the lectures ranged from 8-15 minutes. Yet a final reason could be that we had a 1- to 2-day break built into the intervention, and for unknown reasons, some participants did not log back into the intervention. However, we did not find a dose-response relationship indicating favorable outcomes for those who completed more of the program content. High levels of attrition are not uncommon in self-directed programs such as the one in this study; for example, in a study on a smoking cessation intervention, 37% of the participants never logged into the platform [ 70 ], and in a self-directed intervention for problem gamblers, a majority dropped out after 1 week and none completed the entire program [ 71 ]. Increased intervention adherence is a priority when developing new digital interventions, particularly for young people. One method is to use more persuasive technologies, such as primary tasks, dialogue, and social support [ 72 ]. Considering children whose parents have mental disorders, Grové and Reupert [ 73 ] suggested that digital interventions should include components such as providing information about parental mental illness, access to health care, genetic risk, and suggestions for how children might initiate conversations with parents who have the illness. These suggestions should be considered in future studies on interventions for youths whose parents have substance use problems. Representatives of the target group and other relevant stakeholders should also be involved in coproducing new interventions to increase the probability of developing more engaging programs [ 74 ]. Moreover, one cannot expect study participants to return to the program more than once, and for the sake of adherence, briefer interventions should not encourage participants to log-out for a break. To keep adherence at an acceptable level, similar future interventions for this target group should also consider having symptoms of severe depression as an exclusion criterion [ 28 , 67 ]. Further, to improve adherence, strategies of coproduction could be used where all stakeholders, including the target group, are involved in intervention development [ 75 ]. Other important factors identified to improve adherence to digital interventions are to make the content relatable, useful, and even more interactive [ 76 ]. Those participants who have symptoms of severe depression should be referred to other appropriate health care. Finally, it is probably beneficial to develop shorter psychoeducative film-based lectures than ours, lasting up to 15 minutes. Future self-directed digital interventions targeting this population should, therefore, focus on a very brief and focused intervention, which, based on theory, has the potential to foster healthy coping behaviors that can lead to an increased quality of life and improved mental health for this group of young people.

Another concern for future projects would be to use a data-driven approach during the program development phase, where A/B testing can be used to test different setups of the program to highlight which setup works best. Another aspect that must be considered is the fast-changing world of technology, where young people are exposed to an infinite number of different apps that grab their attention, which also calls for interventions to be short and to the point. Furthermore, if the program is to spread and become generally available, one must consider that keeping the program alive for a longer period will require funding and staffing for both product management and technical support.

Strengths and Limitations

This study had several strengths. First, Alcohol and Coping is a web-based intervention program, and it appears as if the internet is a particularly promising way to provide support to adolescents growing up with parents with alcohol problems because it offers an anonymous means of communicating and makes intervention programs readily accessible [ 25 ]. Our recruitment strategies reached a considerable number of interested and eligible individuals, demonstrating the potential for recruiting through social media and other web platforms. Additionally, this program is one of the first brief web-based interventions aimed at adolescents with parents with alcohol-related problems. We used the CAST-6, which has been validated among Swedish adolescents [ 47 ], to screen eligible participants. Another strength is that the intervention program involved personalized, tailored feedback in the form of prewritten automatic messages and therapist-written personalized feedback, both of which have proven to be important components of web-based interventions aimed at adolescents [ 77 , 78 ]. Finally, this study evaluated the effectiveness of the Alcohol and Coping program using a randomized controlled trial design, which is considered the strongest experimental design with regard to allocation bias.

This study had some limitations. First, the design with a passive waitlist control group and an active intervention group, both unblinded to study allocation, may have resulted in biased estimates of treatment effects. Intervention adherence was low, and most of the study participants had symptoms of depression, where 55% (87/158) had symptoms of severe depression. This may have contributed to the small and overall inconclusive effects on the primary outcomes of this study. Many digital interventions have problems with low adherence, and in a review by Välimäki et al [ 79 ], some studies reported adherence rates as low as 10%. A vast proportion of the study participants were women, making the findings difficult to generalize to men. However, another limitation concerns selection bias and external validity. We recruited study participants through social media and other relevant websites containing health-related information, including information about parents with alcohol-related problems. It is, therefore, possible that the study population can be classified as “information-seeking” adolescents, who may have different personality traits relative to other adolescents in the same home situation. Additionally, as an inclusion criterion was having ready access to computers and the internet, it is possible that participants belonging to a lower socioeconomic class were underrepresented in the study. It should also be noted that the data presented here were collected approximately 10 years ago. However, we believe our findings make an important contribution to the field since, like our intervention, many recent web-based interventions use strategies of psychoeducation, films, exercises, questions, and feedback. Further, the number of web-based interventions for this target group remains scarce in the literature, which underscores the need for future research. Finally, the study was powered to detect a medium effect size. However, given the small effect sizes detected in this study, it is plausible that too few participants were recruited to detect differences between the groups.

Implications for Practice

Although growing up with parents who have alcohol problems per se is not sufficient for developing psychosocial disorders, many children need support to manage their situation. Therefore, it is difficult to recruit children to support these groups. In Sweden, not even 2% of all children growing up with parental alcohol problems attend face-to-face support groups provided by municipalities.

Offering support through web-based intervention programs seems particularly attractive to adolescents whose parents have alcohol-related problems. To date, evidence for such programs is scarce, and there is an urgent need to develop and evaluate digital interventions targeting this group of adolescents. This study makes important contributions to this novel field of research. The results provide insight into effective strategies for delivering intervention programs to children of parents with substance abuse issues, highlighting the potential for digital interventions to reach a vulnerable, hard-to-reach group of adolescents. Our findings underscore the need to develop more engaging interventions in coproduction with the target group.

Conclusions

We found that a digital therapist-assisted self-management intervention for adolescents whose parents have alcohol use problems contributed to a reduction in the adolescents’ own alcohol consumption. This result highlights the potential for digital interventions to reach a large, vulnerable, and hard-to-reach group of adolescents with support efforts. Findings were inconclusive for all other outcomes, which may be attributable to low adherence. This points to the need for future research on developing more engaging digital interventions to increase adherence among adolescents.

Acknowledgments

This work was undertaken on behalf of the Swedish Council for Information on Alcohol and Other Drugs (CAN) and was supported by grants from the Swedish National Institute of Public Health and the Swedish Council for Working Life and Social Research.

Conflicts of Interest

HH and UZ developed the study interventions. However, the parties did not derive direct financial income from these interventions. HW, PK, and THE declare no conflicts of interest.

CONSORT-eHEALTH checklist (V 1.6.1).

• Haugland SH, Elgán TH. Prevalence of parental alcohol problems among a general population sample of 28,047 Norwegian adults: evidence for a socioeconomic gradient. Int J Environ Res Public Health. 2021;18(10):5412. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Elgán TH, Leifman H. Prevalence of adolescents who perceive their parents to have alcohol problems: a Swedish national survey using a web panel. Scand J Public Health. 2013;41(7):680-683. [ CrossRef ] [ Medline ]
• Laslett AM, Ferris J, Dietze P, Room R. Social demography of alcohol-related harm to children in Australia. Addiction. 2012;107(6):1082-1089. [ CrossRef ] [ Medline ]
• Manning V, Best DW, Faulkner N, Titherington E. New estimates of the number of children living with substance misusing parents: results from UK national household surveys. BMC Public Health. 2009;9:377. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Grant BF. Estimates of US children exposed to alcohol abuse and dependence in the family. Am J Public Health. 2000;90(1):112-115. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Raninen J, Elgán TH, Sundin E, Ramstedt M. Prevalence of children whose parents have a substance use disorder: findings from a Swedish general population survey. Scand J Public Health. 2016;44(1):14-17. [ CrossRef ] [ Medline ]
• Christoffersen MN, Soothill K. The long-term consequences of parental alcohol abuse: a cohort study of children in Denmark. J Subst Abuse Treat. 2003;25(2):107-116. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Martikainen PN, Korhonen K, Moustgaard H, Aaltonen M, Remes H. Substance abuse in parents and subsequent risk of offspring psychiatric morbidity in late adolescence and early adulthood: a longitudinal analysis of siblings and their parents. Soc Sci Med. 2018;217:106-111. [ CrossRef ] [ Medline ]
• Jääskeläinen M, Holmila M, Notkola IL, Raitasalo K. Mental disorders and harmful substance use in children of substance abusing parents: a longitudinal register-based study on a complete birth cohort born in 1991. Drug Alcohol Rev. 2016;35(6):728-740. [ CrossRef ] [ Medline ]
• Velleman R, Templeton LJ. Impact of parents' substance misuse on children: an update. BJPsych Adv. Apr 11, 2018;22(2):108-117. [ FREE Full text ] [ CrossRef ]
• Ohannessian CM, Hesselbrock VM, Kramer J, Bucholz KK, Schuckit MA, Kuperman S, et al. Parental substance use consequences and adolescent psychopathology. J Stud Alcohol. 2004;65(6):725-730. [ CrossRef ] [ Medline ]
• Johnson JL, Leff M. Children of substance abusers: overview of research findings. Pediatrics. 1999;103(5 Pt 2):1085-1099. [ CrossRef ] [ Medline ]
• Berg L, Bäck K, Vinnerljung B, Hjern A. Parental alcohol-related disorders and school performance in 16-year-olds-a Swedish national cohort study. Addiction. 2016;111(10):1795-1803. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Casas-Gil MJ, Navarro-Guzman JI. School characteristics among children of alcoholic parents. Psychol Rep. 2002;90(1):341-348. [ CrossRef ] [ Medline ]
• McGrath CE, Watson AL, Chassin L. Academic achievement in adolescent children of alcoholics. J Stud Alcohol. 1999;60(1):18-26. [ CrossRef ] [ Medline ]
• Velleman R, Templeton L, Reuber D, Klein M, Moesgen D. Domestic abuse experienced by young people living in families with alcohol problems: results from a cross‐european study. Child Abuse Rev. Nov 24, 2008;17(6):387-409. [ CrossRef ]
• Rothman EF, Edwards EM, Heeren T, Hingson RW. Adverse childhood experiences predict earlier age of drinking onset: results from a representative US sample of current or former drinkers. Pediatrics. 2008;122(2):e298-e304. [ CrossRef ] [ Medline ]
• Anda RF, Whitfield CL, Felitti VJ, Chapman D, Edwards VJ, Dube SR, et al. Adverse childhood experiences, alcoholic parents, and later risk of alcoholism and depression. Psychiatr Serv. 2002;53(8):1001-1009. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Straussner SLA, Fewell CH. A review of recent literature on the impact of parental substance use disorders on children and the provision of effective services. Curr Opin Psychiatry. 2018;31(4):363-367. [ CrossRef ] [ Medline ]
• Calhoun S, Conner E, Miller M, Messina N. Improving the outcomes of children affected by parental substance abuse: a review of randomized controlled trials. Subst Abuse Rehabil. 2015;6:15-24. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Emshoff JG, Price AW. Prevention and intervention strategies with children of alcoholics. Pediatrics. 1999;103(5 Pt 2):1112-1121. [ CrossRef ] [ Medline ]
• Cuijpers P. Prevention programmes for children of problem drinkers: a review. Drugs Educ Prev Policy. 2009;12(6):465-475. [ CrossRef ]
• Wannberg H. Plats för barnen—Om kommunernas stöd till barn som växer upp med missbrukande föräldrar [Make room for the children—municipalities and their support for children who grow up with parents with substance abuse]. Stockholm, Sweden. Junis, IOGT-NTO's ungdomsförbund; 2023.
• Elgán TH, Leifman H. Children of substance abusing parents: a national survey on policy and practice in Swedish schools. Health Policy. 2011;101(1):29-36. [ CrossRef ] [ Medline ]
• de Sousa D, Fogel A, Azevedo J, Padrão P. The effectiveness of web-based interventions to promote health behaviour change in adolescents: a systematic review. Nutrients. 2022;14(6):1258. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Andersson G. Internet-delivered psychological treatments. Annu Rev Clin Psychol. 2016;12:157-179. [ FREE Full text ] [ CrossRef ] [ Medline ]
• King R, Bambling M, Lloyd C, Gomurra R, Smith S, Reid W, et al. Online counselling: the motives and experiences of young people who choose the internet instead of face to face or telephone counselling. Couns Psychother Res. 2007;6(3):169-174. [ CrossRef ]
• Borghouts J, Eikey E, Mark G, De Leon C, Schueller SM, Schneider M, et al. Barriers to and facilitators of user engagement with digital mental health interventions: systematic review. J Med Internet Res. 2021;23(3):e24387. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Boggs JM, Beck A, Felder JN, Dimidjian S, Metcalf CA, Segal ZV. Web-based intervention in mindfulness meditation for reducing residual depressive symptoms and relapse prophylaxis: a qualitative study. J Med Internet Res. 2014;16(3):e87. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hedman-Lagerlöf E, Carlbring P, Svärdman F, Riper H, Cuijpers P, Andersson G. Therapist-supported internet-based cognitive behaviour therapy yields similar effects as face-to-face therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. World Psychiatry. 2023;22(2):305-314. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lehtimaki S, Martic J, Wahl B, Foster KT, Schwalbe N. Evidence on digital mental health interventions for adolescents and young people: systematic overview. JMIR Ment Health. 2021;8(4):e25847. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Li J, Theng YL, Foo S. Game-based digital interventions for depression therapy: a systematic review and meta-analysis. Cyberpsychol Behav Soc Netw. 2014;17(8):519-527. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pennant ME, Loucas CE, Whittington C, Creswell C, Fonagy P, Fuggle P, et al. Computerised therapies for anxiety and depression in children and young people: a systematic review and meta-analysis. Behav Res Ther. 2015;67:1-18. [ CrossRef ] [ Medline ]
• Woolderink M, Smit F, van der Zanden R, Beecham J, Knapp M, Paulus A, et al. Design of an internet-based health economic evaluation of a preventive group-intervention for children of parents with mental illness or substance use disorders. BMC Public Health. 2010;10:470. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Elgán TH, Kartengren N, Strandberg AK, Ingemarson M, Hansson H, Zetterlind U, et al. A web-based group course intervention for 15-25-year-olds whose parents have substance use problems or mental illness: study protocol for a randomized controlled trial. BMC Public Health. 2016;16(1):1011. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Maybery D, Reupert A, Bartholomew C, Cuff R, Duncan Z, Foster K, et al. A web-based intervention for young adults whose parents have a mental illness or substance use concern: protocol for a randomized controlled trial. JMIR Res Protoc. 2020;9(6):e15626. [ FREE Full text ] [ CrossRef ] [ Medline ]
• EÉk N, Romberg K, Siljeholm O, Johansson M, Andreasson S, Lundgren T, et al. Efficacy of an internet-based community reinforcement and family training program to increase treatment engagement for AUD and to improve psychiatric health for CSOs: a randomized controlled trial. Alcohol Alcohol. 2020;55(2):187-195. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Siljeholm O, Lindner P, Johansson M, Hammarberg A. An online self-directed program combining Community Reinforcement Approach and Family Training and parenting training for concerned significant others sharing a child with a person with problematic alcohol consumption: a randomized controlled trial. Addict Sci Clin Pract. 2022;17(1):49. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Woolderink M. Mind the gap: evaluation of an online preventive programme for adolescents with mentally ill or addicted parents. Maastricht University. 2016. URL: https://cris.maastrichtuniversity.nl/ws/portalfiles/portal/7281149/c5531.pdf [accessed 2024-03-14]
• Maybery D, Reupert A, Bartholomew C, Cuff R, Duncan Z, McAuliffe C, et al. An online intervention for 18-25-year-old youth whose parents have a mental illness and/or substance use disorder: a pilot randomized controlled trial. Early Interv Psychiatry. 2022;16(11):1249-1258. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hansson H, Rundberg J, Zetterlind U, Johnsson KO, Berglund M. An intervention program for university students who have parents with alcohol problems: a randomized controlled trial. Alcohol Alcohol. 2006;41(6):655-663. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hansson H, Rundberg J, Zetterlind U, Johnsson KO, Berglund M. Two-year outcome of an intervention program for university students who have parents with alcohol problems: a randomized controlled trial. Alcohol Clin Exp Res. 2007;31(11):1927-1933. [ CrossRef ] [ Medline ]
• Zetterlind U, Hansson H, Aberg-Orbeck K, Berglund M. Effects of coping skills training, group support, and information for spouses of alcoholics: a controlled randomized study. Nord J Psychiatry. 2001;55(4):257-262. [ CrossRef ] [ Medline ]
• Hansson H, Zetterlind U, Aberg-Orbeck K, Berglund M. Two-year outcome of coping skills training, group support and information for spouses of alcoholics: a randomized controlled trial. Alcohol Alcohol. 2004;39(2):135-140. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hodgins DC, Maticka-Tyndale E, El-Guebaly N, West M. The CAST-6: development of a short-form of the Children of Alcoholics Screening Test. Addict Behav. 1993;18(3):337-345. [ CrossRef ] [ Medline ]
• Hodgins DC, Shimp L. Identifying adult children of alcoholics: methodological review and a comparison of the CAST-6 with other methods. Addiction. 1995;90(2):255-267. [ Medline ]
• Elgán TH, Berman AH, Jayaram-Lindström N, Hammarberg A, Jalling C, Källmén H. Psychometric properties of the short version of the children of alcoholics screening test (CAST-6) among Swedish adolescents. Nord J Psychiatry. 2021;75(2):155-158. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Orford J, Guthrie S, Nicholls P, Oppenheimer E, Egert S, Hensman C. Self-reported coping behavior of wives of alcoholics and its association with drinking outcome. J Stud Alcohol. 1975;36(9):1254-1267. [ CrossRef ] [ Medline ]
• Schoenbach VJ, Kaplan BH, Grimson RC, Wagner EH. Use of a symptom scale to study the prevalence of a depressive syndrome in young adolescents. Am J Epidemiol. 1982;116(5):791-800. [ CrossRef ] [ Medline ]
• Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test. Arch Intern Med. 1998;158(16):1789-1795. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Myers JK, Weissman MM. Use of a self-report symptom scale to detect depression in a community sample. Am J Psychiatry. 1980;137(9):1081-1084. [ CrossRef ] [ Medline ]
• Olsson G, von Knorring AL. Depression among Swedish adolescents measured by the self-rating scale Center for Epidemiology Studies-Depression Child (CES-DC). Eur Child Adolesc Psychiatry. 1997;6(2):81-87. [ CrossRef ] [ Medline ]
• Fendrich M, Weissman MM, Warner V. Screening for depressive disorder in children and adolescents: validating the Center for Epidemiologic Studies Depression Scale for Children. Am J Epidemiol. 1990;131(3):538-551. [ CrossRef ] [ Medline ]
• Bergman H, Källmen H, Rydberg U, Sandahl C. Tio frågor om alkohol identifierar beroendeproblem. Psykometrisk prövning på psykiatrisk akutmottagning [Ten questions about alcohol as identifier of addiction problems. Psychometric tests at an emergency psychiatric department]. Läkartidningen. 1998;95(43):4731-4735. [ FREE Full text ]
• Källmén H, Berman AH, Jayaram-Lindström N, Hammarberg A, Elgán TH. Psychometric properties of the AUDIT, AUDIT-C, CRAFFT and ASSIST-Y among Swedish adolescents. Eur Addict Res. 2019;25(2):68-77. [ CrossRef ] [ Medline ]
• Andrews FM, Withey SB. Developing measures of perceived life quality: results from several national surveys. Soc Indic Res. 1974;1(1):1-26. [ CrossRef ]
• Nagy E. Barns känsla av sammanhang—En valideringsstudie av BarnKASAM i årskurserna 1-6 (ålder 7-12 år) [Children's sense of coherence—a study validating SOC for children in grades 1-6 (7-12 years old)]. Lunds Universitet [Lund University]. 2004. URL: https://lup.lub.lu.se/luur/download?func=downloadFile&recordOId=1358959&fileOId=1358960 [accessed 2024-03-14]
• Saghaei M. Random allocation software for parallel group randomized trials. BMC Med Res Methodol. 2004;4:26. [ FREE Full text ] [ CrossRef ] [ Medline ]
• drugsmart. URL: https://www.drugsmart.se/ [accessed 2024-03-19]
• Dimeff LA, Baer JS, Kivlahan DR, Marlatt GA. Brief Alcohol Screening and Intervention for College Students: A Harm Reduction Approach. New York. The Guilford Press; 1999.
• Cohen J. A power primer. Psychol Bull. 1992;112(1):155-159. [ CrossRef ] [ Medline ]
• Faul F, Erdfelder E, Lang A, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39(2):175-191. [ CrossRef ] [ Medline ]
• Bürkner PC. brms: an R package for bayesian multilevel models using stan. J Stat Soft. 2017;80(1):1-28. [ FREE Full text ] [ CrossRef ]
• Carpenter B, Gelman A, Hoffman MD, Lee D, Goodrich B, Betancourt M, et al. Stan: a probabilistic programming language. J Stat Softw. 2017;76(1):1-32. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Magnusson K, Nilsson A, Carlbring P. Modeling longitudinal gambling data: challenges and opportunities. PsyArxiv. Preprint posted online on September 12, 2019. [ FREE Full text ] [ CrossRef ]
• Coertjens L, Donche V, De Maeyer S, Vanthournout G, Van Petegem P. To what degree does the missing-data technique influence the estimated growth in learning strategies over time? A tutorial example of sensitivity analysis for longitudinal data. PLoS One. 2017;12(9):e0182615. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kunas SL, Lautenbacher LM, Lueken PU, Hilbert K. Psychological predictors of cognitive-behavioral therapy outcomes for anxiety and depressive disorders in children and adolescents: a systematic review and meta-analysis. J Affect Disord. 2021;278:614-626. [ CrossRef ] [ Medline ]
• Stallman HM, Lipson SK, Zhou S, Eisenberg D. How do university students cope? An exploration of the health theory of coping in a US sample. J Am Coll Health. 2022;70(4):1179-1185. [ CrossRef ] [ Medline ]
• Stallman HM, Beaudequin D, Hermens DF, Eisenberg D. Modelling the relationship between healthy and unhealthy coping strategies to understand overwhelming distress: a Bayesian network approach. J Affect Disord Rep. 2021;3:100054. [ FREE Full text ] [ CrossRef ]
• McClure JB, Shortreed SM, Bogart A, Derry H, Riggs K, St John J, et al. The effect of program design on engagement with an internet-based smoking intervention: randomized factorial trial. J Med Internet Res. 2013;15(3):e69. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Humphrey G, Bullen C. Smartphone-based problem gambling evaluation and technology testing initiative ('SPGeTTI'): final report reference 354913/00 for the Ministry of Health. National Institute for Health Innovation (NIHI), Auckland UniServices Ltd, The Univerisy of Auckland. 2019. URL: https:/​/www.​health.govt.nz/​system/​files/​documents/​publications/​20190424-spgetti-354913-00-final-report.​pdf [accessed 2024-03-14]
• Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JEWC. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res. 2012;14(6):e152. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Grové C, Reupert A. Moving the field forward: developing online interventions for children of parents with a mental illness. Child Youth Serv Rev. 2017;82:354-358. [ CrossRef ]
• Moffat BM, Haines-Saah RJ, Johnson JL. From didactic to dialogue: assessing the use of an innovative classroom resource to support decision-making about cannabis use. Drugs Educ Prev Policy. 2016;24(1):85-95. [ FREE Full text ] [ CrossRef ]
• Bevan Jones R, Stallard P, Agha SS, Rice S, Werner-Seidler A, Stasiak K, et al. Practitioner review: co-design of digital mental health technologies with children and young people. J Child Psychol Psychiatry. 2020;61(8):928-940. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Garrido S, Millington C, Cheers D, Boydell K, Schubert E, Meade T, et al. What works and what doesn't work? A systematic review of digital mental health interventions for depression and anxiety in young people. Front Psychiatry. 2019;10:759. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Crutzen R, de Nooijer J, Brouwer W, Oenema A, Brug J, de Vries NK. Strategies to facilitate exposure to internet-delivered health behavior change interventions aimed at adolescents or young adults: a systematic review. Health Educ Behav. 2011;38(1):49-62. [ CrossRef ] [ Medline ]
• Milward J, Drummond C, Fincham-Campbell S, Deluca P. What makes online substance-use interventions engaging? A systematic review and narrative synthesis. Digit Health. 2018;4:2055207617743354. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Välimäki M, Anttila K, Anttila M, Lahti M. Web-based interventions supporting adolescents and young people with depressive symptoms: systematic review and meta-analysis. JMIR Mhealth Uhealth. 2017;5(12):e180. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by YH Lin; submitted 24.08.23; peer-reviewed by X Zhang, C Asuzu, D Liu; comments to author 28.01.24; revised version received 08.02.24; accepted 27.02.24; published 10.04.24.

©Håkan Wall, Helena Hansson, Ulla Zetterlind, Pia Kvillemo, Tobias H Elgán. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 10.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

• Facebook Icon
• Twitter Icon
• LinkedIn Icon

Linking Rapid Aging with Early-onset Cancer Risk

Abstract: An analysis of data from nearly 150,000 individuals has revealed that accelerated aging is more common in recent birth cohorts and correlates with an increased risk of several cancer types. These findings may help explain the recent troubling trend of rising cancer rates among younger adults.

Client Account

Advertisement

• Noted This Week Archive (2011-2023)

AACR Journals

• Blood Cancer Discovery
• Cancer Discovery
• Cancer Epidemiology, Biomarkers & Prevention
• Cancer Immunology Research
• Cancer Prevention Research
• Cancer Research
• Cancer Research Communications
• Clinical Cancer Research
• Molecular Cancer Research
• Molecular Cancer Therapeutics
• Info for Advertisers
• Information for Institutions/Librarians

• Privacy Policy
• Copyright © 2023 by the American Association for Cancer Research.

This Feature Is Available To Subscribers Only

Sign In or Create an Account

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Research bias

Types of Bias in Research | Definition & Examples

Research bias results from any deviation from the truth, causing distorted results and wrong conclusions. Bias can occur at any phase of your research, including during data collection , data analysis , interpretation, or publication. Research bias can occur in both qualitative and quantitative research .

Understanding research bias is important for several reasons.

• Bias exists in all research, across research designs , and is difficult to eliminate.
• Bias can occur at any stage of the research process .
• Bias impacts the validity and reliability of your findings, leading to misinterpretation of data.

It is almost impossible to conduct a study without some degree of research bias. It’s crucial for you to be aware of the potential types of bias, so you can minimize them.

For example, the success rate of the program will likely be affected if participants start to drop out ( attrition ). Participants who become disillusioned due to not losing weight may drop out, while those who succeed in losing weight are more likely to continue. This in turn may bias the findings towards more favorable results.  

Table of contents

Information bias, interviewer bias.

• Publication bias

Researcher bias

Response bias.

Selection bias

Cognitive bias

How to avoid bias in research

Other types of research bias, frequently asked questions about research bias.

Information bias , also called measurement bias, arises when key study variables are inaccurately measured or classified. Information bias occurs during the data collection step and is common in research studies that involve self-reporting and retrospective data collection. It can also result from poor interviewing techniques or differing levels of recall from participants.

The main types of information bias are:

• Recall bias
• Observer bias

Performance bias

Regression to the mean (rtm).

Over a period of four weeks, you ask students to keep a journal, noting how much time they spent on their smartphones along with any symptoms like muscle twitches, aches, or fatigue.

Recall bias is a type of information bias. It occurs when respondents are asked to recall events in the past and is common in studies that involve self-reporting.

As a rule of thumb, infrequent events (e.g., buying a house or a car) will be memorable for longer periods of time than routine events (e.g., daily use of public transportation). You can reduce recall bias by running a pilot survey and carefully testing recall periods. If possible, test both shorter and longer periods, checking for differences in recall.

• A group of children who have been diagnosed, called the case group
• A group of children who have not been diagnosed, called the control group

Since the parents are being asked to recall what their children generally ate over a period of several years, there is high potential for recall bias in the case group.

The best way to reduce recall bias is by ensuring your control group will have similar levels of recall bias to your case group. Parents of children who have childhood cancer, which is a serious health problem, are likely to be quite concerned about what may have contributed to the cancer.

Thus, if asked by researchers, these parents are likely to think very hard about what their child ate or did not eat in their first years of life. Parents of children with other serious health problems (aside from cancer) are also likely to be quite concerned about any diet-related question that researchers ask about.

Observer bias is the tendency of research participants to see what they expect or want to see, rather than what is actually occurring. Observer bias can affect the results in observationa l and experimental studies, where subjective judgment (such as assessing a medical image) or measurement (such as rounding blood pressure readings up or down) is part of the d ata collection process.

Observer bias leads to over- or underestimation of true values, which in turn compromise the validity of your findings. You can reduce observer bias by using double-blinded  and single-blinded research methods.

Based on discussions you had with other researchers before starting your observations , you are inclined to think that medical staff tend to simply call each other when they need specific patient details or have questions about treatments.

At the end of the observation period, you compare notes with your colleague. Your conclusion was that medical staff tend to favor phone calls when seeking information, while your colleague noted down that medical staff mostly rely on face-to-face discussions. Seeing that your expectations may have influenced your observations, you and your colleague decide to conduct semi-structured interviews with medical staff to clarify the observed events. Note: Observer bias and actor–observer bias are not the same thing.

Performance bias is unequal care between study groups. Performance bias occurs mainly in medical research experiments, if participants have knowledge of the planned intervention, therapy, or drug trial before it begins.

Studies about nutrition, exercise outcomes, or surgical interventions are very susceptible to this type of bias. It can be minimized by using blinding , which prevents participants and/or researchers from knowing who is in the control or treatment groups. If blinding is not possible, then using objective outcomes (such as hospital admission data) is the best approach.

When the subjects of an experimental study change or improve their behavior because they are aware they are being studied, this is called the Hawthorne effect (or observer effect). Similarly, the John Henry effect occurs when members of a control group are aware they are being compared to the experimental group. This causes them to alter their behavior in an effort to compensate for their perceived disadvantage.

Regression to the mean (RTM) is a statistical phenomenon that refers to the fact that a variable that shows an extreme value on its first measurement will tend to be closer to the center of its distribution on a second measurement.

Medical research is particularly sensitive to RTM. Here, interventions aimed at a group or a characteristic that is very different from the average (e.g., people with high blood pressure) will appear to be successful because of the regression to the mean. This can lead researchers to misinterpret results, describing a specific intervention as causal when the change in the extreme groups would have happened anyway.

In general, among people with depression, certain physical and mental characteristics have been observed to deviate from the population mean .

This could lead you to think that the intervention was effective when those treated showed improvement on measured post-treatment indicators, such as reduced severity of depressive episodes.

However, given that such characteristics deviate more from the population mean in people with depression than in people without depression, this improvement could be attributed to RTM.

Interviewer bias stems from the person conducting the research study. It can result from the way they ask questions or react to responses, but also from any aspect of their identity, such as their sex, ethnicity, social class, or perceived attractiveness.

Interviewer bias distorts responses, especially when the characteristics relate in some way to the research topic. Interviewer bias can also affect the interviewer’s ability to establish rapport with the interviewees, causing them to feel less comfortable giving their honest opinions about sensitive or personal topics.

Participant: “I like to solve puzzles, or sometimes do some gardening.”

You: “I love gardening, too!”

In this case, seeing your enthusiastic reaction could lead the participant to talk more about gardening.

Establishing trust between you and your interviewees is crucial in order to ensure that they feel comfortable opening up and revealing their true thoughts and feelings. At the same time, being overly empathetic can influence the responses of your interviewees, as seen above.

Publication bias occurs when the decision to publish research findings is based on their nature or the direction of their results. Studies reporting results that are perceived as positive, statistically significant , or favoring the study hypotheses are more likely to be published due to publication bias.

Publication bias is related to data dredging (also called p -hacking ), where statistical tests on a set of data are run until something statistically significant happens. As academic journals tend to prefer publishing statistically significant results, this can pressure researchers to only submit statistically significant results. P -hacking can also involve excluding participants or stopping data collection once a p value of 0.05 is reached. However, this leads to false positive results and an overrepresentation of positive results in published academic literature.

Researcher bias occurs when the researcher’s beliefs or expectations influence the research design or data collection process. Researcher bias can be deliberate (such as claiming that an intervention worked even if it didn’t) or unconscious (such as letting personal feelings, stereotypes, or assumptions influence research questions ).

The unconscious form of researcher bias is associated with the Pygmalion effect (or Rosenthal effect ), where the researcher’s high expectations (e.g., that patients assigned to a treatment group will succeed) lead to better performance and better outcomes.

Researcher bias is also sometimes called experimenter bias, but it applies to all types of investigative projects, rather than only to experimental designs .

• Good question: What are your views on alcohol consumption among your peers?
• Bad question: Do you think it’s okay for young people to drink so much?

Response bias is a general term used to describe a number of different situations where respondents tend to provide inaccurate or false answers to self-report questions, such as those asked on surveys or in structured interviews .

This happens because when people are asked a question (e.g., during an interview ), they integrate multiple sources of information to generate their responses. Because of that, any aspect of a research study may potentially bias a respondent. Examples include the phrasing of questions in surveys, how participants perceive the researcher, or the desire of the participant to please the researcher and to provide socially desirable responses.

Response bias also occurs in experimental medical research. When outcomes are based on patients’ reports, a placebo effect can occur. Here, patients report an improvement despite having received a placebo, not an active medical treatment.

While interviewing a student, you ask them:

“Do you think it’s okay to cheat on an exam?”

Common types of response bias are:

Acquiescence bias

Demand characteristics.

• Social desirability bias

Courtesy bias

• Question-order bias

Extreme responding

Acquiescence bias is the tendency of respondents to agree with a statement when faced with binary response options like “agree/disagree,” “yes/no,” or “true/false.” Acquiescence is sometimes referred to as “yea-saying.”

This type of bias occurs either due to the participant’s personality (i.e., some people are more likely to agree with statements than disagree, regardless of their content) or because participants perceive the researcher as an expert and are more inclined to agree with the statements presented to them.

Q: Are you a social person?

People who are inclined to agree with statements presented to them are at risk of selecting the first option, even if it isn’t fully supported by their lived experiences.

In order to control for acquiescence, consider tweaking your phrasing to encourage respondents to make a choice truly based on their preferences. Here’s an example:

Q: What would you prefer?

• A quiet night in
• A night out with friends

Demand characteristics are cues that could reveal the research agenda to participants, risking a change in their behaviors or views. Ensuring that participants are not aware of the research objectives is the best way to avoid this type of bias.

On each occasion, patients reported their pain as being less than prior to the operation. While at face value this seems to suggest that the operation does indeed lead to less pain, there is a demand characteristic at play. During the interviews, the researcher would unconsciously frown whenever patients reported more post-op pain. This increased the risk of patients figuring out that the researcher was hoping that the operation would have an advantageous effect.

Social desirability bias is the tendency of participants to give responses that they believe will be viewed favorably by the researcher or other participants. It often affects studies that focus on sensitive topics, such as alcohol consumption or sexual behavior.

You are conducting face-to-face semi-structured interviews with a number of employees from different departments. When asked whether they would be interested in a smoking cessation program, there was widespread enthusiasm for the idea.

Note that while social desirability and demand characteristics may sound similar, there is a key difference between them. Social desirability is about conforming to social norms, while demand characteristics revolve around the purpose of the research.

Courtesy bias stems from a reluctance to give negative feedback, so as to be polite to the person asking the question. Small-group interviewing where participants relate in some way to each other (e.g., a student, a teacher, and a dean) is especially prone to this type of bias.

Question order bias

Question order bias occurs when the order in which interview questions are asked influences the way the respondent interprets and evaluates them. This occurs especially when previous questions provide context for subsequent questions.

When answering subsequent questions, respondents may orient their answers to previous questions (called a halo effect ), which can lead to systematic distortion of the responses.

Extreme responding is the tendency of a respondent to answer in the extreme, choosing the lowest or highest response available, even if that is not their true opinion. Extreme responding is common in surveys using Likert scales , and it distorts people’s true attitudes and opinions.

Disposition towards the survey can be a source of extreme responding, as well as cultural components. For example, people coming from collectivist cultures tend to exhibit extreme responses in terms of agreement, while respondents indifferent to the questions asked may exhibit extreme responses in terms of disagreement.

Selection bias is a general term describing situations where bias is introduced into the research from factors affecting the study population.

Common types of selection bias are:

Sampling or ascertainment bias

• Attrition bias
• Self-selection (or volunteer) bias
• Survivorship bias
• Nonresponse bias
• Undercoverage bias

Sampling bias occurs when your sample (the individuals, groups, or data you obtain for your research) is selected in a way that is not representative of the population you are analyzing. Sampling bias threatens the external validity of your findings and influences the generalizability of your results.

The easiest way to prevent sampling bias is to use a probability sampling method . This way, each member of the population you are studying has an equal chance of being included in your sample.

Sampling bias is often referred to as ascertainment bias in the medical field.

Attrition bias occurs when participants who drop out of a study systematically differ from those who remain in the study. Attrition bias is especially problematic in randomized controlled trials for medical research because participants who do not like the experience or have unwanted side effects can drop out and affect your results.

You can minimize attrition bias by offering incentives for participants to complete the study (e.g., a gift card if they successfully attend every session). It’s also a good practice to recruit more participants than you need, or minimize the number of follow-up sessions or questions.

You provide a treatment group with weekly one-hour sessions over a two-month period, while a control group attends sessions on an unrelated topic. You complete five waves of data collection to compare outcomes: a pretest survey, three surveys during the program, and a posttest survey.

Self-selection or volunteer bias

Self-selection bias (also called volunteer bias ) occurs when individuals who volunteer for a study have particular characteristics that matter for the purposes of the study.

Volunteer bias leads to biased data, as the respondents who choose to participate will not represent your entire target population. You can avoid this type of bias by using random assignment —i.e., placing participants in a control group or a treatment group after they have volunteered to participate in the study.

Closely related to volunteer bias is nonresponse bias , which occurs when a research subject declines to participate in a particular study or drops out before the study’s completion.

Considering that the hospital is located in an affluent part of the city, volunteers are more likely to have a higher socioeconomic standing, higher education, and better nutrition than the general population.

Survivorship bias occurs when you do not evaluate your data set in its entirety: for example, by only analyzing the patients who survived a clinical trial.

This strongly increases the likelihood that you draw (incorrect) conclusions based upon those who have passed some sort of selection process—focusing on “survivors” and forgetting those who went through a similar process and did not survive.

Note that “survival” does not always mean that participants died! Rather, it signifies that participants did not successfully complete the intervention.

However, most college dropouts do not become billionaires. In fact, there are many more aspiring entrepreneurs who dropped out of college to start companies and failed than succeeded.

Nonresponse bias occurs when those who do not respond to a survey or research project are different from those who do in ways that are critical to the goals of the research. This is very common in survey research, when participants are unable or unwilling to participate due to factors like lack of the necessary skills, lack of time, or guilt or shame related to the topic.

You can mitigate nonresponse bias by offering the survey in different formats (e.g., an online survey, but also a paper version sent via post), ensuring confidentiality , and sending them reminders to complete the survey.

You notice that your surveys were conducted during business hours, when the working-age residents were less likely to be home.

Undercoverage bias occurs when you only sample from a subset of the population you are interested in. Online surveys can be particularly susceptible to undercoverage bias. Despite being more cost-effective than other methods, they can introduce undercoverage bias as a result of excluding people who do not use the internet.

Cognitive bias refers to a set of predictable (i.e., nonrandom) errors in thinking that arise from our limited ability to process information objectively. Rather, our judgment is influenced by our values, memories, and other personal traits. These create “ mental shortcuts” that help us process information intuitively and decide faster. However, cognitive bias can also cause us to misunderstand or misinterpret situations, information, or other people.

Because of cognitive bias, people often perceive events to be more predictable after they happen.

Although there is no general agreement on how many types of cognitive bias exist, some common types are:

• Anchoring bias  
• Framing effect  
• Actor-observer bias
• Availability heuristic (or availability bias)
• Confirmation bias  
• Halo effect
• The Baader-Meinhof phenomenon  

Anchoring bias

Anchoring bias is people’s tendency to fixate on the first piece of information they receive, especially when it concerns numbers. This piece of information becomes a reference point or anchor. Because of that, people base all subsequent decisions on this anchor. For example, initial offers have a stronger influence on the outcome of negotiations than subsequent ones.

• Framing effect

Framing effect refers to our tendency to decide based on how the information about the decision is presented to us. In other words, our response depends on whether the option is presented in a negative or positive light, e.g., gain or loss, reward or punishment, etc. This means that the same information can be more or less attractive depending on the wording or what features are highlighted.

Actor–observer bias

Actor–observer bias occurs when you attribute the behavior of others to internal factors, like skill or personality, but attribute your own behavior to external or situational factors.

In other words, when you are the actor in a situation, you are more likely to link events to external factors, such as your surroundings or environment. However, when you are observing the behavior of others, you are more likely to associate behavior with their personality, nature, or temperament.

One interviewee recalls a morning when it was raining heavily. They were rushing to drop off their kids at school in order to get to work on time. As they were driving down the highway, another car cut them off as they were trying to merge. They tell you how frustrated they felt and exclaim that the other driver must have been a very rude person.

At another point, the same interviewee recalls that they did something similar: accidentally cutting off another driver while trying to take the correct exit. However, this time, the interviewee claimed that they always drive very carefully, blaming their mistake on poor visibility due to the rain.

• Availability heuristic

Availability heuristic (or availability bias) describes the tendency to evaluate a topic using the information we can quickly recall to our mind, i.e., that is available to us. However, this is not necessarily the best information, rather it’s the most vivid or recent. Even so, due to this mental shortcut, we tend to think that what we can recall must be right and ignore any other information.

• Confirmation bias

Confirmation bias is the tendency to seek out information in a way that supports our existing beliefs while also rejecting any information that contradicts those beliefs. Confirmation bias is often unintentional but still results in skewed results and poor decision-making.

Let’s say you grew up with a parent in the military. Chances are that you have a lot of complex emotions around overseas deployments. This can lead you to over-emphasize findings that “prove” that your lived experience is the case for most families, neglecting other explanations and experiences.

The halo effect refers to situations whereby our general impression about a person, a brand, or a product is shaped by a single trait. It happens, for instance, when we automatically make positive assumptions about people based on something positive we notice, while in reality, we know little about them.

The Baader-Meinhof phenomenon

The Baader-Meinhof phenomenon (or frequency illusion) occurs when something that you recently learned seems to appear “everywhere” soon after it was first brought to your attention. However, this is not the case. What has increased is your awareness of something, such as a new word or an old song you never knew existed, not their frequency.

While very difficult to eliminate entirely, research bias can be mitigated through proper study design and implementation. Here are some tips to keep in mind as you get started.

• Clearly explain in your methodology section how your research design will help you meet the research objectives and why this is the most appropriate research design.
• In quantitative studies , make sure that you use probability sampling to select the participants. If you’re running an experiment, make sure you use random assignment to assign your control and treatment groups.
• Account for participants who withdraw or are lost to follow-up during the study. If they are withdrawing for a particular reason, it could bias your results. This applies especially to longer-term or longitudinal studies .
• Use triangulation to enhance the validity and credibility of your findings.
• Phrase your survey or interview questions in a neutral, non-judgmental tone. Be very careful that your questions do not steer your participants in any particular direction.
• Consider using a reflexive journal. Here, you can log the details of each interview , paying special attention to any influence you may have had on participants. You can include these in your final analysis.
• Baader–Meinhof phenomenon
• Sampling bias
• Ascertainment bias
• Self-selection bias
• Hawthorne effect
• Omitted variable bias
• Pygmalion effect
• Placebo effect

Research bias affects the validity and reliability of your research findings , leading to false conclusions and a misinterpretation of the truth. This can have serious implications in areas like medical research where, for example, a new form of treatment may be evaluated.

Observer bias occurs when the researcher’s assumptions, views, or preconceptions influence what they see and record in a study, while actor–observer bias refers to situations where respondents attribute internal factors (e.g., bad character) to justify other’s behavior and external factors (difficult circumstances) to justify the same behavior in themselves.

Response bias is a general term used to describe a number of different conditions or factors that cue respondents to provide inaccurate or false answers during surveys or interviews. These factors range from the interviewer’s perceived social position or appearance to the the phrasing of questions in surveys.

Nonresponse bias occurs when the people who complete a survey are different from those who did not, in ways that are relevant to the research topic. Nonresponse can happen because people are either not willing or not able to participate.

Is this article helpful?

Other students also liked.

• Attrition Bias | Examples, Explanation, Prevention
• Observer Bias | Definition, Examples, Prevention
• What Is Social Desirability Bias? | Definition & Examples

More interesting articles

• Demand Characteristics | Definition, Examples & Control
• Hostile Attribution Bias | Definition & Examples
• Regression to the Mean | Definition & Examples
• Representativeness Heuristic | Example & Definition
• Sampling Bias and How to Avoid It | Types & Examples
• Self-Fulfilling Prophecy | Definition & Examples
• The Availability Heuristic | Example & Definition
• The Baader–Meinhof Phenomenon Explained
• What Is a Ceiling Effect? | Definition & Examples
• What Is Actor-Observer Bias? | Definition & Examples
• What Is Affinity Bias? | Definition & Examples
• What Is Anchoring Bias? | Definition & Examples
• What Is Ascertainment Bias? | Definition & Examples
• What Is Belief Bias? | Definition & Examples
• What Is Bias for Action? | Definition & Examples
• What Is Cognitive Bias? | Definition, Types, & Examples
• What Is Confirmation Bias? | Definition & Examples
• What Is Conformity Bias? | Definition & Examples
• What Is Correspondence Bias? | Definition & Example
• What Is Explicit Bias? | Definition & Examples
• What Is Generalizability? | Definition & Examples
• What Is Hindsight Bias? | Definition & Examples
• What Is Implicit Bias? | Definition & Examples
• What Is Information Bias? | Definition & Examples
• What Is Ingroup Bias? | Definition & Examples
• What Is Negativity Bias? | Definition & Examples
• What Is Nonresponse Bias? | Definition & Example
• What Is Normalcy Bias? | Definition & Example
• What Is Omitted Variable Bias? | Definition & Examples
• What Is Optimism Bias? | Definition & Examples
• What Is Outgroup Bias? | Definition & Examples
• What Is Overconfidence Bias? | Definition & Examples
• What Is Perception Bias? | Definition & Examples
• What Is Primacy Bias? | Definition & Example
• What Is Publication Bias? | Definition & Examples
• What Is Recall Bias? | Definition & Examples
• What Is Recency Bias? | Definition & Examples
• What Is Response Bias? | Definition & Examples
• What Is Selection Bias? | Definition & Examples
• What Is Self-Selection Bias? | Definition & Example
• What Is Self-Serving Bias? | Definition & Example
• What Is Status Quo Bias? | Definition & Examples
• What Is Survivorship Bias? | Definition & Examples
• What Is the Affect Heuristic? | Example & Definition
• What Is the Egocentric Bias? | Definition & Examples
• What Is the Framing Effect? | Definition & Examples
• What Is the Halo Effect? | Definition & Examples
• What Is the Hawthorne Effect? | Definition & Examples
• What Is the Placebo Effect? | Definition & Examples
• What Is the Pygmalion Effect? | Definition & Examples
• What Is Unconscious Bias? | Definition & Examples
• What Is Undercoverage Bias? | Definition & Example
• What Is Vividness Bias? | Definition & Examples