research questions research objectives

Research Aims, Objectives & Questions

The “Golden Thread” Explained Simply (+ Examples)

By: David Phair (PhD) and Alexandra Shaeffer (PhD) | June 2022

The research aims , objectives and research questions (collectively called the “golden thread”) are arguably the most important thing you need to get right when you’re crafting a research proposal , dissertation or thesis . We receive questions almost every day about this “holy trinity” of research and there’s certainly a lot of confusion out there, so we’ve crafted this post to help you navigate your way through the fog.

Overview: The Golden Thread

• What is the golden thread
• What are research aims ( examples )
• What are research objectives ( examples )
• What are research questions ( examples )
• The importance of alignment in the golden thread

What is the “golden thread”?  

The golden thread simply refers to the collective research aims , research objectives , and research questions for any given project (i.e., a dissertation, thesis, or research paper ). These three elements are bundled together because it’s extremely important that they align with each other, and that the entire research project aligns with them.

Importantly, the golden thread needs to weave its way through the entirety of any research project , from start to end. In other words, it needs to be very clearly defined right at the beginning of the project (the topic ideation and proposal stage) and it needs to inform almost every decision throughout the rest of the project. For example, your research design and methodology will be heavily influenced by the golden thread (we’ll explain this in more detail later), as well as your literature review.

The research aims, objectives and research questions (the golden thread) define the focus and scope ( the delimitations ) of your research project. In other words, they help ringfence your dissertation or thesis to a relatively narrow domain, so that you can “go deep” and really dig into a specific problem or opportunity. They also help keep you on track , as they act as a litmus test for relevance. In other words, if you’re ever unsure whether to include something in your document, simply ask yourself the question, “does this contribute toward my research aims, objectives or questions?”. If it doesn’t, chances are you can drop it.

Alright, enough of the fluffy, conceptual stuff. Let’s get down to business and look at what exactly the research aims, objectives and questions are and outline a few examples to bring these concepts to life.

Research Aims: What are they?

Simply put, the research aim(s) is a statement that reflects the broad overarching goal (s) of the research project. Research aims are fairly high-level (low resolution) as they outline the general direction of the research and what it’s trying to achieve .

Research Aims: Examples  

True to the name, research aims usually start with the wording “this research aims to…”, “this research seeks to…”, and so on. For example:

“This research aims to explore employee experiences of digital transformation in retail HR.”   “This study sets out to assess the interaction between student support and self-care on well-being in engineering graduate students”  

As you can see, these research aims provide a high-level description of what the study is about and what it seeks to achieve. They’re not hyper-specific or action-oriented, but they’re clear about what the study’s focus is and what is being investigated.

Need a helping hand?

Research Objectives: What are they?

The research objectives take the research aims and make them more practical and actionable . In other words, the research objectives showcase the steps that the researcher will take to achieve the research aims.

The research objectives need to be far more specific (higher resolution) and actionable than the research aims. In fact, it’s always a good idea to craft your research objectives using the “SMART” criteria. In other words, they should be specific, measurable, achievable, relevant and time-bound”.

Research Objectives: Examples  

Let’s look at two examples of research objectives. We’ll stick with the topic and research aims we mentioned previously.  

For the digital transformation topic:

To observe the retail HR employees throughout the digital transformation. To assess employee perceptions of digital transformation in retail HR. To identify the barriers and facilitators of digital transformation in retail HR.

And for the student wellness topic:

To determine whether student self-care predicts the well-being score of engineering graduate students. To determine whether student support predicts the well-being score of engineering students. To assess the interaction between student self-care and student support when predicting well-being in engineering graduate students.

  As you can see, these research objectives clearly align with the previously mentioned research aims and effectively translate the low-resolution aims into (comparatively) higher-resolution objectives and action points . They give the research project a clear focus and present something that resembles a research-based “to-do” list.

Research Questions: What are they?

Finally, we arrive at the all-important research questions. The research questions are, as the name suggests, the key questions that your study will seek to answer . Simply put, they are the core purpose of your dissertation, thesis, or research project. You’ll present them at the beginning of your document (either in the introduction chapter or literature review chapter) and you’ll answer them at the end of your document (typically in the discussion and conclusion chapters).  

The research questions will be the driving force throughout the research process. For example, in the literature review chapter, you’ll assess the relevance of any given resource based on whether it helps you move towards answering your research questions. Similarly, your methodology and research design will be heavily influenced by the nature of your research questions. For instance, research questions that are exploratory in nature will usually make use of a qualitative approach, whereas questions that relate to measurement or relationship testing will make use of a quantitative approach.  

Let’s look at some examples of research questions to make this more tangible.

Research Questions: Examples  

Again, we’ll stick with the research aims and research objectives we mentioned previously.  

For the digital transformation topic (which would be qualitative in nature):

How do employees perceive digital transformation in retail HR? What are the barriers and facilitators of digital transformation in retail HR?  

And for the student wellness topic (which would be quantitative in nature):

Does student self-care predict the well-being scores of engineering graduate students? Does student support predict the well-being scores of engineering students? Do student self-care and student support interact when predicting well-being in engineering graduate students?  

You’ll probably notice that there’s quite a formulaic approach to this. In other words, the research questions are basically the research objectives “converted” into question format. While that is true most of the time, it’s not always the case. For example, the first research objective for the digital transformation topic was more or less a step on the path toward the other objectives, and as such, it didn’t warrant its own research question.  

So, don’t rush your research questions and sloppily reword your objectives as questions. Carefully think about what exactly you’re trying to achieve (i.e. your research aim) and the objectives you’ve set out, then craft a set of well-aligned research questions . Also, keep in mind that this can be a somewhat iterative process , where you go back and tweak research objectives and aims to ensure tight alignment throughout the golden thread.

The importance of strong alignment 

Alignment is the keyword here and we have to stress its importance . Simply put, you need to make sure that there is a very tight alignment between all three pieces of the golden thread. If your research aims and research questions don’t align, for example, your project will be pulling in different directions and will lack focus . This is a common problem students face and can cause many headaches (and tears), so be warned.

Take the time to carefully craft your research aims, objectives and research questions before you run off down the research path. Ideally, get your research supervisor/advisor to review and comment on your golden thread before you invest significant time into your project, and certainly before you start collecting data .  

Recap: The golden thread

In this post, we unpacked the golden thread of research, consisting of the research aims , research objectives and research questions . You can jump back to any section using the links below.

As always, feel free to leave a comment below – we always love to hear from you. Also, if you’re interested in 1-on-1 support, take a look at our private coaching service here.

Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

You Might Also Like:

39 Comments

Thank you very much for your great effort put. As an Undergraduate taking Demographic Research & Methodology, I’ve been trying so hard to understand clearly what is a Research Question, Research Aim and the Objectives in a research and the relationship between them etc. But as for now I’m thankful that you’ve solved my problem.

Well appreciated. This has helped me greatly in doing my dissertation.

An so delighted with this wonderful information thank you a lot.

so impressive i have benefited a lot looking forward to learn more on research.

I am very happy to have carefully gone through this well researched article.

Infact,I used to be phobia about anything research, because of my poor understanding of the concepts.

Now,I get to know that my research question is the same as my research objective(s) rephrased in question format.

I please I would need a follow up on the subject,as I intends to join the team of researchers. Thanks once again.

Thanks so much. This was really helpful.

I know you pepole have tried to break things into more understandable and easy format. And God bless you. Keep it up

i found this document so useful towards my study in research methods. thanks so much.

This is my 2nd read topic in your course and I should commend the simplified explanations of each part. I’m beginning to understand and absorb the use of each part of a dissertation/thesis. I’ll keep on reading your free course and might be able to avail the training course! Kudos!

Thank you! Better put that my lecture and helped to easily understand the basics which I feel often get brushed over when beginning dissertation work.

This is quite helpful. I like how the Golden thread has been explained and the needed alignment.

This is quite helpful. I really appreciate!

The article made it simple for researcher students to differentiate between three concepts.

Very innovative and educational in approach to conducting research.

I am very impressed with all these terminology, as I am a fresh student for post graduate, I am highly guided and I promised to continue making consultation when the need arise. Thanks a lot.

A very helpful piece. thanks, I really appreciate it .

Very well explained, and it might be helpful to many people like me.

Wish i had found this (and other) resource(s) at the beginning of my PhD journey… not in my writing up year… 😩 Anyways… just a quick question as i’m having some issues ordering my “golden thread”…. does it matter in what order you mention them? i.e., is it always first aims, then objectives, and finally the questions? or can you first mention the research questions and then the aims and objectives?

Thank you for a very simple explanation that builds upon the concepts in a very logical manner. Just prior to this, I read the research hypothesis article, which was equally very good. This met my primary objective.

My secondary objective was to understand the difference between research questions and research hypothesis, and in which context to use which one. However, I am still not clear on this. Can you kindly please guide?

In research, a research question is a clear and specific inquiry that the researcher wants to answer, while a research hypothesis is a tentative statement or prediction about the relationship between variables or the expected outcome of the study. Research questions are broader and guide the overall study, while hypotheses are specific and testable statements used in quantitative research. Research questions identify the problem, while hypotheses provide a focus for testing in the study.

Exactly what I need in this research journey, I look forward to more of your coaching videos.

This helped a lot. Thanks so much for the effort put into explaining it.

What data source in writing dissertation/Thesis requires?

What is data source covers when writing dessertation/thesis

This is quite useful thanks

I’m excited and thankful. I got so much value which will help me progress in my thesis.

where are the locations of the reserch statement, research objective and research question in a reserach paper? Can you write an ouline that defines their places in the researh paper?

Very helpful and important tips on Aims, Objectives and Questions.

Thank you so much for making research aim, research objectives and research question so clear. This will be helpful to me as i continue with my thesis.

Thanks much for this content. I learned a lot. And I am inspired to learn more. I am still struggling with my preparation for dissertation outline/proposal. But I consistently follow contents and tutorials and the new FB of GRAD Coach. Hope to really become confident in writing my dissertation and successfully defend it.

As a researcher and lecturer, I find splitting research goals into research aims, objectives, and questions is unnecessarily bureaucratic and confusing for students. For most biomedical research projects, including ‘real research’, 1-3 research questions will suffice (numbers may differ by discipline).

Awesome! Very important resources and presented in an informative way to easily understand the golden thread. Indeed, thank you so much.

Well explained

The blog article on research aims, objectives, and questions by Grad Coach is a clear and insightful guide that aligns with my experiences in academic research. The article effectively breaks down the often complex concepts of research aims and objectives, providing a straightforward and accessible explanation. Drawing from my own research endeavors, I appreciate the practical tips offered, such as the need for specificity and clarity when formulating research questions. The article serves as a valuable resource for students and researchers, offering a concise roadmap for crafting well-defined research goals and objectives. Whether you’re a novice or an experienced researcher, this article provides practical insights that contribute to the foundational aspects of a successful research endeavor.

A great thanks for you. it is really amazing explanation. I grasp a lot and one step up to research knowledge.

I really found these tips helpful. Thank you very much Grad Coach.

I found this article helpful. Thanks for sharing this.

thank you so much, the explanation and examples are really helpful

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Research process
• Writing Strong Research Questions | Criteria & Examples

Writing Strong Research Questions | Criteria & Examples

Published on 30 October 2022 by Shona McCombes . Revised on 12 December 2023.

A research question pinpoints exactly what you want to find out in your work. A good research question is essential to guide your research paper , dissertation , or thesis .

All research questions should be:

• Focused on a single problem or issue
• Researchable using primary and/or secondary sources
• Feasible to answer within the timeframe and practical constraints
• Specific enough to answer thoroughly
• Complex enough to develop the answer over the space of a paper or thesis
• Relevant to your field of study and/or society more broadly

Table of contents

How to write a research question, what makes a strong research question, research questions quiz, frequently asked questions.

You can follow these steps to develop a strong research question:

• Choose your topic
• Do some preliminary reading about the current state of the field
• Narrow your focus to a specific niche
• Identify the research problem that you will address

The way you frame your question depends on what your research aims to achieve. The table below shows some examples of how you might formulate questions for different purposes.

Using your research problem to develop your research question

Note that while most research questions can be answered with various types of research , the way you frame your question should help determine your choices.

Prevent plagiarism, run a free check.

Research questions anchor your whole project, so it’s important to spend some time refining them. The criteria below can help you evaluate the strength of your research question.

Focused and researchable

Feasible and specific, complex and arguable, relevant and original.

The way you present your research problem in your introduction varies depending on the nature of your research paper . A research paper that presents a sustained argument will usually encapsulate this argument in a thesis statement .

A research paper designed to present the results of empirical research tends to present a research question that it seeks to answer. It may also include a hypothesis – a prediction that will be confirmed or disproved by your research.

As you cannot possibly read every source related to your topic, it’s important to evaluate sources to assess their relevance. Use preliminary evaluation to determine whether a source is worth examining in more depth.

This involves:

• Reading abstracts , prefaces, introductions , and conclusions
• Looking at the table of contents to determine the scope of the work
• Consulting the index for key terms or the names of important scholars

An essay isn’t just a loose collection of facts and ideas. Instead, it should be centered on an overarching argument (summarised in your thesis statement ) that every part of the essay relates to.

The way you structure your essay is crucial to presenting your argument coherently. A well-structured essay helps your reader follow the logic of your ideas and understand your overall point.

A research hypothesis is your proposed answer to your research question. The research hypothesis usually includes an explanation (‘ x affects y because …’).

A statistical hypothesis, on the other hand, is a mathematical statement about a population parameter. Statistical hypotheses always come in pairs: the null and alternative hypotheses. In a well-designed study , the statistical hypotheses correspond logically to the research hypothesis.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2023, December 12). Writing Strong Research Questions | Criteria & Examples. Scribbr. Retrieved 14 May 2024, from https://www.scribbr.co.uk/the-research-process/research-question/

Is this article helpful?

Shona McCombes

Other students also liked, how to write a research proposal | examples & templates, how to write a results section | tips & examples, what is a research methodology | steps & tips.

• Privacy Policy

Home » Research Questions – Types, Examples and Writing Guide

Research Questions – Types, Examples and Writing Guide

Table of Contents

Research Questions

Definition:

Research questions are the specific questions that guide a research study or inquiry. These questions help to define the scope of the research and provide a clear focus for the study. Research questions are usually developed at the beginning of a research project and are designed to address a particular research problem or objective.

Types of Research Questions

Types of Research Questions are as follows:

Descriptive Research Questions

These aim to describe a particular phenomenon, group, or situation. For example:

• What are the characteristics of the target population?
• What is the prevalence of a particular disease in a specific region?

Exploratory Research Questions

These aim to explore a new area of research or generate new ideas or hypotheses. For example:

• What are the potential causes of a particular phenomenon?
• What are the possible outcomes of a specific intervention?

Explanatory Research Questions

These aim to understand the relationship between two or more variables or to explain why a particular phenomenon occurs. For example:

• What is the effect of a specific drug on the symptoms of a particular disease?
• What are the factors that contribute to employee turnover in a particular industry?

Predictive Research Questions

These aim to predict a future outcome or trend based on existing data or trends. For example :

• What will be the future demand for a particular product or service?
• What will be the future prevalence of a particular disease?

Evaluative Research Questions

These aim to evaluate the effectiveness of a particular intervention or program. For example:

• What is the impact of a specific educational program on student learning outcomes?
• What is the effectiveness of a particular policy or program in achieving its intended goals?

How to Choose Research Questions

Choosing research questions is an essential part of the research process and involves careful consideration of the research problem, objectives, and design. Here are some steps to consider when choosing research questions:

• Identify the research problem: Start by identifying the problem or issue that you want to study. This could be a gap in the literature, a social or economic issue, or a practical problem that needs to be addressed.
• Conduct a literature review: Conducting a literature review can help you identify existing research in your area of interest and can help you formulate research questions that address gaps or limitations in the existing literature.
• Define the research objectives : Clearly define the objectives of your research. What do you want to achieve with your study? What specific questions do you want to answer?
• Consider the research design : Consider the research design that you plan to use. This will help you determine the appropriate types of research questions to ask. For example, if you plan to use a qualitative approach, you may want to focus on exploratory or descriptive research questions.
• Ensure that the research questions are clear and answerable: Your research questions should be clear and specific, and should be answerable with the data that you plan to collect. Avoid asking questions that are too broad or vague.
• Get feedback : Get feedback from your supervisor, colleagues, or peers to ensure that your research questions are relevant, feasible, and meaningful.

How to Write Research Questions

Guide for Writing Research Questions:

• Start with a clear statement of the research problem: Begin by stating the problem or issue that your research aims to address. This will help you to formulate focused research questions.
• Use clear language : Write your research questions in clear and concise language that is easy to understand. Avoid using jargon or technical terms that may be unfamiliar to your readers.
• Be specific: Your research questions should be specific and focused. Avoid broad questions that are difficult to answer. For example, instead of asking “What is the impact of climate change on the environment?” ask “What are the effects of rising sea levels on coastal ecosystems?”
• Use appropriate question types: Choose the appropriate question types based on the research design and objectives. For example, if you are conducting a qualitative study, you may want to use open-ended questions that allow participants to provide detailed responses.
• Consider the feasibility of your questions : Ensure that your research questions are feasible and can be answered with the resources available. Consider the data sources and methods of data collection when writing your questions.
• Seek feedback: Get feedback from your supervisor, colleagues, or peers to ensure that your research questions are relevant, appropriate, and meaningful.

Examples of Research Questions

Some Examples of Research Questions with Research Titles:

Research Title: The Impact of Social Media on Mental Health

• Research Question : What is the relationship between social media use and mental health, and how does this impact individuals’ well-being?

Research Title: Factors Influencing Academic Success in High School

• Research Question: What are the primary factors that influence academic success in high school, and how do they contribute to student achievement?

Research Title: The Effects of Exercise on Physical and Mental Health

• Research Question: What is the relationship between exercise and physical and mental health, and how can exercise be used as a tool to improve overall well-being?

Research Title: Understanding the Factors that Influence Consumer Purchasing Decisions

• Research Question : What are the key factors that influence consumer purchasing decisions, and how do these factors vary across different demographics and products?

Research Title: The Impact of Technology on Communication

• Research Question : How has technology impacted communication patterns, and what are the effects of these changes on interpersonal relationships and society as a whole?

Research Title: Investigating the Relationship between Parenting Styles and Child Development

• Research Question: What is the relationship between different parenting styles and child development outcomes, and how do these outcomes vary across different ages and developmental stages?

Research Title: The Effectiveness of Cognitive-Behavioral Therapy in Treating Anxiety Disorders

• Research Question: How effective is cognitive-behavioral therapy in treating anxiety disorders, and what factors contribute to its success or failure in different patients?

Research Title: The Impact of Climate Change on Biodiversity

• Research Question : How is climate change affecting global biodiversity, and what can be done to mitigate the negative effects on natural ecosystems?

Research Title: Exploring the Relationship between Cultural Diversity and Workplace Productivity

• Research Question : How does cultural diversity impact workplace productivity, and what strategies can be employed to maximize the benefits of a diverse workforce?

Research Title: The Role of Artificial Intelligence in Healthcare

• Research Question: How can artificial intelligence be leveraged to improve healthcare outcomes, and what are the potential risks and ethical concerns associated with its use?

Applications of Research Questions

Here are some of the key applications of research questions:

• Defining the scope of the study : Research questions help researchers to narrow down the scope of their study and identify the specific issues they want to investigate.
• Developing hypotheses: Research questions often lead to the development of hypotheses, which are testable predictions about the relationship between variables. Hypotheses provide a clear and focused direction for the study.
• Designing the study : Research questions guide the design of the study, including the selection of participants, the collection of data, and the analysis of results.
• Collecting data : Research questions inform the selection of appropriate methods for collecting data, such as surveys, interviews, or experiments.
• Analyzing data : Research questions guide the analysis of data, including the selection of appropriate statistical tests and the interpretation of results.
• Communicating results : Research questions help researchers to communicate the results of their study in a clear and concise manner. The research questions provide a framework for discussing the findings and drawing conclusions.

Characteristics of Research Questions

Characteristics of Research Questions are as follows:

• Clear and Specific : A good research question should be clear and specific. It should clearly state what the research is trying to investigate and what kind of data is required.
• Relevant : The research question should be relevant to the study and should address a current issue or problem in the field of research.
• Testable : The research question should be testable through empirical evidence. It should be possible to collect data to answer the research question.
• Concise : The research question should be concise and focused. It should not be too broad or too narrow.
• Feasible : The research question should be feasible to answer within the constraints of the research design, time frame, and available resources.
• Original : The research question should be original and should contribute to the existing knowledge in the field of research.
• Significant : The research question should have significance and importance to the field of research. It should have the potential to provide new insights and knowledge to the field.
• Ethical : The research question should be ethical and should not cause harm to any individuals or groups involved in the study.

Purpose of Research Questions

Research questions are the foundation of any research study as they guide the research process and provide a clear direction to the researcher. The purpose of research questions is to identify the scope and boundaries of the study, and to establish the goals and objectives of the research.

The main purpose of research questions is to help the researcher to focus on the specific area or problem that needs to be investigated. They enable the researcher to develop a research design, select the appropriate methods and tools for data collection and analysis, and to organize the results in a meaningful way.

Research questions also help to establish the relevance and significance of the study. They define the research problem, and determine the research methodology that will be used to address the problem. Research questions also help to determine the type of data that will be collected, and how it will be analyzed and interpreted.

Finally, research questions provide a framework for evaluating the results of the research. They help to establish the validity and reliability of the data, and provide a basis for drawing conclusions and making recommendations based on the findings of the study.

Advantages of Research Questions

There are several advantages of research questions in the research process, including:

• Focus : Research questions help to focus the research by providing a clear direction for the study. They define the specific area of investigation and provide a framework for the research design.
• Clarity : Research questions help to clarify the purpose and objectives of the study, which can make it easier for the researcher to communicate the research aims to others.
• Relevance : Research questions help to ensure that the study is relevant and meaningful. By asking relevant and important questions, the researcher can ensure that the study will contribute to the existing body of knowledge and address important issues.
• Consistency : Research questions help to ensure consistency in the research process by providing a framework for the development of the research design, data collection, and analysis.
• Measurability : Research questions help to ensure that the study is measurable by defining the specific variables and outcomes that will be measured.
• Replication : Research questions help to ensure that the study can be replicated by providing a clear and detailed description of the research aims, methods, and outcomes. This makes it easier for other researchers to replicate the study and verify the results.

Limitations of Research Questions

Limitations of Research Questions are as follows:

• Subjectivity : Research questions are often subjective and can be influenced by personal biases and perspectives of the researcher. This can lead to a limited understanding of the research problem and may affect the validity and reliability of the study.
• Inadequate scope : Research questions that are too narrow in scope may limit the breadth of the study, while questions that are too broad may make it difficult to focus on specific research objectives.
• Unanswerable questions : Some research questions may not be answerable due to the lack of available data or limitations in research methods. In such cases, the research question may need to be rephrased or modified to make it more answerable.
• Lack of clarity : Research questions that are poorly worded or ambiguous can lead to confusion and misinterpretation. This can result in incomplete or inaccurate data, which may compromise the validity of the study.
• Difficulty in measuring variables : Some research questions may involve variables that are difficult to measure or quantify, making it challenging to draw meaningful conclusions from the data.
• Lack of generalizability: Research questions that are too specific or limited in scope may not be generalizable to other contexts or populations. This can limit the applicability of the study’s findings and restrict its broader implications.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Data Collection – Methods Types and Examples

Delimitations in Research – Types, Examples and...

Research Process – Steps, Examples and Tips

Research Design – Types, Methods and Examples

Institutional Review Board – Application Sample...

Evaluating Research – Process, Examples and...

Ohio State nav bar

The Ohio State University

• BuckeyeLink
• Find People
• Search Ohio State

Research Questions & Hypotheses

Generally, in quantitative studies, reviewers expect hypotheses rather than research questions. However, both research questions and hypotheses serve different purposes and can be beneficial when used together.

Research Questions

Clarify the research’s aim (farrugia et al., 2010).

• Research often begins with an interest in a topic, but a deep understanding of the subject is crucial to formulate an appropriate research question.
• Descriptive: “What factors most influence the academic achievement of senior high school students?”
• Comparative: “What is the performance difference between teaching methods A and B?”
• Relationship-based: “What is the relationship between self-efficacy and academic achievement?”
• Increasing knowledge about a subject can be achieved through systematic literature reviews, in-depth interviews with patients (and proxies), focus groups, and consultations with field experts.
• Some funding bodies, like the Canadian Institute for Health Research, recommend conducting a systematic review or a pilot study before seeking grants for full trials.
• The presence of multiple research questions in a study can complicate the design, statistical analysis, and feasibility.
• It’s advisable to focus on a single primary research question for the study.
• The primary question, clearly stated at the end of a grant proposal’s introduction, usually specifies the study population, intervention, and other relevant factors.
• The FINER criteria underscore aspects that can enhance the chances of a successful research project, including specifying the population of interest, aligning with scientific and public interest, clinical relevance, and contribution to the field, while complying with ethical and national research standards.
• The P ICOT approach is crucial in developing the study’s framework and protocol, influencing inclusion and exclusion criteria and identifying patient groups for inclusion.
• Defining the specific population, intervention, comparator, and outcome helps in selecting the right outcome measurement tool.
• The more precise the population definition and stricter the inclusion and exclusion criteria, the more significant the impact on the interpretation, applicability, and generalizability of the research findings.
• A restricted study population enhances internal validity but may limit the study’s external validity and generalizability to clinical practice.
• A broadly defined study population may better reflect clinical practice but could increase bias and reduce internal validity.
• An inadequately formulated research question can negatively impact study design, potentially leading to ineffective outcomes and affecting publication prospects.

Checklist: Good research questions for social science projects (Panke, 2018)

Research Hypotheses

Present the researcher’s predictions based on specific statements.

• These statements define the research problem or issue and indicate the direction of the researcher’s predictions.
• Formulating the research question and hypothesis from existing data (e.g., a database) can lead to multiple statistical comparisons and potentially spurious findings due to chance.
• The research or clinical hypothesis, derived from the research question, shapes the study’s key elements: sampling strategy, intervention, comparison, and outcome variables.
• Hypotheses can express a single outcome or multiple outcomes.
• After statistical testing, the null hypothesis is either rejected or not rejected based on whether the study’s findings are statistically significant.
• Hypothesis testing helps determine if observed findings are due to true differences and not chance.
• Hypotheses can be 1-sided (specific direction of difference) or 2-sided (presence of a difference without specifying direction).
• 2-sided hypotheses are generally preferred unless there’s a strong justification for a 1-sided hypothesis.
• A solid research hypothesis, informed by a good research question, influences the research design and paves the way for defining clear research objectives.

Types of Research Hypothesis

• In a Y-centered research design, the focus is on the dependent variable (DV) which is specified in the research question. Theories are then used to identify independent variables (IV) and explain their causal relationship with the DV.
• Example: “An increase in teacher-led instructional time (IV) is likely to improve student reading comprehension scores (DV), because extensive guided practice under expert supervision enhances learning retention and skill mastery.”
• Hypothesis Explanation: The dependent variable (student reading comprehension scores) is the focus, and the hypothesis explores how changes in the independent variable (teacher-led instructional time) affect it.
• In X-centered research designs, the independent variable is specified in the research question. Theories are used to determine potential dependent variables and the causal mechanisms at play.
• Example: “Implementing technology-based learning tools (IV) is likely to enhance student engagement in the classroom (DV), because interactive and multimedia content increases student interest and participation.”
• Hypothesis Explanation: The independent variable (technology-based learning tools) is the focus, with the hypothesis exploring its impact on a potential dependent variable (student engagement).
• Probabilistic hypotheses suggest that changes in the independent variable are likely to lead to changes in the dependent variable in a predictable manner, but not with absolute certainty.
• Example: “The more teachers engage in professional development programs (IV), the more their teaching effectiveness (DV) is likely to improve, because continuous training updates pedagogical skills and knowledge.”
• Hypothesis Explanation: This hypothesis implies a probable relationship between the extent of professional development (IV) and teaching effectiveness (DV).
• Deterministic hypotheses state that a specific change in the independent variable will lead to a specific change in the dependent variable, implying a more direct and certain relationship.
• Example: “If the school curriculum changes from traditional lecture-based methods to project-based learning (IV), then student collaboration skills (DV) are expected to improve because project-based learning inherently requires teamwork and peer interaction.”
• Hypothesis Explanation: This hypothesis presumes a direct and definite outcome (improvement in collaboration skills) resulting from a specific change in the teaching method.
• Example : “Students who identify as visual learners will score higher on tests that are presented in a visually rich format compared to tests presented in a text-only format.”
• Explanation : This hypothesis aims to describe the potential difference in test scores between visual learners taking visually rich tests and text-only tests, without implying a direct cause-and-effect relationship.
• Example : “Teaching method A will improve student performance more than method B.”
• Explanation : This hypothesis compares the effectiveness of two different teaching methods, suggesting that one will lead to better student performance than the other. It implies a direct comparison but does not necessarily establish a causal mechanism.
• Example : “Students with higher self-efficacy will show higher levels of academic achievement.”
• Explanation : This hypothesis predicts a relationship between the variable of self-efficacy and academic achievement. Unlike a causal hypothesis, it does not necessarily suggest that one variable causes changes in the other, but rather that they are related in some way.

Tips for developing research questions and hypotheses for research studies

• Perform a systematic literature review (if one has not been done) to increase knowledge and familiarity with the topic and to assist with research development.
• Learn about current trends and technological advances on the topic.
• Seek careful input from experts, mentors, colleagues, and collaborators to refine your research question as this will aid in developing the research question and guide the research study.
• Use the FINER criteria in the development of the research question.
• Ensure that the research question follows PICOT format.
• Develop a research hypothesis from the research question.
• Ensure that the research question and objectives are answerable, feasible, and clinically relevant.

If your research hypotheses are derived from your research questions, particularly when multiple hypotheses address a single question, it’s recommended to use both research questions and hypotheses. However, if this isn’t the case, using hypotheses over research questions is advised. It’s important to note these are general guidelines, not strict rules. If you opt not to use hypotheses, consult with your supervisor for the best approach.

Farrugia, P., Petrisor, B. A., Farrokhyar, F., & Bhandari, M. (2010). Practical tips for surgical research: Research questions, hypotheses and objectives.  Canadian journal of surgery. Journal canadien de chirurgie ,  53 (4), 278–281.

Hulley, S. B., Cummings, S. R., Browner, W. S., Grady, D., & Newman, T. B. (2007). Designing clinical research. Philadelphia.

Panke, D. (2018). Research design & method selection: Making good choices in the social sciences.  Research Design & Method Selection , 1-368.

• Research Group
• Job Seekers

Research Tips

Understanding the Difference between Research Questions and Objectives

January 13, 2023

When conducting research, clearly understanding the difference between research questions and objectives is important. While these terms are often used interchangeably, they refer to two distinct aspects of the research process.

Research questions are broad statements that guide the overall direction of the research. They identify the main problem or area of inquiry that the research will address. For example, a research question might be, "What is the impact of social media on teenage mental health?" This question sets the stage for the research and helps to define the scope of the study.

• Research questions are more general and open-ended, while objectives are specific and measurable.
• Research questions identify the main problem or area of inquiry, while objectives define the specific outcomes that the researcher is looking to achieve.
• Research questions help define the study's scope, while objectives help guide the research process.
• Research questions are often used to generate hypotheses or identify gaps in existing knowledge, while objectives are used to establish clear and achievable targets for the research.
• Research questions and objectives are not mutually exclusive, but well-defined research questions should lead to specific objectives necessary to answer the question.

On the other hand, research objectives are specific, measurable goals that the research aims to achieve. They are used to guide the research process and help to define the specific outcomes that the researcher is looking to achieve. For example, an objective for the above research question might be "To determine the correlation between social media usage and rates of depression in teenagers." This objective is more specific and measurable than the research question and helps define the specific outcomes that the researcher is looking to achieve.

It is important to note that research questions and objectives are not mutually exclusive; a study can have one or several questions and objectives. A well-defined research question should lead to specific objectives necessary to answer the question.

In summary, research questions and objectives are two distinct aspects of the research process. Research questions are broad statements that guide the overall direction of the research, while research objectives are specific, measurable goals that the research aims to achieve. Understanding these two terms' differences is essential for conducting effective and meaningful research.

Literature Reviews

• 1. Define your research question
• Getting started
• Types of reviews

Define topic

Brainstorming, limit scope, how ai can help, helpful resources.

• 2. Plan your search
• 3. Search the literature
• 4. Organize your results
• 5. Synthesize your findings
• 6. Write the review
• Artificial intelligence (AI) tools
• Thompson Writing Studio This link opens in a new window
• Need to write a systematic review? This link opens in a new window

Contact a Librarian

Ask a Librarian

Mission animated icons created by Freepik - Flaticon

For example, the following research question is too broad.

Does social media affect mental health in college students?

A more focused research question could be:

Does social media play a role in the number of eating disorder diagnoses in college-aged women?

Need some guidance? Contact your subject librarian for a research consultation or reach out to [email protected] .

Start the brainstorming process:

• Begin by identifying a broad topic of research interest (the more you're personally interested in it, the better!). Write it down to clarify your focus.
• Narrow down your scope  by considering specific areas within this topic that you want to explore.
• Identify key themes and elements  related to these specific areas. These will be the building blocks of your investigation.
• Finally,  integrate these key areas and elements  into a well-crafted research question.

One way to brainstorm is to use a flowchart.

The scope of a topic  encompasses the boundaries and constraints you set for your research. The more precise your research question, the smoother the research process becomes.

Feeling stuck? Here are some ways you can narrow down your topic:

• geographic location
• time period
• discipline/field of study
• age group, etc.

You:  I need to conduct a literature review about social media, mental health, and college students. What are some example research questions within this topic?

ChatGPT: 

• How does the frequency and nature of social media use impact the mental health of college students?
• What role do specific social media platforms play in influencing the mental well-being of college students?
• How do online social interactions on platforms like Facebook, Instagram, or Twitter relate to the mental health outcomes of college students?
• Are there differences in the association between social media use and mental health outcomes based on demographic factors such as gender, age, or academic major among college students?
• What strategies do college students employ to cope with negative mental health effects associated with social media use?
• How does the perception of social support through online platforms affect the mental health of college students?
• To what extent does the comparison of oneself to others on social media contribute to mental health challenges among college students?
• What interventions or educational programs are effective in promoting positive mental health behaviors in the context of social media use for college students?
• How do social media literacy and awareness programs impact the mental health attitudes and behaviors of college students?
• What is the relationship between the use of social media for academic and social purposes and the mental health outcomes of college students?

Generative AI tools : ChatGPT , Google Gemini (formerly Bard) , Claude , Microsoft Copilot

For more information on how to incorporate AI tools into your research, check out the section on AI Tools .

• Developing Your Research Question
• Developing a Researchable Question
• Brainstorming tips from UNC Writing Center
• << Previous: Types of reviews
• Next: 2. Plan your search >>
• Last Updated: May 17, 2024 8:42 AM
• URL: https://guides.library.duke.edu/litreviews

Services for...

• Faculty & Instructors
• Graduate Students
• Undergraduate Students
• International Students
• Patrons with Disabilities

• Harmful Language Statement
• Re-use & Attribution / Privacy
• Support the Libraries

Difference between Aim and Objectives of a Research Study

In research, aim and objective are two important terms that are often used interchangeably. However, there is a fundamental difference between them. Aim refers to the overall goal or purpose of a research study, while objectives are specific statements that describe the steps or actions needed to achieve the aim.

What Is the Aim of a Research?

The aim of a research study is a broad statement that defines the overall purpose of the study. It answers the question of why the study is being conducted and what the researcher hopes to achieve. It provides a general idea of the scope and direction of the research. It also helps to guide the research process and provide a clear focus for the study.

What Are the Objectives of a Research ?

Objectives are specific statements that describe the steps or actions that are needed to achieve the aim of the study. They are more specific and measurable than the aim, and they provide a clear roadmap for the research. Objectives help to clarify the research question, identify the key variables, and outline the research methodology.

Objectives are often used to break down the aim of the research into smaller, more manageable tasks. They help to provide structure and direction to the research and ensure that the researcher stays on track. They are often S pecific, M easurable, A chievable, R elevant, and T ime-bound (acronymed as SMART). This means that they should be clear and concise, include metrics to measure progress and success, be achievable within the timeframe of the research, be relevant to the overall aim of the study, and have a deadline for completion.

To better understand the difference between aim and objectives, here are some examples.

Aim: To investigate the relationship between physical activity and mental health.

Objectives:

• To review the existing literature on the relationship between physical activity and mental health.
• To collect data on the physical activity levels and mental health of a sample population.
• To analyze the data to determine the relationship between physical activity and mental health.
• To draw conclusions and make recommendations based on the findings of the study.

In this example, the aim is to investigate the relationship between physical activity and mental health. The objectives break down the aim into specific tasks, such as reviewing the literature, collecting data, analyzing the data, and drawing conclusions.

Aim: To evaluate the effectiveness of a new teaching method for improving student performance in mathematics.

• To identify the key components of the new teaching method.
• To implement the new teaching method in a sample of classrooms.
• To collect data on the performance of students in the sample classrooms.
• To analyze the data to determine the effectiveness of the new teaching method.
• To make recommendations for the implementation of the new teaching method.

In this example, the aim is to evaluate the effectiveness of a new teaching method for improving student performance in mathematics. The objectives break down the aim into specific tasks, such as identifying the key components of the new teaching method, implementing the method in classrooms, collecting data, analyzing the data, and making recommendations.

In conclusion, aim and objective are two important terms in research. The aim is the overall goal or purpose of the study, while objectives are specific statements that describe the steps or actions needed to achieve the aim. Objectives provide a clear roadmap for the research, help to clarify the research question, and ensure that the researcher stays on track. When writing a research paper, it is important to clearly define the aim and objectives of the study to provide a clear focus and direction for the research.

Did the content interest you? Would you like to read more such content? Do visit our website https://www.manuscriptedit.com/ to explore more blogs and podcasts. You can also reach us at [email protected] for your queries.

Related Posts

Synthesis and study of microcapsules in self-healing dental composites.

The pursuit of enhancing dental resin composites has led to the synthesis and characterization of microcapsules, aiming to introduce self-healing properties. Dental composites often face challenges such as wear, cracking, and degradation over time. The incorporation of microcapsules presents a promising avenue for addressing these issues and extending the longevity of dental restorations. Synthesis: The […]

Creating a Video Abstract for your Research

Want to create an impactful abstract that persuades the readers to read your article?  Here’s an interesting solution – Video Abstracts Introduction What is a video abstract? A video abstract is an innovative way to explain your work to the public and researchers outside of your field that adds value proposition. This 3-5 minute video […]

Anxiety: More Than Just a Feeling

  Fear and anxiety aren’t just in your mind, as psychiatrist and neuroscientist Arash Javanbakht explains. While the brain processes these emotions, they manifest throughout the body. The amygdala, a key brain region, triggers physical responses rapidly when danger is perceived. The hippocampus contextualizes fear, differentiating between real threats and safe situations. The prefrontal cortex […]

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Frontiers in Endocrinology
• Translational Endocrinology
• Research Topics

Endocrine Imbalances of Mineral Ions and Vitamins in Chronic Disease Pathogenesis

Total Downloads

Total Views and Downloads

About this Research Topic

The endocrine system plays a crucial role in the regulation of vitamin and mineral ion homeostasis, which is essential for maintaining normal physiological functions of the body. This regulation is disrupted in various chronic diseases, contributing to the development of debilitating conditions, affecting almost all the major systems of the body, including renal, musculoskeletal, gastrointestinal, and neural systems. Understanding the extent of the hormonal impact on cellular and tissue functions, as well as the potential for reversing such disruptions, is essential for reducing the disease burden in chronic human disorders. This Research Topic invites the submission of various types of scholarly contributions to provide a comprehensive understanding of the endocrine regulation of vitamin and mineral ion metabolism and its clinical implications. The Research Topic intends to publish: 1. Original research works presenting unpublished human studies and experimental research findings. 2. Updated review articles summarizing the current state of knowledge in the existing literature. 3. Brief commentaries discussing emerging areas and developments in mineral ion metabolism and vitamin regulation. The objective of the Research Topic is to gather a diverse range of submissions that collectively (1) offer a thorough and up-to-date understanding of the endocrine control of mineral ion and vitamin metabolism, and (2) identify areas for further research and potential applications in the management of chronic disorders and associated complications. The overarching goal of this Research Topic is to bring together basic scientists and clinicians to explain how miscommunication between vitamins and mineral ions contribute to altered bone mineralization, disrupted energy metabolism, altered cell signaling, neuronal irregularities, and nutrition-associated deformities.

Keywords : Mineral ion, Vitamin, Chronic disease, Pathogenesis, Regulation, Metabolism

Important Note : All contributions to this Research Topic must be within the scope of the section and journal to which they are submitted, as defined in their mission statements. Frontiers reserves the right to guide an out-of-scope manuscript to a more suitable section or journal at any stage of peer review.

Topic Editors

Topic coordinators, submission deadlines, participating journals.

Manuscripts can be submitted to this Research Topic via the following journals:

total views

• Demographics

No records found

total views article views downloads topic views

Top countries

Top referring sites, about frontiers research topics.

With their unique mixes of varied contributions from Original Research to Review Articles, Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author.

This paper is in the following e-collection/theme issue:

Published on 17.5.2024 in Vol 26 (2024)

Reporting of Ethical Considerations in Qualitative Research Utilizing Social Media Data on Public Health Care: Scoping Review

Authors of this article:

• Yujie Zhang 1, 2 * , MD   ; 
• Jiaqi Fu 1, 2 * , MD   ; 
• Jie Lai 1, 2 * , MD   ; 
• Shisi Deng 1, 2 , MD   ; 
• Zihan Guo 1, 2 , MD   ; 
• Chuhan Zhong 1, 2 , MD   ; 
• Jianyao Tang 1, 2 , MD   ; 
• Wenqiong Cao 1 , BSc   ; 
• Yanni Wu 1 , PhD  

1 Nanfang Hospital, Southern Medical University, Guangzhou, China

2 School of Nursing, Southern Medical University, Guangzhou, China

*these authors contributed equally

Corresponding Author:

Yanni Wu, PhD

Nanfang Hospital

Southern Medical University

No 1838 Guangzhou Avenue North

Baiyun District, Guangdong Province

Guangzhou, 510515

Phone: 86 02061641192

Email: [email protected]

Background: The internet community has become a significant source for researchers to conduct qualitative studies analyzing users’ views, attitudes, and experiences about public health. However, few studies have assessed the ethical issues in qualitative research using social media data.

Objective: This study aims to review the reportage of ethical considerations in qualitative research utilizing social media data on public health care.

Methods: We performed a scoping review of studies mining text from internet communities and published in peer-reviewed journals from 2010 to May 31, 2023. These studies, limited to the English language, were retrieved to evaluate the rates of reporting ethical approval, informed consent, and privacy issues. We searched 5 databases, that is, PubMed, Web of Science, CINAHL, Cochrane, and Embase. Gray literature was supplemented from Google Scholar and OpenGrey websites. Studies using qualitative methods mining text from the internet community focusing on health care topics were deemed eligible. Data extraction was performed using a standardized data extraction spreadsheet. Findings were reported using PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines.

Results: After 4674 titles, abstracts, and full texts were screened, 108 studies on mining text from the internet community were included. Nearly half of the studies were published in the United States, with more studies from 2019 to 2022. Only 59.3% (64/108) of the studies sought ethical approval, 45.3% (49/108) mentioned informed consent, and only 12.9% (14/108) of the studies explicitly obtained informed consent. Approximately 86% (12/14) of the studies that reported informed consent obtained digital informed consent from participants/administrators, while 14% (2/14) did not describe the method used to obtain informed consent. Notably, 70.3% (76/108) of the studies contained users’ written content or posts: 68% (52/76) contained verbatim quotes, while 32% (24/76) paraphrased the quotes to prevent traceability. However, 16% (4/24) of the studies that paraphrased the quotes did not report the paraphrasing methods. Moreover, 18.5% (20/108) of the studies used aggregated data analysis to protect users’ privacy. Furthermore, the rates of reporting ethical approval were different between different countries ( P =.02) and between papers that contained users’ written content (both direct and paraphrased quotes) and papers that did not contain users’ written content ( P <.001).

Conclusions: Our scoping review demonstrates that the reporting of ethical considerations is widely neglected in qualitative research studies using social media data; such studies should be more cautious in citing user quotes to maintain user privacy. Further, our review reveals the need for detailed information on the precautions of obtaining informed consent and paraphrasing to reduce the potential bias. A national consensus of ethical considerations such as ethical approval, informed consent, and privacy issues is needed for qualitative research of health care using social media data of internet communities.

Introduction

Social media are web-based computer-mediated tools to collaborate, share, or exchange information, ideas, pictures, or videos in virtual communities and networks such as message boards, communities, chat rooms, forums, Twitter, and Facebook [ 1 ]. Moreover, patients and researchers can use internet communities to provide health care and disseminate health information [ 2 , 3 ]. Health care refers to the efforts made to improve or maintain physical, mental, or emotional well-being, including prevention, diagnosis, treatment, recovery, and other physical and mental impairments [ 4 ]. Currently, with 57% of the global population’s access to social media, more than 40% of the patients and caregivers worldwide utilize the internet community for health care information needs [ 5 ]. With diverse populations accessing internet communities and sharing information about health care topics, researchers have the opportunity to collect and analyze text about health care from a diverse range of participants in the internet community, which was unavailable previously [ 6 ]. Usually, quantitative data are derived from information extraction, which can be analyzed statistically, and the summary results presented cannot be directly linked to individual participants. In contrast, qualitative research within internet community analysis posts and comments qualitatively or thematically involves a more detailed and in-depth analysis and understanding of the full written content [ 7 ]. However, a controversial ethical problem has been raised about conducting qualitative research containing internet users’ verbatim quotes that could lead to traceability of the original post, thereby causing a threat to an individual’s privacy [ 8 ]. Additionally, a previous study investigated public and patients’ views regarding ethics in research using social media data and reported that internet users were aggrieved if they found any of their quotes cited in a medical research paper without obtaining their informed consent [ 9 ]. Further, besides the privacy breach caused by posts being traced, there is greater harm for special groups or vulnerable groups if we do not highlight the importance of the technical standards for text mining and privacy protection in health care. For instance, some unusual postings, abnormal pictures, and interactions that were expressed by individuals with mental disorders in social media can be detected by researchers by using text mining tools without obtaining their consent [ 10 ]. The publication of research on mental disorders, including quotes in posts, can result in a high risk of information harm, which can lead to personal information being revealed and further stigmatization of the condition or disease [ 11 ]. Since 2001, ethical concerns have been debated for decades about ethical approval, informed consent, and how to ensure anonymity and preserve data privacy and confidentiality in qualitative research in the internet community [ 12 - 14 ].

With the rapid development of social media and internet research, some ethical guidelines or standards have been published to ensure that research based on internet communities is conducted ethically. The Association of Internet Researchers (internet research ethical guidelines 2.0 and 3.0) showed that researchers working without the direct approval of ethics review boards would have additional challenges to face, and obtaining informed consent is obviously impracticable in several big data projects. However, with the ethical issues about privacy breaches and harms of risk of discrimination, the Association of Internet Researchers recommended reserving the acquisition of informed consent to the dissemination stage by asking for informed consent from specific participants before publication of their quotes [ 15 , 16 ]. Furthermore, researchers should take responsibility for information confidentiality and anonymity according to the internet research ethics criteria prepared by the National Committee for Research Ethics in the Social Sciences and the Humanities guidelines, which recommend a basic research ethic norm for the analyses, reports, and evaluations that apply to all research [ 17 ]. Moreover, the National Committee for Research Ethics in the Social Sciences and the Humanities guidelines contain more details about the demand for legal consent and privacy standards imposed by the European Union’s General Data Protection Regulation. The General Data Protection Regulation is a European Union–wide regulation targeting the project of personal data processing. The General Data Protection Regulation defines personal data as any information relating to an identifiable person (data subject), including name, online identification number, location data, and other factors related to personal, physical, physiological, mental, or social identity [ 18 ]. The General Data Protection Regulation recommends using anonymous data and deleting identifiable information to ensure the confidentiality of the data. Consent should be obtained from the individual for use in scientific research [ 18 , 19 ]. The British Psychological Society guideline does not explicitly refer to the internet community but suggests that researchers may consider paraphrasing the verbatim quotes to reduce the risk of being traced or identified in qualitative research [ 20 ]. When paraphrasing, steps must be put into place to ensure that the original meaning of the message is maintained. Currently, there is no widespread consensus on ethical considerations by social media researchers.

Some researchers have tried to explore the reporting of existing ethical considerations in research papers using social media data. For instance, Sinnenberg et al [ 6 ] reported that only 32% and 12% of the papers mentioned acquiring ethical approval and informed consent, respectively, by utilizing multiple analysis methods, including surveillance, intervention, recruitment, engagement, content analysis, and network analysis with Twitter data before 2015. Thereafter, Takats et al [ 21 ] conducted an updated examination based on Sinnenberg et al’s [ 6 ] study. They found that of 367 studies using different methodological approaches, including sentiment mining, surveillance, and thematic exploration of public health research using Twitter data between 2010 to 2019, 17% of the studies included verbatim tweets and identifiable information about the internet users [ 21 ]. Similarly, Lathan et al [ 22 ] reviewed papers, including both qualitative and quantitative methods, by using Facebook data to explore public health issues and reported that only 48% and 10% of the papers obtained ethical approval and informed consent, respectively. Furthermore, in a study on research using YouTube data or comments, Tanner et al [ 23 ] found that only 26.1% of these studies sought ethical approval, only 1 paper (0.08%) sought informed consent, and 27.7% contained identifiable information. These findings indicate widespread neglect of ethical issues such as ethical approval, informed consent, and privacy issues in research papers using social media data.

Our study focuses on the ethical challenges of qualitative studies utilizing social media data. First, social media can be considered as sources for qualitative data collection because of the low cost, vast amount of available sources about health information, and users’ health behaviors, experiences, and attitudes. Second, qualitative research is context-dependent and mainly contains quotations and written content to support the viewpoint. It is acknowledged that quote materials from social media would potentially be traced back to the original posts and threaten the users’ privacy [ 24 ]. This is supported by findings reported by Ayers et al [ 25 ] who found that online searches of verbatim Twitter quotes in journal papers described as “content analyses” or “coded Twitter postings” can be traced back to individual internet users 84% of the time. Furthermore, Lathan et al [ 22 ] identified that 46% of the studies with verbatim or paraphrased quotes could be traced to the original posts in 10 minutes. Therefore, it is essential to investigate the extent to which ethical oversight is reported in qualitative studies using social media data. Moreover, qualitative research often involves personally sensitive data about health conditions and diseases; hence, anonymity and proper deidentification would be more important for researchers [ 26 , 27 ].

Previous studies have reviewed the ethical challenges and methodological use of social media platforms such as Twitter [ 6 , 21 ], Facebook [ 22 ], and YouTube [ 23 ] for health care research in both qualitative and quantitative studies. Although there is plenty of qualitative data pouring into social media such as blogs, Twitter, Facebook, and Weibo, evidence is lacking on the investigation of ethical considerations targeting qualitative data in different software and web-based discussion forums to provide a more comprehensive understanding of the ethical issues. To address the ethical considerations in qualitative research of different internet communities and draw the attention of researchers and publishers to ethical issues, we conducted this study to evaluate the ethical practices and ethical considerations of qualitative studies on health care by using data of internet communities. This review aims to (1) assess the rates of reporting institutional review board (IRB) approval and informed consent in studies focused on mining text in the internet community and social media, (2) compare these rates according to the year of publication, country conducting the research, website included in the study’s analysis, and journal’s guidelines about ethical approval for the type of study, and (3) describe whether the studies used anonymized/deidentified data.

Research Design

We conducted a scoping review to investigate how qualitative research mining social media data handles ethical approval, informed consent, and confidential issues. We performed this study according to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines. The completed PRISMA-ScR checklist is provided in Multimedia Appendix 1 .

Search Strategy

All published qualitative studies from 2010 to March 31, 2023, focusing on mining text from online community and social media sources about health care in the following databases were included in this study: PubMed, Web of Science, CINAHL, Cochrane, and Embase. A standardized search string containing Medical Subject Headings (MeSH) and non-MeSH entry terms was used in the search strategy. In addition, the reference lists of the retrieved papers and citation tracking were manually searched as a supplement to database searches to improve comprehensiveness. Gray literature was also identified through internet searches in Google Scholar and OpenGrey websites. The search strategies are represented in Multimedia Appendix 2 .

Inclusion and Exclusion Criteria

We divided the criteria into 2 parts. First, we limited the inclusion and exclusion criteria used at the title and abstract screening stage eligible for (1) studies mining existing text and posts from the internet community and social media data focusing on health care topics, (2) studies using qualitative methods or available qualitative parts in mixed methods studies to analyze data, and (3) studies only written in English. Ineligible studies were those related to investigating the use and dissemination of social media in health care, using social media or internet community as an intervention tool, and using social media to conduct web-based interviews, surveys, or focus groups. We also excluded studies published as reviews, case studies, conference abstracts, commentaries, policies, guidelines, and recommendations. Second, at the full screening stage, the specific eligible inclusion criteria were studies focused on mining text about health care topics with full-text papers. Studies that did not have the full text after contacting the authors and that were not originally in the English language were excluded.

Study Selection

All results of the searches were entered into the EndNote library, and duplicates were removed. Two researchers reviewed the titles and abstracts based on the inclusion and exclusion criteria independently. Those studies that were irrelevant to the study topic were discarded, and then the full text was screened to select eligible papers. Any disagreements were discussed and resolved by consensus or a third person.

Data Extraction

Data were extracted between April 2023 and May 2023. Two researchers independently read the full text carefully, and the results were extracted using a standardized data extraction spreadsheet, including research type, first author, study objective, sample size, publication time, country where the research was conducted or country of the first author, website or internet community the studies focus on, type of data collected from social media, language of collected posts or data, privacy level of data (public or privacy posts), study design, research results, published journal, and information about the ethical considerations. Disagreements were resolved by consensus of a third person. The information about ethical considerations was analyzed to investigate the rates of reporting ethical approval, informed consent, and privacy issues: whether IRB review was reported (IRB approval, IRB exemption, unnecessary, not mentioned) and the reason for not requiring IRB approval; whether informed consent was obtained from participants or the websites’ administrators, consent types (digitally informed consent or written informed consent, informed consent is not required, consent was waived by IRB), and the methods used to obtain consent in each study; and whether quoting a post in papers could lead to the identification of internet users in each study. The description of users’ posts (verbatim quote, paraphrase) was recorded. We also analyzed if posts were paraphrased to maintain the original meaning, if actions were taken to deidentify the internet users, and if the posts contained other identifying information (ie, usernames, photos, links, hashtags) attached to the post. As every journal would provide publication ethical considerations and requirements, we also searched the submission guidelines and editorial policies of each journal submission website to check whether the journal contained any ethical guidance targeting studies using data from internet community and social media platforms. Additional information was included about the details of ethical approval, informed consent, and privacy, for example, whether individuals can withdraw their quotes if they want to be excluded from the study at any time without any reprisal and whether the quotations were tested for deidentification via search engines. There was excellent agreement on the primary outcome between the 2 researchers (k>.95 for all).

Data Analysis

Data were analyzed using SPSS software (IBM Corp). The chi-square test or Fisher exact tests (when cell size was less than 5) were used to test for differences between the rates of informed consent and ethical approval according to publication year, website, and different countries. All P values were 2-sided, and P values <.05 indicated significance.

Study Selection for the Review

We reviewed 4674 papers after removing the duplicates. After screening the titles, abstracts, and full-texts, we reviewed 108 eligible papers ( Figure 1 ). The full list of the included papers and all the extracted information are incorporated in Multimedia Appendix 3 [ 28 - 135 ]. Of the 108 studies reviewed, 73 (67.6%) were qualitative studies and 35 (32.4%) were mixed methods studies. All papers had text mined from internet communities or social media for qualitative analysis. The sample size ranged from 32 to 392,962. Approximately 82.4% (89/108) of the studies were published after 2018, and there was a sharp increase in the number of studies from 2019 to 2022. Moreover, nearly half of the studies (55/108, 50.9%) were published in the United States. Regarding the websites for mining text, the most widely used social media platform was Twitter (42/108, 38.9%), followed by Facebook (17/108, 15.7%).

Ethics Approval in These Studies

Our results indicated that of the 108 studies, 78 (72.2%) reported ethics approval. Of the 78 studies, 31 (40%) explicitly stated that ethics approval was obtained before the study was undertaken, 33 (42%) reported that the ethics approval was granted through exemptions by the local IRB, and 14 (18%) explicitly demonstrated that approval by the ethics committee was not required because publicly available data were collected from internet communities and social media platforms. However, 30 (27.8%) of the 108 studies did not mention about obtaining IRB approval ( Table 1 ).

Based on our exploration of the ethical guidelines of each journal to determine whether there were ethical requirements for studies mining social media data, only 36.1% (39/108) of the studies were published in journals that required ethical considerations for studies gathering data from social media platforms by using internet and digital technologies. Of the 39 studies published in 19 journals, 27 (69%) were published in the Journal of Medical Internet Research and its sister journals. The submission guidelines of the Journal of Medical Internet Research state that authors of manuscripts describing studies of internet, digital tools, and technologies are required to verify that they have adhered to local, national, regional, and international laws and regulations, and are required to verify that they complied with informed consent guidelines. Moreover, 2 journals also provided a specific requirement, that is, when researchers interact with individuals or obtain privacy information gathered from social media platforms, they should obtain ethics approval prior to conducting the study and informed consent from anyone who could potentially be identified. Surprisingly, there were no significant differences in the ethics approval reportage between journals with ethics approval guidelines and those that did not have ethics guidelines for researchers gathering data from social media platforms ( P =.08). Notably, the rates of reporting ethics approval were different between different countries ( P =.02). However, there were no statistically significant differences between the rates of reporting ethical approval and different websites or publication years (all P >.05) ( Table 2 ).

Informed Consent

Of the 108 studies, 59 (54.7%) showed that they did not include any information about informed consent and 49 (45.3%) mentioned informed consent. Of the 49 studies that mentioned informed consent, 14 (13%) demonstrated that informed consent was waived by local institutional boards, and 21 (19%) reported that informed consent was not required because this information is publicly available in websites or did not involve human participants. We interpreted this as not seeking informed consent. Only 14 (12.9%) of the 108 studies explicitly indicated that informed consent was obtained ( Table 1 ). Among the 14 studies, 2 (14%) only provided a generic statement that informed consent was obtained but did not report the process of how the informed consent was obtained, while 12 (86%) received digital informed consent. Of the 12 studies that reported receiving digital informed consent, 6 reported that they sought permission from the communities’ or groups’ administrators and by posting a statement of the research objective on the group’s wall, while 5 studies contacted the participants privately via email, commenting below the posts and software to gain consent, and 1 study reported that it had sent a digital version of the informed consent book. Furthermore, among the studies that had obtained informed consent, 7 studies included the statement that the individuals’ posts would be removed if they wanted to be excluded from the study, and they could withdraw from the study whenever they wanted. In addition, the rates of reporting informed consent showed no statistical significance between publication years, different countries, and different websites (all P >.05) ( Table 2 ).

Confidentiality of the Information

All data sources were obtained from anonymous websites or communities, and the majority (104/108, 96.3%) of the data sources did not contain usernames. Notably, only 3.7% (4/108) of the studies contained the participants’ usernames or pseudonyms. One study reported that pseudonyms like Sasha had been used instead of the real name. The other 3 studies contained the expression for usernames but did not state whether pseudonyms were used. Except for 9 studies that used nonnative language quotes and 3 studies that were transcribed into text via video, among the 108 included studies, 76 (70.3%) quoted at least one native language post in their reports. Additionally, 20 studies presenting aggregated analysis or composite accounts did not include any quotation or written content. Of the 76 studies containing internet users’ written content, 52 (68%) contained just verbatim-quoted participants’ posts and 24 (32%) contained paraphrased posts ( Table 1 ). Among the 52 studies containing direct and verbatim quotations, which are likely to be traced to the original posts from users, only 17 (33%) studies took measures to deidentify the users. The 17 studies mentioned that all names or usernames were removed and personal identifying information was removed to maintain privacy, while 42% (22/52) of the studies did not mention any measures that were taken to deidentify the users and maintain confidentiality. Approximately 32% (24/76) of the studies described that they paraphrased posts and removed any explicitly identified personal information to maintain confidentiality to reduce the likelihood of users being identified via search engines. Of the 24 studies, 20 (83%) reported that the quotations were slightly modified or summarized for readability, the symbol information was removed using “…”, and key identifiable information was removed to protect privacy while maintaining the meaning of posts. Four of the 24 (17%) studies did not report the methods and details of paraphrasing. Notably, only 3% (2/76) of the studies containing users’ written content showed that researchers intentionally entered each quote into search engines to ensure that every quote did not lead to the original posts. Moreover, of the 76 studies containing written content, 62 (82%) did not contain other types of identity information attached to the posts, while 14 (18%) included other identifying data (hashtags, emojis, geolocation, photos, links, screenshots) attached to the original posts for analysis ( Table 1 ). Of the 14 studies including other identifying information, 4 (29%) contained photos and screenshots associated with the website pages. Of the 52 studies that disclosed verbatim quotes and other identifiable information, 26 (46%) studies reported informed consent consideration, and only 8 (15%) obtained explicitly informed consent. Additionally, of the 77% (40/52) of the studies that mentioned IRB or ethical review, 38% (15/40) received IRB approval, and 63% (25/40) of the studies were granted exemption. The proportion of reporting ethical approval in studies containing users’ written content was modestly higher than that in studies not containing users’ written content (60/76, 79% vs 14/32, 44%; P <.001) ( Table 2 ).

Principal Findings and Comparison to Prior Work

In this scoping review, we included 108 studies ( Multimedia Appendix 3 ; [ 28 - 135 ]) that focused on mining text from internet community and social media data for health care research, and we reviewed the ethical consideration reportage and outcome reports in these studies. We found that the rates of reporting IRB approval and informed consent in qualitative research on health care utilizing social media data were 59.3% (64/108) and 12.9% (14/108), respectively. Our findings demonstrate that the key ethical considerations for qualitative research in online communities are insufficiently discussed and described. However, the reporting rates of ethical considerations in the papers in our scoping review were much higher than those reported in systematic reviews including multiple analysis methodologies on only 1 social media platform. For example, ethics approval and informed consent were reported in 48% and 10% of research studies using only Facebook data [ 22 ], 32% and 0% from 2006 to 2019 [ 21 ], 40% and 0.9% (only 1 paper) from 2015 to 2016 in public health research using only Twitter data [ 25 ], and 26.1% and 0.8% (only 1 paper) in researches incorporating only YouTube data [ 23 ], respectively. In fact, previous studies were limited to only a few selected websites such as Twitter, Facebook, and YouTube. There is a lack of research that incorporates a variety of different social media data for comparisons. Differences in the reporting of ethical considerations may be attributed to the different methodologies adopted by studies. For example, Lathan et al [ 22 ] analyzed the ethical considerations in studies including predictive or model development, while our research focuses on the ethical considerations in qualitative studies.

Importantly, our findings indicate that there is a need to develop a standardized and apparent approach for the reporting of ethical considerations in qualitative research of data from social media and online communities. Our research demonstrates that the rates of reporting ethics approval are different in different countries ( P =.02). Specifically, a wide variety of national research ethics governing bodies and over 1000 laws, regulations, and standards provide oversight for human subjects research in 130 countries. Obviously, a guideline is needed for best ethical practices for qualitative research involving posts from social media platforms. Surprisingly, there were no significant differences between the rates of reporting ethical approval and those of journals specifying ethical requirements for studies involving text mining ( P =.08). This inconsistent result of publication guidelines and reports of ethical approval consent is similar to previous findings on the ethical standards in COVID-19 human studies [ 136 ]. Although there are journal publication guidelines for studies mining social media data, the reports of ethical approval and consent in the papers published in such journals do not exactly follow the guidelines. Consequently, this finding indicates that more ethical awareness is needed among researchers, editors, and reviewers for qualitative studies on data mining.

Besides the different legal and regulations in different countries, the inconsistency in the ethics approval in published papers may be because social media research is a highly interdisciplinary science, and computer science researchers may be less experienced or may pay less attention to the key ethical issues of protecting human subjects [ 137 ]. Medical and health science researchers may have considered some ethical concerns about gathering social media data but they may not be familiar with the relevant guidelines. For example, the Association of Internet Researchers has a detailed ethical guideline targeting social scientists conducting digital research, while it may be less popular and less well-known among medical and health care researchers. At the institute level, Ferretti et al [ 138 ] noticed that institutionalized review committees, especially the individual IRB institutes for universities and health care systems lack knowledge about the methodology, text mining technical standards, data security, and ethical harms for studies using big data and social media as sources. Because of this lack of knowledge, institutional ethics committees may have inconsistent ethical criteria and perspectives about web-based projects using social media data [ 139 ]. Therefore, some ethics review committees exclude research on internet communities from ethical oversights because their ethics standards are confined only to medical fields. Above all, it is additionally challenging for ethical approval institutions because of the continuous development and dynamic change of studies using social media data. Furthermore, it is necessary for ethics committee members to be trained about the ethical issues in studies mining text from social media. Inviting interdisciplinary researchers to join in the approval process would be an appropriate method to increase the awareness of ethical considerations [ 140 , 141 ].

Interestingly, the reporting rate of obtaining informed consent for mining social media data in qualitative studies was unexpected. The most influential ethical reports such as the Nuremberg Code [ 142 ], Declaration of Helsinki [ 143 ], and the Belmont Report [ 144 ] have demonstrated the principle of informed consent in research involving humans. Our review shows that only 12.9% (14/108) of the studies explicitly obtained informed consent and 32.4% (35/108) of the studies reported that informed consent was exempted by IRB or was not required, as the information was available publicly in websites or did not involve human participants. Our results are similar to those of Wongkoblap et al [ 145 ] who reported that only 16.7% of the studies received informed consent from participants prior to data analysis on data mining of social network data on mental health disorders.

There are multiple reasons for the challenges in obtaining informed consent in an internet setting. First, it is impractical for researchers to gain individual informed consent from a large number of users in an internet community [ 146 ]. Second, members of ethics review boards lack consensus about the need for informed consent from an internet community for qualitative research under the current legal definition [ 147 ]. Moreover, there has been a debate on the criteria of human subject research in using social media data. The federal regulation recommends that if data in the studies are obtained from public social media websites, where data are identifiable and do not require interaction with individuals, such studies do not constitute human subject research, while studies involving the identification of private information or interaction with the individual can be considered as human subject research [ 148 ]. In contrast, some researchers believe that social media and big data research are not ethically exceptional and should be treated in the same manner and with the same rules as those for traditional forms of research [ 149 ]. There is ambiguity as to what is appropriate or should be standard practice for obtaining informed consent.

Currently, it is challenging to maintain privacy and protect the traceability of individuals posting content in the internet community. Our findings indicated that 70.3% (76/108) of the studies contained internet users’ written content, of which 68% (52/76) included verbatim quotations of users’ posts that could lead to identification, and 18% (14/76) of the studies included other identifiable information such as links, screenshots, and emojis linked to original posts, which are similar to the findings of Ayers et al [ 25 ] and Lathan et al [ 22 ]. Usha Lawrance et al [ 150 ] and Wilkinson and Thelwall [ 151 ] argued that using direct quotes to support findings would lead to the identification of users and breach users’ confidentiality in internet community data. Moreover, quoting social media posts or disclosing usernames violate the International Committee of Medical Journal Editors’ ethics standards, which state that identifying information such as written descriptions and photos should not be published unless the information is essential for scientific purposes and the participants give written informed consent for publication [ 152 ]. Furthermore, our study demonstrates that the proportion of studies containing users’ written content (both direct and paraphrased quotations) is higher than that of studies that do not include any quotation or written content (60/76, 79% vs 14/32, 44%; P <.001)——a tentative explanation is that some researchers realized that ethical reportage should be stricter for qualitative papers with quotations from social media posts due to privacy and security issues. This is supported by Boyd and Crawford [ 153 ] who stated that rigorous thinking about the process of mining and anonymizing big data is required for ethics boards to ensure that people are protected. Our findings show that 32% (24/76) of the studies intentionally paraphrased the quotes to ensure that users could not locate them, and 20 studies used aggregated data interconnected with anonymity. Moreover, it is recommended by Wilkinson and Thelwall [ 151 ], Bond et al [ 154 ], and Markham et al [ 155 ] that researchers should not directly quote and work with aggregate data sets and separate texts from their original context, which is more acceptable to participants. In addition, the British Psychological Society guidelines recommend that researchers consider paraphrasing any verbatim quotes to reduce the risk of these being traced to the source [ 20 ]. Notably, 13 of the 25 papers in this study showed that they did not report the precautions taken for paraphrasing. This may be due to the lack of detailed methodology and consensus about paraphrasing quotes to reduce bias and maintaining the original meaning.

Limitations and Strengths

Our scoping review has several limitations. First, our research was limited to qualitative studies and the qualitative parts in mixed methods studies on text mining from social media, and it is unclear whether ethical considerations are critical in quantitative studies among internet communities. Second, we were restricted to studies published in English language and those with the full text available, and therefore, we could be underestimating the number of relevant papers published in other languages. Third, the rates of reporting ethical approval, informed consent, and privacy of this research relied on self-reported data. Thus, it is possible that although certain studies did not report the process of ethical considerations, such considerations may have been followed during the research. Conversely, some studies may have mentioned about the ethical considerations but may not have conducted them in practice. Hence, there is a bias because of the lack of accurate documentation that must be considered.

Social media text mining can be a useful tool for researchers to understand patient experiences of health conditions and health care. However, as illustrated by the absence of ethical discourse in publications, our analysis indicates significant gaps in the ethical considerations and governance of qualitative research of internet posts. Therefore, a complete and consistent consensus guideline of ethical considerations in qualitative research of internet posts is needed to protect users’ data. With the continued advancing development of text-mining techniques, qualitative studies mining text from social media should be more cautious while using user quotations to maintain user privacy and protect the traceability of the internet users posting content. We suggest that authors should report their results by using aggregated findings or deidentified ways like paraphrasing instead of verbatim quotations, which can prevent internet users from being identified through search engines. In addition, authors should provide more detailed information about the precautions taken for obtaining informed consent and paraphrasing to reduce the potential bias. Furthermore, journals and editors should pay more attention to the reporting standards of ethical consideration and privacy issues in qualitative research involving social media data.

Acknowledgments

This project was funded by the National Natural Science Foundation of China (72304131) and the Outstanding Youths Development Scheme of Nanfang Hospital, Southern Medical University (2023J005). The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of this manuscript. We sincerely thank the funders of this study.

Data Availability

All data extracted and analyzed during this study are presented in this paper and in the multimedia appendices.

Authors' Contributions

YW was responsible for the protocol of the research and redrafted the paper critically. YZ and JF performed literature searches. YZ, JL, and WC performed study identification and screening. ZG, SD, CZ, and JT extracted and analyzed the data from the included journals. YZ and JL wrote the first draft of the paper. All authors read and approved the final manuscript.

Conflicts of Interest

None declared.

PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) checklist.

Search strategy for each database.

Summary of included literature.

• Obar JA, Wildman S. Social media definition and the governance challenge: An introduction to the special issue. Telecommunications Policy. Oct 2015;39(9):745-750. [ CrossRef ]
• Kaplan AM, Haenlein M. Users of the world, unite! The challenges and opportunities of social media. Business Horizons. Jan 2010;53(1):59-68. [ CrossRef ]
• VanDam C, Kanthawala S, Pratt W, Chai J, Huh J. Detecting clinically related content in online patient posts. J Biomed Inform. Nov 2017;75:96-106. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Health care. Wikipedia. URL: https://en.wikipedia.org/wiki/Health_care [accessed 2023-06-01]
• Moorhead SA, Hazlett DE, Harrison L, Carroll JK, Irwin A, Hoving C. A new dimension of health care: systematic review of the uses, benefits, and limitations of social media for health communication. J Med Internet Res. Apr 23, 2013;15(4):e85. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sinnenberg L, Buttenheim AM, Padrez K, Mancheno C, Ungar L, Merchant RM. Twitter as a tool for health research: A systematic review. Am J Public Health. Jan 2017;107(1):e1-e8. [ CrossRef ] [ Medline ]
• Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. Jan 2006;3(2):77-101. [ CrossRef ]
• Hswen Y, Naslund JA, Brownstein JS, Hawkins JB. Monitoring online discussions about suicide among Twitter users with schizophrenia: exploratory study. JMIR Ment Health. Dec 13, 2018;5(4):e11483. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Golder S, Ahmed S, Norman G, Booth A. Attitudes toward the ethics of research using social media: a systematic review. J Med Internet Res. Jun 06, 2017;19(6):e195. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Athanasopoulou C, Sakellari E. Facebook and health information: content analysis of groups related to schizophrenia. Stud Health Technol Inform. 2015;213:255-258. [ Medline ]
• Conway M, O'Connor D. Social media, big data, and mental health: current advances and ethical implications. Curr Opin Psychol. Jun 2016;9:77-82. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Eysenbach G, Till JE. Ethical issues in qualitative research on internet communities. BMJ. Nov 10, 2001;323(7321):1103-1105. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Heilferty CM. Ethical considerations in the study of online illness narratives: a qualitative review. J Adv Nurs. May 2011;67(5):945-953. [ CrossRef ] [ Medline ]
• Roberts LD. Ethical issues in conducting qualitative research in online communities. Qualitative Research in Psychology. Apr 20, 2015;12(3):314-325. [ CrossRef ]
• Franzke A, Bechmann A, Zimmer M, Ess C, the Association of Internet Researchers. Internet research: ethical guidelines 3.0. Association of Internet Researchers. 2020. URL: https://aoir.org/reports/ethics3.pdf [accessed 2023-05-30]
• Markham A, Buchanan E. Ethical decision-making and internet research: recommendations from the AoIR ethics working committee (version 2.0). AoIR. 2012. URL: https://aoir.org/reports/ethics2.pdf [accessed 2023-06-05]
• The NCFREITSSH(. Ethical guidelines for internet research. 2022. URL: https:/​/www.​forskningsetikk.no/​en/​guidelines/​social-sciences-humanities-law-and-theology/​a-guide-to-internet-research-ethics [accessed 2023-06-13]
• Chapter 3: Rights of the data subject. General Data Protection Regulation (GDPR). European Commission. Official Journal of the European Union; 2018. URL: https://gdpr-info.eu/chapter-3/ [accessed 2023-11-29]
• Hand DJ. Aspects of data ethics in a changing world: where are we now? Big Data. Sep 01, 2018;6(3):176-190. [ FREE Full text ] [ CrossRef ] [ Medline ]
• British Psychological Society. Ethics guidelines for internet-mediated research. 2021. URL: https:/​/explore.​bps.org.uk/​binary/​bpsworks/​64374754e0c1dd30/​5a757d7c2d39a9837f0eedec1b9ba28fa5e9e38f0bffa9950b678ab727803959/​rep155_2021.​pdf [accessed 2023-05-30]
• Takats C, Kwan A, Wormer R, Goldman D, Jones HE, Romero D. Ethical and methodological considerations of Twitter data for public health research: systematic review. J Med Internet Res. Nov 29, 2022;24(11):e40380. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lathan HS, Kwan A, Takats C, Tanner JP, Wormer R, Romero D, et al. Ethical considerations and methodological uses of Facebook data in public health research: A systematic review. Soc Sci Med. Apr 2023;322:115807. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tanner J, Takats C, Lathan H, Kwan A, Wormer R, Romero D, et al. Approaches to research ethics in health research on YouTube: systematic review. J Med Internet Res. Oct 04, 2023;25:e43060. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Stockdale J, Cassell J, Ford E. "Giving something back": A systematic review and ethical enquiry into public views on the use of patient data for research in the United Kingdom and the Republic of Ireland. Wellcome Open Res. 2018;3:6. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ayers JW, Caputi TL, Nebeker C, Dredze M. Don't quote me: reverse identification of research participants in social media studies. NPJ Digit Med. 2018;1:30. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Aitken M, de St Jorre J, Pagliari C, Jepson R, Cunningham-Burley S. Public responses to the sharing and linkage of health data for research purposes: a systematic review and thematic synthesis of qualitative studies. BMC Med Ethics. Nov 10, 2016;17(1):73. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hunter RF, Gough A, O’Kane N, McKeown G, Fitzpatrick A, Walker T, et al. Ethical issues in social media research for public health. Am J Public Health. Mar 2018;108(3):343-348. [ CrossRef ]
• Ahmed OH, Sullivan SJ, Schneiders AG, McCrory P. iSupport: do social networking sites have a role to play in concussion awareness? Disabil Rehabil. 2010;32(22):1877-1883. [ CrossRef ] [ Medline ]
• Bender JL, Jimenez-Marroquin M, Jadad AR. Seeking support on Facebook: a content analysis of breast cancer groups. J Med Internet Res. Feb 04, 2011;13(1):e16. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gajaria A, Yeung E, Goodale T, Charach A. Beliefs about attention-deficit/hyperactivity disorder and response to stereotypes: youth postings in Facebook groups. J Adolesc Health. Jul 2011;49(1):15-20. [ CrossRef ] [ Medline ]
• Degroot JM. Maintaining relational continuity with the deceased on Facebook. Omega (Westport). Nov 01, 2012;65(3):195-212. [ CrossRef ]
• Donelle L, Booth RG. Health tweets: an exploration of health promotion on twitter. Online J Issues Nurs. Sep 30, 2012;17(3):4. [ FREE Full text ] [ Medline ]
• Rui JR, Chen Y, Damiano A. Health organizations providing and seeking social support: a Twitter-based content analysis. Cyberpsychol Behav Soc Netw. Sep 2013;16(9):669-673. [ CrossRef ] [ Medline ]
• Lyles CR, López A, Pasick R, Sarkar U. "5 mins of uncomfyness is better than dealing with cancer 4 a lifetime": an exploratory qualitative analysis of cervical and breast cancer screening dialogue on Twitter. J Cancer Educ. Mar 2013;28(1):127-133. [ CrossRef ] [ Medline ]
• Lee JL, DeCamp M, Dredze M, Chisolm MS, Berger ZD. What are health-related users tweeting? A qualitative content analysis of health-related users and their messages on twitter. J Med Internet Res. Oct 15, 2014;16(10):e237. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ahlwardt K, Heaivilin N, Gibbs J, Page J, Gerbert B, Tsoh JY. Tweeting about pain: comparing self-reported toothache experiences with those of backaches, earaches and headaches. J Am Dent Assoc. Jul 2014;145(7):737-743. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Struik LL, Baskerville NB. The role of Facebook in Crush the Crave, a mobile- and social media-based smoking cessation intervention: qualitative framework analysis of posts. J Med Internet Res. Jul 11, 2014;16(7):e170. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Abdel-Razig S, Anglade P, Ibrahim H. Impact of the COVID-19 pandemic on a physician group's WhatsApp chat: qualitative content analysis. JMIR Form Res. Dec 07, 2021;5(12):e31791. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Reddy A. Skincare in social media: analyzing prominent themes in online dermatologic discussions. Cureus. May 7, 2021:e14890. [ CrossRef ] [ Medline ]
• van der Pijl MSG, Hollander MH, van der Linden T, Verweij R, Holten L, Kingma E, et al. Left powerless: A qualitative social media content analysis of the Dutch #breakthesilence campaign on negative and traumatic experiences of labour and birth. PLoS One. 2020;15(5):e0233114. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Rath L, Vijiaratnam N, Skibina O. Alemtuzumab in multiple sclerosis: lessons from social media in enhancing patient care. Int J MS Care. 2017;19(6):323-328. [ CrossRef ] [ Medline ]
• Bloom R, Beck S, Chou W, Reblin M, Ellington L. In their own words: experiences of caregivers of adults with cancer as expressed on social media. ONF. Sep 1, 2019;46(5):617-630. [ CrossRef ]
• Grumme V, Gordon S. Social media use by transplant recipients for support and healing. Comput Inform Nurs. Dec 2016;34(12):570-577. [ CrossRef ] [ Medline ]
• Haug NA, Bielenberg J, Linder SH, Lembke A. Assessment of provider attitudes toward #naloxone on Twitter. Subst Abus. 2016;37(1):35-41. [ CrossRef ] [ Medline ]
• Oren E, Martinez L, Hensley RE, Jain P, Ahmed T, Purnajo I, et al. Twitter communication during an outbreak of hepatitis A in San Diego, 2016–2018. Am J Public Health. Oct 2020;110(S3):S348-S355. [ CrossRef ]
• Parker C, Zomer E, Liew D, Ayton D. Characterising experiences with acute myeloid leukaemia using an Instagram content analysis. PLoS One. 2021;16(5):e0250641. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kurko T, Linden K, Kolstela M, Pietilä K, Airaksinen M. Is nicotine replacement therapy overvalued in smoking cessation? Analysis of smokers' and quitters' communication in social media. Health Expect. Dec 2015;18(6):2962-2977. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Reuter K, Lee D. Perspectives toward seeking treatment among patients with psoriasis: protocol for a Twitter content analysis. JMIR Res Protoc. Feb 18, 2021;10(2):e13731. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mollema L, Harmsen IA, Broekhuizen E, Clijnk R, De Melker H, Paulussen T, et al. Disease detection or public opinion reflection? Content analysis of tweets, other social media, and online newspapers during the measles outbreak in The Netherlands in 2013. J Med Internet Res. May 26, 2015;17(5):e128. [ FREE Full text ] [ CrossRef ] [ Medline ]
• O'Hagan ET, Traeger AC, Bunzli S, Leake HB, Schabrun SM, Wand BM, et al. What do people post on social media relative to low back pain? A content analysis of Australian data. Musculoskelet Sci Pract. Aug 2021;54:102402. [ CrossRef ] [ Medline ]
• Zhou F, Zhang W, Cai H, Cao Y. Portrayals of 2v, 4v and 9vHPV vaccines on Chinese social media: a content analysis of hot posts on Sina Weibo. Hum Vaccin Immunother. Nov 02, 2021;17(11):4433-4441. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Golder S, Bach M, O'Connor K, Gross R, Hennessy S, Gonzalez Hernandez G. Public perspectives on anti-diabetic drugs: exploratory analysis of Twitter posts. JMIR Diabetes. Jan 26, 2021;6(1):e24681. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mercier R, Senter K, Webster R, Henderson RA. Instagram users? experiences of miscarriage. Obstet Gynecol 2020 Jan;? Jan 2020;135(1):166-173. [ CrossRef ]
• Alghamdi A, Abumelha K, Allarakia J, Al-Shehri A. Conversations and misconceptions about chemotherapy in Arabic tweets: content analysis. J Med Internet Res. Jul 29, 2020;22(7):e13979. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Golder S, O'Connor K, Hennessy S, Gross R, Gonzalez-Hernandez G. Assessment of beliefs and attitudes about statins posted on Twitter: a qualitative study. JAMA Netw Open. Jun 01, 2020;3(6):e208953. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Stekelenburg N, Horsham C, O'Hara M, Janda M. Using social media to determine the affective and cognitive components of tweets about sunburn. Dermatology. Feb 27, 2020;236(2):75-80. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Meeking K. Patients' experiences of radiotherapy: Insights from Twitter. Radiography (Lond). Aug 2020;26(3):e146-e151. [ CrossRef ] [ Medline ]
• Shah S, Bradbury-Jones C, Taylor J. Using Facebook to tell stories of premature ageing and sexual and reproductive healthcare across the life course for women with cerebral palsy in the UK and USA. BMJ Open. Feb 17, 2020;10(2):e032172. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pretorius K, Choi E, Kang S, Mackert M. Sudden infant death syndrome on Facebook: qualitative descriptive content analysis to guide prevention efforts. J Med Internet Res. Jul 30, 2020;22(7):e18474. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Karmegam D, Mapillairaju B. What people share about the COVID-19 outbreak on Twitter? An exploratory analysis. BMJ Health Care Inform. Nov 2020;27(3):e100133. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hairston TK, Links AR, Harris V, Tunkel DE, Walsh J, Beach MC, et al. Evaluation of parental perspectives and concerns about pediatric tonsillectomy in social media. JAMA Otolaryngol Head Neck Surg. Jan 01, 2019;145(1):45-52. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Årsand E, Bradway M, Gabarron E. What are diabetes patients versus health care personnel discussing on social media? J Diabetes Sci Technol. Mar 2019;13(2):198-205. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jiang X, Jiang W, Cai J, Su Q, Zhou Z, He L, et al. Characterizing media content and effects of organ donation on a social media platform: content analysis. J Med Internet Res. Mar 12, 2019;21(3):e13058. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Oser TK, Minnehan KA, Wong G, Parascando J, McGinley E, Radico J, et al. Using social media to broaden understanding of the barriers and facilitators to exercise in adults with type 1 diabetes. J Diabetes Sci Technol. May 2019;13(3):457-465. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sutton J, Vos SC, Olson MK, Woods C, Cohen E, Gibson CB, et al. Lung cancer messages on Twitter: content analysis and evaluation. J Am Coll Radiol. Jan 2018;15(1 Pt B):210-217. [ CrossRef ] [ Medline ]
• Thomas J, Prabhu AV, Heron DE, Beriwal S. Twitter and brachytherapy: An analysis of "tweets" over six years by patients and health care professionals. Brachytherapy. 2018;17(6):1004-1010. [ CrossRef ] [ Medline ]
• Kelly-Hedrick M, Grunberg PH, Brochu F, Zelkowitz P. "It's totally okay to be sad, but never lose hope": content analysis of infertility-related videos on YouTube in relation to viewer preferences. J Med Internet Res. May 23, 2018;20(5):e10199. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gage-Bouchard EA, LaValley S, Mollica M, Beaupin LK. Cancer communication on social media: examining how cancer caregivers use Facebook for cancer-related communication. Cancer Nurs. 2017;40(4):332-338. [ CrossRef ] [ Medline ]
• Rael CT, Pierre D, Frye V, Kessler D, Duffy L, Malos N, et al. Evaluating blood donor experiences and barriers/facilitators to blood donation in the United States using YouTube video content. Transfusion. Sep 2021;61(9):2650-2657. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fisher S, Jehassi A, Ziv M. Hidradenitis suppurativa on Facebook: thematic and content analyses of patient support group. Arch Dermatol Res. Aug 2020;312(6):421-426. [ CrossRef ] [ Medline ]
• Abdoli S, Hessler D, Vora A, Smither B, Stuckey H. Descriptions of diabetes burnout from individuals with Type 1 diabetes: an analysis of YouTube videos. Diabet Med. Aug 2020;37(8):1344-1351. [ CrossRef ] [ Medline ]
• Myneni S, Lewis B, Singh T, Paiva K, Kim SM, Cebula AV, et al. Diabetes self-management in the age of social media: large-scale analysis of peer interactions using semiautomated methods. JMIR Med Inform. Jun 30, 2020;8(6):e18441. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Charlie AM, Gao Y, Heller SL. What do patients want to know? questions and concerns regarding mammography expressed through social media. J Am Coll Radiol. Oct 2018;15(10):1478-1486. [ CrossRef ] [ Medline ]
• Watts G, Christou P, Antonarakis G. Experiences of individuals concerning combined orthodontic and orthognathic surgical treatment: a qualitative Twitter analysis. Med Princ Pract. 2018;27(3):227-235. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pai RR, Alathur S. Assessing mobile health applications with Twitter analytics. Int J Med Inform. May 2018;113:72-84. [ CrossRef ] [ Medline ]
• Cheng TY, Liu L, Woo BK. Analyzing Twitter as a platform for Alzheimer-related dementia awareness: thematic analyses of tweets. JMIR Aging. Dec 10, 2018;1(2):e11542. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bridges N, Howell G, Schmied V. Exploring breastfeeding support on social media. Int Breastfeed J. 2018;13:22. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Anderson JG, Hundt E, Dean M, Keim-Malpass J, Lopez RP. "The Church of Online Support": examining the use of blogs among family caregivers of persons with dementia. J Fam Nurs. Feb 2017;23(1):34-54. [ CrossRef ] [ Medline ]
• Kearney MD, Selvan P, Hauer MK, Leader AE, Massey PM. Characterizing HPV vaccine sentiments and content on Instagram. Health Educ Behav. Dec 2019;46(2_suppl):37-48. [ CrossRef ] [ Medline ]
• Davies SH, Langer MD, Klein A, Gonzalez-Hernandez G, Dowshen N. Adolescent perceptions of menstruation on Twitter: opportunities for advocacy and education. J Adolesc Health. Jul 2022;71(1):94-104. [ CrossRef ] [ Medline ]
• Van Diepen C, Rosales Valdes D. A content analysis on the perceptions of LGBTQ+ (centred) health care on Twitter. Health Expect. Dec 2022;25(6):3238-3245. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pleasure ZH, Frohwirth LF, Li N, Polis CB. A content analysis of Reddit users' posts about challenges to contraceptive care-seeking during COVID-19-related restrictions in the United States. J Health Commun. Oct 03, 2022;27(10):746-754. [ CrossRef ] [ Medline ]
• Du Y, Dennis B, Ramirez V, Li C, Wang J, Meireles CL. Experiences and disease self-management in individuals living with chronic kidney disease: qualitative analysis of the National Kidney Foundation's online community. BMC Nephrol. Mar 04, 2022;23(1):88. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sadek Attalla S, Ow NL, McNarry M, De Simoni A. Experiences of exercise in patients with asthma: a qualitative analysis of discussions in a UK asthma online community. BJGP Open. Apr 29, 2022;6(3):BJGPO.2021.0162. [ CrossRef ]
• Bartmess M, Talbot C, O'Dwyer ST, Lopez RP, Rose KM, Anderson JG. Using Twitter to understand perspectives and experiences of dementia and caregiving at the beginning of the COVID-19 pandemic. Dementia (London). Jul 2022;21(5):1734-1752. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Brewer G, Centifanti L, Caicedo JC, Huxley G, Peddie C, Stratton K, et al. Experiences of mental distress during COVID-19: thematic analysis of discussion forum posts for anxiety, depression, and obsessive-compulsive disorder. Illn Crises Loss. Oct 2022;30(4):795-811. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Castillo LIR, Hadjistavropoulos T, Beahm J. Social media discussions about long-term care and the COVID-19 pandemic. J Aging Stud. Dec 2022;63:101076. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Colaceci S, Anderson G, Ricciuto V, Montinaro D, Alazraki G, Mena-Tudela D. Experiences of birth during COVID-19 pandemic in Italy and Spain: A thematic analysis. Int J Environ Res Public Health. Jun 18, 2022;19(12):7488. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tripathi SD, Parker PD, Prabhu AV, Thomas K, Rodriguez A. An examination of patients and caregivers on Reddit Navigating Brain Cancer: content analysis of the brain tumor Subreddit. JMIR Cancer. Jun 22, 2022;8(2):e35324. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jina-Pettersen N. Fear, neglect, coercion, and dehumanization: is inpatient psychiatric trauma contributing to a public health crisis? J Patient Exp. 2022;9:23743735221079138. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Rossi NA, Devarajan K, Chokshi SN, Ochoa VJ, Benavidez M, Malaya LT, et al. Social media depictions of cochlear implants: An Instagram and TikTok analysis. Otol Neurotol. 2023;44(1):e13-e21. [ CrossRef ]
• Singh GK, Rego J, Chambers S, Fox J. Health professionals' perspectives of the role of palliative care during COVID-19: content analysis of articles and blogs posted on Twitter. Am J Hosp Palliat Care. Apr 2022;39(4):487-493. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Koly KN, Tasnim Z, Ahmed S, Saba J, Mahmood R, Farin FT, et al. Mental healthcare-seeking behavior of women in Bangladesh: content analysis of a social media platform. BMC Psychiatry. Dec 19, 2022;22(1):797. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Belcher R, Sim D, Meykler M, Owens-Walton J, Hassan N, Rubin R, et al. A qualitative analysis of female Reddit users' experiences with low libido: how do women perceive their changes in sexual desire? J Sex Med. Feb 27, 2023;20(3):287-297. [ CrossRef ] [ Medline ]
• Culp F, Wu Y, Wu D, Ren Y, Raynor P, Hung P, et al. Understanding alcohol use discourse and stigma patterns in perinatal care on Twitter. Healthcare (Basel). Nov 26, 2022;10(12):2375. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ferrey A, Ashworth G, Cabling M, Rundblad G, Ismail K. A thematic analysis of YouTube comments on a television documentary titled 'Diabulimia: The world's most dangerous eating disorder'. Diabet Med. May 2023;40(5):e15025. [ CrossRef ] [ Medline ]
• Watt S, Salway T, Gómez-Ramírez O, Ablona A, Barton L, Chang H, et al. Rumination, risk, and response: a qualitative analysis of sexual health anxiety among online sexual health chat service users. Sex. Health. May 23, 2022;19(3):182-191. [ CrossRef ]
• Naganathan G, Bilgen I, Cleland J, Reel E, Cil T. #COVID19 and #Breastcancer: a qualitative analysis of tweets. Curr Oncol. Nov 08, 2022;29(11):8483-8500. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Potgieter I, Hoare DJ, Fackrell K. Hyperacusis in children: a thematic analysis of discussions in online forums. Am J Audiol. Mar 03, 2022;31(1):166-174. [ CrossRef ]
• Manning Hutson M, Hosking SM, Mantalvanos S, Berk M, Pasco J, Dunning T. What injured workers with complex claims look for in online communities: netnographic analysis. J Med Internet Res. Apr 07, 2022;24(4):e17180. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lawless MT, Hunter SC, Pinero de Plaza MA, Archibald MM, Kitson AL. "You Are By No Means Alone": a netnographic study of self-care support in an online community for older adults. Qual Health Res. Nov 2022;32(13):1935-1951. [ CrossRef ] [ Medline ]
• Keim-Malpass J, Mitchell EM, Sun E, Kennedy C. Using Twitter to understand public perceptions regarding the #HPV vaccine: opportunities for public health nurses to engage in social marketing. Public Health Nurs. Jul 2017;34(4):316-323. [ CrossRef ] [ Medline ]
• Yamada R, Rasmussen KM, Felice JP. "What is 'enough,' and how do I make it?": a qualitative examination of questions mothers ask on social media about pumping and providing an adequate amount of milk for their infants. Breastfeed Med. 2019;14(1):17-21. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Osadchiy V, Mills JN, Eleswarapu SV. Understanding patient anxieties in the social media era: qualitative analysis and natural language processing of an online male infertility community. J Med Internet Res. Mar 10, 2020;22(3):e16728. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Thomas TH, Nauth-Shelley K, Thompson MA, Attai DJ, Katz MS, Graham D, et al. The needs of women treated for ovarian cancer: results from a #gyncsm Twitter chat. J Patient Cent Res Rev. 2018;5(2):149-157. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mehta N, Zhu L, Lam K, Stall NM, Savage R, Read SH, et al. Health forums and Twitter for dementia research: opportunities and considerations. J Am Geriatr Soc. Dec 2020;68(12):2881-2889. [ CrossRef ] [ Medline ]
• Gupta R, Ariefdjohan M. Mental illness on Instagram: a mixed method study to characterize public content, sentiments, and trends of antidepressant use. J Ment Health. Aug 2021;30(4):518-525. [ CrossRef ] [ Medline ]
• He L, He C, Reynolds T, Bai Q, Huang Y, Li C, et al. Why do people oppose mask wearing? A comprehensive analysis of U.S. tweets during the COVID-19 pandemic. J Am Med Inform Assoc. Jul 14, 2021;28(7):1564-1573. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Loeb S, Mihalcea R, Perez-Rosas V, Xu A, Taylor J, Byrne N, et al. Leveraging social media as a thermometer to gauge patient and caregiver concerns: COVID-19 and prostate cancer. Eur Urol Open Sci. Mar 2021;25:1-4. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Graf I, Gerwing H, Hoefer K, Ehlebracht D, Christ H, Braumann B. Social media and orthodontics: A mixed-methods analysis of orthodontic-related posts on Twitter and Instagram. Am J Orthod Dentofacial Orthop. Aug 2020;158(2):221-228. [ CrossRef ] [ Medline ]
• Boon-Itt S, Skunkan Y. Public perception of the COVID-19 pandemic on Twitter: sentiment analysis and topic modeling study. JMIR Public Health Surveill. Nov 11, 2020;6(4):e21978. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wang J, Zhou Y, Zhang W, Evans R, Zhu C. Concerns expressed by Chinese social media users during the COVID-19 pandemic: content analysis of Sina Weibo microblogging data. J Med Internet Res. Nov 26, 2020;22(11):e22152. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wahbeh A, Nasralah T, Al-Ramahi M, El-Gayar O. Mining physicians' opinions on social media to obtain insights into COVID-19: mixed methods analysis. JMIR Public Health Surveill. Jun 18, 2020;6(2):e19276. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Çınar S, Boztepe H, Prof. The use of social media among parents of infants with cleft lip and/or palate. J Pediatr Nurs. 2020;54:e91-e96. [ CrossRef ] [ Medline ]
• Jiang T, Osadchiy V, Mills JN, Eleswarapu SV. Is It all in my head? self-reported psychogenic erectile dysfunction and depression are common among young men seeking advice on social media. Urology. Aug 2020;142:133-140. [ CrossRef ] [ Medline ]
• Dzubur E, Khalil C, Almario CV, Noah B, Minhas D, Ishimori M, et al. Patient concerns and perceptions regarding biologic therapies in ankylosing spondylitis: insights from a large-scale survey of social media platforms. Arthritis Care Res (Hoboken). Feb 2019;71(2):323-330. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gonzalez G, Vaculik K, Khalil C, Zektser Y, Arnold C, Almario CV, et al. Women's experience with stress urinary incontinence: insights from social media analytics. Journal of Urology. May 2020;203(5):962-968. [ CrossRef ]
• Jha SR, McDonagh J, Prichard R, Newton PJ, Hickman LD, Fung E, et al. #Frailty: A snapshot Twitter report on frailty knowledge translation. Australas J Ageing. Dec 2018;37(4):309-312. [ CrossRef ] [ Medline ]
• Litchman ML, Snider C, Edelman LS, Wawrzynski SE, Gee PM. Diabetes online community user perceptions of successful aging with diabetes: analysis of a #DSMA Tweet chat. JMIR Aging. Jun 22, 2018;1(1):e10176. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jones J, Pradhan M, Hosseini M, Kulanthaivel A, Hosseini M. Novel approach to cluster patient-generated data into actionable topics: case study of a web-based breast cancer forum. JMIR Med Inform. Nov 29, 2018;6(4):e45. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gonzalez G, Vaculik K, Khalil C, Zektser Y, Arnold C, Almario CV, et al. Using large-scale social media analytics to understand patient perspectives about urinary tract infections: thematic analysis. J Med Internet Res. Jan 25, 2022;24(1):e26781. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sormunen T, Westerbotn M, Aanesen A, Fossum B, Karlgren K. Social media in the infertile community-using a text analysis tool to identify the topics of discussion on the multitude of infertility blogs. Womens Health (Lond). 2021;17:17455065211063280. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Berkovic D, Ackerman IN, Briggs AM, Ayton D. Tweets by people with arthritis during the COVID-19 pandemic: content and sentiment analysis. J Med Internet Res. Dec 03, 2020;22(12):e24550. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Della Rosa S, Sen F. Health topics on Facebook Groups: content analysis of posts in multiple sclerosis communities. Interact J Med Res. Feb 11, 2019;8(1):e10146. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Awofeso N, Imam SA, Ahmed A. Content analysis of media coverage of childhood obesity topics in UAE newspapers and popular social media platforms, 2014-2017. Int J Health Policy Manag. Feb 01, 2019;8(2):81-89. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Smith DJ, Mac VV, Hertzberg VS. Using Twitter for nursing research: a tweet analysis on heat illness and health. J Nurs Scholarsh. May 2021;53(3):343-350. [ CrossRef ] [ Medline ]
• Green BM, Van Horn KT, Gupte K, Evans M, Hayes S, Bhowmick A. Assessment of adaptive engagement and support model for people with chronic health conditions in online health communities: combined content analysis. J Med Internet Res. Jul 07, 2020;22(7):e17338. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Osakwe ZT, Cortés YI. Impact of COVID-19: a text mining analysis of Twitter data in Spanish language. Hisp Health Care Int. Dec 2021;19(4):239-245. [ CrossRef ] [ Medline ]
• Sümeyye Yorulmaz D, Karadeniz H. Vaccination refusal debate on social media in Turkey: a content analysis of the comments on Instagram blogs. Iran J Public Health. Mar 2022;51(3):615-623. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Choi E, Becker H, Kim S. A blog text analysis to explore psychosocial support in adolescents and young adults with cancer. Cancer Nurs. Dec 11, 2022;46(2):143-151. [ CrossRef ]
• Jun J, Wickersham K, Zain A, Ford R, Zhang N, Ciccarelli C, et al. Cancer and COVID-19 vaccines on Twitter: the voice and vaccine attitude of cancer community. J Health Commun. Jan 02, 2023;28(1):1-14. [ CrossRef ] [ Medline ]
• Yashpal S, Raghunath A, Gencerliler N, Burns LE. Exploring public perceptions of dental care affordability in the United States: mixed method analysis via Twitter. JMIR Form Res. Jul 01, 2022;6(7):e36315. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Damier P, Henderson EJ, Romero-Imbroda J, Galimam L, Kronfeld N, Warnecke T. Impact of off-time on quality of life in Parkinson's patients and their caregivers: insights from social media. Parkinsons Dis. 2022;2022:1800567. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Miller WR, Malloy C, Mravec M, Sposato MF, Groves D. Nursing in the spotlight: Talk about nurses and the nursing profession on Twitter during the early COVID-19 pandemic. Nurs Outlook. 2022;70(4):580-589. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hriberšek M, Eibensteiner F, Kapral L, Teufel A, Nawaz FA, Cenanovic M, et al. "Loved ones are not 'visitors' in a patient's life"-The importance of including loved ones in the patient's hospital stay: An international Twitter study of #HospitalsTalkToLovedOnes in times of COVID-19. Front Public Health. 2023;11:1100280. [ FREE Full text ] [ CrossRef ] [ Medline ]
• O'Sullivan L, Killeen RP, Doran P, Crowley RK. Adherence with reporting of ethical standards in COVID-19 human studies: a rapid review. BMC Med Ethics. Jun 28, 2021;22(1):80. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Denecke K, Bamidis P, Bond C, Gabarron E, Househ M, Lau AYS, et al. Ethical issues of social media usage in healthcare. Yearb Med Inform. Aug 13, 2015;10(1):137-147. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ferretti A, Ienca M, Sheehan M, Blasimme A, Dove ES, Farsides B, et al. Ethics review of big data research: What should stay and what should be reformed? BMC Med Ethics. Apr 30, 2021;22(1):51. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kohn T, Shore C. The Ethics of University Ethics Committees: risk management and the research imagination. Death Public Univ Uncertain Futur High Educ Knowl Econ. May 2017:229-249. [ CrossRef ]
• Friesen P, Redman B, Caplan A. Of straws, camels, research regulation, and IRBs. Ther Innov Regul Sci. Jul 2019;53(4):526-534. [ CrossRef ] [ Medline ]
• Sellers C, Samuel G, Derrick G. Reasoning "uncharted territory": notions of expertise within ethics review panels assessing research use of social media. J Empir Res Hum Res Ethics. 2020;15(1-2):28-39. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kious B. The Nuremberg Code: its history and implications. Princet J Bioeth. 2001;4:7-19. [ Medline ]
• World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. Nov 27, 2013;310(20):2191-2194. [ CrossRef ] [ Medline ]
• Department of Health‚ Education‚Welfare, National Commission for the Protection of Human Subjects of BiomedicalBehavioral Research. The Belmont Report. Ethical principles and guidelines for the protection of human subjects of research. J Am Coll Dent. 2014;81(3):4-13. [ Medline ]
• Wongkoblap A, Vadillo MA, Curcin V. Researching mental health disorders in the era of social media: systematic review. J Med Internet Res. Jun 29, 2017;19(6):e228. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Alim S. An initial exploration of ethical research practices regarding automated data extraction from online social media user profiles. First Monday. Jul 2014;19(7):105-127. [ CrossRef ] [ Medline ]
• Vitak J, Proferes N, Shilton K, Ashktorab Z. Ethics regulation in social computing research: examining the role of institutional review boards. J Empir Res Hum Res Ethics. Dec 2017;12(5):372-382. [ CrossRef ] [ Medline ]
• Moreno MA, Goniu N, Moreno PS, Diekema D. Ethics of social media research: common concerns and practical considerations. Cyberpsychol Behav Soc Netw. Sep 2013;16(9):708-713. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Metcalf J, Crawford K. Where are human subjects in Big Data research? The emerging ethics divide. Big Data & Society. Jun 01, 2016;3(1):205395171665021. [ CrossRef ]
• Usha Lawrance J, Nayahi Jesudhasan JV. Privacy preserving parallel clustering based anonymization for big data using MapReduce framework. Applied Artificial Intelligence. Oct 17, 2021;35(15):1587-1620. [ CrossRef ]
• Wilkinson D, Thelwall M. Researching personal information on the public web. Social Science Computer Review. Aug 17, 2010;29(4):387-401. [ CrossRef ]
• International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals: writing and editing for biomedical publication. Croat Med J. Dec 2003;44(6):770-783. [ FREE Full text ] [ Medline ]
• Boyd D, Crawford K. Critical questions for big data. Information, Communication & Society. Jun 2012;15(5):662-679. [ CrossRef ]
• Bond CS, Ahmed OH, Hind M, Thomas B, Hewitt-Taylor J. The conceptual and practical ethical dilemmas of using health discussion board posts as research data. J Med Internet Res. Jun 07, 2013;15(6):e112. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Markham A, Tiidenberg K, Herman A. Ethics as methods: doing ethics in the era of big data research—introduction. Social Media + Society. Jul 19, 2018;4(3):205630511878450-205630511878416. [ CrossRef ]

Abbreviations

Edited by T Leung, T de Azevedo Cardoso; submitted 02.08.23; peer-reviewed by E Zibrowski, J Scheibner; comments to author 06.10.23; revised version received 29.11.23; accepted 16.04.24; published 17.05.24.

©Yujie Zhang, Jiaqi Fu, Jie Lai, Shisi Deng, Zihan Guo, Chuhan Zhong, Jianyao Tang, Wenqiong Cao, Yanni Wu. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 17.05.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Velentgas P, Dreyer NA, Nourjah P, et al., editors. Developing a Protocol for Observational Comparative Effectiveness Research: A User's Guide. Rockville (MD): Agency for Healthcare Research and Quality (US); 2013 Jan.

Developing a Protocol for Observational Comparative Effectiveness Research: A User's Guide.

• Hardcopy Version at Agency for Healthcare Research and Quality

Chapter 1 Study Objectives and Questions

Scott R Smith , PhD.

The steps involved in the process of developing research questions and study objectives for conducting observational comparative effectiveness research (CER) are described in this chapter. It is important to begin with identifying decisions under consideration, determining who the decisionmakers and stakeholders in the specific area of research under study are, and understanding the context in which decisions are being made. Synthesizing the current knowledge base and identifying evidence gaps is the next important step in the process, followed by conceptualizing the research problem, which includes developing questions that address the gaps in existing evidence. Understanding the stage of knowledge that the study is designed to address will come from developing these initial questions. Identifying which questions are critical to reduce decisional uncertainty and minimize gaps in the current knowledge base is an important part of developing a successful framework. In particular, it is beneficial to look at what study populations, interventions, comparisons, outcomes, timeframe, and settings (PICOTS framework) are most important to decisionmakers in weighing the balance of harms and benefits of action. Some research questions are easier to operationalize than others, and study limitations should be recognized and accepted from an early stage. The level of new scientific evidence that is required by the decisionmaker to make a decision or to take action must be recognized. Lastly, the magnitude of effect must be specified. This can mean defining what is a clinically meaningful difference in the study endpoints from the perspective of the decisionmaker and/or defining what is a meaningful difference from the patient's perspective.

The foundation for designing a new research protocol is the study's objectives and the questions that will be investigated through its implementation. All aspects of study design and analysis are based on the objectives and questions articulated in a study's protocol. Consequently, it is exceedingly important that a study's objectives and questions be formulated meticulously and written precisely in order for the research to be successful in generating new knowledge that can be used to inform health care decisions and actions.

An important aspect of CER 1 and other forms of translational research is the potential for early involvement and inclusion of patients and other stakeholders to collaborate with researchers in identifying study objectives, key questions, major study endpoints, and the evidentiary standards that are needed to inform decisionmaking. The involvement of stakeholders in formulating the research questions increases the applicability of the study to the end-users and facilitates appropriate translation of the results into health care practice and use by patient communities. While stakeholders may be defined in multiple ways, for the purposes of this User's Guide , a broad definition will be used. Hence, stakeholders are defined as individuals or organizations that use scientific evidence for decisionmaking and therefore have an interest in the results of new research. Implicit in this definition of stakeholders is the importance for stakeholders to understand the scientific process, including considerations of bioethics and the limitations of research, particularly with regard to studies involving human subjects. Ideally, stakeholders also should express commitment to using objective scientific evidence to inform their decisionmaking and recognize that disregarding sound scientific methods often will undermine decisionmaking. For stakeholder organizations, it is also advantageous if the organization has well-established processes for transparently reviewing and incorporating research findings into decisions as well as organized channels for disseminating research results.

There are at least seven essential steps in the conceptualization and development of a research question or set of questions for an observational CER protocol. These steps are presented as a general framework in Table 1.1 below and elaborated upon in the subsequent sections of this chapter. The framework is based on the principle that researchers and stakeholders will work together to objectively lay out the research problems, research questions, study objectives, and key parameters for which scientific evidence is needed to inform decisionmaking or health care actions. The intent of this framework is to facilitate communication between researchers and stakeholders in conceptualizing the research problem and the design of a study (or a program of research involving a series of studies) in order to maximize the potential that new knowledge will be created from the research with results that can inform decisionmaking. To do this, research results must be relevant, applicable, unbiased and sufficient to meet the evidentiary threshold for decisionmaking or action by stakeholders. In order for the results to be valid and credible, all persons involved must be committed to protecting the integrity of the research from bias and conflicts of interest. Most importantly, the study must be designed to protect the rights, welfare, and well-being of subjects involved in the research.

Framework for developing and conceptualizing a CER protocol.

• Identifying Decisions, Decisionmakers, Actions, and Context

In order for research findings to be useful for decisionmaking, the study protocol should clearly articulate the decisions or actions for which stakeholders seek new scientific evidence. While only some studies may be sufficiently robust for making decisions or taking action, statements that describe the stakeholders' decisions will help those who read the protocol understand the rationale for the study and its potential for informing decisions or for translating the findings into changes in health care practices. This information also improves the ability of protocol readers to understand the purpose of the study so they can critically review its design and provide recommendations for ways it may be potentially improved. If stakeholders have a need to make decisions within a critical time frame for regulatory, ethical, or other reasons, this interval should be expressed to researchers and described in the protocol. In some cases, the time frame for decisionmaking may influence the choice of outcomes that can be studied and the study designs that can be used. For some stakeholders' questions, research and decisionmaking may need to be divided into stages, since it may take years for outcomes with long lag times to occur, and research findings will be delayed until they do.

In writing this section of the protocol, investigators should ask stakeholders to describe the context in which the decision will be made or actions will be taken. This context includes the background and rationale for the decision, key areas of uncertainty and controversies surrounding the decision, ways scientific evidence will be used to inform the decision, the process stakeholders will use to reach decisions based on scientific evidence, and a description of the key stakeholders who will use or potentially be affected by the decision. By explaining these contextual factors that surround the decision, investigators will be able to work with stakeholders to determine the study objectives and other major parameters of the study. This work also provides the opportunity to discuss how the tools of science can be applied to generate new evidence for informing stakeholder decisions and what limits may exist in those tools. In addition, this initial step begins to clarify the number of analyses necessary to generate the evidence that stakeholders need to make a decision or take other actions with sufficient certainty about the outcomes of interest. Finally, the contextual information facilitates advance planning and discussions by researchers and stakeholders about approaches to translation and implementation of the study findings once the research is completed.

• Synthesizing the Current Knowledge Base

In designing a new study, investigators should conduct a comprehensive review of the literature, critically appraise published studies, and synthesize what is known related to the research objectives. Specifically, investigators should summarize in the protocol what is known about the efficacy, effectiveness, and safety of the interventions and about the outcomes being studied. Furthermore, investigators should discuss measures used in prior research and whether these measures have changed over time. These descriptions will provide background on the knowledge base for the current protocol. It is equally important to identify which elements of the research problem are unknown because evidence is absent, insufficient, or conflicting.

For some research problems, systematic reviews of the literature may be available and can be useful resources to guide the study design. The AHRQ Evidence-based Practice Centers 2 and the Cochrane Collaboration 3 are examples of established programs that conduct thorough systematic reviews, technology assessments, and specialized comparative effectiveness reviews using standardized methods. When available, systematic reviews and technology assessments should be consulted as resources for investigators to assess the current knowledge base when designing new studies and working with stakeholders.

When reviewing the literature, investigators and stakeholders should identify the most relevant studies and guidelines about the interventions that will be studied. This will allow readers to understand how new research will add to the existing knowledge base. If guidelines are a source of information, then investigators should examine whether these guidelines have been updated to incorporate recent literature. In addition, investigators should assess the health sciences literature to determine what is known about expected effects of the interventions based on current understanding of the pathophysiology of the target condition. Furthermore, clinical experts should be consulted to help identify gaps in current knowledge based on their expertise and interactions with patients. Relevant questions to ask to assess the current knowledge base for development of an observational CER study protocol are:

• What are the most relevant studies and guidelines about the interventions, and why are these studies relevant to the protocol (e.g., because of the study findings, time period conducted, populations studied, etc.)?
• Are there differences in recommendations from clinical guidelines that would indicate clinical equipoise?
• What else is known about the expected effects of the interventions based on current understanding of the pathophysiology of the targeted condition?
• What do clinical experts say about gaps in current knowledge?
• Conceptualizing the Research Problem

In designing studies for addressing stakeholder questions, investigators should engage multiple stakeholders in discussions about how the research problem is conceptualized from the stakeholders' perspectives. These discussions will aid in designing a study that can be used to inform decisionmaking. Together, investigators and stakeholders should work collaboratively to determine the major objectives of the study based on the health care decisions facing stakeholders. As pointed out by Heckman, 4 research objectives should be formalized outside considerations of available data and the inferences that can be made from various statistical estimation approaches. Doing so will allow the study objectives to be determined by stakeholder needs rather than the availability of existing data. A thorough discussion of these considerations is beyond the scope of this chapter, but some important considerations are summarized in supplement 1 of this User's Guide.

In order to conceptualize the problem, stakeholders and other experts should be asked to describe the potential relationships between the intervention and important health outcomes. This description will help researchers develop preliminary hypotheses about the stated relationships. Likewise, stakeholders, researchers, and other experts should be asked to enumerate all major assumptions that affect the conceptualization of the research problem, but will not be directly examined in the study. These assumptions should be described in the study protocol and in reporting final study results. By clearly stating the assumptions, protocol reviewers will be better able to assess how the assumptions may influence the study results.

Based on the conceptualization of the research problem, investigators and stakeholders should make use of applicable scientific theory in designing the study protocol and developing the analytic plan. Research that is designed using a validated theory has a higher potential to reach valid conclusions and improve the overall understanding of a phenomenon. In addition, theory will aid in the interpretation of the study findings, since these results can be put in context with the theory and with past research. Depending on the nature of the inquiry, theory from specific disciplines such as health behavior, sociology, or biology could be the basis for designing the study. In addition, the research team should work with stakeholders to develop a conceptual model or framework to guide the implementation of the study. The protocol should also contain one or more figures that summarize the conceptual model or framework as it applies to the study. These figures will allow readers to understand the theoretical or conceptual basis for the study and how the theory is operationalized for the specific study. The figures should diagram relationships between study variables and outcomes to help readers of the protocol visualize relationships that will be examined in the study.

For research questions about causal associations between exposures and outcomes, causal models such as directed acyclic graphs (DAGs) may be useful tools in designing the conceptual framework for the study and developing the analytic plan. The value of DAGs in the context of refining study questions is that they make assumptions explicit in ways that can clarify gaps in knowledge. Free software such as DAGitty is available for creating, editing, and analyzing causal models. A thorough discussion of DAGs is beyond the scope of this chapter, but more information about DAGs is available in supplement 2 of this User's Guide.

The following list of questions may be useful for defining and describing a study's conceptual framework in a CER protocol:

• What are the main objectives of the study, as related to specific decisions to be made?
• What are the major assumptions of decisionmakers, investigators, and other experts about the problem or phenomenon being studied?
• What relationships, if any, do experts hypothesize exist between interventions and outcomes?

What is known about each element of the model?

Can relationships be expressed by causal diagrams?

• Determining the Stage of Knowledge Development for the Study Design

The scientific method is a process of observation and experimentation in order for the evidence base to be expanded as new knowledge is developed. Therefore, stakeholders and investigators should consider whether a program of research comprising a sequential or concurrent series of studies, rather than a single study, is needed to adequately make a decision. Staging the research into multiple studies and making interim decisions may improve the final decision and make judicious use of scarce research resources. In some cases, the results of preliminary studies, descriptive epidemiology, or pilot work may be helpful in making interim decisions and designing further research. Overall, a planned series of related studies or a program of research may be needed to adequately address stakeholders' decisions.

An example of a structured program of research is the four phases of clinical studies used by the Food and Drug Administration (FDA) to reach a decision about whether or not a new drug is safe and efficacious for market approval in the United States. Using this analogy, the final decision about whether a drug is efficacious and safe to be marketed for specific medical indications is based upon the accumulation of scientific evidence from a series of studies (i.e., not from any individual study), which are conducted in multiple sequential phases. The evidence generated in each phase is reviewed to make interim decisions about the safety and efficacy of a new pharmaceutical until ultimately all the evidence is reviewed to make a final decision about drug approval.

Under the FDA model for decisionmaking, initial research involves laboratory and animal tests. If the evidence generated in these studies indicates that the drug is active and not toxic, the sponsor submits an application to the FDA for an “investigational new drug.” If the FDA approves, human testing for safety and efficacy can begin. The first phase of human testing is usually conducted in a limited number of healthy volunteers (phase 1). If these trials show evidence that the product is safe in healthy volunteers, then the drug is further studied in a small number of volunteers who have the targeted condition (phase 2). If phase 2 studies show that the drug has a therapeutic effect and lacks significant adverse effects, trials with large numbers of people are conducted to determine the drug's safety and efficacy (phase 3). Following these trials, all relevant scientific studies are submitted to the FDA for a decision about whether the drug should be approved for marketing. If there are additional considerations like special safety issues, observational studies may be required to assess the safety of the drug in routine clinical care after the drug is approved for marketing (phase 4). Overall, the decisionmaking and research are staged so that the cumulative findings from all studies are used by the FDA to make interim decisions until the final decision is made about whether a medical product will be approved for marketing.

While most decisions about the comparative effectiveness of interventions will not need such extensive testing, it still may be prudent to stage research in a way that allows for interim decisions and sequentially more rigorous studies. On the other hand conditional approval or interim decisions may risk confusing patients and other stakeholders about the extent to which current evidence indicates that a treatment is effective and safe for all individuals with a health condition. For instance, under this staged approach new treatments could rapidly diffuse into a market even when there is limited evidence of long-term effectiveness and safety for all potential users. An illustrative example of this is the case of lung-volume reduction surgery, which was increasingly being used to treat severe emphysema despite limited evidence supporting its safety and efficacy until new research raised questions about the safety of the procedure. 6

Below is one potential categorization for the stages of knowledge development as related to informing decisions about questions of comparative effectiveness:

• Descriptive analysis
• Hypothesis generation
• Feasibility studies/proof of concept
• Hypothesis supporting
• Hypothesis testing

The first stages (i.e., descriptive analysis, hypothesis generation, and feasibility studies) are not mutually exclusive and usually are not intended to provide conclusive results for most decisions. Instead these stages provide preliminary evidence or feasibility testing before larger, more resource-intensive studies are launched. Results from these categories of studies may allow for interim decisionmaking (e.g., conditional approval for reimbursement of a treatment while further research is conducted). While a phased approach to research may postpone the time when a conclusive decision can be reached it does help to conserve resources such as those that may be consumed in launching a large multicenter study when a smaller study may be sufficient. Investigators will need to engage stakeholders to prioritize what stage of research may be most useful for the practical range of decisions that will be made.

Investigators should discuss in the protocol what stage of knowledge the current study will fulfill in light of the actions available to different stakeholders. This will allow reviewers of the protocol to assess the degree to which the evidence generated in the study holds the potential to fill specific knowledge gaps. For studies that are described in the protocol as preliminary, this may also help readers understand other tradeoffs that were made in the design of the study, in terms of methodological limitations that were accepted a priori in order to gather preliminary information about the research questions.

• Defining and Refining Study Questions Using PICOTS Framework

As recommended in other AHRQ methods guides, 7 investigators should engage stakeholders in a dialogue in order to understand the objectives of the research in practical terms, particularly so that investigators know the types of decisions that the research may affect. In working with stakeholders to develop research questions that can be studied with scientific methods, investigators may ask stakeholders to identify six key components of the research questions that will form the basis for designing the study. These components are reflected in the PICOTS typology and are shown below in Table 1.2 . These components represent the critical elements that will help investigators design a study that will be able to address the stakeholders' needs. Additional references that expand upon how to frame research questions can be found in the literature. 8 - 9

PICOTS typology for developing research questions.

The PICOTS typology outlines the key parts of the research questions that the study will be designed to address. 10 As a new research protocol is developed these questions can be presented in preliminary form and refined as other steps in the process are implemented. After the preliminary questions are refined, investigators should examine the questions to make sure that they will meet the needs of the stakeholders. In addition, they should assess whether the questions can be answered within the timeframe allotted and with the resources that are available for the study.

Since stakeholders ultimately determine effectiveness, it is important for investigators to ensure that the study endpoints and outcomes will meet their needs. Stakeholders need to articulate to investigators the health outcomes that are most important for a particular stakeholder to make decisions about treatment or take other health care actions. The endpoints that stakeholders will use to determine effectiveness may vary considerably. Unlike efficacy trials, in which clinical endpoints and surrogate measures are frequently used to determine efficacy, effectiveness may need to be determined based on several measures, many of which are not biological. These endpoints may be categorized as clinical endpoints, patient-reported outcomes and quality of life, health resource utilization, and utility measures. Types of measures that could be used are mortality, morbidity and adverse effects, quality of life, costs, or multiple outcomes. Chapter 6 gives a more extensive discussion of potential outcome measures of effectiveness.

The reliability, validity, and accuracy of study instruments to validly measure the concepts they purport to measure will also need to be acceptable to stakeholders. For instance, if stakeholders are interested in quality of life as an outcome, but do not believe there is an adequate measure of quality of life, then measurement development may need to be done prior to study initiation or other measures will need to be identified by stakeholders.

• Discussing Evidentiary Need and Uncertainty

Investigators and stakeholders should discuss the tradeoffs of different study designs that may be used for addressing the research questions. This dialogue will help researchers design a study that will be relevant and useful to the needs of stakeholders. All study designs have strengths and weaknesses, the latter of which may limit the conclusiveness of the final study results. Likewise, some decisions may require evidence that cannot be obtained from certain designs. In addition to design weaknesses, there are also practical tradeoffs that need to be considered in terms of research resources, like the time needed to complete the study, the availability of data, investigator expertise, subject recruitment, human subjects protection, research budget, difference to be detected, and lost-opportunity costs of doing the research instead of other studies that have priority for stakeholders. An important decision that will need to be made is whether or not randomization is needed for the questions being studied. There are several reasons why randomization might be needed, such as determining whether an FDA-approved drug can be used for a new use or indication that was not studied as part of the original drug approval process. A paper by Concato includes a thorough discussion of issues to consider when deciding whether randomization is necessary. 11

In discussing the tradeoffs of different study designs, researchers and stakeholders may wish to discuss the principal goals of research and ensure that researchers and stakeholders are aligned in their understanding of what is meant by scientific evidence. Fundamentally, research is a systematic investigation that uses scientific methods to measure, collect, and analyze data for the advancement of knowledge. This advancement is through the independent peer review and publication of study results, which are collectively referred to as scientific evidence. One definition of scientific evidence has been proposed by Normand and McNeil 12 as:

… the accumulation of information to support or refute a theory or hypothesis. … The idea is that assembling all the available information may reduce uncertainty about the effectiveness of the new technology compared to existing technologies in a setting where we believe particular relationships exist but are uncertain about their relevance …

While the primary aim of research is to produce new knowledge , the Normand and McNeil concept of evidence emphasizes that research helps create knowledge by reducing uncertainty about outcomes. However, rarely, if at all, does research eliminate all uncertainty around most decisions. In some cases, successful research will answer an important question and reduce uncertainty related to that question, but it may also increase uncertainty by leading to more, better informed questions regarding unknowns. As a result, nearly all decisions face some level of uncertainty even in a field where a body of research has been completed. This distinction is also critical because it helps to separate the research and subsequent actions that decisionmakers may take based on their assessment of the research results. Those subsequent actions may be informed by the research findings but will also be based on stakeholders' values and resources. Hence, as the definition by Normand and McNeil implies, research generates evidence but stakeholders decide whether to act on the evidence. Scientific evidence informs decisions to the extent it can adequately reduce the uncertainty about the problem for the stakeholder. Ultimately, treatment decisions are only guided by an assessment of the certainty that a course of therapy will lead to the outcomes of interest and the likelihood that this conclusion will be affected by the results of future studies.

In conceptualizing a study design, it is important for investigators to understand what constitutes sufficient and valid evidence from the stakeholder's perspective. In other words, what is the type of evidence that will be required to inform the stakeholder's decision to act or make a conscious decision not to take action? Evidence needed for action may vary by type of stakeholder and the scope of decisions that the stakeholder is making. For instance, a stakeholder who is making a population-based decision such as whether to provide insurance coverage for a new medical device with many alternatives may need substantially robust research findings in order to take action and provide that insurance coverage. In this example, the stakeholder may only accept as evidence a study with strong internal validity and generalizability (i.e., one conducted in a nationally representative sample of patients with the disease). On the other hand a patient who has a health condition where there are few treatments may be willing to accept lower-quality evidence in order to make a decision about whether to proceed with treatment despite a higher level of uncertainty about the outcome.

In many cases, there may exist a gradient of actions that can be taken based on available evidence. Quanstrum and Hayward 13 have discussed this gradient and argued that health care decisionmaking is changing, partly because more information is available to patients and other stakeholders about treatment options. As shown in the upper panel (A) in Figure 1.1 , many people may currently believe that health care treatment decisions are basically uniform for most people and under most circumstances. Panel A represents a hypothetical treatment whereby there is an evidentiary threshold or a point at which treatment is always beneficial and should be recommended. On the other hand below this threshold care provides no benefits and treatment should be discouraged. Quanstrum and Hayward argue that increasingly health care decisions are more like the lower panel (B). This panel portrays health care treatments as providing a large zone of discretion where benefits may be low or modest for most people. While above this zone treatment may always be recommended, individuals who fall within the zone may have questionable health benefits from treatment. As a result, different decisionmakers may take different actions based on their individual preferences.

Conceptualization of clinical decisionmaking. See Quanstrum KH, Hayward RA (Reference #). This figure is copyrighted by the Massachusetts Medical Society and reprinted with permission.

In light of this illustration, the following questions are suggested for discussion with stakeholders to help elicit the amount of uncertainty that is acceptable so that the study design can reach an appropriate level of evidence for the decision at hand:

• What level of new scientific evidence does the decisionmaker need to make a decision or take action?
• What quality of evidence is needed for the decisionmaker to act?
• What level of certainty of the outcome is needed by the decisionmaker(s)?
• How specific does the evidence need to be?
• Will decisions require consensus of multiple parties?

Additional Considerations When Considering Evidentiary Needs

As mentioned earlier, different stakeholders may disagree on the usefulness of different research designs, but it should be pointed out that this disagreement may be because stakeholders have different scopes of decisions to make. For example, high-quality research that is conclusive may be needed to make a decision that will affect the entire nation. On the other hand, results with more uncertainty as to the magnitude of the effect estimate(s) may be acceptable in making some decisions such as those affecting fewer people or where the risks to health are low. Often this disagreement occurs when different stakeholders debate whether evidence is needed from a new randomized controlled trial or whether evidence can be obtained from an analysis of an existing database. In this debate, both sides need to clarify whether they are facing the same decision or the decisions are different, particularly in terms of their scope.

Groups committed to evidence-based decisionmaking recognize that scientific evidence is only one component of the process of making decisions. Evidence generation is the goal of research, but evidence alone is not the only facet of evidence-based decisionmaking. In addition to scientific evidence, decisionmaking involves the consideration of (a) values, particularly the values placed on benefits and harms, and (b) resources. 14 Stakeholder differences in values and resources may mean that different decisions are made based on the same scientific evidence. Moreover, differences in values may create conflict in the decisionmaking process. One stakeholder may believe a particular study outcome is most important from their perspective, while another stakeholder may believe a different outcome is the most important for determining effectiveness.

Likewise, there may be inherent conflicts in values between individual decisionmaking and population decisionmaking, even though these decisions are often interrelated. For example, an individual may have a higher tolerance for treatment risk in light of the expected treatment benefits for him or her. On the other hand a regulatory health authority may determine that the population risk is too great without sufficient evidence that treatment provides benefits to the population. An example of this difference in perspective can be seen with how different decisionmakers responded to evidence about the drug Avastin ® (bevacizumab) for the treatment of metastatic breast cancer. In this case, the FDA revoked their approval of the breast cancer indication for Avastin after concluding that the drug had not been shown to be safe and effective for that use. Nonetheless, Medicare, the public insurance program for the elderly and disabled continued to allow coverage when a physician prescribes the drug, even for breast cancer. Likewise, some patient groups were reported to be concerned by the decision since it presumably would deny some women access to Avastin treatment. For a more thorough discussion of these issues around differences in perspective, the reader is referred to an article by Atkins 15 and the examples in Table 1.3 below.

Examples of individual versus population decisions (Adapted from Atkins, 2007).

• Specifying Magnitude of Effect

In order for decisions to be objective, it is important for there to be an a priori discussion with stakeholders about the magnitude of effect that stakeholders believe represents a meaningful difference between treatment options. Researchers will be familiar with the basic tenet that statistically significant differences do not always represent clinically meaningful differences. Hence, researchers and stakeholders will need to have knowledge of the instruments that are used to measure differences and the accuracy, limitations, and properties of those instruments. Three key questions are recommended to use when eliciting from stakeholders the effect sizes that are important to them for making a decision or taking action:

• How do patients and other stakeholders define a meaningful difference between interventions?
• How do previous studies and reviews define a meaningful difference?
• Are patients and other stakeholders interested in superiority or noninferiority as it relates to decisionmaking?
• Challenges to Developing Study Questions and Initial Solutions

In developing CER study objectives and questions, there are some potential challenges that face researchers and stakeholders. The involvement of patients and other stakeholders in determining study objectives and questions is a relatively new paradigm, but one that is consistent with established principles of translational research. A key principle of translational research is that users need to be involved in research at the earliest stages for the research to be adopted. 16 In addition, most research is currently initiated by an investigator, and traditionally there have been few incentives (and some disincentives) to involving others in designing a new research study. Although the research paradigm is rapidly shifting, 17 there is little information about how to structure, process, and evaluate outcomes from initiatives that attempt to engage stakeholders in developing study questions and objectives with researchers. As different approaches are taken to involve stakeholders in the research process, researchers will learn how to optimize the process of stakeholder involvement and improve the applicability of research to the end-users.

The bringing together of stakeholders may create some general challenges to the research team. For instance, it may be difficult to identify, engage, or manage all stakeholders who are interested in developing and using scientific evidence for addressing a problem. A process that allows for public commenting on research protocols through Internet postings may be helpful in reaching the widest network of interested stakeholders. Nevertheless, finding stakeholders who can represent all perspectives may not always be practical or available to the study team. In addition, competing interests among stakeholders may make prioritization of research questions challenging. Different stakeholders have different needs and this may make prioritization of research difficult. Nonetheless, as the science of translational research evolves, the collaboration of researchers with stakeholders will likely become increasingly the standard of practice in designing new research.

To assist researchers and stakeholders with working together, AHRQ has published several online resources to facilitate the involvement of stakeholders in the research process. These include a brief guide for stakeholders that highlights opportunities for taking part in AHRQ's Effective Health Care Program, a facilitation primer with strategies for working with diverse stakeholder groups, a table of suggested tasks for researchers to involve stakeholders in the identification and prioritization of future research, and learning modules with slide presentations on engaging stakeholders in the Effective Health Care Program. 18 - 19 In addition, AHRQ supports the Evidence-based Practice Centers in working with various stakeholders to further develop and prioritize decisionmakers' future research needs, which are published in a series of reports on AHRQ's Web site and on the National Library of Medicine's open-access Bookshelf. 20

Likewise, AHRQ supports the active involvement of patients and other stakeholders in the AHRQ DEcIDE program, in which different models of engagement have been used. These models include hosting in-person meetings with stakeholders to create research agendas; 21 - 22 developing research based on questions posed by public payers such as Centers for Medicare and Medicaid Services; addressing knowledge gaps that have been identified in AHRQ systematic reviews through new research; and supporting five research consortia, each of which involves researchers, patients, and other stakeholders working together to develop, prioritize, and implement research studies.

• Summary and Conclusion

This chapter provides a framework for formulating study objectives and questions, for a research protocol on a CER topic. Implementation of the framework involves collaboration between researchers and stakeholders in conceptualizing the research objectives and questions and the design of the study. In this process, there is a shared commitment to protect the integrity of the research results from bias and conflicts of interest, so that the results are valid for informing decisions and health care actions. Due to the complexity of some health care decisions, the evidence needed for decisionmaking or action may need to be developed from multiple studies, including preliminary research that becomes the underpinning for larger studies. The principles described in this chapter are intended to strengthen the writing of research protocols and enhance the results from the emanating studies, for informing the important decisions facing patients, providers, and other stakeholders about health care treatments and new technologies. Subsequent chapters in this User's Guide provide specific principles for operationalizing the study objectives and research questions in writing a complete study protocol that can be executed as new research.

Checklist: Guidance and key considerations for developing study objectives and questions for observational CER protocols

View in own window

Developing a Protocol for Observational Comparative Effectiveness Research: A User’s Guide is copyrighted by the Agency for Healthcare Research and Quality (AHRQ). The product and its contents may be used and incorporated into other materials on the following three conditions: (1) the contents are not changed in any way (including covers and front matter), (2) no fee is charged by the reproducer of the product or its contents for its use, and (3) the user obtains permission from the copyright holders identified therein for materials noted as copyrighted by others. The product may not be sold for profit or incorporated into any profitmaking venture without the expressed written permission of AHRQ.

• Cite this Page Smith SR. Study Objectives and Questions. In: Velentgas P, Dreyer NA, Nourjah P, et al., editors. Developing a Protocol for Observational Comparative Effectiveness Research: A User's Guide. Rockville (MD): Agency for Healthcare Research and Quality (US); 2013 Jan. Chapter 1.
• PDF version of this title (5.8M)

In this Page

Other titles in these collections.

• AHRQ Methods for Effective Health Care
• Health Services/Technology Assessment Text (HSTAT)

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Study Objectives and Questions - Developing a Protocol for Observational Compara... Study Objectives and Questions - Developing a Protocol for Observational Comparative Effectiveness Research: A User's Guide

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers