research methods article topics

Choose Your Test

Sat / act prep online guides and tips, 113 great research paper topics.

General Education

One of the hardest parts of writing a research paper can be just finding a good topic to write about. Fortunately we've done the hard work for you and have compiled a list of 113 interesting research paper topics. They've been organized into ten categories and cover a wide range of subjects so you can easily find the best topic for you.

In addition to the list of good research topics, we've included advice on what makes a good research paper topic and how you can use your topic to start writing a great paper.

What Makes a Good Research Paper Topic?

Not all research paper topics are created equal, and you want to make sure you choose a great topic before you start writing. Below are the three most important factors to consider to make sure you choose the best research paper topics.

#1: It's Something You're Interested In

A paper is always easier to write if you're interested in the topic, and you'll be more motivated to do in-depth research and write a paper that really covers the entire subject. Even if a certain research paper topic is getting a lot of buzz right now or other people seem interested in writing about it, don't feel tempted to make it your topic unless you genuinely have some sort of interest in it as well.

#2: There's Enough Information to Write a Paper

Even if you come up with the absolute best research paper topic and you're so excited to write about it, you won't be able to produce a good paper if there isn't enough research about the topic. This can happen for very specific or specialized topics, as well as topics that are too new to have enough research done on them at the moment. Easy research paper topics will always be topics with enough information to write a full-length paper.

Trying to write a research paper on a topic that doesn't have much research on it is incredibly hard, so before you decide on a topic, do a bit of preliminary searching and make sure you'll have all the information you need to write your paper.

#3: It Fits Your Teacher's Guidelines

Don't get so carried away looking at lists of research paper topics that you forget any requirements or restrictions your teacher may have put on research topic ideas. If you're writing a research paper on a health-related topic, deciding to write about the impact of rap on the music scene probably won't be allowed, but there may be some sort of leeway. For example, if you're really interested in current events but your teacher wants you to write a research paper on a history topic, you may be able to choose a topic that fits both categories, like exploring the relationship between the US and North Korea. No matter what, always get your research paper topic approved by your teacher first before you begin writing.

113 Good Research Paper Topics

Below are 113 good research topics to help you get you started on your paper. We've organized them into ten categories to make it easier to find the type of research paper topics you're looking for.

Arts/Culture

• Discuss the main differences in art from the Italian Renaissance and the Northern Renaissance .
• Analyze the impact a famous artist had on the world.
• How is sexism portrayed in different types of media (music, film, video games, etc.)? Has the amount/type of sexism changed over the years?
• How has the music of slaves brought over from Africa shaped modern American music?
• How has rap music evolved in the past decade?
• How has the portrayal of minorities in the media changed?

Current Events

• What have been the impacts of China's one child policy?
• How have the goals of feminists changed over the decades?
• How has the Trump presidency changed international relations?
• Analyze the history of the relationship between the United States and North Korea.
• What factors contributed to the current decline in the rate of unemployment?
• What have been the impacts of states which have increased their minimum wage?
• How do US immigration laws compare to immigration laws of other countries?
• How have the US's immigration laws changed in the past few years/decades?
• How has the Black Lives Matter movement affected discussions and view about racism in the US?
• What impact has the Affordable Care Act had on healthcare in the US?
• What factors contributed to the UK deciding to leave the EU (Brexit)?
• What factors contributed to China becoming an economic power?
• Discuss the history of Bitcoin or other cryptocurrencies  (some of which tokenize the S&P 500 Index on the blockchain) .
• Do students in schools that eliminate grades do better in college and their careers?
• Do students from wealthier backgrounds score higher on standardized tests?
• Do students who receive free meals at school get higher grades compared to when they weren't receiving a free meal?
• Do students who attend charter schools score higher on standardized tests than students in public schools?
• Do students learn better in same-sex classrooms?
• How does giving each student access to an iPad or laptop affect their studies?
• What are the benefits and drawbacks of the Montessori Method ?
• Do children who attend preschool do better in school later on?
• What was the impact of the No Child Left Behind act?
• How does the US education system compare to education systems in other countries?
• What impact does mandatory physical education classes have on students' health?
• Which methods are most effective at reducing bullying in schools?
• Do homeschoolers who attend college do as well as students who attended traditional schools?
• Does offering tenure increase or decrease quality of teaching?
• How does college debt affect future life choices of students?
• Should graduate students be able to form unions?

• What are different ways to lower gun-related deaths in the US?
• How and why have divorce rates changed over time?
• Is affirmative action still necessary in education and/or the workplace?
• Should physician-assisted suicide be legal?
• How has stem cell research impacted the medical field?
• How can human trafficking be reduced in the United States/world?
• Should people be able to donate organs in exchange for money?
• Which types of juvenile punishment have proven most effective at preventing future crimes?
• Has the increase in US airport security made passengers safer?
• Analyze the immigration policies of certain countries and how they are similar and different from one another.
• Several states have legalized recreational marijuana. What positive and negative impacts have they experienced as a result?
• Do tariffs increase the number of domestic jobs?
• Which prison reforms have proven most effective?
• Should governments be able to censor certain information on the internet?
• Which methods/programs have been most effective at reducing teen pregnancy?
• What are the benefits and drawbacks of the Keto diet?
• How effective are different exercise regimes for losing weight and maintaining weight loss?
• How do the healthcare plans of various countries differ from each other?
• What are the most effective ways to treat depression ?
• What are the pros and cons of genetically modified foods?
• Which methods are most effective for improving memory?
• What can be done to lower healthcare costs in the US?
• What factors contributed to the current opioid crisis?
• Analyze the history and impact of the HIV/AIDS epidemic .
• Are low-carbohydrate or low-fat diets more effective for weight loss?
• How much exercise should the average adult be getting each week?
• Which methods are most effective to get parents to vaccinate their children?
• What are the pros and cons of clean needle programs?
• How does stress affect the body?
• Discuss the history of the conflict between Israel and the Palestinians.
• What were the causes and effects of the Salem Witch Trials?
• Who was responsible for the Iran-Contra situation?
• How has New Orleans and the government's response to natural disasters changed since Hurricane Katrina?
• What events led to the fall of the Roman Empire?
• What were the impacts of British rule in India ?
• Was the atomic bombing of Hiroshima and Nagasaki necessary?
• What were the successes and failures of the women's suffrage movement in the United States?
• What were the causes of the Civil War?
• How did Abraham Lincoln's assassination impact the country and reconstruction after the Civil War?
• Which factors contributed to the colonies winning the American Revolution?
• What caused Hitler's rise to power?
• Discuss how a specific invention impacted history.
• What led to Cleopatra's fall as ruler of Egypt?
• How has Japan changed and evolved over the centuries?
• What were the causes of the Rwandan genocide ?

• Why did Martin Luther decide to split with the Catholic Church?
• Analyze the history and impact of a well-known cult (Jonestown, Manson family, etc.)
• How did the sexual abuse scandal impact how people view the Catholic Church?
• How has the Catholic church's power changed over the past decades/centuries?
• What are the causes behind the rise in atheism/ agnosticism in the United States?
• What were the influences in Siddhartha's life resulted in him becoming the Buddha?
• How has media portrayal of Islam/Muslims changed since September 11th?

Science/Environment

• How has the earth's climate changed in the past few decades?
• How has the use and elimination of DDT affected bird populations in the US?
• Analyze how the number and severity of natural disasters have increased in the past few decades.
• Analyze deforestation rates in a certain area or globally over a period of time.
• How have past oil spills changed regulations and cleanup methods?
• How has the Flint water crisis changed water regulation safety?
• What are the pros and cons of fracking?
• What impact has the Paris Climate Agreement had so far?
• What have NASA's biggest successes and failures been?
• How can we improve access to clean water around the world?
• Does ecotourism actually have a positive impact on the environment?
• Should the US rely on nuclear energy more?
• What can be done to save amphibian species currently at risk of extinction?
• What impact has climate change had on coral reefs?
• How are black holes created?
• Are teens who spend more time on social media more likely to suffer anxiety and/or depression?
• How will the loss of net neutrality affect internet users?
• Analyze the history and progress of self-driving vehicles.
• How has the use of drones changed surveillance and warfare methods?
• Has social media made people more or less connected?
• What progress has currently been made with artificial intelligence ?
• Do smartphones increase or decrease workplace productivity?
• What are the most effective ways to use technology in the classroom?
• How is Google search affecting our intelligence?
• When is the best age for a child to begin owning a smartphone?
• Has frequent texting reduced teen literacy rates?

How to Write a Great Research Paper

Even great research paper topics won't give you a great research paper if you don't hone your topic before and during the writing process. Follow these three tips to turn good research paper topics into great papers.

#1: Figure Out Your Thesis Early

Before you start writing a single word of your paper, you first need to know what your thesis will be. Your thesis is a statement that explains what you intend to prove/show in your paper. Every sentence in your research paper will relate back to your thesis, so you don't want to start writing without it!

As some examples, if you're writing a research paper on if students learn better in same-sex classrooms, your thesis might be "Research has shown that elementary-age students in same-sex classrooms score higher on standardized tests and report feeling more comfortable in the classroom."

If you're writing a paper on the causes of the Civil War, your thesis might be "While the dispute between the North and South over slavery is the most well-known cause of the Civil War, other key causes include differences in the economies of the North and South, states' rights, and territorial expansion."

#2: Back Every Statement Up With Research

Remember, this is a research paper you're writing, so you'll need to use lots of research to make your points. Every statement you give must be backed up with research, properly cited the way your teacher requested. You're allowed to include opinions of your own, but they must also be supported by the research you give.

#3: Do Your Research Before You Begin Writing

You don't want to start writing your research paper and then learn that there isn't enough research to back up the points you're making, or, even worse, that the research contradicts the points you're trying to make!

Get most of your research on your good research topics done before you begin writing. Then use the research you've collected to create a rough outline of what your paper will cover and the key points you're going to make. This will help keep your paper clear and organized, and it'll ensure you have enough research to produce a strong paper.

What's Next?

Are you also learning about dynamic equilibrium in your science class? We break this sometimes tricky concept down so it's easy to understand in our complete guide to dynamic equilibrium .

Thinking about becoming a nurse practitioner? Nurse practitioners have one of the fastest growing careers in the country, and we have all the information you need to know about what to expect from nurse practitioner school .

Want to know the fastest and easiest ways to convert between Fahrenheit and Celsius? We've got you covered! Check out our guide to the best ways to convert Celsius to Fahrenheit (or vice versa).

These recommendations are based solely on our knowledge and experience. If you purchase an item through one of our links, PrepScholar may receive a commission.

Christine graduated from Michigan State University with degrees in Environmental Biology and Geography and received her Master's from Duke University. In high school she scored in the 99th percentile on the SAT and was named a National Merit Finalist. She has taught English and biology in several countries.

Student and Parent Forum

Our new student and parent forum, at ExpertHub.PrepScholar.com , allow you to interact with your peers and the PrepScholar staff. See how other students and parents are navigating high school, college, and the college admissions process. Ask questions; get answers.

Ask a Question Below

Have any questions about this article or other topics? Ask below and we'll reply!

Improve With Our Famous Guides

• For All Students

The 5 Strategies You Must Be Using to Improve 160+ SAT Points

How to Get a Perfect 1600, by a Perfect Scorer

Series: How to Get 800 on Each SAT Section:

Score 800 on SAT Math

Score 800 on SAT Reading

Score 800 on SAT Writing

Series: How to Get to 600 on Each SAT Section:

Score 600 on SAT Math

Score 600 on SAT Reading

Score 600 on SAT Writing

Free Complete Official SAT Practice Tests

What SAT Target Score Should You Be Aiming For?

15 Strategies to Improve Your SAT Essay

The 5 Strategies You Must Be Using to Improve 4+ ACT Points

How to Get a Perfect 36 ACT, by a Perfect Scorer

Series: How to Get 36 on Each ACT Section:

36 on ACT English

36 on ACT Math

36 on ACT Reading

36 on ACT Science

Series: How to Get to 24 on Each ACT Section:

24 on ACT English

24 on ACT Math

24 on ACT Reading

24 on ACT Science

What ACT target score should you be aiming for?

ACT Vocabulary You Must Know

ACT Writing: 15 Tips to Raise Your Essay Score

How to Get Into Harvard and the Ivy League

How to Get a Perfect 4.0 GPA

How to Write an Amazing College Essay

What Exactly Are Colleges Looking For?

Is the ACT easier than the SAT? A Comprehensive Guide

Should you retake your SAT or ACT?

When should you take the SAT or ACT?

Stay Informed

Get the latest articles and test prep tips!

Looking for Graduate School Test Prep?

Check out our top-rated graduate blogs here:

GRE Online Prep Blog

GMAT Online Prep Blog

TOEFL Online Prep Blog

Holly R. "I am absolutely overjoyed and cannot thank you enough for helping me!”

1000+ FREE Research Topics & Title Ideas

If you’re at the start of your research journey and are trying to figure out which research topic you want to focus on, you’ve come to the right place. Select your area of interest below to view a comprehensive collection of potential research ideas.

Research Topic FAQs

What (exactly) is a research topic.

A research topic is the subject of a research project or study – for example, a dissertation or thesis. A research topic typically takes the form of a problem to be solved, or a question to be answered.

A good research topic should be specific enough to allow for focused research and analysis. For example, if you are interested in studying the effects of climate change on agriculture, your research topic could focus on how rising temperatures have impacted crop yields in certain regions over time.

To learn more about the basics of developing a research topic, consider our free research topic ideation webinar.

What constitutes a good research topic?

A strong research topic comprises three important qualities : originality, value and feasibility.

• Originality – a good topic explores an original area or takes a novel angle on an existing area of study.
• Value – a strong research topic provides value and makes a contribution, either academically or practically.
• Feasibility – a good research topic needs to be practical and manageable, given the resource constraints you face.

To learn more about what makes for a high-quality research topic, check out this post .

What's the difference between a research topic and research problem?

A research topic and a research problem are two distinct concepts that are often confused. A research topic is a broader label that indicates the focus of the study , while a research problem is an issue or gap in knowledge within the broader field that needs to be addressed.

To illustrate this distinction, consider a student who has chosen “teenage pregnancy in the United Kingdom” as their research topic. This research topic could encompass any number of issues related to teenage pregnancy such as causes, prevention strategies, health outcomes for mothers and babies, etc.

Within this broad category (the research topic) lies potential areas of inquiry that can be explored further – these become the research problems . For example:

• What factors contribute to higher rates of teenage pregnancy in certain communities?
• How do different types of parenting styles affect teen pregnancy rates?
• What interventions have been successful in reducing teenage pregnancies?

Simply put, a key difference between a research topic and a research problem is scope ; the research topic provides an umbrella under which multiple questions can be asked, while the research problem focuses on one specific question or set of questions within that larger context.

How can I find potential research topics for my project?

There are many steps involved in the process of finding and choosing a high-quality research topic for a dissertation or thesis. We cover these steps in detail in this video (also accessible below).

How can I find quality sources for my research topic?

Finding quality sources is an essential step in the topic ideation process. To do this, you should start by researching scholarly journals, books, and other academic publications related to your topic. These sources can provide reliable information on a wide range of topics. Additionally, they may contain data or statistics that can help support your argument or conclusions.

Identifying Relevant Sources

When searching for relevant sources, it’s important to look beyond just published material; try using online databases such as Google Scholar or JSTOR to find articles from reputable journals that have been peer-reviewed by experts in the field.

You can also use search engines like Google or Bing to locate websites with useful information about your topic. However, be sure to evaluate any website before citing it as a source—look for evidence of authorship (such as an “About Us” page) and make sure the content is up-to-date and accurate before relying on it.

Evaluating Sources

Once you’ve identified potential sources for your research project, take some time to evaluate them thoroughly before deciding which ones will best serve your purpose. Consider factors such as author credibility (are they an expert in their field?), publication date (is the source current?), objectivity (does the author present both sides of an issue?) and relevance (how closely does this source relate to my specific topic?).

By researching the current literature on your topic, you can identify potential sources that will help to provide quality information. Once you’ve identified these sources, it’s time to look for a gap in the research and determine what new knowledge could be gained from further study.

How can I find a good research gap?

Finding a strong gap in the literature is an essential step when looking for potential research topics. We explain what research gaps are and how to find them in this post.

How should I evaluate potential research topics/ideas?

When evaluating potential research topics, it is important to consider the factors that make for a strong topic (we discussed these earlier). Specifically:

• Originality
• Feasibility

So, when you have a list of potential topics or ideas, assess each of them in terms of these three criteria. A good topic should take a unique angle, provide value (either to academia or practitioners), and be practical enough for you to pull off, given your limited resources.

Finally, you should also assess whether this project could lead to potential career opportunities such as internships or job offers down the line. Make sure that you are researching something that is relevant enough so that it can benefit your professional development in some way. Additionally, consider how each research topic aligns with your career goals and interests; researching something that you are passionate about can help keep motivation high throughout the process.

How can I assess the feasibility of a research topic?

When evaluating the feasibility and practicality of a research topic, it is important to consider several factors.

First, you should assess whether or not the research topic is within your area of competence. Of course, when you start out, you are not expected to be the world’s leading expert, but do should at least have some foundational knowledge.

Time commitment

When considering a research topic, you should think about how much time will be required for completion. Depending on your field of study, some topics may require more time than others due to their complexity or scope.

Additionally, if you plan on collaborating with other researchers or institutions in order to complete your project, additional considerations must be taken into account such as coordinating schedules and ensuring that all parties involved have adequate resources available.

Resources needed

It’s also critically important to consider what type of resources are necessary in order to conduct the research successfully. This includes physical materials such as lab equipment and chemicals but can also include intangible items like access to certain databases or software programs which may be necessary depending on the nature of your work. Additionally, if there are costs associated with obtaining these materials then this must also be factored into your evaluation process.

Potential risks

It’s important to consider the inherent potential risks for each potential research topic. These can include ethical risks (challenges getting ethical approval), data risks (not being able to access the data you’ll need), technical risks relating to the equipment you’ll use and funding risks (not securing the necessary financial back to undertake the research).

If you’re looking for more information about how to find, evaluate and select research topics for your dissertation or thesis, check out our free webinar here . Alternatively, if you’d like 1:1 help with the topic ideation process, consider our private coaching services .

Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

News alert: UC Berkeley has announced its next university librarian

Secondary menu

• Log in to your Library account
• Hours and Maps
• Connect from Off Campus
• UC Berkeley Home

Search form

Research methods--quantitative, qualitative, and more: overview.

• Quantitative Research
• Qualitative Research
• Data Science Methods (Machine Learning, AI, Big Data)
• Text Mining and Computational Text Analysis
• Evidence Synthesis/Systematic Reviews
• Get Data, Get Help!

About Research Methods

This guide provides an overview of research methods, how to choose and use them, and supports and resources at UC Berkeley. 

As Patten and Newhart note in the book Understanding Research Methods , "Research methods are the building blocks of the scientific enterprise. They are the "how" for building systematic knowledge. The accumulation of knowledge through research is by its nature a collective endeavor. Each well-designed study provides evidence that may support, amend, refute, or deepen the understanding of existing knowledge...Decisions are important throughout the practice of research and are designed to help researchers collect evidence that includes the full spectrum of the phenomenon under study, to maintain logical rules, and to mitigate or account for possible sources of bias. In many ways, learning research methods is learning how to see and make these decisions."

The choice of methods varies by discipline, by the kind of phenomenon being studied and the data being used to study it, by the technology available, and more.  This guide is an introduction, but if you don't see what you need here, always contact your subject librarian, and/or take a look to see if there's a library research guide that will answer your question. 

Suggestions for changes and additions to this guide are welcome! 

START HERE: SAGE Research Methods

Without question, the most comprehensive resource available from the library is SAGE Research Methods.  HERE IS THE ONLINE GUIDE  to this one-stop shopping collection, and some helpful links are below:

• SAGE Research Methods
• Little Green Books  (Quantitative Methods)
• Little Blue Books  (Qualitative Methods)
• Dictionaries and Encyclopedias  
• Case studies of real research projects
• Sample datasets for hands-on practice
• Streaming video--see methods come to life
• Methodspace- -a community for researchers
• SAGE Research Methods Course Mapping

Library Data Services at UC Berkeley

Library Data Services Program and Digital Scholarship Services

The LDSP offers a variety of services and tools !  From this link, check out pages for each of the following topics:  discovering data, managing data, collecting data, GIS data, text data mining, publishing data, digital scholarship, open science, and the Research Data Management Program.

Be sure also to check out the visual guide to where to seek assistance on campus with any research question you may have!

Library GIS Services

Other Data Services at Berkeley

D-Lab Supports Berkeley faculty, staff, and graduate students with research in data intensive social science, including a wide range of training and workshop offerings Dryad Dryad is a simple self-service tool for researchers to use in publishing their datasets. It provides tools for the effective publication of and access to research data. Geospatial Innovation Facility (GIF) Provides leadership and training across a broad array of integrated mapping technologies on campu Research Data Management A UC Berkeley guide and consulting service for research data management issues

General Research Methods Resources

Here are some general resources for assistance:

• Assistance from ICPSR (must create an account to access): Getting Help with Data , and Resources for Students
• Wiley Stats Ref for background information on statistics topics
• Survey Documentation and Analysis (SDA) .  Program for easy web-based analysis of survey data.

Consultants

• D-Lab/Data Science Discovery Consultants Request help with your research project from peer consultants.
• Research data (RDM) consulting Meet with RDM consultants before designing the data security, storage, and sharing aspects of your qualitative project.
• Statistics Department Consulting Services A service in which advanced graduate students, under faculty supervision, are available to consult during specified hours in the Fall and Spring semesters.

Related Resourcex

• IRB / CPHS Qualitative research projects with human subjects often require that you go through an ethics review.
• OURS (Office of Undergraduate Research and Scholarships) OURS supports undergraduates who want to embark on research projects and assistantships. In particular, check out their "Getting Started in Research" workshops
• Sponsored Projects Sponsored projects works with researchers applying for major external grants.
• Next: Quantitative Research >>
• Last Updated: Apr 3, 2023 3:14 PM
• URL: https://guides.lib.berkeley.edu/researchmethods

• Methodology
• Open access
• Published: 11 October 2016

Reviewing the research methods literature: principles and strategies illustrated by a systematic overview of sampling in qualitative research

• Stephen J. Gentles 1 , 4 ,
• Cathy Charles 1 ,
• David B. Nicholas 2 ,
• Jenny Ploeg 3 &
• K. Ann McKibbon 1  

Systematic Reviews volume  5 , Article number:  172 ( 2016 ) Cite this article

51k Accesses

26 Citations

13 Altmetric

Metrics details

Overviews of methods are potentially useful means to increase clarity and enhance collective understanding of specific methods topics that may be characterized by ambiguity, inconsistency, or a lack of comprehensiveness. This type of review represents a distinct literature synthesis method, although to date, its methodology remains relatively undeveloped despite several aspects that demand unique review procedures. The purpose of this paper is to initiate discussion about what a rigorous systematic approach to reviews of methods, referred to here as systematic methods overviews , might look like by providing tentative suggestions for approaching specific challenges likely to be encountered. The guidance offered here was derived from experience conducting a systematic methods overview on the topic of sampling in qualitative research.

The guidance is organized into several principles that highlight specific objectives for this type of review given the common challenges that must be overcome to achieve them. Optional strategies for achieving each principle are also proposed, along with discussion of how they were successfully implemented in the overview on sampling. We describe seven paired principles and strategies that address the following aspects: delimiting the initial set of publications to consider, searching beyond standard bibliographic databases, searching without the availability of relevant metadata, selecting publications on purposeful conceptual grounds, defining concepts and other information to abstract iteratively, accounting for inconsistent terminology used to describe specific methods topics, and generating rigorous verifiable analytic interpretations. Since a broad aim in systematic methods overviews is to describe and interpret the relevant literature in qualitative terms, we suggest that iterative decision making at various stages of the review process, and a rigorous qualitative approach to analysis are necessary features of this review type.

Conclusions

We believe that the principles and strategies provided here will be useful to anyone choosing to undertake a systematic methods overview. This paper represents an initial effort to promote high quality critical evaluations of the literature regarding problematic methods topics, which have the potential to promote clearer, shared understandings, and accelerate advances in research methods. Further work is warranted to develop more definitive guidance.

Peer Review reports

While reviews of methods are not new, they represent a distinct review type whose methodology remains relatively under-addressed in the literature despite the clear implications for unique review procedures. One of few examples to describe it is a chapter containing reflections of two contributing authors in a book of 21 reviews on methodological topics compiled for the British National Health Service, Health Technology Assessment Program [ 1 ]. Notable is their observation of how the differences between the methods reviews and conventional quantitative systematic reviews, specifically attributable to their varying content and purpose, have implications for defining what qualifies as systematic. While the authors describe general aspects of “systematicity” (including rigorous application of a methodical search, abstraction, and analysis), they also describe a high degree of variation within the category of methods reviews itself and so offer little in the way of concrete guidance. In this paper, we present tentative concrete guidance, in the form of a preliminary set of proposed principles and optional strategies, for a rigorous systematic approach to reviewing and evaluating the literature on quantitative or qualitative methods topics. For purposes of this article, we have used the term systematic methods overview to emphasize the notion of a systematic approach to such reviews.

The conventional focus of rigorous literature reviews (i.e., review types for which systematic methods have been codified, including the various approaches to quantitative systematic reviews [ 2 – 4 ], and the numerous forms of qualitative and mixed methods literature synthesis [ 5 – 10 ]) is to synthesize empirical research findings from multiple studies. By contrast, the focus of overviews of methods, including the systematic approach we advocate, is to synthesize guidance on methods topics. The literature consulted for such reviews may include the methods literature, methods-relevant sections of empirical research reports, or both. Thus, this paper adds to previous work published in this journal—namely, recent preliminary guidance for conducting reviews of theory [ 11 ]—that has extended the application of systematic review methods to novel review types that are concerned with subject matter other than empirical research findings.

Published examples of methods overviews illustrate the varying objectives they can have. One objective is to establish methodological standards for appraisal purposes. For example, reviews of existing quality appraisal standards have been used to propose universal standards for appraising the quality of primary qualitative research [ 12 ] or evaluating qualitative research reports [ 13 ]. A second objective is to survey the methods-relevant sections of empirical research reports to establish current practices on methods use and reporting practices, which Moher and colleagues [ 14 ] recommend as a means for establishing the needs to be addressed in reporting guidelines (see, for example [ 15 , 16 ]). A third objective for a methods review is to offer clarity and enhance collective understanding regarding a specific methods topic that may be characterized by ambiguity, inconsistency, or a lack of comprehensiveness within the available methods literature. An example of this is a overview whose objective was to review the inconsistent definitions of intention-to-treat analysis (the methodologically preferred approach to analyze randomized controlled trial data) that have been offered in the methods literature and propose a solution for improving conceptual clarity [ 17 ]. Such reviews are warranted because students and researchers who must learn or apply research methods typically lack the time to systematically search, retrieve, review, and compare the available literature to develop a thorough and critical sense of the varied approaches regarding certain controversial or ambiguous methods topics.

While systematic methods overviews , as a review type, include both reviews of the methods literature and reviews of methods-relevant sections from empirical study reports, the guidance provided here is primarily applicable to reviews of the methods literature since it was derived from the experience of conducting such a review [ 18 ], described below. To our knowledge, there are no well-developed proposals on how to rigorously conduct such reviews. Such guidance would have the potential to improve the thoroughness and credibility of critical evaluations of the methods literature, which could increase their utility as a tool for generating understandings that advance research methods, both qualitative and quantitative. Our aim in this paper is thus to initiate discussion about what might constitute a rigorous approach to systematic methods overviews. While we hope to promote rigor in the conduct of systematic methods overviews wherever possible, we do not wish to suggest that all methods overviews need be conducted to the same standard. Rather, we believe that the level of rigor may need to be tailored pragmatically to the specific review objectives, which may not always justify the resource requirements of an intensive review process.

The example systematic methods overview on sampling in qualitative research

The principles and strategies we propose in this paper are derived from experience conducting a systematic methods overview on the topic of sampling in qualitative research [ 18 ]. The main objective of that methods overview was to bring clarity and deeper understanding of the prominent concepts related to sampling in qualitative research (purposeful sampling strategies, saturation, etc.). Specifically, we interpreted the available guidance, commenting on areas lacking clarity, consistency, or comprehensiveness (without proposing any recommendations on how to do sampling). This was achieved by a comparative and critical analysis of publications representing the most influential (i.e., highly cited) guidance across several methodological traditions in qualitative research.

The specific methods and procedures for the overview on sampling [ 18 ] from which our proposals are derived were developed both after soliciting initial input from local experts in qualitative research and an expert health librarian (KAM) and through ongoing careful deliberation throughout the review process. To summarize, in that review, we employed a transparent and rigorous approach to search the methods literature, selected publications for inclusion according to a purposeful and iterative process, abstracted textual data using structured abstraction forms, and analyzed (synthesized) the data using a systematic multi-step approach featuring abstraction of text, summary of information in matrices, and analytic comparisons.

For this article, we reflected on both the problems and challenges encountered at different stages of the review and our means for selecting justifiable procedures to deal with them. Several principles were then derived by considering the generic nature of these problems, while the generalizable aspects of the procedures used to address them formed the basis of optional strategies. Further details of the specific methods and procedures used in the overview on qualitative sampling are provided below to illustrate both the types of objectives and challenges that reviewers will likely need to consider and our approach to implementing each of the principles and strategies.

Organization of the guidance into principles and strategies

For the purposes of this article, principles are general statements outlining what we propose are important aims or considerations within a particular review process, given the unique objectives or challenges to be overcome with this type of review. These statements follow the general format, “considering the objective or challenge of X, we propose Y to be an important aim or consideration.” Strategies are optional and flexible approaches for implementing the previous principle outlined. Thus, generic challenges give rise to principles, which in turn give rise to strategies.

We organize the principles and strategies below into three sections corresponding to processes characteristic of most systematic literature synthesis approaches: literature identification and selection ; data abstraction from the publications selected for inclusion; and analysis , including critical appraisal and synthesis of the abstracted data. Within each section, we also describe the specific methodological decisions and procedures used in the overview on sampling in qualitative research [ 18 ] to illustrate how the principles and strategies for each review process were applied and implemented in a specific case. We expect this guidance and accompanying illustrations will be useful for anyone considering engaging in a methods overview, particularly those who may be familiar with conventional systematic review methods but may not yet appreciate some of the challenges specific to reviewing the methods literature.

Results and discussion

Literature identification and selection.

The identification and selection process includes search and retrieval of publications and the development and application of inclusion and exclusion criteria to select the publications that will be abstracted and analyzed in the final review. Literature identification and selection for overviews of the methods literature is challenging and potentially more resource-intensive than for most reviews of empirical research. This is true for several reasons that we describe below, alongside discussion of the potential solutions. Additionally, we suggest in this section how the selection procedures can be chosen to match the specific analytic approach used in methods overviews.

Delimiting a manageable set of publications

One aspect of methods overviews that can make identification and selection challenging is the fact that the universe of literature containing potentially relevant information regarding most methods-related topics is expansive and often unmanageably so. Reviewers are faced with two large categories of literature: the methods literature , where the possible publication types include journal articles, books, and book chapters; and the methods-relevant sections of empirical study reports , where the possible publication types include journal articles, monographs, books, theses, and conference proceedings. In our systematic overview of sampling in qualitative research, exhaustively searching (including retrieval and first-pass screening) all publication types across both categories of literature for information on a single methods-related topic was too burdensome to be feasible. The following proposed principle follows from the need to delimit a manageable set of literature for the review.

Principle #1:

Considering the broad universe of potentially relevant literature, we propose that an important objective early in the identification and selection stage is to delimit a manageable set of methods-relevant publications in accordance with the objectives of the methods overview.

Strategy #1:

To limit the set of methods-relevant publications that must be managed in the selection process, reviewers have the option to initially review only the methods literature, and exclude the methods-relevant sections of empirical study reports, provided this aligns with the review’s particular objectives.

We propose that reviewers are justified in choosing to select only the methods literature when the objective is to map out the range of recognized concepts relevant to a methods topic, to summarize the most authoritative or influential definitions or meanings for methods-related concepts, or to demonstrate a problematic lack of clarity regarding a widely established methods-related concept and potentially make recommendations for a preferred approach to the methods topic in question. For example, in the case of the methods overview on sampling [ 18 ], the primary aim was to define areas lacking in clarity for multiple widely established sampling-related topics. In the review on intention-to-treat in the context of missing outcome data [ 17 ], the authors identified a lack of clarity based on multiple inconsistent definitions in the literature and went on to recommend separating the issue of how to handle missing outcome data from the issue of whether an intention-to-treat analysis can be claimed.

In contrast to strategy #1, it may be appropriate to select the methods-relevant sections of empirical study reports when the objective is to illustrate how a methods concept is operationalized in research practice or reported by authors. For example, one could review all the publications in 2 years’ worth of issues of five high-impact field-related journals to answer questions about how researchers describe implementing a particular method or approach, or to quantify how consistently they define or report using it. Such reviews are often used to highlight gaps in the reporting practices regarding specific methods, which may be used to justify items to address in reporting guidelines (for example, [ 14 – 16 ]).

It is worth recognizing that other authors have advocated broader positions regarding the scope of literature to be considered in a review, expanding on our perspective. Suri [ 10 ] (who, like us, emphasizes how different sampling strategies are suitable for different literature synthesis objectives) has, for example, described a two-stage literature sampling procedure (pp. 96–97). First, reviewers use an initial approach to conduct a broad overview of the field—for reviews of methods topics, this would entail an initial review of the research methods literature. This is followed by a second more focused stage in which practical examples are purposefully selected—for methods reviews, this would involve sampling the empirical literature to illustrate key themes and variations. While this approach is seductive in its capacity to generate more in depth and interpretive analytic findings, some reviewers may consider it too resource-intensive to include the second step no matter how selective the purposeful sampling. In the overview on sampling where we stopped after the first stage [ 18 ], we discussed our selective focus on the methods literature as a limitation that left opportunities for further analysis of the literature. We explicitly recommended, for example, that theoretical sampling was a topic for which a future review of the methods sections of empirical reports was justified to answer specific questions identified in the primary review.

Ultimately, reviewers must make pragmatic decisions that balance resource considerations, combined with informed predictions about the depth and complexity of literature available on their topic, with the stated objectives of their review. The remaining principles and strategies apply primarily to overviews that include the methods literature, although some aspects may be relevant to reviews that include empirical study reports.

Searching beyond standard bibliographic databases

An important reality affecting identification and selection in overviews of the methods literature is the increased likelihood for relevant publications to be located in sources other than journal articles (which is usually not the case for overviews of empirical research, where journal articles generally represent the primary publication type). In the overview on sampling [ 18 ], out of 41 full-text publications retrieved and reviewed, only 4 were journal articles, while 37 were books or book chapters. Since many books and book chapters did not exist electronically, their full text had to be physically retrieved in hardcopy, while 11 publications were retrievable only through interlibrary loan or purchase request. The tasks associated with such retrieval are substantially more time-consuming than electronic retrieval. Since a substantial proportion of methods-related guidance may be located in publication types that are less comprehensively indexed in standard bibliographic databases, identification and retrieval thus become complicated processes.

Principle #2:

Considering that important sources of methods guidance can be located in non-journal publication types (e.g., books, book chapters) that tend to be poorly indexed in standard bibliographic databases, it is important to consider alternative search methods for identifying relevant publications to be further screened for inclusion.

Strategy #2:

To identify books, book chapters, and other non-journal publication types not thoroughly indexed in standard bibliographic databases, reviewers may choose to consult one or more of the following less standard sources: Google Scholar, publisher web sites, or expert opinion.

In the case of the overview on sampling in qualitative research [ 18 ], Google Scholar had two advantages over other standard bibliographic databases: it indexes and returns records of books and book chapters likely to contain guidance on qualitative research methods topics; and it has been validated as providing higher citation counts than ISI Web of Science (a producer of numerous bibliographic databases accessible through institutional subscription) for several non-biomedical disciplines including the social sciences where qualitative research methods are prominently used [ 19 – 21 ]. While we identified numerous useful publications by consulting experts, the author publication lists generated through Google Scholar searches were uniquely useful to identify more recent editions of methods books identified by experts.

Searching without relevant metadata

Determining what publications to select for inclusion in the overview on sampling [ 18 ] could only rarely be accomplished by reviewing the publication’s metadata. This was because for the many books and other non-journal type publications we identified as possibly relevant, the potential content of interest would be located in only a subsection of the publication. In this common scenario for reviews of the methods literature (as opposed to methods overviews that include empirical study reports), reviewers will often be unable to employ standard title, abstract, and keyword database searching or screening as a means for selecting publications.

Principle #3:

Considering that the presence of information about the topic of interest may not be indicated in the metadata for books and similar publication types, it is important to consider other means of identifying potentially useful publications for further screening.

Strategy #3:

One approach to identifying potentially useful books and similar publication types is to consider what classes of such publications (e.g., all methods manuals for a certain research approach) are likely to contain relevant content, then identify, retrieve, and review the full text of corresponding publications to determine whether they contain information on the topic of interest.

In the example of the overview on sampling in qualitative research [ 18 ], the topic of interest (sampling) was one of numerous topics covered in the general qualitative research methods manuals. Consequently, examples from this class of publications first had to be identified for retrieval according to non-keyword-dependent criteria. Thus, all methods manuals within the three research traditions reviewed (grounded theory, phenomenology, and case study) that might contain discussion of sampling were sought through Google Scholar and expert opinion, their full text obtained, and hand-searched for relevant content to determine eligibility. We used tables of contents and index sections of books to aid this hand searching.

Purposefully selecting literature on conceptual grounds

A final consideration in methods overviews relates to the type of analysis used to generate the review findings. Unlike quantitative systematic reviews where reviewers aim for accurate or unbiased quantitative estimates—something that requires identifying and selecting the literature exhaustively to obtain all relevant data available (i.e., a complete sample)—in methods overviews, reviewers must describe and interpret the relevant literature in qualitative terms to achieve review objectives. In other words, the aim in methods overviews is to seek coverage of the qualitative concepts relevant to the methods topic at hand. For example, in the overview of sampling in qualitative research [ 18 ], achieving review objectives entailed providing conceptual coverage of eight sampling-related topics that emerged as key domains. The following principle recognizes that literature sampling should therefore support generating qualitative conceptual data as the input to analysis.

Principle #4:

Since the analytic findings of a systematic methods overview are generated through qualitative description and interpretation of the literature on a specified topic, selection of the literature should be guided by a purposeful strategy designed to achieve adequate conceptual coverage (i.e., representing an appropriate degree of variation in relevant ideas) of the topic according to objectives of the review.

Strategy #4:

One strategy for choosing the purposeful approach to use in selecting the literature according to the review objectives is to consider whether those objectives imply exploring concepts either at a broad overview level, in which case combining maximum variation selection with a strategy that limits yield (e.g., critical case, politically important, or sampling for influence—described below) may be appropriate; or in depth, in which case purposeful approaches aimed at revealing innovative cases will likely be necessary.

In the methods overview on sampling, the implied scope was broad since we set out to review publications on sampling across three divergent qualitative research traditions—grounded theory, phenomenology, and case study—to facilitate making informative conceptual comparisons. Such an approach would be analogous to maximum variation sampling.

At the same time, the purpose of that review was to critically interrogate the clarity, consistency, and comprehensiveness of literature from these traditions that was “most likely to have widely influenced students’ and researchers’ ideas about sampling” (p. 1774) [ 18 ]. In other words, we explicitly set out to review and critique the most established and influential (and therefore dominant) literature, since this represents a common basis of knowledge among students and researchers seeking understanding or practical guidance on sampling in qualitative research. To achieve this objective, we purposefully sampled publications according to the criterion of influence , which we operationalized as how often an author or publication has been referenced in print or informal discourse. This second sampling approach also limited the literature we needed to consider within our broad scope review to a manageable amount.

To operationalize this strategy of sampling for influence , we sought to identify both the most influential authors within a qualitative research tradition (all of whose citations were subsequently screened) and the most influential publications on the topic of interest by non-influential authors. This involved a flexible approach that combined multiple indicators of influence to avoid the dilemma that any single indicator might provide inadequate coverage. These indicators included bibliometric data (h-index for author influence [ 22 ]; number of cites for publication influence), expert opinion, and cross-references in the literature (i.e., snowball sampling). As a final selection criterion, a publication was included only if it made an original contribution in terms of novel guidance regarding sampling or a related concept; thus, purely secondary sources were excluded. Publish or Perish software (Anne-Wil Harzing; available at http://www.harzing.com/resources/publish-or-perish ) was used to generate bibliometric data via the Google Scholar database. Figure  1 illustrates how identification and selection in the methods overview on sampling was a multi-faceted and iterative process. The authors selected as influential, and the publications selected for inclusion or exclusion are listed in Additional file 1 (Matrices 1, 2a, 2b).

Literature identification and selection process used in the methods overview on sampling [ 18 ]

In summary, the strategies of seeking maximum variation and sampling for influence were employed in the sampling overview to meet the specific review objectives described. Reviewers will need to consider the full range of purposeful literature sampling approaches at their disposal in deciding what best matches the specific aims of their own reviews. Suri [ 10 ] has recently retooled Patton’s well-known typology of purposeful sampling strategies (originally intended for primary research) for application to literature synthesis, providing a useful resource in this respect.

Data abstraction

The purpose of data abstraction in rigorous literature reviews is to locate and record all data relevant to the topic of interest from the full text of included publications, making them available for subsequent analysis. Conventionally, a data abstraction form—consisting of numerous distinct conceptually defined fields to which corresponding information from the source publication is recorded—is developed and employed. There are several challenges, however, to the processes of developing the abstraction form and abstracting the data itself when conducting methods overviews, which we address here. Some of these problems and their solutions may be familiar to those who have conducted qualitative literature syntheses, which are similarly conceptual.

Iteratively defining conceptual information to abstract

In the overview on sampling [ 18 ], while we surveyed multiple sources beforehand to develop a list of concepts relevant for abstraction (e.g., purposeful sampling strategies, saturation, sample size), there was no way for us to anticipate some concepts prior to encountering them in the review process. Indeed, in many cases, reviewers are unable to determine the complete set of methods-related concepts that will be the focus of the final review a priori without having systematically reviewed the publications to be included. Thus, defining what information to abstract beforehand may not be feasible.

Principle #5:

Considering the potential impracticality of defining a complete set of relevant methods-related concepts from a body of literature one has not yet systematically read, selecting and defining fields for data abstraction must often be undertaken iteratively. Thus, concepts to be abstracted can be expected to grow and change as data abstraction proceeds.

Strategy #5:

Reviewers can develop an initial form or set of concepts for abstraction purposes according to standard methods (e.g., incorporating expert feedback, pilot testing) and remain attentive to the need to iteratively revise it as concepts are added or modified during the review. Reviewers should document revisions and return to re-abstract data from previously abstracted publications as the new data requirements are determined.

In the sampling overview [ 18 ], we developed and maintained the abstraction form in Microsoft Word. We derived the initial set of abstraction fields from our own knowledge of relevant sampling-related concepts, consultation with local experts, and reviewing a pilot sample of publications. Since the publications in this review included a large proportion of books, the abstraction process often began by flagging the broad sections within a publication containing topic-relevant information for detailed review to identify text to abstract. When reviewing flagged text, the reviewer occasionally encountered an unanticipated concept significant enough to warrant being added as a new field to the abstraction form. For example, a field was added to capture how authors described the timing of sampling decisions, whether before (a priori) or after (ongoing) starting data collection, or whether this was unclear. In these cases, we systematically documented the modification to the form and returned to previously abstracted publications to abstract any information that might be relevant to the new field.

The logic of this strategy is analogous to the logic used in a form of research synthesis called best fit framework synthesis (BFFS) [ 23 – 25 ]. In that method, reviewers initially code evidence using an a priori framework they have selected. When evidence cannot be accommodated by the selected framework, reviewers then develop new themes or concepts from which they construct a new expanded framework. Both the strategy proposed and the BFFS approach to research synthesis are notable for their rigorous and transparent means to adapt a final set of concepts to the content under review.

Accounting for inconsistent terminology

An important complication affecting the abstraction process in methods overviews is that the language used by authors to describe methods-related concepts can easily vary across publications. For example, authors from different qualitative research traditions often use different terms for similar methods-related concepts. Furthermore, as we found in the sampling overview [ 18 ], there may be cases where no identifiable term, phrase, or label for a methods-related concept is used at all, and a description of it is given instead. This can make searching the text for relevant concepts based on keywords unreliable.

Principle #6:

Since accepted terms may not be used consistently to refer to methods concepts, it is necessary to rely on the definitions for concepts, rather than keywords, to identify relevant information in the publication to abstract.

Strategy #6:

An effective means to systematically identify relevant information is to develop and iteratively adjust written definitions for key concepts (corresponding to abstraction fields) that are consistent with and as inclusive of as much of the literature reviewed as possible. Reviewers then seek information that matches these definitions (rather than keywords) when scanning a publication for relevant data to abstract.

In the abstraction process for the sampling overview [ 18 ], we noted the several concepts of interest to the review for which abstraction by keyword was particularly problematic due to inconsistent terminology across publications: sampling , purposeful sampling , sampling strategy , and saturation (for examples, see Additional file 1 , Matrices 3a, 3b, 4). We iteratively developed definitions for these concepts by abstracting text from publications that either provided an explicit definition or from which an implicit definition could be derived, which was recorded in fields dedicated to the concept’s definition. Using a method of constant comparison, we used text from definition fields to inform and modify a centrally maintained definition of the corresponding concept to optimize its fit and inclusiveness with the literature reviewed. Table  1 shows, as an example, the final definition constructed in this way for one of the central concepts of the review, qualitative sampling .

We applied iteratively developed definitions when making decisions about what specific text to abstract for an existing field, which allowed us to abstract concept-relevant data even if no recognized keyword was used. For example, this was the case for the sampling-related concept, saturation , where the relevant text available for abstraction in one publication [ 26 ]—“to continue to collect data until nothing new was being observed or recorded, no matter how long that takes”—was not accompanied by any term or label whatsoever.

This comparative analytic strategy (and our approach to analysis more broadly as described in strategy #7, below) is analogous to the process of reciprocal translation —a technique first introduced for meta-ethnography by Noblit and Hare [ 27 ] that has since been recognized as a common element in a variety of qualitative metasynthesis approaches [ 28 ]. Reciprocal translation, taken broadly, involves making sense of a study’s findings in terms of the findings of the other studies included in the review. In practice, it has been operationalized in different ways. Melendez-Torres and colleagues developed a typology from their review of the metasynthesis literature, describing four overlapping categories of specific operations undertaken in reciprocal translation: visual representation, key paper integration, data reduction and thematic extraction, and line-by-line coding [ 28 ]. The approaches suggested in both strategies #6 and #7, with their emphasis on constant comparison, appear to fall within the line-by-line coding category.

Generating credible and verifiable analytic interpretations

The analysis in a systematic methods overview must support its more general objective, which we suggested above is often to offer clarity and enhance collective understanding regarding a chosen methods topic. In our experience, this involves describing and interpreting the relevant literature in qualitative terms. Furthermore, any interpretative analysis required may entail reaching different levels of abstraction, depending on the more specific objectives of the review. For example, in the overview on sampling [ 18 ], we aimed to produce a comparative analysis of how multiple sampling-related topics were treated differently within and among different qualitative research traditions. To promote credibility of the review, however, not only should one seek a qualitative analytic approach that facilitates reaching varying levels of abstraction but that approach must also ensure that abstract interpretations are supported and justified by the source data and not solely the product of the analyst’s speculative thinking.

Principle #7:

Considering the qualitative nature of the analysis required in systematic methods overviews, it is important to select an analytic method whose interpretations can be verified as being consistent with the literature selected, regardless of the level of abstraction reached.

Strategy #7:

We suggest employing the constant comparative method of analysis [ 29 ] because it supports developing and verifying analytic links to the source data throughout progressively interpretive or abstract levels. In applying this approach, we advise a rigorous approach, documenting how supportive quotes or references to the original texts are carried forward in the successive steps of analysis to allow for easy verification.

The analytic approach used in the methods overview on sampling [ 18 ] comprised four explicit steps, progressing in level of abstraction—data abstraction, matrices, narrative summaries, and final analytic conclusions (Fig.  2 ). While we have positioned data abstraction as the second stage of the generic review process (prior to Analysis), above, we also considered it as an initial step of analysis in the sampling overview for several reasons. First, it involved a process of constant comparisons and iterative decision-making about the fields to add or define during development and modification of the abstraction form, through which we established the range of concepts to be addressed in the review. At the same time, abstraction involved continuous analytic decisions about what textual quotes (ranging in size from short phrases to numerous paragraphs) to record in the fields thus created. This constant comparative process was analogous to open coding in which textual data from publications was compared to conceptual fields (equivalent to codes) or to other instances of data previously abstracted when constructing definitions to optimize their fit with the overall literature as described in strategy #6. Finally, in the data abstraction step, we also recorded our first interpretive thoughts in dedicated fields, providing initial material for the more abstract analytic steps.

Summary of progressive steps of analysis used in the methods overview on sampling [ 18 ]

In the second step of the analysis, we constructed topic-specific matrices , or tables, by copying relevant quotes from abstraction forms into the appropriate cells of matrices (for the complete set of analytic matrices developed in the sampling review, see Additional file 1 (matrices 3 to 10)). Each matrix ranged from one to five pages; row headings, nested three-deep, identified the methodological tradition, author, and publication, respectively; and column headings identified the concepts, which corresponded to abstraction fields. Matrices thus allowed us to make further comparisons across methodological traditions, and between authors within a tradition. In the third step of analysis, we recorded our comparative observations as narrative summaries , in which we used illustrative quotes more sparingly. In the final step, we developed analytic conclusions based on the narrative summaries about the sampling-related concepts within each methodological tradition for which clarity, consistency, or comprehensiveness of the available guidance appeared to be lacking. Higher levels of analysis thus built logically from the lower levels, enabling us to easily verify analytic conclusions by tracing the support for claims by comparing the original text of publications reviewed.

Integrative versus interpretive methods overviews

The analytic product of systematic methods overviews is comparable to qualitative evidence syntheses, since both involve describing and interpreting the relevant literature in qualitative terms. Most qualitative synthesis approaches strive to produce new conceptual understandings that vary in level of interpretation. Dixon-Woods and colleagues [ 30 ] elaborate on a useful distinction, originating from Noblit and Hare [ 27 ], between integrative and interpretive reviews. Integrative reviews focus on summarizing available primary data and involve using largely secure and well defined concepts to do so; definitions are used from an early stage to specify categories for abstraction (or coding) of data, which in turn supports their aggregation; they do not seek as their primary focus to develop or specify new concepts, although they may achieve some theoretical or interpretive functions. For interpretive reviews, meanwhile, the main focus is to develop new concepts and theories that integrate them, with the implication that the concepts developed become fully defined towards the end of the analysis. These two forms are not completely distinct, and “every integrative synthesis will include elements of interpretation, and every interpretive synthesis will include elements of aggregation of data” [ 30 ].

The example methods overview on sampling [ 18 ] could be classified as predominantly integrative because its primary goal was to aggregate influential authors’ ideas on sampling-related concepts; there were also, however, elements of interpretive synthesis since it aimed to develop new ideas about where clarity in guidance on certain sampling-related topics is lacking, and definitions for some concepts were flexible and not fixed until late in the review. We suggest that most systematic methods overviews will be classifiable as predominantly integrative (aggregative). Nevertheless, more highly interpretive methods overviews are also quite possible—for example, when the review objective is to provide a highly critical analysis for the purpose of generating new methodological guidance. In such cases, reviewers may need to sample more deeply (see strategy #4), specifically by selecting empirical research reports (i.e., to go beyond dominant or influential ideas in the methods literature) that are likely to feature innovations or instructive lessons in employing a given method.

In this paper, we have outlined tentative guidance in the form of seven principles and strategies on how to conduct systematic methods overviews, a review type in which methods-relevant literature is systematically analyzed with the aim of offering clarity and enhancing collective understanding regarding a specific methods topic. Our proposals include strategies for delimiting the set of publications to consider, searching beyond standard bibliographic databases, searching without the availability of relevant metadata, selecting publications on purposeful conceptual grounds, defining concepts and other information to abstract iteratively, accounting for inconsistent terminology, and generating credible and verifiable analytic interpretations. We hope the suggestions proposed will be useful to others undertaking reviews on methods topics in future.

As far as we are aware, this is the first published source of concrete guidance for conducting this type of review. It is important to note that our primary objective was to initiate methodological discussion by stimulating reflection on what rigorous methods for this type of review should look like, leaving the development of more complete guidance to future work. While derived from the experience of reviewing a single qualitative methods topic, we believe the principles and strategies provided are generalizable to overviews of both qualitative and quantitative methods topics alike. However, it is expected that additional challenges and insights for conducting such reviews have yet to be defined. Thus, we propose that next steps for developing more definitive guidance should involve an attempt to collect and integrate other reviewers’ perspectives and experiences in conducting systematic methods overviews on a broad range of qualitative and quantitative methods topics. Formalized guidance and standards would improve the quality of future methods overviews, something we believe has important implications for advancing qualitative and quantitative methodology. When undertaken to a high standard, rigorous critical evaluations of the available methods guidance have significant potential to make implicit controversies explicit, and improve the clarity and precision of our understandings of problematic qualitative or quantitative methods issues.

A review process central to most types of rigorous reviews of empirical studies, which we did not explicitly address in a separate review step above, is quality appraisal . The reason we have not treated this as a separate step stems from the different objectives of the primary publications included in overviews of the methods literature (i.e., providing methodological guidance) compared to the primary publications included in the other established review types (i.e., reporting findings from single empirical studies). This is not to say that appraising quality of the methods literature is not an important concern for systematic methods overviews. Rather, appraisal is much more integral to (and difficult to separate from) the analysis step, in which we advocate appraising clarity, consistency, and comprehensiveness—the quality appraisal criteria that we suggest are appropriate for the methods literature. As a second important difference regarding appraisal, we currently advocate appraising the aforementioned aspects at the level of the literature in aggregate rather than at the level of individual publications. One reason for this is that methods guidance from individual publications generally builds on previous literature, and thus we feel that ahistorical judgments about comprehensiveness of single publications lack relevance and utility. Additionally, while different methods authors may express themselves less clearly than others, their guidance can nonetheless be highly influential and useful, and should therefore not be downgraded or ignored based on considerations of clarity—which raises questions about the alternative uses that quality appraisals of individual publications might have. Finally, legitimate variability in the perspectives that methods authors wish to emphasize, and the levels of generality at which they write about methods, makes critiquing individual publications based on the criterion of clarity a complex and potentially problematic endeavor that is beyond the scope of this paper to address. By appraising the current state of the literature at a holistic level, reviewers stand to identify important gaps in understanding that represent valuable opportunities for further methodological development.

To summarize, the principles and strategies provided here may be useful to those seeking to undertake their own systematic methods overview. Additional work is needed, however, to establish guidance that is comprehensive by comparing the experiences from conducting a variety of methods overviews on a range of methods topics. Efforts that further advance standards for systematic methods overviews have the potential to promote high-quality critical evaluations that produce conceptually clear and unified understandings of problematic methods topics, thereby accelerating the advance of research methodology.

Hutton JL, Ashcroft R. What does “systematic” mean for reviews of methods? In: Black N, Brazier J, Fitzpatrick R, Reeves B, editors. Health services research methods: a guide to best practice. London: BMJ Publishing Group; 1998. p. 249–54.

Google Scholar  

Cochrane handbook for systematic reviews of interventions. In. Edited by Higgins JPT, Green S, Version 5.1.0 edn: The Cochrane Collaboration; 2011.

Centre for Reviews and Dissemination: Systematic reviews: CRD’s guidance for undertaking reviews in health care . York: Centre for Reviews and Dissemination; 2009.

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700–0.

Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009;9(1):59.

Article   PubMed   PubMed Central   Google Scholar  

Kastner M, Tricco AC, Soobiah C, Lillie E, Perrier L, Horsley T, Welch V, Cogo E, Antony J, Straus SE. What is the most appropriate knowledge synthesis method to conduct a review? Protocol for a scoping review. BMC Med Res Methodol. 2012;12(1):1–1.

Article   Google Scholar  

Booth A, Noyes J, Flemming K, Gerhardus A. Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions. In: Integrate-HTA. 2016.

Booth A, Sutton A, Papaioannou D. Systematic approaches to successful literature review. 2nd ed. London: Sage; 2016.

Hannes K, Lockwood C. Synthesizing qualitative research: choosing the right approach. Chichester: Wiley-Blackwell; 2012.

Suri H. Towards methodologically inclusive research syntheses: expanding possibilities. New York: Routledge; 2014.

Campbell M, Egan M, Lorenc T, Bond L, Popham F, Fenton C, Benzeval M. Considering methodological options for reviews of theory: illustrated by a review of theories linking income and health. Syst Rev. 2014;3(1):1–11.

Cohen DJ, Crabtree BF. Evaluative criteria for qualitative research in health care: controversies and recommendations. Ann Fam Med. 2008;6(4):331–9.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reportingqualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

Article   PubMed   Google Scholar  

Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7(2):e1000217.

Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4(3):e78.

Chan AW, Altman DG. Epidemiology and reporting of randomised trials published in PubMed journals. Lancet. 2005;365(9465):1159–62.

Alshurafa M, Briel M, Akl EA, Haines T, Moayyedi P, Gentles SJ, Rios L, Tran C, Bhatnagar N, Lamontagne F, et al. Inconsistent definitions for intention-to-treat in relation to missing outcome data: systematic review of the methods literature. PLoS One. 2012;7(11):e49163.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Gentles SJ, Charles C, Ploeg J, McKibbon KA. Sampling in qualitative research: insights from an overview of the methods literature. Qual Rep. 2015;20(11):1772–89.

Harzing A-W, Alakangas S. Google Scholar, Scopus and the Web of Science: a longitudinal and cross-disciplinary comparison. Scientometrics. 2016;106(2):787–804.

Harzing A-WK, van der Wal R. Google Scholar as a new source for citation analysis. Ethics Sci Environ Polit. 2008;8(1):61–73.

Kousha K, Thelwall M. Google Scholar citations and Google Web/URL citations: a multi‐discipline exploratory analysis. J Assoc Inf Sci Technol. 2007;58(7):1055–65.

Hirsch JE. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci U S A. 2005;102(46):16569–72.

Booth A, Carroll C. How to build up the actionable knowledge base: the role of ‘best fit’ framework synthesis for studies of improvement in healthcare. BMJ Quality Safety. 2015;24(11):700–8.

Carroll C, Booth A, Leaviss J, Rick J. “Best fit” framework synthesis: refining the method. BMC Med Res Methodol. 2013;13(1):37.

Carroll C, Booth A, Cooper K. A worked example of “best fit” framework synthesis: a systematic review of views concerning the taking of some potential chemopreventive agents. BMC Med Res Methodol. 2011;11(1):29.

Cohen MZ, Kahn DL, Steeves DL. Hermeneutic phenomenological research: a practical guide for nurse researchers. Thousand Oaks: Sage; 2000.

Noblit GW, Hare RD. Meta-ethnography: synthesizing qualitative studies. Newbury Park: Sage; 1988.

Book   Google Scholar  

Melendez-Torres GJ, Grant S, Bonell C. A systematic review and critical appraisal of qualitative metasynthetic practice in public health to develop a taxonomy of operations of reciprocal translation. Res Synthesis Methods. 2015;6(4):357–71.

Article   CAS   Google Scholar  

Glaser BG, Strauss A. The discovery of grounded theory. Chicago: Aldine; 1967.

Dixon-Woods M, Agarwal S, Young B, Jones D, Sutton A. Integrative approaches to qualitative and quantitative evidence. In: UK National Health Service. 2004. p. 1–44.

Download references

Acknowledgements

Not applicable.

There was no funding for this work.

Availability of data and materials

The systematic methods overview used as a worked example in this article (Gentles SJ, Charles C, Ploeg J, McKibbon KA: Sampling in qualitative research: insights from an overview of the methods literature. The Qual Rep 2015, 20(11):1772-1789) is available from http://nsuworks.nova.edu/tqr/vol20/iss11/5 .

Authors’ contributions

SJG wrote the first draft of this article, with CC contributing to drafting. All authors contributed to revising the manuscript. All authors except CC (deceased) approved the final draft. SJG, CC, KAB, and JP were involved in developing methods for the systematic methods overview on sampling.

Authors’ information

Competing interests.

The authors declare that they have no competing interests.

Consent for publication

Ethics approval and consent to participate, author information, authors and affiliations.

Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada

Stephen J. Gentles, Cathy Charles & K. Ann McKibbon

Faculty of Social Work, University of Calgary, Alberta, Canada

David B. Nicholas

School of Nursing, McMaster University, Hamilton, Ontario, Canada

Jenny Ploeg

CanChild Centre for Childhood Disability Research, McMaster University, 1400 Main Street West, IAHS 408, Hamilton, ON, L8S 1C7, Canada

Stephen J. Gentles

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Stephen J. Gentles .

Additional information

Cathy Charles is deceased

Additional file

Additional file 1:.

Submitted: Analysis_matrices. (DOC 330 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Gentles, S.J., Charles, C., Nicholas, D.B. et al. Reviewing the research methods literature: principles and strategies illustrated by a systematic overview of sampling in qualitative research. Syst Rev 5 , 172 (2016). https://doi.org/10.1186/s13643-016-0343-0

Download citation

Received : 06 June 2016

Accepted : 14 September 2016

Published : 11 October 2016

DOI : https://doi.org/10.1186/s13643-016-0343-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Systematic review
• Literature selection
• Research methods
• Research methodology
• Overview of methods
• Systematic methods overview
• Review methods

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology

Research Methods | Definition, Types, Examples

Research methods are specific procedures for collecting and analysing data. Developing your research methods is an integral part of your research design . When planning your methods, there are two key decisions you will make.

First, decide how you will collect data . Your methods depend on what type of data you need to answer your research question :

• Qualitative vs quantitative : Will your data take the form of words or numbers?
• Primary vs secondary : Will you collect original data yourself, or will you use data that have already been collected by someone else?
• Descriptive vs experimental : Will you take measurements of something as it is, or will you perform an experiment?

Second, decide how you will analyse the data .

• For quantitative data, you can use statistical analysis methods to test relationships between variables.
• For qualitative data, you can use methods such as thematic analysis to interpret patterns and meanings in the data.

Table of contents

Methods for collecting data, examples of data collection methods, methods for analysing data, examples of data analysis methods, frequently asked questions about methodology.

Data are the information that you collect for the purposes of answering your research question . The type of data you need depends on the aims of your research.

Qualitative vs quantitative data

Your choice of qualitative or quantitative data collection depends on the type of knowledge you want to develop.

For questions about ideas, experiences and meanings, or to study something that can’t be described numerically, collect qualitative data .

If you want to develop a more mechanistic understanding of a topic, or your research involves hypothesis testing , collect quantitative data .

You can also take a mixed methods approach, where you use both qualitative and quantitative research methods.

Primary vs secondary data

Primary data are any original information that you collect for the purposes of answering your research question (e.g. through surveys , observations and experiments ). Secondary data are information that has already been collected by other researchers (e.g. in a government census or previous scientific studies).

If you are exploring a novel research question, you’ll probably need to collect primary data. But if you want to synthesise existing knowledge, analyse historical trends, or identify patterns on a large scale, secondary data might be a better choice.

Descriptive vs experimental data

In descriptive research , you collect data about your study subject without intervening. The validity of your research will depend on your sampling method .

In experimental research , you systematically intervene in a process and measure the outcome. The validity of your research will depend on your experimental design .

To conduct an experiment, you need to be able to vary your independent variable , precisely measure your dependent variable, and control for confounding variables . If it’s practically and ethically possible, this method is the best choice for answering questions about cause and effect.

Prevent plagiarism, run a free check.

Your data analysis methods will depend on the type of data you collect and how you prepare them for analysis.

Data can often be analysed both quantitatively and qualitatively. For example, survey responses could be analysed qualitatively by studying the meanings of responses or quantitatively by studying the frequencies of responses.

Qualitative analysis methods

Qualitative analysis is used to understand words, ideas, and experiences. You can use it to interpret data that were collected:

• From open-ended survey and interview questions, literature reviews, case studies, and other sources that use text rather than numbers.
• Using non-probability sampling methods .

Qualitative analysis tends to be quite flexible and relies on the researcher’s judgement, so you have to reflect carefully on your choices and assumptions.

Quantitative analysis methods

Quantitative analysis uses numbers and statistics to understand frequencies, averages and correlations (in descriptive studies) or cause-and-effect relationships (in experiments).

You can use quantitative analysis to interpret data that were collected either:

• During an experiment.
• Using probability sampling methods .

Because the data are collected and analysed in a statistically valid way, the results of quantitative analysis can be easily standardised and shared among researchers.

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to test a hypothesis by systematically collecting and analysing data, while qualitative methods allow you to explore ideas and experiences in depth.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

A sample is a subset of individuals from a larger population. Sampling means selecting the group that you will actually collect data from in your research.

For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

Statistical sampling allows you to test a hypothesis about the characteristics of a population. There are various sampling methods you can use to ensure that your sample is representative of the population as a whole.

The research methods you use depend on the type of data you need to answer your research question .

• If you want to measure something or test a hypothesis , use quantitative methods . If you want to explore ideas, thoughts, and meanings, use qualitative methods .
• If you want to analyse a large amount of readily available data, use secondary data. If you want data specific to your purposes with control over how they are generated, collect primary data.
• If you want to establish cause-and-effect relationships between variables , use experimental methods. If you want to understand the characteristics of a research subject, use descriptive methods.

Methodology refers to the overarching strategy and rationale of your research project . It involves studying the methods used in your field and the theories or principles behind them, in order to develop an approach that matches your objectives.

Methods are the specific tools and procedures you use to collect and analyse data (e.g. experiments, surveys , and statistical tests ).

In shorter scientific papers, where the aim is to report the findings of a specific study, you might simply describe what you did in a methods section .

In a longer or more complex research project, such as a thesis or dissertation , you will probably include a methodology section , where you explain your approach to answering the research questions and cite relevant sources to support your choice of methods.

Is this article helpful?

More interesting articles.

• A Quick Guide to Experimental Design | 5 Steps & Examples
• Between-Subjects Design | Examples, Pros & Cons
• Case Study | Definition, Examples & Methods
• Cluster Sampling | A Simple Step-by-Step Guide with Examples
• Confounding Variables | Definition, Examples & Controls
• Construct Validity | Definition, Types, & Examples
• Content Analysis | A Step-by-Step Guide with Examples
• Control Groups and Treatment Groups | Uses & Examples
• Controlled Experiments | Methods & Examples of Control
• Correlation vs Causation | Differences, Designs & Examples
• Correlational Research | Guide, Design & Examples
• Critical Discourse Analysis | Definition, Guide & Examples
• Cross-Sectional Study | Definitions, Uses & Examples
• Data Cleaning | A Guide with Examples & Steps
• Data Collection Methods | Step-by-Step Guide & Examples
• Descriptive Research Design | Definition, Methods & Examples
• Doing Survey Research | A Step-by-Step Guide & Examples
• Ethical Considerations in Research | Types & Examples
• Explanatory Research | Definition, Guide, & Examples
• Explanatory vs Response Variables | Definitions & Examples
• Exploratory Research | Definition, Guide, & Examples
• External Validity | Types, Threats & Examples
• Extraneous Variables | Examples, Types, Controls
• Face Validity | Guide with Definition & Examples
• How to Do Thematic Analysis | Guide & Examples
• How to Write a Strong Hypothesis | Guide & Examples
• Inclusion and Exclusion Criteria | Examples & Definition
• Independent vs Dependent Variables | Definition & Examples
• Inductive Reasoning | Types, Examples, Explanation
• Inductive vs Deductive Research Approach (with Examples)
• Internal Validity | Definition, Threats & Examples
• Internal vs External Validity | Understanding Differences & Examples
• Longitudinal Study | Definition, Approaches & Examples
• Mediator vs Moderator Variables | Differences & Examples
• Mixed Methods Research | Definition, Guide, & Examples
• Multistage Sampling | An Introductory Guide with Examples
• Naturalistic Observation | Definition, Guide & Examples
• Operationalisation | A Guide with Examples, Pros & Cons
• Population vs Sample | Definitions, Differences & Examples
• Primary Research | Definition, Types, & Examples
• Qualitative vs Quantitative Research | Examples & Methods
• Quasi-Experimental Design | Definition, Types & Examples
• Questionnaire Design | Methods, Question Types & Examples
• Random Assignment in Experiments | Introduction & Examples
• Reliability vs Validity in Research | Differences, Types & Examples
• Reproducibility vs Replicability | Difference & Examples
• Research Design | Step-by-Step Guide with Examples
• Sampling Methods | Types, Techniques, & Examples
• Semi-Structured Interview | Definition, Guide & Examples
• Simple Random Sampling | Definition, Steps & Examples
• Stratified Sampling | A Step-by-Step Guide with Examples
• Structured Interview | Definition, Guide & Examples
• Systematic Review | Definition, Examples & Guide
• Systematic Sampling | A Step-by-Step Guide with Examples
• Textual Analysis | Guide, 3 Approaches & Examples
• The 4 Types of Reliability in Research | Definitions & Examples
• The 4 Types of Validity | Types, Definitions & Examples
• Transcribing an Interview | 5 Steps & Transcription Software
• Triangulation in Research | Guide, Types, Examples
• Types of Interviews in Research | Guide & Examples
• Types of Research Designs Compared | Examples
• Types of Variables in Research | Definitions & Examples
• Unstructured Interview | Definition, Guide & Examples
• What Are Control Variables | Definition & Examples
• What Is a Case-Control Study? | Definition & Examples
• What Is a Cohort Study? | Definition & Examples
• What Is a Conceptual Framework? | Tips & Examples
• What Is a Double-Barrelled Question?
• What Is a Double-Blind Study? | Introduction & Examples
• What Is a Focus Group? | Step-by-Step Guide & Examples
• What Is a Likert Scale? | Guide & Examples
• What is a Literature Review? | Guide, Template, & Examples
• What Is a Prospective Cohort Study? | Definition & Examples
• What Is a Retrospective Cohort Study? | Definition & Examples
• What Is Action Research? | Definition & Examples
• What Is an Observational Study? | Guide & Examples
• What Is Concurrent Validity? | Definition & Examples
• What Is Content Validity? | Definition & Examples
• What Is Convenience Sampling? | Definition & Examples
• What Is Convergent Validity? | Definition & Examples
• What Is Criterion Validity? | Definition & Examples
• What Is Deductive Reasoning? | Explanation & Examples
• What Is Discriminant Validity? | Definition & Example
• What Is Ecological Validity? | Definition & Examples
• What Is Ethnography? | Meaning, Guide & Examples
• What Is Non-Probability Sampling? | Types & Examples
• What Is Participant Observation? | Definition & Examples
• What Is Peer Review? | Types & Examples
• What Is Predictive Validity? | Examples & Definition
• What Is Probability Sampling? | Types & Examples
• What Is Purposive Sampling? | Definition & Examples
• What Is Qualitative Observation? | Definition & Examples
• What Is Qualitative Research? | Methods & Examples
• What Is Quantitative Observation? | Definition & Examples
• What Is Quantitative Research? | Definition & Methods
• What Is Quota Sampling? | Definition & Examples
• What is Secondary Research? | Definition, Types, & Examples
• What Is Snowball Sampling? | Definition & Examples
• Within-Subjects Design | Explanation, Approaches, Examples

Articles on Research methods

Displaying 1 - 20 of 26 articles.

E. coli is one of the most widely studied organisms – and that may be a problem for both science and medicine

Megan Keller , Cornell University and Tobias Dörr , Cornell University

Visualizing the inside of cells at previously impossible resolutions provides vivid insights into how they work

Jeremy Berg , University of Pittsburgh

Nanomedicines for various diseases are in development – but research facilities produce vastly inconsistent results on how the body will react to them

Morteza Mahmoudi , Michigan State University

Collaborative Indigenous Research is a way to repair the legacy of harmful research practices

Eve Tuck , University of Toronto

Psychedelic drugs: how to tell good research from bad

James Rucker , King's College London

African ubuntu can deepen how research is done

Obaa Akua Konadu-Osei , Stellenbosch University

Studying how people spend their time reveals how societies differ and change over time

Ugo Lachapelle , Université du Québec à Montréal (UQAM)

Young Canadians are asking to be included in research — here’s how to engage them

Roberta L. Woodgate , University of Manitoba

For too long, research was done on First Nations peoples, not with them. Universities can change this

Victoria Rawlings , University of Sydney ; James L. Flexner , University of Sydney , and Lynette Riley , University of Sydney

Studying social media can give us insight into human behaviour. It can also give us nonsense

Jason Burton , Birkbeck, University of London ; Nicole Cruz , UNSW Sydney , and Ulrike Hahn , Birkbeck, University of London

Predicting research results can mean better science and better advice

Eva Vivalt , Australian National University

Should I eat red meat? Confusing studies diminish trust in nutrition science

Scott Lear , Simon Fraser University

Historians’ archival research looks quite different in the digital age

Ian Milligan , University of Waterloo

There is no great salt debate: we should be consuming less

Feng He , Queen Mary University of London

You look but do not find: why the absence of evidence can be a useful thing

Deborah Brown , The University of Queensland and Brian Key , The University of Queensland

Unscrambling the egg: how research works out what really leads to an increased disease risk

Sean Martin , University of Adelaide

When the numbers aren’t enough: how different data work together in research

Jacqueline Stephens , University of South Australia

If ‘indigenizing’ education feels this good, we aren’t doing it right

Monique Giroux , University of Lethbridge

Decolonising research methodology must include undoing its dirty history

Sabelo Ndlovu-Gatsheni , University of South Africa

How I am learning to include Indigenous knowledge in the classroom

Kathleen Gallagher , University of Toronto

Related Topics

• Colonialism
• Decolonisation
• Mental health
• Replicability
• Reproducibility
• Research quality
• Science research
• Scientific method

Top contributors

Visiting associate professor, Massachusetts Institute of Technology (MIT)

Postdoctoral Research Fellow in Psychiatry and Behavioral Sciences, Medical University of South Carolina

Senior Lecturer, University of Sydney School of Education and Social Work, University of Sydney

Associate Professor of Archaeology, University of Washington

Research Faculty, MIT Sloan School of Management

Assistant Professor of Cognitive and Information Sciences, University of California, Merced

Professor and Chair of Epistemologies of the Global South, University of Bareuth, Germany, University of South Africa

Associate Professor in Public Health, Flinders University

Professor of Physics and Astronomy, Bucknell University

Assistant Professor of Psychology, Grand Valley State University

Georgia Institute of Technology

Program Head of Psychology, University of Guelph-Humber

Distinguished Professor, Department of Curriculum, Teaching and Learning, Ontario Institute for Studies in Education, University of Toronto

Assistant Professor of Indigenous Studies, University of Lethbridge

Professor, University of Wollongong

• X (Twitter)
• Unfollow topic Follow topic

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Korean Med Sci
• v.37(16); 2022 Apr 25

A Practical Guide to Writing Quantitative and Qualitative Research Questions and Hypotheses in Scholarly Articles

Edward barroga.

1 Department of General Education, Graduate School of Nursing Science, St. Luke’s International University, Tokyo, Japan.

Glafera Janet Matanguihan

2 Department of Biological Sciences, Messiah University, Mechanicsburg, PA, USA.

The development of research questions and the subsequent hypotheses are prerequisites to defining the main research purpose and specific objectives of a study. Consequently, these objectives determine the study design and research outcome. The development of research questions is a process based on knowledge of current trends, cutting-edge studies, and technological advances in the research field. Excellent research questions are focused and require a comprehensive literature search and in-depth understanding of the problem being investigated. Initially, research questions may be written as descriptive questions which could be developed into inferential questions. These questions must be specific and concise to provide a clear foundation for developing hypotheses. Hypotheses are more formal predictions about the research outcomes. These specify the possible results that may or may not be expected regarding the relationship between groups. Thus, research questions and hypotheses clarify the main purpose and specific objectives of the study, which in turn dictate the design of the study, its direction, and outcome. Studies developed from good research questions and hypotheses will have trustworthy outcomes with wide-ranging social and health implications.

INTRODUCTION

Scientific research is usually initiated by posing evidenced-based research questions which are then explicitly restated as hypotheses. 1 , 2 The hypotheses provide directions to guide the study, solutions, explanations, and expected results. 3 , 4 Both research questions and hypotheses are essentially formulated based on conventional theories and real-world processes, which allow the inception of novel studies and the ethical testing of ideas. 5 , 6

It is crucial to have knowledge of both quantitative and qualitative research 2 as both types of research involve writing research questions and hypotheses. 7 However, these crucial elements of research are sometimes overlooked; if not overlooked, then framed without the forethought and meticulous attention it needs. Planning and careful consideration are needed when developing quantitative or qualitative research, particularly when conceptualizing research questions and hypotheses. 4

There is a continuing need to support researchers in the creation of innovative research questions and hypotheses, as well as for journal articles that carefully review these elements. 1 When research questions and hypotheses are not carefully thought of, unethical studies and poor outcomes usually ensue. Carefully formulated research questions and hypotheses define well-founded objectives, which in turn determine the appropriate design, course, and outcome of the study. This article then aims to discuss in detail the various aspects of crafting research questions and hypotheses, with the goal of guiding researchers as they develop their own. Examples from the authors and peer-reviewed scientific articles in the healthcare field are provided to illustrate key points.

DEFINITIONS AND RELATIONSHIP OF RESEARCH QUESTIONS AND HYPOTHESES

A research question is what a study aims to answer after data analysis and interpretation. The answer is written in length in the discussion section of the paper. Thus, the research question gives a preview of the different parts and variables of the study meant to address the problem posed in the research question. 1 An excellent research question clarifies the research writing while facilitating understanding of the research topic, objective, scope, and limitations of the study. 5

On the other hand, a research hypothesis is an educated statement of an expected outcome. This statement is based on background research and current knowledge. 8 , 9 The research hypothesis makes a specific prediction about a new phenomenon 10 or a formal statement on the expected relationship between an independent variable and a dependent variable. 3 , 11 It provides a tentative answer to the research question to be tested or explored. 4

Hypotheses employ reasoning to predict a theory-based outcome. 10 These can also be developed from theories by focusing on components of theories that have not yet been observed. 10 The validity of hypotheses is often based on the testability of the prediction made in a reproducible experiment. 8

Conversely, hypotheses can also be rephrased as research questions. Several hypotheses based on existing theories and knowledge may be needed to answer a research question. Developing ethical research questions and hypotheses creates a research design that has logical relationships among variables. These relationships serve as a solid foundation for the conduct of the study. 4 , 11 Haphazardly constructed research questions can result in poorly formulated hypotheses and improper study designs, leading to unreliable results. Thus, the formulations of relevant research questions and verifiable hypotheses are crucial when beginning research. 12

CHARACTERISTICS OF GOOD RESEARCH QUESTIONS AND HYPOTHESES

Excellent research questions are specific and focused. These integrate collective data and observations to confirm or refute the subsequent hypotheses. Well-constructed hypotheses are based on previous reports and verify the research context. These are realistic, in-depth, sufficiently complex, and reproducible. More importantly, these hypotheses can be addressed and tested. 13

There are several characteristics of well-developed hypotheses. Good hypotheses are 1) empirically testable 7 , 10 , 11 , 13 ; 2) backed by preliminary evidence 9 ; 3) testable by ethical research 7 , 9 ; 4) based on original ideas 9 ; 5) have evidenced-based logical reasoning 10 ; and 6) can be predicted. 11 Good hypotheses can infer ethical and positive implications, indicating the presence of a relationship or effect relevant to the research theme. 7 , 11 These are initially developed from a general theory and branch into specific hypotheses by deductive reasoning. In the absence of a theory to base the hypotheses, inductive reasoning based on specific observations or findings form more general hypotheses. 10

TYPES OF RESEARCH QUESTIONS AND HYPOTHESES

Research questions and hypotheses are developed according to the type of research, which can be broadly classified into quantitative and qualitative research. We provide a summary of the types of research questions and hypotheses under quantitative and qualitative research categories in Table 1 .

Research questions in quantitative research

In quantitative research, research questions inquire about the relationships among variables being investigated and are usually framed at the start of the study. These are precise and typically linked to the subject population, dependent and independent variables, and research design. 1 Research questions may also attempt to describe the behavior of a population in relation to one or more variables, or describe the characteristics of variables to be measured ( descriptive research questions ). 1 , 5 , 14 These questions may also aim to discover differences between groups within the context of an outcome variable ( comparative research questions ), 1 , 5 , 14 or elucidate trends and interactions among variables ( relationship research questions ). 1 , 5 We provide examples of descriptive, comparative, and relationship research questions in quantitative research in Table 2 .

Hypotheses in quantitative research

In quantitative research, hypotheses predict the expected relationships among variables. 15 Relationships among variables that can be predicted include 1) between a single dependent variable and a single independent variable ( simple hypothesis ) or 2) between two or more independent and dependent variables ( complex hypothesis ). 4 , 11 Hypotheses may also specify the expected direction to be followed and imply an intellectual commitment to a particular outcome ( directional hypothesis ) 4 . On the other hand, hypotheses may not predict the exact direction and are used in the absence of a theory, or when findings contradict previous studies ( non-directional hypothesis ). 4 In addition, hypotheses can 1) define interdependency between variables ( associative hypothesis ), 4 2) propose an effect on the dependent variable from manipulation of the independent variable ( causal hypothesis ), 4 3) state a negative relationship between two variables ( null hypothesis ), 4 , 11 , 15 4) replace the working hypothesis if rejected ( alternative hypothesis ), 15 explain the relationship of phenomena to possibly generate a theory ( working hypothesis ), 11 5) involve quantifiable variables that can be tested statistically ( statistical hypothesis ), 11 6) or express a relationship whose interlinks can be verified logically ( logical hypothesis ). 11 We provide examples of simple, complex, directional, non-directional, associative, causal, null, alternative, working, statistical, and logical hypotheses in quantitative research, as well as the definition of quantitative hypothesis-testing research in Table 3 .

Research questions in qualitative research

Unlike research questions in quantitative research, research questions in qualitative research are usually continuously reviewed and reformulated. The central question and associated subquestions are stated more than the hypotheses. 15 The central question broadly explores a complex set of factors surrounding the central phenomenon, aiming to present the varied perspectives of participants. 15

There are varied goals for which qualitative research questions are developed. These questions can function in several ways, such as to 1) identify and describe existing conditions ( contextual research question s); 2) describe a phenomenon ( descriptive research questions ); 3) assess the effectiveness of existing methods, protocols, theories, or procedures ( evaluation research questions ); 4) examine a phenomenon or analyze the reasons or relationships between subjects or phenomena ( explanatory research questions ); or 5) focus on unknown aspects of a particular topic ( exploratory research questions ). 5 In addition, some qualitative research questions provide new ideas for the development of theories and actions ( generative research questions ) or advance specific ideologies of a position ( ideological research questions ). 1 Other qualitative research questions may build on a body of existing literature and become working guidelines ( ethnographic research questions ). Research questions may also be broadly stated without specific reference to the existing literature or a typology of questions ( phenomenological research questions ), may be directed towards generating a theory of some process ( grounded theory questions ), or may address a description of the case and the emerging themes ( qualitative case study questions ). 15 We provide examples of contextual, descriptive, evaluation, explanatory, exploratory, generative, ideological, ethnographic, phenomenological, grounded theory, and qualitative case study research questions in qualitative research in Table 4 , and the definition of qualitative hypothesis-generating research in Table 5 .

Qualitative studies usually pose at least one central research question and several subquestions starting with How or What . These research questions use exploratory verbs such as explore or describe . These also focus on one central phenomenon of interest, and may mention the participants and research site. 15

Hypotheses in qualitative research

Hypotheses in qualitative research are stated in the form of a clear statement concerning the problem to be investigated. Unlike in quantitative research where hypotheses are usually developed to be tested, qualitative research can lead to both hypothesis-testing and hypothesis-generating outcomes. 2 When studies require both quantitative and qualitative research questions, this suggests an integrative process between both research methods wherein a single mixed-methods research question can be developed. 1

FRAMEWORKS FOR DEVELOPING RESEARCH QUESTIONS AND HYPOTHESES

Research questions followed by hypotheses should be developed before the start of the study. 1 , 12 , 14 It is crucial to develop feasible research questions on a topic that is interesting to both the researcher and the scientific community. This can be achieved by a meticulous review of previous and current studies to establish a novel topic. Specific areas are subsequently focused on to generate ethical research questions. The relevance of the research questions is evaluated in terms of clarity of the resulting data, specificity of the methodology, objectivity of the outcome, depth of the research, and impact of the study. 1 , 5 These aspects constitute the FINER criteria (i.e., Feasible, Interesting, Novel, Ethical, and Relevant). 1 Clarity and effectiveness are achieved if research questions meet the FINER criteria. In addition to the FINER criteria, Ratan et al. described focus, complexity, novelty, feasibility, and measurability for evaluating the effectiveness of research questions. 14

The PICOT and PEO frameworks are also used when developing research questions. 1 The following elements are addressed in these frameworks, PICOT: P-population/patients/problem, I-intervention or indicator being studied, C-comparison group, O-outcome of interest, and T-timeframe of the study; PEO: P-population being studied, E-exposure to preexisting conditions, and O-outcome of interest. 1 Research questions are also considered good if these meet the “FINERMAPS” framework: Feasible, Interesting, Novel, Ethical, Relevant, Manageable, Appropriate, Potential value/publishable, and Systematic. 14

As we indicated earlier, research questions and hypotheses that are not carefully formulated result in unethical studies or poor outcomes. To illustrate this, we provide some examples of ambiguous research question and hypotheses that result in unclear and weak research objectives in quantitative research ( Table 6 ) 16 and qualitative research ( Table 7 ) 17 , and how to transform these ambiguous research question(s) and hypothesis(es) into clear and good statements.

a These statements were composed for comparison and illustrative purposes only.

b These statements are direct quotes from Higashihara and Horiuchi. 16

a This statement is a direct quote from Shimoda et al. 17

The other statements were composed for comparison and illustrative purposes only.

CONSTRUCTING RESEARCH QUESTIONS AND HYPOTHESES

To construct effective research questions and hypotheses, it is very important to 1) clarify the background and 2) identify the research problem at the outset of the research, within a specific timeframe. 9 Then, 3) review or conduct preliminary research to collect all available knowledge about the possible research questions by studying theories and previous studies. 18 Afterwards, 4) construct research questions to investigate the research problem. Identify variables to be accessed from the research questions 4 and make operational definitions of constructs from the research problem and questions. Thereafter, 5) construct specific deductive or inductive predictions in the form of hypotheses. 4 Finally, 6) state the study aims . This general flow for constructing effective research questions and hypotheses prior to conducting research is shown in Fig. 1 .

Research questions are used more frequently in qualitative research than objectives or hypotheses. 3 These questions seek to discover, understand, explore or describe experiences by asking “What” or “How.” The questions are open-ended to elicit a description rather than to relate variables or compare groups. The questions are continually reviewed, reformulated, and changed during the qualitative study. 3 Research questions are also used more frequently in survey projects than hypotheses in experiments in quantitative research to compare variables and their relationships.

Hypotheses are constructed based on the variables identified and as an if-then statement, following the template, ‘If a specific action is taken, then a certain outcome is expected.’ At this stage, some ideas regarding expectations from the research to be conducted must be drawn. 18 Then, the variables to be manipulated (independent) and influenced (dependent) are defined. 4 Thereafter, the hypothesis is stated and refined, and reproducible data tailored to the hypothesis are identified, collected, and analyzed. 4 The hypotheses must be testable and specific, 18 and should describe the variables and their relationships, the specific group being studied, and the predicted research outcome. 18 Hypotheses construction involves a testable proposition to be deduced from theory, and independent and dependent variables to be separated and measured separately. 3 Therefore, good hypotheses must be based on good research questions constructed at the start of a study or trial. 12

In summary, research questions are constructed after establishing the background of the study. Hypotheses are then developed based on the research questions. Thus, it is crucial to have excellent research questions to generate superior hypotheses. In turn, these would determine the research objectives and the design of the study, and ultimately, the outcome of the research. 12 Algorithms for building research questions and hypotheses are shown in Fig. 2 for quantitative research and in Fig. 3 for qualitative research.

EXAMPLES OF RESEARCH QUESTIONS FROM PUBLISHED ARTICLES

• EXAMPLE 1. Descriptive research question (quantitative research)
• - Presents research variables to be assessed (distinct phenotypes and subphenotypes)
• “BACKGROUND: Since COVID-19 was identified, its clinical and biological heterogeneity has been recognized. Identifying COVID-19 phenotypes might help guide basic, clinical, and translational research efforts.
• RESEARCH QUESTION: Does the clinical spectrum of patients with COVID-19 contain distinct phenotypes and subphenotypes? ” 19
• EXAMPLE 2. Relationship research question (quantitative research)
• - Shows interactions between dependent variable (static postural control) and independent variable (peripheral visual field loss)
• “Background: Integration of visual, vestibular, and proprioceptive sensations contributes to postural control. People with peripheral visual field loss have serious postural instability. However, the directional specificity of postural stability and sensory reweighting caused by gradual peripheral visual field loss remain unclear.
• Research question: What are the effects of peripheral visual field loss on static postural control ?” 20
• EXAMPLE 3. Comparative research question (quantitative research)
• - Clarifies the difference among groups with an outcome variable (patients enrolled in COMPERA with moderate PH or severe PH in COPD) and another group without the outcome variable (patients with idiopathic pulmonary arterial hypertension (IPAH))
• “BACKGROUND: Pulmonary hypertension (PH) in COPD is a poorly investigated clinical condition.
• RESEARCH QUESTION: Which factors determine the outcome of PH in COPD?
• STUDY DESIGN AND METHODS: We analyzed the characteristics and outcome of patients enrolled in the Comparative, Prospective Registry of Newly Initiated Therapies for Pulmonary Hypertension (COMPERA) with moderate or severe PH in COPD as defined during the 6th PH World Symposium who received medical therapy for PH and compared them with patients with idiopathic pulmonary arterial hypertension (IPAH) .” 21
• EXAMPLE 4. Exploratory research question (qualitative research)
• - Explores areas that have not been fully investigated (perspectives of families and children who receive care in clinic-based child obesity treatment) to have a deeper understanding of the research problem
• “Problem: Interventions for children with obesity lead to only modest improvements in BMI and long-term outcomes, and data are limited on the perspectives of families of children with obesity in clinic-based treatment. This scoping review seeks to answer the question: What is known about the perspectives of families and children who receive care in clinic-based child obesity treatment? This review aims to explore the scope of perspectives reported by families of children with obesity who have received individualized outpatient clinic-based obesity treatment.” 22
• EXAMPLE 5. Relationship research question (quantitative research)
• - Defines interactions between dependent variable (use of ankle strategies) and independent variable (changes in muscle tone)
• “Background: To maintain an upright standing posture against external disturbances, the human body mainly employs two types of postural control strategies: “ankle strategy” and “hip strategy.” While it has been reported that the magnitude of the disturbance alters the use of postural control strategies, it has not been elucidated how the level of muscle tone, one of the crucial parameters of bodily function, determines the use of each strategy. We have previously confirmed using forward dynamics simulations of human musculoskeletal models that an increased muscle tone promotes the use of ankle strategies. The objective of the present study was to experimentally evaluate a hypothesis: an increased muscle tone promotes the use of ankle strategies. Research question: Do changes in the muscle tone affect the use of ankle strategies ?” 23

EXAMPLES OF HYPOTHESES IN PUBLISHED ARTICLES

• EXAMPLE 1. Working hypothesis (quantitative research)
• - A hypothesis that is initially accepted for further research to produce a feasible theory
• “As fever may have benefit in shortening the duration of viral illness, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response when taken during the early stages of COVID-19 illness .” 24
• “In conclusion, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response . The difference in perceived safety of these agents in COVID-19 illness could be related to the more potent efficacy to reduce fever with ibuprofen compared to acetaminophen. Compelling data on the benefit of fever warrant further research and review to determine when to treat or withhold ibuprofen for early stage fever for COVID-19 and other related viral illnesses .” 24
• EXAMPLE 2. Exploratory hypothesis (qualitative research)
• - Explores particular areas deeper to clarify subjective experience and develop a formal hypothesis potentially testable in a future quantitative approach
• “We hypothesized that when thinking about a past experience of help-seeking, a self distancing prompt would cause increased help-seeking intentions and more favorable help-seeking outcome expectations .” 25
• “Conclusion
• Although a priori hypotheses were not supported, further research is warranted as results indicate the potential for using self-distancing approaches to increasing help-seeking among some people with depressive symptomatology.” 25
• EXAMPLE 3. Hypothesis-generating research to establish a framework for hypothesis testing (qualitative research)
• “We hypothesize that compassionate care is beneficial for patients (better outcomes), healthcare systems and payers (lower costs), and healthcare providers (lower burnout). ” 26
• Compassionomics is the branch of knowledge and scientific study of the effects of compassionate healthcare. Our main hypotheses are that compassionate healthcare is beneficial for (1) patients, by improving clinical outcomes, (2) healthcare systems and payers, by supporting financial sustainability, and (3) HCPs, by lowering burnout and promoting resilience and well-being. The purpose of this paper is to establish a scientific framework for testing the hypotheses above . If these hypotheses are confirmed through rigorous research, compassionomics will belong in the science of evidence-based medicine, with major implications for all healthcare domains.” 26
• EXAMPLE 4. Statistical hypothesis (quantitative research)
• - An assumption is made about the relationship among several population characteristics ( gender differences in sociodemographic and clinical characteristics of adults with ADHD ). Validity is tested by statistical experiment or analysis ( chi-square test, Students t-test, and logistic regression analysis)
• “Our research investigated gender differences in sociodemographic and clinical characteristics of adults with ADHD in a Japanese clinical sample. Due to unique Japanese cultural ideals and expectations of women's behavior that are in opposition to ADHD symptoms, we hypothesized that women with ADHD experience more difficulties and present more dysfunctions than men . We tested the following hypotheses: first, women with ADHD have more comorbidities than men with ADHD; second, women with ADHD experience more social hardships than men, such as having less full-time employment and being more likely to be divorced.” 27
• “Statistical Analysis
• ( text omitted ) Between-gender comparisons were made using the chi-squared test for categorical variables and Students t-test for continuous variables…( text omitted ). A logistic regression analysis was performed for employment status, marital status, and comorbidity to evaluate the independent effects of gender on these dependent variables.” 27

EXAMPLES OF HYPOTHESIS AS WRITTEN IN PUBLISHED ARTICLES IN RELATION TO OTHER PARTS

• EXAMPLE 1. Background, hypotheses, and aims are provided
• “Pregnant women need skilled care during pregnancy and childbirth, but that skilled care is often delayed in some countries …( text omitted ). The focused antenatal care (FANC) model of WHO recommends that nurses provide information or counseling to all pregnant women …( text omitted ). Job aids are visual support materials that provide the right kind of information using graphics and words in a simple and yet effective manner. When nurses are not highly trained or have many work details to attend to, these job aids can serve as a content reminder for the nurses and can be used for educating their patients (Jennings, Yebadokpo, Affo, & Agbogbe, 2010) ( text omitted ). Importantly, additional evidence is needed to confirm how job aids can further improve the quality of ANC counseling by health workers in maternal care …( text omitted )” 28
• “ This has led us to hypothesize that the quality of ANC counseling would be better if supported by job aids. Consequently, a better quality of ANC counseling is expected to produce higher levels of awareness concerning the danger signs of pregnancy and a more favorable impression of the caring behavior of nurses .” 28
• “This study aimed to examine the differences in the responses of pregnant women to a job aid-supported intervention during ANC visit in terms of 1) their understanding of the danger signs of pregnancy and 2) their impression of the caring behaviors of nurses to pregnant women in rural Tanzania.” 28
• EXAMPLE 2. Background, hypotheses, and aims are provided
• “We conducted a two-arm randomized controlled trial (RCT) to evaluate and compare changes in salivary cortisol and oxytocin levels of first-time pregnant women between experimental and control groups. The women in the experimental group touched and held an infant for 30 min (experimental intervention protocol), whereas those in the control group watched a DVD movie of an infant (control intervention protocol). The primary outcome was salivary cortisol level and the secondary outcome was salivary oxytocin level.” 29
• “ We hypothesize that at 30 min after touching and holding an infant, the salivary cortisol level will significantly decrease and the salivary oxytocin level will increase in the experimental group compared with the control group .” 29
• EXAMPLE 3. Background, aim, and hypothesis are provided
• “In countries where the maternal mortality ratio remains high, antenatal education to increase Birth Preparedness and Complication Readiness (BPCR) is considered one of the top priorities [1]. BPCR includes birth plans during the antenatal period, such as the birthplace, birth attendant, transportation, health facility for complications, expenses, and birth materials, as well as family coordination to achieve such birth plans. In Tanzania, although increasing, only about half of all pregnant women attend an antenatal clinic more than four times [4]. Moreover, the information provided during antenatal care (ANC) is insufficient. In the resource-poor settings, antenatal group education is a potential approach because of the limited time for individual counseling at antenatal clinics.” 30
• “This study aimed to evaluate an antenatal group education program among pregnant women and their families with respect to birth-preparedness and maternal and infant outcomes in rural villages of Tanzania.” 30
• “ The study hypothesis was if Tanzanian pregnant women and their families received a family-oriented antenatal group education, they would (1) have a higher level of BPCR, (2) attend antenatal clinic four or more times, (3) give birth in a health facility, (4) have less complications of women at birth, and (5) have less complications and deaths of infants than those who did not receive the education .” 30

Research questions and hypotheses are crucial components to any type of research, whether quantitative or qualitative. These questions should be developed at the very beginning of the study. Excellent research questions lead to superior hypotheses, which, like a compass, set the direction of research, and can often determine the successful conduct of the study. Many research studies have floundered because the development of research questions and subsequent hypotheses was not given the thought and meticulous attention needed. The development of research questions and hypotheses is an iterative process based on extensive knowledge of the literature and insightful grasp of the knowledge gap. Focused, concise, and specific research questions provide a strong foundation for constructing hypotheses which serve as formal predictions about the research outcomes. Research questions and hypotheses are crucial elements of research that should not be overlooked. They should be carefully thought of and constructed when planning research. This avoids unethical studies and poor outcomes by defining well-founded objectives that determine the design, course, and outcome of the study.

Disclosure: The authors have no potential conflicts of interest to disclose.

Author Contributions:

• Conceptualization: Barroga E, Matanguihan GJ.
• Methodology: Barroga E, Matanguihan GJ.
• Writing - original draft: Barroga E, Matanguihan GJ.
• Writing - review & editing: Barroga E, Matanguihan GJ.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts

Latest science news, discoveries and analysis

The Maldives is racing to create new land. Why are so many people concerned?

Mini-colon and brain 'organoids' shed light on cancer and other diseases

Retractions are part of science, but misconduct isn’t — lessons from a superconductivity lab

Monkeypox virus: dangerous strain gains ability to spread through sex, new data suggest

Dna from ancient graves reveals the culture of a mysterious nomadic people, atomic clock keeps ultra-precise time aboard a rocking naval ship, who redefines airborne transmission: what does that mean for future pandemics, ecologists: don’t lose touch with the joy of fieldwork chris mantegna, european ruling linking climate change to human rights could be a game changer — here’s how charlotte e. blattner.

Lethal AI weapons are here: how can we control them?

Living on Mars would probably suck — here's why

Dozens of genes are linked to post-traumatic stress disorder

What toilets can reveal about COVID, cancer and other health threats

How gliding marsupials got their ‘wings’, plastic pollution: three numbers that support a crackdown, first glowing animals lit up the oceans half a billion years ago, how to freeze a memory: putting worms on ice stops them forgetting.

Any plan to make smoking obsolete is the right step

Will AI accelerate or delay the race to net-zero emissions?

Citizenship privilege harms science

We must protect the global plastics treaty from corporate interference martin wagner, un plastics treaty: don’t let lobbyists drown out researchers, current issue.

Surprise hybrid origins of a butterfly species

Stripped-envelope supernova light curves argue for central engine activity, optical clocks at sea, research analysis.

A chemical method for selective labelling of the key amino acid tryptophan

Charles Darwin investigates: the curious case of primrose punishment

Nanoparticle fix opens up tricky technique to forensic applications

Coupled neural activity controls working memory in humans

Robust optical clocks promise stable timing in a portable package, targeting rna opens therapeutic avenues for timothy syndrome, bioengineered ‘mini-colons’ shed light on cancer progression, ancient dna traces family lines and political shifts in the avar empire.

Breaking ice, and helicopter drops: winning photos of working scientists

Shrouded in secrecy: how science is harmed by the bullying and harassment rumour mill

Londoners see what a scientist looks like up close in 50 photographs

How ground glass might save crops from drought on a caribbean island, deadly diseases and inflatable suits: how i found my niche in virology research, books & culture.

How volcanoes shaped our planet — and why we need to be ready for the next big eruption

Dogwhistles, drilling and the roots of Western civilization: Books in brief

Cosmic rentals

Las boriqueñas remembers the forgotten puerto rican women who tested the first pill, dad always mows on summer saturday mornings, nature podcast.

Latest videos

Nature briefing.

An essential round-up of science news, opinion and analysis, delivered to your inbox every weekday.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Maintenance work is planned for Wednesday 1st May 2024 from 9:00am to 11:00am (BST).

During this time, the performance of our website may be affected - searches may run slowly and some pages may be temporarily unavailable. If this happens, please try refreshing your web browser or try waiting two to three minutes before trying again.

We apologise for any inconvenience this might cause and thank you for your patience.

Solution-Processed Colloidal Quantum Dots for Internet of Things

Colloidal quantum dots (CQDs) have been a hot research topic ever since they were successfully fabricated in 1993 via the hot injection method. The Nobel Prize in Chemistry 2023 was awarded to Moungi G. Bawendi, Louis E. Brus and Aleksey Yekimov for the discovery and synthesis of quantum dots. The Internet of Things (IoT) has also attracted a lot of attention due to the technological advancements and digitalisation of the world. This review first aims to give the basics behind QD physics. After, the history behind CQD synthesis and the different methods used to synthesize most widely researched CQD materials (CdSe, PbS and InP) are revisited. A brief introduction to what IoT is and how it works is also mentioned. Then, the most widely researched CQD devices that can be used for the main IoT components are reviewed, where the history, physics, the figure of merits (FoMs) and the state-of-the-art are discussed. Finally, the challenges and different methods for integrating CQDs into IoT devices are discussed, mentioning the future possibilities that await CQDs.

• This article is part of the themed collection: Recent Review Articles

Article information

Download Citation

Permissions.

D. M. Othman, J. Weinstein, N. Huang, W. Ming, Q. Lyu and B. Hou, Nanoscale , 2024, Accepted Manuscript , DOI: 10.1039/D4NR00203B

This article is licensed under a Creative Commons Attribution 3.0 Unported Licence . You can use material from this article in other publications without requesting further permissions from the RSC, provided that the correct acknowledgement is given.

Read more about how to correctly acknowledge RSC content .

Social activity

Search articles by author.

This article has not yet been cited.

Advertisements

Numbers, Facts and Trends Shaping Your World

Read our research on:

Full Topic List

Regions & Countries

• Publications
• Our Methods
• Short Reads
• Tools & Resources

Read Our Research On:

Americans are divided on whether society overlooks racial discrimination or sees it where it doesn’t exist

Ahead of the 60th anniversary of the March on Washington for Black Americans’ civil rights, we asked U.S. adults what they think is the bigger problem when it comes to racial discrimination in the country today.

Pew Research Center conducted this analysis to explore how Americans view racial discrimination in the United States today compared with previous years. This question is a part of a broader study that asked Americans about their views on the Black Lives Matter movement and Martin Luther King Jr.’s legacy .

This analysis is based on a survey of 5,073 U.S. adults conducted April 10-16, 2023. Everyone who took part is a member of the Center’s American Trends Panel (ATP), an online survey panel that is recruited through national, random sampling of residential addresses. Address-based sampling ensures that nearly all U.S. adults have a chance of selection. The survey is weighted to be representative of the U.S. adult population by gender, race, ethnicity, partisan affiliation, education and other categories. Read more about the ATP’s methodology .

Here are the questions used for this analysis , along with responses, and the survey methodology .

• 53% say people not seeing racial discrimination where it really does exist is the bigger problem.
• 45% point to people seeing racial discrimination where it really doesn’t exist as the larger issue.

Views on this have changed in recent years, according to Pew Research Center surveys. In 2019, 57% said people overlooking racial discrimination was the bigger problem, while 42% pointed to people seeing it where it really didn’t exist. That gap has narrowed from 15 to 8 percentage points.

Americans’ current views on this question differ greatly by:

• Race and ethnicity: 88% of Black adults say people overlooking discrimination is the bigger problem. Smaller majorities of Asian (66%) and Hispanic (58%) adults say the same, as do 45% of White adults.

• Partisanship: 80% of Democrats and Democratic-leaning independents say people not seeing racial discrimination where it does exist is the larger issue. About three-quarters (74%) of Republicans and Republican leaners give the opposite answer.

How views on racial discrimination differ within political parties

Majorities of Republicans across age groups say people seeing racial discrimination where it doesn’t exist is the larger issue. But Republicans ages 50 and older are more likely than those under 50 to say this (78% vs. 68%).

Among Democrats, age differences aren’t as large, but there are differences by race and ethnicity. Hispanic Democrats are the most likely to say people seeing discrimination where it doesn’t exist is the bigger problem. Some 29% say this, compared with 20% of Asian Democrats, 19% of White Democrats and 8% of Black Democrats.

Note: Here are the questions used for this analysis , along with responses, and the survey methodology .

• Black Americans
• Discrimination & Prejudice
• Partisanship & Issues
• Racial Bias & Discrimination

Kiley Hurst is a research analyst focusing on social and demographic research at Pew Research Center

A look at Black-owned businesses in the U.S.

8 facts about black americans and the news, black americans’ views on success in the u.s., among black adults, those with higher incomes are most likely to say they are happy, fewer than half of black americans say the news often covers the issues that are important to them, most popular.

1615 L St. NW, Suite 800 Washington, DC 20036 USA (+1) 202-419-4300 | Main (+1) 202-857-8562 | Fax (+1) 202-419-4372 |  Media Inquiries

Research Topics

• Age & Generations
• Coronavirus (COVID-19)
• Economy & Work
• Family & Relationships
• Gender & LGBTQ
• Immigration & Migration
• International Affairs
• Internet & Technology
• Methodological Research
• News Habits & Media
• Non-U.S. Governments
• Other Topics
• Politics & Policy
• Race & Ethnicity
• Email Newsletters

ABOUT PEW RESEARCH CENTER  Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of  The Pew Charitable Trusts .

Copyright 2024 Pew Research Center

Terms & Conditions

Privacy Policy

Cookie Settings

Reprints, Permissions & Use Policy

This paper is in the following e-collection/theme issue:

Published on 23.4.2024 in Vol 26 (2024)

Electronic Media Use and Sleep Quality: Updated Systematic Review and Meta-Analysis

Authors of this article:

• Xiaoning Han * , PhD   ; 
• Enze Zhou * , MA   ; 
• Dong Liu * , PhD  

School of Journalism and Communication, Renmin University of China, Beijing, China

*all authors contributed equally

Corresponding Author:

Dong Liu, PhD

School of Journalism and Communication

Renmin University of China

No. 59 Zhongguancun Street, Haidian District

Beijing, 100872

Phone: 86 13693388506

Email: [email protected]

Background: This paper explores the widely discussed relationship between electronic media use and sleep quality, indicating negative effects due to various factors. However, existing meta-analyses on the topic have some limitations.

Objective: The study aims to analyze and compare the impacts of different digital media types, such as smartphones, online games, and social media, on sleep quality.

Methods: Adhering to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, the study performed a systematic meta-analysis of literature across multiple databases, including Web of Science, MEDLINE, PsycINFO, PubMed, Science Direct, Scopus, and Google Scholar, from January 2018 to October 2023. Two trained coders coded the study characteristics independently. The effect sizes were calculated using the correlation coefficient as a standardized measure of the relationship between electronic media use and sleep quality across studies. The Comprehensive Meta-Analysis software (version 3.0) was used to perform the meta-analysis. Statistical methods such as funnel plots were used to assess the presence of asymmetry and a p -curve test to test the p -hacking problem, which can indicate publication bias.

Results: Following a thorough screening process, the study involved 55 papers (56 items) with 41,716 participants from over 20 countries, classifying electronic media use into “general use” and “problematic use.” The meta-analysis revealed that electronic media use was significantly linked with decreased sleep quality and increased sleep problems with varying effect sizes across subgroups. A significant cultural difference was also observed in these effects. General use was associated with a significant decrease in sleep quality ( P <.001). The pooled effect size was 0.28 (95% CI 0.21-0.35; k =20). Problematic use was associated with a significant increase in sleep problems ( P ≤.001). The pooled effect size was 0.33 (95% CI 0.28-0.38; k =36). The subgroup analysis indicated that the effect of general smartphone use and sleep problems was r =0.33 (95% CI 0.27-0.40), which was the highest among the general group. The effect of problematic internet use and sleep problems was r =0.51 (95% CI 0.43-0.59), which was the highest among the problematic groups. There were significant differences among these subgroups (general: Q between =14.46, P =.001; problematic: Q between =27.37, P <.001). The results of the meta-regression analysis using age, gender, and culture as moderators indicated that only cultural difference in the relationship between Eastern and Western culture was significant ( Q between =6.69; P =.01). All funnel plots and p -curve analyses showed no evidence of publication and selection bias.

Conclusions: Despite some variability, the study overall confirms the correlation between increased electronic media use and poorer sleep outcomes, which is notably more significant in Eastern cultures.

Introduction

Sleep is vital to our health. Research has shown that high sleep quality can lead to improvements in a series of health outcomes, such as an improved immune system, better mood and mental health, enhanced physical performance, lower risk of chronic diseases, and a longer life span [ 1 - 5 ].

Electronic media refers to forms of media or communication that use electronic devices or technology to create, distribute, and display content. This can include various forms of digital media such as smartphones, tablets, instant messaging, phone calls, social media, online games, short video platforms, etc. Electronic media has permeated every aspect of our lives [ 6 ]. Many prefer to use smartphones or tablets before sleep, which can negatively affect sleep in many aspects, including delayed sleep onset, disrupted sleep patterns, shortened sleep duration, and poor sleep quality [ 7 - 10 ]. Furthermore, problematic use occurs when the behavior surpasses a certain limit. In this study, problematic use of electronic media is not solely determined by the amount of time spent on these platforms, but rather by behavioral indicators that suggest an unhealthy or harmful relationship with them.

Smartphones or tablet use can affect sleep quality in many ways. At first, the use of these devices may directly displace, delay, or interrupt sleep time, resulting in inadequate sleep quantity [ 11 ]. The sound of notifications and vibrations of these devices may interrupt sleep. Second, the screens of smartphones and tablets emit blue light, which can suppress the production of melatonin, the hormone responsible for regulating sleep-wake cycles [ 12 ]. Third, consuming emotionally charged content, such as news, suspenseful movies, or engaging in online arguments, can increase emotional arousal, making it harder to relax and fall asleep. This emotional arousal can also lead to disrupted sleep and nightmares [ 13 ]. Finally, the use of electronic devices before bedtime can lead to a delay in bedtime and a shortened sleep duration, as individuals may lose track of time while engaging with their devices. This can result in a disrupted sleep routine and decreased sleep quality [ 14 ].

Some studies have conducted meta-analyses on screen media use and sleep outcomes in 2016, 2019, and 2021 [ 15 - 17 ]. However, these studies had their own limitations. First, the sample size included in their meta-analyses was small (around 10). Second, these studies only focused on 1 aspect of the effect of digital media on sleep quality. For example, Carter et al [ 16 ] focused only on adolescents, and both Alimoradi et al [ 15 ] and Kristensen et al [ 17 ] only reviewed the relationship between problematic use of digital media or devices and sleep quality. Despite of the high heterogeneity found in the meta-analyses, none have compared the effects of different digital media or devices. This study aims to clarify and compare the effects of these different channels.

Literature Search

The research adhered to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines ( Multimedia Appendix 1 ) and followed a predetermined protocol [ 18 , 19 ]. As the idea and scope of this study evolved over time, the meta-analysis was not preregistered. However, the methodology was defined a priori and strictly followed to reduce biases, and the possible influence of post hoc decisions was minimized. All relevant studies in English, published from January 1, 2018, to October 9, 2023, were searched. We searched the following databases: Web of Science, MEDLINE, PsycINFO, PubMed, Science Direct, Scopus, and Google Scholar. The abstracts were examined manually. The keywords used to search were the combination of the following words: “sleep” OR “sleep duration” OR “sleep quality” OR “sleep problems” AND “electronic media” OR “smartphone” OR “tablet” OR “social media” OR “Facebook” OR “Twitter” OR “online gaming” OR “internet” OR “addiction” OR “problematic” ( Multimedia Appendix 2 ). Additionally, the reference lists of relevant studies were examined.

Two trained coders independently screened the titles and abstracts of the identified papers for eligibility, followed by a full-text review of the selected studies. Discrepancies between the coders were resolved through discussion until a consensus was reached. The reference lists of the included studies were also manually screened to identify any additional relevant studies. Through this rigorous process, we ensured a comprehensive and replicable literature search that could contribute to the robustness of our meta-analysis findings.

Inclusion or Exclusion Criteria

Titles and abstracts from search results were scrutinized for relevance, with duplicates removed. Full texts of pertinent papers were obtained, and their eligibility for inclusion was evaluated. We mainly included correlational studies that used both continuous measures of time spent using electronic media use and sleep quality. Studies must have been available in English. Four criteria were used to screen studies: (1) only peer-reviewed empirical studies, published in English, were considered for inclusion in the meta-analysis; (2) the studies should report quantitative statistics on electronic media use and sleep quality, including sample size and essential information to calculate the effect size, and review papers, qualitative studies, case studies, and conference abstracts were excluded; (3) studies on both general use and problematic use of electronic media or devices should be included; and (4) only studies that used correlation, regression, or odds ratio were included to ensure consistency.

Study Coding

Two trained coders were used to code the characteristics of the studies independently. Discrepancies were discussed with the first author of the paper to resolve. Sample size and characteristics of participants were coded: country, female ratio, average age, publication year, and electronic types. Effect sizes were either extracted directly from the original publications or manually calculated. If a study reported multiple dependent effects, the effects were merged into one. If a study reported multiple independent effects from different samples, the effects were included separately. Additionally, to evaluate the study quality, the papers were classified into 3 tiers (high, middle, and low) according to Journal Citation Reports 2022 , a ranking of journals based on their impact factor as reported in the Web of Science. The few unindexed papers were rated based on their citation counts as reported in Google Scholar.

Meta-Analysis and Moderator Analyses

The effect sizes were calculated using the correlation coefficient ( r ) as a standardized measure of the relationship between electronic media or device use and sleep quality across studies. When studies reported multiple effect sizes, we selected the one that best represented the overall association between electronic media use and sleep quality. If studies did not provide correlation coefficients, we converted other reported statistics (eg, standardized regression coefficients) into correlation coefficients using established formulas. Once calculated, the correlation coefficients were transformed into Fisher z scores to stabilize the variance and normalize the distribution.

Previous meta-studies have shown high levels of heterogeneity. Hence, the random effects model was adopted for all analyses. To explore potential factors contributing to the heterogeneity and to further understand the relationship between electronic media use and sleep quality, we conducted moderator analyses. The following categorical and continuous moderators were examined: media types (online gaming, social media, smartphone, or intent), participants’ average age, culture, female ratio, and sleep quality assessment method. For categorical moderators, subgroup analyses were performed, while for continuous moderators, meta-regression analyses were conducted. All analyses were completed in the Comprehensive Meta-Analysis software (version 3.0; Biostat, Inc).

Publication Bias

We used statistical methods such as funnel plots to assess the presence of asymmetry and a p -curve test to test the p -hacking problem, which may indicate publication bias. In case of detected asymmetry, we applied techniques such as the trim-and-fill method to adjust the effect size estimates.

By addressing publication bias, we aimed to provide a more accurate and reliable synthesis of the available evidence, enhancing the validity and generalizability of our meta-analytic findings. Nevertheless, it is essential for readers to interpret the results cautiously, considering the potential limitations imposed by publication bias and other methodological concerns.

Search Findings

A total of 98,806 studies were identified from databases, especially Scopus (n=49,643), Google Scholar (n=18,600), Science Direct (n=15,084), and Web of Science (n=11,689). Upon removing duplicate records and excluding studies that did not meet the inclusion criteria, 754 studies remained for the screening phase. After screening titles, abstracts, and full texts, 703 studies were excluded. A total of 4 additional studies were identified from the references of relevant reviews. Finally, 55 studies [ 20 - 74 ] were included in the meta-analysis. The flow diagram of the selection is shown in Figure 1 .

Characteristics of Included Studies

In 20 studies, 21,594 participants were included in the analysis of the general use of electronic media and sleep quality. The average age of the sample ranged from 9.9 to 44 years. The category of general online gaming and sleep quality included 4 studies, with 14,837 participants; the category of general smartphone use and sleep quality included 10 studies, with 5011 participants; and the category of general social media use and sleep quality included 6 studies, with 1746 participants.

These studies came from the following countries or areas: Germany, Serbia, Indonesia, India, China, Italy, Saudi Arabia, New Zealand, the United Kingdom, the United States, Spain, Qatar, Egypt, Argentina, and Portugal. The most frequently used measure of electronic media use was the time spent on it. The most frequently used measure of sleep was the Pittsburgh Sleep Quality Index.

In 35 studies, 20,122 participants were included in the analysis of the problematic use of electronic media and sleep quality. The average age of the sample ranged from 14.76 to 65.62 years. The category of problematic online gaming and sleep quality included 5 studies, with 1874 participants; the category of problematic internet use and sleep quality included 2 studies, with 774 participants; the category of problematic smartphone use and sleep quality included 18 studies, with 12,204 participants; and the category of problematic social media use and sleep quality included 11 studies, with 5270 participants. There was a study that focused on both social media and online gaming, which led to its inclusion in the analysis. These studies came from 14 countries or areas: Turkey, the United States, Indonesia, China, France, Taiwan, India, South Korea, Hong Kong, Iran, Poland, Israel, Hungary, and Saudi Arabia. The most frequently used measures of problematic electronic media use were the Internet Gaming Disorder Scale-Short Form, Smartphone Addiction Scale-Short Form, and Bergen Social Media Addiction Scale.

With respect to study quality, the 56 papers were published in 50 journals, 41 of which were indexed in Journal Citation Reports 2022 , while the remaining 9 journals were rated based on their citation counts as reported in Google Scholar. As a result, of the 56 papers included in the study, 22 papers were assigned a high rating, 18 papers were assigned a middle rating, and 16 papers were assigned a low rating. More information about the included studies is listed in Multimedia Appendix 3 [ 20 - 74 ].

Meta-Analysis

The results of the meta-analysis of the relationship between general electronic media use and sleep quality showed that electronic media use was associated with a significant decrease in sleep quality ( P <.001). The pooled effect size was 0.28 (95% CI 0.21-0.35; k =20), indicating that individuals who used electronic media more frequently were generally associated with more sleeping problems.

The second meta-analysis showed that problematic electronic media use was associated with a significant increase in sleep problems ( P ≤.001). The pooled effect size was 0.33 (95% CI 0.28-0.38; k =36), indicating that participants who used electronic media more frequently were more likely to have more sleep problems.

Moderator Analyses

At first, we conducted subgroup analyses for different media or devices. The results are shown in Tables 1 and 2 . The effect of the relationship between general online gaming and sleep problems was r =0.14 (95% CI 0.06-0.22); the effect of the relationship between general smartphone use and sleep problems was r =0.33 (95% CI 0.27-0.40); and the effect of the relationship between general social media use and sleep problems was r =0.28 (95% CI 0.21-0.34). There are significant differences among these groups ( Q between =14.46; P =.001).

The effect of the relationship between problematic gaming and sleep problems was r =0.49, 95% CI 0.23-0.69; the effect of the relationship between problematic internet use and sleep problems was r =0.51 (95% CI 0.43-0.59); the effect of the relationship between problematic smartphone use and sleep problems was r =0.25 (95% CI 0.20-0.30); and the effect of the relationship between problematic social media use and sleep problems was r =0.35 (95% CI 0.29-0.40). There are significant differences among these groups ( Q between =27.37; P <.001).

We also used age, gender, and culture as moderators to conduct meta-regression analyses. The results are shown in Tables 3 and 4 . Only cultural difference in the relationship between Eastern and Western culture was significant ( Q between =6.694; P =.01). All other analyses were not significant.

a Not applicable.

All funnel plots of the analyses were symmetrical, showing no evidence of publication bias ( Figures 2 - 5 ). We also conducted p -curve analyses to see whether there were any selection biases. The results also showed that there were no biases.

Principal Findings

This study indicated that electronic media use was significantly linked with decreased sleep quality and increased sleep problems with varying effect sizes across subgroups. General use was associated with a significant decrease in sleep quality. Problematic use was associated with a significant increase in sleep problems. A significant cultural difference was also observed by the meta-regression analysis.

First, there is a distinction in the impact on sleep quality between problematic use and general use, with the former exhibiting a higher correlation strength. However, both have a positive correlation, suggesting that the deeper the level of use, the more sleep-related issues are observed. In addressing this research question, the way in which electronic media use is conceptualized and operationalized may have a bearing on the ultimate outcomes. Problematic use is measured through addiction scales, while general use is predominantly assessed by duration of use (time), leading to divergent results stemming from these distinct approaches. The key takeaway is that each measurement possesses unique strengths and weaknesses, and the pathways affecting sleep quality differ. Consequently, the selection of a measurement approach should be tailored to the specific research question at hand. The duration of general use reflects an individual’s comprehensive involvement with electronic media, and its impact on sleep quality is evident in factors such as an extended time to fall asleep and reduced sleep duration. The addiction scale for problematic use illuminates an individual’s preferences, dependencies, and other associations with electronic media. Its impact on sleep quality is evident through physiological and psychological responses, including anxiety, stress, and emotional reactions.

Second, notable variations exist in how different types of electronic media affect sleep quality. In general, the positive predictive effects of smartphone, social media, and online gaming use durations on sleep problems gradually decrease. In the problematic context, the intensity of addiction to the internet and online gaming has the most significant positive impact on sleep problems, followed by social media, while smartphones exert the least influence. On one hand, longitudinal comparisons within the same context reveal that the content and format of electronic media can have varying degrees of negative impact on sleep quality, irrespective of whether it involves general or problematic use. On the other hand, cross-context comparisons suggest that both general and problematic use play a role in moderating the impact of electronic media types on sleep quality. As an illustration, problematic use reinforces the positive impact of online gaming and social media on sleep problems, while mitigating the influence of smartphones. Considering smartphones as electronic media, an extended duration of general use is associated with lower sleep quality. However, during problematic use, smartphones serve as the platform for other electronic media such as games and social media, resulting in a weakened predictive effect on sleep quality. Put differently, in the context of problematic use, the specific type of electronic media an individual consumes on their smartphones becomes increasingly pivotal in shaping sleep quality.

Third, cultural differences were found to be significant moderators of the relationship between electronic media use and sleep problems in both our study and Carter et al [ 16 ]. Kristensen et al [ 17 ], however, did not specifically address the role of cultural differences but revealed that there was a strong and consistent association between bedtime media device use and sleep outcomes across the studies included. Our findings showed that the association between problematic social media use was significantly larger in Eastern culture. We speculate that the difference may be attributed to cultural differences in social media use patterns, perceptions of social norms and expectations, variations in bedtime routines and habits, and diverse coping mechanisms for stress. These speculations warrant further investigation to understand better the underlying factors contributing to the observed cultural differences in the relationship between social media use and sleep quality.

Fourth, it was observed that gender and age had no significant impact on sleep quality. The negative effects of electronic media use are not only confined to the sleep quality of adults, and the association with gender differences remains unclear. Recent studies point out that electronic media use among preschoolers may result in a “time-shifting” process, disrupting their sleep patterns [ 75 ]. Similarly, children and adolescent sleep patterns have been reported to be adversely affected by electronic media use [ 76 - 78 ]. These findings underscore the necessity of considering age group variations in future research, as electronic media use may differently impact sleep quality across age demographics.

In conclusion, our study, Carter et al [ 16 ], and Kristensen et al [ 17 ] collectively emphasize the importance of understanding and addressing the negative impact of electronic media use, particularly problematic online gaming and smartphone use, on sleep quality and related issues. Further research is warranted to explore the underlying mechanisms and specific factors contributing to the relationship between electronic media use and sleep problems.

Strengths and Limitations

Our study, supplemented with research by Carter et al [ 16 ] and Kristensen et al [ 17 ], contributes to the growing evidence supporting a connection between electronic media use and sleep quality. We found that both general and problematic use of electronic media correlates with sleep issues, with the strength of the correlation varying based on the type of electronic media and cultural factors, with no significant relationship observed with age or gender.

Despite the vast amount of research on the relationship between electronic media use and sleep, several gaps and limitations still exist.

First, the inclusion criteria were restricted to English-language, peer-reviewed empirical studies published between January 2018 and October 2023. This may have led to the exclusion of relevant studies published in other languages or before 2018, potentially limiting the generalizability of our findings. Furthermore, the exclusion of non–peer-reviewed studies and conference abstracts may have introduced publication bias, as significant results are more likely to be published in peer-reviewed journals.

Second, although we used a comprehensive search strategy, the possibility remains that some relevant studies may have been missed. Additionally, the search strategies were not linked with Medical Subject Headings headers and may not have captured all possible electronic media types, resulting in an incomplete representation of the effects of electronic media use on sleep quality.

Third, the studies included in our meta-analysis exhibited considerable heterogeneity in sample characteristics, electronic media types, and measures of sleep quality. This heterogeneity might have contributed to the variability in effect sizes observed across studies. Although we conducted moderator analyses to explore potential sources of heterogeneity, other unexamined factors may still have influenced the relationship between electronic media use and sleep quality.

Fourth, our meta-analysis relied on the correlation coefficient ( r ) as the primary effect size measure, which may not fully capture the complex relationships between electronic media use and sleep quality. Moreover, the conversion of other reported statistics into correlation coefficients could introduce additional sources of error. The correlational nature of the included studies limited our ability to draw causal inferences between electronic media use and sleep quality. Experimental and longitudinal research designs would provide stronger evidence for the directionality of this relationship.

Given these limitations, future research should aim to include a more diverse range of studies, examine additional potential moderators, and use more robust research designs to better understand the complex relationship between electronic media use and sleep quality.

Conclusions

In conclusion, our updated meta-analysis affirms the consistent negative impact of electronic media use on sleep outcomes, with problematic online gaming and smartphone use being particularly impactful. Notably, the negative effect of problematic social media use on sleep quality appears more pronounced in Eastern cultures. This research emphasizes the need for public health initiatives to increase awareness of these impacts, particularly for adolescents. Further research, including experimental and longitudinal studies, is necessary to delve deeper into the complex relationship between electronic media use and sleep quality, considering potential moderators like cultural differences.

Acknowledgments

This research was supported by the Journalism and Marxism Research Center, Renmin University of China (MXG202215), and by funds for building world-class universities (disciplines) of Renmin University of China (23RXW195).

A statement on the use of ChatGPT in the process of writing this paper can be found in Multimedia Appendix 4.

Data Availability

The data sets analyzed during this study are available from the corresponding author on reasonable request.

Conflicts of Interest

None declared.

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 checklist.

Search strategies.

Characteristics of included studies.

Large language model statement.

• Brink-Kjaer A, Leary EB, Sun H, Westover MB, Stone KL, Peppard PE, et al. Age estimation from sleep studies using deep learning predicts life expectancy. NPJ Digit Med. 2022;5(1):103. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Killgore WDS. Effects of sleep deprivation on cognition. Prog Brain Res. 2010;185:105-129. [ CrossRef ] [ Medline ]
• Lee S, Mu CX, Wallace ML, Andel R, Almeida DM, Buxton OM, et al. Sleep health composites are associated with the risk of heart disease across sex and race. Sci Rep. 2022;12(1):2023. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Prather AA. Sleep, stress, and immunity. In: Grandner MA, editor. Sleep and Health, 1st Edition. Cambridge. Academic Press; 2019;319-330.
• Scott AJ, Webb TL, Martyn-St James M, Rowse G, Weich S. Improving sleep quality leads to better mental health: a meta-analysis of randomised controlled trials. Sleep Med Rev. Dec 2021;60:101556. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Guttmann A. Statista. 2023. URL: https://www.statista.com/topics/1536/media-use/#topicOverview [accessed 2023-06-10]
• Hysing M, Pallesen S, Stormark KM, Jakobsen R, Lundervold AJ, Sivertsen B. Sleep and use of electronic devices in adolescence: results from a large population-based study. BMJ Open. Feb 02, 2015;5(1):e006748. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lavender RM. Electronic media use and sleep quality. Undergrad J Psychol. 2015;28(1):55-62. [ FREE Full text ]
• Exelmans L, Van den Bulck J. Bedtime mobile phone use and sleep in adults. Soc Sci Med. 2016;148:93-101. [ CrossRef ] [ Medline ]
• Twenge JM, Krizan Z, Hisler G. Decreases in self-reported sleep duration among U.S. adolescents 2009-2015 and association with new media screen time. Sleep Med. 2017;39:47-53. [ CrossRef ] [ Medline ]
• Exelmans L. Electronic media use and sleep: a self-control perspective. Curr Sleep Med Rep. 2019;5:135-140. [ CrossRef ]
• Jniene A, Errguig L, El Hangouche AJ, Rkain H, Aboudrar S, El Ftouh M, et al. Perception of sleep disturbances due to bedtime use of blue light-emitting devices and its impact on habits and sleep quality among young medical students. Biomed Res Int. 2019;2019:7012350. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Munezawa T, Kaneita Y, Osaki Y, Kanda H, Minowa M, Suzuki K, et al. The association between use of mobile phones after lights out and sleep disturbances among Japanese adolescents: a nationwide cross-sectional survey. Sleep. 2011;34(8):1013-1020. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Smith LJ, Gradisar M, King DL, Short M. Intrinsic and extrinsic predictors of video-gaming behaviour and adolescent bedtimes: the relationship between flow states, self-perceived risk-taking, device accessibility, parental regulation of media and bedtime. Sleep Med. 2017;30:64-70. [ CrossRef ] [ Medline ]
• Alimoradi Z, Lin CY, Broström A, Bülow PH, Bajalan Z, Griffiths MD, et al. Internet addiction and sleep problems: a systematic review and meta-analysis. Sleep Med Rev. 2019;47:51-61. [ CrossRef ] [ Medline ]
• Carter B, Rees P, Hale L, Bhattacharjee D, Paradkar MS. Association between portable screen-based media device access or use and sleep outcomes: a systematic review and meta-analysis. JAMA Pediatr. 2016;170(12):1202-1208. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kristensen JH, Pallesen S, King DL, Hysing M, Erevik EK. Problematic gaming and sleep: a systematic review and meta-analysis. Front Psychiatry. 2021;12:675237. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Akçay D, Akçay BD. The effect of computer game playing habits of university students on their sleep states. Perspect Psychiatr Care. 2020;56(4):820-826. [ CrossRef ] [ Medline ]
• Alahdal WM, Alsaedi AA, Garrni AS, Alharbi FS. The impact of smartphone addiction on sleep quality among high school students in Makkah, Saudi Arabia. Cureus. 2023;15(6):e40759. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Alam A, Alshakhsi S, Al-Thani D, Ali R. The role of objectively recorded smartphone usage and personality traits in sleep quality. PeerJ Comput Sci. 2023;9:e1261. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Almeida F, Marques DR, Gomes AA. A preliminary study on the association between social media at night and sleep quality: the relevance of FOMO, cognitive pre-sleep arousal, and maladaptive cognitive emotion regulation. Scand J Psychol. 2023;64(2):123-132. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Alshobaili FA, AlYousefi NA. The effect of smartphone usage at bedtime on sleep quality among Saudi non-medical staff at King Saud University Medical City. J Family Med Prim Care. 2019;8(6):1953-1957. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Alsulami A, Bakhsh D, Baik M, Merdad M, Aboalfaraj N. Assessment of sleep quality and its relationship to social media use among medical students. Med Sci Educ. 2019;29(1):157-161. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Altintas E, Karaca Y, Hullaert T, Tassi P. Sleep quality and video game playing: effect of intensity of video game playing and mental health. Psychiatry Res. 2019;273:487-492. [ CrossRef ] [ Medline ]
• Asbee J, Slavish D, Taylor DJ, Dietch JR. Using a frequentist and Bayesian approach to examine video game usage, substance use, and sleep among college students. J Sleep Res. 2023;32(4):e13844. [ CrossRef ] [ Medline ]
• Bae ES, Kang HS, Lee HN. The mediating effect of sleep quality in the relationship between academic stress and social network service addiction tendency among adolescents. J Korean Acad Community Health Nurs. 2020;31(3):290-299. [ FREE Full text ] [ CrossRef ]
• Chatterjee S, Kar SK. Smartphone addiction and quality of sleep among Indian medical students. Psychiatry. 2021;84(2):182-191. [ CrossRef ] [ Medline ]
• Chung JE, Choi SA, Kim KT, Yee J, Kim JH, Seong JW, et al. Smartphone addiction risk and daytime sleepiness in Korean adolescents. J Paediatr Child Health. 2018;54(7):800-806. [ CrossRef ] [ Medline ]
• Demir YP, Sumer MM. Effects of smartphone overuse on headache, sleep and quality of life in migraine patients. Neurosciences (Riyadh). 2019;24(2):115-121. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Dewi RK, Efendi F, Has EMM, Gunawan J. Adolescents' smartphone use at night, sleep disturbance and depressive symptoms. Int J Adolesc Med Health. 2018;33(2):20180095. [ CrossRef ] [ Medline ]
• Eden A, Ellithorpe ME, Meshi D, Ulusoy E, Grady SM. All night long: problematic media use is differentially associated with sleep quality and depression by medium. Commun Res Rep. 2021;38(3):143-149. [ CrossRef ]
• Ellithorpe ME, Meshi D, Tham SM. Problematic video gaming is associated with poor sleep quality, diet quality, and personal hygiene. Psychol Pop Media. 2023;12(2):248-253. [ CrossRef ]
• Elsheikh AA, Elsharkawy SA, Ahmed DS. Impact of smartphone use at bedtime on sleep quality and academic activities among medical students at Al -Azhar University at Cairo. J Public Health (Berl.). Jun 15, 2023.:1-10. [ FREE Full text ] [ CrossRef ]
• Gaya AR, Brum R, Brites K, Gaya A, de Borba Schneiders L, Duarte Junior MA, et al. Electronic device and social network use and sleep outcomes among adolescents: the EHDLA study. BMC Public Health. 2023;23(1):919. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gezgin DM. Understanding patterns for smartphone addiction: age, sleep duration, social network use and fear of missing out. Cypriot J Educ Sci. 2018;13(2):166-177. [ CrossRef ]
• Graham S, Mason A, Riordan B, Winter T, Scarf D. Taking a break from social media improves wellbeing through sleep quality. Cyberpsychol Behav Soc Netw. 2021;24(6):421-425. [ CrossRef ] [ Medline ]
• Guerrero MD, Barnes JD, Chaput JP, Tremblay MS. Screen time and problem behaviors in children: exploring the mediating role of sleep duration. Int J Behav Nutr Phys Act. 2019;16(1):105. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hamvai C, Kiss H, Vörös H, Fitzpatrick KM, Vargha A, Pikó BF. Association between impulsivity and cognitive capacity decrease is mediated by smartphone addiction, academic procrastination, bedtime procrastination, sleep insufficiency and daytime fatigue among medical students: a path analysis. BMC Med Educ. 2023;23(1):537. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Herlache AD, Lang KM, Krizan Z. Withdrawn and wired: problematic internet use accounts for the link of neurotic withdrawal to sleep disturbances. Sleep Sci. 2018;11(2):69-73. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Huang Q, Li Y, Huang S, Qi J, Shao T, Chen X, et al. Smartphone use and sleep quality in chinese college students: a preliminary study. Front Psychiatry. 2020;11:352. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hussain Z, Griffiths MD. The associations between problematic social networking site use and sleep quality, attention-deficit hyperactivity disorder, depression, anxiety and stress. Int J Ment Health Addict. 2021;19:686-700. [ FREE Full text ] [ CrossRef ]
• Imani V, Ahorsu DK, Taghizadeh N, Parsapour Z, Nejati B, Chen HP, et al. The mediating roles of anxiety, depression, sleepiness, insomnia, and sleep quality in the association between problematic social media use and quality of life among patients with cancer. Healthcare (Basel). 2022;10(9):1745. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jeong CY, Seo YS, Cho EH. The effect of SNS addiction tendency on trait-anxiety and quality of sleep in university students'. J Korean Clin Health Sci. 2018;6(2):1147-1155. [ CrossRef ]
• Karaş H, Küçükparlak İ, Özbek MG, Yılmaz T. Addictive smartphone use in the elderly: relationship with depression, anxiety and sleep quality. Psychogeriatrics. 2023;23(1):116-125. [ CrossRef ] [ Medline ]
• Kater MJ, Schlarb AA. Smartphone usage in adolescents: motives and link to sleep disturbances, stress and sleep reactivity. Somnologie. 2020;24(4):245-252. [ CrossRef ]
• Kharisma AC, Fitryasari R, Rahmawati PD. Online games addiction and the decline in sleep quality of college student gamers in the online game communities in Surabaya, Indonesia. Int J Psychosoc Rehabil. 2020;24(7):8987-8993. [ FREE Full text ] [ CrossRef ]
• Kumar VA, Chandrasekaran V, Brahadeeswari H. Prevalence of smartphone addiction and its effects on sleep quality: a cross-sectional study among medical students. Ind Psychiatry J. 2019;28(1):82-85. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lee Y, Blebea J, Janssen F, Domoff SE. The impact of smartphone and social media use on adolescent sleep quality and mental health during the COVID-19 pandemic. Hum Behav Emerg Technol. 2023;2023:3277040. [ FREE Full text ] [ CrossRef ]
• Li L, Griffiths MD, Mei S, Niu Z. Fear of missing out and smartphone addiction mediates the relationship between positive and negative affect and sleep quality among Chinese university students. Front Psychiatry. 2020;11:877. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Li Y, Mu W, Sun C, Kwok SYCL. Surrounded by smartphones: relationship between peer phubbing, psychological distress, problematic smartphone use, daytime sleepiness, and subjective sleep quality. Appl Res Qual Life. 2023;18:1099-1114. [ CrossRef ]
• Luo X, Hu C. Loneliness and sleep disturbance among first-year college students: the sequential mediating effect of attachment anxiety and mobile social media dependence. Psychol Sch. 2022;59(9):1776-1789. [ CrossRef ]
• Luqman A, Masood A, Shahzad F, Shahbaz M, Feng Y. Untangling the adverse effects of late-night usage of smartphone-based SNS among university students. Behav Inf Technol. 2021;40(15):1671-1687. [ CrossRef ]
• Makhfudli, Aulia A, Pratiwi A. Relationship intensity of social media use with quality of sleep, social interaction, and self-esteem in urban adolescents in Surabaya. Sys Rev Pharm. 2020;11(5):783-788. [ CrossRef ]
• Ozcan B, Acimis NM. Sleep quality in Pamukkale university students and its relationship with smartphone addiction. Pak J Med Sci. 2021;37(1):206-211. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Peltz JS, Bodenlos JS, Kingery JN, Abar C. Psychological processes linking problematic smartphone use to sleep disturbance in young adults. Sleep Health. 2023;9(4):524-531. [ CrossRef ] [ Medline ]
• Pérez-Chada D, Bioch SA, Schönfeld D, Gozal D, Perez-Lloret S, Sleep in Adolescents Collaborative Study Group. Screen use, sleep duration, daytime somnolence, and academic failure in school-aged adolescents. PLoS One. 2023;18(2):e0281379. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Przepiorka A, Blachnio A. The role of Facebook intrusion, depression, and future time perspective in sleep problems among adolescents. J Res Adolesc. 2020;30(2):559-569. [ CrossRef ] [ Medline ]
• Rudolf K, Bickmann P, Froböse I, Tholl C, Wechsler K, Grieben C. Demographics and health behavior of video game and eSports players in Germany: the eSports study 2019. Int J Environ Res Public Health. 2020;17(6):1870. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sami H, Danielle L, Lihi D, Elena S. The effect of sleep disturbances and internet addiction on suicidal ideation among adolescents in the presence of depressive symptoms. Psychiatry Res. 2018;267:327-332. [ CrossRef ] [ Medline ]
• Scott H, Woods HC. Fear of missing out and sleep: cognitive behavioural factors in adolescents' nighttime social media use. J Adolesc. 2018;68:61-65. [ CrossRef ] [ Medline ]
• Spagnoli P, Balducci C, Fabbri M, Molinaro D, Barbato G. Workaholism, intensive smartphone use, and the sleep-wake cycle: a multiple mediation analysis. Int J Environ Res Public Health. 2019;16(19):3517. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Stanković M, Nešić M, Čičević S, Shi Z. Association of smartphone use with depression, anxiety, stress, sleep quality, and internet addiction. empirical evidence from a smartphone application. Pers Individ Differ. Jan 2021;168:110342. [ CrossRef ]
• Tandon A, Kaur P, Dhir A, Mäntymäki M. Sleepless due to social media? investigating problematic sleep due to social media and social media sleep hygiene. Comput Hum Behav. Dec 2020;113:106487. [ FREE Full text ] [ CrossRef ]
• Wang PY, Chen KL, Yang SY, Lin PH. Relationship of sleep quality, smartphone dependence, and health-related behaviors in female junior college students. PLoS One. 2019;14(4):e0214769. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wang Q, Zhong Y, Zhao G, Song R, Zeng C. Relationship among content type of smartphone use, technostress, and sleep difficulty: a study of university students in China. Educ Inf Technol. Aug 02, 2022;28(2):1697-1714. [ CrossRef ]
• Wong HY, Mo HY, Potenza MN, Chan MNM, Lau WM, Chui TK, et al. Relationships between severity of internet gaming disorder, severity of problematic social media use, sleep quality and psychological distress. Int J Environ Res Public Health. 2020;17(6):1879. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Xie X, Dong Y, Wang J. Sleep quality as a mediator of problematic smartphone use and clinical health symptoms. J Behav Addict. 2018;7(2):466-472. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Yang SY, Chen KL, Lin PH, Wang PY. Relationships among health-related behaviors, smartphone dependence, and sleep duration in female junior college students. Soc Health Behav. 2019;2(1):26-31. [ FREE Full text ] [ CrossRef ]
• Yıldırım M, Öztürk A, Solmaz F. Fear of COVID-19 and sleep problems in Turkish young adults: mediating roles of happiness and problematic social networking sites use. Psihologija. 2023;56(4):497-515. [ FREE Full text ] [ CrossRef ]
• Zhai X, Ye M, Wang C, Gu Q, Huang T, Wang K, et al. Associations among physical activity and smartphone use with perceived stress and sleep quality of Chinese college students. Mental Health and Physical Activity. Mar 2020;18:100323. [ CrossRef ]
• Zhang MX, Wu AMS. Effects of smartphone addiction on sleep quality among Chinese university students: the mediating role of self-regulation and bedtime procrastination. Addict Behav. 2020;111:106552. [ CrossRef ] [ Medline ]
• Zhang MX, Zhou H, Yang HM, Wu AMS. The prospective effect of problematic smartphone use and fear of missing out on sleep among Chinese adolescents. Curr Psychol. May 24, 2021;42(7):5297-5305. [ CrossRef ]
• Beyens I, Nathanson AI. Electronic media use and sleep among preschoolers: evidence for time-shifted and less consolidated sleep. Health Commun. 2019;34(5):537-544. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mazurek MO, Engelhardt CR, Hilgard J, Sohl K. Bedtime electronic media use and sleep in children with autism spectrum disorder. J Dev Behav Pediatr. 2016;37(7):525-531. [ CrossRef ] [ Medline ]
• King DL, Delfabbro PH, Zwaans T, Kaptsis D. Sleep interference effects of pathological electronic media use during adolescence. Int J Ment Health Addict. 2014;12:21-35. [ CrossRef ]
• Kubiszewski V, Fontaine R, Rusch E, Hazouard E. Association between electronic media use and sleep habits: an eight-day follow-up study. Int J Adolesc Youth. 2013;19(3):395-407. [ FREE Full text ] [ CrossRef ]

Abbreviations

Edited by G Eysenbach, T Leung; submitted 20.04.23; peer-reviewed by M Behzadifar, F Estévez-López, R Prieto-Moreno; comments to author 18.05.23; revised version received 15.06.23; accepted 26.03.24; published 23.04.24.

©Xiaoning Han, Enze Zhou, Dong Liu. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 23.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.