quantitative research titles with authors

• Write my thesis
• Thesis writers
• Buy thesis papers
• Bachelor thesis
• Master's thesis
• Thesis editing services
• Thesis proofreading services
• Buy a thesis online
• Write my dissertation
• Dissertation proposal help
• Pay for dissertation
• Custom dissertation
• Dissertation help online
• Buy dissertation online
• Cheap dissertation
• Dissertation editing services
• Write my research paper
• Buy research paper online
• Pay for research paper
• Research paper help
• Order research paper
• Custom research paper
• Cheap research paper
• Research papers for sale
• Thesis subjects
• How It Works

100+ Quantitative Research Topics For Students

Quantitative research is a research strategy focusing on quantified data collection and analysis processes. This research strategy emphasizes testing theories on various subjects. It also includes collecting and analyzing non-numerical data.

Quantitative research is a common approach in the natural and social sciences , like marketing, business, sociology, chemistry, biology, economics, and psychology. So, if you are fond of statistics and figures, a quantitative research title would be an excellent option for your research proposal or project.

How to Get a Title of Quantitative Research

How to make quantitative research title, what is the best title for quantitative research, amazing quantitative research topics for students, creative quantitative research topics, perfect quantitative research title examples, unique quantitative research titles, outstanding quantitative research title examples for students, creative example title of quantitative research samples, outstanding quantitative research problems examples, fantastic quantitative research topic examples, the best quantitative research topics, grade 12 quantitative research title for students, list of quantitative research titles for high school, easy quantitative research topics for students, trending topics for quantitative research, quantitative research proposal topics, samples of quantitative research titles, research title about business quantitative.

Finding a great title is the key to writing a great quantitative research proposal or paper. A title for quantitative research prepares you for success, failure, or mediocre grades. This post features examples of quantitative research titles for all students.

Putting together a research title and quantitative research design is not as easy as some students assume. So, an example topic of quantitative research can help you craft your own. However, even with the examples, you may need some guidelines for personalizing your research project or proposal topics.

So, here are some tips for getting a title for quantitative research:

• Consider your area of studies
• Look out for relevant subjects in the area
• Expert advice may come in handy
• Check out some sample quantitative research titles

Making a quantitative research title is easy if you know the qualities of a good title in quantitative research. Reading about how to make a quantitative research title may not help as much as looking at some samples. Looking at a quantitative research example title will give you an idea of where to start.

However, let’s look at some tips for how to make a quantitative research title:

• The title should seem interesting to readers
• Ensure that the title represents the content of the research paper
• Reflect on the tone of the writing in the title
• The title should contain important keywords in your chosen subject to help readers find your paper
• The title should not be too lengthy
• It should be grammatically correct and creative
• It must generate curiosity

An excellent quantitative title should be clear, which implies that it should effectively explain the paper and what readers can expect. A research title for quantitative research is the gateway to your article or proposal. So, it should be well thought out. Additionally, it should give you room for extensive topic research.

A sample of quantitative research titles will give you an idea of what a good title for quantitative research looks like. Here are some examples:

• What is the correlation between inflation rates and unemployment rates?
• Has climate adaptation influenced the mitigation of funds allocation?
• Job satisfaction and employee turnover: What is the link?
• A look at the relationship between poor households and the development of entrepreneurship skills
• Urbanization and economic growth: What is the link between these elements?
• Does education achievement influence people’s economic status?
• What is the impact of solar electricity on the wholesale energy market?
• Debt accumulation and retirement: What is the relationship between these concepts?
• Can people with psychiatric disorders develop independent living skills?
• Children’s nutrition and its impact on cognitive development

Quantitative research applies to various subjects in the natural and social sciences. Therefore, depending on your intended subject, you have numerous options. Below are some good quantitative research topics for students:

• The difference between the colorific intake of men and women in your country
• Top strategies used to measure customer satisfaction and how they work
• Black Friday sales: are they profitable?
• The correlation between estimated target market and practical competitive risk assignment
• Are smartphones making us brighter or dumber?
• Nuclear families Vs. Joint families: Is there a difference?
• What will society look like in the absence of organized religion?
• A comparison between carbohydrate weight loss benefits and high carbohydrate diets?
• How does emotional stability influence your overall well-being?
• The extent of the impact of technology in the communications sector

Creativity is the key to creating a good research topic in quantitative research. Find a good quantitative research topic below:

• How much exercise is good for lasting physical well-being?
• A comparison of the nutritional therapy uses and contemporary medical approaches
• Does sugar intake have a direct impact on diabetes diagnosis?
• Education attainment: Does it influence crime rates in society?
• Is there an actual link between obesity and cancer rates?
• Do kids with siblings have better social skills than those without?
• Computer games and their impact on the young generation
• Has social media marketing taken over conventional marketing strategies?
• The impact of technology development on human relationships and communication
• What is the link between drug addiction and age?

Need more quantitative research title examples to inspire you? Here are some quantitative research title examples to look at:

• Habitation fragmentation and biodiversity loss: What is the link?
• Radiation has affected biodiversity: Assessing its effects
• An assessment of the impact of the CORONA virus on global population growth
• Is the pandemic truly over, or have human bodies built resistance against the virus?
• The ozone hole and its impact on the environment
• The greenhouse gas effect: What is it and how has it impacted the atmosphere
• GMO crops: are they good or bad for your health?
• Is there a direct link between education quality and job attainment?
• How have education systems changed from traditional to modern times?
• The good and bad impacts of technology on education qualities

Your examiner will give you excellent grades if you come up with a unique title and outstanding content. Here are some quantitative research examples titles.

• Online classes: are they helpful or not?
• What changes has the global CORONA pandemic had on the population growth curve?
• Daily habits influenced by the global pandemic
• An analysis of the impact of culture on people’s personalities
• How has feminism influenced the education system’s approach to the girl child’s education?
• Academic competition: what are its benefits and downsides for students?
• Is there a link between education and student integrity?
• An analysis of how the education sector can influence a country’s economy
• An overview of the link between crime rates and concern for crime
• Is there a link between education and obesity?

Research title example quantitative topics when well-thought guarantees a paper that is a good read. Look at the examples below to get started.

• What are the impacts of online games on students?
• Sex education in schools: how important is it?
• Should schools be teaching about safe sex in their sex education classes?
• The correlation between extreme parent interference on student academic performance
• Is there a real link between academic marks and intelligence?
• Teacher feedback: How necessary is it, and how does it help students?
• An analysis of modern education systems and their impact on student performance
• An overview of the link between academic performance/marks and intelligence
• Are grading systems helpful or harmful to students?
• What was the impact of the pandemic on students?

Irrespective of the course you take, here are some titles that can fit diverse subjects pretty well. Here are some creative quantitative research title ideas:

• A look at the pre-corona and post-corona economy
• How are conventional retail businesses fairing against eCommerce sites like Amazon and Shopify?
• An evaluation of mortality rates of heart attacks
• Effective treatments for cardiovascular issues and their prevention
• A comparison of the effectiveness of home care and nursing home care
• Strategies for managing effective dissemination of information to modern students
• How does educational discrimination influence students’ futures?
• The impacts of unfavorable classroom environment and bullying on students and teachers
• An overview of the implementation of STEM education to K-12 students
• How effective is digital learning?

If your paper addresses a problem, you must present facts that solve the question or tell more about the question. Here are examples of quantitative research titles that will inspire you.

• An elaborate study of the influence of telemedicine in healthcare practices
• How has scientific innovation influenced the defense or military system?
• The link between technology and people’s mental health
• Has social media helped create awareness or worsened people’s mental health?
• How do engineers promote green technology?
• How can engineers raise sustainability in building and structural infrastructures?
• An analysis of how decision-making is dependent on someone’s sub-conscious
• A comprehensive study of ADHD and its impact on students’ capabilities
• The impact of racism on people’s mental health and overall wellbeing
• How has the current surge in social activism helped shape people’s relationships?

Are you looking for an example of a quantitative research title? These ten examples below will get you started.

• The prevalence of nonverbal communication in social control and people’s interactions
• The impacts of stress on people’s behavior in society
• A study of the connection between capital structures and corporate strategies
• How do changes in credit ratings impact equality returns?
• A quantitative analysis of the effect of bond rating changes on stock prices
• The impact of semantics on web technology
• An analysis of persuasion, propaganda, and marketing impact on individuals
• The dominant-firm model: what is it, and how does it apply to your country’s retail sector?
• The role of income inequality in economy growth
• An examination of juvenile delinquents’ treatment in your country

Excellent Topics For Quantitative Research

Here are some titles for quantitative research you should consider:

• Does studying mathematics help implement data safety for businesses
• How are art-related subjects interdependent with mathematics?
• How do eco-friendly practices in the hospitality industry influence tourism rates?
• A deep insight into how people view eco-tourisms
• Religion vs. hospitality: Details on their correlation
• Has your country’s tourist sector revived after the pandemic?
• How effective is non-verbal communication in conveying emotions?
• Are there similarities between the English and French vocabulary?
• How do politicians use persuasive language in political speeches?
• The correlation between popular culture and translation

Here are some quantitative research titles examples for your consideration:

• How do world leaders use language to change the emotional climate in their nations?
• Extensive research on how linguistics cultivate political buzzwords
• The impact of globalization on the global tourism sector
• An analysis of the effects of the pandemic on the worldwide hospitality sector
• The influence of social media platforms on people’s choice of tourism destinations
• Educational tourism: What is it and what you should know about it
• Why do college students experience math anxiety?
• Is math anxiety a phenomenon?
• A guide on effective ways to fight cultural bias in modern society
• Creative ways to solve the overpopulation issue

An example of quantitative research topics for 12 th -grade students will come in handy if you want to score a good grade. Here are some of the best ones:

• The link between global warming and climate change
• What is the greenhouse gas impact on biodiversity and the atmosphere
• Has the internet successfully influenced literacy rates in society
• The value and downsides of competition for students
• A comparison of the education system in first-world and third-world countries
• The impact of alcohol addiction on the younger generation
• How has social media influenced human relationships?
• Has education helped boost feminism among men and women?
• Are computers in classrooms beneficial or detrimental to students?
• How has social media improved bullying rates among teenagers?

High school students can apply research titles on social issues  or other elements, depending on the subject. Let’s look at some quantitative topics for students:

• What is the right age to introduce sex education for students
• Can extreme punishment help reduce alcohol consumption among teenagers?
• Should the government increase the age of sexual consent?
• The link between globalization and the local economy collapses
• How are global companies influencing local economies?

There are numerous possible quantitative research topics you can write about. Here are some great quantitative research topics examples:

• The correlation between video games and crime rates
• Do college studies impact future job satisfaction?
• What can the education sector do to encourage more college enrollment?
• The impact of education on self-esteem
• The relationship between income and occupation

You can find inspiration for your research topic from trending affairs on social media or in the news. Such topics will make your research enticing. Find a trending topic for quantitative research example from the list below:

• How the country’s economy is fairing after the pandemic
• An analysis of the riots by women in Iran and what the women gain to achieve
• Is the current US government living up to the voter’s expectations?
• How is the war in Ukraine affecting the global economy?
• Can social media riots affect political decisions?

A proposal is a paper you write proposing the subject you would like to cover for your research and the research techniques you will apply. If the proposal is approved, it turns to your research topic. Here are some quantitative titles you should consider for your research proposal:

• Military support and economic development: What is the impact in developing nations?
• How does gun ownership influence crime rates in developed countries?
• How can the US government reduce gun violence without influencing people’s rights?
• What is the link between school prestige and academic standards?
• Is there a scientific link between abortion and the definition of viability?

You can never have too many sample titles. The samples allow you to find a unique title you’re your research or proposal. Find a sample quantitative research title here:

• Does weight loss indicate good or poor health?
• Should schools do away with grading systems?
• The impact of culture on student interactions and personalities
• How can parents successfully protect their kids from the dangers of the internet?
• Is the US education system better or worse than Europe’s?

If you’re a business major, then you must choose a research title quantitative about business. Let’s look at some research title examples quantitative in business:

• Creating shareholder value in business: How important is it?
• The changes in credit ratings and their impact on equity returns
• The importance of data privacy laws in business operations
• How do businesses benefit from e-waste and carbon footprint reduction?
• Organizational culture in business: what is its importance?

We Are A Call Away

Interesting, creative, unique, and easy quantitative research topics allow you to explain your paper and make research easy. Therefore, you should not take choosing a research paper or proposal topic lightly. With your topic ready, reach out to us today for excellent research paper writing services .

Leave a Reply Cancel reply

> q-bio > q-bio.QM

Help | Advanced Search

Quantitative Methods

Authors and titles for recent submissions.

• Tue, 23 Apr 2024
• Mon, 22 Apr 2024
• Fri, 19 Apr 2024
• Thu, 18 Apr 2024
• Wed, 17 Apr 2024

Links to: arXiv , form interface , find , q-bio , new , 2404 , contact , h elp   ( Access key information)

Quantitative Methods

• Living reference work entry
• First Online: 11 June 2021
• Cite this living reference work entry

• Juwel Rana 2 , 3 , 4 ,
• Patricia Luna Gutierrez 5 &
• John C. Oldroyd 6  

339 Accesses

1 Citations

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Babones S (2016) Interpretive quantitative methods for the social sciences. Sociology. https://doi.org/10.1177/0038038515583637

Balnaves M, Caputi P (2001) Introduction to quantitative research methods: an investigative approach. Sage, London

Book   Google Scholar  

Brenner PS (2020) Understanding survey methodology: sociological theory and applications. Springer, Boston

Google Scholar  

Creswell JW (2014) Research design: qualitative, quantitative, and mixed methods approaches. Sage, London

Leavy P (2017) Research design. The Gilford Press, New York

Mertens W, Pugliese A, Recker J (2018) Quantitative data analysis, research methods: information, systems, and contexts: second edition. https://doi.org/10.1016/B978-0-08-102220-7.00018-2

Neuman LW (2014) Social research methods: qualitative and quantitative approaches. Pearson Education Limited, Edinburgh

Treiman DJ (2009) Quantitative data analysis: doing social research to test ideas. Jossey-Bass, San Francisco

Download references

Author information

Authors and affiliations.

Department of Public Health, School of Health and Life Sciences, North South University, Dhaka, Bangladesh

Department of Biostatistics and Epidemiology, School of Health and Health Sciences, University of Massachusetts Amherst, MA, USA

Department of Research and Innovation, South Asia Institute for Social Transformation (SAIST), Dhaka, Bangladesh

Independent Researcher, Masatepe, Nicaragua

Patricia Luna Gutierrez

School of Behavioral and Health Sciences, Australian Catholic University, Fitzroy, VIC, Australia

John C. Oldroyd

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Juwel Rana .

Editor information

Editors and affiliations.

Florida Atlantic University, Boca Raton, FL, USA

Ali Farazmand

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this entry

Cite this entry.

Rana, J., Gutierrez, P.L., Oldroyd, J.C. (2021). Quantitative Methods. In: Farazmand, A. (eds) Global Encyclopedia of Public Administration, Public Policy, and Governance. Springer, Cham. https://doi.org/10.1007/978-3-319-31816-5_460-1

Download citation

DOI : https://doi.org/10.1007/978-3-319-31816-5_460-1

Received : 31 January 2021

Accepted : 14 February 2021

Published : 11 June 2021

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-31816-5

Online ISBN : 978-3-319-31816-5

eBook Packages : Springer Reference Economics and Finance Reference Module Humanities and Social Sciences Reference Module Business, Economics and Social Sciences

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• Quantitative Methods
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

Quantitative methods emphasize objective measurements and the statistical, mathematical, or numerical analysis of data collected through polls, questionnaires, and surveys, or by manipulating pre-existing statistical data using computational techniques . Quantitative research focuses on gathering numerical data and generalizing it across groups of people or to explain a particular phenomenon.

Babbie, Earl R. The Practice of Social Research . 12th ed. Belmont, CA: Wadsworth Cengage, 2010; Muijs, Daniel. Doing Quantitative Research in Education with SPSS . 2nd edition. London: SAGE Publications, 2010.

Need Help Locating Statistics?

Resources for locating data and statistics can be found here:

Statistics & Data Research Guide

Characteristics of Quantitative Research

Your goal in conducting quantitative research study is to determine the relationship between one thing [an independent variable] and another [a dependent or outcome variable] within a population. Quantitative research designs are either descriptive [subjects usually measured once] or experimental [subjects measured before and after a treatment]. A descriptive study establishes only associations between variables; an experimental study establishes causality.

Quantitative research deals in numbers, logic, and an objective stance. Quantitative research focuses on numeric and unchanging data and detailed, convergent reasoning rather than divergent reasoning [i.e., the generation of a variety of ideas about a research problem in a spontaneous, free-flowing manner].

Its main characteristics are :

• The data is usually gathered using structured research instruments.
• The results are based on larger sample sizes that are representative of the population.
• The research study can usually be replicated or repeated, given its high reliability.
• Researcher has a clearly defined research question to which objective answers are sought.
• All aspects of the study are carefully designed before data is collected.
• Data are in the form of numbers and statistics, often arranged in tables, charts, figures, or other non-textual forms.
• Project can be used to generalize concepts more widely, predict future results, or investigate causal relationships.
• Researcher uses tools, such as questionnaires or computer software, to collect numerical data.

The overarching aim of a quantitative research study is to classify features, count them, and construct statistical models in an attempt to explain what is observed.

  Things to keep in mind when reporting the results of a study using quantitative methods :

• Explain the data collected and their statistical treatment as well as all relevant results in relation to the research problem you are investigating. Interpretation of results is not appropriate in this section.
• Report unanticipated events that occurred during your data collection. Explain how the actual analysis differs from the planned analysis. Explain your handling of missing data and why any missing data does not undermine the validity of your analysis.
• Explain the techniques you used to "clean" your data set.
• Choose a minimally sufficient statistical procedure ; provide a rationale for its use and a reference for it. Specify any computer programs used.
• Describe the assumptions for each procedure and the steps you took to ensure that they were not violated.
• When using inferential statistics , provide the descriptive statistics, confidence intervals, and sample sizes for each variable as well as the value of the test statistic, its direction, the degrees of freedom, and the significance level [report the actual p value].
• Avoid inferring causality , particularly in nonrandomized designs or without further experimentation.
• Use tables to provide exact values ; use figures to convey global effects. Keep figures small in size; include graphic representations of confidence intervals whenever possible.
• Always tell the reader what to look for in tables and figures .

NOTE:   When using pre-existing statistical data gathered and made available by anyone other than yourself [e.g., government agency], you still must report on the methods that were used to gather the data and describe any missing data that exists and, if there is any, provide a clear explanation why the missing data does not undermine the validity of your final analysis.

Babbie, Earl R. The Practice of Social Research . 12th ed. Belmont, CA: Wadsworth Cengage, 2010; Brians, Craig Leonard et al. Empirical Political Analysis: Quantitative and Qualitative Research Methods . 8th ed. Boston, MA: Longman, 2011; McNabb, David E. Research Methods in Public Administration and Nonprofit Management: Quantitative and Qualitative Approaches . 2nd ed. Armonk, NY: M.E. Sharpe, 2008; Quantitative Research Methods. Writing@CSU. Colorado State University; Singh, Kultar. Quantitative Social Research Methods . Los Angeles, CA: Sage, 2007.

Basic Research Design for Quantitative Studies

Before designing a quantitative research study, you must decide whether it will be descriptive or experimental because this will dictate how you gather, analyze, and interpret the results. A descriptive study is governed by the following rules: subjects are generally measured once; the intention is to only establish associations between variables; and, the study may include a sample population of hundreds or thousands of subjects to ensure that a valid estimate of a generalized relationship between variables has been obtained. An experimental design includes subjects measured before and after a particular treatment, the sample population may be very small and purposefully chosen, and it is intended to establish causality between variables. Introduction The introduction to a quantitative study is usually written in the present tense and from the third person point of view. It covers the following information:

• Identifies the research problem -- as with any academic study, you must state clearly and concisely the research problem being investigated.
• Reviews the literature -- review scholarship on the topic, synthesizing key themes and, if necessary, noting studies that have used similar methods of inquiry and analysis. Note where key gaps exist and how your study helps to fill these gaps or clarifies existing knowledge.
• Describes the theoretical framework -- provide an outline of the theory or hypothesis underpinning your study. If necessary, define unfamiliar or complex terms, concepts, or ideas and provide the appropriate background information to place the research problem in proper context [e.g., historical, cultural, economic, etc.].

Methodology The methods section of a quantitative study should describe how each objective of your study will be achieved. Be sure to provide enough detail to enable the reader can make an informed assessment of the methods being used to obtain results associated with the research problem. The methods section should be presented in the past tense.

• Study population and sampling -- where did the data come from; how robust is it; note where gaps exist or what was excluded. Note the procedures used for their selection;
• Data collection – describe the tools and methods used to collect information and identify the variables being measured; describe the methods used to obtain the data; and, note if the data was pre-existing [i.e., government data] or you gathered it yourself. If you gathered it yourself, describe what type of instrument you used and why. Note that no data set is perfect--describe any limitations in methods of gathering data.
• Data analysis -- describe the procedures for processing and analyzing the data. If appropriate, describe the specific instruments of analysis used to study each research objective, including mathematical techniques and the type of computer software used to manipulate the data.

Results The finding of your study should be written objectively and in a succinct and precise format. In quantitative studies, it is common to use graphs, tables, charts, and other non-textual elements to help the reader understand the data. Make sure that non-textual elements do not stand in isolation from the text but are being used to supplement the overall description of the results and to help clarify key points being made. Further information about how to effectively present data using charts and graphs can be found here .

• Statistical analysis -- how did you analyze the data? What were the key findings from the data? The findings should be present in a logical, sequential order. Describe but do not interpret these trends or negative results; save that for the discussion section. The results should be presented in the past tense.

Discussion Discussions should be analytic, logical, and comprehensive. The discussion should meld together your findings in relation to those identified in the literature review, and placed within the context of the theoretical framework underpinning the study. The discussion should be presented in the present tense.

• Interpretation of results -- reiterate the research problem being investigated and compare and contrast the findings with the research questions underlying the study. Did they affirm predicted outcomes or did the data refute it?
• Description of trends, comparison of groups, or relationships among variables -- describe any trends that emerged from your analysis and explain all unanticipated and statistical insignificant findings.
• Discussion of implications – what is the meaning of your results? Highlight key findings based on the overall results and note findings that you believe are important. How have the results helped fill gaps in understanding the research problem?
• Limitations -- describe any limitations or unavoidable bias in your study and, if necessary, note why these limitations did not inhibit effective interpretation of the results.

Conclusion End your study by to summarizing the topic and provide a final comment and assessment of the study.

• Summary of findings – synthesize the answers to your research questions. Do not report any statistical data here; just provide a narrative summary of the key findings and describe what was learned that you did not know before conducting the study.
• Recommendations – if appropriate to the aim of the assignment, tie key findings with policy recommendations or actions to be taken in practice.
• Future research – note the need for future research linked to your study’s limitations or to any remaining gaps in the literature that were not addressed in your study.

Black, Thomas R. Doing Quantitative Research in the Social Sciences: An Integrated Approach to Research Design, Measurement and Statistics . London: Sage, 1999; Gay,L. R. and Peter Airasain. Educational Research: Competencies for Analysis and Applications . 7th edition. Upper Saddle River, NJ: Merril Prentice Hall, 2003; Hector, Anestine. An Overview of Quantitative Research in Composition and TESOL . Department of English, Indiana University of Pennsylvania; Hopkins, Will G. “Quantitative Research Design.” Sportscience 4, 1 (2000); "A Strategy for Writing Up Research Results. The Structure, Format, Content, and Style of a Journal-Style Scientific Paper." Department of Biology. Bates College; Nenty, H. Johnson. "Writing a Quantitative Research Thesis." International Journal of Educational Science 1 (2009): 19-32; Ouyang, Ronghua (John). Basic Inquiry of Quantitative Research . Kennesaw State University.

Strengths of Using Quantitative Methods

Quantitative researchers try to recognize and isolate specific variables contained within the study framework, seek correlation, relationships and causality, and attempt to control the environment in which the data is collected to avoid the risk of variables, other than the one being studied, accounting for the relationships identified.

Among the specific strengths of using quantitative methods to study social science research problems:

• Allows for a broader study, involving a greater number of subjects, and enhancing the generalization of the results;
• Allows for greater objectivity and accuracy of results. Generally, quantitative methods are designed to provide summaries of data that support generalizations about the phenomenon under study. In order to accomplish this, quantitative research usually involves few variables and many cases, and employs prescribed procedures to ensure validity and reliability;
• Applying well established standards means that the research can be replicated, and then analyzed and compared with similar studies;
• You can summarize vast sources of information and make comparisons across categories and over time; and,
• Personal bias can be avoided by keeping a 'distance' from participating subjects and using accepted computational techniques .

Babbie, Earl R. The Practice of Social Research . 12th ed. Belmont, CA: Wadsworth Cengage, 2010; Brians, Craig Leonard et al. Empirical Political Analysis: Quantitative and Qualitative Research Methods . 8th ed. Boston, MA: Longman, 2011; McNabb, David E. Research Methods in Public Administration and Nonprofit Management: Quantitative and Qualitative Approaches . 2nd ed. Armonk, NY: M.E. Sharpe, 2008; Singh, Kultar. Quantitative Social Research Methods . Los Angeles, CA: Sage, 2007.

Limitations of Using Quantitative Methods

Quantitative methods presume to have an objective approach to studying research problems, where data is controlled and measured, to address the accumulation of facts, and to determine the causes of behavior. As a consequence, the results of quantitative research may be statistically significant but are often humanly insignificant.

Some specific limitations associated with using quantitative methods to study research problems in the social sciences include:

• Quantitative data is more efficient and able to test hypotheses, but may miss contextual detail;
• Uses a static and rigid approach and so employs an inflexible process of discovery;
• The development of standard questions by researchers can lead to "structural bias" and false representation, where the data actually reflects the view of the researcher instead of the participating subject;
• Results provide less detail on behavior, attitudes, and motivation;
• Researcher may collect a much narrower and sometimes superficial dataset;
• Results are limited as they provide numerical descriptions rather than detailed narrative and generally provide less elaborate accounts of human perception;
• The research is often carried out in an unnatural, artificial environment so that a level of control can be applied to the exercise. This level of control might not normally be in place in the real world thus yielding "laboratory results" as opposed to "real world results"; and,
• Preset answers will not necessarily reflect how people really feel about a subject and, in some cases, might just be the closest match to the preconceived hypothesis.

Research Tip

Finding Examples of How to Apply Different Types of Research Methods

SAGE publications is a major publisher of studies about how to design and conduct research in the social and behavioral sciences. Their SAGE Research Methods Online and Cases database includes contents from books, articles, encyclopedias, handbooks, and videos covering social science research design and methods including the complete Little Green Book Series of Quantitative Applications in the Social Sciences and the Little Blue Book Series of Qualitative Research techniques. The database also includes case studies outlining the research methods used in real research projects. This is an excellent source for finding definitions of key terms and descriptions of research design and practice, techniques of data gathering, analysis, and reporting, and information about theories of research [e.g., grounded theory]. The database covers both qualitative and quantitative research methods as well as mixed methods approaches to conducting research.

SAGE Research Methods Online and Cases

• << Previous: Qualitative Methods
• Next: Insiderness >>
• Last Updated: Apr 22, 2024 9:12 AM
• URL: https://libguides.usc.edu/writingguide

Quantitative Research Methods for Political Science, Public Policy and Public Administration (With Applications in R) - 3rd Edition

(8 reviews)

Hank Jenkins-Smith, University of Oklahoma

Joseph Ripberger, University of Oklahoma

Copyright Year: 2017

Publisher: University of Oklahoma Libraries

Language: English

Formats Available

Conditions of use.

Learn more about reviews.

Reviewed by Saahir Shafi, Assistant Professor, California State University, Dominguez Hills on 12/13/22

This textbook provides a solid introduction to quantitative methods in social science research. It is ideal for introducing early stage researchers to R as a tool of quantitative research. From a broad overview of the scientific method and... read more

Comprehensiveness rating: 4 see less

This textbook provides a solid introduction to quantitative methods in social science research. It is ideal for introducing early stage researchers to R as a tool of quantitative research. From a broad overview of the scientific method and research design to OLS and logit regression, researchers can expect to become comfortable using R for data analysis. The authors could expand this volume to introduce more intermediate and advanced examples of quantitative methods such as ridge regression, panel regression, etc.

Content Accuracy rating: 5

The content is accurate, error-free, and quite straightforward - R scripts are broken down with clear discussions on what the script is evaluating and how to interpret results.

Relevance/Longevity rating: 5

The content is up-to-date. Although newer R packages continue to be made available, this text provides a foundational knowledge of basic statistical analysis which is unlikely to become obsolete anytime soon.

Clarity rating: 5

The text is highly accessible and may be successfully used by graduate students with little to no prior knowledge of R. A base understanding of research methods and quantitative analysis would be beneficial for students to get the most out of this text.

Consistency rating: 5

The text is consistent in terms of terminology and presentation of material.

Modularity rating: 5

The text is easily divisible into sections and concepts that progressively build upon each other and ideal for college level coursework. The book is split into 16 sections which would fit ideally within the scope of a 16 week course.

Organization/Structure/Flow rating: 5

Topics are presented in a logical progression moving from general research design to variables and model specification.

Interface rating: 5

There are no interface issues. Text is presented in an organized and accessible format.

Grammatical Errors rating: 5

The text contains no grammatical errors.

Cultural Relevance rating: 5

The text is not culturally insensitive or offensive in any way. In future editions, the authors could make efforts to include more diverse demographic groupings within the specified models to demonstrate the best way to evaluate such variables.

Reviewed by Lindsay Benstead, Associate Professor, Portland State University on 9/3/22

This book is highly comprehensive in the sense that it effectively marries a discussion of the theoretical foundations of research design and statistics with concrete examples and syntax for applying the concepts in R. Although there is neither an... read more

This book is highly comprehensive in the sense that it effectively marries a discussion of the theoretical foundations of research design and statistics with concrete examples and syntax for applying the concepts in R. Although there is neither an index nor a glossary, the table of contents is detailed and the book itself is effectively organized. Key words are presented in bold.

In my review of the textbook, I found no evidence of errors of bias. The book appears to be carefully edited with well-chosen examples pertinent to the field of study.

Given the subject matter (i.e., statistics and mathematics), it is unlikely that the material will date quickly. It is plausible but unlikely that the R syntax will no longer work in future iterations of the program. The textbook is on its third edition, suggesting that the authors are attentive to implementing improvements. Additionally, while there are screenshots to pages where students can download resources, the instructions are described in the text without the use of many links which can stop working and create frustration for readers.

This book contains very clear descriptions of key topics. Specific chapters could be assigned in courses, even if the application in R is not being used in the course, in which case some of the chapters would be less relevant.

This book is thoroughly edited and presents the material in a clear and well-organized way as one would find in any quality textbook on the subject.

The early chapters on research design and statistical theory could be assigned on their own, while the same would be true of the latter chapters for a course needing only instructions on how to use R.

The organization of the book is effective. I would like to see potential material on how to conduct a literature review and research ethics. Creation of examples using SPSS or Stata would also be welcome.

The formatting and interface is problem free.

This book is well edited and free from grammatical errors.

This book is free from insensitive or offensive material.

Reviewed by Caitlin Jewitt, Associate Professor, Virginia Tech on 12/14/21

This book is very comprehensive, beginning with the scientific method, progressing through research design, data visualization, inference, regression, and culminating with a chapter on generalized linear models (logit). The table of contents is... read more

Comprehensiveness rating: 3 see less

This book is very comprehensive, beginning with the scientific method, progressing through research design, data visualization, inference, regression, and culminating with a chapter on generalized linear models (logit). The table of contents is thorough, which is helpful to the reader, especially for a methods textbook, which often is not read cover-to-cover, but is referenced time and time again. There is also an appendix which introduces the reader to utilizing R to implement some of the statistical techniques provided in the text. I would like to see more time spent on interaction terms, as this is an important component of teaching quantitative methods to political scientists. While the book covers a breadth of topics, it provides only a surface coverage of many fundamental aspects of research methods (e.g. independent and dependent variables). In other words, the coverage is broad but not deep. Compared to other methods textbooks, there are not as many examples and there are not problems or questions that are often very helpful to students.

The content is accurate and unbiased and its presentation is straightforward. The authors could spend more time explaining how to apply these concepts in R and what version of R they are using, so that they may be more easily replicated.

Due to the content (methods-based), it will remain relevant and not be quickly outdated like many texts in political science. The text is also on its third edition, which demonstrates that the authors are continuing to update and improve the text.

Clarity rating: 4

There are some missed opportunities for the authors to define terms. For instance, they discuss a working hypothesis and null hypothesis in the first paragraph (and put these terms in bold), but do not explicitly define them in the text. In other places, they define terms (such as the definition of theory on page 5). Defining terms consistently throughout the text would be helpful and would improve the text's clarity.

Overall, the language is clear and straightforward. It is written in a manner that is easily accessible to undergraduates. Additional examples, however, would provide a useful supplement to aid in understanding.

The presentation of graphs could be enhanced by using variable labels as opposed to variable names.

The book is consistent in its approach and terminology.

There are many short sections within each chapter. This makes it ideal and easy to assign sections of a chapter to students, rather than requiring that they read the whole chapter. Other than understanding the basics of research methods, readers could easily move between sections and read portions out of order.

Organization/Structure/Flow rating: 4

The book is well organized, and progresses in an expected fashion. It begins with theories of social science and the scientific method, discusses fundamentals of research methods, describing and displaying data, discusses inference, then presents bivariate and multivariate OLS regression, and finally general linear models. This coincides with the order in which I would teach these topics.

Interface rating: 4

The book is produced in latex, and so the format (including figures) should be familiar to many political scientists who utilize this software. One revision the authors could make for future revisions would be to include hyperlinks in the table of contents to link the reader to the sections, chapters, and figures.

The book is well-written.

The book is not culturally insensitive or offensive. The examples are straightforward and brief. In a future revision, the authors could consider discussing the measurement of demographic variables (e.g. gender and sex, race) in greater depth. This would provide the reader with a stronger grasp of the advantages and disadvantages of utilizing different measurement strategies.

The book is a good, comprehensive overview of research methods. It would be difficult to use it as the sole textbook for undergraduates, due to the lack of examples. It would be a strong choice for a supplemental text or may be more appropriate for a graduate course.

Reviewed by Kimberly Wilson, Assistant Professor, East Tennessee State University on 3/22/20

The book's overall approach is great -- framing quantitative methods in terms of social scientific research more broadly. If I was teaching a quantitative methods course, I would most likely use this book, as it covers a nice range of essentials,... read more

The book's overall approach is great -- framing quantitative methods in terms of social scientific research more broadly. If I was teaching a quantitative methods course, I would most likely use this book, as it covers a nice range of essentials, particularly regression, while the open source nature ensures that students can always return to this book for reference. The book's use of R is similarly ideal. There are a few areas where an instructor may wish to expand upon the book's content, but this can easily be done through lecture or by assigning one or two additional and complementary readings. I do wish the book did a bit more in terms of clearly defining key terms and concepts. For instance, null hypothesis is first mentioned on page 4, but is not defined until page 10, and one only learns this by reading the full chapter. While the book description says that the book is designed for upper level undergraduates and graduate students, I assume that most students do not encounter terms like null hypothesis until their first methods course, which is usually where they are also learning quantitative methods (and where this textbook would be appropriate). In short, a glossary of the terms set in bold would be a strong addition to this book.

I saw no inaccuracies worthy of note. One always has preferences for the way in which methods are explained, but I saw nothing that would cause me to view this book as inaccurate.

The book tackles fundamentals in social scientific research and quantitative methods, and these will stay relevant.

As mentioned above, a glossary would be an easy addition that would greatly strengthen the text. Students at all levels can become intimidated by a methods book with unfamiliar terminology. A glossary can help alleviate some anxiety.

I saw no inconsistencies.

The book is organized in a consistent and clear manner. The headings and subheadings are easily understood and navigated. Chapters can easily be broken down into smaller sections for class readings.

The text builds in a clear and logical fashion, appropriate to the subject matter of this type of course.

I saw no interface issues.

There are only trivial grammatical errors, of the kind similar to all textbooks.

I did not see anything culturally insensitive or offensive in the text. I have to admit, I only understood the Monty Python reference after googling it, but that's life.

I have one relatively minor suggestion. In the first two chapters, where theory and social scientific methodology is discussed, it might help to use a consistent, versatile example to illustrate many concepts of those chapters. For instance, when the text discusses the goal of generalization, and uses the example of why a president's approval rating may have dropped, why not also use this example later to discuss independent and dependent variables? The discussion of dependent and independent variables on page 6 doesn't use an example, and I think students would greatly benefit by having an example to illustrate this content.

Reviewed by Christina Ladam, Graduate Part-time Instructor, CU Boulder on 6/5/19, updated 7/1/19

This text does a solid job in providing an introduction to statistical analysis with a focus on regression. Additionally, it provides a light introduction to statistical computing in R. This is mostly a tool for teaching regression, with a light... read more

This text does a solid job in providing an introduction to statistical analysis with a focus on regression. Additionally, it provides a light introduction to statistical computing in R. This is mostly a tool for teaching regression, with a light introduction to maximum likelihood estimation and generalized linear models through a chapter on logistic regression. The text briefly discusses some other methods, though, for instance, the discussion on experimental research designs is quite minimal. There is no discussion of survey experiments, which are increasingly used by social scientists as research design. Perhaps the text should be more clearly framed as one to teach regression. Additionally, there could be more instruction provided on R, specifically in teaching best practices for conducting analyses in R.

Content Accuracy rating: 4

I found the content in the text to be mostly accurate. The "Inference" section could use some editing in reference to p-values and how we interpret them. This is notoriously difficult, but could be improved.

Relevance/Longevity rating: 4

While I cannot foresee the content regarding regression becoming obsolete any time soon, there are some limitations to the relevance of the text. For instance, many more recent developments in methodology are not included. That is fine, as no one book can address that many streams of quantitative research. However, the framing of the book makes it seem like it would address more than regression. Additionally, the text would be improved by providing an updated, more thorough introduction to R, including a "best practices" approach to analysis in R.

The text is written quite clearly, and would be very appropriate for its target audience. Complex econometric concepts are written in an approachable way, with illustrative and complementary examples. I can see this text being especially useful for public policy and public administration students. While the text is framed as being designed for graduate students, it also seems appropriate for teaching undergraduate statistics courses.

I found the text to be consistent in its notation, which is important in statistics texts.

I really appreciated the way in which chapters were organized. Subjects were broken down to manageable chapter lengths, and the use of headings and subheadings was very clear. I can easily picture assigning readings throughout the semester without much modification to chapters.

I appreciate the authors' decision to structure the the text as similar to the way in which scientific research is conducted, beginning with the development of theory, moving to research design, and ending with statistical analyses and model evaluation. It is important to place an emphasis on following the scientific method when conducting statistical analyses. While the Appendix on R is helpful, it may make sense to incorporate some introduction to R in the main text. When R is introduced in the main text, it somewhat assumes a baseline familiarity with R.

The PDF version was mostly free of interface issues. It would be nice to incorporate hyperlinks within the text, so that one can simply click on a page number to navigate to a section rather than being limited to scrolling to find things. There also seems to be some inconsistency in formatting of tables and figures -- while most are center-aligned, some are left-aligned.

I did not encounter problematic grammatical errors.

I did not find the text to be culturally insensitive in any way.

Reviewed by Chris Garmon, Assistant Professor of Health Administration, University of Missouri - Kansas City on 5/24/19

The book's coverage of regression is outstanding. In particular, this is the most comprehensive coverage of regression diagnostics I've seen in a research methods text. There is also an entire chapter on logit regression, whereas most texts may... read more

The book's coverage of regression is outstanding. In particular, this is the most comprehensive coverage of regression diagnostics I've seen in a research methods text. There is also an entire chapter on logit regression, whereas most texts may devote a paragraph to it at best. Most texts jump right into inference after descriptive statistics, but the authors add a chapter on probability before discussing inference, which is a nice addition. However, there are certain topics that are not covered or barely covered. There is only a cursory discussion of sampling distributions and only one paragraph on the Central Limit Theorem. The authors fly through the discussion of t tests and there is no coverage of the assumptions needed for independent sample t tests. The only coverage of ANOVA is in the discussion of model fit. I think this is an excellent text for instructors who want to emphasize regressions, but those who like to build up to regressions with t tests and ANOVA might find this text lacking.

The book is accurate and thorough, particularly regarding regression and regression diagnostics. On a few occasions, the authors talked about "accepting the null hypothesis" if the p value is greater than 0.05, which is too strong, but apart from that, I saw no problems with the analysis or interpretations.

A nice feature of this text is that it is written in open source R markdown, so instructors can adapt and add content as desired, making updates easy to implement.

The book has the right tone and level of technical information for Ph.D. students in the social sciences, but I think parts of it are too advanced for the typical MPA student. There are entire chapters on calculus (chapter 8) and matrix algebra (chapter 11), which in my opinion are unnecessary for and would likely intimidate most MPA students.

The terminology and framework are consistent and easy to follow.

Modularity rating: 2

This book is best covered as a whole. I think it would be difficult to use only a subset of chapters as they all build off and reference each other. For instance, there are numerous instances where terminology (e.g., null hypothesis, Likert scale) are briefly introduced with the promise to cover them in more detail in future chapters.

The book starts off with an emphasis on theory as the basis and guiding force of quality social science research and the topics are presented with this theme in mind. I applaud the authors for making theory and causality a guiding principle for the organization of the text because too many research methods texts leave the students with the impression that quantitative research involves looking at the data, discerning patterns, and then developing a theory. I think the organization of the text is ideal with the emphasis of theory and testable hypotheses as the starting point of research.

The text has no interface or navigation problems.

Grammatical Errors rating: 4

There are a number of minor grammatical mistakes and typos, but nothing that would cause confusion for the reader.

Cultural Relevance rating: 3

The text uses only one example throughout (an analysis of a survey of perceptions of climate change risk by political ideology and sex). Students outside of political science might not find the example interesting or relevant for the research problems they are likely to face. The title of the text implies that it is designed for public administration students. The text should illustrate at least some of the concepts with research problems public administrators are likely to face.

Overall, I think this is an excellent text, but I think it is too advanced and technical, and has too much of a political science focus, for MPA students.

Reviewed by Sarah Fisher, Assistant Professor, Emory and Henry College on 3/20/19

In terms of content, this text contains nearly everything I generally cover in my introductory statistics class. This book is aimed at graduate students, but I am reviewing it for undergraduate social science majors. Overall, I think this will... read more

Comprehensiveness rating: 5 see less

In terms of content, this text contains nearly everything I generally cover in my introductory statistics class. This book is aimed at graduate students, but I am reviewing it for undergraduate social science majors. Overall, I think this will be a good text. It does seem that the authors assume some level of knowledge of R before beginning the book. There is additional information available online and in the appendix, but I think more of an introduction to R placed at the beginning of the book would have been useful, given how prominently R features in the text. I share the author's frustration with teaching this course-- the cost of these textbooks is high and the relationship between statistics and actual research is sometimes spotty. I think this text does a good job of really connecting statistical techniques to social science research.

I saw no glaring inaccuracies in the text.

One great thing about statistics books is that the formula for standard deviation is unlikely to change any time soon. I see this text as having a long self life. The only thing that might change would be the R code, but the authors have noted that there is more information available about the R online.

I found the writing in this text to be very clear. One nice addition would be titles for all of the R code that corresponded with a quick reference list for the code included. Then, if a student was looking for the R code to recode a variable (page 80), for instance, they could quickly find it. Given the online format, one can search for this information in the text, but I think students who print the text might find it useful.

The book is generally consistent in terms of format.

This is one of the text's strong points. They cover a lot of information in an efficient manner, and they also include some useful asides. For instance, in section 5.3.3, when discussing statistical inference, they have a header entitled "Some Miscellaneous Notes about Hypothesis Testing." I find this sort of discussion very useful. This section included information on why .05 is a standard, Type I and Type II errors, etc. While these are clearly important, they are secondary compared to general ideas about inference. In this sense, I think the layout of the text is very reader friendly. The bolded terms are also crucial. I also appreciate the "Summary" sections at the end of chapters.

I think the organization is very good. In an undergraduate course, I'm not sure I will go in as much depth as is included in some of the later chapters (ex: having students do quartile plots for residuals), but I still find it useful. Moreover, an instructor could easily pick and choose which sections they wanted students to read given the section headers. I might just move the R appendix to the beginning of the text.

I think that the graphics (some in color) are particularly useful. Moreover, I think that the inclusion of R output throughout the text was generally useful. I would like to have seen more presentations of "cleaned" data, to show students how they should present their data output. There are several points in the text where the R code seems to be out of place. For instance, on page 76, part of the code goes outside the grey shaded box.

The grammar and writing style of the textbook was good. I saw no major problems.

Cultural Relevance rating: 4

The text has the occasional nerd-culture reference (ex: page 40 contained a Monty Python reference) and sports references (ex: lots of baseball references in the probability chapter). In another example, when talking about sampling strategies, the authors write about how one might observe a potential partner in a variety of circumstances to determine whether they would me a good match. While I find this example a bit odd, I think the impulse to include interesting examples is a good one.

I am planning on using this for an undergraduate class, and it seems like the authors have pitched this for graduate students. I don't anticipate too many differences, but I'm excited to see how this textbook works for undergrads.

Reviewed by Saleheh Sharifmoghaddam, Adjunct Lecturer, Lehman College, City University of New York on 5/21/18

This book definitely tackles many of the issues facing students doing quantitative analysis in social sciences. The authors try to cover the main data analysis techniques, providing readers with ample examples to better appreciate the complexity... read more

This book definitely tackles many of the issues facing students doing quantitative analysis in social sciences. The authors try to cover the main data analysis techniques, providing readers with ample examples to better appreciate the complexity and dynamism of each model. While no text can attend to all models with detail, this book tries to educate the reader holistically and achieves this breadth, in my opinion, very effectively.

The book is accurate and error-free.

The book is certainly up-to-date and includes R codes to apply the models in the R interface.

The forte of the book is explaining complex econometrics models in very simple language with ample examples.

The text is internally consistent.

The books has various subheadings and makes the division of material very clear at the outset.

The topics are presented in a logical and clear fashion.

There are no significant interface issues.

The text is free of grammatical errors.

The books is not culturally offensive in any way.

The authors can improve the teaching capacity of the material by adding a sequel to the book, discussing more complicated models used in social sciences.

Table of Contents

I Theory and Empirical Social Science

• 1 Theories and Social Science
• 2 Research Design
• 3 Exploring and Visualizing Data
• 4 Probability
• 5 Inference
• 6 Association of Variables

II Simple Regression

• 7 The Logic of Ordinary Least Squares Estimation
• 8 Linear Estimation and Minimizing Error
• 9 Bi-Variate Hypothesis Testing and Model Fit
• 10 OLS Assumptions and Simple Regression Diagnostics

III Multiple Regression

• 11 Introduction to Multiple Regression
• 12 The Logic of Multiple Regression
• 13 Multiple Regression and Model Building
• 14 Topics in Multiple Regression
• 15 The Art of Regression Diagnostic

IV Generalized Linear Model

• 16 Logit Regression

V Appendices

• 17 Appendix: Basic

Ancillary Material

About the book.

The focus of this book is on using quantitative research methods to test hypotheses and build theory in political science, public policy and public administration. It is designed for advanced undergraduate courses, or introductory and intermediate graduate-level courses. The first part of the book introduces the scientific method, then covers research design, measurement, descriptive statistics, probability, inference, and basic measures of association. The second part of the book covers bivariate and multiple linear regression using the ordinary least squares, the calculus and matrix algebra that are necessary for understanding bivariate and multiple linear regression, the assumptions that underlie these methods, and then provides a short introduction to generalized linear models. The book fully embraces the open access and open source philosophies. The book is freely available in the SHAREOK repository; it is written in R Markdown files that are available in a public GitHub repository; it uses and teaches R and RStudio for data analysis, visualization and data management; and it uses publically available survey data (from the Meso-Scale Integrated Socio-geographic Network) to illustrate important concepts and methods. We encourage students to download the data, replicate the examples, and explore further! We also encourage instructors to download the R Markdown files and modify the text for use in different courses.

About the Contributors

Hank Jenkins-Smith earned his PhD in political science from the University of Rochester (1985). He is a George Lynn Cross Research Professor in the Political Science Department at the University of Oklahoma, and serves as a co-Director of the National Institute for Risk and Resilience. Professor Jenkins-Smith has published books and articles on public policy processes, national security, weather, and energy and environmental policy. He has served on National Research Council Committees, as an elected member on the National Council on Radiation Protection and Measurement, and as a member of the governing Council of the American Political Science Association. His current research focuses on theories of the public policy process, with particular emphasis on the management (and mismanagement) of controversial technical issues involving high risk perceptions on the part of the public. In 2012 he and collaborators initiated a series of studies focused on social responses to the risks posed by severe weather. This work continues with a panel survey of Oklahoma households, funded by the National Science Foundation, to track perceptions of and responses to changing weather patterns. In his spare time, Professor Jenkins-Smith engages in personal experiments in risk perception and management via skiing, scuba diving and motorcycling.

Joseph Ripberger currently works at the Center for Risk and Crisis Management, University of Oklahoma. Joseph does research in Public Policy. Their current project is 'Glen Canyon Dam.'

Contribute to this Page

• Privacy Policy

Buy Me a Coffee

Home » Quantitative Research – Methods, Types and Analysis

Quantitative Research – Methods, Types and Analysis

Table of Contents

Quantitative Research

Quantitative research is a type of research that collects and analyzes numerical data to test hypotheses and answer research questions . This research typically involves a large sample size and uses statistical analysis to make inferences about a population based on the data collected. It often involves the use of surveys, experiments, or other structured data collection methods to gather quantitative data.

Quantitative Research Methods

Quantitative Research Methods are as follows:

Descriptive Research Design

Descriptive research design is used to describe the characteristics of a population or phenomenon being studied. This research method is used to answer the questions of what, where, when, and how. Descriptive research designs use a variety of methods such as observation, case studies, and surveys to collect data. The data is then analyzed using statistical tools to identify patterns and relationships.

Correlational Research Design

Correlational research design is used to investigate the relationship between two or more variables. Researchers use correlational research to determine whether a relationship exists between variables and to what extent they are related. This research method involves collecting data from a sample and analyzing it using statistical tools such as correlation coefficients.

Quasi-experimental Research Design

Quasi-experimental research design is used to investigate cause-and-effect relationships between variables. This research method is similar to experimental research design, but it lacks full control over the independent variable. Researchers use quasi-experimental research designs when it is not feasible or ethical to manipulate the independent variable.

Experimental Research Design

Experimental research design is used to investigate cause-and-effect relationships between variables. This research method involves manipulating the independent variable and observing the effects on the dependent variable. Researchers use experimental research designs to test hypotheses and establish cause-and-effect relationships.

Survey Research

Survey research involves collecting data from a sample of individuals using a standardized questionnaire. This research method is used to gather information on attitudes, beliefs, and behaviors of individuals. Researchers use survey research to collect data quickly and efficiently from a large sample size. Survey research can be conducted through various methods such as online, phone, mail, or in-person interviews.

Quantitative Research Analysis Methods

Here are some commonly used quantitative research analysis methods:

Statistical Analysis

Statistical analysis is the most common quantitative research analysis method. It involves using statistical tools and techniques to analyze the numerical data collected during the research process. Statistical analysis can be used to identify patterns, trends, and relationships between variables, and to test hypotheses and theories.

Regression Analysis

Regression analysis is a statistical technique used to analyze the relationship between one dependent variable and one or more independent variables. Researchers use regression analysis to identify and quantify the impact of independent variables on the dependent variable.

Factor Analysis

Factor analysis is a statistical technique used to identify underlying factors that explain the correlations among a set of variables. Researchers use factor analysis to reduce a large number of variables to a smaller set of factors that capture the most important information.

Structural Equation Modeling

Structural equation modeling is a statistical technique used to test complex relationships between variables. It involves specifying a model that includes both observed and unobserved variables, and then using statistical methods to test the fit of the model to the data.

Time Series Analysis

Time series analysis is a statistical technique used to analyze data that is collected over time. It involves identifying patterns and trends in the data, as well as any seasonal or cyclical variations.

Multilevel Modeling

Multilevel modeling is a statistical technique used to analyze data that is nested within multiple levels. For example, researchers might use multilevel modeling to analyze data that is collected from individuals who are nested within groups, such as students nested within schools.

Applications of Quantitative Research

Quantitative research has many applications across a wide range of fields. Here are some common examples:

• Market Research : Quantitative research is used extensively in market research to understand consumer behavior, preferences, and trends. Researchers use surveys, experiments, and other quantitative methods to collect data that can inform marketing strategies, product development, and pricing decisions.
• Health Research: Quantitative research is used in health research to study the effectiveness of medical treatments, identify risk factors for diseases, and track health outcomes over time. Researchers use statistical methods to analyze data from clinical trials, surveys, and other sources to inform medical practice and policy.
• Social Science Research: Quantitative research is used in social science research to study human behavior, attitudes, and social structures. Researchers use surveys, experiments, and other quantitative methods to collect data that can inform social policies, educational programs, and community interventions.
• Education Research: Quantitative research is used in education research to study the effectiveness of teaching methods, assess student learning outcomes, and identify factors that influence student success. Researchers use experimental and quasi-experimental designs, as well as surveys and other quantitative methods, to collect and analyze data.
• Environmental Research: Quantitative research is used in environmental research to study the impact of human activities on the environment, assess the effectiveness of conservation strategies, and identify ways to reduce environmental risks. Researchers use statistical methods to analyze data from field studies, experiments, and other sources.

Characteristics of Quantitative Research

Here are some key characteristics of quantitative research:

• Numerical data : Quantitative research involves collecting numerical data through standardized methods such as surveys, experiments, and observational studies. This data is analyzed using statistical methods to identify patterns and relationships.
• Large sample size: Quantitative research often involves collecting data from a large sample of individuals or groups in order to increase the reliability and generalizability of the findings.
• Objective approach: Quantitative research aims to be objective and impartial in its approach, focusing on the collection and analysis of data rather than personal beliefs, opinions, or experiences.
• Control over variables: Quantitative research often involves manipulating variables to test hypotheses and establish cause-and-effect relationships. Researchers aim to control for extraneous variables that may impact the results.
• Replicable : Quantitative research aims to be replicable, meaning that other researchers should be able to conduct similar studies and obtain similar results using the same methods.
• Statistical analysis: Quantitative research involves using statistical tools and techniques to analyze the numerical data collected during the research process. Statistical analysis allows researchers to identify patterns, trends, and relationships between variables, and to test hypotheses and theories.
• Generalizability: Quantitative research aims to produce findings that can be generalized to larger populations beyond the specific sample studied. This is achieved through the use of random sampling methods and statistical inference.

Examples of Quantitative Research

Here are some examples of quantitative research in different fields:

• Market Research: A company conducts a survey of 1000 consumers to determine their brand awareness and preferences. The data is analyzed using statistical methods to identify trends and patterns that can inform marketing strategies.
• Health Research : A researcher conducts a randomized controlled trial to test the effectiveness of a new drug for treating a particular medical condition. The study involves collecting data from a large sample of patients and analyzing the results using statistical methods.
• Social Science Research : A sociologist conducts a survey of 500 people to study attitudes toward immigration in a particular country. The data is analyzed using statistical methods to identify factors that influence these attitudes.
• Education Research: A researcher conducts an experiment to compare the effectiveness of two different teaching methods for improving student learning outcomes. The study involves randomly assigning students to different groups and collecting data on their performance on standardized tests.
• Environmental Research : A team of researchers conduct a study to investigate the impact of climate change on the distribution and abundance of a particular species of plant or animal. The study involves collecting data on environmental factors and population sizes over time and analyzing the results using statistical methods.
• Psychology : A researcher conducts a survey of 500 college students to investigate the relationship between social media use and mental health. The data is analyzed using statistical methods to identify correlations and potential causal relationships.
• Political Science: A team of researchers conducts a study to investigate voter behavior during an election. They use survey methods to collect data on voting patterns, demographics, and political attitudes, and analyze the results using statistical methods.

How to Conduct Quantitative Research

Here is a general overview of how to conduct quantitative research:

• Develop a research question: The first step in conducting quantitative research is to develop a clear and specific research question. This question should be based on a gap in existing knowledge, and should be answerable using quantitative methods.
• Develop a research design: Once you have a research question, you will need to develop a research design. This involves deciding on the appropriate methods to collect data, such as surveys, experiments, or observational studies. You will also need to determine the appropriate sample size, data collection instruments, and data analysis techniques.
• Collect data: The next step is to collect data. This may involve administering surveys or questionnaires, conducting experiments, or gathering data from existing sources. It is important to use standardized methods to ensure that the data is reliable and valid.
• Analyze data : Once the data has been collected, it is time to analyze it. This involves using statistical methods to identify patterns, trends, and relationships between variables. Common statistical techniques include correlation analysis, regression analysis, and hypothesis testing.
• Interpret results: After analyzing the data, you will need to interpret the results. This involves identifying the key findings, determining their significance, and drawing conclusions based on the data.
• Communicate findings: Finally, you will need to communicate your findings. This may involve writing a research report, presenting at a conference, or publishing in a peer-reviewed journal. It is important to clearly communicate the research question, methods, results, and conclusions to ensure that others can understand and replicate your research.

When to use Quantitative Research

Here are some situations when quantitative research can be appropriate:

• To test a hypothesis: Quantitative research is often used to test a hypothesis or a theory. It involves collecting numerical data and using statistical analysis to determine if the data supports or refutes the hypothesis.
• To generalize findings: If you want to generalize the findings of your study to a larger population, quantitative research can be useful. This is because it allows you to collect numerical data from a representative sample of the population and use statistical analysis to make inferences about the population as a whole.
• To measure relationships between variables: If you want to measure the relationship between two or more variables, such as the relationship between age and income, or between education level and job satisfaction, quantitative research can be useful. It allows you to collect numerical data on both variables and use statistical analysis to determine the strength and direction of the relationship.
• To identify patterns or trends: Quantitative research can be useful for identifying patterns or trends in data. For example, you can use quantitative research to identify trends in consumer behavior or to identify patterns in stock market data.
• To quantify attitudes or opinions : If you want to measure attitudes or opinions on a particular topic, quantitative research can be useful. It allows you to collect numerical data using surveys or questionnaires and analyze the data using statistical methods to determine the prevalence of certain attitudes or opinions.

Purpose of Quantitative Research

The purpose of quantitative research is to systematically investigate and measure the relationships between variables or phenomena using numerical data and statistical analysis. The main objectives of quantitative research include:

• Description : To provide a detailed and accurate description of a particular phenomenon or population.
• Explanation : To explain the reasons for the occurrence of a particular phenomenon, such as identifying the factors that influence a behavior or attitude.
• Prediction : To predict future trends or behaviors based on past patterns and relationships between variables.
• Control : To identify the best strategies for controlling or influencing a particular outcome or behavior.

Quantitative research is used in many different fields, including social sciences, business, engineering, and health sciences. It can be used to investigate a wide range of phenomena, from human behavior and attitudes to physical and biological processes. The purpose of quantitative research is to provide reliable and valid data that can be used to inform decision-making and improve understanding of the world around us.

Advantages of Quantitative Research

There are several advantages of quantitative research, including:

• Objectivity : Quantitative research is based on objective data and statistical analysis, which reduces the potential for bias or subjectivity in the research process.
• Reproducibility : Because quantitative research involves standardized methods and measurements, it is more likely to be reproducible and reliable.
• Generalizability : Quantitative research allows for generalizations to be made about a population based on a representative sample, which can inform decision-making and policy development.
• Precision : Quantitative research allows for precise measurement and analysis of data, which can provide a more accurate understanding of phenomena and relationships between variables.
• Efficiency : Quantitative research can be conducted relatively quickly and efficiently, especially when compared to qualitative research, which may involve lengthy data collection and analysis.
• Large sample sizes : Quantitative research can accommodate large sample sizes, which can increase the representativeness and generalizability of the results.

Limitations of Quantitative Research

There are several limitations of quantitative research, including:

• Limited understanding of context: Quantitative research typically focuses on numerical data and statistical analysis, which may not provide a comprehensive understanding of the context or underlying factors that influence a phenomenon.
• Simplification of complex phenomena: Quantitative research often involves simplifying complex phenomena into measurable variables, which may not capture the full complexity of the phenomenon being studied.
• Potential for researcher bias: Although quantitative research aims to be objective, there is still the potential for researcher bias in areas such as sampling, data collection, and data analysis.
• Limited ability to explore new ideas: Quantitative research is often based on pre-determined research questions and hypotheses, which may limit the ability to explore new ideas or unexpected findings.
• Limited ability to capture subjective experiences : Quantitative research is typically focused on objective data and may not capture the subjective experiences of individuals or groups being studied.
• Ethical concerns : Quantitative research may raise ethical concerns, such as invasion of privacy or the potential for harm to participants.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Questionnaire – Definition, Types, and Examples

Case Study – Methods, Examples and Guide

Observational Research – Methods and Guide

Qualitative Research Methods

Explanatory Research – Types, Methods, Guide

Survey Research – Types, Methods, Examples

Find the top authors in a research field: What you need to know

Joanna Wilkinson

Our blog to help you find top authors is part of our Research Smarter series. This series is dedicated to helping you get familiar with your research field. Download our cheat sheet , which brings together top tips for finding relevant journals, papers, and authors in a field. You can also read the related blog posts for each, here .

Before starting any academic writing effort, it is important to get familiar with the shining stars in your research field. The established, prolific and emerging researchers each contribute to the scholarly conversation. It’s up to you to learn who they are and what they achieved in order to further their ideas.

The Web of Science™ makes it easy to find these leading authors, and confidently assess their output and associated citation impact thanks to the built-in citation network. This blog will help you uncover:

• Specific researchers and their papers
• The most prolific authors in your field
• Authors of Hot and Highly Cited Papers
• An author’s impact as a researcher, editor and reviewer

Find specific researchers and their paper

  You will likely have a few researchers in mind when beginning your author search. The Web of Science’s Author Search helps you easily find researchers and their publications in the Web of Science Core Collection™ – regardless of how common or complex their name, or how their name may be presented in different publications over time.

Start your search by entering the full name of the researcher you’re looking for or enter their ResearcherID or ORCiD. These are persistent author identifiers that help to distinguish researchers from their peers, and to ensure a researcher’s work is correctly attributed to them.

You will then be taken to the researcher’s Author Record. This is an initial snapshot of the author’s publications, citation impact and citing articles in the Core Collection, which is based on the intersection of a machine learning algorithm and human curation via Publons™ and ORCID .

Author Records enable you to discover a researcher’s published work and assess their output and associated impact. If you want to dig even deeper, simply click the “View as a set of results to export, analyze, and link to full text” button at the top of the page. You will be directed to a new results page where you can filter and refine the author’s publications, as well as make use of the Analyze Results tool, all of which we discuss below.

You can learn more about Author Search via our Quick Reference Guide or try it out now.

Find the most prolific authors in your field

The Analyze Results tool in the Web of Science helps you discover who the established authors are in your field. They also help you dig deeper into any known and specific author. You can discover the top authors for any keyword search, their organizational affiliation and even the funding agencies they’re most commonly associated with. This tool also reveals the top co-authors for any researchers or group authors for any topic. You can also analyze your results by timespan, which delivers insight into an author’s full body of work.

Access Analyze Results from the results page of any Web of Science keyword search or via an Author Record as described in the section above. Learn more about the tool’s capabilities   or by watching the video below.

Want to gain access to the Web of Science for your organization? Contact our sales team .

Discover top authors of Hot and Highly Cited Papers in your field

Citation counts are a traditional and key measure of scholarly impact. While no metric is perfect in understanding the influence of a researcher as a whole, citations can help. Citations enable you to gauge the research community’s interest in a given paper. They also help signal the impact this work has within a particular discipline or across fields. This impact can either be recent and highlighted as a Hot Paper, or more sustained as a Highly Cited Paper™.

To view the authors of both Highly Cited and Hot Papers, simply:

• Start a Topic Search
• Refine your results by Hot or Highly Cited Papers (located on the left-hand side)
• Analyze results by author ( Find out more about this here).

Uncover an author’s impact as a researcher, editor and reviewer

We mentioned earlier that no metric is perfect in measuring the impact of an author’s complete body of work. Each metric has a purpose and the key is knowing what each metric can measure and what it cannot. Rather than limiting your author search to papers and citations, you can see a more complete picture of an author’s contribution to their field on Publons . Publons is a free platform for researchers to track their publications, citations, peer review metrics and journal editing work. You can view a summary of all researchers’ work on their public profile. This includes their engagement in open and transparent peer review.

Publons profiles also list certain accolades that will help you understand a researcher’s influence at a glance. One is the Highly Cited Researcher™ recognition , presented annually by Clarivate to recognize the world’s most influential researchers. This is demonstrated by the production of multiple highly-cited papers that rank in the top 1% by citations for field and year in the Web of Science. Furthermore, you’ll also see the global Peer Review Awards listed on various profiles. This award recognizes the quantity and quality of researchers’ peer review efforts in their field over the last 12 months.

Browse researchers on Publons today or learn more about creating your own Publons profile in the video below.

Stay connected

Interested in discovering more tips to really understand your research field? You can download our cheat sheet , which brings together top tips for finding relevant journals, papers and authors in your field. You can also read the related blog posts in our Research Smarter series, here.

If you don’t have access to the Web of Science and want to enquire on behalf of your organization, please contact our sales team .

Related posts

Demonstrating socioeconomic impact – a historical perspective of ancient wisdom and modern challenges.

Unlocking U.K. research excellence: Key insights from the Research Professional News Live summit

For better insights, assess research performance at the department level

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• HHS Author Manuscripts

Appraising Quantitative Research in Health Education: Guidelines for Public Health Educators

Leonard jack, jr..

Associate Dean for Research and Endowed Chair of Minority Health Disparities, College of Pharmacy, Xavier University of Louisiana, 1 Drexel Drive, New Orleans, Louisiana 70125; Telephone: 504-520-5345; Fax: 504-520-7971

Sandra C. Hayes

Central Mississippi Area Health Education Center, 350 West Woodrow Wilson, Suite 3320, Jackson, MS 39213; Telephone: 601-987-0272; Fax: 601-815-5388

Jeanfreau G. Scharalda

Louisiana State University Health Sciences Center School of Nursing, 1900 Gravier Street, New Orleans, Louisiana 70112; Telephone: 504-568-4140; Fax: 504-568-5853

Barbara Stetson

Department of Psychological and Brain Sciences, 317 Life Sciences Building, University of Louisville, Louisville, KY 40292; Telephone: 502-852-2540; Fax: 502-852-8904

Nkenge H. Jones-Jack

Epidemiologist & Evaluation Consultant, Metairie, Louisiana 70002. Telephone: 678-524-1147; Fax: 504-267-4080

Matthew Valliere

Chronic Disease Prevention and Control, Bureau of Primary Care and Rural Health, Office of the Secretary, 628 North 4th Street, Baton Rouge, LA 70821-3118; Telephone: 225-342-2655; Fax: 225-342-2652

William R. Kirchain

Division of Clinical and Administrative Sciences, College of Pharmacy, Xavier University of Louisiana, 1 Drexel Drive, Room 121, New Orleans, Louisiana 70125; Telephone: 504-520-5395; Fax: 504-520-7971

Michael Fagen

Co-Associate Editor for the Evaluation and Practice section of Health Promotion Practice , Department of Community Health Sciences, School of Public Health, University of Illinois at Chicago, 1603 W. Taylor St., M/C 923, Chicago, IL 60608-1260, Telephone: 312-355-0647; Fax: 312-996-3551

Cris LeBlanc

Centers of Excellence Scholar, College of Pharmacy, Xavier University of Louisiana, 1 Drexel Drive, New Orleans, Louisiana 70125; Telephone: 504-520-5345; Fax: 504-520-7971

Many practicing health educators do not feel they possess the skills necessary to critically appraise quantitative research. This publication is designed to help provide practicing health educators with basic tools helpful to facilitate a better understanding of quantitative research. This article describes the major components—title, introduction, methods, analyses, results and discussion sections—of quantitative research. Readers will be introduced to information on the various types of study designs and seven key questions health educators can use to facilitate the appraisal process. Upon reading, health educators will be in a better position to determine whether research studies are well designed and executed.

Appraising the Quality of Quantitative Research in Health Education

Practicing health educators often find themselves with little time to read published research in great detail. Some health educators with limited time to read scientific papers may get frustrated as they get bogged down trying to understand research terminology, methods, and approaches. The purpose of appraising a scientific publication is to assess whether the study’s research questions (hypotheses), methods and results (findings) are sufficiently valid to produce useful information ( Fowkes and Fulton, 1991 ; Donnelly, 2004 ; Greenhalgh and Taylor, 1997 ; Johnson and Onwuegbuze, 2004 ; Greenhalgh, 1997 ; Yin, 2003; and Hennekens and Buring, 1987 ). Having the ability to deconstruct and reconstruct scientific publications is a critical skill in a results-oriented environment linked to increasing demands and expectations for improved program outcomes and strong justifications to program focus and direction. Health educators do must not solely rely on the opinions of researchers, but, rather, increase their confidence in their own abilities to discern the quality of published scientific research. Health educators with little experience reading and appraising scientific publications, may find this task less difficult if they: 1) become more familiar with the key components of a research publication, and 2) utilize questions presented in this article to critically appraise the strengths and weaknesses of published research.

Key Components of a Scientific Research Publication

The key components of a research publication should provide important information that is needed to assess the strengths and weaknesses of the research. Key components typically include the: publication title , abstract , introduction , research methods used to address the research question(s) or hypothesis, statistical analysis used, results , and the researcher’s interpretation and conclusion or recommended use of results to inform future research or practice. A brief description of these components follows:

Publication Title

A general heading or description should provide immediate insight into the intent of the research. Titles may include information regarding the focus of the research, population or target audience being studied, and study design.

An abstract provides the reader with a brief description of the overall research, how it was done, statistical techniques employed, key results,and relevant implications or recommendations.

Introduction

This section elaborates on the content mentioned in the abstract and provides a better idea of what to anticipate in the manuscript. The introduction provides a succinct presentation of previously published literature, thus offering a purpose (rationale) for the study.

This component of the publication provides critical information on the type of research methods used to conduct the study. Common examples of study designs used to conduct quantitative research include cross sectional study, cohort study, case-control study, and controlled trial. The methods section should contain information on the inclusion and exclusion criteria used to identify participants in the study.

Quantitative data contains information that is quantifiable, perhaps through surveys that are analyzed using statistical tests to determine if the results happened by chance. Two types of statistical analyses are used: descriptive and inferential ( Johnson and Onwuegbuze, 2004 ). Descriptive statistics are used to describe the basic features of the study data and provide simple summaries about the sample and measures. With inferential statistics, researchers are trying to reach conclusions that extend beyond the immediate data alone. Thus, they use inferential statistics to make inferences from the data to more general conditions.

This section presents the reader with the researcher’s data and results of statistical analyses described in the method section. Thus, this section must align closely with the methods section.

Discussion (Conclusion)

This section should explain what the data means thereby summarizing main results and findings for the reader. Important limitations (such as the use of a non-random sample, the absence of a control group, and short duration of the intervention) should be discussed. Researchers should discuss how each limitation can impact the applicability and use of study results. This section also presents recommendations on ways the study can help advance future health education and practice.

Critically Appraising the Strengths and Weaknesses of Published Research

During careful reading of the analysis, results, and discussion (conclusion) sections, what key questions might you ask yourself in order to critically appraise the strengths and weaknesses of the research? Based on a careful review of the literature ( Greenhalgh and Taylor, 1997 ; Greenhalgh, 1997 ; and Hennekens and Buring, 1987 ) and our research experiences, we have identified seven key questions around which to guide your assessment of quantitative research.

1) Is a study design identified and appropriately applied?

Study designs refer to the methodology used to investigate a particular health phenomenon. Becoming familiar with the various study designs will help prepare you to critically assess whether its selection was applied adequately to answer the research questions (or hypotheses). As mentioned previously, common examples of study designs frequently used to conduct quantitative research include cross sectional study, cohort study, case-control study, and controlled trail. A brief description of each can be found in Table 1 .

Definitions of Study Designs

2) Is the study sample representative of the group from which it is drawn?

The study sample must be representative of the group from which it is drawn. The study sample must therefore be typical of the wider target audience to whom the research might apply. Addressing whether the study sample is representative of the group from which it is drawn will require the researcher to take into consideration the sampling method and sample size.

Sampling Method

Many sampling methods are used individually or in combination. Keep in mind that sampling methods are divided into two categories: probability sampling and non-probability sampling ( Last, 2001 ). Probability sampling (also called random sampling) is any sampling scheme in which the probability of choosing each individual is the same (or at least known, so it can be readjusted mathematically to be equal). Non-probability sampling is any sampling scheme in which the probability of an individual being chosen is unknown. Typically, researchers should offer a rationale for utilizing non-probability sampling, and when utilized, be aware of its limitations. For example, use of a convenience sample (choosing individuals in an unstructured manner) can be justified when collecting pilot data around which future studies employing more rigorous sampling methods will be utilized.

Sample Size

Established statistical theories and formulas are used to generate sample size calculations—the recommended number of individuals necessary in order to have sufficient power to detect meaningful results at a certain level of statistical significance. In the methods section, look for a statement or two confirming whether steps where taken to obtain the appropriate sample size.

3) In research studies using a control group, is this group adequate for the purpose of the study?

Source of controls.

In case-control and cohort studies, the source of controls should be such that the distribution of characteristics not under investigation are similar to those in the cases or study cohort.

In case-control studies both cases and controls are often matched on certain characteristics such as age, sex, income, and race. The criteria used for including and excluding study participants must be adequately described and examined carefully. Inclusion and exclusion criteria may include: ethnicity, age of diagnosis, length of time living with a health condition, geographic location, and presence or absence of complications. You should critically assess whether matching across these characteristics actually occurred.

4) What is the validity of measurements and outcomes identified in the study?

Validity is the extent to which a measurement captures what it claims to measure. This might take the form of questions contained on a survey, questionnaire or instrument. Researchers should address one or more of the following types of validity: face, content, criterion-related, and construct ( Last, 2001 ; William and Donnelly, 2008).

Face validity

Face validity assures that, upon examination, the variable of interest can measure what it intends to measure. If the researcher has chosen to study a variable that has not been studied before, he/she usually will need to start with face validity.

Content validity

Content validity involves comparing the content of the measurement technique to the known literature on the topic and validating the fact that the tool (e.g., survey, questionnaire) does represent the literature accurately.

Criterion-related validity

Criterion-related validity involves making sure the measures within a survey when tested proves to be effective in predicting criterion or indicators of a construct.

Construct validity

Construct validity deals with the validation of the construct that underlies the research. Here, researchers test the theory that underlies the hypothesis or research question.

5) To what extent is a common source of bias called blindness taken into account?

During data collection, a common source of bias is that subjects and/or those collecting the data are not blind to the purpose of the research. This can likely be the result of researchers going the extra mile to make sure those in the experimental group benefit from the intervention ( Fowkes and Fulton, 1991 ). Inadequate blindness can be a problem in studies utilizing all types of study designs. While total blindness is not possible, appraising whether steps were taken to be sure issues related to ensure blindness occurred is essential.

6) To what extent is the study considered complete with regard to drop outs and missing data?

Regardless of the study design employed, one must assess not only the proportion of drop outs in each group, but also why they dropped out. This may point to possible bias, as well as determine what efforts were taken to retain participants in the study.

Missing data

Despite the fact that missing data are a part of almost all research, it should still be appraised. There are several reasons why the data may be missing. The nature and extent to which data is missing should be explained.

7) To what extent are study results influenced by factors that negatively impact their credibility?

Contamination.

In research studies comparing the effectiveness of a structured intervention, contamination occurs when the control group makes changes based on learning what those participating in the intervention are doing. Despite the fact that researchers typically do not report the extent to which contamination occurs, you should nevertheless try to assess whether contamination negatively impacted the credibility of study results.

Confounding factors

A confounding factor in a study is a variable which is related to one or more of the measurements (measures or variables) defined in a study. A confounding factor may mask an actual association or falsely demonstrate an apparent association between the study variables where no real association between them exists. If confounding factors are not measured and considered, study results may be biased and compromised.

The guidelines and questions presented in this article are by no means exhaustive. However, when applied, they can help health education practitioners obtain a deeper understanding of the quality of published research. While no study is 100% perfect, we do encourage health education practitioners to pause before taking researchers at their word that study results are both accurate and impressive. If you find yourself answering ‘no’ to a majority of the key questions provided, then it is probably safe to say that, from your perspective, the quality of the research is questionable.

Over time, as you repeatedly apply the guidelines presented in this article, you will become more confident and interested in reading research publications from beginning to end. While this article is geared to health educators, it can help anyone interested in learning how to appraise published research. Table 2 lists additional reading resources that can help improve one’s understanding and knowledge of quantitative research. This article and the reading resources identified in Table 2 can serve as useful tools to frame informative conversations with your peers regarding the strengths and weaknesses of published quantitative research in health education.

Publications on How to Read, Write and Appraise Quantitative Research

Contributor Information

Leonard Jack, Jr., Associate Dean for Research and Endowed Chair of Minority Health Disparities, College of Pharmacy, Xavier University of Louisiana, 1 Drexel Drive, New Orleans, Louisiana 70125; Telephone: 504-520-5345; Fax: 504-520-7971.

Sandra C. Hayes, Central Mississippi Area Health Education Center, 350 West Woodrow Wilson, Suite 3320, Jackson, MS 39213; Telephone: 601-987-0272; Fax: 601-815-5388.

Jeanfreau G. Scharalda, Louisiana State University Health Sciences Center School of Nursing, 1900 Gravier Street, New Orleans, Louisiana 70112; Telephone: 504-568-4140; Fax: 504-568-5853.

Barbara Stetson, Department of Psychological and Brain Sciences, 317 Life Sciences Building, University of Louisville, Louisville, KY 40292; Telephone: 502-852-2540; Fax: 502-852-8904.

Nkenge H. Jones-Jack, Epidemiologist & Evaluation Consultant, Metairie, Louisiana 70002. Telephone: 678-524-1147; Fax: 504-267-4080.

Matthew Valliere, Chronic Disease Prevention and Control, Bureau of Primary Care and Rural Health, Office of the Secretary, 628 North 4th Street, Baton Rouge, LA 70821-3118; Telephone: 225-342-2655; Fax: 225-342-2652.

William R. Kirchain, Division of Clinical and Administrative Sciences, College of Pharmacy, Xavier University of Louisiana, 1 Drexel Drive, Room 121, New Orleans, Louisiana 70125; Telephone: 504-520-5395; Fax: 504-520-7971.

Michael Fagen, Co-Associate Editor for the Evaluation and Practice section of Health Promotion Practice , Department of Community Health Sciences, School of Public Health, University of Illinois at Chicago, 1603 W. Taylor St., M/C 923, Chicago, IL 60608-1260, Telephone: 312-355-0647; Fax: 312-996-3551.

Cris LeBlanc, Centers of Excellence Scholar, College of Pharmacy, Xavier University of Louisiana, 1 Drexel Drive, New Orleans, Louisiana 70125; Telephone: 504-520-5345; Fax: 504-520-7971.

• Fowkes FG, Fulton PM. Critical appraisal of published research: introductory guidelines. British Medical Journal. 1991; 302 :1136–40. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Donnelly RA. The Complete Idiots Guide to Statistics. Alpha Books; New York, NY: 2004. pp. 6–7. [ Google Scholar ]
• Greenhalgh T, Taylor R. How to read a paper: Papers that go beyond numbers (qualitative research) British Medical Journal. 1997; 315 :740–743. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Greenhalgh T. How to read a paper: Assessing the methodological quality of published papers. British Medical Journal. 315 :305–308. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Johnson RB, Onwuegbuze AJ. Mixed methods research: A research paradigm whose time has come. Educational Researcher. 2004; 33 :14–26. [ Google Scholar ]
• Hennekens CH, Buring JE. Epidemiology in Medicine. Little, Brown and Company; Boston, Massachusetts: 1987. pp. 106–108. [ Google Scholar ]
• Last JM. A dictionary of epidemiology. 4. Oxford University Press, Inc; New York, New York: 2001. [ Google Scholar ]
• Trochim WM, Donnelly J. Research methods knowledge base. 3. Atomic Dog; Mason, Ohio: 2008. pp. 6–8. [ Google Scholar ]

Advertisement

• Previous Issue
• Previous Article
• Next Article

Clarifying the Research Purpose

Methodology, measurement, data analysis and interpretation, tools for evaluating the quality of medical education research, research support, competing interests, quantitative research methods in medical education.

Submitted for publication January 8, 2018. Accepted for publication November 29, 2018.

• Split-Screen
• Article contents
• Figures & tables
• Supplementary Data
• Peer Review
• Open the PDF for in another window
• Cite Icon Cite
• Get Permissions
• Search Site

John T. Ratelle , Adam P. Sawatsky , Thomas J. Beckman; Quantitative Research Methods in Medical Education. Anesthesiology 2019; 131:23–35 doi: https://doi.org/10.1097/ALN.0000000000002727

Download citation file:

• Ris (Zotero)
• Reference Manager

There has been a dramatic growth of scholarly articles in medical education in recent years. Evaluating medical education research requires specific orientation to issues related to format and content. Our goal is to review the quantitative aspects of research in medical education so that clinicians may understand these articles with respect to framing the study, recognizing methodologic issues, and utilizing instruments for evaluating the quality of medical education research. This review can be used both as a tool when appraising medical education research articles and as a primer for clinicians interested in pursuing scholarship in medical education.

Image: J. P. Rathmell and Terri Navarette.

There has been an explosion of research in the field of medical education. A search of PubMed demonstrates that more than 40,000 articles have been indexed under the medical subject heading “Medical Education” since 2010, which is more than the total number of articles indexed under this heading in the 1980s and 1990s combined. Keeping up to date requires that practicing clinicians have the skills to interpret and appraise the quality of research articles, especially when serving as editors, reviewers, and consumers of the literature.

While medical education shares many characteristics with other biomedical fields, substantial particularities exist. We recognize that practicing clinicians may not be familiar with the nuances of education research and how to assess its quality. Therefore, our purpose is to provide a review of quantitative research methodologies in medical education. Specifically, we describe a structure that can be used when conducting or evaluating medical education research articles.

Clarifying the research purpose is an essential first step when reading or conducting scholarship in medical education. 1   Medical education research can serve a variety of purposes, from advancing the science of learning to improving the outcomes of medical trainees and the patients they care for. However, a well-designed study has limited value if it addresses vague, redundant, or unimportant medical education research questions.

What is the research topic and why is it important? What is unknown about the research topic? Why is further research necessary?

What is the conceptual framework being used to approach the study?

What is the statement of study intent?

What are the research methodology and study design? Are they appropriate for the study objective(s)?

Which threats to internal validity are most relevant for the study?

What is the outcome and how was it measured?

Can the results be trusted? What is the validity and reliability of the measurements?

How were research subjects selected? Is the research sample representative of the target population?

Was the data analysis appropriate for the study design and type of data?

What is the effect size? Do the results have educational significance?

Fortunately, there are steps to ensure that the purpose of a research study is clear and logical. Table 1   2–5   outlines these steps, which will be described in detail in the following sections. We describe these elements not as a simple “checklist,” but as an advanced organizer that can be used to understand a medical education research study. These steps can also be used by clinician educators who are new to the field of education research and who wish to conduct scholarship in medical education.

Steps in Clarifying the Purpose of a Research Study in Medical Education

Literature Review and Problem Statement

A literature review is the first step in clarifying the purpose of a medical education research article. 2 , 5 , 6   When conducting scholarship in medical education, a literature review helps researchers develop an understanding of their topic of interest. This understanding includes both existing knowledge about the topic as well as key gaps in the literature, which aids the researcher in refining their study question. Additionally, a literature review helps researchers identify conceptual frameworks that have been used to approach the research topic. 2  

When reading scholarship in medical education, a successful literature review provides background information so that even someone unfamiliar with the research topic can understand the rationale for the study. Located in the introduction of the manuscript, the literature review guides the reader through what is already known in a manner that highlights the importance of the research topic. The literature review should also identify key gaps in the literature so the reader can understand the need for further research. This gap description includes an explicit problem statement that summarizes the important issues and provides a reason for the study. 2 , 4   The following is one example of a problem statement:

“Identifying gaps in the competency of anesthesia residents in time for intervention is critical to patient safety and an effective learning system… [However], few available instruments relate to complex behavioral performance or provide descriptors…that could inform subsequent feedback, individualized teaching, remediation, and curriculum revision.” 7  

This problem statement articulates the research topic (identifying resident performance gaps), why it is important (to intervene for the sake of learning and patient safety), and current gaps in the literature (few tools are available to assess resident performance). The researchers have now underscored why further research is needed and have helped readers anticipate the overarching goals of their study (to develop an instrument to measure anesthesiology resident performance). 4  

The Conceptual Framework

Following the literature review and articulation of the problem statement, the next step in clarifying the research purpose is to select a conceptual framework that can be applied to the research topic. Conceptual frameworks are “ways of thinking about a problem or a study, or ways of representing how complex things work.” 3   Just as clinical trials are informed by basic science research in the laboratory, conceptual frameworks often serve as the “basic science” that informs scholarship in medical education. At a fundamental level, conceptual frameworks provide a structured approach to solving the problem identified in the problem statement.

Conceptual frameworks may take the form of theories, principles, or models that help to explain the research problem by identifying its essential variables or elements. Alternatively, conceptual frameworks may represent evidence-based best practices that researchers can apply to an issue identified in the problem statement. 3   Importantly, there is no single best conceptual framework for a particular research topic, although the choice of a conceptual framework is often informed by the literature review and knowing which conceptual frameworks have been used in similar research. 8   For further information on selecting a conceptual framework for research in medical education, we direct readers to the work of Bordage 3   and Irby et al. 9  

To illustrate how different conceptual frameworks can be applied to a research problem, suppose you encounter a study to reduce the frequency of communication errors among anesthesiology residents during day-to-night handoff. Table 2 10 , 11   identifies two different conceptual frameworks researchers might use to approach the task. The first framework, cognitive load theory, has been proposed as a conceptual framework to identify potential variables that may lead to handoff errors. 12   Specifically, cognitive load theory identifies the three factors that affect short-term memory and thus may lead to communication errors:

Conceptual Frameworks to Address the Issue of Handoff Errors in the Intensive Care Unit

Intrinsic load: Inherent complexity or difficulty of the information the resident is trying to learn ( e.g. , complex patients).

Extraneous load: Distractions or demands on short-term memory that are not related to the information the resident is trying to learn ( e.g. , background noise, interruptions).

Germane load: Effort or mental strategies used by the resident to organize and understand the information he/she is trying to learn ( e.g. , teach back, note taking).

Using cognitive load theory as a conceptual framework, researchers may design an intervention to reduce extraneous load and help the resident remember the overnight to-do’s. An example might be dedicated, pager-free handoff times where distractions are minimized.

The second framework identified in table 2 , the I-PASS (Illness severity, Patient summary, Action list, Situational awareness and contingency planning, and Synthesis by receiver) handoff mnemonic, 11   is an evidence-based best practice that, when incorporated as part of a handoff bundle, has been shown to reduce handoff errors on pediatric wards. 13   Researchers choosing this conceptual framework may adapt some or all of the I-PASS elements for resident handoffs in the intensive care unit.

Note that both of the conceptual frameworks outlined above provide researchers with a structured approach to addressing the issue of handoff errors; one is not necessarily better than the other. Indeed, it is possible for researchers to use both frameworks when designing their study. Ultimately, we provide this example to demonstrate the necessity of selecting conceptual frameworks to clarify the research purpose. 3 , 8   Readers should look for conceptual frameworks in the introduction section and should be wary of their omission, as commonly seen in less well-developed medical education research articles. 14  

Statement of Study Intent

After reviewing the literature, articulating the problem statement, and selecting a conceptual framework to address the research topic, the final step in clarifying the research purpose is the statement of study intent. The statement of study intent is arguably the most important element of framing the study because it makes the research purpose explicit. 2   Consider the following example:

This study aimed to test the hypothesis that the introduction of the BASIC Examination was associated with an accelerated knowledge acquisition during residency training, as measured by increments in annual ITE scores. 15  

This statement of study intent succinctly identifies several key study elements including the population (anesthesiology residents), the intervention/independent variable (introduction of the BASIC Examination), the outcome/dependent variable (knowledge acquisition, as measure by in In-training Examination [ITE] scores), and the hypothesized relationship between the independent and dependent variable (the authors hypothesize a positive correlation between the BASIC examination and the speed of knowledge acquisition). 6 , 14  

The statement of study intent will sometimes manifest as a research objective, rather than hypothesis or question. In such instances there may not be explicit independent and dependent variables, but the study population and research aim should be clearly identified. The following is an example:

“In this report, we present the results of 3 [years] of course data with respect to the practice improvements proposed by participating anesthesiologists and their success in implementing those plans. Specifically, our primary aim is to assess the frequency and type of improvements that were completed and any factors that influence completion.” 16  

The statement of study intent is the logical culmination of the literature review, problem statement, and conceptual framework, and is a transition point between the Introduction and Methods sections of a medical education research report. Nonetheless, a systematic review of experimental research in medical education demonstrated that statements of study intent are absent in the majority of articles. 14   When reading a medical education research article where the statement of study intent is absent, it may be necessary to infer the research aim by gathering information from the Introduction and Methods sections. In these cases, it can be useful to identify the following key elements 6 , 14 , 17   :

Population of interest/type of learner ( e.g. , pain medicine fellow or anesthesiology residents)

Independent/predictor variable ( e.g. , educational intervention or characteristic of the learners)

Dependent/outcome variable ( e.g. , intubation skills or knowledge of anesthetic agents)

Relationship between the variables ( e.g. , “improve” or “mitigate”)

Occasionally, it may be difficult to differentiate the independent study variable from the dependent study variable. 17   For example, consider a study aiming to measure the relationship between burnout and personal debt among anesthesiology residents. Do the researchers believe burnout might lead to high personal debt, or that high personal debt may lead to burnout? This “chicken or egg” conundrum reinforces the importance of the conceptual framework which, if present, should serve as an explanation or rationale for the predicted relationship between study variables.

Research methodology is the “…design or plan that shapes the methods to be used in a study.” 1   Essentially, methodology is the general strategy for answering a research question, whereas methods are the specific steps and techniques that are used to collect data and implement the strategy. Our objective here is to provide an overview of quantitative methodologies ( i.e. , approaches) in medical education research.

The choice of research methodology is made by balancing the approach that best answers the research question against the feasibility of completing the study. There is no perfect methodology because each has its own potential caveats, flaws and/or sources of bias. Before delving into an overview of the methodologies, it is important to highlight common sources of bias in education research. We use the term internal validity to describe the degree to which the findings of a research study represent “the truth,” as opposed to some alternative hypothesis or variables. 18   Table 3   18–20   provides a list of common threats to internal validity in medical education research, along with tactics to mitigate these threats.

Threats to Internal Validity and Strategies to Mitigate Their Effects

Experimental Research

The fundamental tenet of experimental research is the manipulation of an independent or experimental variable to measure its effect on a dependent or outcome variable.

True Experiment

True experimental study designs minimize threats to internal validity by randomizing study subjects to experimental and control groups. Through ensuring that differences between groups are—beyond the intervention/variable of interest—purely due to chance, researchers reduce the internal validity threats related to subject characteristics, time-related maturation, and regression to the mean. 18 , 19  

Quasi-experiment

There are many instances in medical education where randomization may not be feasible or ethical. For instance, researchers wanting to test the effect of a new curriculum among medical students may not be able to randomize learners due to competing curricular obligations and schedules. In these cases, researchers may be forced to assign subjects to experimental and control groups based upon some other criterion beyond randomization, such as different classrooms or different sections of the same course. This process, called quasi-randomization, does not inherently lead to internal validity threats, as long as research investigators are mindful of measuring and controlling for extraneous variables between study groups. 19  

Single-group Methodologies

All experimental study designs compare two or more groups: experimental and control. A common experimental study design in medical education research is the single-group pretest–posttest design, which compares a group of learners before and after the implementation of an intervention. 21   In essence, a single-group pre–post design compares an experimental group ( i.e. , postintervention) to a “no-intervention” control group ( i.e. , preintervention). 19   This study design is problematic for several reasons. Consider the following hypothetical example: A research article reports the effects of a year-long intubation curriculum for first-year anesthesiology residents. All residents participate in monthly, half-day workshops over the course of an academic year. The article reports a positive effect on residents’ skills as demonstrated by a significant improvement in intubation success rates at the end of the year when compared to the beginning.

This study does little to advance the science of learning among anesthesiology residents. While this hypothetical report demonstrates an improvement in residents’ intubation success before versus after the intervention, it does not tell why the workshop worked, how it compares to other educational interventions, or how it fits in to the broader picture of anesthesia training.

Single-group pre–post study designs open themselves to a myriad of threats to internal validity. 20   In our hypothetical example, the improvement in residents’ intubation skills may have been due to other educational experience(s) ( i.e. , implementation threat) and/or improvement in manual dexterity that occurred naturally with time ( i.e. , maturation threat), rather than the airway curriculum. Consequently, single-group pre–post studies should be interpreted with caution. 18  

Repeated testing, before and after the intervention, is one strategy that can be used to reduce the some of the inherent limitations of the single-group study design. Repeated pretesting can mitigate the effect of regression toward the mean, a statistical phenomenon whereby low pretest scores tend to move closer to the mean on subsequent testing (regardless of intervention). 20   Likewise, repeated posttesting at multiple time intervals can provide potentially useful information about the short- and long-term effects of an intervention ( e.g. , the “durability” of the gain in knowledge, skill, or attitude).

Observational Research

Unlike experimental studies, observational research does not involve manipulation of any variables. These studies often involve measuring associations, developing psychometric instruments, or conducting surveys.

Association Research

Association research seeks to identify relationships between two or more variables within a group or groups (correlational research), or similarities/differences between two or more existing groups (causal–comparative research). For example, correlational research might seek to measure the relationship between burnout and educational debt among anesthesiology residents, while causal–comparative research may seek to measure differences in educational debt and/or burnout between anesthesiology and surgery residents. Notably, association research may identify relationships between variables, but does not necessarily support a causal relationship between them.

Psychometric and Survey Research

Psychometric instruments measure a psychologic or cognitive construct such as knowledge, satisfaction, beliefs, and symptoms. Surveys are one type of psychometric instrument, but many other types exist, such as evaluations of direct observation, written examinations, or screening tools. 22   Psychometric instruments are ubiquitous in medical education research and can be used to describe a trait within a study population ( e.g. , rates of depression among medical students) or to measure associations between study variables ( e.g. , association between depression and board scores among medical students).

Psychometric and survey research studies are prone to the internal validity threats listed in table 3 , particularly those relating to mortality, location, and instrumentation. 18   Additionally, readers must ensure that the instrument scores can be trusted to truly represent the construct being measured. For example, suppose you encounter a research article demonstrating a positive association between attending physician teaching effectiveness as measured by a survey of medical students, and the frequency with which the attending physician provides coffee and doughnuts on rounds. Can we be confident that this survey administered to medical students is truly measuring teaching effectiveness? Or is it simply measuring the attending physician’s “likability”? Issues related to measurement and the trustworthiness of data are described in detail in the following section on measurement and the related issues of validity and reliability.

Measurement refers to “the assigning of numbers to individuals in a systematic way as a means of representing properties of the individuals.” 23   Research data can only be trusted insofar as we trust the measurement used to obtain the data. Measurement is of particular importance in medical education research because many of the constructs being measured ( e.g. , knowledge, skill, attitudes) are abstract and subject to measurement error. 24   This section highlights two specific issues related to the trustworthiness of data: the validity and reliability of measurements.

Validity regarding the scores of a measurement instrument “refers to the degree to which evidence and theory support the interpretations of the [instrument’s results] for the proposed use of the [instrument].” 25   In essence, do we believe the results obtained from a measurement really represent what we were trying to measure? Note that validity evidence for the scores of a measurement instrument is separate from the internal validity of a research study. Several frameworks for validity evidence exist. Table 4 2 , 22 , 26   represents the most commonly used framework, developed by Messick, 27   which identifies sources of validity evidence—to support the target construct—from five main categories: content, response process, internal structure, relations to other variables, and consequences.

Sources of Validity Evidence for Measurement Instruments

Reliability

Reliability refers to the consistency of scores for a measurement instrument. 22 , 25 , 28   For an instrument to be reliable, we would anticipate that two individuals rating the same object of measurement in a specific context would provide the same scores. 25   Further, if the scores for an instrument are reliable between raters of the same object of measurement, then we can extrapolate that any difference in scores between two objects represents a true difference across the sample, and is not due to random variation in measurement. 29   Reliability can be demonstrated through a variety of methods such as internal consistency ( e.g. , Cronbach’s alpha), temporal stability ( e.g. , test–retest reliability), interrater agreement ( e.g. , intraclass correlation coefficient), and generalizability theory (generalizability coefficient). 22 , 29  

Example of a Validity and Reliability Argument

This section provides an illustration of validity and reliability in medical education. We use the signaling questions outlined in table 4 to make a validity and reliability argument for the Harvard Assessment of Anesthesia Resident Performance (HARP) instrument. 7   The HARP was developed by Blum et al. to measure the performance of anesthesia trainees that is required to provide safe anesthetic care to patients. According to the authors, the HARP is designed to be used “…as part of a multiscenario, simulation-based assessment” of resident performance. 7  

Content Validity: Does the Instrument’s Content Represent the Construct Being Measured?

To demonstrate content validity, instrument developers should describe the construct being measured and how the instrument was developed, and justify their approach. 25   The HARP is intended to measure resident performance in the critical domains required to provide safe anesthetic care. As such, investigators note that the HARP items were created through a two-step process. First, the instrument’s developers interviewed anesthesiologists with experience in resident education to identify the key traits needed for successful completion of anesthesia residency training. Second, the authors used a modified Delphi process to synthesize the responses into five key behaviors: (1) formulate a clear anesthetic plan, (2) modify the plan under changing conditions, (3) communicate effectively, (4) identify performance improvement opportunities, and (5) recognize one’s limits. 7 , 30  

Response Process Validity: Are Raters Interpreting the Instrument Items as Intended?

In the case of the HARP, the developers included a scoring rubric with behavioral anchors to ensure that faculty raters could clearly identify how resident performance in each domain should be scored. 7  

Internal Structure Validity: Do Instrument Items Measuring Similar Constructs Yield Homogenous Results? Do Instrument Items Measuring Different Constructs Yield Heterogeneous Results?

Item-correlation for the HARP demonstrated a high degree of correlation between some items ( e.g. , formulating a plan and modifying the plan under changing conditions) and a lower degree of correlation between other items ( e.g. , formulating a plan and identifying performance improvement opportunities). 30   This finding is expected since the items within the HARP are designed to assess separate performance domains, and we would expect residents’ functioning to vary across domains.

Relationship to Other Variables’ Validity: Do Instrument Scores Correlate with Other Measures of Similar or Different Constructs as Expected?

As it applies to the HARP, one would expect that the performance of anesthesia residents will improve over the course of training. Indeed, HARP scores were found to be generally higher among third-year residents compared to first-year residents. 30  

Consequence Validity: Are Instrument Results Being Used as Intended? Are There Unintended or Negative Uses of the Instrument Results?

While investigators did not intentionally seek out consequence validity evidence for the HARP, unanticipated consequences of HARP scores were identified by the authors as follows:

“Data indicated that CA-3s had a lower percentage of worrisome scores (rating 2 or lower) than CA-1s… However, it is concerning that any CA-3s had any worrisome scores…low performance of some CA-3 residents, albeit in the simulated environment, suggests opportunities for training improvement.” 30  

That is, using the HARP to measure the performance of CA-3 anesthesia residents had the unintended consequence of identifying the need for improvement in resident training.

Reliability: Are the Instrument’s Scores Reproducible and Consistent between Raters?

The HARP was applied by two raters for every resident in the study across seven different simulation scenarios. The investigators conducted a generalizability study of HARP scores to estimate the variance in assessment scores that was due to the resident, the rater, and the scenario. They found little variance was due to the rater ( i.e. , scores were consistent between raters), indicating a high level of reliability. 7  

Sampling refers to the selection of research subjects ( i.e. , the sample) from a larger group of eligible individuals ( i.e. , the population). 31   Effective sampling leads to the inclusion of research subjects who represent the larger population of interest. Alternatively, ineffective sampling may lead to the selection of research subjects who are significantly different from the target population. Imagine that researchers want to explore the relationship between burnout and educational debt among pain medicine specialists. The researchers distribute a survey to 1,000 pain medicine specialists (the population), but only 300 individuals complete the survey (the sample). This result is problematic because the characteristics of those individuals who completed the survey and the entire population of pain medicine specialists may be fundamentally different. It is possible that the 300 study subjects may be experiencing more burnout and/or debt, and thus, were more motivated to complete the survey. Alternatively, the 700 nonresponders might have been too busy to respond and even more burned out than the 300 responders, which would suggest that the study findings were even more amplified than actually observed.

When evaluating a medical education research article, it is important to identify the sampling technique the researchers employed, how it might have influenced the results, and whether the results apply to the target population. 24  

Sampling Techniques

Sampling techniques generally fall into two categories: probability- or nonprobability-based. Probability-based sampling ensures that each individual within the target population has an equal opportunity of being selected as a research subject. Most commonly, this is done through random sampling, which should lead to a sample of research subjects that is similar to the target population. If significant differences between sample and population exist, those differences should be due to random chance, rather than systematic bias. The difference between data from a random sample and that from the population is referred to as sampling error. 24  

Nonprobability-based sampling involves selecting research participants such that inclusion of some individuals may be more likely than the inclusion of others. 31   Convenience sampling is one such example and involves selection of research subjects based upon ease or opportuneness. Convenience sampling is common in medical education research, but, as outlined in the example at the beginning of this section, it can lead to sampling bias. 24   When evaluating an article that uses nonprobability-based sampling, it is important to look for participation/response rate. In general, a participation rate of less than 75% should be viewed with skepticism. 21   Additionally, it is important to determine whether characteristics of participants and nonparticipants were reported and if significant differences between the two groups exist.

Interpreting medical education research requires a basic understanding of common ways in which quantitative data are analyzed and displayed. In this section, we highlight two broad topics that are of particular importance when evaluating research articles.

The Nature of the Measurement Variable

Measurement variables in quantitative research generally fall into three categories: nominal, ordinal, or interval. 24   Nominal variables (sometimes called categorical variables) involve data that can be placed into discrete categories without a specific order or structure. Examples include sex (male or female) and professional degree (M.D., D.O., M.B.B.S., etc .) where there is no clear hierarchical order to the categories. Ordinal variables can be ranked according to some criterion, but the spacing between categories may not be equal. Examples of ordinal variables may include measurements of satisfaction (satisfied vs . unsatisfied), agreement (disagree vs . agree), and educational experience (medical student, resident, fellow). As it applies to educational experience, it is noteworthy that even though education can be quantified in years, the spacing between years ( i.e. , educational “growth”) remains unequal. For instance, the difference in performance between second- and third-year medical students is dramatically different than third- and fourth-year medical students. Interval variables can also be ranked according to some criteria, but, unlike ordinal variables, the spacing between variable categories is equal. Examples of interval variables include test scores and salary. However, the conceptual boundaries between these measurement variables are not always clear, as in the case where ordinal scales can be assumed to have the properties of an interval scale, so long as the data’s distribution is not substantially skewed. 32  

Understanding the nature of the measurement variable is important when evaluating how the data are analyzed and reported. Medical education research commonly uses measurement instruments with items that are rated on Likert-type scales, whereby the respondent is asked to assess their level of agreement with a given statement. The response is often translated into a corresponding number ( e.g. , 1 = strongly disagree, 3 = neutral, 5 = strongly agree). It is remarkable that scores from Likert-type scales are sometimes not normally distributed ( i.e. , are skewed toward one end of the scale), indicating that the spacing between scores is unequal and the variable is ordinal in nature. In these cases, it is recommended to report results as frequencies or medians, rather than means and SDs. 33  

Consider an article evaluating medical students’ satisfaction with a new curriculum. Researchers measure satisfaction using a Likert-type scale (1 = very unsatisfied, 2 = unsatisfied, 3 = neutral, 4 = satisfied, 5 = very satisfied). A total of 20 medical students evaluate the curriculum, 10 of whom rate their satisfaction as “satisfied,” and 10 of whom rate it as “very satisfied.” In this case, it does not make much sense to report an average score of 4.5; it makes more sense to report results in terms of frequency ( e.g. , half of the students were “very satisfied” with the curriculum, and half were not).

Effect Size and CIs

In medical education, as in other research disciplines, it is common to report statistically significant results ( i.e. , small P values) in order to increase the likelihood of publication. 34 , 35   However, a significant P value in itself does necessarily represent the educational impact of the study results. A statement like “Intervention x was associated with a significant improvement in learners’ intubation skill compared to education intervention y ( P < 0.05)” tells us that there was a less than 5% chance that the difference in improvement between interventions x and y was due to chance. Yet that does not mean that the study intervention necessarily caused the nonchance results, or indicate whether the between-group difference is educationally significant. Therefore, readers should consider looking beyond the P value to effect size and/or CI when interpreting the study results. 36 , 37  

Effect size is “the magnitude of the difference between two groups,” which helps to quantify the educational significance of the research results. 37   Common measures of effect size include Cohen’s d (standardized difference between two means), risk ratio (compares binary outcomes between two groups), and Pearson’s r correlation (linear relationship between two continuous variables). 37   CIs represent “a range of values around a sample mean or proportion” and are a measure of precision. 31   While effect size and CI give more useful information than simple statistical significance, they are commonly omitted from medical education research articles. 35   In such instances, readers should be wary of overinterpreting a P value in isolation. For further information effect size and CI, we direct readers the work of Sullivan and Feinn 37   and Hulley et al. 31  

In this final section, we identify instruments that can be used to evaluate the quality of quantitative medical education research articles. To this point, we have focused on framing the study and research methodologies and identifying potential pitfalls to consider when appraising a specific article. This is important because how a study is framed and the choice of methodology require some subjective interpretation. Fortunately, there are several instruments available for evaluating medical education research methods and providing a structured approach to the evaluation process.

The Medical Education Research Study Quality Instrument (MERSQI) 21   and the Newcastle Ottawa Scale-Education (NOS-E) 38   are two commonly used instruments, both of which have an extensive body of validity evidence to support the interpretation of their scores. Table 5 21 , 39   provides more detail regarding the MERSQI, which includes evaluation of study design, sampling, data type, validity, data analysis, and outcomes. We have found that applying the MERSQI to manuscripts, articles, and protocols has intrinsic educational value, because this practice of application familiarizes MERSQI users with fundamental principles of medical education research. One aspect of the MERSQI that deserves special mention is the section on evaluating outcomes based on Kirkpatrick’s widely recognized hierarchy of reaction, learning, behavior, and results ( table 5 ; fig .). 40   Validity evidence for the scores of the MERSQI include its operational definitions to improve response process, excellent reliability, and internal consistency, as well as high correlation with other measures of study quality, likelihood of publication, citation rate, and an association between MERSQI score and the likelihood of study funding. 21 , 41   Additionally, consequence validity for the MERSQI scores has been demonstrated by its utility for identifying and disseminating high-quality research in medical education. 42  

Kirkpatrick’s hierarchy of outcomes as applied to education research. Reaction = Level 1, Learning = Level 2, Behavior = Level 3, Results = Level 4. Outcomes become more meaningful, yet more difficult to achieve, when progressing from Level 1 through Level 4. Adapted with permission from Beckman and Cook, 2007. 2  

The Medical Education Research Study Quality Instrument for Evaluating the Quality of Medical Education Research

The NOS-E is a newer tool to evaluate the quality of medication education research. It was developed as a modification of the Newcastle-Ottawa Scale 43   for appraising the quality of nonrandomized studies. The NOS-E includes items focusing on the representativeness of the experimental group, selection and compatibility of the control group, missing data/study retention, and blinding of outcome assessors. 38 , 39   Additional validity evidence for NOS-E scores includes operational definitions to improve response process, excellent reliability and internal consistency, and its correlation with other measures of study quality. 39   Notably, the complete NOS-E, along with its scoring rubric, can found in the article by Cook and Reed. 39  

A recent comparison of the MERSQI and NOS-E found acceptable interrater reliability and good correlation between the two instruments 39   However, noted differences exist between the MERSQI and NOS-E. Specifically, the MERSQI may be applied to a broad range of study designs, including experimental and cross-sectional research. Additionally, the MERSQI addresses issues related to measurement validity and data analysis, and places emphasis on educational outcomes. On the other hand, the NOS-E focuses specifically on experimental study designs, and on issues related to sampling techniques and outcome assessment. 39   Ultimately, the MERSQI and NOS-E are complementary tools that may be used together when evaluating the quality of medical education research.

Conclusions

This article provides an overview of quantitative research in medical education, underscores the main components of education research, and provides a general framework for evaluating research quality. We highlighted the importance of framing a study with respect to purpose, conceptual framework, and statement of study intent. We reviewed the most common research methodologies, along with threats to the validity of a study and its measurement instruments. Finally, we identified two complementary instruments, the MERSQI and NOS-E, for evaluating the quality of a medical education research study.

Bordage G: Conceptual frameworks to illuminate and magnify. Medical education. 2009; 43(4):312–9.

Cook DA, Beckman TJ: Current concepts in validity and reliability for psychometric instruments: Theory and application. The American journal of medicine. 2006; 119(2):166. e7–166. e116.

Franenkel JR, Wallen NE, Hyun HH: How to Design and Evaluate Research in Education. 9th edition. New York, McGraw-Hill Education, 2015.

Hulley SB, Cummings SR, Browner WS, Grady DG, Newman TB: Designing clinical research. 4th edition. Philadelphia, Lippincott Williams & Wilkins, 2011.

Irby BJ, Brown G, Lara-Alecio R, Jackson S: The Handbook of Educational Theories. Charlotte, NC, Information Age Publishing, Inc., 2015

Standards for Educational and Psychological Testing (American Educational Research Association & American Psychological Association, 2014)

Swanwick T: Understanding medical education: Evidence, theory and practice, 2nd edition. Wiley-Blackwell, 2013.

Sullivan GM, Artino Jr AR: Analyzing and interpreting data from Likert-type scales. Journal of graduate medical education. 2013; 5(4):541–2.

Sullivan GM, Feinn R: Using effect size—or why the P value is not enough. Journal of graduate medical education. 2012; 4(3):279–82.

Tavakol M, Sandars J: Quantitative and qualitative methods in medical education research: AMEE Guide No 90: Part II. Medical teacher. 2014; 36(10):838–48.

Support was provided solely from institutional and/or departmental sources.

The authors declare no competing interests.

Citing articles via

Most viewed, email alerts, related articles, social media, affiliations.

• ASA Practice Parameters
• Online First
• Author Resource Center
• About the Journal
• Editorial Board
• Rights & Permissions
• Online ISSN 1528-1175
• Print ISSN 0003-3022
• Anesthesiology
• ASA Monitor

• Terms & Conditions Privacy Policy
• Manage Cookie Preferences
• © Copyright 2024 American Society of Anesthesiologists

This Feature Is Available To Subscribers Only

Sign In or Create an Account