case selection qualitative research

Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options

• Related Documents

Case Selection for Case‐Study Analysis: Qualitative and Quantitative Techniques

This article presents some guidance by cataloging nine different techniques for case selection: typical, diverse, extreme, deviant, influential, crucial, pathway, most similar, and most different. It also indicates that if the researcher is starting from a quantitative database, then methods for finding influential outliers can be used. In particular, the article clarifies the general principles that might guide the process of case selection in case-study research. Cases are more or less representative of some broader phenomenon and, on that score, may be considered better or worse subjects for intensive analysis. The article then draws attention to two ambiguities in case-selection strategies in case-study research. The first concerns the admixture of several case-selection strategies. The second concerns the changing status of a case as a study proceeds. Some case studies follow only one strategy of case selection.

Case Selection and Hypothesis Testing

This chapter discusses quantitative and qualitative practices of case-study selection when the goal of the analysis is to evaluate causal hypotheses. More specifically, it considers how the different causal models used in the qualitative and quantitative research cultures shape the kind of cases that provide the most leverage for hypothesis testing. The chapter examines whether one should select cases based on their value on the dependent variable. It also evaluates the kinds of cases that provide the most leverage for causal inference when conducting case-study research. It shows that differences in research goals between quantitative and qualitative scholars yield distinct ideas about best strategies of case selection. Qualitative research places emphasis on explaining particular cases; quantitative research does not.

Paying Attention to the Trees in the Forest, or a Call to Examine Agency-Specific Stories

Public administration scholarship needs to strike a better balance between large sample studies and in-depth case studies. The availability of large data sets has led us to engage in empirical research that is broad in scope but is frequently devoid of rich context. In-depth case studies can help to explain why we observe particular relationships and can help us to clarify gaps and inconsistencies in theory. Our argument for more case studies aims to encourage researchers to bridge insights from qualitative and quantitative research through triangulation. We describe the value of case study research, and qualitative and quantitative design options. We then propose opportunities for case study research in public personnel scholarship on patronage pressures, performance management, and diversity management.

Case Selection Techniques in Case Study Research

Blending the focused ethnographic method and case study research: implications regarding case selection and generalization of results.

In this article, we present the benefits of blending the methodological characteristics of the focused ethnographic method (FEM) and case study research (CSR) for a study on auxiliary work processes in a hospital in Barcelona, Spain. We argue that incorporating CSR logic and principles in the FEM produces a better form of inquiry, as this combination improves the quality of data and the ability to make comparisons in addition to enhancing the transferability of findings. These better outcomes are achieved because the characteristics of the case study methodological design reinforce many of the strengths of the FEM, such as its thematic concreteness and its targeted data collection. The role played by the theoretical framework of the study thus makes it easier to focus on the key variables of the initial theoretical model and to introduce a logic of multiple case comparison.

Qualitative Case Study Research Design

Qualitative case study research can be a valuable tool for answering complex, real-world questions. This method is often misunderstood or neglected due to a lack of understanding by researchers and reviewers. This tutorial defines the characteristics of qualitative case study research and its application to a broader understanding of stuttering that cannot be defined through other methodologies. This article will describe ways that data can be collected and analyzed.

Intersecting Mixed Methods and Case Study Research: Design Possibilities and Challenges

Robert k. yin. (2014). case study research design and methods (5th ed.). thousand oaks, ca: sage. 282 pages., cooperative banks & local government units - areas of collaboration (poland case study - research results), from the ‘frying-pan’ into the ‘fire’ antidotes to confirmatory bias in case-study research, export citation format, share document.

• Search Menu
• Browse content in Arts and Humanities
• Browse content in Archaeology
• Anglo-Saxon and Medieval Archaeology
• Archaeological Methodology and Techniques
• Archaeology by Region
• Archaeology of Religion
• Archaeology of Trade and Exchange
• Biblical Archaeology
• Contemporary and Public Archaeology
• Environmental Archaeology
• Historical Archaeology
• History and Theory of Archaeology
• Industrial Archaeology
• Landscape Archaeology
• Mortuary Archaeology
• Prehistoric Archaeology
• Underwater Archaeology
• Urban Archaeology
• Zooarchaeology
• Browse content in Architecture
• Architectural Structure and Design
• History of Architecture
• Residential and Domestic Buildings
• Theory of Architecture
• Browse content in Art
• Art Subjects and Themes
• History of Art
• Industrial and Commercial Art
• Theory of Art
• Biographical Studies
• Byzantine Studies
• Browse content in Classical Studies
• Classical History
• Classical Philosophy
• Classical Mythology
• Classical Literature
• Classical Reception
• Classical Art and Architecture
• Classical Oratory and Rhetoric
• Greek and Roman Epigraphy
• Greek and Roman Law
• Greek and Roman Papyrology
• Greek and Roman Archaeology
• Late Antiquity
• Religion in the Ancient World
• Digital Humanities
• Browse content in History
• Colonialism and Imperialism
• Diplomatic History
• Environmental History
• Genealogy, Heraldry, Names, and Honours
• Genocide and Ethnic Cleansing
• Historical Geography
• History by Period
• History of Emotions
• History of Agriculture
• History of Education
• History of Gender and Sexuality
• Industrial History
• Intellectual History
• International History
• Labour History
• Legal and Constitutional History
• Local and Family History
• Maritime History
• Military History
• National Liberation and Post-Colonialism
• Oral History
• Political History
• Public History
• Regional and National History
• Revolutions and Rebellions
• Slavery and Abolition of Slavery
• Social and Cultural History
• Theory, Methods, and Historiography
• Urban History
• World History
• Browse content in Language Teaching and Learning
• Language Learning (Specific Skills)
• Language Teaching Theory and Methods
• Browse content in Linguistics
• Applied Linguistics
• Cognitive Linguistics
• Computational Linguistics
• Forensic Linguistics
• Grammar, Syntax and Morphology
• Historical and Diachronic Linguistics
• History of English
• Language Acquisition
• Language Evolution
• Language Reference
• Language Variation
• Language Families
• Lexicography
• Linguistic Anthropology
• Linguistic Theories
• Linguistic Typology
• Phonetics and Phonology
• Psycholinguistics
• Sociolinguistics
• Translation and Interpretation
• Writing Systems
• Browse content in Literature
• Bibliography
• Children's Literature Studies
• Literary Studies (Asian)
• Literary Studies (European)
• Literary Studies (Eco-criticism)
• Literary Studies (Romanticism)
• Literary Studies (American)
• Literary Studies (Modernism)
• Literary Studies - World
• Literary Studies (1500 to 1800)
• Literary Studies (19th Century)
• Literary Studies (20th Century onwards)
• Literary Studies (African American Literature)
• Literary Studies (British and Irish)
• Literary Studies (Early and Medieval)
• Literary Studies (Fiction, Novelists, and Prose Writers)
• Literary Studies (Gender Studies)
• Literary Studies (Graphic Novels)
• Literary Studies (History of the Book)
• Literary Studies (Plays and Playwrights)
• Literary Studies (Poetry and Poets)
• Literary Studies (Postcolonial Literature)
• Literary Studies (Queer Studies)
• Literary Studies (Science Fiction)
• Literary Studies (Travel Literature)
• Literary Studies (War Literature)
• Literary Studies (Women's Writing)
• Literary Theory and Cultural Studies
• Mythology and Folklore
• Shakespeare Studies and Criticism
• Browse content in Media Studies
• Browse content in Music
• Applied Music
• Dance and Music
• Ethics in Music
• Ethnomusicology
• Gender and Sexuality in Music
• Medicine and Music
• Music Cultures
• Music and Religion
• Music and Media
• Music and Culture
• Music Education and Pedagogy
• Music Theory and Analysis
• Musical Scores, Lyrics, and Libretti
• Musical Structures, Styles, and Techniques
• Musicology and Music History
• Performance Practice and Studies
• Race and Ethnicity in Music
• Sound Studies
• Browse content in Performing Arts
• Browse content in Philosophy
• Aesthetics and Philosophy of Art
• Epistemology
• Feminist Philosophy
• History of Western Philosophy
• Metaphysics
• Moral Philosophy
• Non-Western Philosophy
• Philosophy of Science
• Philosophy of Language
• Philosophy of Mind
• Philosophy of Perception
• Philosophy of Action
• Philosophy of Law
• Philosophy of Religion
• Philosophy of Mathematics and Logic
• Practical Ethics
• Social and Political Philosophy
• Browse content in Religion
• Biblical Studies
• Christianity
• East Asian Religions
• History of Religion
• Judaism and Jewish Studies
• Qumran Studies
• Religion and Education
• Religion and Health
• Religion and Politics
• Religion and Science
• Religion and Law
• Religion and Art, Literature, and Music
• Religious Studies
• Browse content in Society and Culture
• Cookery, Food, and Drink
• Cultural Studies
• Customs and Traditions
• Ethical Issues and Debates
• Hobbies, Games, Arts and Crafts
• Lifestyle, Home, and Garden
• Natural world, Country Life, and Pets
• Popular Beliefs and Controversial Knowledge
• Sports and Outdoor Recreation
• Technology and Society
• Travel and Holiday
• Visual Culture
• Browse content in Law
• Arbitration
• Browse content in Company and Commercial Law
• Commercial Law
• Company Law
• Browse content in Comparative Law
• Systems of Law
• Competition Law
• Browse content in Constitutional and Administrative Law
• Government Powers
• Judicial Review
• Local Government Law
• Military and Defence Law
• Parliamentary and Legislative Practice
• Construction Law
• Contract Law
• Browse content in Criminal Law
• Criminal Procedure
• Criminal Evidence Law
• Sentencing and Punishment
• Employment and Labour Law
• Environment and Energy Law
• Browse content in Financial Law
• Banking Law
• Insolvency Law
• History of Law
• Human Rights and Immigration
• Intellectual Property Law
• Browse content in International Law
• Private International Law and Conflict of Laws
• Public International Law
• IT and Communications Law
• Jurisprudence and Philosophy of Law
• Law and Politics
• Law and Society
• Browse content in Legal System and Practice
• Courts and Procedure
• Legal Skills and Practice
• Primary Sources of Law
• Regulation of Legal Profession
• Medical and Healthcare Law
• Browse content in Policing
• Criminal Investigation and Detection
• Police and Security Services
• Police Procedure and Law
• Police Regional Planning
• Browse content in Property Law
• Personal Property Law
• Study and Revision
• Terrorism and National Security Law
• Browse content in Trusts Law
• Wills and Probate or Succession
• Browse content in Medicine and Health
• Browse content in Allied Health Professions
• Arts Therapies
• Clinical Science
• Dietetics and Nutrition
• Occupational Therapy
• Operating Department Practice
• Physiotherapy
• Radiography
• Speech and Language Therapy
• Browse content in Anaesthetics
• General Anaesthesia
• Neuroanaesthesia
• Browse content in Clinical Medicine
• Acute Medicine
• Cardiovascular Medicine
• Clinical Genetics
• Clinical Pharmacology and Therapeutics
• Dermatology
• Endocrinology and Diabetes
• Gastroenterology
• Genito-urinary Medicine
• Geriatric Medicine
• Infectious Diseases
• Medical Toxicology
• Medical Oncology
• Pain Medicine
• Palliative Medicine
• Rehabilitation Medicine
• Respiratory Medicine and Pulmonology
• Rheumatology
• Sleep Medicine
• Sports and Exercise Medicine
• Clinical Neuroscience
• Community Medical Services
• Critical Care
• Emergency Medicine
• Forensic Medicine
• Haematology
• History of Medicine
• Browse content in Medical Dentistry
• Oral and Maxillofacial Surgery
• Paediatric Dentistry
• Restorative Dentistry and Orthodontics
• Surgical Dentistry
• Browse content in Medical Skills
• Clinical Skills
• Communication Skills
• Nursing Skills
• Surgical Skills
• Medical Ethics
• Medical Statistics and Methodology
• Browse content in Neurology
• Clinical Neurophysiology
• Neuropathology
• Nursing Studies
• Browse content in Obstetrics and Gynaecology
• Gynaecology
• Occupational Medicine
• Ophthalmology
• Otolaryngology (ENT)
• Browse content in Paediatrics
• Neonatology
• Browse content in Pathology
• Chemical Pathology
• Clinical Cytogenetics and Molecular Genetics
• Histopathology
• Medical Microbiology and Virology
• Patient Education and Information
• Browse content in Pharmacology
• Psychopharmacology
• Browse content in Popular Health
• Caring for Others
• Complementary and Alternative Medicine
• Self-help and Personal Development
• Browse content in Preclinical Medicine
• Cell Biology
• Molecular Biology and Genetics
• Reproduction, Growth and Development
• Primary Care
• Professional Development in Medicine
• Browse content in Psychiatry
• Addiction Medicine
• Child and Adolescent Psychiatry
• Forensic Psychiatry
• Learning Disabilities
• Old Age Psychiatry
• Psychotherapy
• Browse content in Public Health and Epidemiology
• Epidemiology
• Public Health
• Browse content in Radiology
• Clinical Radiology
• Interventional Radiology
• Nuclear Medicine
• Radiation Oncology
• Reproductive Medicine
• Browse content in Surgery
• Cardiothoracic Surgery
• Gastro-intestinal and Colorectal Surgery
• General Surgery
• Neurosurgery
• Paediatric Surgery
• Peri-operative Care
• Plastic and Reconstructive Surgery
• Surgical Oncology
• Transplant Surgery
• Trauma and Orthopaedic Surgery
• Vascular Surgery
• Browse content in Science and Mathematics
• Browse content in Biological Sciences
• Aquatic Biology
• Biochemistry
• Bioinformatics and Computational Biology
• Developmental Biology
• Ecology and Conservation
• Evolutionary Biology
• Genetics and Genomics
• Microbiology
• Molecular and Cell Biology
• Natural History
• Plant Sciences and Forestry
• Research Methods in Life Sciences
• Structural Biology
• Systems Biology
• Zoology and Animal Sciences
• Browse content in Chemistry
• Analytical Chemistry
• Computational Chemistry
• Crystallography
• Environmental Chemistry
• Industrial Chemistry
• Inorganic Chemistry
• Materials Chemistry
• Medicinal Chemistry
• Mineralogy and Gems
• Organic Chemistry
• Physical Chemistry
• Polymer Chemistry
• Study and Communication Skills in Chemistry
• Theoretical Chemistry
• Browse content in Computer Science
• Artificial Intelligence
• Computer Architecture and Logic Design
• Game Studies
• Human-Computer Interaction
• Mathematical Theory of Computation
• Programming Languages
• Software Engineering
• Systems Analysis and Design
• Virtual Reality
• Browse content in Computing
• Business Applications
• Computer Security
• Computer Games
• Computer Networking and Communications
• Digital Lifestyle
• Graphical and Digital Media Applications
• Operating Systems
• Browse content in Earth Sciences and Geography
• Atmospheric Sciences
• Environmental Geography
• Geology and the Lithosphere
• Maps and Map-making
• Meteorology and Climatology
• Oceanography and Hydrology
• Palaeontology
• Physical Geography and Topography
• Regional Geography
• Soil Science
• Urban Geography
• Browse content in Engineering and Technology
• Agriculture and Farming
• Biological Engineering
• Civil Engineering, Surveying, and Building
• Electronics and Communications Engineering
• Energy Technology
• Engineering (General)
• Environmental Science, Engineering, and Technology
• History of Engineering and Technology
• Mechanical Engineering and Materials
• Technology of Industrial Chemistry
• Transport Technology and Trades
• Browse content in Environmental Science
• Applied Ecology (Environmental Science)
• Conservation of the Environment (Environmental Science)
• Environmental Sustainability
• Environmentalist Thought and Ideology (Environmental Science)
• Management of Land and Natural Resources (Environmental Science)
• Natural Disasters (Environmental Science)
• Nuclear Issues (Environmental Science)
• Pollution and Threats to the Environment (Environmental Science)
• Social Impact of Environmental Issues (Environmental Science)
• History of Science and Technology
• Browse content in Materials Science
• Ceramics and Glasses
• Composite Materials
• Metals, Alloying, and Corrosion
• Nanotechnology
• Browse content in Mathematics
• Applied Mathematics
• Biomathematics and Statistics
• History of Mathematics
• Mathematical Education
• Mathematical Finance
• Mathematical Analysis
• Numerical and Computational Mathematics
• Probability and Statistics
• Pure Mathematics
• Browse content in Neuroscience
• Cognition and Behavioural Neuroscience
• Development of the Nervous System
• Disorders of the Nervous System
• History of Neuroscience
• Invertebrate Neurobiology
• Molecular and Cellular Systems
• Neuroendocrinology and Autonomic Nervous System
• Neuroscientific Techniques
• Sensory and Motor Systems
• Browse content in Physics
• Astronomy and Astrophysics
• Atomic, Molecular, and Optical Physics
• Biological and Medical Physics
• Classical Mechanics
• Computational Physics
• Condensed Matter Physics
• Electromagnetism, Optics, and Acoustics
• History of Physics
• Mathematical and Statistical Physics
• Measurement Science
• Nuclear Physics
• Particles and Fields
• Plasma Physics
• Quantum Physics
• Relativity and Gravitation
• Semiconductor and Mesoscopic Physics
• Browse content in Psychology
• Affective Sciences
• Clinical Psychology
• Cognitive Psychology
• Cognitive Neuroscience
• Criminal and Forensic Psychology
• Developmental Psychology
• Educational Psychology
• Evolutionary Psychology
• Health Psychology
• History and Systems in Psychology
• Music Psychology
• Neuropsychology
• Organizational Psychology
• Psychological Assessment and Testing
• Psychology of Human-Technology Interaction
• Psychology Professional Development and Training
• Research Methods in Psychology
• Social Psychology
• Browse content in Social Sciences
• Browse content in Anthropology
• Anthropology of Religion
• Human Evolution
• Medical Anthropology
• Physical Anthropology
• Regional Anthropology
• Social and Cultural Anthropology
• Theory and Practice of Anthropology
• Browse content in Business and Management
• Business Strategy
• Business Ethics
• Business History
• Business and Government
• Business and Technology
• Business and the Environment
• Comparative Management
• Corporate Governance
• Corporate Social Responsibility
• Entrepreneurship
• Health Management
• Human Resource Management
• Industrial and Employment Relations
• Industry Studies
• Information and Communication Technologies
• International Business
• Knowledge Management
• Management and Management Techniques
• Operations Management
• Organizational Theory and Behaviour
• Pensions and Pension Management
• Public and Nonprofit Management
• Strategic Management
• Supply Chain Management
• Browse content in Criminology and Criminal Justice
• Criminal Justice
• Criminology
• Forms of Crime
• International and Comparative Criminology
• Youth Violence and Juvenile Justice
• Development Studies
• Browse content in Economics
• Agricultural, Environmental, and Natural Resource Economics
• Asian Economics
• Behavioural Finance
• Behavioural Economics and Neuroeconomics
• Econometrics and Mathematical Economics
• Economic Systems
• Economic History
• Economic Methodology
• Economic Development and Growth
• Financial Markets
• Financial Institutions and Services
• General Economics and Teaching
• Health, Education, and Welfare
• History of Economic Thought
• International Economics
• Labour and Demographic Economics
• Law and Economics
• Macroeconomics and Monetary Economics
• Microeconomics
• Public Economics
• Urban, Rural, and Regional Economics
• Welfare Economics
• Browse content in Education
• Adult Education and Continuous Learning
• Care and Counselling of Students
• Early Childhood and Elementary Education
• Educational Equipment and Technology
• Educational Strategies and Policy
• Higher and Further Education
• Organization and Management of Education
• Philosophy and Theory of Education
• Schools Studies
• Secondary Education
• Teaching of a Specific Subject
• Teaching of Specific Groups and Special Educational Needs
• Teaching Skills and Techniques
• Browse content in Environment
• Applied Ecology (Social Science)
• Climate Change
• Conservation of the Environment (Social Science)
• Environmentalist Thought and Ideology (Social Science)
• Natural Disasters (Environment)
• Social Impact of Environmental Issues (Social Science)
• Browse content in Human Geography
• Cultural Geography
• Economic Geography
• Political Geography
• Browse content in Interdisciplinary Studies
• Communication Studies
• Museums, Libraries, and Information Sciences
• Browse content in Politics
• African Politics
• Asian Politics
• Chinese Politics
• Comparative Politics
• Conflict Politics
• Elections and Electoral Studies
• Environmental Politics
• European Union
• Foreign Policy
• Gender and Politics
• Human Rights and Politics
• Indian Politics
• International Relations
• International Organization (Politics)
• International Political Economy
• Irish Politics
• Latin American Politics
• Middle Eastern Politics
• Political Methodology
• Political Communication
• Political Philosophy
• Political Sociology
• Political Behaviour
• Political Economy
• Political Institutions
• Political Theory
• Politics and Law
• Public Administration
• Public Policy
• Quantitative Political Methodology
• Regional Political Studies
• Russian Politics
• Security Studies
• State and Local Government
• UK Politics
• US Politics
• Browse content in Regional and Area Studies
• African Studies
• Asian Studies
• East Asian Studies
• Japanese Studies
• Latin American Studies
• Middle Eastern Studies
• Native American Studies
• Scottish Studies
• Browse content in Research and Information
• Research Methods
• Browse content in Social Work
• Addictions and Substance Misuse
• Adoption and Fostering
• Care of the Elderly
• Child and Adolescent Social Work
• Couple and Family Social Work
• Developmental and Physical Disabilities Social Work
• Direct Practice and Clinical Social Work
• Emergency Services
• Human Behaviour and the Social Environment
• International and Global Issues in Social Work
• Mental and Behavioural Health
• Social Justice and Human Rights
• Social Policy and Advocacy
• Social Work and Crime and Justice
• Social Work Macro Practice
• Social Work Practice Settings
• Social Work Research and Evidence-based Practice
• Welfare and Benefit Systems
• Browse content in Sociology
• Childhood Studies
• Community Development
• Comparative and Historical Sociology
• Economic Sociology
• Gender and Sexuality
• Gerontology and Ageing
• Health, Illness, and Medicine
• Marriage and the Family
• Migration Studies
• Occupations, Professions, and Work
• Organizations
• Population and Demography
• Race and Ethnicity
• Social Theory
• Social Movements and Social Change
• Social Research and Statistics
• Social Stratification, Inequality, and Mobility
• Sociology of Religion
• Sociology of Education
• Sport and Leisure
• Urban and Rural Studies
• Browse content in Warfare and Defence
• Defence Strategy, Planning, and Research
• Land Forces and Warfare
• Military Administration
• Military Life and Institutions
• Naval Forces and Warfare
• Other Warfare and Defence Issues
• Peace Studies and Conflict Resolution
• Weapons and Equipment

• < Previous chapter
• Next chapter >

6 Case Study and Selection

• Published: March 2024
• Cite Icon Cite
• Permissions Icon Permissions

This chapter presents and critically examines best practices for case study research and case selection. In reviewing prominent case study criteria that prioritize objective, analytical needs of research design (“appropriateness”), the chapter argues that overlooked considerations of feasibility (including positionality, resources, and skills) and interest play equally important roles when selecting and studying cases. The chapter concludes by arguing for an iterative process of research design, one that equally weighs pragmatic considerations and academic concerns of best fit. This strategy hopes to make qualitative research more accessible and transparent and to diversify the types of cases we study and questions we ask.

Signed in as

Institutional accounts.

• Google Scholar Indexing
• GoogleCrawler [DO NOT DELETE]

Personal account

• Sign in with email/username & password
• Get email alerts
• Save searches
• Purchase content
• Activate your purchase/trial code

Institutional access

• Sign in with a library card Sign in with username/password Recommend to your librarian
• Institutional account management
• Get help with access

Access to content on Oxford Academic is often provided through institutional subscriptions and purchases. If you are a member of an institution with an active account, you may be able to access content in one of the following ways:

IP based access

Typically, access is provided across an institutional network to a range of IP addresses. This authentication occurs automatically, and it is not possible to sign out of an IP authenticated account.

Sign in through your institution

Choose this option to get remote access when outside your institution. Shibboleth/Open Athens technology is used to provide single sign-on between your institution’s website and Oxford Academic.

• Click Sign in through your institution.
• Select your institution from the list provided, which will take you to your institution's website to sign in.
• When on the institution site, please use the credentials provided by your institution. Do not use an Oxford Academic personal account.
• Following successful sign in, you will be returned to Oxford Academic.

If your institution is not listed or you cannot sign in to your institution’s website, please contact your librarian or administrator.

Sign in with a library card

Enter your library card number to sign in. If you cannot sign in, please contact your librarian.

Society Members

Society member access to a journal is achieved in one of the following ways:

Sign in through society site

Many societies offer single sign-on between the society website and Oxford Academic. If you see ‘Sign in through society site’ in the sign in pane within a journal:

• Click Sign in through society site.
• When on the society site, please use the credentials provided by that society. Do not use an Oxford Academic personal account.

If you do not have a society account or have forgotten your username or password, please contact your society.

Sign in using a personal account

Some societies use Oxford Academic personal accounts to provide access to their members. See below.

A personal account can be used to get email alerts, save searches, purchase content, and activate subscriptions.

Some societies use Oxford Academic personal accounts to provide access to their members.

Viewing your signed in accounts

Click the account icon in the top right to:

• View your signed in personal account and access account management features.
• View the institutional accounts that are providing access.

Signed in but can't access content

Oxford Academic is home to a wide variety of products. The institutional subscription may not cover the content that you are trying to access. If you believe you should have access to that content, please contact your librarian.

For librarians and administrators, your personal account also provides access to institutional account management. Here you will find options to view and activate subscriptions, manage institutional settings and access options, access usage statistics, and more.

Our books are available by subscription or purchase to libraries and institutions.

• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Rights and permissions
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

• Help & FAQ

Case-Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options

• Political Science

Research output : Contribution to journal › Article › peer-review

Fingerprint

• Quantitative Social Sciences 100%
• Selection Procedure Social Sciences 100%
• Approach Social Sciences 100%
• Case Study Research Social Sciences 100%
• Case Study Psychology 100%
• Research Psychology 100%
• sampling INIS 100%
• Analysis Social Sciences 66%

T1 - Case-Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options

AU - Seawright, Jason

N2 - How can scholars select cases from a large universe for in-depth case study analysis? Random sampling is not typically a viable approach when the total number of cases to be selected is small. Hence attention to purposive modes of sampling is needed. Yet, while the existing qualitative literature on case selection offers a wide range of suggestions for case selection, most techniques discussed require in-depth familiarity of each case. Seven case selection procedures are considered, each of which facilitates a different strategy for within-case analysis. The case selection procedures considered focus on typical, diverse, extreme, deviant, influential, most similar, and most different cases. For each case selection procedure, quantitative approaches are discussed that meet the goals of the approach, while still requiring information that can reasonably be gathered for a large number of cases.

AB - How can scholars select cases from a large universe for in-depth case study analysis? Random sampling is not typically a viable approach when the total number of cases to be selected is small. Hence attention to purposive modes of sampling is needed. Yet, while the existing qualitative literature on case selection offers a wide range of suggestions for case selection, most techniques discussed require in-depth familiarity of each case. Seven case selection procedures are considered, each of which facilitates a different strategy for within-case analysis. The case selection procedures considered focus on typical, diverse, extreme, deviant, influential, most similar, and most different cases. For each case selection procedure, quantitative approaches are discussed that meet the goals of the approach, while still requiring information that can reasonably be gathered for a large number of cases.

M3 - Article

JO - Political Research Quarterly

JF - Political Research Quarterly

• Open access
• Published: 02 May 2024

Use of the International IFOMPT Cervical Framework to inform clinical reasoning in postgraduate level physiotherapy students: a qualitative study using think aloud methodology

• Katie L. Kowalski 1 ,
• Heather Gillis 1 ,
• Katherine Henning 1 ,
• Paul Parikh 1 ,
• Jackie Sadi 1 &
• Alison Rushton 1  

BMC Medical Education volume  24 , Article number:  486 ( 2024 ) Cite this article

225 Accesses

Metrics details

Vascular pathologies of the head and neck are rare but can present as musculoskeletal problems. The International Federation of Orthopedic Manipulative Physical Therapists (IFOMPT) Cervical Framework (Framework) aims to assist evidence-based clinical reasoning for safe assessment and management of the cervical spine considering potential for vascular pathology. Clinical reasoning is critical to physiotherapy, and developing high-level clinical reasoning is a priority for postgraduate (post-licensure) educational programs.

To explore the influence of the Framework on clinical reasoning processes in postgraduate physiotherapy students.

Qualitative case study design using think aloud methodology and interpretive description, informed by COnsolidated criteria for REporting Qualitative research. Participants were postgraduate musculoskeletal physiotherapy students who learned about the Framework through standardized delivery. Two cervical spine cases explored clinical reasoning processes. Coding and analysis of transcripts were guided by Elstein’s diagnostic reasoning components and the Postgraduate Musculoskeletal Physiotherapy Practice model. Data were analyzed using thematic analysis (inductive and deductive) for individuals and then across participants, enabling analysis of key steps in clinical reasoning processes and use of the Framework. Trustworthiness was enhanced with multiple strategies (e.g., second researcher challenged codes).

For all participants ( n  = 8), the Framework supported clinical reasoning using primarily hypothetico-deductive processes. It informed vascular hypothesis generation in the patient history and testing the vascular hypothesis through patient history questions and selection of physical examination tests, to inform clarity and support for diagnosis and management. Most participant’s clinical reasoning processes were characterized by high-level features (e.g., prioritization), however there was a continuum of proficiency. Clinical reasoning processes were informed by deep knowledge of the Framework integrated with a breadth of wider knowledge and supported by a range of personal characteristics (e.g., reflection).

Conclusions

Findings support use of the Framework as an educational resource in postgraduate physiotherapy programs to inform clinical reasoning processes for safe and effective assessment and management of cervical spine presentations considering potential for vascular pathology. Individualized approaches may be required to support students, owing to a continuum of clinical reasoning proficiency. Future research is required to explore use of the Framework to inform clinical reasoning processes in learners at different levels.

Peer Review reports

Introduction

Musculoskeletal neck pain and headache are highly prevalent and among the most disabling conditions globally that require effective rehabilitation [ 1 , 2 , 3 , 4 ]. A range of rehabilitation professionals, including physiotherapists, assess and manage musculoskeletal neck pain and headache. Assessment of the cervical spine can be a complex process. Patients can present to physiotherapy with vascular pathology masquerading as musculoskeletal pain and dysfunction, as neck pain and/or headache as a common first symptom [ 5 ]. While vascular pathologies of the head and neck are rare [ 6 ], they are important considerations within a cervical spine assessment to facilitate the best possible patient outcomes [ 7 ]. The International IFOMPT (International Federation of Orthopedic Manipulative Physical Therapists) Cervical Framework (Framework) provides guidance in the assessment and management of the cervical spine region, considering the potential for vascular pathologies of the neck and head [ 8 ]. Two separate, but related, risks are considered: risk of misdiagnosis of an existing vascular pathology and risk of serious adverse event following musculoskeletal interventions [ 8 ].

The Framework is a consensus document iteratively developed through rigorous methods and the best contemporary evidence [ 8 ], and is also published as a Position Statement [ 7 ]. Central to the Framework are clinical reasoning and evidence-based practice, providing guidance in the assessment of the cervical spine region, considering the potential for vascular pathologies in advance of planned interventions [ 7 , 8 ]. The Framework was developed and published to be a resource for practicing musculoskeletal clinicians and educators. It has been implemented widely within IFOMPT postgraduate (post-licensure) educational programs, influencing curricula by enabling a comprehensive and systemic approach when considering the potential for vascular pathology [ 9 ]. Frequently reported curricula changes include an emphasis on the patient history and incorporating Framework recommended physical examination tests to evaluate a vascular hypothesis [ 9 ]. The Framework aims to assist musculoskeletal clinicians in their clinical reasoning processes, however no study has investigated students’ use of the Framework to inform their clinical reasoning.

Clinical reasoning is a critical component to physiotherapy practice as it is fundamental to assessment and diagnosis, enabling physiotherapists to provide safe and effective patient-centered care [ 10 ]. This is particularly important for postgraduate physiotherapy educational programs, where developing a high level of clinical reasoning is a priority for educational curricula [ 11 ] and critical for achieving advanced practice physiotherapy competency [ 12 , 13 , 14 , 15 ]. At this level of physiotherapy, diagnostic reasoning is emphasized as an important component of a high level of clinical reasoning, informed by advanced use of domain-specific knowledge (e.g., propositional, experiential) and supported by a range of personal characteristics (e.g., adaptability, reflective) [ 12 ]. Facilitating the development of clinical reasoning improves physiotherapist’s performance and patient outcomes [ 16 ], underscoring the importance of clinical reasoning to physiotherapy practice. Understanding students’ use of the Framework to inform their clinical reasoning can support optimal implementation of the Framework within educational programs to facilitate safe and effective assessment and management of the cervical spine for patients.

To explore the influence of the Framework on the clinical reasoning processes in postgraduate level physiotherapy students.

Using a qualitative case study design, think aloud case analyses enabled exploration of clinical reasoning processes in postgraduate physiotherapy students. Case study design allows evaluation of experiences in practice, providing knowledge and accounts of practical actions in a specific context [ 17 ]. Case studies offer opportunity to generate situationally dependent understandings of accounts of clinical practice, highlighting the action and interaction that underscore the complexity of clinical decision-making in practice [ 17 ]. This study was informed by an interpretive description methodological approach with thematic analysis [ 18 , 19 ]. Interpretive description is coherent with mixed methods research and pragmatic orientations [ 20 , 21 ], and enables generation of evidence-based disciplinary knowledge and clinical understanding to inform practice [ 18 , 19 , 22 ]. Interpretive description has evolved for use in educational research to generate knowledge of educational experiences and the complexities of health care education to support achievement of educational objectives and professional practice standards [ 23 ]. The COnsolidated criteria for REporting Qualitative research (COREQ) informed the design and reporting of this study [ 24 ].

Research team

All research team members hold physiotherapy qualifications, and most hold advanced qualifications specializing in musculoskeletal physiotherapy. The research team is based in Canada and has varying levels of academic credentials (ranging from Clinical Masters to PhD or equivalent) and occupations (ranging from PhD student to Director of Physical Therapy). The final author (AR) is also an author of the Framework, which represents international and multiprofessional consensus. Authors HG and JS are lecturers on one of the postgraduate programs which students were recruited from. The primary researcher and first author (KK) is a US-trained Physical Therapist and Postdoctoral Research Associate investigating spinal pain and clinical reasoning in the School of Physical Therapy at Western University. Authors KK, KH and PP had no prior relationship with the postgraduate educational programs, students, or the Framework.

Study setting

Western University in London, Ontario, Canada offers a one-year Advanced Health Care Practice (AHCP) postgraduate IFOMPT-approved Comprehensive Musculoskeletal Physiotherapy program (CMP) and a postgraduate Sport and Exercise Medicine (SEM) program. Think aloud case analyses interviews were conducted using Zoom, a viable option for qualitative data collection and audio-video recording of interviews that enables participation for students who live in geographically dispersed areas across Canada [ 25 ]. Interviews with individual participants were conducted by one researcher (KK or KH) in a calm and quiet environment to minimize disruption to the process of thinking aloud [ 26 ].

Participants

AHCP postgraduate musculoskeletal physiotherapy students ≥ 18 years of age in the CMP and SEM programs were recruited via email and an introduction to the research study during class by KK, using purposive sampling to ensure theoretical representation. The purposive sample ensured key characteristics of participants were included, specifically gender, ethnicity, and physiotherapy experience (years, type). AHCP students must have attended standardized teaching about the Framework to be eligible to participate. Exclusion criteria included inability to communicate fluently in English. As think-aloud methodology seeks rich, in-depth data from a small sample [ 27 ], this study sought to recruit 8–10 AHCP students. This range was informed by prior think aloud literature and anticipated to balance diversity of participant characteristics, similarities in musculoskeletal physiotherapy domain knowledge and rich data supporting individual clinical reasoning processes [ 27 , 28 ].

Learning about the IFOMPT Cervical Framework

CMP and SEM programs included standardized teaching of the Framework to inform AHCP students’ clinical reasoning in practice. Delivery included a presentation explaining the Framework, access to the full Framework document [ 8 ], and discussion of its role to inform practice, including a case analysis of a cervical spine clinical presentation, by research team members AR and JS. The full Framework document that is publicly available through IFOMPT [ 8 ] was provided to AHCP students as the Framework Position Statement [ 7 ] was not yet published. Discussion and case analysis was led by AHCP program leads in November 2021 (CMP, including research team member JS) and January 2022 (SEM).

Think aloud case analyses data collection

Using think aloud methodology, the analytical processes of how participants use the Framework to inform clinical reasoning were explored in an interview with one research team member not involved in AHCP educational programs (KK or KH). The think aloud method enables description and explanation of complex information paralleling the clinical reasoning process and has been used previously in musculoskeletal physiotherapy [ 29 , 30 ]. It facilitates the generation of rich verbal [ 27 ]as participants verbalize their clinical reasoning protocols [ 27 , 31 ]. Participants were aware of the aim of the research study and the research team’s clinical and research backgrounds, supporting an open environment for depth of data collection [ 32 ]. There was no prior relationship between participants and research team members conducting interviews.

Participants were instructed to think aloud their analysis of two clinical cases, presented in random order (Supplementary  1 ). Case information was provided in stages to reflect the chronology of assessment of patients in practice (patient history, planning the physical examination, physical examination, treatment). Use of the Framework to inform clinical reasoning was discussed at each stage. The cases enabled participants to identify and discuss features of possible vascular pathology, treatment indications and contraindications/precautions, etc. Two research study team members (HG, PP) developed cases designed to facilitate and elicit clinical reasoning processes in neck and head pain presentations. Cases were tested against the research team to ensure face validity. Cases and think aloud prompts were piloted prior to use with three physiotherapists at varying levels of practice to ensure they were fit for purpose.

Data collection took place from March 30-August 15, 2022, during the final terms of the AHCP programs and an average of 5 months after standardized teaching about the Framework. During case analysis interviews, participants were instructed to constantly think aloud, and if a pause in verbalizations was sustained, they were reminded to “keep thinking aloud” [ 27 ]. As needed, prompts were given to elicit verbalization of participants’ reasoning processes, including use of the Framework to inform their clinical reasoning at each stage of case analysis (Supplementary  2 ). Aside from this, all interactions between participants and researchers minimized to not interfere with the participant’s thought processes [ 27 , 31 ]. When analysis of the first case was complete, the researcher provided the second case, each lasting 35–45 min. A break between cases was offered. During and after interviews, field notes were recorded about initial impressions of the data collection session and potential patterns appearing to emerge [ 33 ].

Data analysis

Data from think aloud interviews were analyzed using thematic analysis [ 30 , 34 ], facilitating identification and analysis of patterns in data and key steps in the clinical reasoning process, including use of the Framework to enable its characterization (Fig.  1 ). As established models of clinical reasoning exist, a hybrid approach to thematic analysis was employed, incorporating inductive and deductive processes [ 35 ], which proceeded according to 5 iterative steps: [ 34 ]

Data analysis steps

Familiarize with data: Audio-visual recordings were transcribed verbatim by a physiotherapist external to the research team. All transcripts were read and re-read several times by one researcher (KK), checking for accuracy by reviewing recordings as required. Field notes supported depth of familiarization with data.

Generate initial codes: Line-by-line coding of transcripts by one researcher (KK) supported generation of initial codes that represented components, patterns and meaning in clinical reasoning processes and use of the Framework. Established preliminary coding models were used as a guide. Elstein’s diagnostic reasoning model [ 36 ] guided generating initial codes of key steps in clinical reasoning processes (Table  1 a) [ 29 , 36 ]. Leveraging richness of data, further codes were generated guided by the Postgraduate Musculoskeletal Physiotherapy Practice model, which describes masters level clinical practice (Table  1 b) [ 12 ]. Codes were refined as data analysis proceeded. All codes were collated within participants along with supporting data.

Generate initial themes within participants: Coded data was inductively grouped into initial themes within each participant, reflecting individual clinical reasoning processes and use of the Framework. This inductive stage enabled a systematic, flexible approach to describe each participant’s unique thinking path, offering insight into the complexities of their clinical reasoning processes. It also provided a comprehensive understanding of the Framework informing clinical reasoning and a rich characterization of its components, aiding the development of robust, nuanced insights [ 35 , 37 , 38 ]. Initial themes were repeatedly revised to ensure they were grounded in and reflected raw data.

Develop, review and refine themes across participants: Initial themes were synthesized across participants to develop themes that represented all participants. Themes were reviewed and refined, returning to initial themes and codes at the individual participant level as needed.

Organize themes into established models: Themes were deductively organized into established clinical reasoning models; first into Elstein’s diagnostic reasoning model, second into the Postgraduate Musculoskeletal Physiotherapy Practice model to characterize themes within each diagnostic reasoning component [ 12 , 36 ].

Trustworthiness of findings

The research study was conducted according to an a priori protocol and additional steps were taken to establish trustworthiness of findings [ 39 ]. Field notes supported deep familiarization with data and served as a means of data source triangulation during analysis [ 40 ]. One researcher coded transcripts and a second researcher challenged codes, with codes and themes rigorously and iteratively reviewed and refined. Frequent debriefing sessions with the research team, reflexive discussions with other researchers and peer scrutiny of initial findings enabled wider perspectives and experiences to shape analysis and interpretation of findings. Several strategies were implemented to minimize the influence of prior relationships between participants and researchers, including author KK recruiting participants, KK and KH collecting/analyzing data, and AR, JS, HG and PP providing input on de-identified data at the stage of synthesis and interpretation.

Nine AHCP postgraduate level students were recruited and participated in data collection. One participant was withdrawn because of unfamiliarity with the standardized teaching session about use of the Framework (no recall of session), despite confirmation of attendance. Data from eight participants were used for analysis (CMP: n  = 6; SEM: n  = 2; Table  2 ), which achieved sample size requirements for think aloud methodology of rich and in-depth data [ 27 , 28 ].

Diagnostic reasoning components

Informed by the Framework, all components of Elstein’s diagnostic reasoning processes [ 36 ] were used by participants, including use of treatment with physiotherapy interventions to aid diagnostic reasoning. An illustrative example is presented in Supplement  3 . Clinical reasoning used primarily hypothetico-deductive processes reflecting a continuum of proficiency, was informed by deep Framework knowledge and breadth of prior knowledge (e.g., experiential), and supported by a range of personal characteristics (e.g., justification for decisions).

Cue acquisition

All participants sought to acquire additional cues early in the patient history, and for some this persisted into the medical history and physical examination. Cue acquisition enabled depth and breadth of understanding patient history information to generate hypotheses and factors contributing to the patient’s pain experience (Table  3 ). All participants asked further questions to understand details of the patients’ pain and their presentation, while some also explored the impact of pain on patient functioning and treatments received to date. There was a high degree of specificity to questions for most participants. Ongoing clinical reasoning processes through a thorough and complete assessment, even if the patient had previously received treatment for similar symptoms, was important for some participants. Cue acquisition was supported by personal characteristics including a patient-centered approach (e.g., understanding the patient’s beliefs about pain) and one participant reflected on their approach to acquiring patient history cues.

Hypothesis generation

Participants generated an average of 4.5 hypotheses per case (range: 2–8) and most hypotheses (77%) were generated rapidly early in the patient history. Knowledge from the Framework about patient history features of vascular pathology informed vascular hypothesis generation in the patient history for all participants in both cases (Table  4 ). Vascular hypotheses were also generated during the past medical history, where risk factors for vascular pathology were identified and interpreted by some participants who had high levels of suspicion for cervical articular involvement. Non-vascular hypotheses were generated during the physical examination by some participants to explain individual physical examination or patient history cues. Deep knowledge of the patient history section in the Framework supported high level of cue identification and interpretation for generating vascular hypotheses. Initial hypotheses were prioritized by some participants, however the level of specificity of hypotheses varied.

Cue evaluation

All participants evaluated cues throughout the patient history and physical examination in relationship to hypotheses generated, indicating use of hypothetico-deductive reasoning processes (Table  5 ). Framework knowledge of patient history features of vascular pathology was used to test vascular hypotheses and aid differential diagnosis. The patient history section supported high level of cue identification and interpretation of patient history features for all but one participant, and generation of further patient history questions for all participants. The level of specificity of these questions was high for all but one participant. Framework knowledge of recommended physical examination tests, including removal of positional testing, supported planning a focused and prioritized physical examination to further test vascular hypotheses for all participants. No participant indicated intention to use positional testing as part of their physical examination. Treatment with physiotherapy interventions served as a form of cue evaluation, and cues were evaluated to inform prognosis for some participants. At times during the physical examination, some participants demonstrated occasional errors or difficulty with cue evaluation by omitting key physical exam tests (e.g., no cranial nerve assessment despite concerns for trigeminal nerve involvement), selecting physical exam tests in advance of hypothesis generation (e.g., cervical spine instability testing), difficulty interpreting cues, or late selection of a physical examination test. Cue acquisition was supported by a range of personal characteristics. Most participants justified selection of physical examination tests, and some self-reflected on their ability to collect useful physical examination information to inform selection of tests. Precaution to the physical examination was identified by all participants but one, which contributed to an adaptable approach, prioritizing patient safety and comfort. Critical analysis of physical examination information aided interpretation within the context of the patient for most participants.

Hypothesis evaluation

All participants used the Framework to evaluate their hypotheses throughout the patient history and physical examination, continuously shifting their level of support for hypotheses (Table  6 , Supplement  4 ). This informed clarity in the overall level of suspicion for vascular pathology or musculoskeletal diagnoses, which were specific for most participants. Response to treatment with physiotherapy interventions served as a form of hypothesis evaluation for most participants who had low level suspicion for vascular pathology, highlighting ongoing reasoning processes. Hypotheses evaluated were prioritized by ranking according to level of suspicion by some participants. Difficulties weighing patient history and physical examination cues to inform judgement on overall level of suspicion for vascular pathology was demonstrated by some participants who reported that incomplete physical examination data and not being able to see the patient contributed to difficulties. Hypothesis evaluation was supported by the personal characteristic of reflection, where some students reflected on the Framework’s emphasis on the patient history to evaluate a vascular hypothesis.

The Framework supported all participants in clinical reasoning related to treatment (Table  7 ). Treatment decisions were always linked to the participant’s overall level of suspicion for vascular pathology or musculoskeletal diagnosis. Framework knowledge supported participants with high level of suspicion for vascular pathology to refer for further investigations. Participants with a musculoskeletal diagnosis kept the patient for physiotherapy interventions. The Framework patient history section supported patient education about symptoms of vascular pathology and safety netting for some participants. Framework knowledge influenced informed consent processes and risk-benefit analysis to support the selection of musculoskeletal physiotherapy interventions, which were specific and prioritized for some participants. Less Framework knowledge related to treatment was demonstrated by some students, generating unclear recommendations regarding the urgency of referral and use of the Framework to inform musculoskeletal physiotherapy interventions. Treatment was supported by a range of personal characteristics. An adaptable approach that prioritized patient safety and was supported by justification was demonstrated in all participants except one. Shared decision-making enabled the selection of physiotherapy interventions, which were patient-centered (individualized, considered whole person, identified future risk for vascular pathology). Communication with the patient’s family doctor facilitated collaborative patient-centered care for most participants.

This is the first study to explore the influence of the Framework on clinical reasoning processes in postgraduate physiotherapy students. The Framework supported clinical reasoning that used primarily hypothetico-deductive processes. The Framework informed vascular hypothesis generation in the patient history and testing the vascular hypothesis through patient history questions and selection of physical examination tests to inform clarity and support for diagnosis and management. Most postgraduate students’ clinical reasoning processes were characterized by high-level features (e.g. specificity, prioritization). However, some demonstrated occasional difficulties or errors, reflecting a continuum of clinical reasoning proficiency. Clinical reasoning processes were informed by deep knowledge of the Framework integrated with a breadth of wider knowledge and supported by a range of personal characteristics (e.g., justification for decisions, reflection).

Use of the Framework to inform clinical reasoning processes

The Framework provided a structured and comprehensive approach to support postgraduate students’ clinical reasoning processes in assessment and management of the cervical spine region, considering the potential for vascular pathology. Patient history and physical examination information was evaluated to inform clarity and support the decision to refer for further vascular investigations or proceed with musculoskeletal physiotherapy diagnosis/interventions. The Framework is not intended to lead to a vascular pathology diagnosis [ 7 , 8 ], and following the Framework does not guarantee vascular pathologies will be identified [ 41 ]. Rather, it aims to support a process of clinical reasoning to elicit and interpret appropriate patient history and physical examination information to estimate the probability of vascular pathology and inform judgement about the need to refer for further investigations [ 7 , 8 , 42 ]. Results of this study suggest the Framework has achieved this aim for postgraduate physiotherapy students.

The Framework supported postgraduate students in using primarily hypothetico-deductive diagnostic reasoning processes. This is expected given the diversity of vascular pathology clinical presentations precluding a definite clinical pattern and inherent complexity as a potential masquerader of a musculoskeletal problem [ 7 ]. It is also consistent with prior research investigating clinical reasoning processes in musculoskeletal physiotherapy postgraduate students [ 12 ] and clinical experts [ 29 ] where hypothetico-deductive and pattern recognition diagnostic reasoning are employed according to the demands of the clinical situation [ 10 ]. Diagnostic reasoning of most postgraduate students in this study demonstrated features suggestive of high-level clinical reasoning in musculoskeletal physiotherapy [ 12 ], including ongoing reasoning with high-level cue identification and interpretation, specificity and prioritization during assessment and treatment, use of physiotherapy interventions to aid diagnostic reasoning, and prognosis determination [ 12 , 29 , 43 ]. Expert physiotherapy practice has been further described as using a dialectical model of clinical reasoning with seamless transitions between clinical reasoning strategies [ 44 ]. While diagnostic reasoning was a focus in this study, postgraduate students considered a breadth of information as important to their reasoning (e.g., patient’s perspectives of the reason for their pain). This suggests wider reasoning strategies (e.g., narrative, collaborative) were employed to enable shared decision-making within the context of patient-centered care.

Study findings also highlighted a continuum of proficiency in use of the Framework to inform clinical reasoning processes. Not all students demonstrated all characteristics of high-level clinical reasoning and there are suggestions of incomplete reasoning processes, for example occasional errors in evaluating cues. Some students offered explanations such as incomplete case information as factors contributing to difficulties with clinical reasoning processes. However, the ability to critically evaluate incomplete and potentially conflicting clinical information is consistently identified as an advanced clinical practice competency [ 14 , 43 ]. A continuum of proficiency in clinical reasoning in musculoskeletal physiotherapy is supported by wider healthcare professions describing acquisition and application of clinical knowledge and skills as a developmental continuum of clinical competence progressing from novice to expert [ 45 , 46 ]. The range of years of clinical practice experience in this cohort of students (3–14 years) or prior completed postgraduate education may have contributed to the continuum of proficiency, as high-quality and diverse experiential learning is essential for the development of high-level clinical reasoning [ 14 , 47 ].

Deep knowledge of the Framework informs clinical reasoning processes

Postgraduate students demonstrated deep Framework knowledge to inform clinical reasoning processes. All students demonstrated knowledge of patient history features of vascular pathology, recommended physical examination tests to test a vascular hypothesis, and the need to refer if there is a high level of suspicion for vascular pathology. A key development in the recent Framework update is the removal of the recommendation to perform positional testing [ 8 ]. All students demonstrated knowledge of this development, and none wanted to test a vascular hypothesis with positional testing. Most also demonstrated Framework knowledge about considerations for planning treatment with physiotherapy interventions (e.g., risk-benefit analysis, informed consent), though not all, which underscores the continuum of proficiency in postgraduate students. Rich organization of multidimensional knowledge is a required component for high level clinical reasoning and is characteristic of expert physiotherapy practice [ 10 , 48 , 49 ]. Most postgraduate physiotherapy students displayed this expert practice characteristic through integration of deep Framework knowledge with a breadth of prior knowledge (e.g., experiential, propositional) to inform clinical reasoning processes. This highlights the utility of the Framework in postgraduate physiotherapy education to develop advanced level evidence-based knowledge informing clinical reasoning processes for safe assessment and management of the cervical spine, considering the potential for vascular pathology [ 9 , 8 , 50 , 51 , 52 ].

Framework supports personal characteristics to facilitate integration of knowledge and clinical reasoning

The Framework supported personal characteristics of postgraduate students, which are key drivers for the complex integration of advanced knowledge and high-level clinical reasoning [ 10 , 12 , 48 ]. For all students, the Framework supported justification for decisions and patient-centered care, emphasizing a whole-person approach and shared decision-making. Further demonstrating a continuum of proficiency, the Framework supported a wider breadth of personal characteristics for some students, including critical analysis, reflection, self-analysis, and adaptability. These personal characteristics illustrate the interwoven cognitive and metacognitive skills that influence and support a high level of clinical reasoning [ 10 , 12 ] and the development of clinical expertise [ 48 , 53 ]. For example [ 54 ], reflection is critical to developing high-level clinical reasoning and advanced level practice [ 12 , 55 ]. Postgraduate students reflected on prior knowledge, experiences, and action within the context of current Framework knowledge, emphasizing active engagement in cognitive processes to inform clinical reasoning processes. Reflection-in-action is highlighted by self-analysis and adaptability. These characteristics require continuous cognitive processing to consider personal strengths and limitations in the context of the patient and evidence-based practice, adapting the clinical encounter as required [ 53 , 55 ]. These findings highlight use of the Framework in postgraduate education to support development of personal characteristics that are indicative of an advanced level of clinical practice [ 12 ].

Synthesis of findings

Derived from synthesis of research study findings and informed by the Postgraduate Musculoskeletal Physiotherapy Practice model [ 12 ], use of the Framework to inform clinical reasoning processes in postgraduate students is illustrated in Fig.  2 . Overlapping clinical reasoning, knowledge and personal characteristic components emphasize the complex interaction of factors contributing to clinical reasoning processes. Personal characteristics of postgraduate students underpin clinical reasoning and knowledge, highlighting their role in facilitating the integration of these two components. Bolded subcomponents indicate convergence of results reflecting all postgraduate students and underscores the variability among postgraduate students contributing to a continuum of clinical reasoning proficiency. The relative weighting of the components is approximately equal to balance the breadth and convergence of subcomponents. Synthesis of findings align with the Postgraduate Musculoskeletal Physiotherapy Practice model [ 12 ], though some differences exist. Limited personal characteristics were identified in this study with little convergence across students, which may be due to the objective of this study and the case analysis approach.

Use of the Framework to inform clinical reasoning in postgraduate level musculoskeletal physiotherapy students. Adapted from the Postgraduate Musculoskeletal Physiotherapy Practice model [ 12 ].

Strengths and limitations

Think aloud case analyses enabled situationally dependent understanding of the Framework to inform clinical reasoning processes in postgraduate level students [ 17 ], considering the rare potential for vascular pathology. A limitation of this approach was the standardized nature of case information provided to students, which may have influenced clinical reasoning processes. Future research studies may consider patient case simulation to address this limitation [ 30 ]. Interviews were conducted during the second half of the postgraduate educational program, and this timing could have influenced clinical reasoning processes compared to if interviews were conducted at the end of the program. Future research can explore use of the Framework to inform clinical reasoning processes in established advanced practice physiotherapists. The sample size of this study aligns with recommendations for think aloud methodology [ 27 , 28 ], achieved rich data, and purposive sampling enabled wide representation of key characteristics (e.g., gender, ethnicity, country of training, physiotherapy experiences), which enhances transferability of findings. Students were aware of the study objective in advance of interviews which may have contributed to a heightened level of awareness of vascular pathology. The prior relationship between students and researchers may have also influenced results, however several strategies were implemented to minimize this influence.

Implications

The Framework is widely implemented within IFOMPT postgraduate educational programs and has led to important shifts in educational curricula [ 9 ]. Findings of this study support use of the Framework as an educational resource in postgraduate physiotherapy programs to inform clinical reasoning processes for safe and effective assessment and management of cervical spine presentations considering the potential for vascular pathology. Individualized approaches may be required to support each student, owing to a continuum of clinical reasoning proficiency. As the Framework was written for practicing musculoskeletal clinicians, future research is required to explore use of the Framework to inform clinical reasoning in learners at different levels, for example entry-level physiotherapy students.

The Framework supported clinical reasoning that used primarily hypothetico-deductive processes in postgraduate physiotherapy students. It informed vascular hypothesis generation in the patient history and testing the vascular hypothesis through patient history questions and selection of physical examination tests, to inform clarity and support for diagnosis and management. Most postgraduate students clinical reasoning processes were characterized as high-level, informed by deep Framework knowledge integrated with a breadth of wider knowledge, and supported by a range of personal characteristics to facilitate the integration of advanced knowledge and high-level clinical reasoning. Future research is required to explore use of the Framework to inform clinical reasoning in learners at different levels.

Data availability

The dataset used and analyzed during the current study are available from the corresponding author on reasonable request.

Safiri S, Kolahi AA, Hoy D, Buchbinder R, Mansournia MA, Bettampadi D et al. Global, regional, and national burden of neck pain in the general population, 1990–2017: systematic analysis of the global burden of Disease Study 2017. BMJ. 2020;368.

Stovner LJ, Nichols E, Steiner TJ, Abd-Allah F, Abdelalim A, Al-Raddadi RM, et al. Global, regional, and national burden of migraine and tension-type headache, 1990–2016: a systematic analysis for the global burden of Disease Study 2016. Lancet Neurol. 2018;17:954–76.

Article   Google Scholar  

Cieza A, Causey K, Kamenov K, Hanson SW, Chatterji S, Vos T. Global estimates of the need for rehabilitation based on the Global Burden of Disease study 2019: a systematic analysis for the global burden of Disease Study 2019. Lancet. 2020;396:2006–17.

Côté P, Yu H, Shearer HM, Randhawa K, Wong JJ, Mior S et al. Non-pharmacological management of persistent headaches associated with neck pain: A clinical practice guideline from the Ontario protocol for traffic injury management (OPTIMa) collaboration. European Journal of Pain (United Kingdom). 2019;23.

Diamanti S, Longoni M, Agostoni EC. Leading symptoms in cerebrovascular diseases: what about headache? Neurological Sciences. 2019.

Debette S, Compter A, Labeyrie MA, Uyttenboogaart M, Metso TM, Majersik JJ, et al. Epidemiology, pathophysiology, diagnosis, and management of intracranial artery dissection. Lancet Neurol. 2015;14:640–54.

Rushton A, Carlesso LC, Flynn T, Hing WA, Rubinstein SM, Vogel S, et al. International Framework for examination of the Cervical Region for potential of vascular pathologies of the Neck Prior to Musculoskeletal intervention: International IFOMPT Cervical Framework. J Orthop Sports Phys Therapy. 2023;53:7–22.

Rushton A, Carlesso LC, Flynn T, Hing WA, Kerry R, Rubinstein SM, et al. International framework for examination of the cervical region for potential of vascular pathologies of the neck prior to orthopaedic manual therapy (OMT) intervention: International IFOMPT Cervical Framework. International IFOMPT Cervical Framework; 2020.

Hutting N, Kranenburg R, Taylor A, Wilbrink W, Kerry R, Mourad F. Implementation of the International IFOMPT Cervical Framework: a survey among educational programmes. Musculoskelet Sci Pract. 2022;62:102619.

Jones MA, Jensen G, Edwards I. Clinical reasoning in physiotherapy. In: Campbell S, Watkins V, editors. Clinical reasoning in the health professions. Third. Philadelphia: Elsevier; 2008. pp. 245–56.

Google Scholar  

Fennelly O, Desmeules F, O’Sullivan C, Heneghan NR, Cunningham C. Advanced musculoskeletal physiotherapy practice: informing education curricula. Musculoskelet Sci Pract. 2020;48:102174.

Rushton A, Lindsay G. Defining the construct of masters level clinical practice in manipulative physiotherapy. Man Ther. 2010;15.

Rushton A, Lindsay G. Defining the construct of masters level clinical practice in healthcare based on the UK experience. Med Teach. 2008;30:e100–7.

Noblet T, Heneghan NR, Hindle J, Rushton A. Accreditation of advanced clinical practice of musculoskeletal physiotherapy in England: a qualitative two-phase study to inform implementation. Physiotherapy (United Kingdom). 2021;113.

Tawiah AK, Stokes E, Wieler M, Desmeules F, Finucane L, Lewis J, et al. Developing an international competency and capability framework for advanced practice physiotherapy: a scoping review with narrative synthesis. Physiotherapy. 2023;122:3–16.

Williams A, Rushton A, Lewis JJ, Phillips C. Evaluation of the clinical effectiveness of a work-based mentoring programme to develop clinical reasoning on patient outcome: a stepped wedge cluster randomised controlled trial. PLoS ONE. 2019;14.

Miles R. Complexity, representation and practice: case study as method and methodology. Issues Educational Res. 2015;25.

Thorne S, Kirkham SR, MacDonald-Emes J. Interpretive description: a noncategorical qualitative alternative for developing nursing knowledge. Res Nurs Health. 1997;20.

Thorne S, Kirkham SR, O’Flynn-Magee K. The Analytic challenge in interpretive description. Int J Qual Methods. 2004;3.

Creswell JW. Research design: qualitative, quantitative, and mixed methods approaches. Sage; 2003.

Dolan S, Nowell L, Moules NJ. Interpretive description in applied mixed methods research: exploring issues of fit, purpose, process, context, and design. Nurs Inq. 2023;30.

Thorne S. Interpretive description. In: Routledge International Handbook of Qualitative Nursing Research. 2013. pp. 295–306.

Thompson Burdine J, Thorne S, Sandhu G. Interpretive description: a flexible qualitative methodology for medical education research. Med Educ. 2021;55.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus group. Int J Qual Health Care. 2007;19:349–57.

Archibald MM, Ambagtsheer RC, Casey MG, Lawless M. Using zoom videoconferencing for qualitative data Collection: perceptions and experiences of researchers and participants. Int J Qual Methods. 2019;18.

Van Someren M, Barnard YF, Sandberg J. The think aloud method: a practical approach to modelling cognitive. Volume 11. London: Academic; 1994.

Fonteyn ME, Kuipers B, Grobe SJ. A description of think aloud Method and Protocol Analysis. Qual Health Res. 1993;3:430–41.

Lundgrén-Laine H, Salanterä S. Think-Aloud technique and protocol analysis in clinical decision-making research. Qual Health Res. 2010;20:565–75.

Doody C, McAteer M. Clinical reasoning of expert and novice physiotherapists in an outpatient orthopaedic setting. Physiotherapy. 2002;88.

Gilliland S. Physical therapist students’ development of diagnostic reasoning: a longitudinal study. J Phys Therapy Educ. 2017;31.

Ericsson KA, Simon HA. How to study thinking in Everyday Life: contrasting think-aloud protocols with descriptions and explanations of thinking. Mind Cult Act. 1998;5:178–86.

Dwyer SC, Buckle JL. The space between: on being an insider-outsider in qualitative research. Int J Qual Methods. 2009;8.

Shenton AK. Strategies for ensuring trustworthiness in qualitative research projects. Educ Inform. 2004;22:63–75.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3:77–101.

Fereday J, Muir-Cochrane E. Demonstrating Rigor using thematic analysis: a Hybrid Approach of Inductive and deductive coding and theme development. Int J Qual Methods. 2006;5.

Elstein ASLSS. Medical problem solving: an analysis of clinical reasoning. Harvard University Press; 1978.

Proudfoot K. Inductive/Deductive Hybrid Thematic Analysis in mixed methods research. J Mix Methods Res. 2023;17.

Charters E. The use of think-aloud methods in qualitative research an introduction to think-aloud methods. Brock Educ J. 2003;12.

Nowell LS, Norris JM, White DE, Moules NJ. Thematic analysis: striving to meet the trustworthiness Criteria. Int J Qual Methods. 2017;16:1–13.

Thurmond VA. The point of triangulation. J Nurs Scholarsh. 2001;33.

Hutting N, Wilbrink W, Taylor A, Kerry R. Identifying vascular pathologies or flow limitations: important aspects in the clinical reasoning process. Musculoskelet Sci Pract. 2021;53:102343.

de Best RF, Coppieters MW, van Trijffel E, Compter A, Uyttenboogaart M, Bot JC, et al. Risk assessment of vascular complications following manual therapy and exercise for the cervical region: diagnostic accuracy of the International Federation of Orthopaedic Manipulative physical therapists framework (the Go4Safe project). J Physiother. 2023;69:260–6.

Petty NJ. Becoming an expert: a masterclass in developing clinical expertise. Int J Osteopath Med. 2015;18:207–18.

Edwards I, Jones M, Carr J, Braunack-Mayer A, Jensen GM. Clinical reasoning strategies in physical therapy. Phys Ther. 2004;84.

Carraccio CL, Benson BJ, Nixon LJ, Derstine PL. Clinical teaching from the Educational Bench to the clinical Bedside: Translating the Dreyfus Developmental Model to the Learning of Clinical Skills.

Benner P. Using the Dreyfus Model of Skill Acquisition to describe and interpret Skill Acquisition and Clinical Judgment in nursing practice and education. Bull Sci Technol Soc. 2004;24:188–99.

Benner P. From novice to expert: Excellence and power in clinical nursing practice. Upper Saddle River, New Jersey: Prentice Hall;: Commemorative Ed; 2001.

Jensen GM, Gwyer J, Shepard KF, Hack LM. Expert practice in physical therapy. Phys Ther. 2000;80.

Huhn K, Gilliland SJ, Black LL, Wainwright SF, Christensen N. Clinical reasoning in physical therapy: a Concept Analysis. Phys Ther. 2019;99.

Hutting N, Kranenburg HA, Rik KR. Yes, we should abandon pre-treatment positional testing of the cervical spine. Musculoskelet Sci Pract. 2020;49:102181.

Kranenburg HA, Tyer R, Schmitt M, Luijckx GJ, Schans C, Van Der, Hutting N, et al. Effects of head and neck positions on blood flow in the vertebral, internal carotid, and intracranial arteries: a systematic review. J Orthop Sports Phys Ther. 2019;49:688–97.

Hutting N, Kerry R, Coppieters MW, Scholten-Peeters GGM. Considerations to improve the safety of cervical spine manual therapy. Musculoskelet Sci Pract. 2018;33.

Wainwright SF, Shepard KF, Harman LB, Stephens J. Novice and experienced physical therapist clinicians: a comparison of how reflection is used to inform the clinical decision-making process. Phys Ther. 2010;90:75–88.

Dy SM, Purnell TS. Key concepts relevant to quality of complex and shared decision-making in health care: a literature review. Soc Sci Med. 2012;74:582–7.

Christensen N, Jones MA, Higgs J, Edwards I. Dimensions of clinical reasoning capability. In: Campbell S, Watkins V, editors. Clinical reasoning in the health professions. 3rd edition. Philadelphia: Elsevier; 2008. pp. 101–10.

Download references

Acknowledgements

The authors would like to acknowledge study participants and the transcriptionist for their time in completing and transcribing think aloud interviews.

No funding was received to conduct this research study.

Author information

Authors and affiliations.

School of Physical Therapy, Western University, London, Ontario, Canada

Katie L. Kowalski, Heather Gillis, Katherine Henning, Paul Parikh, Jackie Sadi & Alison Rushton

You can also search for this author in PubMed   Google Scholar

Contributions

Katie Kowalski: Conceptualization, methodology, validation, formal analysis, investigation, data curation, writing– original draft, visualization, project administration. Heather Gillis: Validation, resources, writing– review & editing. Katherine Henning: Investigation, formal analysis, writing– review & editing. Paul Parikh: Validation, resources, writing– review & editing. Jackie Sadi: Validation, resources, writing– review & editing. Alison Rushton: Conceptualization, methodology, validation, writing– review & editing, supervision.

Corresponding author

Correspondence to Katie L. Kowalski .

Ethics declarations

Ethics approval and consent to participate.

Western University Health Science Research Ethics Board granted ethical approval (Project ID: 119934). Participants provided written informed consent prior to participating in think aloud interviews.

Consent for publication

Not applicable.

Competing interests

Author AR is an author of the IFOMPT Cervical Framework. Authors JS and HG are lecturers on the AHCP CMP program. AR and JS led standardized teaching of the Framework. Measures to reduce the influence of potential competing interests on the conduct and results of this study included: the Framework representing international and multiprofessional consensus, recruitment of participants by author KK, data collection and analysis completed by KK with input from AR, JS and HG at the stage of data synthesis and interpretation, and wider peer scrutiny of initial findings. KK, KH and PP have no potential competing interests.

Authors’ information

The lead author of this study (AR) is the first author of the International IFOMPT Cervical Framework.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Kowalski, K.L., Gillis, H., Henning, K. et al. Use of the International IFOMPT Cervical Framework to inform clinical reasoning in postgraduate level physiotherapy students: a qualitative study using think aloud methodology. BMC Med Educ 24 , 486 (2024). https://doi.org/10.1186/s12909-024-05399-x

Download citation

Received : 11 February 2024

Accepted : 08 April 2024

Published : 02 May 2024

DOI : https://doi.org/10.1186/s12909-024-05399-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• International IFOMPT Cervical Framework
• Clinical reasoning
• Postgraduate students
• Physiotherapy
• Educational research
• Qualitative research
• Think aloud methodology

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Approaches to sampling and case selection in qualitative research: examples in the geography of health

Affiliation.

• 1 Department of Geography, Queen Mary and Westfield College, University of London, UK. [email protected]
• PMID: 10714922
• DOI: 10.1016/s0277-9536(99)00350-0

This paper focuses on the question of sampling (or selection of cases) in qualitative research. Although the literature includes some very useful discussions of qualitative sampling strategies, the question of sampling often seems to receive less attention in methodological discussion than questions of how data is collected or is analysed. Decisions about sampling are likely to be important in many qualitative studies (although it may not be an issue in some research). There are varying accounts of the principles applicable to sampling or case selection. Those who espouse 'theoretical sampling', based on a 'grounded theory' approach, are in some ways opposed to those who promote forms of 'purposive sampling' suitable for research informed by an existing body of social theory. Diversity also results from the many different methods for drawing purposive samples which are applicable to qualitative research. We explore the value of a framework suggested by Miles and Huberman [Miles, M., Huberman,, A., 1994. Qualitative Data Analysis, Sage, London.], to evaluate the sampling strategies employed in three examples of research by the authors. Our examples comprise three studies which respectively involve selection of: 'healing places'; rural places which incorporated national anti-malarial policies; young male interviewees, identified as either chronically ill or disabled. The examples are used to show how in these three studies the (sometimes conflicting) requirements of the different criteria were resolved, as well as the potential and constraints placed on the research by the selection decisions which were made. We also consider how far the criteria Miles and Huberman suggest seem helpful for planning 'sample' selection in qualitative research.

Publication types

• Research Support, Non-U.S. Gov't
• Chronic Disease
• Disabled Persons
• Malaria / prevention & control
• Mental Healing
• Models, Theoretical
• North Carolina
• Sampling Studies*

Help | Advanced Search

Computer Science > Human-Computer Interaction

Title: exploring the potential of human-llm synergy in advancing qualitative analysis: a case study on mental-illness stigma.

Abstract: Qualitative analysis is a challenging, yet crucial aspect of advancing research in the field of Human-Computer Interaction (HCI). Recent studies show that large language models (LLMs) can perform qualitative coding within existing schemes, but their potential for collaborative human-LLM discovery and new insight generation in qualitative analysis is still underexplored. To bridge this gap and advance qualitative analysis by harnessing the power of LLMs, we propose CHALET, a novel methodology that leverages the human-LLM collaboration paradigm to facilitate conceptualization and empower qualitative research. The CHALET approach involves LLM-supported data collection, performing both human and LLM deductive coding to identify disagreements, and performing collaborative inductive coding on these disagreement cases to derive new conceptual insights. We validated the effectiveness of CHALET through its application to the attribution model of mental-illness stigma, uncovering implicit stigmatization themes on cognitive, emotional and behavioral dimensions. We discuss the implications for future research, methodology, and the transdisciplinary opportunities CHALET presents for the HCI community and beyond.

Submission history

Access paper:.

• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Multilevel Research Designs: Case Selection, Levels of Analysis, and Scope Conditions

• Published: 15 August 2020
• Volume 55 , pages 460–480, ( 2020 )

Cite this article

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

• Tomas Dosek   ORCID: orcid.org/0000-0003-4297-8456 1  

1230 Accesses

2 Citations

30 Altmetric

Explore all metrics

Subnational research in comparative politics has been growing steadily in the last two decades. However, methodological advances have been rather limited. This article builds upon Snyder’s (Stud Comp Int Dev 36(1):93–110, 2001 ) subnational comparative method and extends its logic to the comparison of subnational units from different countries. It proposes a novel typology of multilevel research designs that focuses particularly on cross-national small-N analysis (CSNA). This research design offers three different logics of qualitative case selection to achieve a sound trade-off between internal and external validity. This article analyzes the advantages and limitations of the underlying logics of CSNA and illustrates their use with recent empirical research from Latin American countries. It concludes by highlighting its versatility and offers a series of best practices in order to produce more generalizable findings than the majority of single-country subnational comparisons.

Similar content being viewed by others

What Is Globalisation?

The KOF Globalisation Index – revisited

Gender as a Social Structure

Avoid common mistakes on your manuscript.

Almost two decades ago, Richard Snyder ( 2001 ) argued in his path-breaking article for more subnational research and for the comparison of subnational units. Snyder stressed the advantages of the so-called subnational comparative method (SCM). Basically, this method Footnote 1 allows for the increasing of the number of observations, as well as a more controlled comparison, a more accurate coding of cases and a comprehension of within-country variations of many social and political phenomena. Snyder’s SCM embraced both large and small-N studies, although his empirical examples were drawn mainly from qualitative comparative studies.

Ironically, given the revival of qualitative methods at the beginning of the 2000s (see Brady and Collier 2010 , among many others), Snyder’s call for “scaling down” was mainly understood as a solution to the “many variables, small N” problem famously posed by Arend Lijphart ( 1971 ). A meta-analysis of all the studies that cite Snyder’s article shows that most of them use it to justify a subnational large-N (mostly econometric) analysis within a single country. To a lesser extent, they compare few subnational cases within a single country in a more qualitative fashion. Thus, the majority of them focus on a single country, seeking to explain the subnational variation of a given phenomenon and emphasizing internal validity. These studies offer a more fine-grained knowledge of within-country differences and in many cases some theoretical innovation, Footnote 2 but their scope is normally limited to a single country. Footnote 3

However, Snyder ( 2001 ) made a particularly strong argument for combining within-country and between-country comparisons in a single study. Footnote 4 Using non-contiguous subnational units from different countries, this research design provides a strategy that not only “could help maximize both control over potential explanatory factors and independence among the cases”, but could also help overcome the limited “generalizability” of within-country comparisons (Snyder 2001 : 97, 103–104). Snyder concluded by urging future research to search for strategies that could achieve these goals in a single research design.

Despite its advantages, very few authors have combined within- and between-country comparisons in empirical studies, and limited methodological advances have been made since 2001 and the recent “subnational turn” Footnote 5 in comparative politics. In line with Snyder ( 2001 ), this paper argues that comparativists should use this research design more frequently, but also more consciously, since there are different logics not envisioned in the original proposal regarding how cases are selected and generalizability is achieved. Footnote 6 While no one logic is a priori superior to the others, they serve different purposes and have different limitations.

By unpacking the research strategy that combines comparison of subnational units from different countries, this paper identifies three different underlying logics of qualitative case selection. Footnote 7 The first one seeks to understand different results in similar subnational cases; the second one studies similar subnational phenomena in different national-level settings; and the final one analyzes similar phenomena in a more exploratory fashion, independently of the national context. These logics serve different research purposes and have different (dis)advantages, but all provide new insights on subnational phenomena in different countries and thus more generalizable findings than single-country studies.

This paper also argues that the SCM can be better understood in terms of what is called here a “multilevel research design” (MRD). Footnote 8 This type of research design includes cases from two (or more) levels of analysis Footnote 9 and the principal units of analysis are normally at the lower level. What this paper proposes is a novel typology of multilevel research designs that helps to locate the qualitative combination of within- and between-country comparison within a broader frame. This typology yields four ideal types of research design. In this context, a small-N case selection at two different levels (referred to here as cross-national small-N analysis, CSNA) with its three different logics is located halfway between the within-country subnational comparisons and cross-national hierarchical statistical large-N models.

The rest of this paper is organized as follows. First, I conduct a meta-analysis of approximately 800 studies that cite Snyder ( 2001 ) in order to show some basic patterns in the subnational comparative research of the past years and to underscore the predominance of within-country subnational large-N analysis research design. Second, the paper introduces a novel typology of multilevel research designs (MRDs) that combines the number of countries with the number of cases to produce four ideal types of MRDs. Third, the article focuses specifically on cross-national small-N (comparative) analysis, and fourth, it explores its three different logics and its (dis)advantages. Fifth, the article examines possible application of these logics to within-country multilevel comparisons. Sixth, it discusses the lessons learnt. Finally, the concluding section sums up main findings and suggests agenda for future research.

Meta-Analysis of Snyder’s “Scaling Down: The Subnational Comparative Method”

Different ways exist to grasp the impact of a particular piece of scholarship in the literature. Many authors choose a meta-analysis of leading journals in the field in order to evaluate the presence of a certain methodological approach or substantive topic (Munck and Snyder 2007 ; Sellers 2019 ). However, these studies normally have trouble justifying which particular journals to include in the sample Footnote 10 and in assessing the extent to which they reflect broader tendencies in the field. Here, I prefer to analyze the citations of Snyder’s ( 2001 ) article in Google Scholar, as it allows for the capture of a larger picture of the discipline. Footnote 11 This is true insofar Snyder’s method has been used in an increasing number of empirical studies Footnote 12 and also because it contributed to the consolidation of subnational politics as both a subfield in comparative politics and as a methodological approach in the discipline. Footnote 13 Moreover, many (unpublished) dissertations have used his approach, as well as articles focusing exclusively on a single country normally published in area studies journals or regional/national political science journals. For all these reasons, this meta-analysis is based on Google Scholar and not only on a selection of leading journals.

The meta-analysis of all 792 works Footnote 14 that cite Snyder’s ( 2001 ) article shows three main types of references. Footnote 15 Figure 1 reveals that these can be classified as follows: (i) single-country large-N (statistical) analyses at the subnational level that use Snyder (more or less explicitly) to justify their research design; (ii) single-country small-N (comparative) analyses that, again, in the majority of cases use Snyder to justify their research design; and (iii) a mix of works that highlight either the necessity to analyze subnational politics (for either theoretical or methodological reasons, or both), the fact that the politics is more and more decentralized and therefore subnational levels of government more important, or the need for more subnational comparisons. Footnote 16 Only number categories (i) and (ii) are clearly empirical studies that use the Snyder article to justify their research design.

Meta-analysis of Snyder’s ( 2001 ) article. Source: prepared by the author

Other categories follow. The fourth group includes empirical works with different mixed research designs, mainly in the form of statistical subnational analysis followed by a few cases studies (some explicitly in Lieberman’s nested fashion) or subnational large-N analysis in two or more countries. The fifth group is made up of works that cite Snyder “by error.” Footnote 17 The sixth group consists of methodology studies that refer to the paper or discuss some of its arguments. The seventh group consists in empirical works that use single-country multilevel comparisons. The eighth group is made up of book reviews and state-of-the-art articles or chapters. Only the ninth group includes works that use the cross-national small-N (comparative) analysis MRD, which shows how limited its use still is. The tenth group consists of classic multilevel statistical analysis. The last three groups comprise single-country subnational case studies, hybrid research designs that combine different types of research design and that in many cases even include national-level cross-sectional analyses, and, lastly, national-level comparisons that for some reason cite Snyder ( 2001 ).

It is clear from this brief analysis that the empirical studies that justify their research design in terms of Snyder’s framework are principally limited to within-country studies. They take the form of small-N (qualitative) or large-N (usually statistical) comparisons and together comprise more than one-third of all the works that cite his article. Footnote 18 As Fig.  1 shows, the latter prevail over the former. These large-N works highlight, as Snyder suggested, the possibility of seeking explanations of subnational variation in a more controlled context. Thus, researchers are able to better control for national-level characteristics, mainly cultural, socioeconomic, and institutional variables common to a given country Footnote 19 as explanatory variables, and can focus more directly on subnational factors (or on interactions between subnational- and national-level variables). Consequently, these studies contribute to a better understanding of within-country variation of sociopolitical phenomena and place more emphasis on the internal validity of the theoretical arguments.

However, only a few of these studies also acknowledge the challenges of this type of large-N within-country research design. In a recent article, Pepinsky ( 2017 ) highlights the importance of what he calls “regions of exception.” According to this author, these particular regions within a given country can pose limits to subnational comparative analysis and face serious limitations for causal inference, given “either [their] unobservability or indeterminacy.” Footnote 20 In broader terms, these cases can question the unit homogeneity assumption common in subnational (large-N) studies and should not be set aside as outliers. On the contrary, they must receive close attention and more emphasis needs to be put on idiographic explanations (Pepinsky 2017 ).

Another limitation rarely stated directly, particularly in the large-N analysis, is the assumption of the independence of the units of observation. Snyder ( 2001 ) already argued that policy diffusion or borrowing can be a problem, as it is highly likely that this process takes place among subnational units within a given country. Snyder himself offered a more research design-based solution: selecting non-contiguous subnational units from different countries. However, other authors like Ingram ( 2015 ) argue that this is a problem only in large-N studies Footnote 21 and that process tracing in more qualitatively oriented studies can take this possibility into account.

Within-country comparisons also suffer from other limitations. For one, there is an inherent (and in many cases acknowledged) scope condition that limits the theoretical arguments to only a single country. As Pepinsky ( 2019 ) claims, current single-country studies face the problem of “external validity and generalizability.” Despite the fact that recent studies have shown how case and small-N studies can help with testing and generalizing theoretical arguments (Rueschemeyer 2003 ; Slater and Ziblatt 2013 ), subnational within-country comparisons normally do not adequately justify the case selection at the country level.

In addition, in many cases, these studies test theories derived from national politics, but would benefit from a more explicit discussion about the possibility of incurring in “theory stretching” (Giraudy et al. 2019 ). Moreover, the meta-analysis reveals an important “large-nation bias” (Rokkan 1970 : 49) regarding which countries are studied; these are mainly Russia, Brazil, Argentina, Mexico, India, China, and Germany. Footnote 22 This probably occurs due to an implicit assumption that there is more territorial variation in this type of countries and due to better data availability. Finally, the low number of qualitative comparative studies of subnational units across countries is probably due to high costs in terms of immersion and thick knowledge acquisition for the understanding of both subnational and national politics, as well as costs in terms of time spent in the field, language requirements, and establishing networks of informants and other sources of information.

Typology of Multilevel Research Designs

This paper proposes a novel typology of multilevel research designs (MRDs). Footnote 23 A MRD is defined here as a research design in which cases from two (or more) levels of analysis are included and the principal units of analysis are at the subnational level. The typology is based on a combination of two criteria: number of countries and number of cases of analysis (Table 1 ). This combination yields four ideal types of MRDs. Footnote 24 On the one hand, and in line with Snyder’s original argument, subnational cases can be selected from a single country or two (or more) countries, yielding within- or between-country comparisons. Footnote 25 On the other hand, either there can be a purposeful case selection or (almost) all the subnational units are analyzed.

These MRDs do not presuppose the type of subnational units of analysis. They can be both jurisdictional (regions, districts, etc.) and non-jurisdictional (institutional o unbound), even though the former are more common. The latter can refer to processes, events, or phenomena that do not correspond with formal administrative boundaries (e.g., ethnic violence, organized crime, environmental problems, peace zones, borderlands). Footnote 26 Some studies even focus on subnational/territorial actors (e.g., insurgent or guerrilla groups, indigenous organizations, organized crime groups) as units of analysis. The MRDs are also agnostic regarding the type and direction of causal relationships (top-down, bottom-up, or interaction based Footnote 27 ). Besides, they do not assume the use of any specific method or instrument of data collection and analysis, despite the fact that elective affinities can exist. Particularly, when all the subnational units are included in the analysis, the author normally uses statistical methods or, to a lesser degree, qualitative comparative analysis (QCA).

First, when the researcher compares (subnational) cases in a single country and she in fact selects cases, it means that she is using the “single-country small-N (comparative) analysis” (SSNA). Two caveats are in place. First, the justification of case selection at the national level is normally vague or non-existent and, in any case, receives much less attention than the selection of subnational cases. Second, in this paper, and in line with Fig.  1 , this research design focuses on analysis with at least two subnational cases (be they from the same or different countries) and for the purposes of this typology does not engage with subnational single case studies (if they even exist, see Fig.  1 ). Footnote 28 Thus, the SSNA roughly corresponds to the qualitative version of Snyder’s SCM. This type of MRD normally involves selection of intermediate or local-level subnational units within a single country. Footnote 29

SSNA normally takes (even if implicitly) the form of a “most similar system design” (Przeworski and Teune 1970 ). Researchers seek to explain differences in the results of a certain phenomenon within the same country, assuming that they control for nationallevel variables. Therefore, they can focus on local-level variables (and their interaction with national ones) and explain differing outcomes in otherwise similar cases and the same national-level context.

Second, when the researcher analyzes (subnational) cases within a country and (almost) all subnational units are included in the analysis, Footnote 30 she is employing a “single-country large-N (statistical) analysis” (SLNA). Strictly speaking, there is no case selection at all at the subnational level. Normally, this comes in some version of a statistical analysis of cross-sectional (and even time series) format or QCA. In addition, there is often some kind of justification for the country selection Footnote 31 in terms of the “relationship to theory or posited X/Y relationship”; in other cases, more pragmatic reasons Footnote 32 seem to guide the selection of the country case (Koivu and Hinze 2017 ).

As mentioned before, this type of MRD is the most common in terms of the works that cite Snyder’s ( 2001 ) article. Unsurprisingly, this type of studies underlines the possibility of increasing N and of running a statistical model with all the subnational units to explain territorial variation in a given country with more internal validity. Accordingly, the advantages of this MRD reflect Snyder’s reasoning. Rogers ( 2014 : 11), for example, states that “by looking within a single country, we can hold cultural and historical factors constant while still investigating a relatively large number of cases,” while Murillo and Ronconi ( 2004 : 81) claim that “this subnational statistical comparison allows us to keep national historic and institutional variables constant, thereby facilitating the obtaining of findings that support valid causal inferences.” As Slater and Ziblatt ( 2013 ) stress, in these cases, large-N regression analyses serve to generate internal validity (and not to enhance external validity, as traditionally proposed).

Third, when the researcher selects cases from two (or more) countries and there is in fact a purposeful case selection at both levels, she is then using what is here called “cross-national small-N (comparative) analysis” (CSNA). The development of CSNA and its different logics is the main goal of this article and will be dealt with more depth in the following section. Suffice is to say that despite the advantages that this MRD offers, it remains rather underused and undertheorized.

Fourth, when the researcher selects cases from two or more countries and analyzes all the subnational units, she is making use of a “cross-national large-N (statistical) analysis” (CLNA). Normally, there is almost no theoretical justification for the cases included in the analysis and the case selection responds largely to data availability. For example, a researcher analyzes the data for all the municipalities in all the countries of one region. Generally, this type of MRD uses statistical analysis of cross-sectional (and time series) data, both on a national and subnational level. Hierarchical statistical models have long been used, and there is a vast (methodological) literature on them, Footnote 33 so it receives less attention here. Footnote 34

Overall, the within-country MRDs represent the majority of the works that has applied Snyder’s SCM. At the same time, both the qualitative and the quantitative versions put emphasis on internal validity (Pepinsky 2019 ). CLNA, on the contrary, prefers external validity and pays scarce attention to causal mechanisms and individual cases. Therefore, CSNA (with its three different logics) offers a middle ground between the two options. It strives for a reasonable trade-off between internal and external validity (Goertz and Mahoney 2012 ). It also focuses on causal mechanisms, processes, and contextual richness. Contrary to within-country MRD, it allows for the evaluation of the relative importance of national-level factors and the interaction between subnational and national variables. Moreover, given its qualitative nature and purposeful case selection at both levels, it tends to pay more attention to scope conditions.

Given its promise, the following section explores the different logics underpinning CSNA and evaluates its pros and cons. CSNA roughly corresponds to Snyder’s ( 2001 ) combination of within- and between-country comparisons. However, in the case of CSNA, there are not necessarily two or more subnational units within a single country. Consequently, there need not be, strictly speaking, a within-country comparison in this MRD. In some cases, authors choose just one subnational case per country (e.g., Eaton 2011 ; Simmons 2016 ; Holland 2017 ).

Cross-National Small-N Analysis

Cross-national small-N (comparative) analysis is a specific type of MRD that combines case selection and comparison of subnational units in different countries. As mentioned before, its increasing use demands a methodological discussion of its (dis)advantages. Snyder ( 2001 ) argued that, when executed properly, it contributes to overcoming some of the limitations of the other MRDs. On the one hand, it helps to control potential explanatory variables, and, on the other, it avoids the problems that arise with the (more or less) implicit assumption of independence among subnational cases.

CSNA also has other advantages. It compares subnational units in at least two countries. As such, the theoretical scope of the empirical argument (in terms of countries) is normally broader than in either SSNA or SLNA. In addition, its multilevel character with purposeful case selections at the national level allows it to identify causal interactions with the national level and evaluate its importance for the causal argument (Giraudy et al. 2019 ). Moreover, in contrast with CLNA, a smaller number of subnational cases allows for an easier detection of diffusion effects both within and between countries (Ingram 2015 ), usually in combination with process tracing. Finally, it is better equipped for explorative studies of new subnational phenomena that occur in different countries and whose principal goal is theory building.

The interest in comparing subnational units in different countries is hardly new in comparative politics. While Eulau ( 1962 ) and Smelser ( 1965 ) highlighted the possibility of “controlled comparison” within a single country, Footnote 35 Elder ( 1976 ) reviewed the works proposing cross-national comparisons of within-country units, be they subnational jurisdictional units or others. In this vein, Linz and de Miguel ( 1966 ) underlined the within-country differences in Spain ( ocho Españas ) and argued that a cross-national comparison of Spanish and Italian regions would be more meaningful. More specifically, “advanced and backward sections” of both countries could be compared more fruitfully in order to evaluate the role of national-level institutions. O’Donnell ( 1973 ) used the same logic in advocating for the comparison of “modern” and “peripheral” areas in Argentina and Brazil, to avoid erroneous conclusions derived from mean-spirited analyses Footnote 36 of socioeconomic indicators.

Both Linz and de Miguel and O’Donnell argued for pairing units from different countries, as they would be more similar in terms of cultural and socioeconomic characteristics. However, it was at an even earlier date that Lipset ( 1950 ) combined both types of comparison: within- and between-country (see Snyder 2001 ). Indeed, Lipset compared the strength and radicalism of agrarian movements in three Canadian provinces (Alberta, Manitoba, and Saskatchewan) to discern the role of subnational variables in the differences between these provinces. He also compared the bordering subnational units of Saskatchewan (Canadian province) and North Dakota (US state). They were very similar in subnational characteristics but under the influence of different national-level governments. This allowed him to ascertain the role of national-level variables in the resulting differences between Saskatchewan and North Dakota. Footnote 37

In more methodological terms, Rokkan ( 1970 ) warned early against the “whole-nation bias” that studies of political development suffered. This referred to the neglected variations in the statistical indicators used to characterized countries, which meant that Rokkan ( 1970 ) had in effect shed light on within-country differences (in socioeconomic indicators). More than three decades later, sociologists coined the term “methodological nationalism” (Wimmer and Schiller 2002 ) to question the fact that the nation-state had long been taken as a natural unit of analysis of social and political phenomena. Footnote 38 Only recently, political scientists have articulated the shift toward “regional political science” (Jeffery and Schakel 2013 ), by arguing for the need to focus more directly on subnational levels of political systems and take subnational units as units of analysis (Snyder 2001 ; Jeffery and Wincott 2010 ) of, for example, state presence, governments, decision-making, elections, coalitions, political parties, among many others.

Given this context, it is all the more striking that few methodological advances have been made since 2001. In a review of the “second generation” of subnational research, Moncada and Snyder ( 2012 ) mention the works of Lieberman ( 2005 ) and Coppedge ( 2005 ) and their “nested analysis” and “nested inference,” Footnote 39 respectively, as the methodological advances in the field. Footnote 40 However, strictly speaking, both authors contribute more to the mixed-methods literature than to the subnational or multilevel methodology. On her part, Riedl ( 2017 ) also proposes some methodological innovations related to “subnational to cross-national” research. She suggests that there are four ways to deal with “subnational to national variation.” Footnote 41 Still, these four research designs are more related to how causal inference is conceived in different types of MRDs and less to the case selection and levels of analysis themselves. In that sense, it speaks more directly to the types of causal relationships in Giraudy et al. ( 2019 ). As mentioned before, the types of MRDs proposed here are agnostic regarding the type and direction of causal relationships, as well as regarding the specific methods and instruments of data collection and analysis.

Arguably, Sellers ( 2019 ) offers the most similar (inductive) classification of what he calls “subnational comparison across borders” or “transnational comparison” to the one proposed here. The author identifies six different comparative research designs. The typology put forward here differs from Sellers ( 2019 ) in several ways. First, this paper focuses only on CSNA and it locates this type of research design within a broader typology. Second, as will be argued below, MRD can be used also for a within-country analysis as well. In other words, the multilevel nature of a research design can focus on comparing, for example, selected municipalities nested within regions (in a single country). By design, Sellers’ ( 2019 ) scheme does not contemplate this option. Third, the four MRDs examined here do not include the analysis of territorial variation of a national-level phenomenon, for instance national party system nationalization. Footnote 42 Fourth, as mentioned before, this paper does not contemplate the inclusion of the international level in the levels of analysis, although it acknowledges its importance. Fifth, it does not consider natural experiments Footnote 43 a separate type of MRD but rather as a subtype of CSNA, based, as Sellers ( 2019 ) himself argues, on the most similar design system logic.

Three Different Logics of Cross-National Small-N Analysis

This paper suggests that CSNA has been used with (at least) three different logics. Although there is a relatively small number of studies using this design (and not all cite Snyder’s article), some common patterns can be found. Broadly speaking, these relate to the logics of most similar and most different cases and exploratory studies, with selection on dependent variables. However, these research designs are more complex than the Przeworski and Teune’s ( 1970 ) traditional ones, as they include (at least) two different levels of case selection.

Most Similar Subnational Cases

The first logic of CSNA follows case selection based on the most similar system design (at the subnational level). Studies using this type of CSNA normally present the most sophisticated research designs and seek hypothesis testing. They can analyze both subnational phenomena as well as territorially uneven implementation and effects of national-level processes or politics. The goal of these research designs is to explain the territorial variation at the subnational level, be this local or intermediate. Thus, they compare similar paired subnational units (that normally differ only in one or two variables of theoretical interest) both within a single country and cross-nationally and look for the determinants of the different results. These similarities between the units are maximized in natural experiments using jurisdictional borders for as-if randomization.

However, this MRD goes beyond the SSNA as it combines within-country and between-country comparison. Therefore, it offers more external validity to the argument. Footnote 44 The exact justification for case selection at the national level may vary, despite the tendency to choose similar cases. On the one hand, researchers seek to demonstrate that the subnational explanation depends on the national level (either because of contextual differences or inter-level interactions) and thus select similar national cases with variation on one or more variable of interest. In both cases, country-level case selection is important and is usually well justified. On the other hand, in some studies, the authors use quite different national-level cases in order to discard important national-level variables and to show that the argument holds strong independently of the country’s context. Footnote 45

First, various works offer comparative subnational case studies, paired either within the same country (Giraudy 2015 ; McMann 2006 ) Footnote 46 or even between countries (Montambeault 2016 ). Footnote 47 In both cases, the underlying logic is the same. The authors seek to explain different outcomes (McMann 2006 ; Montambeault 2016 ) or different causal pathways (Giraudy 2015 ) in the most similar within-country cases with differences on the main independent variables’ effects. The differing national contexts play different roles for these authors. In the case of Giraudy ( 2015 ), she shows how subnational cases from different countries arrive at the same outcome via similar causal pathways Footnote 48 (although she does not pair them explicitly). On her part, Montambeault ( 2016 ) pairs the cases even between countries and finds that the reasons why local participatory institutions are more (or less) successful are quite similar in different national settings. Footnote 49

Second, various studies analyze national-level phenomena, but focus on its territorial variation Footnote 50 and use CSNA for the qualitative part of the project (Luna 2014 ; Ingram 2015 ; Niedzwiecki 2018 ). Footnote 51 In some cases (Ingram 2015 ; Niedzwiecki 2018 ), they make use of the nested research design (Lieberman 2005 ) in its subnational version, where a number of case studies follows a large-N statistical analysis of subnational units from a given country. Footnote 52 Again, at the lowest level of analysis, they seek to enhance the internal validity Footnote 53 of their argument by comparing similar subnational units in structured, controlled comparisons. However, the sample of congressional districts and municipalities (Luna 2014 ), states and municipalities (Niedzwiecki 2018 ), and states (Ingram 2015 ) for qualitative examination is chosen in order to maximize the variance on a host of socioeconomic and political factors, Footnote 54 and thus bolster the external validity of the argument within the countries.

Finally, research designs based on natural experiments (e.g., borders) exploit the same logic. Normally, the border creates as-if randomization in the treatment variable (Dunning 2012 : chapter 2) and the authors stress the importance of national-level factors (or their interaction with the subnational unit/actors). Posner’s ( 2004 ) study of cultural differences between Chewas and Tumbukas in Zambia and Malawi is an oft-cited example of this type of research design. Footnote 55 Posner argues that the cultural cleavage between both people is politically mobilized in Malawi (but not in Zambia) because they represent relatively large groups vis-à-vis the whole population and therefore serve for political coalition building. In this context, what really matters is the national context and how these groups relate to country-level characteristics.

Most Different National Cases, Same Subnational Phenomenon

The second logic of CSNA Footnote 56 is based on the selection of most different national cases, in combination with one subnational case in each country. Despite the fact that the case selection on the national level is not explicit in many cases, Footnote 57 it is on this level where the cases differ substantially. In other words, in this logic, CSNA analyzes cases from different national-level contexts and shows how certain subnational phenomena work the same way, according to broadly the same causal mechanisms. Footnote 58 Studies that use this logic try to maximize the variation in structural (national level) factors, precisely in order to rule them out as potential explanatory variables of the subnational phenomena. If the logic of subnational processes is similar in many different contexts, then the explanation is expected to be found at the subnational level. The main goal of these studies is normally theory building.

This type of MRD is particularly powerful in terms of external validity and the scope of the theoretical argument. Authors that use this research design seek to show that some subnational phenomena follow a similar logic in different parts of the world. As Simmons ( 2016 : 31) puts it, “the cases for comparison are selected based on the presence of apparently similar mechanisms, dynamics, and processes.” Indeed, this usually implies that the explanation is based on local factors, and national context may account for some contextual nuances but does not interfere with the main causal mechanism. Thus, in terms of generalization, it would be expected that where similar subnational phenomena shall unfold, the explanatory logic would tend to be the same. Within countries, however, the subnational cases are usually limited to a certain subset of jurisdictional units, actors, or phenomena, for example big or capital cities, certain type of municipalities, or social conflicts.

Various recent studies illustrate the basic logic of this CSNA design (Ostrom 1990 ; Simmons 2016 ; Pasotti 2017 ; Dosek 2019 ). Pasotti’s ( 2017 ) book offers an interesting application of this logic. She studies the shift from machine politics to a candidate-centered political branding politics in three cities: Bogotá, Naples, and Chicago. Pasotti shows in her theory-building book how a similar pattern of branding politics of city mayors developed in different national contexts. As she makes clear, these vary in terms of “socioeconomic, cultural, and institutional factors, thus offering a broad perspective.” Indeed, this variation sheds light on the contextual adaptation of these new political brands, as these brands are obviously not exactly the same. In the concluding part of her book, she inserts the “brand mayors” of these three cities in a comparative perspective, limiting her typology to the universe of large cities with established mass electoral politics. Therefore, she imposes scope conditions on the type of city and context and with secondary evidence proposes how “mayoral regimes” differ in cities where her three main independent variables (direct elections, self-finance, party discipline) vary.

Exploratory Dependent Variable Subnational Cases

The third logic of CSNA follows selecting on the dependent variables and is normally of an exploratory nature. It deals with new and rare cases of the same subnational phenomenon in different countries. However, authors using this MRD normally do not adequately justify the selecting on the national level, as they analyze subnational phenomena on its own. As those are few in number, country-level variables either are not really important (or taken into account) or help to explain only partially the differences in the development of the cases.

These case studies usually seek to understand, describe, and explain new phenomena. As such, in many cases, they trace the causal mechanism behind them. Footnote 59 It normally engages with theory building or offers theoretical insights and adjustment to existing theories. As to the latter, it often explores how national-level theories work on subnational levels and to what extent they are useful for understanding subnational phenomena. These studies are thus particularly advantageous for exploring the limits of current (national level) theories (i.e., their scope in terms of level of analysis) and to avoid “theory stretching” (Giraudy et al. 2019 ). Apart from theory construction, they also engage with new or understudied phenomena or propose new concepts.

Several studies illustrate this logic (Gibson 2005 ; Goldfrank 2007 ; Eaton 2011 ; Diez 2013 ). For example, take Eaton’s ( 2011 ) study. He explores the dynamics of “conservative autonomy movements” in Santa Cruz (Bolivia) and Guayas (Ecuador). These two cases of the same (new) phenomenon (subnational pro-market resistance to statist government at the national level) show slightly different results in terms of their success in promoting autonomy demands. Theoretically, Eaton engages with the social movement literature and offers a number of innovations. In terms of levels of analysis, the author argues that the Bolivian movement has more chances of confronting the central government thanks to a combination of (mainly) national and subnational factors. As to the former, the Bolivian case featured more pronounced ethnic divisions, more successful attempts of party building, and more profound neoliberal reforms in the past; regarding the latter, the Santa Cruz movement has more (financial) resources. Thus, different results are explained by a combination of national and subnational factors. Eaton explicitly shows that the national context is crucially important for the theoretical argument. However, similarly to Diez ( 2013 ) and Goldfrank ( 2007 ), the theoretical scope of the argument is not discussed Footnote 60 and the article does not mention other possible cases. Footnote 61

Single-Country Multilevel Comparisons

Small-N comparative analysis can also be carried out in a multilevel fashion within a single country. Although this type of research designs is much less common in existing literature, it offers a useful extension of research designs that combine within- and between-country comparisons. Thus, single-country multilevel comparisons (SCMC) can be considered a multilevel research design for studying subnational units at different levels in a single country.

Certainly, this MRD gains additional complexity by adding a third level. In this case, the national level can play different roles depending on the research goal. On the one hand, it can be taken as given when, for example, an electoral system or levels of decentralization in a country are part of the analysis, as there is normally no variation across the territory. On the other hand, it must be adequately taken into account if, for example, vertical interactions between political actors are to be examined. In general, single-country multilevel comparisons employ intermediate level subnational units as higher level units, and local jurisdictions, actors, policies, or other phenomena are the lower level units of analysis.

Generalizability is lower than in the cross-national research designs given that these studies are single-country studies. However, they are especially adequate for exploring territorial heterogeneity within a single country, offer an even more fine-grained picture of territorial variations, and permit the analysis of interactions between subnational levels or of the varying influence of national-level factors in the territory. Arguably, all three of the abovementioned logics of CSNA could be fruitfully applied in empirical SCMC studies, despite the first one appearing to be the most common, as the following examples show.

Two recent studies of clientelism in Argentina seek to explain the differences in its working across the provinces and generate more external validity to theories derived from a single city or province. On the one hand, Szwarcberg ( 2017 ) focuses on how clientelism works across the Argentine territory. She chooses two provinces (Buenos Aires and Cordoba) and eight municipalities. In both selections, she seeks to maximize the variation in theoretically important variables at both levels in order to obtain more similar cases for a structured comparison. On the other hand, Weitz-Shapiro ( 2016 ) seeks to explain why in some municipalities, politicians opt to not use clientelism as a political strategy. She chooses three provinces and a large number of municipalities maximizing the variation in historical, political, and economic conditions. Contrary to Szwarcberg ( 2017 ), she runs an econometric analysis at the municipal level to explain the variation in party leaders’ strategies and the decision not to engage in clientelistic practices. Similar multilevel design has been also used in unitary countries like Peru. Paredes and Došek ( 2020 ) explore the determinants of substantive indigenous representation in four provinces, pairing two of them within different regions of the country in order to explore their importance. In all these studies, the authors seek to maximize the territorial variation of the analyzed social phenomena and control for or take into account the variable of interest at the intermediate level.

Lessons Learned

Despite the diversity of logics that CSNA exhibits and its purported advantages over the other MRDs, the use of cross-national small-N analysis in empirical research is still relatively limited. This section offers some lessons learned and proposes some best practices for the use of CSNA. Some of them may also be applied more broadly to case selection processes in the other MRDs.

Indeed, as Snyder ( 2001 ) already suggested, CSNA offers a comparative advantage with respect to SSNA and SLNA in terms of theoretical/geographical scope of the principal argument and thus yields important gains in generalizability. This is in line with Slater and Ziblatt’s ( 2013 : 1313) recent claim that small-N–controlled comparison can be used to “generate” more external validity, although large-N tests may be needed to “confirm” it. Regardless, authors using CSNA need to be more explicit about the scope conditions of their arguments, which in turn requires better justification of case selections at both levels.

Particularly at the higher level, case selection should be more explicit and it should be clearer what its function is and why the authors choose precisely the cases that they do and not others (Koivu and Hinze 2017 ). Note that at the higher (normally, national) level the first two logics of CSNA achieve external validity in different ways (Table 2 ). While in the first one the author purposefully selects cases with variation on a certain variable of interest in order to test its influence, in the second, one authors tend to maximize national level differences.

At the subnational level, the scope conditions should be particularly clear in the second logic of CSNA. Subnational cases normally represent a certain type of jurisdiction, actor, or phenomenon. Typologies or classifications can help the authors locate them in a broader context of, for example, cities, as in the case of Pasotti ( 2017 ). These typologies help to understand what is particular about them and why the theory does not apply in the other types of subnational units or actors. In the case of exploratory CSNA, authors should consider how new and particular to a given region the studied phenomenon is and what other possible positive cases or non-trivial negative cases could be.

In many cases, particularly those using the first CSNA logic, the efforts to generalize and externally validate the theoretical argument seem rather superficial and anecdotal. In some cases, this might be due to the federal bias of the subnational comparisons (Zuo 2015 ), as relatively little is known about subnational dynamics in unitary countries. Extrapolating theoretical arguments derived from federal countries to unitary ones might therefore need further elaboration. Paradoxically, in some studies, the original case selection is justified in terms of federal countries but then the findings are sought to be generalized to unitary countries.

Different CSNA logics also serve different theoretical purposes. Generally, the first logics seeks theory testing whereas the second two logics seek theory building or adjustment. With units of analysis at the lower level, authors should clarify what the theoretical role of national-level variables is, be it whether they stress solely subnational factors (as is normally the case with the second CSNA logic) or whether the explanation resides in those national-level variables or in inter-level interactions. In this sense, recent advantages in the field should be taken advantage of and “theory stretching” from one level to another (Giraudy et al. 2019 ) should be avoided.

Conclusions

This article has sought to contribute to the debate on (multilevel) research designs. Since the publication of Snyder’s ( 2001 ) path-breaking article almost two decades ago, the number of within-country comparisons has grown steadily. His call for more subnational research was understood mainly as an invitation for within-country studies, particularly large-N analyses. These studies take advantage of making comparisons in a more controlled fashion and focus mainly on explaining territorial variation in a single country. Consequently, these single-country analyses lose between-country comparisons from sight and limit their generalizability. On the contrary, the use of research designs that combine (qualitative) within- and between-country comparisons has been rather scarce. This is ironic given that Snyder himself considered this type of MRD as the strongest in terms of the trade-off between generalization and case-oriented analysis.

The paper has identified and proposed three different logics that combine within- and between-country comparisons in a qualitative fashion, the cross-national small-N analysis (CSNA). It has inserted the CSNA within a broader typology of multilevel research design. In line with Snyder’s ( 2001 ) original argument, it shows how these three logics of CSNA (with their limitations) seek to maximize both internal and external validity. Thus, CSNA shows important versatility in terms of units of analysis (cases can be jurisdictional or not), goals (theory building or theory testing), ways to achieve external validity, and also in terms of its applicability to subnational levels within a single country.

Despite CSNA’s versatility, different logics, and abovementioned methodological advantages, subnational comparative research is still rather limited. In empirical terms, it suffers from a “federal bias,” typical in the study of subnational authoritarianism in Latin America (Pino Uribe 2017 ), as well as generally in research in the region (Zuo 2015 ). In any case, more empirical work should be done using CSNA in order to evaluate its contributions. The limited number of this type of study might be due in part to high costs in terms of knowledge acquisition of local-level political contexts and of variation in different countries. This is probably also why the majority of the CSNA works here cited are part of or are based on doctoral dissertation research.

Theoretically, authors should be more aware of the possibility of “theory stretching” (Giraudy et al. 2019 ) and should clarify more how their theories apply at different levels of analysis. More methodological advances are also needed to further explore both the potential and limitations of CSNA. Specifically, future research should focus more on how to analyze non-jurisdictional subnational units, on how to productively combine different research methods and logics, and the application of CSNA within a single country.

Although there is no consensus in the literature on whether SCM is a distinctive “method,” most recent works seem to assume so, more or less explicitly (Fox 2007 ; Pepinsky 2017 ; Giraudy et al. 2019 ). In any case, what makes it distinctive is not so much the logic of case selection as the levels where the cases are selected. Selecting both at the subnational and national level introduces further complexity and need for justification and also requires the analysis of (possible) vertical interactions between levels. In this sense, it goes beyond the Millean methods (see also Sellers 2019 ).

See Giraudy et al. ( 2019 ).

See Pepinsky ( 2019 ) for a recent discussion on “the return of the single country study” and its tradeoffs.

For the same point, see Lankina ( 2012 ) and Sinha ( 2015 ) who further argues for cross-regional comparisons.

Many authors use the term in the passing (Pepinsky 2017 ; Post 2018 ; Giraudy et al. 2019 ) but none really develop the concept. It seems to broadly refer to increased interest in the study of politics at the subnational level.

Indeed, rather few of the works reviewed by Snyder ( 2001 : footnote 1) featured comparison of subnational units from different countries (see Lipset 1950 ; Linz and de Miguel 1966 ; O’Donnell 1973 ; Tarrow 1976 ; Montero 2001 ); the vast majority were single-country studies.

These three logics partly concur with those proposed recently by Sellers ( 2019 ). However, see below for differences in conceptualization and logic of case selection.

The term “multilevel research design” is certainly not new. It has been used before by other authors, see, e.g., Murillo ( 2001 ), Luna ( 2014 ), Niedzwiecki ( 2018 ), or Giraudy et al. ( 2019 ). However, Giraudy et al. ( 2019 ) use it more in terms of theoretical explanations than research design (case selection).

Normally, the higher level refers to countries. However, as will be made clear below, this research design can also be applied at two different subnational levels within a given country.

Some important outlets for comparative research like Democratization, Studies in Comparative International Development, Politics and Society or Local Government Studies, to name a few, are normally not included.

They most probably include the vast majority of the articles found by Sellers ( 2019 ), assuming they cite Snyder ( 2001 ).

See Sellers ( 2019 ) for a quantitative analysis of leading comparative politics journals and monographs.

Indeed, much of what has been somewhat ambiguously dubbed “subnational turn” owes a big debt to Snyder’s seminal article and goes beyond the sphere of publications.

As of October 2018, according to Google Scholar.

This analysis does not mean to imply that there were no empirical works using multilevel research design or the subnational method before Snyder’s article was published. Of course, as his review shows, there were many of them (see particularly footnote 1, Snyder 2001 : 104). However, only one (Montero 2001 ) among the more recent works compares subnational units from different countries. Some of the classic studies are discussed below.

In fact, in Fig.  1 the third group consists of works that I was unable to check. These include studies that are unavailable (Google Scholar does not provide any link) or had restricted access (mainly dissertations). Also, there are books and book chapters I do not have access to and works in languages I do not speak (mainly, German, Russian, or Chinese).

On the one hand, many works include his article in the bibliography but do not cite it in the text of the article and thus make it impossible to determine how it was used. On the other hand, Google Scholar erroneously includes a number of studies that do not cite the article.

Indeed, this proportion is even larger if one considers only original empirical works.

Some authors nevertheless state, and Snyder ( 2001 ) himself recognizes, that in some countries the within-country variation of these variables can be huge (Pepinsky 2017 ).

Pepinsky ( 2017 : 1034) puts forward a solution to this problem in what he calls a “problem-driven approach” to examine how “such cases are defined by their relationship to the causal questions under consideration.”

Nonetheless, authors like Hiskey and Canache ( 2005 ) propose an empirical statistical model designed specifically to test the diffusion argument in subnational settings.

See Zuo ( 2015 : 324) for similar findings.

This typology differs from the one proposed by Giraudy et al. ( 2019 ) concerning “strategies of subnational research.” These authors combine the criteria of “type of causal relationship” and “number of levels of analysis.”

In general terms, all these designs are observational in nature. It is nevertheless true that they can be combined with experiments as a method in some of the lower level

cases or that the case selection at this level can take the form of a natural experiment.

In this paper, the international level is excluded for the sake of clarity.

See Moncada and Snyder ( 2012 ) and, particularly, Rodrigues-Silveira ( 2013 ) for the notion of “institutionally unbounded processes”.

See Giraudy et al. ( 2019 ) for a thorough discussion.

This is in line with Snyder’s article, where the author does not discuss single case studies and all the works that use the qualitative version of SCM analyze at least two subnational units. However, case and small-N studies are often grouped together for the purposes of research design (Munck and Verkuilen 2005 ; Gerring 2007 ).

Indeed, the same logic of case selection can be applied at different subnational levels, e.g., comparing municipalities within one province.

Here, all subnational units refer to all states, provinces, departments, or municipalities in a given country. The criterion for selecting (almost) all units might depend on data availability and may be more flexible, as long as the analysis is large-N and no additional qualitative case selection is employed.

For example, in her subnational analysis of clientelism in Argentina, Weitz-Shapiro ( 2016 : 18) justifies her national case selection basically by referring to the “relative pervasiveness of the practice” in this country. However, it could be argued that in other Latin American countries clientelism is equally or even more pervasive (Gonzalez-Ocantos and Oliveros 2019 ), though it has certainly received less attention in the comparative literature. Again, as with SSNA, the justification of the national-level case selection tends to be rather loose.

This refers to situations where two or more theoretically justifiable cases can be selected and choosing between them entails practical motives (e.g., time or logistical constraints, previous contacts and networks, funding access to information or language skills).

See Raudenbush and Bryk ( 2002 ) or Gelman and Hill ( 2007 ), among many others. Sellers ( 2019 ) includes this research design in his “subnational comparison across borders” classification, as an “encompassing subnational comparison.”

Indeed, as the above meta-analysis shows, Snyder’s article is also least cited in this type of studies.

Lijphart ( 1971 ) calls them “intranation comparisons.”

This refers to the use of national averages in cross-national comparison that conceal territorial variation in a given phenomenon within a country (Snyder 2001 : 98).

See below for a discussion on using borders for research designs based on natural experiments.

This challenge, in fact, advocated for a shift of focus to the transnational sphere.

Nested analysis or nested inference follows a similar logic and both refer to a mixed-method research strategy, whereby a scholar combines large-N statistical analysis with a subsequent intensive small-N analysis. The results of the former guide the case selection for the latter (Lieberman 2005 ). Lieberman was thinking of national-level cases; for application on at the subnational level, see Ingram ( 2015 ) and Niedzwiecki ( 2018 ).

Moreover, Denk ( 2010 ) sparked a discussion on using a new type of Qualitative Comparative Analysis (QCA) for a “comparative multilevel analysis” (Rohlfing 2012 ; Denk and Lethinen 2014 ; Thiem 2016 ). Harbers and Ingram’s ( 2017 ) article contributes to the discussion. Of course, there is now much more debate on mixed- and multi-method methodology. For two recent contributions, see Seawright ( 2016 ) and Goertz ( 2017 ). For a review of the current literature and a new research agenda, see Brookes ( 2017 ).

These four ways are (i) multilevel analysis: two level interaction; (ii) multilevel analysis: hierarchical model; (iii) testing the uniformity of national traits; and (iv) borders as quasi-experiments, see Riedl ( 2017 : 933-943) for details on each of them.

On the contrary, if, for example, the researcher tries to explain the variation of turnout in subnational elections using municipal data, it is considered SLNA.

Natural experiments are considered strictly speaking, observational studies (Dunning 2012 ), see Keele and Titiunik ( 2016 ) for natural experiments based on geography.

As Montambeault ( 2016 : 13) points out, “the weaknesses of small-N comparison are overcome by the fact that the analysis is based on multilevel comparisons. In fact, comparing both similar and different cities within two countries generates findings that increase the potential for midrange generalizations”.

However, they differ from the second logic of CSNA (see below), in that they select more than one case at the subnational level and use paired comparisons.

In fact, both Giraudy and McMann choose bordering subnational units. Both authors study the reproduction of subnational non-democratic regimes; Giraudy ( 2015 ) compares Mexican states and Argentine provinces and McMann ( 2006 ) Russian and Kyrgyzstani provinces ( oblast ).

Montambeault ( 2016 ) compares local participatory governance mechanisms and their effect on the quality of democracy in Brazilian and Mexican cities.

These pathways permit subnational undemocratic regimes’ continuity, although the predominant instrument of presidential power and the principal attribute of subnational governments to neutralize presidential power vary by country (Giraudy 2015 ).

These reasons relate to the interaction between the institutional design and local actors’ attitudes and strategies (Montambeault 2016 ).

In more theoretical terms, Giraudy et al. ( 2019 ) argue that some national-level events can have heterogeneous effects at the subnational level (top-down theories). They mention, among others, economic reforms, violence, or state capacities.

Luna ( 2014 ) studies differences in political parties’ linkage strategies in Chile and Uruguay. Ingram ( 2015 ) examines the variation in subnational judicial capacity and the institutional reforms in Brazilian and Mexican states. Niedzwiecki ( 2018 ) seeks to explain territorial differences in the implementation of social policies in Brazilian states and Argentine provinces and municipalities in both countries.

Both Ingram ( 2015 ) and Niedzwiecki ( 2018 ) introduce time dimension to their analysis and run time-series cross-section models with intermediate subnational units. Luna captures the temporal dimension by comparing data from fieldwork in two different moments of time.

As Luna ( 2014 : 14) underlines, this case selection confers “greater levels of internal validity to my causal inferences, even where there are limited degrees of freedom.”

This is less so in the case of Ingram ( 2015 ). The variation is based only on two variables.

Even before natural experiments became popular (and the terminology coined), Lancaster ( 1987 ) offered a study with a similar logic (see also Lipset 1950 ). In his article, the author compared Basques in Spain and France and identified significant differences in their national self-identification. Lancaster ( 1987 ) argued that this phenomenon was due to differing state policies toward them in both countries, see also Linz ( 1986 ) for a similar comparison.

This logic roughly corresponds to what Sellers ( 2019 ) calls “Matched Subnational Cases in ‘Most Different’ National Systems.”

See for example Pasotti ( 2017 ).

Simmons ( 2016 : 6) maintains that in these cases “a micro-level analysis reveals important commonalities.”

In this sense, they resemble the “explaining-outcome process tracing” proposed by Beach and Pedersen ( 2016 ).

Although Eaton does make it clear that these conservative movements differ from their progressive counterparts, and that consequently so do the explanations of their possibilities of success.

However, Eaton ( 2011 ) briefly mentions Guatemala, Peru, and Venezuela.

Beach D, Pedersen RB. Process-tracing methods. Foundations and guidelines. Ann Arbor: The University of Michigan Press; 2016.

Google Scholar  

Brady HE, Collier D. Editors. Rethinking social inquiry. Diverse tool, shared standards. Lanham: Rowman & Littlefield Publishers, Inc; 2010.

Brookes M. The road less traveled: an agenda for mixed-methods research. PS: Pol Sci & Pol. 2017;50(4):1015–8.

Coppedge M. Explaining democratic deterioration in Venezuela through nested inference. In: Mainwaring SP, Hagopian F, editors. The Third Wave of Democratization in Latin America. Advances and Setbacks. New York: Cambridge University Press; 2005. p. 289–316.

Denk T. Comparative multilevel analysis: proposal for a methodology. Int J Soc Res Methodol. 2010;13(1):29–39.

Denk T, Lethinen S. Contextual analyses with QCA-methods. Qual Quant. 2014;48(6):3475–87.

Diez J. Explaining policy outcomes: the adoption of same-sex unions in Buenos Aires and Mexico City. Comp Polit Stud. 2013;46(2):212–35.

Došek T. La persistencia de caudillos locales en América Latina [Ph.D.]. Santiago: Pontificia Universidad Católica de Chile; 2019.

Dunning T. Natural experiments in the social sciences: a design-based approach. New York: Cambridge University Press; 2012.

Eaton K. Conservative Autonomy Movements. Territorial dimensions of ideological conflict in Bolivia and Ecuador. Comp Polit. 2011;43(3):291–310.

Elder JW. Comparative cross-national methodology. Annu Rev Soc. 1976;2:209–30.

Article   Google Scholar  

Eulau H. Comparative political analysis: a methodological note. Midwest J Pol Sci. 1962;6(4):397–407.

Fox J. Accountability politics: power and voice in rural Mexico. New York: Oxford University Press; 2007.

Book   Google Scholar  

Gelman A, Hill J. Data analysis using regression and multilevel/hierarchical models. New York: Cambridge University Press; 2007.

Gerring J. Case study research: principles and practices. Cambridge: Cambridge University Press; 2007.

Gibson EL. Boundary control. Subnational authoritarianism in democratic countries. World Pol. 2005;58(1):101–32.

Giraudy A. Democrats and autocrats. Pathways of subnational undemocratic regime continuity within democratic countries. New York: Oxford University Press; 2015.

Giraudy A, Moncada E, Snyder R. Subnational research in comparative politics: achievements and future directions. In: Giraudy A, Moncada E, Snyder R, editors. Inside countries: subnational research in comparative politics. New York: Cambridge University Press; 2019. p. 3–54.

Goertz G. Multimethod research, causal mechanisms, and case studies. An integrated approach. Princeton: Princeton University Press; 2017.

Goertz G, Mahoney J. A tale of two cultures. Qualitative and quantitative research in the social sciences. Princeton: Princeton University Press; 2012.

Goldfrank B. The politics of deepening local democracy: decentralization, party institutionalization, and participation. Comp Polit. 2007;39(2):147–68.

Gonzalez-Ocantos E, Oliveros E. Clientelism in Latin American politics. In: Gary Prevost G, Vanden H, editors. The Oxford Encyclopedia of Latin American Politics. Oxford: Oxford University Press; 2019. https://doi.org/10.1093/acrefore/9780190228637.013.1677 .

Harbers I, Ingram MC. Incorporating space in multimethod research: combining spatial analysis with case-study research. PS: Pol Sci & Pol. 2017;50(4):1032–7.

Hiskey J, Canache D. The demise of one-party politics in Mexican municipal elections. Br J Pol Sci. 2005;35(2):257–84.

Holland AC. Forbearance as Redistribution. The politics of informal welfare in Latin America. New York: Cambridge University Press; 2017.

Ingram MC. Crafting courts in new democracies: the politics of subnational judicial reform in Brazil and Mexico. New York: Cambridge University Press; 2015.

Jeffery C, Schakel AH. Editorial: towards a regional political Science. Reg Stud. 2013;47(3):299–302.

Jeffery C, Wincott D. The challenge of territorial politics: beyond methodological nationalism. In: Hay C, editor. New directions in political science. London: Palgrave; 2010. p. 167–88.

Keele L, Titiunik R. Natural experiments based on geography. Pol Sci Res Met. 2016;4(1):65–95.

Koivu KL, Hinze AM. 2017. Cases of convenience? The divergence of theory from practice in case selection in qualitative and mixed-methods research. PS: Polit Sci Pol. 2017;50(4):1023–7.

Lancaster TD. Comparative nationalism: the Basques in Spain and France. Eur J Polit Res. 1987;15(5):561–90.

Lankina T. Sisyphean endeavor or worthwhile undertaking? Transcending within-nation, within region sub-national democracy analysis. American Political Science Association, APSA Comparative Democratization Newsletter. 2012;2:14–8.

Lieberman ES. Nested analysis as a mixed-method strategy for comparative research. Am Pol Sci Rev. 2005;99(3):435–52.

Lijphart A. Comparative politics and the comparative method. Am Pol Sci Rev. 1971;65(3):682–93.

Linz JJ. Conflicto en Euskadi. Madrid: Espasa Calpe; 1986.

Linz JJ, de Miguel A. Within-nation differences and comparisons: the eight Spains. In: Merritt RL, Rokkan S, editors. Comparing nations: the use of quantitative data in cross-National Research. New Haven: Yale University Press; 1966. p. 267–319.

Lipset SM. Agrarian socialism: the cooperative commonwealth Federation in Saskatchewan. Berkeley: University of California Press; 1950.

Luna JP. Segmented representation. Political party strategies in unequal democracies. New York: Oxford University Press; 2014.

McMann KM. Economic autonomy and democracy. Hybrid regimes in Russia and Kyrgyzstan. New York: Cambridge University Press; 2006.

Moncada E, Snyder R. Subnational comparative research on democracy: taking stock and looking forward. American Political Science Association, APSA Comparative Democratization Newsletter 2012;1:4–9.

Montambeault F. The politics of local participatory democracy in Latin America. Institutions, actors, and interactions. Stanford: Stanford University Press; 2016.

Montero AP. Delegative dilemmas and horizontal logics: Subnational industrial policy in Spain and Brazil. Stud in Comp Int Dev. 2001;36(3):58–87.

Munck G; Verkuilen J. Research Designs. In: Kempf-Leonard K., editor. Encyclopedia of Social Measurement (Vol. 3). San Diego: Academic Press; 2005. p. 385–395.

Munck GL, Snyder, R. Debating the direction of comparative politics. An analysis of leading journals. Comp Polit Stud. 2007;(40)1:5–31.

Murillo MV. Labor unions, partisan coalitions, and market reforms in Latin America. New York: Cambridge University Press; 2001.

Murillo MV, Ronconi L. Teachers’ strikes in Argentina: partisan alignments and public-sector labor relations. Stud Comp Int Dev. 2004;39(l):77–98.

Niedzwiecki S. Uneven social policies. The politics of subnational variation in Latin America. New York: Cambridge University Press; 2018.

O’Donnell G. Modernization and bureaucratic-authoritarianism: studies in south American politics. Berkeley: Institute of International Studies, University of California, Berkeley; 1973.

Ostrom E. Governing the commons: the evolution of institutions for collective action. New York: Cambridge University Press; 1990.

Paredes M, Došek T. The subnational indigenous quota in Peru: the paradoxes of political representation. Lat Am Pol Soc. 2020;62(3):123–48.

Pasotti E. Political branding in cities. The decline of machine politics in Bogotá, Naples, and Chicago. New York: Cambridge University Press; 2017.

Pepinsky T. Regions of exception. Perspectives on Pol. 2017;15(4):1034–52.

Pepinsky T. The return of the single country study. Annu Rev of Pol Sci. 2019;22:187–203.

Pino Uribe JF. Entre democracias y autoritarismos: una mirada crítica al estudio de la democracia subnacional en Colombia y Latinoamérica. Col Int. 2017;91:215–42.

Posner DN. The political salience of cultural difference: why Chewas and Tumbukas are allies in Zambia and adversaries in Malawi. Am Pol Sci Rev. 2004;98(4):529–45.

Post AF. Cities and politics in the developing world. Annu Rev of Pol Sci. 2018;21:115–33.

Przeworski A, Teune H. The logic of comparative social inquiry. New York: John Wiley and Sons; 1970.

Raudenbush SW, Bryk AS. Hierarchical linear models. Applications and data analysis methods. Thousand Oaks: Sage Publications; 2002.

Riedl RB. Sub-National–Cross-National Variation: Method and analysis in sub-Saharan Africa. Am Behav Sci. 2017;61(8):932–59.

Rodrigues-Silveira R. The Subnational Method and Social Policy Provision: Socioeconomic Context, Political Institutions and Spatial Inequality. Working Paper No. 36, desiguALdades.net , Berlin, Germany; 2013.

Rogers MZ. Taxing with dictators and democrats: regime effects, transfers and revenue in Argentina’s provinces, in: journal of politics in Latin America. J Pol Latin Am. 2014;6(1):3–44.

Rohlfing I. Analyzing multilevel data with QCA: a straightforward procedure. Int J Soc Res Methodol. 2012;15(6):497–506.

Rokkan S. Citizens, elections, parties: approaches to the comparative study of the processes of development. New York: David McKay Company; 1970.

Rueschemeyer D. Can one of a few cases yield theoretical gains. In: Mahoney J, Rueschemeyer D, editors. New York: Cambridge University Press; 2003. p. 305–36.

Seawright J. Multi-Method Social Science. Combining qualitative and quantitative tools. New York: Cambridge University Press; 2016.

Sellers JM. From within to between nations: subnational comparison across borders. Perspectives on Pol. 2019;17(1):85–105.

Simmons ES. Meaningful resistance: market reforms and the roots of social protest in Latin America. New York: Cambridge University Press; 2016.

Sinha A. Scaling up: beyond the subnational comparative method for India. Stud Ind Pol. 2015;3(1):128–33.

Slater D, Ziblatt D. The enduring indispensability of the controlled comparison. Comp Polit Stud. 2013;46(10):1301–27.

Smelser NJ. Notes on the methodology of comparative analysis of economic activity. Paris: UNESCO; 1965.

Snyder R. Scaling down: the subnational comparative method. Stud Comp Int Dev. 2001;36(1):93–110.

Szwarcberg M. Mobilizing Poor Voters. Machine politics, clientelism, and social networks in Argentina. New York: Cambridge University Press; 2017.

Tarrow S. From Center to Periphery: Alternative models of national-local policy impact and an application to France and Italy. Ithaca: Western Societies Program, Cornell University; 1976.

Thiem A. Analyzing multilevel data with QCA: yet another straightforward procedure. Qual & Quant. 2016;50(1):121–8.

Weitz-Shapiro R. Curbing clientelism in Argentina. Politics, poverty, and social policy. New York: Cambridge University Press; 2016.

Wimmer A, Schiller NG. Methodological nationalism, the social sciences, and the study of migration: an essay in historical epistemology. Int Migr Rev. 2002;37(3):576–610.

Zuo C. Scaling down: Subnational comparative case studies in comparative politics and Chinese politics. Eur Pol Sci. 2015;14(3):318–39.

Download references

Acknowledgments

I thank two anonymous reviewers and members of the SCID Editorial Collective for their helpful comments and suggestions on earlier drafts. I am also grateful to the participants of Fourth Southwest Workshop on Mixed Methods Research, University of California, Santa Cruz, November 8-9, 2018, for their insights.

Author information

Authors and affiliations.

Departamento de Ciencias Sociales, Pontificia Universidad Católica del Perú, Lima, Peru

Tomas Dosek

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Tomas Dosek .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Dosek, T. Multilevel Research Designs: Case Selection, Levels of Analysis, and Scope Conditions. St Comp Int Dev 55 , 460–480 (2020). https://doi.org/10.1007/s12116-020-09313-6

Download citation

Published : 15 August 2020

Issue Date : December 2020

DOI : https://doi.org/10.1007/s12116-020-09313-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Multilevel research design
• Cross-national small-N analysis
• Case selection
• Scope conditions
• Subnational comparative method
• Find a journal
• Publish with us
• Track your research