qualitative research validation

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Family Med Prim Care
• v.4(3); Jul-Sep 2015

Validity, reliability, and generalizability in qualitative research

Lawrence leung.

1 Department of Family Medicine, Queen's University, Kingston, Ontario, Canada

2 Centre of Studies in Primary Care, Queen's University, Kingston, Ontario, Canada

In general practice, qualitative research contributes as significantly as quantitative research, in particular regarding psycho-social aspects of patient-care, health services provision, policy setting, and health administrations. In contrast to quantitative research, qualitative research as a whole has been constantly critiqued, if not disparaged, by the lack of consensus for assessing its quality and robustness. This article illustrates with five published studies how qualitative research can impact and reshape the discipline of primary care, spiraling out from clinic-based health screening to community-based disease monitoring, evaluation of out-of-hours triage services to provincial psychiatric care pathways model and finally, national legislation of core measures for children's healthcare insurance. Fundamental concepts of validity, reliability, and generalizability as applicable to qualitative research are then addressed with an update on the current views and controversies.

Nature of Qualitative Research versus Quantitative Research

The essence of qualitative research is to make sense of and recognize patterns among words in order to build up a meaningful picture without compromising its richness and dimensionality. Like quantitative research, the qualitative research aims to seek answers for questions of “how, where, when who and why” with a perspective to build a theory or refute an existing theory. Unlike quantitative research which deals primarily with numerical data and their statistical interpretations under a reductionist, logical and strictly objective paradigm, qualitative research handles nonnumerical information and their phenomenological interpretation, which inextricably tie in with human senses and subjectivity. While human emotions and perspectives from both subjects and researchers are considered undesirable biases confounding results in quantitative research, the same elements are considered essential and inevitable, if not treasurable, in qualitative research as they invariable add extra dimensions and colors to enrich the corpus of findings. However, the issue of subjectivity and contextual ramifications has fueled incessant controversies regarding yardsticks for quality and trustworthiness of qualitative research results for healthcare.

Impact of Qualitative Research upon Primary Care

In many ways, qualitative research contributes significantly, if not more so than quantitative research, to the field of primary care at various levels. Five qualitative studies are chosen to illustrate how various methodologies of qualitative research helped in advancing primary healthcare, from novel monitoring of chronic obstructive pulmonary disease (COPD) via mobile-health technology,[ 1 ] informed decision for colorectal cancer screening,[ 2 ] triaging out-of-hours GP services,[ 3 ] evaluating care pathways for community psychiatry[ 4 ] and finally prioritization of healthcare initiatives for legislation purposes at national levels.[ 5 ] With the recent advances of information technology and mobile connecting device, self-monitoring and management of chronic diseases via tele-health technology may seem beneficial to both the patient and healthcare provider. Recruiting COPD patients who were given tele-health devices that monitored lung functions, Williams et al. [ 1 ] conducted phone interviews and analyzed their transcripts via a grounded theory approach, identified themes which enabled them to conclude that such mobile-health setup and application helped to engage patients with better adherence to treatment and overall improvement in mood. Such positive findings were in contrast to previous studies, which opined that elderly patients were often challenged by operating computer tablets,[ 6 ] or, conversing with the tele-health software.[ 7 ] To explore the content of recommendations for colorectal cancer screening given out by family physicians, Wackerbarth, et al. [ 2 ] conducted semi-structure interviews with subsequent content analysis and found that most physicians delivered information to enrich patient knowledge with little regard to patients’ true understanding, ideas, and preferences in the matter. These findings suggested room for improvement for family physicians to better engage their patients in recommending preventative care. Faced with various models of out-of-hours triage services for GP consultations, Egbunike et al. [ 3 ] conducted thematic analysis on semi-structured telephone interviews with patients and doctors in various urban, rural and mixed settings. They found that the efficiency of triage services remained a prime concern from both users and providers, among issues of access to doctors and unfulfilled/mismatched expectations from users, which could arouse dissatisfaction and legal implications. In UK, a care pathways model for community psychiatry had been introduced but its benefits were unclear. Khandaker et al. [ 4 ] hence conducted a qualitative study using semi-structure interviews with medical staff and other stakeholders; adopting a grounded-theory approach, major themes emerged which included improved equality of access, more focused logistics, increased work throughput and better accountability for community psychiatry provided under the care pathway model. Finally, at the US national level, Mangione-Smith et al. [ 5 ] employed a modified Delphi method to gather consensus from a panel of nominators which were recognized experts and stakeholders in their disciplines, and identified a core set of quality measures for children's healthcare under the Medicaid and Children's Health Insurance Program. These core measures were made transparent for public opinion and later passed on for full legislation, hence illustrating the impact of qualitative research upon social welfare and policy improvement.

Overall Criteria for Quality in Qualitative Research

Given the diverse genera and forms of qualitative research, there is no consensus for assessing any piece of qualitative research work. Various approaches have been suggested, the two leading schools of thoughts being the school of Dixon-Woods et al. [ 8 ] which emphasizes on methodology, and that of Lincoln et al. [ 9 ] which stresses the rigor of interpretation of results. By identifying commonalities of qualitative research, Dixon-Woods produced a checklist of questions for assessing clarity and appropriateness of the research question; the description and appropriateness for sampling, data collection and data analysis; levels of support and evidence for claims; coherence between data, interpretation and conclusions, and finally level of contribution of the paper. These criteria foster the 10 questions for the Critical Appraisal Skills Program checklist for qualitative studies.[ 10 ] However, these methodology-weighted criteria may not do justice to qualitative studies that differ in epistemological and philosophical paradigms,[ 11 , 12 ] one classic example will be positivistic versus interpretivistic.[ 13 ] Equally, without a robust methodological layout, rigorous interpretation of results advocated by Lincoln et al. [ 9 ] will not be good either. Meyrick[ 14 ] argued from a different angle and proposed fulfillment of the dual core criteria of “transparency” and “systematicity” for good quality qualitative research. In brief, every step of the research logistics (from theory formation, design of study, sampling, data acquisition and analysis to results and conclusions) has to be validated if it is transparent or systematic enough. In this manner, both the research process and results can be assured of high rigor and robustness.[ 14 ] Finally, Kitto et al. [ 15 ] epitomized six criteria for assessing overall quality of qualitative research: (i) Clarification and justification, (ii) procedural rigor, (iii) sample representativeness, (iv) interpretative rigor, (v) reflexive and evaluative rigor and (vi) transferability/generalizability, which also double as evaluative landmarks for manuscript review to the Medical Journal of Australia. Same for quantitative research, quality for qualitative research can be assessed in terms of validity, reliability, and generalizability.

Validity in qualitative research means “appropriateness” of the tools, processes, and data. Whether the research question is valid for the desired outcome, the choice of methodology is appropriate for answering the research question, the design is valid for the methodology, the sampling and data analysis is appropriate, and finally the results and conclusions are valid for the sample and context. In assessing validity of qualitative research, the challenge can start from the ontology and epistemology of the issue being studied, e.g. the concept of “individual” is seen differently between humanistic and positive psychologists due to differing philosophical perspectives:[ 16 ] Where humanistic psychologists believe “individual” is a product of existential awareness and social interaction, positive psychologists think the “individual” exists side-by-side with formation of any human being. Set off in different pathways, qualitative research regarding the individual's wellbeing will be concluded with varying validity. Choice of methodology must enable detection of findings/phenomena in the appropriate context for it to be valid, with due regard to culturally and contextually variable. For sampling, procedures and methods must be appropriate for the research paradigm and be distinctive between systematic,[ 17 ] purposeful[ 18 ] or theoretical (adaptive) sampling[ 19 , 20 ] where the systematic sampling has no a priori theory, purposeful sampling often has a certain aim or framework and theoretical sampling is molded by the ongoing process of data collection and theory in evolution. For data extraction and analysis, several methods were adopted to enhance validity, including 1 st tier triangulation (of researchers) and 2 nd tier triangulation (of resources and theories),[ 17 , 21 ] well-documented audit trail of materials and processes,[ 22 , 23 , 24 ] multidimensional analysis as concept- or case-orientated[ 25 , 26 ] and respondent verification.[ 21 , 27 ]

Reliability

In quantitative research, reliability refers to exact replicability of the processes and the results. In qualitative research with diverse paradigms, such definition of reliability is challenging and epistemologically counter-intuitive. Hence, the essence of reliability for qualitative research lies with consistency.[ 24 , 28 ] A margin of variability for results is tolerated in qualitative research provided the methodology and epistemological logistics consistently yield data that are ontologically similar but may differ in richness and ambience within similar dimensions. Silverman[ 29 ] proposed five approaches in enhancing the reliability of process and results: Refutational analysis, constant data comparison, comprehensive data use, inclusive of the deviant case and use of tables. As data were extracted from the original sources, researchers must verify their accuracy in terms of form and context with constant comparison,[ 27 ] either alone or with peers (a form of triangulation).[ 30 ] The scope and analysis of data included should be as comprehensive and inclusive with reference to quantitative aspects if possible.[ 30 ] Adopting the Popperian dictum of falsifiability as essence of truth and science, attempted to refute the qualitative data and analytes should be performed to assess reliability.[ 31 ]

Generalizability

Most qualitative research studies, if not all, are meant to study a specific issue or phenomenon in a certain population or ethnic group, of a focused locality in a particular context, hence generalizability of qualitative research findings is usually not an expected attribute. However, with rising trend of knowledge synthesis from qualitative research via meta-synthesis, meta-narrative or meta-ethnography, evaluation of generalizability becomes pertinent. A pragmatic approach to assessing generalizability for qualitative studies is to adopt same criteria for validity: That is, use of systematic sampling, triangulation and constant comparison, proper audit and documentation, and multi-dimensional theory.[ 17 ] However, some researchers espouse the approach of analytical generalization[ 32 ] where one judges the extent to which the findings in one study can be generalized to another under similar theoretical, and the proximal similarity model, where generalizability of one study to another is judged by similarities between the time, place, people and other social contexts.[ 33 ] Thus said, Zimmer[ 34 ] questioned the suitability of meta-synthesis in view of the basic tenets of grounded theory,[ 35 ] phenomenology[ 36 ] and ethnography.[ 37 ] He concluded that any valid meta-synthesis must retain the other two goals of theory development and higher-level abstraction while in search of generalizability, and must be executed as a third level interpretation using Gadamer's concepts of the hermeneutic circle,[ 38 , 39 ] dialogic process[ 38 ] and fusion of horizons.[ 39 ] Finally, Toye et al. [ 40 ] reported the practicality of using “conceptual clarity” and “interpretative rigor” as intuitive criteria for assessing quality in meta-ethnography, which somehow echoed Rolfe's controversial aesthetic theory of research reports.[ 41 ]

Food for Thought

Despite various measures to enhance or ensure quality of qualitative studies, some researchers opined from a purist ontological and epistemological angle that qualitative research is not a unified, but ipso facto diverse field,[ 8 ] hence any attempt to synthesize or appraise different studies under one system is impossible and conceptually wrong. Barbour argued from a philosophical angle that these special measures or “technical fixes” (like purposive sampling, multiple-coding, triangulation, and respondent validation) can never confer the rigor as conceived.[ 11 ] In extremis, Rolfe et al. opined from the field of nursing research, that any set of formal criteria used to judge the quality of qualitative research are futile and without validity, and suggested that any qualitative report should be judged by the form it is written (aesthetic) and not by the contents (epistemic).[ 41 ] Rolfe's novel view is rebutted by Porter,[ 42 ] who argued via logical premises that two of Rolfe's fundamental statements were flawed: (i) “The content of research report is determined by their forms” may not be a fact, and (ii) that research appraisal being “subject to individual judgment based on insight and experience” will mean those without sufficient experience of performing research will be unable to judge adequately – hence an elitist's principle. From a realism standpoint, Porter then proposes multiple and open approaches for validity in qualitative research that incorporate parallel perspectives[ 43 , 44 ] and diversification of meanings.[ 44 ] Any work of qualitative research, when read by the readers, is always a two-way interactive process, such that validity and quality has to be judged by the receiving end too and not by the researcher end alone.

In summary, the three gold criteria of validity, reliability and generalizability apply in principle to assess quality for both quantitative and qualitative research, what differs will be the nature and type of processes that ontologically and epistemologically distinguish between the two.

Source of Support: Nil.

Conflict of Interest: None declared.

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals

You are here

• Volume 18, Issue 2
• Issues of validity and reliability in qualitative research
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Helen Noble 1 ,
• Joanna Smith 2
• 1 School of Nursing and Midwifery, Queens's University Belfast , Belfast , UK
• 2 School of Human and Health Sciences, University of Huddersfield , Huddersfield , UK
• Correspondence to Dr Helen Noble School of Nursing and Midwifery, Queens's University Belfast, Medical Biology Centre, 97 Lisburn Rd, Belfast BT9 7BL, UK; helen.noble{at}qub.ac.uk

https://doi.org/10.1136/eb-2015-102054

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Evaluating the quality of research is essential if findings are to be utilised in practice and incorporated into care delivery. In a previous article we explored ‘bias’ across research designs and outlined strategies to minimise bias. 1 The aim of this article is to further outline rigour, or the integrity in which a study is conducted, and ensure the credibility of findings in relation to qualitative research. Concepts such as reliability, validity and generalisability typically associated with quantitative research and alternative terminology will be compared in relation to their application to qualitative research. In addition, some of the strategies adopted by qualitative researchers to enhance the credibility of their research are outlined.

Are the terms reliability and validity relevant to ensuring credibility in qualitative research?

Although the tests and measures used to establish the validity and reliability of quantitative research cannot be applied to qualitative research, there are ongoing debates about whether terms such as validity, reliability and generalisability are appropriate to evaluate qualitative research. 2–4 In the broadest context these terms are applicable, with validity referring to the integrity and application of the methods undertaken and the precision in which the findings accurately reflect the data, while reliability describes consistency within the employed analytical procedures. 4 However, if qualitative methods are inherently different from quantitative methods in terms of philosophical positions and purpose, then alterative frameworks for establishing rigour are appropriate. 3 Lincoln and Guba 5 offer alternative criteria for demonstrating rigour within qualitative research namely truth value, consistency and neutrality and applicability. Table 1 outlines the differences in terminology and criteria used to evaluate qualitative research.

• View inline

Terminology and criteria used to evaluate the credibility of research findings

What strategies can qualitative researchers adopt to ensure the credibility of the study findings?

Unlike quantitative researchers, who apply statistical methods for establishing validity and reliability of research findings, qualitative researchers aim to design and incorporate methodological strategies to ensure the ‘trustworthiness’ of the findings. Such strategies include:

Accounting for personal biases which may have influenced findings; 6

Acknowledging biases in sampling and ongoing critical reflection of methods to ensure sufficient depth and relevance of data collection and analysis; 3

Meticulous record keeping, demonstrating a clear decision trail and ensuring interpretations of data are consistent and transparent; 3 , 4

Establishing a comparison case/seeking out similarities and differences across accounts to ensure different perspectives are represented; 6 , 7

Including rich and thick verbatim descriptions of participants’ accounts to support findings; 7

Demonstrating clarity in terms of thought processes during data analysis and subsequent interpretations 3 ;

Engaging with other researchers to reduce research bias; 3

Respondent validation: includes inviting participants to comment on the interview transcript and whether the final themes and concepts created adequately reflect the phenomena being investigated; 4

Data triangulation, 3 , 4 whereby different methods and perspectives help produce a more comprehensive set of findings. 8 , 9

Table 2 provides some specific examples of how some of these strategies were utilised to ensure rigour in a study that explored the impact of being a family carer to patients with stage 5 chronic kidney disease managed without dialysis. 10

Strategies for enhancing the credibility of qualitative research

In summary, it is imperative that all qualitative researchers incorporate strategies to enhance the credibility of a study during research design and implementation. Although there is no universally accepted terminology and criteria used to evaluate qualitative research, we have briefly outlined some of the strategies that can enhance the credibility of study findings.

• Sandelowski M
• Lincoln YS ,
• Barrett M ,
• Mayan M , et al
• Greenhalgh T
• Lingard L ,

Twitter Follow Joanna Smith at @josmith175 and Helen Noble at @helnoble

Competing interests None.

Read the full text or download the PDF:

Qualitative Research and Content Validity

• Reference work entry
• pp 5257–5265
• Cite this reference work entry

• Meryl Brod 3 ,
• Betsy Pohlman 3 &
• Laura Tesler Waldman 4  

1010 Accesses

4 Citations

Index of sustainable economic well-being

Qualitative research investigates events that are difficult to quantify mathematically, such as beliefs.

Content validity , derived during concept elicitation, is the measurement property that assesses whether items are comprehensive and adequately reflect the patient perspective for the population of interest.

Description

Establishing content validity for both new and existing patient-reported outcome (PRO) measures is central to a scientifically sound instrument development process (Brod, Tesler, & Christensen, 2009 ). Content validity, derived during concept elicitation, is the measurement property that assesses whether items are comprehensive and adequately reflect the patient perspective for the population of interest. Content validation provides evidence that the conceptual framework, content of items, and overall measurement approach are consistent with the perspective, experience, and words of the patient group and is...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Beatty, P., & Willis, G. (2007). Research synthesis: the practice of cognitive interviewing. Public Opinion Quarterly, 71 (2), 287–311.

Google Scholar  

Brod, M., Tesler, L., & Christensen, T. (2009). Qualitative research and content validity: developing best practices based on science and experience. Quality of Life Research, 18 (9), 1263–1278.

Charmaz, K. (2003). Qualitative interviewing and grounded theory analysis. In J. Holstein & J. Gubrium (Eds.), Inside interviewing: new lenses, new concerns (pp. 311–330). Thousand Oaks, CA: SAGE.

Corbin, J., & Strauss, A. (1990). Grounded theory research: procedures, canons and evaluative criteria. Qualitative Sociology, 13 (1), 3–21.

Corbin, J., & Strauss, A. (2008). Basics of Qualitative Research (3rd ed.). Newbury Park, CA: SAGE Publications.

Cutcliffe, J. (2000). Methodological issues in grounded theory. Journal of Advanced Nursing, 31 (6), 1476–1484.

FDA. (2009). Patient-reported outcome measures: use in medical product development to support labeling claims . Rockville, MD: Food and Drug Administration. http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf .

Frost, M., Reeve, B., Liepa, A., Stauffer, J., & Hays, R. (2007). What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value in Health, 10 (2), S94–S105.

Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough?: an experiment with data saturation and variability. Field Methods, 18 (1), 59–82.

Lasch, K., Marquis, P., Vigneux, M., Abetz, L., Arnould, B., Bayliss, M., et al. (2010). PRO development: rigorous qualitative research as the crucial foundation. Quality of Life Research, 19 (8), 1087–1096.

Leidy, N., & Vernon, M. (2008). Perspectives on patient-reported outcomes: content validity and qualitative research in a changing clinical trial environment. PharmacoEconomics, 26 (5), 363–370.

Magasi, S., Ryan, G., Revicki, D., Lenderking, W., Hays, R., Brod, M., et al. (2012). Content validity of patient-reported outcome measures: perspectives from a PROMIS meeting. Quality of Life Research, 21 (5), 739–46.

Morgan, D. (1996). Focus groups. Annual Review of Sociology, 22 , 129–152.

Patrick, D., Burke, L., Gwaltney, C., Leidy, N., Martin, M., Molsen, E., et al. (2011a). Content validity - establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1 - eliciting concepts for a new PRO instrument. Value in Health, 14 (8), 967–977.

Patrick, D., Burke, L., Gwaltney, C., Leidy, N., Martin, M., Molsen, E., et al. (2011b). Content validity - establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 2 - assessing respondent understanding. Value in Health, 14 (8), 978–988.

Patrick, D., Burke, L., Powers, J., Scott, J., Rock, E., Dawisha, S., O’Neill, R., et al. (2007). Patient-reported outcomes to support medical product labeling claims: FDA perspective. Value in Health, 10 (2), S125–137.

Patton, M. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: SAGE Publications.

Poland, B. (2003). Transcription quality. In J. Holstein & J. Gubrium (Eds.), Inside interviewing: new lenses, new concerns . Thousand Oaks, CA: SAGE Publications.

Ritchie, J., Spencer, L., & O’Connor, W. (2003). Carrying out qualitative analysis. In J. Ritchie & J. Lewis (Eds.), Qualitative research practice: a guide for social science students and researchers (pp. 219–262). London: SAGE Publications.

Rothman, M., Beltran, P., Cappelleri, J., Lipscomb, J., & Teschendorf, B. (2007). Patient-reported outcomes: conceptual issues. Value in Health, 10 (2), S66–75.

Stewart, D., Shamdasani, P., & Rook, D. (2007). Focus groups (2nd ed.). Thousand Oaks, CA: SAGE Publications.

Turner, R., Quittner, A., Parasuraman, B., Kallich, J., & Cleeland, C. (2007). Patient-reported outcomes: instrument development and selection issues. Value in Health, 10 (2), S86–93.

Willis, G. (1999). Cognitive interviewing: A “how to” guide . Research Triangle Park, NC: Research Triangle Institute. http://appliedresearch.cancer.gov/areas/cognitive/interview.pdf .

Willis, G. (2004a). Cognitive interviewing revisited: a useful technique, in theory? In S. Presser, J. Rothgeb, M. Couper, J. Lessler, E. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 23–44). Hoboken, NJ: John Wiley & Sons.

Willis, G. (2005). Cognitive interviewing: a tool for improving questionnaire design . Thousand Oaks, CA: SAGE Publications.

Download references

Author information

Authors and affiliations.

The Brod Group, Inc, 219 Julia Ave, Mill Valley, CA, 94941, USA

Meryl Brod & Betsy Pohlman

Dana-Farber Cancer Institute, 450 Brookline Avenue, D1022, Boston, MA, 02215, USA

Laura Tesler Waldman

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Meryl Brod .

Editor information

Editors and affiliations.

University of Northern British Columbia, Prince George, BC, Canada

Alex C. Michalos

(residence), Brandon, MB, Canada

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media Dordrecht

About this entry

Cite this entry.

Brod, M., Pohlman, B., Tesler Waldman, L. (2014). Qualitative Research and Content Validity. In: Michalos, A.C. (eds) Encyclopedia of Quality of Life and Well-Being Research. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-0753-5_3848

Download citation

DOI : https://doi.org/10.1007/978-94-007-0753-5_3848

Publisher Name : Springer, Dordrecht

Print ISBN : 978-94-007-0752-8

Online ISBN : 978-94-007-0753-5

eBook Packages : Humanities, Social Sciences and Law

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Validity in Qualitative Research

How do we assess and assure Validity in Qualitative Research ?  This can be a bit of a tricky topic, as qualitative research involves humans understanding humans, a necessarily subjective practice from the get-go.  Nevertheless, there are some questions the researcher can ask and some techniques he or she can employ to establish a reasonable level of validity.

Whether it is employed in business or the social sciences, it is often used to inform decisions that have important implications, thus assuring a high level of validity is essential.  While the results should never be extrapolated over a larger population, (as they never come from a large enough sample to be statistically significant), validity can be established such that it can be used to inform meaningful decisions.

One measure of validity in qualitative research is to ask questions such as: “Does it make sense?” and “Can I trust it?”  This may seem like a fuzzy measure of validity to someone disciplined in quantitative research, for example, but in a science that deals in themes and context, these questions are important.

Steps in Ensuring Validity

The first step in ensuring validity is choosing a well-trained and skilled moderator (or facilitator).  A good moderator will check personal bias and expectations at the door.  He or she is interested in learning as much candid information from the research participants as possible, and respectful neutrality is a must if the goal is valid qualitative research.  For this reason, organizations often employ moderators from outside the group or organization to help ensure that the responses are genuine and not influenced by “what we want to hear.”  For some academic applications, the moderator will disclose his or her perspectives and biases in the reporting of the data as a matter of full disclosure.

While a good moderator is key, a good sample group is also essential.  Are the participants truly members of the segment from which they are recruited?  Ethical recruiting is an important issue in qualitative research, as data collected from individuals who are not truly representative of their segment will not lead to valid results.

Another way to promote validity is to employ a strategy known as triangulation.  To accomplish this, the research is done from multiple perspectives.   This could take the form of using several moderators, different locations, multiple individuals analyzing the same data . . . essentially any technique that would inform the results from different angles.   For some applications, for example, an organization may choose to run focus groups in parallel through two entirely different researchers and then compare the results.

Validity in qualitative research can also be checked by a technique known as respondent validation.  This technique involves testing initial results with participants to see if they still ring true.   Although the research has been interpreted and condensed, participants should still recognize the results as authentic and, at this stage, may even be able to refine the researcher’s understanding.

When the study permits, deep saturation into the research will also promote validity.  If responses become more consistent across larger numbers of samples, the data becomes more reliable.

Another technique to establish validity is to actively seek alternative explanations to what appear to be research results.  If the researcher is able to exclude other scenarios, he is or she is able to strengthen the validity of the findings.   Related to this technique is asking questions in an inverse format.

While the techniques to establish validity in qualitative research may seem less concrete and defined than in some of the other scientific disciplines, strong research techniques will, indeed, assure an appropriate level of validity in qualitative research.

Additional Webpages Related to Validity in Qualitative Research

• Intellectus Qualitative : The ultimate platform designed to redefine your qualitative research experience.
• Conducting Qualitative Research
• Content Analysis
• Ethnography
• The Focus Group

• Subscribe to journal Subscribe
• Get new issue alerts Get alerts

Secondary Logo

Journal logo.

Colleague's E-mail is Invalid

Your message has been successfully sent to your colleague.

Save my selection

Rigor or Reliability and Validity in Qualitative Research: Perspectives, Strategies, Reconceptualization, and Recommendations

Cypress, Brigitte S. EdD, RN, CCRN

Brigitte S. Cypress, EdD, RN, CCRN , is an assistant professor of nursing, Lehman College and The Graduate Center, City University of New York.

The author has disclosed that she has no significant relationships with, or financial interest in, any commercial companies pertaining to this article.

Address correspondence and reprint requests to: Brigitte S. Cypress, EdD, RN, CCRN, Lehman College and The Graduate Center, City University of New York, PO Box 2205, Pocono Summit, PA 18346 ( [email protected] ).

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site ( www.dccnjournal.com ).

Issues are still raised even now in the 21st century by the persistent concern with achieving rigor in qualitative research. There is also a continuing debate about the analogous terms reliability and validity in naturalistic inquiries as opposed to quantitative investigations. This article presents the concept of rigor in qualitative research using a phenomenological study as an exemplar to further illustrate the process. Elaborating on epistemological and theoretical conceptualizations by Lincoln and Guba, strategies congruent with qualitative perspective for ensuring validity to establish the credibility of the study are described. A synthesis of the historical development of validity criteria evident in the literature during the years is explored. Recommendations are made for use of the term rigor instead of trustworthiness and the reconceptualization and renewed use of the concept of reliability and validity in qualitative research, that strategies for ensuring rigor must be built into the qualitative research process rather than evaluated only after the inquiry, and that qualitative researchers and students alike must be proactive and take responsibility in ensuring the rigor of a research study. The insights garnered here will move novice researchers and doctoral students to a better conceptual grasp of the complexity of reliability and validity and its ramifications for qualitative inquiry.

Conducting a naturalistic inquiry in general is not an easy task. Qualitative studies are more complex in many ways than a traditional investigation. Quantitative research follows a structured, rigid, preset design with the methods all prescribed. In naturalistic inquiries, planning and implementation are simultaneous, and the research design can change or is emergent. Preliminary steps must be accomplished before the design is fully implemented from making initial contact and gaining entry to site, negotiating consent, building and maintaining trust, and identifying participants. The steps of a qualitative inquiry are also repeated multiple times during the process. As the design unfolds, the elements of this design are put into place, and the inquirer has minimal control and should be flexible. There is continuous reassessment and reiteration. Data collection is carried out using multiple techniques, and whatever the source maybe, it is the researcher who is the sole instrument of the study and the primary mode of collecting the information. All the while during these processes, the qualitative inquirer must be concerned with rigor. 1 Appropriate activities must be conducted to ensure that rigor had been attended to in the research process rather than only adhering to set criteria for rigor after the completion of the study. 1-4

Reliability and validity are 2 key aspects of all research. Researchers assert that rigor of qualitative research equates to the concepts reliability and validity and all are necessary components of quality. 5,6 However, the precise definition of quality has created debates among naturalistic inquirers. Other scholars consider different criteria to describe rigor in qualitative research process. 7 The 2 concepts of reliability and validity have been operationalized eloquently in quantitative texts but at the same time were deemed not pertinent to qualitative inquiries in the 1990s. Meticulous attention to the reliability and validity of research studies is particularly vital in qualitative work, where the researcher's subjectivity can so readily cloud the interpretation of the data and where research findings are often questioned or viewed with skepticism by the scientific community (Brink, 1993).

This article will discuss the issue of rigor in relation to qualitative research and further illustrate the process using a phenomenological study as an exemplar based on Lincoln and Guba's 1 (1985) techniques. This approach will clarify and define some of these complex concepts. There are numerous articles about trustworthiness in the literature that are too complex, confusing, and full of jargon. Some of these published articles also discuss rigor vis-à-vis reliability and validity in a very complicated way. Rigor will be first defined followed by how “reliability and validity” should be applied to qualitative research methods during the inquiry (constructive) rather than only post hoc evaluation. Strategies to attain reliability and validity will be described including the criteria and techniques for ensuring its attainment in a study. This discussion will critically focus on the misuse or nonuse of the concept of reliability and validity in qualitative inquiries, reestablish its importance, and relate both to the concept of rigor. Reflecting on my own research experience, recommendations for the renewed use of the concept of reliability and validity in qualitative research will be presented.

RIGOR VERSUS TRUSTWORTHINESS

Rigor of qualitative research continues to be challenged even now in the 21st century—from the very idea that qualitative research alone is open to questions, so with the terms rigor and trustworthiness . It is critical to understand rigor in research. Rigor is simply defined as the quality or state of being very exact, careful, or with strict precision 8 or the quality of being thorough and accurate. 9 The term qualitative rigor itself is an oxymoron, considering that qualitative research is a journey of explanation and discovery that does not lend to stiff boundaries. 10

Rigor and truth are always of concern for qualitative research. 11 Rigor has also been used to express attributes related to the qualitative research process. 12,13 Per Morse et al 4 (2002), without rigor, research is worthless, becomes fiction, and loses its use. The authors further defined rigor as the strength of the research design and the appropriateness of the method to answer the questions. It is expected that qualitative studies be conducted with extreme rigor because of the potential of subjectivity that is inherent in this type of research. This is a more difficult task when dealing with narratives and people than numbers and statistics. 14 Davies and Dodd 13 (2002) refer rigor to the reliability and validity of research and that, inherent to the conception, the concept is a quantitative bias. Several researchers argued that reliability and validity pertain to quantitative research, which is unrelated or not pertinent to qualitative inquiry because it is aligned with the positivist view. 15 It is also suggested that a new way of looking at reliability and validity will ensure rigor in qualitative inquiry. 1,16 From Lincoln and Guba's crucial work in the 1980s, reliability and validity were replaced with the concept “trustworthiness.” Lincoln and Guba 1 (1985) were the first to address rigor in their model of trustworthiness of qualitative research. Trustworthiness is used as the central concept in their framework to appraise the rigor of a qualitative study.

Trustworthiness is described in different ways by researchers. Trustworthiness refers to quality, authenticity, and truthfulness of findings of qualitative research. It relates to the degree of trust, or confidence, readers have in results. 14 Yin 17 (1994) describes trustworthiness as a criterion to judge the quality of a research design. Trustworthiness addressed methods that can ensure one has carried out the research process correctly. 18 Manning 19 (1997) considered trustworthiness as parallel to the empiricist concepts of internal and external validity, reliability, and objectivity. Seale 20 (1999) asserted that trustworthiness of a research study is based on the concepts of reliability and validity. Guba 2 (1981), Guba and Lincoln 3 (1982), and Lincoln and Guba 1 (1985) refer to trustworthiness as something that evolved from 4 major concerns that relate to it in which the set of criteria were based on. Trustworthiness is a goal of the study and, at the same time, something to be judged during the study and after the research is conducted. The 4 major traditional criteria are summarized into 4 questions about truth value, applicability, consistency, and neutrality. From these, they proposed 4 analogous terms within the naturalistic paradigm to replace the rationalistic terms: credibility, transferability, dependability, and confirmability. 1 For each of these 4 naturalistic terms are research activities or steps that the inquirer should be engage in to be able to safeguard or satisfy each of the previously mentioned criteria and thus attain trustworthiness (Supplemental Digital Content 1, https://links.lww.com/DCCN/A18 ). Guba and Lincoln 1 (1985) stated:

The criteria aid inquirers in monitoring themselves and in guiding activities in the field, as a way of determining whether or not various stages in the research are meeting standards for quality and rigor. Finally, the same criteria may be used to render ex-post facto judgments on the products of research, including reports, case studies, or proposed publications.

Standards and checklist were developed in the 1990s based on Lincoln and Guba's 1 (1985) established criteria, which were then discarded in favor of principles. 21 These standards and checklists consisted of long list of strategies used by qualitative researchers, which were thought to cause harm because of the confusion on which strategies were appropriate for certain designs or what type of naturalistic inquiry is being evaluated. Thus, researchers interpreted missing data as faults and flaws. 21 Morse 21 (2012) further claimed that these standards became the qualitative researchers' “worst enemies” and such an approach was not appropriate. Guba and Lincoln 18 (1989) later proposed a set of guidelines for post hoc evaluation of a naturalistic inquiry to ensure trustworthiness based on the framework of naturalism and constructivism and beyond the conventional methodological ideas. The aspects of their criteria have been fundamental to development of standards used to evaluate the quality of qualitative inquiry. 4

THE RIGOR DEBATES: TRUSTWORTHINESS OR RELIABILITY AND VALIDITY?

A research endeavor, whether quantitative or qualitative, is always evaluated for its worth and merits by peers, experts, reviewers, and readers. Does this mean that a study is differentiated between “good” and “bad”? What determines a “good” from a “bad” inquiry? For a quantitative study, this would mean determining the reliability and validity, and for qualitative inquiries, this would mean determining rigor and trustworthiness. According to Golafshani 22 (2003), if the issues of reliability, validity, trustworthiness, and rigor are meant to differentiating a “good” from “bad” research, then testing and increasing the reliability, validity, trustworthiness, and rigor will be important to the research in any paradigm. However, do reliability and validity in quantitative research equate totally to rigor and trustworthiness in qualitative research? There are many ways to assess the “goodness” of a naturalistic inquiry. Guba and Lincoln 18 (1989) asked, “‘What standards ought apply?’… goodness criteria like paradigms are rooted in certain assumptions. Thus, it is not appropriate to judge constructivist evaluations by positivistic criteria or standards or vice versa. To each its proper and appropriate set.”

Reliability and validity are analogues and are determined differently than in quantitative inquiry. 21 The nature and purpose of the quantitative and qualitative traditions are also different that it is erroneous to apply the same criteria of worthiness or merit. 23,24 The qualitative researcher should not focus on quantitatively defined indicators of reliability and validity, but that does not mean that rigorous standards are not appropriate for evaluating findings. 11 Evaluation, like democracy, is a process that, to be at its best, depends on the application of enlightened and informed self-interest. 18 Agar 24 (1986), on the other hand, suggested that terms such as reliability and validity are comparative with the quantitative view and do not fit the details of qualitative research. A different language is needed to fit the qualitative view. From Leininger 25 (1985), Krefting 23 (1991) asserted that addressing reliability and validity in qualitative research is such a different process that quantitative labels should not be used. The incorrect application of the qualitative criteria of rigor to studies is as problematic as the application of inappropriate quantitative criteria. 23 Smith 26 (1989) argued that, for qualitative research, this means that the basis of truth or trustworthiness becomes a social agreement. He emphasizes that what is judged true or trustworthy is what we can agree, conditioned by time and place, and is true or trustworthy. Validity standards in qualitative research are also even more challenging because of the necessity to incorporate rigor and subjectivity, as well as creativity into the scientific process. 27 Furthermore, Leininger 25 (1985) claimed that it is not whether the data are reliable or valid but how the terms reliability and validity are defined. Aside from the debate whether reliability and validity criteria should be used similarly in qualitative inquiries, there is also an issue of not using the concepts at all in naturalistic studies.

Designing a naturalistic inquiry is very different from a traditional quantitative notion of design and that defining a “good” qualitative inquiry is controversial and has gone through many changes. 21 First is the confusion on the use of terminologies “rigor” and “trustworthiness.” Morse 28 (2015) suggested that it is time to return to the terminology of mainstream social science and to use “rigor” rather than “trustworthiness.” Debates also continue about why some qualitative researchers do not use the concept of reliability and validity in their studies referring to Lincoln and Guba's 1 (1985) criteria for trustworthiness, namely, transferability, dependability, confirmability, and credibility. Morse 28 (2015) further suggested replacing these criteria to reliability, validity, and generalizability. The importance and centrality of reliability and validity to qualitative inquiries have in some way been disregarded even in the current times. Researchers from the United Kingdom and Europe continue to do so but not much so in North America. 4 According to Morse 21 (2012), this gives the impression that these concepts are of no concern to qualitative research. Morse 29 (1999) stated, “Is the terminology worth making a fuzz about?”, when Lincoln and Guba 1 (1985) described trustworthiness and reliability and validity as analogs. Morse 29 (1999) further articulated that:

To state that reliability and validity are not pertinent to qualitative inquiry places qualitative research in the realm of being not reliable and not valid. Science is concerned with rigor, and by definition, good rigorous research must be reliable and valid. If qualitative research is unreliable and invalid, then it must not be science. If it is not science, then why should it be funded, published, implemented, or taken seriously?

RELIABILITY AND VALIDITY IN QUALITATIVE RESEARCH

Reliability and validity should be taken into consideration by qualitative inquirers while designing a study, analyzing results, and judging the quality of the study, 30 but for too long, the criteria used for evaluating rigor are applied after a research is completed—a considerably wrong tactic. 4 Morse and colleagues 4 (2002) argued that, for reliability and validity to be actively attained, strategies for ensuring rigor must be built into the qualitative research process per se not to be proclaimed only at the end of the inquiry. The authors suggest that focusing on strategies to establish rigor at the completion of the study (post hoc), rather than during the inquiry, exposes the investigators to the risk of missing and addressing serious threats to the reliability and validity until it is too late to correct them. They further asserted that the interface between reliability and validity is important especially for the direction of the analysis process and the development of the study itself.

Reliability

In the social sciences, the whole notion of reliability in and of itself is problematic. 31 The scientific aspect of reliability assumes that repeated measures of a phenomenon (with the same results) using objective methods establish the truth of the findings. 32-35 Merriam 36 (1995) stated that, “The more times the findings of a study can be replicated, the more stable or reliable the phenomenon is thought to be.” In other words, it is the idea of replicability, 22,34,37 repeatability, 21,22,26,30,31,36,38-40 and stability of results or observation. 25,39,41 The issues are that human behaviors and interactions are never static or the same. Measurements and observations can also be repeatedly wrong. Furthermore, researchers have argued that the concept reliability is misleading and has no relevance in qualitative research related to the notion of “measurement method,” as in quantitative studies. 40,42 It is a fact that quantitative research is supported by the positivist or scientific paradigm that regards the world as made up of observable, measurable facts. Qualitative research, on the other hand, produces findings not arrived at by means of statistical procedures or other means of quantification. On the basis of the constructivist paradigm, it is a naturalistic inquiry that seeks to understand phenomena in context-specific settings in which the researcher does not attempt to manipulate the phenomenon of interest. 23 If reliability is used as a criterion in qualitative research, it would mean that the study is “not good.” A thorough description of the entire research process that allows for intersubjectivity is what indicates good quality when using qualitative methodology. Reliability is based on consistency and care in the application of research practices, which are reflected in the visibility of research practices, analysis, and conclusions, reflected in an open account that remains mindful of the partiality and limits of the research findings. 13 Reliability and similar terms are presented in Supplemental Digital Content 2 (see Supplemental Digital Content 2, https://links.lww.com/DCCN/A19 ).

Validity is broadly defined as the state of being well grounded or justifiable, relevant, meaningful, logical, confirming to accepted principles or the quality of being sound, just, and well founded. 8 The issues surrounding the use and nature of the term validity in qualitative research are controversial and many. It is a highly debated topic both in social and educational research and is still often a subject of debate. 43 The traditional criteria for validity find their roots in a positivist tradition, and to an extent, positivism has been defined by a systematic theory of validity. 22 Validity is rooted from empirical conceptions as universal laws, evidence, objectivity, truth, actuality, deduction, reason, fact, and mathematical data, to name only a few. Validity in research is concerned with the accuracy and truthfulness of scientific findings. 44 A valid study should demonstrate what actually exists and is accurate, and a valid instrument or measure should actually measure what it is supposed to measure. 5,22,29,31,42,45

Novice researchers can become easily perplexed in attempting to understand the notion of validity in qualitative inquiry. 44 There is a multiple array of terms similar to validity in the literature, which the authors equate to same such as authenticity, goodness, adequacy, trustworthiness, verisimilitude, credibility, and plausibility. 1,45-51 Validity is not a single, fixed, or universal concept but rather a contingent construct, inescapably grounded in the processes and intentions of particular research methodologies. 39 Some qualitative researchers have argued that the term validity is not applicable to qualitative research and have related it to terms such as quality, rigor, and trustworthiness. 1,13,22,38,42,52-54 I argue that the concepts of reliability and validity are overarching constructs that can be appropriately used in both quantitative and qualitative methodologies. To validate means to investigate, to question, and to theorize, which are all activities to ensure rigor in a qualitative inquiry. For Leininger 25 (1985), the term validity in a qualitative sense means gaining knowledge and understanding of the nature (ie, the meaning, attributes, and characteristics) of the phenomenon under study. A qualitative method seeks for a certain quality that is typical for a phenomenon or that makes the phenomenon different than others.

Some naturalistic inquirers agree that assuring validity is a process whereby ideals are sought through attention to specified criteria, and appropriate techniques are used to address any threats to validity of a naturalistic inquiry. However, other researchers argue that procedures and techniques are not an assurance of validity and will not necessarily produce sound data or credible conclusions. 38,48,55 Thus, some argued that they should abandon the concept of validity and seek alternative criteria with which to judge their work. Criteria are the standards or rules to be upheld as ideals in qualitative research on which a judgment or decisions may be based, 4,56 whereas the techniques are the methods used to diminish identified validity threats. 56 Criteria, for some researchers, are used to test the quality of the research design, whereas for some, they are the goal of the study. There is also the trend to treat standards, goals, and criteria synonymously. I concur with Morse 29 (1999) that introducing parallel terminology and criteria diminishes qualitative inquiry from mainstream science and scientific legitimacy. The development of alternative criteria compromises the issue of rigor. We must work to have a consensus toward criteria and terminology that are used in mainstream science and how it is attained within the qualitative inquiry during the research process rather than at the end of the study. Despite all these, researchers developed validity criteria and techniques during the years. A synthesis of validity criteria development is summarized in Supplemental Digital Content 3 (see Supplemental Digital Content 3, https://links.lww.com/DCCN/A20 ). The techniques for demonstrating validity are presented in Supplemental Digital Content 4 (see Supplemental Digital Content 4, https://links.lww.com/DCCN/A21 ).

Reliability and Validity as Means in Ensuring the Quality of Findings of a Phenomenological Study in Intensive Care Unit

Reliability and validity are 2 factors that any qualitative researcher should be concerned about while designing a study, analyzing results, and judging its quality. Just as the quantitative investigator must attend to the question of how external and internal validity, reliability, and objectivity will be provided for in the design, so must the naturalistic inquirer arrange for credibility, transferability, dependability, and confirmability. 1 Lincoln and Guba 1 (1985) clearly established these 4 criteria as benchmarks for quality based on the identification of 4 aspects of trustworthiness that are relevant to both quantitative and qualitative studies, which are truth value, applicability, consistency, and neutrality. Guba 2 (1981) stated, “It is to these concerns that the criteria must speak.”

Rigor of a naturalistic inquiry such as phenomenology may be operationalized using the criteria of credibility, transferability, dependability, and confirmability. This phenomenological study aimed to understand and illuminate the meaning of the phenomenon of the lived experiences of patients, their family members, and the nurses during critical illness in the intensive care unit (ICU). From Lincoln and Guba 1 (1985), I first asked the question, “How can I persuade my audience that the research findings of my inquiry are worth paying attention to, and worth taking account of?” My answers to these questions were based on the identified 4 criteria set forth by Lincoln and Guba 1 (1985).

Credibility, the accurate and truthful depiction of a participant's lived experience, was achieved in this study through prolonged engagement and persistent observation to learn the context of the phenomenon in which it is embedded and to minimize distortions that might creep into the data. To achieve this, I spent 6 months with nurses, patients, and their families in the ICU to become oriented to the situation and also to build trust and rapport with the participants. Peer debriefing was conducted through meetings and discussions with an expert qualitative researcher to allow for questions and critique of field journals and research activities. Triangulation was achieved by cross-checking the data and interpretations within and across each category of participants by 2 qualitative researchers. Member checks were accomplished by constantly checking data and interpretations with the participants from which data were solicited.

Transferability was enhanced by using purposive sampling method and providing a thick description and a robust data with a wide possible range of information through the detailed and accurate descriptions of the patients, their family members, and the nurses' lived ICU experiences and by continuously returning to the texts. In this study, recruitment of participants and data collection continued until the data are saturated and complete and replicate. According to Morse et al 4 (2002), interviewing additional participants is for the purpose of increasing the scope, adequacy, and appropriateness of the data. I immersed myself into the phenomenon to know, describe, and understand it fully, comprehensively, and thoroughly. Special care was given to the collection, identification, and analysis of all data pertinent to the study. The audiotaped data were meticulously transcribed by a professional transcriber for future scrutiny. During the analysis phase, every attempt was made to document all aspects of the analysis. Analysis in qualitative research refers to the categorization and ordering of information in such a way as to make sense of the data and to writing a final report that is true and accurate. 36 Every effort was made to coordinate methodological and analytical materials. After I categorized and was able to make sense of the transcribed data, all efforts were exhausted to illuminate themes and descriptors as they emerge.

Lincoln and Guba 1 (1985) use “dependability” in qualitative research, which closely corresponds to the notion of “reliability” in quantitative research. Dependability was achieved by having 2 expert qualitative nursing researchers review the transcribed material to validate the themes and descriptors identified. To be able to validate my findings related to the themes, a doctoral-prepared nursing colleague was asked to review some of the transcribed materials. Any new themes and descriptors illuminated by my colleague were acknowledged and considered. It was then compared with my own thematic analysis from the entire participant's transcribed data. If the theme identified by the colleague did not appear in my own thematic analysis, it was agreed by both analysts not to use the said theme. It was my goal that both analysts agree on the findings related to themes and meanings within the transcribed material.

Confirmability was met by maintaining a reflexive journal during the research process to keep notes and document introspections daily that would be beneficial and pertinent during the study. An audit trail also took place to examine the processes whereby data were collected and analyzed and interpretations were made. The audit trail took the form of documentation (the actual interview notes taken) and a running account of the process (my daily field journal). I maintained self-awareness of my role as the sole instrument of this study. After each interview, I retired in 1 private room to document additional perceptions and recollections from the interviews (Supplemental Digital Content 5, https://links.lww.com/DCCN/A22 ).

Through reflexivity and bracketing, I was always on guard of my own biases, assumptions, beliefs, and presuppositions that I might bring to the study but was also aware that complete reduction is not possible. Van Manen 44 (1990) stated that “if we simply try to forget or ignore what we already know, we may find that the presuppositions persistently creep back into our reflections.” During data collection and analysis, I made my orientation and preunderstanding of critical illness and critical care explicit but held them deliberately at bay and bracketed them. Aside from Lincoln and Guba's 1 (1985) 4 criteria for trustworthiness, a question arises as to the reliability of the researcher as the sole instrument of the study.

Reliability related to the researcher as the sole instrument who conducted the data collection and analysis is a limitation of any phenomenological study. The use of humans as instruments is not a new concept. Lincoln and Guba 1 (1985) articulated that humans uniquely qualify as the instrument of choice for naturalistic inquiry. Some of the giants of conventional inquiry have recognized that humans can provide data very nearly as reliable as that produced by “more” objective means. These are formidable characteristics, but they are meaningless if the human instrument is not also trustworthy. However, no human instrument is expected to be perfect. Humans have flaws, and errors could be committed. When Lincoln and Guba 1 (1985) asserted that qualitative methods come more easily to hand when the instrument is a human being, they mean that the human as instrument is inclined toward methods that are extensions of normal activities. They believe that the human will tend therefore toward interviewing, observing, mining available documents and records, taking account of nonverbal cues, and interpreting inadvertent unobtrusive measures. All of which are complex tasks. In addition, one would not expect an individual to function adequately as human instruments without an extensive background or training and experience. This study has reliability in that I have acquired knowledge and the required training for research at a doctoral level with the professional and expert guidance of a mentor. As Lincoln and Guba 1 (1985) said, “Performance can be improved…when that learning is guided by an experienced mentor, remarkable improvements in human instrumental performance can be achieved.” Whereas reliability in quantitative research depends on instrument construction, in qualitative research, the researcher is the instrument of the study. 31 A reliable research is a credible research. Credibility of a qualitative research depends on the ability and effort of the researcher. 22 We have established that a study can be reliable without being valid, but a study cannot be valid without being reliable.

Establishing validity is a major challenge when a qualitative research project is based on a single, cross-sectional, unstructured interview as the basis for data collection. How do I make judgments about the validity of the data? In qualitative research, the validity of the findings is related to the careful recording and continual verification of the data that the researcher undertakes during the investigative practice. If the validity or trustworthiness can be maximized or tested, then more credible and defensible result may lead to generalizability as the structure for both doing and documenting high-quality qualitative research. Therefore, the quality of a research is related to generalizability of the result and thereby to the testing and increasing of the validity or trustworthiness of the research.

One potential threat to validity that researchers need to consider is researcher bias. Researcher bias is frequently an issue because qualitative research is open and less structured than quantitative research. This is because qualitative research tends to be exploratory. Researcher bias tends to result from selective observation and selective recording of information and from allowing one's personal views and perspectives to affect how data are interpreted and how the research is conducted. Therefore, it is very important that the researchers are aware of their own perceptions and opinions because they may taint their research findings and conclusions. I brought all past experiences and knowledge into the study but learned to set aside my own strongly held perceptions, preconceptions, and opinions. I truly listened to the participants to learn their stories, experiences, and meanings.

The key strategy used to understand researcher bias is called reflexivity. Reflexivity means that the researchers actively engage in critical self-reflection about their potential biases and predispositions that they bring to the qualitative study. Through reflexivity, researchers become more self-aware and monitor and attempt to control their biases. Phenomenological researchers can recognize that their interpretation is correct because the reflective process awakens an inner moral impulse. 4,59 I did my best to be always on guard of my own biases, preconceptions, and assumptions that I might bring to this study. Bracketing was also applied.

Husserl 60 (1931) has made some key conceptual elaborations, which led him to assert that an attempt to hold a previous belief about the phenomena under study in suspension to perceive it more clearly is needed in phenomenological research. This technique is called bracketing. Bracketing is another strategy used to control bias. Husserl 60 (1931) explained further that phenomenological reduction is the process of defining the pure essence of a psychological phenomenon. Phenomenological reduction is a process whereby empirical subjectivity is suspended, so that pure consciousness may be defined in its essential and absolute “being.” This is accomplished by a method of bracketing empirical data away from consideration. Bracketing empirical data away from further investigation leaves pure consciousness, pure phenomena, and pure ego as the residue of phenomenological reduction. Husserl 60 (1931) uses the term epoche (Greek word for “a cessation”) to refer to this suspension of judgment regarding the true nature of reality. Bracketed judgment is an epoche or suspension of inquiry, which places in brackets whatever facts belong to essential “being.”

Bracketing was conducted to separate the assumptions and biases from the essences and therefore achieve an understanding of the phenomenon as experienced by the participants of the study. The collected and analyzed data were presented to the participants, and they were asked whether the narrative is accurate and a true reflection of their experience. My interpretation and descriptions of the narratives were presented to the participants to achieve credibility. They were given the opportunity to review the transcripts and modify it if they wished to do so. As I was the one who served as the sole instrument in obtaining data for this phenomenological study, my goal was that my perceptions would reflect the participant's ICU experiences and that the participants would be able to see their lived experience through the researcher's eyes. Because qualitative research designs are flexible and emergent in nature, there will always be study limitations.

Awareness of the limitations of a research study is crucial for researchers. The purpose of this study was to understand the ICU experiences of patients, their family members, and the nurses during critical illness. One limitation of this phenomenological study as a naturalistic inquiry was the inability of the researcher to fully design and provide specific ideas needed for the study. According to Lincoln and Guba 1 (1985), naturalistic studies are virtually impossible to design in any definitive way before the study is actually undertaken. The authors stated:

Designing a naturalistic study means something very different from the traditional notion of “design”—which as often as not meant the specification of a statistical design with its attendant field conditions and controls. Most of the requirements normally laid down for a design statement cannot be met by naturalists because the naturalistic inquiry is largely emergent.

Within the naturalistic paradigm, designs must be emergent rather than preordinate because (1) meaning is determined by context to such a great extent. For this particular study, the phenomenon and context were the experience of critical illness in the ICU; (2) the existence of multiple realities constrains the development of a design based on only 1 (the investigator's) construction; (3) what will be learned at a site is always dependent on the interaction between the investigator and the context, and the interaction is also not fully predictable; and (4) the nature of mutual shapings cannot be known until they are witnessed. These factors underscore the indeterminacy under which naturalistic inquirer functions. The design must therefore be “played by ear”; it must unfold, cascade, and emerge. It does not follow, however, that, because not all of the elements of the design can be prespecified in a naturalistic inquiry, none of them can. Design in the naturalistic sense means planning for certain broad contingencies without however indicating exactly what will be conducted on relation to each. 1

Reliability and validity are such fundamental concepts that should be continually operationalized to meet the conditions of a qualitative inquiry. Morse et al 4,29 (2002) articulated that “by refusing to acknowledge the centrality of reliability and validity in qualitative methods, qualitative methodologists have inadvertently fostered the default notion that qualitative research must therefore be unreliable and invalid, lacking in rigor, and unscientific.” Sparkes 59 (2001) asserted that Morse et al 4,26 (2002) is right in warning us that turning our backs on such fundamental concepts as validity could cost us dearly. This will in turn affect how we mentor novices, early career researchers, and doctoral students in their qualitative research works.

Reliability is inherently integrated and internally needed to attain validity. 1,26 I concur with the use of the term rigor rather than trustworthiness in naturalistic studies. I have also discussed that I accede that strategies for ensuring rigor must be built into the qualitative research process per se rather than evaluated only after the inquiry is conducted. Threats to reliability and validity cannot be actively addressed by using standards and criteria applied at the end of the study. Ensuring rigor must be upheld by the researcher during the investigation rather than the external judges of the completed study. Whether a study is quantitative or qualitative, rigor is a desired goal that is met through the inclusion of different philosophical perspectives inherent in a qualitative inquiry and the strategies that are specific to each methodological approach including the verification techniques to be observed during the research process. It also involves the researcher's creativity, sensitivity, flexibility, and skill in using the verification strategies that determine the reliability and validity of the evolving study.

Some naturalistic inquirers agree that assuring validity is a process whereby ideals are sought through attention to specified criteria, and appropriate techniques are used to address any threats to validity of a naturalistic inquiry. However, other researchers argue that procedures and techniques are not an assurance of validity and will not necessarily produce sound data or credible conclusions. 38,48,55 Thus, some argued that they should abandon the concept of validity and seek alternative criteria with which to judge their work.

Lincoln and Guba's 1 (1985) standards of validity demonstrate the necessity and convenience of overarching principles to all qualitative research, yet there is a need for a reconceptualization of criteria of validity in qualitative research. The development of validity criteria in qualitative research poses theoretical issues, not simply technical problems. 60 Whittemore et al 58 (2001) explored the historical development of validity criteria in qualitative research and synthesized the findings that reflect a contemporary reconceptualization of the debate and dialogue that have ensued in the literature during the years. The authors further presented primary (credibility, authenticity, criticality, and integrity) and secondary (explicitness, vividness, creativity, thoroughness, congruence, and sensitivity) validity criteria to be used in the evaluative process. 56 Before the work of Whittemore and colleagues, 58 Creswell and Miller 48 (2000) asserted that the constructivist lens and paradigm choice should guide validity evaluation and procedures from the perspective of the researcher (disconfirming evidence), the study participants (prolonged engagement in the field), and external reviewers/readers (thick, rich description). Morse et al 4 in 2002 presented 6 major evaluation criteria for validity and asserted that they are congruent and are appropriate within the philosophy of the qualitative tradition. These 6 criteria are credibility, confirmability, meaning in context, recurrent patterning, saturation, and transferability. Synthesis of validity criteria is presented in Supplemental Digital Content 3 (see Supplemental Digital Content 3, https://links.lww.com/DCCN/A20 ).

Common validity techniques in qualitative research refer to design consideration, data generation, analytic procedures, and presentation. 56 First is the design consideration. Developing a self-conscious design, the paradigm assumption, the purposeful choice of small sample of informants relevant to the study, and the use of inductive approach are some techniques to be considered. Purposive sampling enhances the transferability of the results. Interpretivist and constructivist inquiry follows an inductive approach that is flexible and emergent in design with some uncertainty and fluidity within the context of the phenomenon of interest 56,58 and not based on a set of determinate rules. 61 The researcher does not work with a priori theory; rather, these are expected to emerge from the inquiry. Data are analyzed inductively from specific, raw units of information to subsuming categories to define questions that can be followed up. 1 Qualitative studies also follow a naturalistic and constructivist paradigm. Creswell and Miller 48 (2000) suggest that the validity is affected by the researchers' perception of validity in the study and their choice of paradigm assumption. Determining fit of paradigm to focus is an essential aspect of a naturalistic inquiry. 1 Paradigms rest on sets of beliefs called axioms. 1 On the basis of the naturalistic axioms, the researcher should ask questions related to multiplicity or complex constructions of the phenomenon, the degree of investigator-phenomenon interaction and the indeterminacy it will introduce into the study, the degree of context dependence, whether values are likely to be crucial to the outcome, and the constraints that may be placed on the researcher by a variety of significant others. 1

Validity during data generation is evaluated through the researcher's ability to articulate data collection decisions, demonstrate prolonged engagement and persistent observation, provide verbatim transcription, and achieve data saturation. 56 Methods are means to collect evidence to support validity, and this refers to the data obtained by considering a context for a purpose. The human instrument operating in an indeterminate situation falls back on techniques such as interview, observation, unobtrusive measures, document and record analysis, and nonverbal cues. 1 Others remarked that rejecting methods or technical procedures as assurance of truth, thus validity of a qualitative study, lies in the skills and sensitivities of the researchers and how they use themselves as a knower and an inquirer. 57,62 The understanding of the phenomenon is valid if the participants are given the opportunity to speak freely according to their own knowledge structures and perceptions. Validity is therefore achieved when using the method of open-ended, unstructured interviews with strategically chosen participants. 42 We also know that a thorough description of the entire research process enabling unconditional intersubjectivity is what indicates good quality when using a qualitative method. This enhances a clearer and better analysis of data.

Analytical procedures are vital in qualitative research. 56 Not very much can be said about data analysis in advance of a qualitative study. 1 Data analysis is not an inclusive phase that can be marked out as occurring at some singular time during the inquiry. 1 It begins from the very first data collection to facilitate the emergent design and grounding of theory. Validity in a study thus is represented by truthfulness of findings after a careful analysis. 56 Consequently, qualitative researchers seek to illuminate and extrapolate findings to similar situations. 22,63 It is a fact that the interpretations of any given social phenomenon may reflect, in part, the biases and prejudices the interpreters bring to the task and the criteria and logic they follow in completing it. 64 In any case, individuals will draw different conclusions to the debate surrounding validity and will make different judgments as a result. 50 There is a wide array of analytic techniques that the qualitative researcher can choose from based on the contextual factors that will help contribute to the decision as to which technique will optimally reflect specific criteria of validity. 65 Presentation of findings is accomplished by providing an audit trail and evidence that support interpretations, acknowledging the researcher's perspective and providing thick descriptions. Morse et al 4 in 2002 set forth strategies for ensuring validity that include investigator responsiveness and verification through methodological coherence, theoretical sampling and sampling adequacy, an active analytic stance, and saturation. The authors further stated that “these strategies, when used appropriately, force the researcher to correct both the direction of the analysis and the development of the study as necessary, thus ensuring reliability and validity of the completed project (p17). Recently in 2015, Morse 28 presented that the strategies for ensuring validity in a qualitative study are prolonged engagement, persistent observation, thick and rich description, negative case analysis, peer review or debriefing, clarifying researcher's bias, member checking, external audits, and triangulation. These strategies can be upheld with the help of an expert mentor who can in turn guide and affect the reliability and validity of early career researchers and doctoral students' qualitative research works. Techniques for demonstrating validity are summarized in Supplemental Digital Content 4 (see Supplemental Digital Content 4, https://links.lww.com/DCCN/A21 ).

Qualitative researchers and students alike must be proactive and take responsibility in ensuring the rigor of a research study. A lot of times, rigor is at the backseat in some researchers and doctoral students' work related to their novice abilities, lack of proper mentorship, and issues with time and funding. Students should conduct projects that are smaller in scope guided by an expert naturalistic inquirer to come up with the product with depth and, at the same time, gain the grounding experience necessary to become an excellent researcher. Attending to rigor throughout the research process will have important ramifications for qualitative inquiry. 4,26

Qualitative research is not intended to be scary or beyond the grasp of novices and doctoral students. Conducting a naturalistic inquiry is an experience of exploration, discovery, description, and understanding of a phenomenon that transcends one's own research journey. Attending to the rigor of qualitative research is a vital part of the investigative process that offers critique and thus further development of the science.

• Cited Here |
• Google Scholar

Phenomenology; Qualitative research; Reliability; Rigor; Validity

Supplemental Digital Content

• DCCN_2017_04_11_CYPRESS_DCCN-D-16-00060_SDC1.pdf; [PDF] (3 KB)
• DCCN_2017_04_11_CYPRESS_DCCN-D-16-00060_SDC2.pdf; [PDF] (4 KB)
• DCCN_2017_04_11_CYPRESS_DCCN-D-16-00060_SDC3.pdf; [PDF] (78 KB)
• DCCN_2017_04_11_CYPRESS_DCCN-D-16-00060_SDC4.pdf; [PDF] (70 KB)
• DCCN_2017_04_11_CYPRESS_DCCN-D-16-00060_SDC5.pdf; [PDF] (4 KB)
• + Favorites
• View in Gallery

Readers Of this Article Also Read

Increasing access to palliative care services in the intensive care unit, exploring the philosophical, paradigmatic, conceptual-theoretical underpinnings ..., educational interventions to improve support for family presence during..., nursing practices and policies related to family presence during resuscitation.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is Qualitative Research? | Methods & Examples

What Is Qualitative Research? | Methods & Examples

Published on June 19, 2020 by Pritha Bhandari . Revised on June 22, 2023.

Qualitative research involves collecting and analyzing non-numerical data (e.g., text, video, or audio) to understand concepts, opinions, or experiences. It can be used to gather in-depth insights into a problem or generate new ideas for research.

Qualitative research is the opposite of quantitative research , which involves collecting and analyzing numerical data for statistical analysis.

Qualitative research is commonly used in the humanities and social sciences, in subjects such as anthropology, sociology, education, health sciences, history, etc.

• How does social media shape body image in teenagers?
• How do children and adults interpret healthy eating in the UK?
• What factors influence employee retention in a large organization?
• How is anxiety experienced around the world?
• How can teachers integrate social issues into science curriculums?

Table of contents

Approaches to qualitative research, qualitative research methods, qualitative data analysis, advantages of qualitative research, disadvantages of qualitative research, other interesting articles, frequently asked questions about qualitative research.

Qualitative research is used to understand how people experience the world. While there are many approaches to qualitative research, they tend to be flexible and focus on retaining rich meaning when interpreting data.

Common approaches include grounded theory, ethnography , action research , phenomenological research, and narrative research. They share some similarities, but emphasize different aims and perspectives.

Note that qualitative research is at risk for certain research biases including the Hawthorne effect , observer bias , recall bias , and social desirability bias . While not always totally avoidable, awareness of potential biases as you collect and analyze your data can prevent them from impacting your work too much.

Prevent plagiarism. Run a free check.

Each of the research approaches involve using one or more data collection methods . These are some of the most common qualitative methods:

• Observations: recording what you have seen, heard, or encountered in detailed field notes.
• Interviews:  personally asking people questions in one-on-one conversations.
• Focus groups: asking questions and generating discussion among a group of people.
• Surveys : distributing questionnaires with open-ended questions.
• Secondary research: collecting existing data in the form of texts, images, audio or video recordings, etc.
• You take field notes with observations and reflect on your own experiences of the company culture.
• You distribute open-ended surveys to employees across all the company’s offices by email to find out if the culture varies across locations.
• You conduct in-depth interviews with employees in your office to learn about their experiences and perspectives in greater detail.

Qualitative researchers often consider themselves “instruments” in research because all observations, interpretations and analyses are filtered through their own personal lens.

For this reason, when writing up your methodology for qualitative research, it’s important to reflect on your approach and to thoroughly explain the choices you made in collecting and analyzing the data.

Qualitative data can take the form of texts, photos, videos and audio. For example, you might be working with interview transcripts, survey responses, fieldnotes, or recordings from natural settings.

Most types of qualitative data analysis share the same five steps:

• Prepare and organize your data. This may mean transcribing interviews or typing up fieldnotes.
• Review and explore your data. Examine the data for patterns or repeated ideas that emerge.
• Develop a data coding system. Based on your initial ideas, establish a set of codes that you can apply to categorize your data.
• Assign codes to the data. For example, in qualitative survey analysis, this may mean going through each participant’s responses and tagging them with codes in a spreadsheet. As you go through your data, you can create new codes to add to your system if necessary.
• Identify recurring themes. Link codes together into cohesive, overarching themes.

There are several specific approaches to analyzing qualitative data. Although these methods share similar processes, they emphasize different concepts.

Qualitative research often tries to preserve the voice and perspective of participants and can be adjusted as new research questions arise. Qualitative research is good for:

• Flexibility

The data collection and analysis process can be adapted as new ideas or patterns emerge. They are not rigidly decided beforehand.

• Natural settings

Data collection occurs in real-world contexts or in naturalistic ways.

• Meaningful insights

Detailed descriptions of people’s experiences, feelings and perceptions can be used in designing, testing or improving systems or products.

• Generation of new ideas

Open-ended responses mean that researchers can uncover novel problems or opportunities that they wouldn’t have thought of otherwise.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Researchers must consider practical and theoretical limitations in analyzing and interpreting their data. Qualitative research suffers from:

• Unreliability

The real-world setting often makes qualitative research unreliable because of uncontrolled factors that affect the data.

• Subjectivity

Due to the researcher’s primary role in analyzing and interpreting data, qualitative research cannot be replicated . The researcher decides what is important and what is irrelevant in data analysis, so interpretations of the same data can vary greatly.

• Limited generalizability

Small samples are often used to gather detailed data about specific contexts. Despite rigorous analysis procedures, it is difficult to draw generalizable conclusions because the data may be biased and unrepresentative of the wider population .

• Labor-intensive

Although software can be used to manage and record large amounts of text, data analysis often has to be checked or performed manually.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Chi square goodness of fit test
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Inclusion and exclusion criteria

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

There are five common approaches to qualitative research :

• Grounded theory involves collecting data in order to develop new theories.
• Ethnography involves immersing yourself in a group or organization to understand its culture.
• Narrative research involves interpreting stories to understand how people make sense of their experiences and perceptions.
• Phenomenological research involves investigating phenomena through people’s lived experiences.
• Action research links theory and practice in several cycles to drive innovative changes.

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

There are various approaches to qualitative data analysis , but they all share five steps in common:

• Prepare and organize your data.
• Review and explore your data.
• Develop a data coding system.
• Assign codes to the data.
• Identify recurring themes.

The specifics of each step depend on the focus of the analysis. Some common approaches include textual analysis , thematic analysis , and discourse analysis .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 22). What Is Qualitative Research? | Methods & Examples. Scribbr. Retrieved June 9, 2024, from https://www.scribbr.com/methodology/qualitative-research/

Is this article helpful?

Pritha Bhandari

Other students also liked, qualitative vs. quantitative research | differences, examples & methods, how to do thematic analysis | step-by-step guide & examples, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Qualitative vs Quantitative Research Methods & Data Analysis

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

What is the difference between quantitative and qualitative?

The main difference between quantitative and qualitative research is the type of data they collect and analyze.

Quantitative research collects numerical data and analyzes it using statistical methods. The aim is to produce objective, empirical data that can be measured and expressed in numerical terms. Quantitative research is often used to test hypotheses, identify patterns, and make predictions.

Qualitative research , on the other hand, collects non-numerical data such as words, images, and sounds. The focus is on exploring subjective experiences, opinions, and attitudes, often through observation and interviews.

Qualitative research aims to produce rich and detailed descriptions of the phenomenon being studied, and to uncover new insights and meanings.

Quantitative data is information about quantities, and therefore numbers, and qualitative data is descriptive, and regards phenomenon which can be observed but not measured, such as language.

What Is Qualitative Research?

Qualitative research is the process of collecting, analyzing, and interpreting non-numerical data, such as language. Qualitative research can be used to understand how an individual subjectively perceives and gives meaning to their social reality.

Qualitative data is non-numerical data, such as text, video, photographs, or audio recordings. This type of data can be collected using diary accounts or in-depth interviews and analyzed using grounded theory or thematic analysis.

Qualitative research is multimethod in focus, involving an interpretive, naturalistic approach to its subject matter. This means that qualitative researchers study things in their natural settings, attempting to make sense of, or interpret, phenomena in terms of the meanings people bring to them. Denzin and Lincoln (1994, p. 2)

Interest in qualitative data came about as the result of the dissatisfaction of some psychologists (e.g., Carl Rogers) with the scientific study of psychologists such as behaviorists (e.g., Skinner ).

Since psychologists study people, the traditional approach to science is not seen as an appropriate way of carrying out research since it fails to capture the totality of human experience and the essence of being human.  Exploring participants’ experiences is known as a phenomenological approach (re: Humanism ).

Qualitative research is primarily concerned with meaning, subjectivity, and lived experience. The goal is to understand the quality and texture of people’s experiences, how they make sense of them, and the implications for their lives.

Qualitative research aims to understand the social reality of individuals, groups, and cultures as nearly as possible as participants feel or live it. Thus, people and groups are studied in their natural setting.

Some examples of qualitative research questions are provided, such as what an experience feels like, how people talk about something, how they make sense of an experience, and how events unfold for people.

Research following a qualitative approach is exploratory and seeks to explain ‘how’ and ‘why’ a particular phenomenon, or behavior, operates as it does in a particular context. It can be used to generate hypotheses and theories from the data.

Qualitative Methods

There are different types of qualitative research methods, including diary accounts, in-depth interviews , documents, focus groups , case study research , and ethnography.

The results of qualitative methods provide a deep understanding of how people perceive their social realities and in consequence, how they act within the social world.

The researcher has several methods for collecting empirical materials, ranging from the interview to direct observation, to the analysis of artifacts, documents, and cultural records, to the use of visual materials or personal experience. Denzin and Lincoln (1994, p. 14)

Here are some examples of qualitative data:

Interview transcripts : Verbatim records of what participants said during an interview or focus group. They allow researchers to identify common themes and patterns, and draw conclusions based on the data. Interview transcripts can also be useful in providing direct quotes and examples to support research findings.

Observations : The researcher typically takes detailed notes on what they observe, including any contextual information, nonverbal cues, or other relevant details. The resulting observational data can be analyzed to gain insights into social phenomena, such as human behavior, social interactions, and cultural practices.

Unstructured interviews : generate qualitative data through the use of open questions.  This allows the respondent to talk in some depth, choosing their own words.  This helps the researcher develop a real sense of a person’s understanding of a situation.

Diaries or journals : Written accounts of personal experiences or reflections.

Notice that qualitative data could be much more than just words or text. Photographs, videos, sound recordings, and so on, can be considered qualitative data. Visual data can be used to understand behaviors, environments, and social interactions.

Qualitative Data Analysis

Qualitative research is endlessly creative and interpretive. The researcher does not just leave the field with mountains of empirical data and then easily write up his or her findings.

Qualitative interpretations are constructed, and various techniques can be used to make sense of the data, such as content analysis, grounded theory (Glaser & Strauss, 1967), thematic analysis (Braun & Clarke, 2006), or discourse analysis.

For example, thematic analysis is a qualitative approach that involves identifying implicit or explicit ideas within the data. Themes will often emerge once the data has been coded .

Key Features

• Events can be understood adequately only if they are seen in context. Therefore, a qualitative researcher immerses her/himself in the field, in natural surroundings. The contexts of inquiry are not contrived; they are natural. Nothing is predefined or taken for granted.
• Qualitative researchers want those who are studied to speak for themselves, to provide their perspectives in words and other actions. Therefore, qualitative research is an interactive process in which the persons studied teach the researcher about their lives.
• The qualitative researcher is an integral part of the data; without the active participation of the researcher, no data exists.
• The study’s design evolves during the research and can be adjusted or changed as it progresses. For the qualitative researcher, there is no single reality. It is subjective and exists only in reference to the observer.
• The theory is data-driven and emerges as part of the research process, evolving from the data as they are collected.

Limitations of Qualitative Research

• Because of the time and costs involved, qualitative designs do not generally draw samples from large-scale data sets.
• The problem of adequate validity or reliability is a major criticism. Because of the subjective nature of qualitative data and its origin in single contexts, it is difficult to apply conventional standards of reliability and validity. For example, because of the central role played by the researcher in the generation of data, it is not possible to replicate qualitative studies.
• Also, contexts, situations, events, conditions, and interactions cannot be replicated to any extent, nor can generalizations be made to a wider context than the one studied with confidence.
• The time required for data collection, analysis, and interpretation is lengthy. Analysis of qualitative data is difficult, and expert knowledge of an area is necessary to interpret qualitative data. Great care must be taken when doing so, for example, looking for mental illness symptoms.

Advantages of Qualitative Research

• Because of close researcher involvement, the researcher gains an insider’s view of the field. This allows the researcher to find issues that are often missed (such as subtleties and complexities) by the scientific, more positivistic inquiries.
• Qualitative descriptions can be important in suggesting possible relationships, causes, effects, and dynamic processes.
• Qualitative analysis allows for ambiguities/contradictions in the data, which reflect social reality (Denscombe, 2010).
• Qualitative research uses a descriptive, narrative style; this research might be of particular benefit to the practitioner as she or he could turn to qualitative reports to examine forms of knowledge that might otherwise be unavailable, thereby gaining new insight.

What Is Quantitative Research?

Quantitative research involves the process of objectively collecting and analyzing numerical data to describe, predict, or control variables of interest.

The goals of quantitative research are to test causal relationships between variables , make predictions, and generalize results to wider populations.

Quantitative researchers aim to establish general laws of behavior and phenomenon across different settings/contexts. Research is used to test a theory and ultimately support or reject it.

Quantitative Methods

Experiments typically yield quantitative data, as they are concerned with measuring things.  However, other research methods, such as controlled observations and questionnaires , can produce both quantitative information.

For example, a rating scale or closed questions on a questionnaire would generate quantitative data as these produce either numerical data or data that can be put into categories (e.g., “yes,” “no” answers).

Experimental methods limit how research participants react to and express appropriate social behavior.

Findings are, therefore, likely to be context-bound and simply a reflection of the assumptions that the researcher brings to the investigation.

There are numerous examples of quantitative data in psychological research, including mental health. Here are a few examples:

Another example is the Experience in Close Relationships Scale (ECR), a self-report questionnaire widely used to assess adult attachment styles .

The ECR provides quantitative data that can be used to assess attachment styles and predict relationship outcomes.

Neuroimaging data : Neuroimaging techniques, such as MRI and fMRI, provide quantitative data on brain structure and function.

This data can be analyzed to identify brain regions involved in specific mental processes or disorders.

For example, the Beck Depression Inventory (BDI) is a clinician-administered questionnaire widely used to assess the severity of depressive symptoms in individuals.

The BDI consists of 21 questions, each scored on a scale of 0 to 3, with higher scores indicating more severe depressive symptoms. 

Quantitative Data Analysis

Statistics help us turn quantitative data into useful information to help with decision-making. We can use statistics to summarize our data, describing patterns, relationships, and connections. Statistics can be descriptive or inferential.

Descriptive statistics help us to summarize our data. In contrast, inferential statistics are used to identify statistically significant differences between groups of data (such as intervention and control groups in a randomized control study).

• Quantitative researchers try to control extraneous variables by conducting their studies in the lab.
• The research aims for objectivity (i.e., without bias) and is separated from the data.
• The design of the study is determined before it begins.
• For the quantitative researcher, the reality is objective, exists separately from the researcher, and can be seen by anyone.
• Research is used to test a theory and ultimately support or reject it.

Limitations of Quantitative Research

• Context: Quantitative experiments do not take place in natural settings. In addition, they do not allow participants to explain their choices or the meaning of the questions they may have for those participants (Carr, 1994).
• Researcher expertise: Poor knowledge of the application of statistical analysis may negatively affect analysis and subsequent interpretation (Black, 1999).
• Variability of data quantity: Large sample sizes are needed for more accurate analysis. Small-scale quantitative studies may be less reliable because of the low quantity of data (Denscombe, 2010). This also affects the ability to generalize study findings to wider populations.
• Confirmation bias: The researcher might miss observing phenomena because of focus on theory or hypothesis testing rather than on the theory of hypothesis generation.

Advantages of Quantitative Research

• Scientific objectivity: Quantitative data can be interpreted with statistical analysis, and since statistics are based on the principles of mathematics, the quantitative approach is viewed as scientifically objective and rational (Carr, 1994; Denscombe, 2010).
• Useful for testing and validating already constructed theories.
• Rapid analysis: Sophisticated software removes much of the need for prolonged data analysis, especially with large volumes of data involved (Antonius, 2003).
• Replication: Quantitative data is based on measured values and can be checked by others because numerical data is less open to ambiguities of interpretation.
• Hypotheses can also be tested because of statistical analysis (Antonius, 2003).

Antonius, R. (2003). Interpreting quantitative data with SPSS . Sage.

Black, T. R. (1999). Doing quantitative research in the social sciences: An integrated approach to research design, measurement and statistics . Sage.

Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology . Qualitative Research in Psychology , 3, 77–101.

Carr, L. T. (1994). The strengths and weaknesses of quantitative and qualitative research : what method for nursing? Journal of advanced nursing, 20(4) , 716-721.

Denscombe, M. (2010). The Good Research Guide: for small-scale social research. McGraw Hill.

Denzin, N., & Lincoln. Y. (1994). Handbook of Qualitative Research. Thousand Oaks, CA, US: Sage Publications Inc.

Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968). The discovery of grounded theory; strategies for qualitative research. Nursing research, 17(4) , 364.

Minichiello, V. (1990). In-Depth Interviewing: Researching People. Longman Cheshire.

Punch, K. (1998). Introduction to Social Research: Quantitative and Qualitative Approaches. London: Sage

Further Information

• Designing qualitative research
• Methods of data collection and analysis
• Introduction to quantitative and qualitative research
• Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?
• Qualitative research in health care: Analysing qualitative data
• Qualitative data analysis: the framework approach
• Using the framework method for the analysis of
• Qualitative data in multi-disciplinary health research
• Content Analysis
• Grounded Theory
• Thematic Analysis

Related Articles

Research Methodology

Qualitative Data Coding

What Is a Focus Group?

Cross-Cultural Research Methodology In Psychology

What Is Internal Validity In Research?

Research Methodology , Statistics

What Is Face Validity In Research? Importance & How To Measure

Criterion Validity: Definition & Examples

• Research Note
• Open access
• Published: 07 June 2024

A proposed analytical framework for qualitative evaluation of access to medicines from a health systems perspective

• I. R. Joosse 1 ,
• H. A. van den Ham 1 ,
• A. K. Mantel-Teeuwisse 1 &
• F. Suleman 2  

BMC Research Notes volume  17 , Article number:  159 ( 2024 ) Cite this article

19 Accesses

Metrics details

Despite global recognition that access to medicines is shaped by various interacting processes within a health system, a suitable analytical framework for identifying barriers and facilitators from a system’s perspective was needed. We propose a framework specifically designed to find drivers to access to medicines from a country’s health system perspective. This framework could enable the systematic evaluation of access across countries, disease areas and populations and facilitate targeted policy development. This framework is the byproduct of a larger study on the barriers and facilitators to childhood oncology medicines in South Africa.

Eight core (pharmaceutical) functional processes were identified from existing frameworks: (I) medicine regulation , (II) public financing and pricing , (III) selection , (IV) reimbursement , (V) procurement and supply , (VI) healthcare delivery , (VII) dispensing and (VIII) use . National contextual components included policy and legislation and health information systems . To emphasize the interlinkage of processes, the proposed framework was structured as a pharmaceutical value chain. This framework focusses on national processes that are within a country’s control as opposed to global factors, and functional mechanisms versus a country’s performance or policy objectives. Further refinement and validation of the framework following application in other contexts is encouraged.

Peer Review reports

Introduction

Medicines are considered a key component of health systems and a major contributor to health outcomes [ 1 ]. Access to medicines is determined by the interaction between a multitude of factors and processes, not just in the pharmaceutical value chain but also within the broader context of the health system [ 2 ]. Research on accessibility of medicines is often focused on a particular process in isolation from related elements [ 3 , 4 , 5 ], or on the downstream effects for the patient– e.g. availability and affordability [ 6 , 7 , 8 , 9 , 10 ]. However, the entirety of the pharmaceutical system must be taken into consideration to get a comprehensive understanding of the drivers of accessibility.

To that end, we set out to get a better understanding of the drivers and barriers that determine access to pediatric oncology medicines in South Africa. To enable a comprehensive analysis of the issues influencing health system efficiencies and its ability to provide equitable access to childhood cancer medicines, we looked to available analytical tools to inform our qualitative analyses and development of an interview guide. The Paediatric Oncology System Integration Tool (POSIT) was reviewed and deemed suitable for constructing an interview guide [ 11 ]. POSIT was developed to facilitate analyses of the performance of childhood cancer programs in low- and middle-income countries (LMIC) within the context of their health system. Although medicines are only one element of a health system, many of the system functions and performance goals for health systems outlined in POSIT align with those for medicines.

However, thematic analysis of in-depth interviews with stakeholders soon revealed the limitations of POSIT in applying it specifically in the context of access to medicines, with several functional domains that are unique for pharmaceuticals missing from the framework. Missing elements included the regulation and registration of medicines, the selection of essential medicines, and preparing, dispensing and safe administration of pharmaceuticals. Other existing frameworks on access to medicines were either limited in scope or lacked specificity on pharmaceutical processes and were therefore not considered appropriate alternatives [ 2 , 12 , 13 , 14 , 15 , 16 ].

With no single existing framework fit for the qualitative analysis of barriers and enablers in access to medicines, we aimed to develop a new analytical framework specifically designed to evaluate access to medicines from a health systems perspective within a country. Such a framework could yield a comprehensive health systems overview of drivers of access and concrete recommendations for improvement and policy development from stakeholders’ perspectives.

Core components of an access to medicines framework

To inform core pharmaceutical functional processes, elements from two existing frameworks were used to construct a new framework: (1) the childhood cancer system functional domains in POSIT [ 11 ], and (2) the pharmaceutical management framework published in ‘MDS-3: Managing Access to Medicines and other Health Technologies’ [ 12 ]. Managing Drug Supply-3 (MDS-3) is a reference guide detailing sustainable management of essential medicines in LMIC.

The analytical framework proposed by Bigdeli and colleagues was not used to construct our framework [ 2 ]. Although their framework is tailored to medicines and provides a complete overview of the complex components, levels and interconnections that determine access to medicines, it was developed for use in policy design whereas we sought to analyze drivers of access by understanding how effective collective action across the value chain through numerous stakeholders supports access to medicines. Additionally, we wanted to focus on national processes that are within a country’s sphere of influence as compared to global mechanisms. The framework proposed by the World Health Organization (WHO) in 2004 targets performance dimensions (e.g. sustainability , affordability , etc.) rather than functional processes (e.g. regulation , reimbursement , dispensing , etc.) and focusses on key outcomes for coordinated global action, and was therefore not used [ 13 ]. The WHO guidelines on developing National Medicines Policies [ 14 ] and derivatives [ 15 , 16 ] are policy oriented and may not adequately capture the practical effects and lived experiences of such policies or the performance of functional pharmaceutical processes, and position medicines vertically rather than integrated in the health system.

The five functional domains of POSIT (e.g. governance , financing , demand generation , health information systems and service delivery ) were combined with the four basic functions of pharmaceutical management (e.g. selection , procurement , distribution and use (including prescribing and dispensing )) and contextual elements management support and policy, law and regulation . Figure  1 provides a schematic representation of the generation of the framework.

From that, we identified eight core functional process: (I) medicine regulation , (II) public financing and pricing , (III) selection , (IV) reimbursement , (V) procurement and supply , (VI) healthcare delivery , (VII) dispensing and (VIII) use (including social and societal aspects). Other core components that influence the context under which the functional processes are taking place are policy and legislation and health information systems . Recognizing that each element builds on a previous component, we chose to map the framework as a pharmaceutical value chain (panel A, Fig.  2 ) [ 17 ]. This figure also illustrates how the framework was applied to the qualitative study of barriers and facilitators in access to pediatric cancer medicines in South Africa (panel B), while highlighting additional aspects that did not emerge in this specific case study but were recognized as potentially relevant elements in analogous frameworks (panel C) [ 11 , 12 , 13 , 14 , 15 , 16 ].

Policy and legislation

Policy and legislation captures how the pharmaceutical system and broader healthcare structures within a country are organized, managed, and regulated through policies, laws or mandates [ 11 ]. The political environment is also covered within this theme.

Medicine regulation

Regulation involves the marketing registration of medicines, pharmacovigilance activities, the licensing for manufacturing, distributing, storage and sale of pharmaceuticals, as well as importation and exportation [ 12 ]. Substandard and falsified medicines may also be considered here.

Public financing and pricing

Public financing involves the generation, pooling, and allocation of public funds to cover medicines and services [ 11 ]. This may also include donations. Private funding is considered under reimbursement . Pricing considers the prices and affordability of medicines and mechanisms used to regulate prices. Frequently used pricing mechanisms include internal reference pricing, external reference pricing and value-based pricing [ 18 ].

Selection encompasses the identification of prevalent health problems and selecting evidence based treatments of choice, choosing individual medicines and preferred dosage forms, and deciding which medicines will be available at each level of a health care system [ 12 ], usually in the form of a national Essential Medicines List (NEML) or formulary, and Standard Treatment Guidelines (STGs). This element also considers processes for making non- NEML or non-formulary listed medicines available to patients.

Reimbursement

We consider reimbursement to include the coverage of pharmaceuticals in national or social medical insurance plans, subsequent reimbursement prices by third party payers, mechanisms to determine reimbursement prices, co-payments and the regulation of private sector medical insurance schemes [ 11 , 19 ]. This element also considers processes for reimbursement/payment of medicines that are not covered by insurance schemes.

Procurement and supply

Procurement and supply entails the selection and management of procurement methods– including tenders. In addition, distribution processes are also covered within this theme, encompassing aspects related to customs, stock control, and delivery to drug depots and health facilities [ 12 ]. Availability of medicines in health facilities is also considered here.

Healthcare delivery

Healthcare delivery encompasses a range of structures, resources, services, healthcare professionals and other individuals required for the diagnosis and provision of care [ 11 ]. We consider prescribing of medicines to be part of this component.

Dispensing comprises the process of preparing and giving or administering a medicine by a pharmacist or other healthcare professional to a named patient, frequently on the basis of a prescription [ 12 ]. However, over-the-counter (OTC) use and self-medication may also be considered here.

We consider use as the proper medicine consumption by the patient, as well as the ability of people to command appropriate healthcare resources. This includes patients’ knowledge on available health services and treatments, physical accessibility of services, and acceptability of services and medicines within associated social and societal structures [ 20 ].

Health information systems

This component captures by which means data about disease burden and clinical patterns, health outcomes, and the achievement of objectives in the health system is collected, analyzed and reviewed [ 11 , 12 ]. Monitoring and surveillance are a critical element herein.

In current literature, an increasing number of studies describe and analyze access to medicines from a country’s health system perspective [ 21 , 22 , 23 , 24 , 25 ], but a complete and suitable framework to facilitate a qualitative analysis was missing. In previous studies, elements from different frameworks needed to be combined to arrive at a suitable structure for analysis [ 22 , 23 , 24 , 25 ], similar to our own experience. This illustrates the necessity for an amended framework for qualitative research on access to medicines that encompasses the full scope of national functional domains in the pharmaceutical value chain. Similar to Bigdeli et al. and POSIT, we have adopted a health systems perspective on access to medicines, to highlight the interconnectedness of medicines and pharmaceutical processes with other key variables within the health system [ 2 , 11 ].

Unlike most other existing framework, functional domains (‘ what we need to do’ ) rather than performance dimensions (‘ what we aim to achieve’ ) were taken as basis for this framework [ 2 , 11 , 13 ]. Although we consider these performance dimensions to be critical in policy design and development, the level of detail required to identify barriers is missing when performance is suboptimal. The proposed framework was specifically designed to address this gap in understanding how effective collective action across the value chain by numerous stakeholders supports access to medicines. Additionally, when applying our framework to a case study of childhood oncology medicines in South Africa, we have experienced that important performance dimensions of access spontaneously emerge (e.g. availability , equity , affordability , etc.), further enriching the findings.

Rather than providing a checklist through which one could perform a gap-analysis of whether specific policies or processes are in place, we provide an open structure for a qualitative assessment of how functional processes operate and how they impact access. For even when a given policy or process is theoretically in place, its practical effects and lived experiences may differ from what was intended. For example, a national tender process might be in place, but tenders can fail due to too strict participation requirements. Our open structure distinguishes our framework from prior works, and allows for more in-depth discussion and probing. Recognizing that existing frameworks and key documents were designed for different purposes, the proposed framework is meant to complement earlier work rather than replace them. An important strength of this novel framework is its intuitiveness. Critical processes that take place between the moment a medicine is registered for use in the country and actual use by the patient are compartmentalized to facilitate each component’s in-depth analysis, while also emphasizing the interaction between pharmaceutical processes, other healthcare services and actors and social factors. Correspondingly, all six WHO health system building blocks are captured within our framework [ 1 ]. The proposed framework was designed to be used in conjunction with different qualitative pharmaceutical policy analysis methods to derive a complete picture of the situation, including analyses of policy documents (such as a national medicines policy) and public information sources, key informant or patient interviews, and health facility surveys. Finally, in order to adequately capture all practical effects and lived experiences of existing policies and processes, analyses should encompass stakeholders across the value chain and not be restricted to policy-makers.

Limitations

Inevitably, the compartmentalization of functional components in the pharmaceutical value chain oversimplifies the complexities of the health system and may underplay the importance of upstream factors that determine access to care. It also divides processes which are highly interlinked. At the same time, separating these components helps to put boundaries around complex processes, which minimizes the risk of key functional processes being overlooked, and thus facilitates identification of a range of barriers and enablers. Furthermore, this framework is not all-encompassing. We provide a general structure for systematic analysis of drivers of inaccessibility, but the analysis of access in different countries, therapeutic areas or populations may require the evaluation of other subdomains within these core components. We emphasize that this tool should not be considered a universal checklist, and adaptions to national health systems may be necessary. Additional cross-cutting themes that cannot be captured in a single core component may also be identified during analysis. Besides these emerging themes, global processes such as market forces , innovation and manufacturing – that undeniably have an impact on access to medicines as well– are not included in this framework as these are often beyond a country’s influence [ 2 ].

This framework is one outcome of a larger study looking into the barriers and facilitators to childhood oncology medicines in South Africa. With that, there was no protocolized, systematic approach to develop this framework. However, we have nonetheless taken careful approaches to ascertain that it reflects key processes and factors, having taken existing frameworks into consideration in the design of our framework [ 2 , 11 , 12 , 13 ] and undertaken further verification through iterative discussions among authors.

We propose a widely applicable analytical framework for studying qualitative access to medicines from a country’s health system perspective, outlining critical functional processes in the pharmaceutical value chain. We believe this framework could facilitate future analyses of barriers and enablers in accessing medicines, leading to a systematic understanding of determinants of access and potentially guiding targeted policy development. Although we expect the framework to be appropriate for studying other countries, diseases and populations in a structured manner, it is the derivative of a single case study in South Africa. It has yet to prove its usefulness across different contexts, and refinements may be needed to ensure its broad applicability and comprehensiveness. Testing and implementing the proposed framework in various contexts will contribute to its refinement and practical utility.

Mapping of POSIT [ 11 ] and MDS-3 [ 12 ] to construct a new analytical framework to identify barriers and facilitators to medicines’ access. Performance goals and dimensions as well as functional subdomains of POSIT not shown. POSIT: Paediatric Oncology System Integration Tool; MSD-3: Managing Drug Supply-3

Proposed analytical framework for qualitative evaluation of barriers and facilitators in access to medicines. Panel A: general proposed analytical framework for access to medicines. Panel B: analytical framework applied to the qualitative study of barriers and facilitators in accessing childhood oncology medicines in South Africa (emerging cross-cutting themes not shown). Panel C: additional aspects to be considered in alternative contexts

Data availability

Data sharing is not applicable to this article as no data from the study is described in the presentation of the proposed analytical framework.

Abbreviations

Low- and Middle-Income Countries

Managing Drug Supply edition 3

National Essential Medicines List

Paediatric Oncology System Integration Tool

Standard Treatment Guideline

World Health Organization

World Health Organization. Monitoring the building blocks of health systems: a handbook of indicators and their measurement strategies. Geneva: World Health Organization. 2010. https://apps.who.int/iris/handle/10665/258734 .

Bigdeli M, Jacobs B, Tomson G, Laing R, Ghaffar A, Dujardin B, et al. Access to medicines from a health system perspective. Health Policy Plan. 2013;28(7):692–704.

Article   PubMed   Google Scholar  

Lyus R, Pollock A, Ocan M, Brhlikova P. Registration of antimicrobials, Kenya, Uganda and United Republic of Tanzania, 2018. Bull World Health Organ. 2020;98(8):530–8.

Article   PubMed   PubMed Central   Google Scholar  

Brhlikova P, Maigetter K, Murison J, Agaba AG, Tusiimire J, Pollock AM. Registration and local production of essential medicines in Uganda. J Pharm Policy Pract. 2020;13:31.

Omer S, Pan M, Ali S, Shukar S, Fang Y, Yang C. Perceptions of pharmacists towards drug shortages in the healthcare system of Pakistan and its impact on patient care: findings from a cross-sectional survey. BMJ Open. 2021;11(12):e050196.

Robertson J, Forte G, Trapsida J-M, Hill S. What essential medicines for children are on the shelf? Bull World Health Organ. 2009;87(3):231–7.

Musoke D, Sodemann M. Availability of essential medicines across levels of care in Gulu District, northern Uganda. Am J Pharmacol Sci. 2016;4(1):11–4.

Google Scholar  

Birabwa C, Murison J, Evans V, Obua C, Agaba A, Waako P, et al. The availability of six tracer medicines in private medicine outlets in Uganda. J Pharm Policy Pract. 2014;7(1):18.

Sun X, Wei J, Yao Y, Chen Q, You D, Xu X, et al. Availability, prices and affordability of essential medicines for children: a cross-sectional survey in Jiangsu Province, China. BMJ Open. 2018;8(10):e023646.

Saeed A, Saeed F, Saeed H, Saleem Z, Yang C, Chang J, et al. Access to Essential Cardiovascular Medicines in Pakistan: A National Survey on the availability, price, and affordability, using WHO/HAI methodology. Front Pharmacol. 2021;11:595008.

Maser B, Force L, Friedrich P, Antillon F, Arora RS, Herrera CA, et al. Pediatric Oncology System Integration Tool (POSIT): a health systems performance assessment framework for childhood cancer care in low- and middle-income countries. J Cancer Policy. 2020;23:100208.

Article   Google Scholar  

Management Sciences for Health. MDS-3: managing access to medicines and health technologies. 2014. https://msh.org/wp-content/uploads/2014/01/mds3-jan2014.pdf [cited 3 April 2023].

World Health Organization. Equitable access to essential medicines: a framework for collective action. Geneva: World Health Organization. 2004. https://iris.who.int/handle/10665/68571 .

World Health Organization. How to develop and implement a national drug policy, second edition. Geneva: World Health Organization. 2001. https://www.who.int/publications/i/item/924154547X .

Wirtz VJ, Hogerzeil HV, Gray AL, Bigdeli M, de Joncheere CP, Ewen MA, et al. Essential medicines for universal health coverage. Lancet. 2017;389(10067):403–76.

Perehudoff SK, Alexandrov NV, Hogerzeil HV. The right to health as the basis for universal health coverage: a cross-national analysis of national medicines policies of 71 countries. PLoS ONE. 2019;14(6):e0215577.

Article   CAS   PubMed   PubMed Central   Google Scholar  

United Nations/UN Toolkit on Synthetic Drugs. The Pharmaceutical Value Chain [internet]. https://syntheticdrugs.unodc.org/syntheticdrugs/en/access/pharmaceutical/index.html [cited 3 April 2023].

World Health Organization. WHO guideline on country pharmaceutical pricing policies. 2nd ed. Geneva: World Health Organization; 2020.

PPRI. Glossary [internet]. https://ppri.goeg.at/ppri-glossary [cited 3 April 2023].

Gulliford M, Figueroa-Munoz J, Morgan M, Hughes D, Gibson B, Beech R, et al. What does ‘access to health care’ mean? J Health Serv Res Policy. 2002;7(3):186–8.

Zaidi S, Bigdeli M, Aleem N, Rashidian A. Access to essential medicines in Pakistan: policy and health systems research concerns. PLoS ONE. 2013;8(5):e63515.

Rizk HI, Elkholy MM, Barakat AA, Elsayed RMM, Abd El Fatah SAM. Perspectives of pharmaceutical stakeholders on determinants of medicines accessibility at the primary care level. J Egypt Public Health Assoc. 2021;96(1):1.

Boateng R, Renner L, Petricca K, Gupta S, Denburg A. Health system determinants of access to essential medicines for children with cancer in Ghana. BMJ Glob Health. 2020;5(9):e002906.

Tang B, Bodkyn C, Gupta S, Denburg A. Access to WHO Essential Medicines for Childhood Cancer Care in Trinidad and Tobago: A Health System Analysis of barriers and enablers. JCO Glob Oncol. 2020;6:67–79.

Boateng R, Petricca K, Tang B, Parikh S, SinQuee-Brown C, Alexis C, et al. Determinants of access to childhood cancer medicines: a comparative, mixed-methods analysis of four Caribbean countries. Lancet Glob Health. 2021;9(9):e1314–24.

Article   CAS   PubMed   Google Scholar  

Download references

Acknowledgements

Not applicable.

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Author information

Authors and affiliations.

Utrecht WHO Collaborating Centre for Pharmaceutical Policy and Regulation, Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht Institute for Pharmaceutical Sciences (UIPS), Utrecht University, Universiteitsweg 99, Utrecht, 7324, 3584 CG, The Netherlands

I. R. Joosse, H. A. van den Ham & A. K. Mantel-Teeuwisse

WHO Collaborating Centre for Pharmaceutical Policy and Evidence Based Practice, School of Health Sciences, University of KwaZulu-Natal, Durban, South Africa

You can also search for this author in PubMed   Google Scholar

Contributions

All authors were involved in conception and design of the framework. IRJ drafted the manuscript and HAvdH, AKM-T and FS were involved in critical revision of the article. All authors approved the final article.

Corresponding author

Correspondence to H. A. van den Ham .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Joosse, I.R., van den Ham, H.A., Mantel-Teeuwisse, A.K. et al. A proposed analytical framework for qualitative evaluation of access to medicines from a health systems perspective. BMC Res Notes 17 , 159 (2024). https://doi.org/10.1186/s13104-024-06764-1

Download citation

Received : 18 April 2023

Accepted : 02 April 2024

Published : 07 June 2024

DOI : https://doi.org/10.1186/s13104-024-06764-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

BMC Research Notes

ISSN: 1756-0500

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 29 September 2022

Determinants of prescribing decisions for off-patent biological medicines in Belgium: a qualitative study

• Yannick Vandenplas 1 ,
• Steven Simoens 1 ,
• Philippe Van Wilder 2 ,
• Arnold G. Vulto 1 , 3 ,
• Florian Turk 4 &
• Isabelle Huys 1  

BMC Health Services Research volume  22 , Article number:  1211 ( 2022 ) Cite this article

1388 Accesses

2 Citations

2 Altmetric

Metrics details

A competitive market for off-patent biologicals leads to more affordable and high-quality healthcare. In recent years, Belgium has been characterized by its low use of biosimilars and by its shifts from off-patent biologicals toward new alternative therapies. Yet, the prescribing decisions involved in these observations are poorly understood. This study aims to better understand prescribing choices among Belgian physicians in the ambulatory care setting.

This study consisted of two phases. First, a scoping literature review to identify determinants of prescribing choices was conducted. Scientific databases (Embase and PubMed) were searched until 4 November 2021. Second, the nominal group technique (NGT) was employed during focus group discussions with Belgian physicians to consider and validate these determinants for off-patent biologicals in the Belgian context. The qualitative data resulting from the literature review and focus group discussions were analyzed using the thematic framework method.

Fifty-three scientific articles that discussed elements that determine prescribing choices were identified. Out of these, 17 determinants of prescribing choices were found. These were divided into five categories: (1) product-related, (2) physicians’ personal, (3) healthcare system-related, (4) patient-related, and (5) determinants related to the pharmaceutical company or brand. Nineteen Belgian physicians from different therapeutic areas that regularly prescribe biologicals then participated in focus group discussions. Using the NGT, the group discussions revealed that prescribing choices for off-patent biologicals are determined by a complex set of elements. Clinical data, geographical region, working environment, pharmaceutical marketing, patient profile, clinical guidelines, and preference of key opinion leaders (KOL) were considered most influential. Physicians indicated that the importance of these determinants differs depending on product classes or therapeutic domain.

Conclusions

Multiple elements determine the choice of an off-patent biological or biosimilar product. The importance of each of these determinants varies depending on the context in which the prescribing choice is made. To increase the prescription of best-value biologicals in the Belgian ambulatory care, a set of synergistic measures is required including information for healthcare providers (HCP) and patients, prescribing feedback, prescribing targets, tangible incentives, KOL involvement, guidelines regarding pharmaceutical promotion, and regular revision of reimbursement modalities.

Peer Review reports

Governments have increasingly experienced difficulties in keeping their healthcare budgets under control over the past few decades. This problem has become even more pressing as there is a growing number of challenges faced by healthcare systems worldwide, such as ageing populations, the emergence of expensive innovative therapies like precision medicine, and the recent COVID-19 pandemic [ 1 ]. Belgium is no exception and faces these challenges as well. Belgian healthcare spending has increased annually and is now among the highest in the European Union [ 2 ]. A significant portion of healthcare spending is on pharmaceuticals [ 3 ]. The most recent data show that pharmaceuticals represent 18% of the total Belgian healthcare budget, with an overall pharmaceutical budget of approximately 5 billion euros in 2021 [ 4 , 5 , 6 ]. Moreover, the pharmaceutical budget in Belgium has risen rapidly in recent years, with a growth of 10.7% in 2021. This situation is no longer sustainable given the limited resources available in a country where healthcare is mainly publicly funded. The national health insurer (the National Institute of Health and Disability Insurance or NIHDI) also identified this problem during their recent budget discussions [ 4 , 7 ].

Therefore, there is a clear need to develop strategies to maintain the financial sustainability of the Belgian healthcare system. One of these strategies should focus on the rational and cost-effective use of medicines, more specifically of biological medicines. Biological medicines or biologicals are medicines produced by living organisms, such as bacteria or animal cells [ 8 ]. They have been successful in treating several life-threatening or chronic disorders such as cancer, diabetes, and immune mediated inflammatory disorders. Because of a costly development and manufacturing process, biological medicines are generally more expensive compared to traditional chemical molecules [ 9 ]. Together with their widespread use, this results in biologicals representing a high cost to national healthcare systems. To date, biologicals represent about 30% of all pharmaceutical spending across Europe [ 5 ]. The expected advent of a large number of new biological therapies in the coming years will probably result in a further increase of this share [ 1 ]. After the expiry of market exclusivities of biologicals and the market entry of similar versions of authorized biologicals (i.e., biosimilars), off-patent biologicals obtain a more favorable cost-effectiveness profile [ 10 ]. Yet, in order to benefit from these more cost-effective medicines, a sustainable off-patent biologicals market is required. A sustainable market refers to a situation in which both off-patent biological and biosimilar products can coexist. It therefore not only strives towards a competitive market, but also towards economic viability for pharmaceutical industry [ 11 ]. However, it has become clear that the Belgian market is not sustainable [ 12 , 13 ]. Significant shifts from off-patent biologicals towards new therapeutic classes (such as interleukin (IL) inhibitors, Janus Kinase (JAK) inhibitors, etc.) and the dominant market position of originator biologicals that lost their market exclusivities (and thus a low use of biosimilars) are the main elements supporting this observation [ 12 , 13 , 14 ]. Biosimilars (including etanercept, adalimumab, and insulin glargine) have been part of Belgium’s ambulatory care since 2016. Yet, their market shares have only increased slowly over the past few years and remain very limited [ 12 , 13 ]. This is especially so in the ambulatory segment of the market where the choice for a particular biological is made by the prescribing physician and is not subject to tender decisions [ 12 , 13 ].

Due to substantial mandatory price reductions for both biosimilars and reference biological products, price differences are limited and both generate savings for the Belgian healthcare system. Therefore, the concept of best-value biologicals was introduced, which recognizes that both originator biologicals and biosimilars can contribute to a more sustainable healthcare system [ 13 , 15 ]. The Belgian government supports the concept of best-value biologicals and introduced several initiatives to encourage the use of cost-effective biologicals in the past years [ 12 ]. In addition, given the very low biosimilar market shares in Belgium’s ambulatory care, the Belgian government has aimed to specifically increase the use of biosimilars as well [ 4 , 12 , 16 , 17 ]. However, these initiatives have not been successful to date, underlining a poor understanding of what drives prescribing decisions within the off-patent biologicals market [ 12 , 13 ]. As a result, it is of particular interest to better understand what elements determine the choice between different molecules, as well as between a reference biological and its biosimilar(s). In order to design policy interventions to stimulate physicians to prescribe cost-effective medicines in the future, the underlying drivers behind prescribing decisions have to be clearly understood [ 18 , 19 ].

This study aimed to identify elements that determine the prescription choice for off-patent biologicals in the Belgian ambulatory setting. In doing so, we must not lose sight of the current reality and set the goal of looking beyond the choice between reference products and their biosimilar(s). As described earlier, when choosing a biological product, new molecules or follow-on products are also considered. Ultimately, a better understanding of prescribing choices can be used to inform Belgian policymakers when they are designing interventions to stimulate the prescription of best-value biological medicines or biosimilars.

Since there currently is no published evidence on the kinds of elements that determine the prescription of off-patent biological medicines, a two-step qualitative approach was used to identify these determinants. During the first step, an overview was made of elements that generally determine prescribing behavior based on a scoping literature review of scientific databases. This served to inform the next step where these identified elements were discussed in focus groups with Belgian physicians. An overview of the consecutive steps of the used methodological approach is provided in Fig.  1 .

Overview of the methodological approach of this study

Scoping literature review

A scoping review of the literature in Embase and PubMed (MEDLINE) was conducted using a predefined search strategy. Scoping literature reviews aim to identify and map the existing evidence related to a certain topic of interest in a systematic way [ 20 , 21 ]. Full-text, peer-reviewed, English articles that were published between 2008 and 2021 were eligible for inclusion. Both original research articles and literature reviews were considered. Only articles that described determinants of prescribing decisions were included. Search terms were therefore related to ‘prescribing’, ‘behavior’, and ‘physician’. These search terms were adjusted according to the respective scientific database. Initially, the literature was searched up to March 2021 but was subsequently updated to 4 November 2021. The full literature search protocol can be found in the Supplementary Material .

All identified articles were imported into Mendeley software and duplicates were removed. The remaining articles were then screened by one reviewer (YV) based on title and abstract using Rayyan software [ 22 ]. Subsequent screening based on the full text of the selected articles was conducted by the same researcher (YV), and the remaining articles were included in the qualitative analysis. Peer-reviewed articles describing determinants that influence prescribing choices were analyzed qualitatively using the thematic framework method described by Gale et al. [ 23 ]. This process included mostly inductive coding as the researchers did not know in advance which determinants were involved in prescribing decisions. During the analysis of the first articles by two researchers (YV and IH), similar codes were identified as determinants of prescribing behavior and were grouped together to form a coding tree (i.e., thematic framework). The codes of the developed framework were agreed upon by the two researchers who coded the first articles. Subsequently, the remaining articles were analyzed by placing relevant passages under the developed thematic framework. In case new themes or codes were found, these were added to the existing framework. In the last phase, the results were summarized in a framework matrix and interpreted by the researchers.

Focus group discussions using the nominal group technique (NGT)

After general determinants of prescribing decisions were identified from the scoping literature review, the purpose of this study was to further examine these determinants in relation to off-patent biological medicines in the Belgian context. The results of the scoping literature review were therefore used as a starting point. Each of the identified determinants were discussed separately during focus group discussions and were scored quantitatively by the participants according to their relative importance. The nominal group technique (NGT) was used to achieve the latter. The NGT is a method to establish structured group discussions or interactions, which encourages all participants to provide their insights and to react to these insights [ 24 ]. It was chosen since the NGT allows for structured group discussions with the purpose to achieve an agreement related to a particular topic of interest. The NGT was considered appropriate for this research question as the technique also allows for ranking of priorities and the generation of new ideas [ 24 ].

The NGT consists of four distinct parts as defined by Delbecq and Van de Ven: (i) silent generation, (ii) round robin (i.e., the moderator asks each participant to share a single idea with the group), (iii) clarification, and (iv) ranking [ 24 , 25 ]. However, the NGT is a highly adaptable method depending on the specific research question and desired outcomes [ 24 ]. This allows researchers to replace or skip certain steps. For example, in this study, silent generation was omitted and replaced by a comprehensive literature review. Based on the ideas identified (in this case, determinants of prescribing behavior), the participants performed an initial ranking step prior to the group discussions. During the ranking exercise, each participant was asked to score the importance of each of the identified determinants on a five-point Likert scale in the following way: 1 = very low impact, 2 = low impact, 3 = some impact, 4 = high impact, 5 = very high impact. Average ranking values were calculated per determinant. In addition, participants were asked if there were other determinants influencing prescribing decisions regarding off-patent biologicals or if certain elements were irrelevant in this regard. After the initial ranking step, the traditional steps of the NGT were followed. As a result, the NGT consisted of the following steps:

First grading step prior to the group discussion, based on the list of determinants of prescribing decisions resulting from the scoping literature review

Presentation of the aggregated results of the first grading step to the participants (i.e., average ranking values per determinant)

Group discussion where each participant was asked to comment on the identified determinants of prescribing decisions and the results of the first grading step

Second grading step where participants were asked to rank the identified determinants again

Concluding step where each participant could share any final thoughts or remarks.

Usually, group discussions take place face-to-face, but the COVID-19 pandemic situation did not allow for physical interactions at the time when the study was conducted. Therefore, all the group discussions took place remotely via Microsoft Teams. The focus group discussions were anticipated to last between 90 and 120 minutes. All focus groups were moderated by the same researchers (YV and IH), who have experience with conducting qualitative research.

Participant selection and recruitment

Participants were Belgian specialist physicians working in therapeutic domains where off-patent biologicals and biosimilars are being prescribed in the ambulatory care setting. These products are listed in the Supplementary Material and mainly include tumor necrosis factor (TNF)-alpha inhibitors and insulin analogues. Therefore, eligible participating physicians were endocrinologists, dermatologists, gastroenterologists, and rheumatologists. The physicians were required to have a good knowledge of biologics and experience with prescribing both originator biologicals and biosimilars. The physicians were selected based on purposeful sampling, that is the identification and selection of individuals or groups who are especially knowledgeable about the topic of interest [ 26 ]. Participants were contacted via e-mail. Their contact details were obtained via publicly available information or through the network of the research unit where this study was conducted.

NGT group discussions usually involve two to fourteen participants, with an ideal number of seven participants [ 24 ]. To ensure a good balance between input coming from a variety of participants and giving each participant sufficient time to speak, the researchers aimed to have a minimum of four and a maximum of ten participants per group. The researchers ensured that a variety of backgrounds (i.e., region, working environment, therapeutic area) were present in each of the focus groups. Differences in language (i.e., Dutch or French) were also considered. Therefore, participants were asked to speak English during the focus groups. However, if necessary, participants could express themselves in their native language and the moderators acted as translators. Eligible participants were invited via e-mail and provided with all relevant study information and the informed consent form (Cfr. Supplementary Material ). All participants were required to provide written informed consent prior via e-mail to the start of the study. The study was approved by the Ethics Committee Research of UZ/KU Leuven (S65328).

Data analysis

The group discussions were audio recorded and transcribed verbatim manually. The transcripts were analyzed according to the thematic framework with NVivo software (Release 1.3), using a combination of inductive and deductive coding [ 23 ]. The deductive coding was based on the findings of the first step of this study (i.e., the scoping literature review). The transcripts were analyzed by categorizing the relevant transcript sections to the thematic framework that was developed during the first step of this study. Additional codes that arose during the group discussions were added inductively to the coding tree. The group discussions also served to detect subcodes under the existing themes that were identified through the scoping literature review. Next, the results of the coding process were charted into a framework matrix and interpreted by the researchers. Ultimately, the results were narratively described, and illustrative quotes supporting the findings were extracted.

To guarantee the confidentiality of all participants, the recordings and transcripts were pseudonymized by removing all names and references towards the identifiable person (Cfr. Supplementary Material ). These were replaced by a code randomly assigned to each participant per focus group, of which only the researchers have the code key to identify each participant.

The NGT also generates a small amount of quantitative data in addition to qualitative data. These are the result of the first and second ranking step (i.e., the five-point Likert scale) and are reported as an average score per determinant. The results of the first ranking step were presented to the participants, while the average scores per determinant of the second ranking were reported as the final result of this study (Cfr. Table  3 ). Prior to the study, a mean cutoff ranking score of 3.5 was defined by the researchers to identify the predominant determinants. The cutoff score was based on previous similar research, as well as on the perceptions of the NGT participants [ 27 ]. Finally, all average scores per determinant were ranked from high to low (Table 3 ).

As discussed above, a scoping literature review was conducted prior to the focus group discussions in order to identify elements that drive prescribing decisions in general. These identified determinants were used as a starting point for the focus group discussions with Belgian physicians.

The predefined search strategy identified a total of 2587 records. After the removal of duplicates and screening of the title and abstract, 48 articles remained. Further screening of the full-text articles led to 10 articles being excluded. The literature search was updated at the time the researchers finalized the analysis, by again using the same search strategy to look for newly published articles. This resulted in an additional 15 records that were included in the qualitative analysis. Therefore, 53 articles were included in the final qualitative analysis. An overview of the literature search process, including the flow chart of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [ 28 ], is included in the Supplementary Material .

Five main categories of determinants that drive prescribing choices were identified: (1) product-related, (2) physicians’ personal, (3) healthcare system-related, (4) patient-related, and (5) determinants related to the pharmaceutical company or brand. Each of these main categories was subdivided into specific determinants. An overview of all elements determining prescribing decisions with a brief explanation is provided in Table  1 . A detailed overview of the different determinants of prescribing choices identified in each article can be found in the Supplementary Material , along with a summary of the key characteristics of each study (i.e., title, author, year, location, study objectives, design, and sample size).

Demographics of participants

Nineteen physicians participated in the focus group discussions. Most participants were working in Flanders ( n  = 11), while the rest worked either in Wallonia ( n  = 6) or Brussels ( n  = 2). They included rheumatologists ( n  = 7), gastroenterologists ( n  = 6), dermatologists ( n  = 3), and endocrinologists ( n  = 3). All age groups were represented in this study, except for the youngest group of 18 to 29-year-olds. This was expected given the age at which physician specialists graduate in Belgium. Most of the physicians were working in an academic hospital setting ( n  = 10). A summary of the demographic characteristics of all participants can be found in Table  2 .

The results of the focus group discussions are presented below for each of the 17 determinants identified from the scoping literature review. An overview of all these elements, along with their average ranking after the second round, is summarized in Table 3 .

Product-related

Clinical data.

Physicians stated that the need for clinical data regarding biosimilars has evolved over the years. In the past, Belgian physicians had insufficient knowledge about the clinical development of biosimilars, which influenced their prescription practices. Until recently, uncertainty existed among Belgian physicians about the clinical differences when transitioning from an original biological to a biosimilar. These doubts are less prevalent nowadays due to an increase of positive experiences with biosimilars. It was agreed that when choosing between a biosimilar and its originator biological, doubts regarding reduced efficacy or safety are becoming less of an issue in Belgium. According to the participating physicians, the concept of extrapolation of indication does not pose an issue anymore for most Belgian physicians.

However, the situation differs when choosing between products with different mechanisms of action (MOA). In this case, the clinical data were considered important. For example, among the dermatologists, IL-17/23 inhibitors are considered clinically superior to existing off-patent alternatives (i.e., TNF-alpha inhibitors). This is one of the main reasons why existing off-patent biologicals are rarely prescribed for the treatment of new patients with psoriasis. This importance was also reflected in the average ranking score of 3.6 (Cfr. Table 3 ).

Physicians agreed that cost should be an important determinant, given the rising pharmaceutical expenditure in Belgium over the last few years. However, despite the important price differences between off-patent and competing patented products, it was believed that this only mattered for a small number of physicians when choosing a particular biological. The main reason for this would be that Belgian physicians are not sufficiently aware of the cost of medicines. Participants mentioned that there is a general lack of awareness about increasing pharmaceutical costs and the importance of a financially sustainable healthcare system. Moreover, the other determinants discussed in this article were believed to outweigh the impact of price considerations, according to the participating physicians.

According to the participants, cost considerations become an almost irrelevant determinant when choosing between an originator biological and its biosimilar(s) because of the limited price differences in the ambulatory care.

Overall, it was concluded that the cost of biological products has a limited influence on prescribing decisions in Belgium.

Administration

Physicians agreed that the administration modalities play a role for certain therapeutic areas. However, the influence of this depends greatly on the product or on whether different administration routes, devices, or frequencies are available. Off-patent biologicals in the ambulatory setting are always subcutaneously (SC) administered. Hence, the route of administration was considered to be unimportant. However, for certain therapeutic groups, oral (non-biological) alternatives are also available (e.g., JAK inhibitors). The oral administration route was thought to be one of the reasons for the current increase of JAK inhibitors being prescribed in Belgium, especially for patients with rheumatoid arthritis.

For most off-patent SC products, there are products available with different injection devices, frequencies of administration, and injection comfort. Physicians agreed that these differences, if present, play a role in the choice of a particular off-patent biological, although they are mostly related to patient comfort and patient preferences (see patient-related). Physicians concluded that administration modalities are certainly taken into consideration but should not be overestimated when looking at the overall picture.

Notwithstanding the success of certain new alternatives (such as IL-17/23 inhibitors, patented TNF-alpha inhibitors, JAK inhibitors, etc.) compared to off-patent biologicals, the mere fact that these are new products was believed to only play a small part in prescribing choices. The other elements discussed in this study are more likely to be causing the observed shift in prescribing behavior.

Physicians’ personal

Working environment.

Regarding the working environment of prescribers, the main point raised during the focus group discussions was that Belgian physicians working in hospitals or private practices differ in their access to information and certain support services. In academic hospitals, physicians would often have more access to information and are visited more frequently by pharmaceutical representatives. In addition, physicians in academic hospitals often already have experience with new MOA products through research studies, which would lead to a faster prescription uptake of these products. Differences in support services between individual hospitals, however, was considered to be even more influential by the participating physicians. Also, participants expressed that academic hospitals often have access to a larger nursing staff or more hospital beds. As a result, when pharmaceutical companies offer value-added services as a tool to promote their products to prescribers, this would influence their prescribing decisions to a lesser extent than for physicians working in general hospitals. Participants indicated that services provided by the pharmaceutical industry will have a lower impact on physicians working in private practices, which is particularly common among Belgian dermatologists and rheumatologists.

In general, physicians noted the importance of the prescriber’s working environment, but this difference was mostly linked to other determinants, such as pharmaceutical promotion, prescribing habits, and the influence of KOLs.

Several physicians pointed out some demonstrable differences between Belgian regions. For example, in rheumatology the transition from existing off-patent TNF inhibitors to JAK inhibitors is more prominent in Wallonia. However, the participants could not provide a particular reason for this. In addition, it was also indicated that there is a difference in the use of SC-administered biosimilars between Flanders and Wallonia, with higher SC biosimilar prescription rates in Flanders. It was argued that such differences may be due to different pharmaceutical marketing approaches and prescribing habits in both regions.

Prescribing habits

Prescribing habits were described by physicians as one of the most important determinants of prescribing choices. It was confirmed across all specialties that physicians tend to stick with products they have experience with. As a result, originator biologicals are often preferred over biosimilars in Belgium as prescribers have had limited experiences with biosimilars in the ambulatory care to date. An associated fear of loss of clinical efficacy was believed to be the main underlying reason for this therapeutic inertia when making prescribing choices. However, participants mentioned that prescribing habits tend to be more influencial when choosing between reference products and biosimilars compared to when new MOA products are being considered. For the latter, physicians had the impression that the abovementioned therapeutic inertia plays less of a role as moves to new MOAs are common practice in the Belgian ambulatory care.

Participating physicians agreed that the age of the prescribing physician plays little or no role in their prescribing choices. However, it was mentioned several times that older physicians, who have also known a time without the abundance of therapeutic options, are probably more inclined to prescribe the products that have been on the market for a longer period (i.e., off-patent products). Moreover, it was argued that younger physicians will have accumulated less experience with the more mature biologicals and are more likely to transition to new MOAs for this reason.

Healthcare system-related

Preference of key opinion leaders (kol).

The preference of KOLs was regarded as a major determining element when making prescribing choices for off-patent biological medicines. Physicians indicated that when KOLs express a particular preference regarding biological medicines, this generally has an impact on the prescribing choices of their peers. However, the preferences of KOLs were said to be mostly shown for specific MOAs, and not for reference biologicals and their biosimilars. According to the participants, this might have resulted in the shift that has been seen from off-patent biologicals towards new MOA products. KOLs have been less outspoken about their preferences towards biosimilars. Physicians indicated that it is precisely because KOLs give little attention to biosimilars, fewer biosimilars are being prescribed.

Clinical guidelines

Regarding the impact of clinical guidelines as a determinant for choosing a biological, all physicians emphasized that there is a substantial difference between choosing a particular molecule versus an originator biological or biosimilar. Belgian guidelines are generally scarce, and so European guidelines are seen as impactful. Clinical guidelines were thought to be a substantial determinant when choosing between different MOAs and thus between different off-patent (or patented) biologicals. However, it was indicated that clinical guidelines usually do not influence the decision when choosing between an originator biological and its biosimilar. The main reason for this is that both at the Belgian and the European level, this choice is often not explicitly incorporated in the clinical guidelines.

Time pressure

There was agreement among the participating physicians that it takes time to sufficiently explain to patients a transition from an original biological to a biosimilar. It was indicated that the limited length of a consultation does not allow for enough time to provide an explanation of a possible switch to a biosimilar. However, it was emphasized that time constraints are a commonly used excuse not to prescribe biosimilars. Physicians from the various therapeutic areas admitted that time pressure only plays a minor role when choosing between an originator biological or biosimilar medicine. In reality, there are other more predominant elements that determine this choice.

Across the different therapeutic areas, there was an agreement that transitioning between different molecules was regarded as common practice without the concerns of time constraints. Hence, there is no reason to assume that this should be much different when transitioning to a biosimilar. Therefore, even if more explanation is required when transitioning to a biosimilar, this should not be a valid reason to avoid prescribing biosimilars.

Financial incentives

During the group discussions, it emerged that financial incentives can be an important determinant of prescribing choices for off-patent biologicals. It was pointed out that the right incentives to increase the use of biosimilars may have an important impact, although these are currently absent in Belgium. The financial incentive introduced in 2019 to promote biosimilar prescription was cited as an example of the negligible impact of individual financial incentives. An incentive that supports the quality of care could have a much larger impact according to the participants.

Moreover, the impact of an incentive to promote biosimilar prescription would differ between hospitals. As indicated earlier, the amount of support varies from hospital to hospital. Therefore, the potential impact of supporting incentives from the government is partly dependent on this. Physicians working in private practices indicated that financial incentives, whether on an individual basis or at the hospital level, have a minor impact on their prescribing choices.

Patient-related

Patient profile.

Participating physicians agreed that the patient profile plays little or no role in the choice of an originator biological versus its biosimilar. When it comes to choosing between different molecules or MOAs, the patient profile does play a role. Comorbidities, age, lifestyle, socio-professional context, and comedication were mentioned as relevant examples in this context. Physicians indicated that these determinants, together with the administration modality, can largely determine medication adherence. For example, a less frequent administration would be more appropriate for an active or young patient. Participants also indicated that these aspects are often mentioned in clinical guidelines. Therefore, the importance patient profile is also linked to the availability of clinical guidelines.

Patient preference

Physicians indicated that the choice of a biological is only marginally influenced by the patient as most patients have limited preferences. If patients do have a preference, which is mostly for a particular injection device or administration route, it was mentioned that this obviously has a significant impact on the prescribing choice. Physicians confirmed that apart from a small group of patients, most patients generally have no preference regarding a biosimilar or originator biological.

Knowledge or perception of patients

Patient knowledge or perception about a particular type of medicine only has a minor effect on prescribing choices, as patients typically have a limited knowledge about biologicals. Physicians agreed that it may play a role with a small number of patients who have a rather negative perception towards biosimilar medicines. Nonetheless, if strong negative preconceptions about a particular medicine exists, this could have a significant impact on prescribing choices since these negative preconceptions may induce nocebo effects (i.e., the induction or exacerbation of symptoms through negative expectations about the therapy). Endocrinologists indicated that a resistance to taking insulin therapy can be an important barrier for patients with type 2 diabetes, as there is often an underlying perception of failure and shame when being treated with insulin.

Related to the pharmaceutical company or brand

Pharmaceutical marketing and promotion.

The impact of promotion and marketing activities from the pharmaceutical industry was considered by all participating physicians to be very large. It was striking that the focus group participants mentioned that although its impact is undoubtably significant, physicians often deny this impact. The influence of everything related to pharmaceutical marketing was rated as especially high when it comes to prescribing an originator biological or its biosimilar(s). It was postulated that this is because physicians usually know that the originator biological and its biosimilar(s) are clinically equivalent products. Therefore, physicians acknowledged that marketing and promotion become more important determinants in this context, compared to when choosing between different molecules or MOAs where clinical differences might come into play.

Based on the suggestions provided during the focus group discussions, this category was divided into three subcategories: (a) value-added services, (b) visits by pharmaceutical companies, and (c) sponsorship or grants.

Value-added services

When choosing between an original biological or a biosimilar, the services that a company offers to physicians or hospitals were considered the most important determinant within the category of marketing and promotion. Examples cited by participating physicians included patient support programs, therapeutic drug monitoring (TDM), and nursing support. Companies offering such services are preferred when choosing between an original biological and its biosimilar(s). Usually, companies that market the originator biological offer these services already before biosimilars become available. Physicians, therefore, questioned why they would transition to a biosimilar if prescribers and patients would lose this important support. However, it was also indicated that physicians generally do not find this a transparent and correct system, and therefore, government incentives to support biosimilar prescribing remain important. As elaborated earlier, it became clear during the group discussions that the impact of these services also depends in part on the support already in place in the hospital where the prescriber works.

Visits by pharmaceutical companies

Physicians agreed that one should not underestimate the impact of representatives from the pharmaceutical industry. Consciously or unconsciously, prescribers will be influenced by this when choosing an off-patent biological. A good personal contact with the prescriber would certainly have an impact. Physicians pointed out that this was a reason why prescribers do not transition to a biosimilar, but also why they do transition more quickly to another MOA. After all, the alternative MOA products are often marketed by the same company as the originator biological, and thus, by the same pharmaceutical representative. The information provided by representatives during visits was also considered to have a great effect. The physicians indicated that they usually have little time to inform themselves about the latest products, and as a result, they are sometimes limited to the information provided to them by industry representatives.

Sponsorships or grants

Many pharmaceutical companies provide additional sponsorships to prescribers to conduct research with their pharmaceutical product. This was also mentioned as a common practice for biological medicines, both with innovator and off-patent biologicals. Physicians indicated that such sponsorships certainly increase the likelihood of prescribing these products. This may occur because the prescriber would like to give something back to the company. Also, the fact that the physician has already some experience with the product increases the likelihood of future prescriptions.

Reputation of the pharmaceutical company or brand

Participating physicians mentioned that the reputation and trust in the pharmaceutical company is an element that matters when choosing a medicine, but is certainly of less importance than promotional activities. When deciding between an original biological and a biosimilar, certain companies have a less favorable reputation among Belgian physicians than others. Participating physicians appear to prefer companies that also invest in drug research and development, rather than companies that purely produce or market biosimilars. Participants also indicated that prescribers are more likely to prefer biologicals produced by companies that have experience with producing biological products. Both elements were viewed by participants as reasons why biosimilars are less often considered in prescribing decisions. In general, it was assumed that reputation has an impact on prescribing behavior especially when choosing between products with the same MOA, such as between an originator biological and it biosimilar(s).

Key study findings

This study identified five main categories of elements that may determine prescribing choices for off-patent biologicals in Belgium. These five categories were the result of a scoping literature review that looked for determinants of prescribing choices in general: (1) product-related, (2) physicians’ personal, (3) healthcare system-related, (4) patient-related, and (5) determinants related to the pharmaceutical company or brand. Each of these was subdivided into several subcategories, leading to a total of seventeen determinants. In the second phase of this study, focus group discussions using the NGT with Belgian physicians were conducted to validate these determinants for prescribing off-patent biologicals in Belgium. From this, seven elements emerged as predominant in this setting: clinical data, region, working environment, marketing and promotion, patient profile, clinical guidelines, and preference of KOLs.

One of the most discussed topics was the influence of pharmaceutical promotion. Physicians generally agreed during the group discussions that the influence of pharmaceutical promotion is substantial. Several reviews on the influence of pharmaceutical promotion have concluded as well that its impact on prescribing decisions is considerable [ 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 ]. This study confirms these findings also for off-patent biological medicines.

The other much debated element was the impact of the medication cost. Although not strongly reflected in the ranking scores, participating physicians appeared to agree that too little attention is paid to medication cost among Belgian prescribers. This was also demonstrated in a previous study of Belgian prescribing practices where shifts towards more expensive molecules rather than off-patent biologicals were observed [ 13 ].

It is clear that prescription choices are complex and determined by a wide range of elements [ 36 ]. This was illustrated by the considerable number of determinants that were identified from the literature, and the fact that almost all were considered important to some degree by the Belgian participants in the context of off-patent biological prescribing.

Avenues for further research

This study, in addition to the existing knowledge [ 12 , 13 , 14 ], made an important step towards informing Belgian policymakers to create tailored measures for a more sustainable market for off-patent biologicals and biosimilars in Belgium. Nonetheless, in the future, a more thorough and quantitative examination of the determinants of prescribing behavior would be interesting. A quantitative approach with a behavioral model is therefore advisable. This qualitative study could serve as a base for creating such a behavioral model and for thoroughly investigating prescribing behavior in the future.

Opportunities for policymakers

Investments should be made to continuously increase the awareness of physicians about the importance of cost-effective prescribing. In this regard, several opportunities for policymakers were listed in Table  4 based on the identified determinants of prescribing decisions for off-patent biologicals in Belgium.

Benefit-sharing incentives appear to have an influence on prescribing choices, although they will need to be introduced in the form of healthcare support within hospital departments and not as individual financial incentives at the level of the prescriber [ 37 ]. This study made clear that these needs differ between types of hospitals, regions, and therapeutic areas. Therefore, a certain amount of freedom on how hospital departments choose to deploy these incentives within their working environment is appropriate. More restrictive measures such as prescription targets (on biosimilars or on best-value biologicals) are helpful but should not be used as a standalone measure [ 27 , 38 , 39 ]. Invariably, prescribing targets should be accompanied by information, benefit-sharing initiatives, and other measures to empower physicians in their prescribing choices [ 40 ].

In an ideal world, resources are infinite and governments can provide all patients with access to the medicine with the best clinical outcomes or therapeutic value. However, in reality, budgets are limited and therefore the use of cost-effective therapies is essential [ 41 ]. In this way, clinical outcomes are maximized and as many patients as possible can be treated with the limited financial resources available [ 42 , 43 ]. For this reason, clinical guidelines increasingly focus on the importance of cost-effectiveness in medicine selection [ 44 , 45 , 46 , 47 ]. The Belgian reimbursement committee usually adopts these clinical guidelines in their reimbursement modalities. However, in the specific case of JAK inhibitors versus TNF-alpha inhibitors in rheumatology, these have not yet been included in the reimbursement modalities. As a result, physicians are free to choose the various second-line products that are in competition with off-patent biologicals, regardless of the significant differences in treatment costs that have emerged after biosimilar market entry. The Belgian reimbursement committee should, therefore, periodically revise reimbursement modalities for off-patent biologicals as soon as they lose their market exclusivity or when biosimilars enter the market [ 10 , 13 ]. This may result in bringing more cost-effective products into the treatment line sooner or at an earlier disease stage [ 48 ].

As mentioned above, pharmaceutical promotion may be an important determinant of prescribing decisions. Despite the existing regulations (i.e., Article 10 of the Belgian Law on Pharmaceutical products of March 1964 and Directive 2001/83/EC on Medicines for human use) and code of ethics (i.e., Mdeon) at the Belgian and European level [ 49 , 50 , 51 , 52 ], more guidelines on transparency and authorized promotion are desirable. Belgian physicians indicated that there is a need for more detailed transparency regarding what support physicians receive from pharmaceutical companies. This was also previously proposed by Medaxes, the umbrella association for the Belgian generic and biosimilar industry [ 53 ]. Moreover, in particular for value-added services and patient support programs, guidelines are advised on which services are allowed (and which are prohibited).

Given the rapidly evolving off-patent biologics landscape with new biological medicines losing their market exclusivities, new second-generation products, and new therapeutic classes coming to the market, it is important that policymakers act proactively. Policy measures should therefore be tailored to the product-specific context and implemented prior to the changing market environment when biosimilars enter the market.

Study strengths and limitations

There have been no peer-reviewed research articles published that have examined prescribing choices for off-patent biologicals to date. Therefore, as far as the authors know, this study is the first to do so in an academic setting. By starting from elements found in the existing scientific literature and evaluating these further in the context of biological medicines with various Belgian physicians during focus group discussions, a comprehensive picture was obtained. The NGT was appropriate for this purpose as qualitative insights from the group discussions could be combined with a quantitative aspect to estimate the relative impact of each determinant in prescribing decisions.

This study sought to identify which aspects influence prescribing choices through a combination of a literature review and focus group discussions. The scoping literature review aimed to identify general determinants of prescribing choices from the scientific literature. Because of the broad search strategy and the fact that only one independent investigator performed the screening, relevant articles may have been missed (i.e., selection bias). However, the purpose of this scoping review was not to provide a complete overview of all articles describing determinants of prescribing behavior. Instead, the researchers aimed to give an overall picture of the existing literature on this topic, in order to allow for further exploration in a second step during focus group discussions.

A significant part of human decisions happens unconsciously, and people do not consciously make specific trade-offs all the time. This aspect is challenging to map through focus group discussions. In addition, we should also consider possible bias in the results because physicians may be more likely to name elements that should drive prescribing choices rather than those that actually drive their choices. However, this was avoided as much as possible by clearly informing the participating physicians about the research aims.

A disadvantage of the chosen qualitative method is that it is more difficult to draw conclusions about the entire population due to the relatively small sample. This selection bias is inherent to the purposeful sampling method [ 54 ]. Also, the majority of participants were active in academic hospitals. Considering the differences between academic hospitals and private practices or general hospitals, this may also have led to a certain bias in the results. Lastly, a generalization of the obtained conclusions to a wider geographical scope other than Belgium would not be appropriate.

As is inherent in qualitative research, personal biases and opinions may influence the interpretation of the research findings [ 55 ]. However, through close consultation with the researchers involved and a standardized protocol for each step of this study, the probabilities of this were kept to a minimum. In this way, reflexivity was embedded in the research process.

Prescribing is inextricably linked to human choices and behavior, and thus should be understood in this context. Product-, patient-, healthcare system-, personal-, and pharmaceutical marketing-related elements play a role in prescribing decisions, underlining the complexity of prescribing choices. To increase the prescribing of best-value biologicals in the future, policy measures should be tailored to the product-specific context and implemented prior to the changing market environment when biosimilars enter the market.

Availability of data and materials

The data generated or analyzed related to the literature review of this study are included in this published article and its supplementary information files. The datasets generated and/or analyzed during the focus group discussions are not publicly available due to privacy constraints but are available from the corresponding author on reasonable request.

Abbreviations

Nominal Group Technique

Key Opinion Leader

National Institute for Health and Disability Insurance

Interleukin

Janus Kinase

Tumor Necrosis Factor

Mechanism of Action

Subcutaneous

Therapeutic Drug Monitoring

IQVIA. The global use of medicines 2022: outlook to 2026. 2021. https://www.iqvia.com/-/media/iqvia/pdfs/institute-reports/the-global-use-of-medicines-2022/global-use-of-medicines-2022-outlook-to-2026-12-21-forweb.pdf . Accessed 16 Dec 2021.

Google Scholar  

OECD. Health spending (indicator). 2022. https://data.oecd.org/healthres/health-spending.htm#indicator-chart . Accessed 5 May 2022.

OECD. Pharmaceutical spending (indicator). 2022. https://data.oecd.org/healthres/pharmaceutical-spending.htm#indicator-chart . Accessed 3 May 2022.

National Institute for Health and Disability Insurance (NIHDI). Budget 2021 – Begrotingsvoorstel van het Verzekeringscomité. 2020. https://www.riziv.fgov.be/SiteCollectionDocuments/RIZIV_ARGV_2020_065.pdf . Accessed 16 Dec 2020.

IQVIA. The Impact of Biosimilar Competition in Europe. 2020. https://www.iqvia.com/library/white-papers/the-impact-of-biosimilar-competition-in-europe . Accessed 16 Jan 2021.

OECD. Health at a Glance. 2021. https://www.oecd-ilibrary.org/docserver/ae3016b9-en.pdf?expires=1639586364&id=id&accname=guest&checksum=9754877E2D983E7EA02C3901126787D3 . Accessed 15 Dec 2021.

National Institute for Health and Disability Insurance (NIHDI). Budget 2022 – Begrotingsvoorstel van het Verzekeringscomité. 2022. https://www.riziv.fgov.be/SiteCollectionDocuments/voorstel_budget_2022_verzekeringscomite.pdf . Accessed 15 Dec 2021.

European Medicines Agency (EMA). Biosimilars in the EU: Information guide for healthcare professionals. 2017. https://www.ema.europa.eu/en/documents/leaflet/biosimilars-eu-information-guide-healthcare-professionals_en.pdf . Accessed 12 Apr 2020.

Cornes P, Bennett D. Biosimilars. Oxford: Health Press Limited; 2018.

Simoens S, Vulto AG. A health economic guide to market access of biosimilars. Expert Opin Biol Ther. 2021;21:9–17.

Article   PubMed   Google Scholar  

Simoens S, Huys I. Emerging insights into European Markets of Biologics, Including Biosimilars. Pharmaceuticals. 2022;15:615.

Article   PubMed   PubMed Central   Google Scholar  

Moorkens E, Vulto AG, Huys I, Vulto AG. Biosimilars in Belgium : a proposal for a more competitive market. Acta Clin Belg. 2020;76:1–12.

Vandenplas Y, Simoens S, Van Wilder P, Vulto AG, Huys I. Off-patent biological and biosimilar medicines in Belgium: a market landscape analysis. Front Pharmacol. 2021. https://doi.org/10.3389/fphar.2021.644187 .

Dylst P, Vulto A, Simoens S. Barriers to the uptake of biosimilars and possible solutions: a Belgian case study. Pharmacoeconomics. 2014;32:681–91.

HSE Medicines Management Programme. Best-Value Biological Medicines: Tumour Necrosis Factor-α Inhibitors on the High Tech Drug Scheme. 2019. https://www.hse.ie/eng/about/who/cspd/ncps/medicines-management/best-value-medicines/best-value-biological-medicines/mmp report bvb medicines tnf alpha inhibitors may 2019.pdf. Accessed 3 Dec 2021.

National Institute for Health and Disability Insurance (NIHDI). Convenant “Doorstart voor biosimilaire geneesmiddelen in Begië.” 2016. https://www.inami.fgov.be/nl/themas/kost-terugbetaling/door-ziekenfonds/geneesmiddel-gezondheidsproduct/geneesmiddel-voorschrijven/Paginas/biosimilaire-geneesmiddelen.aspx . Accessed 19 Mar 2020.

National Institute for Health and Disability Insurance (NIHDI). Biosimilaire geneesmiddelen: Incentive voor het voorschrijven van biosimilaire geneesmiddelen buiten het ziekenhuis. 2019. https://www.inami.fgov.be/nl/themas/kost-terugbetaling/door-ziekenfonds/geneesmiddel-gezondheidsproduct/geneesmiddel-voorschrijven/Paginas/biosimilaire-geneesmiddelen-buiten-ziekenhuis.aspx#.XUlIX-gzaUk . Accessed 4 Dec 2019.

Matjasko JL, Cawley JH, Baker-Goering MM, Yokum DV. Applying behavioral economics to public health policy: illustrative examples and promising directions. Am J Prev Med. 2016;50:S13–9.

Connelly LB, Birch S. Sustainability of publicly funded health care systems: what does Behavioural economics offer? Pharmacoeconomics. 2020;38:1289–95.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18:143.

Paré G, Trudel M-C, Jaana M, Kitsiou S. Synthesizing information systems knowledge: a typology of literature reviews. Inf Manag. 2015;52:183–99.

Article   Google Scholar  

Qatar computing research institute. Rayyan. 2022. https://www.rayyan.ai/ . Accessed 5 Aug 2022.

Gale NK, Heath G, Cameron E. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;7:117.

McMillan SS, King M, Tully MP. How to use the nominal group and Delphi techniques. Int J Clin Pharm. 2016;38:655–62.

PubMed   PubMed Central   Google Scholar  

Lago PP, Beruvides MG, Jian J-Y, Canto AM, Sandoval A, Taraban R. Structuring group decision making in a web-based environment by using the nominal group technique. Comput Ind Eng. 2007;52:277–95.

Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K, et al. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health. 2016;42:533–44.

Barbier L, Simoens S, Declerck P, Vulto AG, Huys I. Biosimilar use and switching in Belgium: avenues for integrated policy making. Front Pharmacol. 2022. https://doi.org/10.3389/fphar.2022.821616/ .

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.

Hulscher MEJL, Grol RPTM, van der Meer JWM. Antibiotic prescribing in hospitals: a social and behavioural scientific approach. Lancet Infect Dis. 2010;10:167–75.

Teixeira Rodrigues A, Roque F, Falcão A, Figueiras A, Herdeiro MT. Understanding physician antibiotic prescribing behaviour: a systematic review of qualitative studies. Int J Antimicrob Agents. 2013;41:203–12.

Article   CAS   PubMed   Google Scholar  

Rose J, Crosbie M, Stewart A. A qualitative literature review exploring the drivers influencing antibiotic over-prescribing by GPs in primary care and recommendations to reduce unnecessary prescribing. Perspect Public Health. 2021;141:19–27.

Davari M, Khorasani E, Tigabu BM. Factors influencing prescribing decisions of physicians: a review. Ethiop J Health Sci. 2018;28:795–804.

Md Rezal RS, Hassali MA, Alrasheedy AA, Saleem F, Md Yusof FA, Godman B. Physicians’ knowledge, perceptions and behaviour towards antibiotic prescribing: a systematic review of the literature. Expert Rev Anti-Infect Ther. 2015;13:665–80.

Machowska A, Lundborg CS, Stålsby LC. Drivers of irrational use of antibiotics in Europe. Int J Environ Res Public Health. 2019;16:27.

Alowi M, Kani Y. Impact of pharmaceutical companies’ promotional tools on physicians’ prescription patterns: a systematic review. J Appl Pharm. 2018;10.

Lublóy Á. Factors affecting the uptake of new medicines: a systematic literature review. BMC Health Serv Res. 2014;14:469.

Barcina Lacosta T, Vulto AG, Turcu-Stiolica A, Huys I, Simoens S. Qualitative analysis of the design and implementation of benefit-sharing programs for biologics across Europe. BioDrugs. 2022;36:217–29.

Remuzat C, Dorey J, Cristeau O, Ionescu D, Radiere G, Toumi M, et al. Key drivers for market penetration of biosimilars in Europe. J Mark access Heal policy. 2017;5:1272308.

Moorkens E, Jonker-Exler C, Huys I, Declerck P, Simoens S, Vulto AG. Overcoming barriers to the market access of biosimilars in the European union: The case of biosimilar monoclonal antibodies. Front Pharmacol. 2016;7 JUN:1–9.

Barbier L, Simoens S, Vulto AG, Huys I. European stakeholder learnings regarding Biosimilars: part II—improving biosimilar use in clinical practice. BioDrugs. 2020;34:797–808.

Sculpher M. Evaluating the cost-effectiveness of interventions designed to increase the utilization of evidence-based guidelines. Fam Pract. 2000;17 suppl_1:S26–31.

Chalkidou K, Glassman A, Marten R, Vega J, Teerawattananon Y, Tritasavit N, et al. Priority-setting for achieving universal health coverage. Bull World Health Organ. 2016;94:462.

Bilinski A, Neumann P, Cohen J, Thorat T, McDaniel K, Salomon JA. When cost-effective interventions are unaffordable: integrating cost-effectiveness and budget impact in priority setting for global health programs. PLoS Med. 2017;14:e1002397.

Smolen JS, Landewé RBM, Bijlsma JWJ, Burmester GR, Dougados M, Kerschbaumer A, et al. EULAR recommendations for the management of rheumatoid arthritis with synthetic and biological disease-modifying antirheumatic drugs: 2019 update. Ann Rheum Dis. 2020;79:1–15.

Torres J, Bonovas S, Doherty G, Kucharzik T, Gisbert JP, Raine T, et al. ECCO guidelines on therapeutics in Crohn’s disease: medical treatment. J Crohn's Colitis. 2020;14:4–22.

Gossec L, Smolen JS, Ramiro S, De Wit M, Cutolo M, Dougados M, et al. European league against rheumatism (EULAR) recommendations for the management of psoriatic arthritis with pharmacological therapies: 2015 update. Ann Rheum Dis. 2016;75:499–510.

Davies MJ, Alessio DAD, Fradkin J, Kernan WN, Mathieu C, Management of hyperglycaemia in type 2 diabetes. 2018 A consensus report by the American Diabetes Association (ADA) and the European Association for the Study of diabetes (EASD). Diabetologia. 2018;61:2461–98.

Dutta B, Huys I, Vulto AG, Simoens S. Identifying key benefits in European off-patent biologics and biosimilar markets: it is not only about Price! BioDrugs. 2020. https://doi.org/10.1007/s40259-019-00395-w .

European Parliament. Directive 2001/83/EC On the Community Code relating to Medicinal Products for Human Use.

Ministerie van Volksgezondheid en Leefmilieu. 25 maart 1964 - Wet op de geneesmiddelen. 1964.

MDeon. Code voor Deontologie: Deontologisch Gezondheidsplatform 2018. https://www.mdeon.be/wp-content/uploads/2020/10/Code-Mdeon-NL-01.10.2020.pdf . Accessed 8 Aug 2022.

Federal Agency for Medicines and Health Products (FAMHP). Reclame, premies, voordelen, monsters. 2021. https://www.fagg-afmps.be/nl/MENSELIJK_gebruik/geneesmiddelen/geneesmiddelen/goed_gebruik_geneesmiddel/reclame . Accessed 8 Aug 2022.

Medaxes. Medaxes Memorandum 2019. 2019.

Sharma G. Pros and cons of different sampling techniques. Int J Appl Res. 2017;3:749–52.

Guillemin M, Gillam L. Ethics, reflexivity, and “ethically important moments” in research. Qual Inq. 2004;10:261–80.

Download references

Acknowledgements

The authors would like to express their appreciation to the Pharmaceutical Policy Department of NIHDI for their financial support in this research project. In addition, the authors would like to thank all participating physicians who shared their expertise during the focus group discussions.

This manuscript is supported and funded by KU Leuven and the Belgian National Institute for Health and Disability Insurance (NIHDI).

Author information

Authors and affiliations.

Department of Pharmaceutical and Pharmacological Sciences, KU Leuven, Leuven, Belgium, Germany

Yannick Vandenplas, Steven Simoens, Arnold G. Vulto & Isabelle Huys

Ecole de Santé Publique, Université Libre de Bruxelles (ULB), Brussels, Belgium, Germany

Philippe Van Wilder

Hospital Pharmacy, Erasmus University Medical Center, Rotterdam, the Netherlands

Arnold G. Vulto

Unversity of Paderborn, Paderborn, Germany

Florian Turk

You can also search for this author in PubMed   Google Scholar

Contributions

IH, FT, AGV, SS, and YV developed the idea for the study and were involved in its design. YV conducted the data collection and drafted the initial version of the manuscript. IH, FT, AGV, SS, and PVW critically reviewed the manuscript. All authors have read and approved the final manuscript.

Authors’ information

YV works as a PhD researcher in the Department of Clinical Pharmacology and Pharmacotherapy at KU Leuven (Belgium) and is involved in a research project entitled ‘Policy for best-value biological medicines in Belgium’. This research project is supported by the Belgian national health insurer, RIZIV-INAMI.

SS is a professor in Health Economics in the Department of Clinical Pharmacology and Pharmacotherapy at KU Leuven (Belgium).

PVW is a professor in Health Economics at the Ecole de Santé Publique of ULB (Belgium).

AGV is a professor of Hospital Pharmacy & Practical Therapeutics at the department of Clinical Pharmacology and Pharmacotherapy at KU Leuven (Belgium) and Erasmus MC in Rotterdam (the Netherlands).

FT is a professor at the University of Paderborn (Germany), with profound expertise in behavioral sciences and biosimilar policymaking.

IH is a professor in Regulatory Sciences in the Department of Clinical Pharmacology and Pharmacotherapy at KU Leuven (Belgium).

Corresponding author

Correspondence to Yannick Vandenplas .

Ethics declarations

Ethics approval and consent to participate.

The authors confirm that all experiments were performed in accordance with relevant guidelines and regulations (including the Declaration of Helsinki). This study was approved by the Ethics Committee Research of UZ/KU Leuven (S65328). Written informed consent was obtained from all participants prior to the study.

Consent for publication

Not applicable.

Competing interests

This research project is funded by the Belgian National Institute for Health and Disability Insurance (NIHDI). NIHDI was not involved in the design of the study and collection, analysis, interpretation of data, nor in writing the manuscript.

SS, IH and AGV founded the KU Leuven Fund for Market Analysis of Biologics and Biosimilars following Loss of Exclusivity (MABEL). SS was involved in a stakeholder roundtable on biologics and biosimilars sponsored by Amgen, Pfizer, and MSD; he has participated in advisory board meetings for Pfizer, Sandoz, and Amgen; he has contributed to studies on biologics and biosimilars for Hospira (together with AGV and IH), Celltrion, Mundipharma, and Pfizer; and he has had speaking engagements for Amgen, Celltrion, and Sandoz. AGV is involved in consulting, advisory work, and speaking engagements for several companies, including AbbVie, Accord, Amgen, Biogen, EGA, Effik Benelux, Pfizer/Hospira, Mundipharma, Roche, and Sandoz. PVW acted as a healthcare consultant to public and private organizations, including pharmaceutical companies and their professional associations. FT acts as an advisor to, and is a consultant for, several pharmaceutical organizations and has represented pharmaceutical organizations in professional associations.

All other authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Vandenplas, Y., Simoens, S., Van Wilder, P. et al. Determinants of prescribing decisions for off-patent biological medicines in Belgium: a qualitative study. BMC Health Serv Res 22 , 1211 (2022). https://doi.org/10.1186/s12913-022-08591-1

Download citation

Received : 15 June 2022

Accepted : 21 September 2022

Published : 29 September 2022

DOI : https://doi.org/10.1186/s12913-022-08591-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Biological medicine
• Prescription

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]