give two comprehensive case study protocol

• Search Menu
• Author Guidelines
• Submission Site
• Open Access Options
• Self-Archiving Policy
• Reasons to Submit
• About Journal of Surgical Protocols and Research Methodologies
• Editorial Board
• Advertising & Corporate Services
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Introduction, contents of a research study protocol, conflict of interest statement, how to write a research study protocol.

• Article contents
• Figures & tables
• Supplementary Data

Julien Al Shakarchi, How to write a research study protocol, Journal of Surgical Protocols and Research Methodologies , Volume 2022, Issue 1, January 2022, snab008, https://doi.org/10.1093/jsprm/snab008

• Permissions Icon Permissions

A study protocol is an important document that specifies the research plan for a clinical study. Many funders such as the NHS Health Research Authority encourage researchers to publish their study protocols to create a record of the methodology and reduce duplication of research effort. In this paper, we will describe how to write a research study protocol.

A study protocol is an essential part of a research project. It describes the study in detail to allow all members of the team to know and adhere to the steps of the methodology. Most funders, such as the NHS Health Research Authority in the United Kingdom, encourage researchers to publish their study protocols to create a record of the methodology, help with publication of the study and reduce duplication of research effort. In this paper, we will explain how to write a research protocol by describing what should be included.

Introduction

The introduction is vital in setting the need for the planned research and the context of the current evidence. It should be supported by a background to the topic with appropriate references to the literature. A thorough review of the available evidence is expected to document the need for the planned research. This should be followed by a brief description of the study and the target population. A clear explanation for the rationale of the project is also expected to describe the research question and justify the need of the study.

Methods and analysis

A suitable study design and methodology should be chosen to reflect the aims of the research. This section should explain the study design: single centre or multicentre, retrospective or prospective, controlled or uncontrolled, randomised or not, and observational or experimental. Efforts should be made to explain why that particular design has been chosen. The studied population should be clearly defined with inclusion and exclusion criteria. These criteria will define the characteristics of the population the study is proposing to investigate and therefore outline the applicability to the reader. The size of the sample should be calculated with a power calculation if possible.

The protocol should describe the screening process about how, when and where patients will be recruited in the process. In the setting of a multicentre study, each participating unit should adhere to the same recruiting model or the differences should be described in the protocol. Informed consent must be obtained prior to any individual participating in the study. The protocol should fully describe the process of gaining informed consent that should include a patient information sheet and assessment of his or her capacity.

The intervention should be described in sufficient detail to allow an external individual or group to replicate the study. The differences in any changes of routine care should be explained. The primary and secondary outcomes should be clearly defined and an explanation of their clinical relevance is recommended. Data collection methods should be described in detail as well as where the data will be kept secured. Analysis of the data should be explained with clear statistical methods. There should also be plans on how any reported adverse events and other unintended effects of trial interventions or trial conduct will be reported, collected and managed.

Ethics and dissemination

A clear explanation of the risk and benefits to the participants should be included as well as addressing any specific ethical considerations. The protocol should clearly state the approvals the research has gained and the minimum expected would be ethical and local research approvals. For multicentre studies, the protocol should also include a statement of how the protocol is in line with requirements to gain approval to conduct the study at each proposed sites.

It is essential to comment on how personal information about potential and enrolled participants will be collected, shared and maintained in order to protect confidentiality. This part of the protocol should also state who owns the data arising from the study and for how long the data will be stored. It should explain that on completion of the study, the data will be analysed and a final study report will be written. We would advise to explain if there are any plans to notify the participants of the outcome of the study, either by provision of the publication or via another form of communication.

The authorship of any publication should have transparent and fair criteria, which should be described in this section of the protocol. By doing so, it will resolve any issues arising at the publication stage.

Funding statement

It is important to explain who are the sponsors and funders of the study. It should clarify the involvement and potential influence of any party. The sponsor is defined as the institution or organisation assuming overall responsibility for the study. Identification of the study sponsor provides transparency and accountability. The protocol should explicitly outline the roles and responsibilities of any funder(s) in study design, data analysis and interpretation, manuscript writing and dissemination of results. Any competing interests of the investigators should also be stated in this section.

A study protocol is an important document that specifies the research plan for a clinical study. It should be written in detail and researchers should aim to publish their study protocols as it is encouraged by many funders. The spirit 2013 statement provides a useful checklist on what should be included in a research protocol [ 1 ]. In this paper, we have explained a straightforward approach to writing a research study protocol.

None declared.

Chan   A-W , Tetzlaff   JM , Gøtzsche   PC , Altman   DG , Mann   H , Berlin   J , et al.    SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials . BMJ   2013 ; 346 : e7586 .

Google Scholar

• conflict of interest
• national health service (uk)

Email alerts

Citing articles via.

• Advertising and Corporate Services
• Journals Career Network
• JSPRM Twitter

Affiliations

• Online ISSN 2752-616X
• Copyright © 2024 Oxford University Press and JSCR Publishing Ltd
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

To read this content please select one of the options below:

Please note you do not have access to teaching notes, developing a robust case study protocol.

Management Research Review

ISSN : 2040-8269

Article publication date: 4 July 2023

Issue publication date: 11 January 2024

Case study research has been applied across numerous fields and provides an established methodology for exploring and understanding various research contexts. This paper aims to aid in developing methodological rigor by investigating the approaches of establishing validity and reliability.

Design/methodology/approach

Based on a systematic review of relevant literature, this paper catalogs the use of validity and reliability measures within academic publications between 2008 and 2018. The review analyzes case study research across 15 peer-reviewed journals (total of 1,372 articles) and highlights the application of validity and reliability measures.

The evidence of the systematic literature review suggests that validity measures appear well established and widely reported within case study–based research articles. However, measures and test procedures related to research reliability appear underrepresented within analyzed articles.

Originality/value

As shown by the presented results, there is a need for more significant reporting of the procedures used related to research reliability. Toward this, the features of a robust case study protocol are defined and discussed.

• Case study research
• Systematic literature review

Burnard, K.J. (2024), "Developing a robust case study protocol", Management Research Review , Vol. 47 No. 2, pp. 204-225. https://doi.org/10.1108/MRR-11-2021-0821

Emerald Publishing Limited

Copyright © 2023, Emerald Publishing Limited

Related articles

We’re listening — tell us what you think, something didn’t work….

Report bugs here

All feedback is valuable

Please share your general feedback

Join us on our journey

Platform update page.

Visit emeraldpublishing.com/platformupdate to discover the latest news and updates

Questions & More Information

Answers to the most commonly asked questions here

• Open access
• Published: 16 February 2022

The “case” for case studies: why we need high-quality examples of global implementation research

• Blythe Beecroft   ORCID: orcid.org/0000-0002-6254-421X 1 ,
• Rachel Sturke 1 ,
• Gila Neta 2 &
• Rohit Ramaswamy 3  

Implementation Science Communications volume  3 , Article number:  15 ( 2022 ) Cite this article

3561 Accesses

5 Citations

20 Altmetric

Metrics details

Rigorous and systematic documented examples of implementation research in global contexts can be a valuable resource and help build research capacity. In the context of low- and middle-income countries (LMICs), there is a need for practical examples of how to conduct implementation studies. To address this gap, Fogarty’s Center for Global Health Studies in collaboration with the Cincinnati Children's Hospital Medical Center and the National Cancer Institute is commissioning a collection of implementation science case studies in LMICs that describe key components of conducting implementation research, including how to select, adapt, and apply implementation science models, theories, and frameworks to these settings; develop and test implementation strategies; and evaluate implementation processes and outcomes. The case studies describe implementation research in various disease areas in LMICs around the world. This commentary highlights the value of case study methods commonly used in law and business schools as a source of “thick” (i.e., context-rich) description and a teaching tool for global implementation researchers. It addresses the independent merit of case studies as an evaluation approach for disseminating high-quality research in a format that is useful to a broad range of stakeholders. This commentary finally describes an approach for developing high-quality case studies of global implementation research, in order to be of value to a broad audience of researchers and practitioners.

Peer Review reports

Contributions to the literature

Reinforcing the need for “thick” (i.e., context-rich) descriptions of implementation studies

Highlighting the utility of case studies as a dissemination strategy for researchers, practitioners, and policymakers

Articulating the value of detailed case studies as a teaching tool for global implementation researchers

Describing a method for developing high-quality case studies of global implementation research

Research capacity for implementation science remains limited in low- and middle-income countries (LMICs). Various stakeholders, including NIH-funded implementation researchers and practitioners, often inquire about how to apply implementation science methods and have requested additional resources and training to support implementation capacity building. This is in part due to a dearth of practical examples for both researchers and practitioners of how to select, adapt, and apply implementation science models, theories, and frameworks to these settings; how to evaluate implementation processes and outcomes; and how to develop and test implementation strategies. The need for detailed documentation of implementation research in all settings has been well established, and guidelines for documentation of implementation research studies have been created [ 1 , 2 ]. But the mere availability of checklists and guidelines in and of themselves does not result in comprehensive documentation that is useful for learning, as has been pointed out by many systematic reviews of implementation science and quality improvement studies ([ 3 , 4 ]). It has also been observed that documentation alone is not enough, and there is a need for mentors to translate abstract theories into context-appropriate research designs and practice approaches [ 5 ]. Because of the especially acute shortage of mentors and coaches in LMIC settings, we propose that documentation with “thick” descriptions that go beyond checklists and guidelines are needed to make the field more useful to emerging professionals [ 6 ]. We suggest that the case study method intended to “explore the space between the world of theory and the experience of practice” [ 7 ] that has been used successfully for over a century by law and business schools as a teaching aid can be of value to develop detailed narratives of implementation research projects. In this definition, we are not referring to the case study as a qualitative research method [ 8 ], but as a rich and detailed method of retrospective documentation to aid teaching, practice, and research. In this context, our case studies are akin to “single-institution or single-patient descriptions” [ 9 ] called “case reports” or “case examples” in other fields. As these terms are rarely used in global health, we have used the words “case studies” in this paper but reiterate that they do not refer to case study research designs.

Fogarty’s Center for Global Health Studies (CGHS) in collaboration with the Cincinnati Children's Hospital Medical Center and the National Cancer Institute (NCI) is commissioning a collection of implementation science case studies that describe implementation research focusing on various disease areas in different (LMIC) contexts around the world. These case study descriptions will provide guidance on the process of conducting implementation science studies and will highlight the impact these studies have had on practice and policy in global health contexts. This brief note makes a case for using case studies to document and disseminate implementation research, describes the CGHS approach to case study development and poses evaluation questions that need to be answered to better understand the utility of case studies. This effort is intended to develop a set of useful examples for LMIC researchers, practitioners, and policymakers, but also to assess and improve the use of case studies as a tested documentation methodology in implementation research.

The “case” for case studies

A preliminary landscape analysis of the field conducted by CGHS found that there are not many descriptions of global implementation science projects in a case study format in the peer-reviewed or gray literature, and those that exist are embedded in the content of academic teaching materials. There is not a cohesive collection, especially relating to health, that illustrates how implementation research has been conducted in varied organizations, countries, or disease areas. This new collection will add value in three different ways: as a dissemination strategy, as a tool for capacity building, and as a vehicle for stimulating better research.

Case studies as a dissemination strategy

Case studies have independent merit as an evaluation approach for disseminating high-quality research in a format that is useful to a broad range of stakeholders. The Medical Research Council (MRC) has recommended process evaluation as a useful approach to examine complex implementation, mechanisms of impact, and context [ 10 ]. Guidelines on documentation of implementation recommend that researchers should provide “detailed descriptions of interventions (and implementation strategies) in published papers, clarify assumed change processes and design principles, provide access to manuals and protocols that provide information about the clinical interventions or implementation strategies, and give detailed descriptions of active control conditions” [ 1 ]. Case studies can be thought of as a form of post hoc process evaluation, to disseminate how the delivery of an intervention is achieved, the mechanisms by which implementation strategies produce change, or how context impacts implementation and related outcomes.

Case studies as a capacity building tool

In addition, case studies can address the universal recognition of the need for more capacity building in Implementation Science , especially in LMIC settings. Case studies have been shown to address common pedagogical challenges in helping students learn by allowing students to dissect and explore limitations, adaptations, and utilization of theories, thereby creating a bridge between theories presented in a classroom and their application in the field [ 11 ]. A recent learning needs assessment for implementation researchers, practitioners, and policymakers in LMICs conducted by Turner et al. [ 12 ] reflected a universal consensus on the need for context-specific knowledge about how to apply implementation science in practice, delivered in an interactive format supported by mentorship. A collection of case studies is a valuable and scalable resource to meet this need.

Using case studies to strengthen implementation research

Descriptions of research using studies can illustrate not just whether implementation research had an impact on practice and policy, but how, why, under what circumstances, and to whom, which is the ultimate goal of generating generalizable knowledge about how to implement. Using diverse cases to demonstrate how a variety of research designs have been used to answer complex implementation questions provides researchers with a palette of design options and examples of their use. A framework developed by Minary et al. [ 13 ] illustrates the wide variety of research designs that are useful for complex interventions, depending on whether the emphasis is on internal and external validity or whether knowledge about content and process or about outcomes is more important. A collection of case studies would be invaluable to researchers seeking to develop appropriate designs for their work. In addition, the detailed documentation provided through these case descriptions will hopefully motivate researchers to document their own studies better using the guidelines described earlier.

Developing and testing the case study creation process: the CGHS approach

Writing case studies that satisfy the objectives described above is an implementation science undertaking in itself that requires the engagement of a variety of stakeholders and planned implementation strategies. The CGHS team responsible for commissioning the case studies began this process in 2017 and followed the approach detailed below to test the process of case study development.

Conducted 25+ consultations with various implementation science experts on gaps in the field and the relevance of global case studies

Convened a 15-member Steering Committee Footnote 1 of implementation scientists with diverse expertise, from various academic institutions and NIH institutes to serve as technical experts and to help guide the development and execution of the project

Developed a case study protocol in partnership with the Steering Committee to guide the inclusion of key elements in the case studies

Commissioned two pilot cases Footnote 2 to assess the feasibility and utility of the case study protocol and elicited feedback on the writing experience and how it could be improved as the collection expands

Led an iterative pilot writing process where each case study writing team developed several drafts, which were reviewed by CGHS staff and a designated member of the Steering Committee

Truncated and adjusted the protocol in response to input from the pilot case study authors teams

Developed a comprehensive grid with the Steering Committee, outlining the key dimensions of implementation science that are significant and would be important areas of focus for future case studies. The grid will be used to evaluate potential case applicants and is intended to help foster diversity of focus and content, in addition to geography

Implementing the process: the call for case studies

In March of 2021, CGHS issued a closed call for case studies to solicit applications from a pool of researchers. Potential applicants completed the comprehensive grid in addition to a case study proposal. Applicants will go through a three-tier screening and review process. CGHS will initially screen the applications for completeness to ensure all required elements are present. Each case study application will then be reviewed by two Steering Committee members for content and scientific rigor and given a numerical score based on the selection criteria. Finally, the CGHS team will screen the applications to ensure diversity of implementation elements, geography, and disease area. Approximately 10 case studies will be selected for development in an iterative process. Each case team will present their case drafts to the Steering Committee, which will collectively workshop the drafts in multiple sittings, drawing on the committee’s implementation science expertise. Once case study manuscripts are accepted by the Steering Committee, they will be submitted to Implementation Science Communications for independent review by the journal. CGHS intends for the case studies to be published collectively, but on a rolling basis as they are accepted for publication.

Future research: evaluating the effectiveness of the case study approach

This commentary has put forth arguments for the potential value of case studies for documenting implementation research for researchers, practitioners, and policymakers. Case studies not only provide a way to underscore how implementation science can advance practice and policy in LMICs, but also offer guidance on how to conduct implementation research tailored to global contexts. However, there is little empirical evidence about the validity of these arguments. The creation of this body of case studies will allow us to study why, how, how often, and by whom these case studies are used. This is a valuable opportunity to learn and use that information to better inform future use of this approach as a capacity-building or dissemination strategy.

Conclusions

Similar to their use in law and business, case study descriptions of implementation research could be an important mechanism to counteract the paucity of training programs and mentors to meet the demands of global health researchers. If the evaluation results indicate that the case study creation process produces useful products that enhance learning to improve future implementation research, a mechanism needs to be put in place to create more case studies than the small set that can be generated through this initiative. There will be a need to create a set of documentation guidelines that complement those that currently exist and a mechanism to solicit, review, publish, and disseminate case studies from a wide variety of researchers and practitioners. Journals such as Implementation Science or Implementation Science Communications can facilitate this effort by either creating a new article type or by considering a new journal with a focus on rigorous and systematic case study descriptions of implementation research and practice. An example that could serve as a guide is BMJ Open Quality , which is a peer-reviewed, open-access journal focused on healthcare improvement. In addition to original research and systematic reviews, the journal publishes two article types: Quality Improvement Report and Quality Education Report to document healthcare quality improvement programs and training. The journal offers resources for authors to document their work rigorously. Recently, a new journal titled BMJ Open Quality South Asia has been released to disseminate regional research. We hope that our efforts in sponsoring and publishing these cases, and in setting up a process to support their creation, will make an important contribution to the field and become a mechanism for sharing knowledge that accelerates the growth of implementation science in LMIC settings.

Availability of data and materials

Not applicable.

Rohit Ramaswamy, CCHMC, Gila Neta, NCI NIH, Theresa Betancourt, BC, Ross Brownson, WASU, David Chambers, NCI NIH, Sharon Straus, University of Toronto, Greg Aarons, UCSD, Bryan Weiner, UW, Sonia Lee, NICHD NIH, Andrea Horvath Marques, NIMH NIH, Susannah Allison, NIMH NIH, Suzy Pollard, NIMH NIH, Chris Gordon, NIMH NIH, Kenny Sherr, UW, Usman Hamdani, HDR Foundation Pakistan, Linda Kupfer, FIC NIH

The first pilot case was led by the Human Development Research Foundation (HDRF) in Pakistan and examines scaling up evidenced-based care for children with developmental disorders in rural Pakistan. The second pilot was led by Boston College and investigates alternate delivery platforms and implementation models for bringing evidence-based behavioral Interventions to scale for youth facing adversity in Sierra Leone to close the mental health treatment gap.

Abbreviations

Low- and middle-income countries

Center for Global Health Studies

National Cancer Institute

Medical Research Council

Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implementation Sci. 2013;8(139) [cited 2021 May];. Available from: https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-8-139 .

Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. for the StaRI group. Standards for Reporting Implementation Studies (StaRI) statement. BMJ. 2017;356:i6795 [cited 2021Sept]. Available from: https://www.bmj.com/content/356/bmj.i6795 .

Article   Google Scholar  

Brouwers MC, De Vito C, Bahirathan L, Carol A, Carroll JC, Cotterchio M, et al. What implementation interventions increase cancer screening rates? A systematic review. Implementation Sci. 2011; [cited 2021 Sept];6:111. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3197548/ .

Nadeem E, Gleacher A, Beidas RS. Consultation as an implementation strategy for evidence-based practices across multiple contexts: unpacking the black box. Adm Policy Ment Health. 2013;40(6):439–50. [cited 2021 Sept]. Available from: https://link.springer.com/article/10.1007%2Fs10488-013-0502-8 .

Darnell D, Dorsey CN, Melvin A, Chi J, Lyon AR, Lewis CC. A content analysis of dissemination and implementation science resource initiatives: what types of resources do they offer to advance the field? Implement Sci. 2017;12(1):137 [cited 2021 Sept];. Available from: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-017-0673-x .

Geertz C. Thick description: toward an interpretive theory of culture In: The interpretation of cultures: selected essays. New York: Basic Books; 1973. p. 3–32.

Breslin M, Buchanan R. On the case study method of research and teaching in design. Design Issues. 2008;24(1):36–40 [cited 2021 Sept]. Available from: http://www.jstor.org/stable/25224148 .

Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Med Res Methodol. 2011;11:100. [cited 2021 Sept]. Available from. https://doi.org/10.1186/1471-2288-11-100 .

Article   PubMed   PubMed Central   Google Scholar  

Alpi KM, Evans JJ. Distinguishing case study as a research method from case reports as a publication type. J Med Libr Assoc. 2019;107(1):1–5 Jan [cited 2021 Sept]. Available from: https://jmla.pitt.edu/ojs/jmla/article/view/615 .

Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258. March [cited 2021 May]. Available from: https://www.bmj.com/content/350/bmj.h1258 . https://doi.org/10.1136/bmj.h1258 .

Velenchik AD. The case method as a strategy for teaching policy analysis to undergraduates. J. Econ. Educ. 1995;26(1):29–38.

Turner MW, Bogdewic S, Agha E, Blanchard C, Sturke R, Pettifor A, et al. Learning needs assessment for multi-stakeholder implementation science training in LMIC settings: findings and recommendations. Implement Sci Commun. 2021;2(134) [cited 2021 May]. Available from: https://implementationsciencecomms.biomedcentral.com/articles/10.1186/s43058-021-00238-2 .

Minary L, Trompette J, Kivits J, Cambon L, Tarquinio C, Alla F. Which design to evaluate complex interventions? Toward a methodological framework through a systematic review. BMC Med Res Methodol. 2019;19(92) [cited 2021 May]. Available from: https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-019-0736-6 .

Download references

Acknowledgements

The findings and conclusions in this manuscript are those of the authors and do not necessarily represent any official position or policy of the US National Institutes of Health or the US Department of Health and Human Services or any other institutions with which authors are affiliated.

Author information

Authors and affiliations.

Fogarty International Center, US National Institutes of Health, Bethesda, USA

Blythe Beecroft & Rachel Sturke

National Cancer Institute, US National Institutes of Health, Bethesda, USA

Cincinnati Children’s Hospital Medical Center, Cincinnati, USA

Rohit Ramaswamy

You can also search for this author in PubMed   Google Scholar

Contributions

BB, RS, and GN contributed to the conceptualization of the manuscript with leadership from RR. BB and RS drafted the main text. RR and GN reviewed and contributed additional content to further develop the text. All authors have read and agreed to the contents of the final draft of the manuscript.

Corresponding author

Correspondence to Blythe Beecroft .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Beecroft, B., Sturke, R., Neta, G. et al. The “case” for case studies: why we need high-quality examples of global implementation research. Implement Sci Commun 3 , 15 (2022). https://doi.org/10.1186/s43058-021-00227-5

Download citation

Received : 05 August 2021

Accepted : 27 September 2021

Published : 16 February 2022

DOI : https://doi.org/10.1186/s43058-021-00227-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Case studies
• Global implementation science

Implementation Science Communications

ISSN: 2662-2211

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Basic Methods Handbook for Clinical Orthopaedic Research pp 65–73 Cite as

How to Write a Study Protocol

• Lukas B. Moser 8 , 9 &
• Michael T. Hirschmann 8 , 9  
• First Online: 02 February 2019

2366 Accesses

This chapter aims to provide a guide for young trainees writing their first study protocol. It includes important aspects junior researchers should consider before getting started and preparing their first study protocol. After having read the chapter, the reader should have a good idea about what a study protocol is about and be able to answer the question why, when, and how a study protocol should be written. Finally, the reader will be prepared to master the very first step of conducting a successful study—writing a brief, concise, but comprehensive study protocol.

Study protocol examples of typical clinical scenarios further illustrate the approach to this mandatory and important part of a research project.

• Scientific writing
• Instruction
• Study protocol

This is a preview of subscription content, log in via an institution .

Buying options

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Al-Jundi A, Sakka S. Protocol writing in clinical research. J Clin Diagn Res. 2016;10:ZE10–3.

PubMed   PubMed Central   Google Scholar  

Babyak MA. What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models. Psychosom Med. 2004;66:411–21.

PubMed   Google Scholar  

Bando K, Sato T. Did you write a protocol before starting your project? Gen Thorac Cardiovasc Surg. 2015;63:71–7.

Article   Google Scholar  

Barletta JF. Conducting a successful residency research project. Am J Pharm Educ. 2008;72:92.

Brody BA, McCullough LB, Sharp RR. Consensus and controversy in clinical research ethics. JAMA. 2005;294:1411–4.

Article   CAS   Google Scholar  

Doran G. There’s a S.M.A.R.T. way to write management’s goals and objectives. Manag Rev. 1981;70:35–6.

Google Scholar  

Doran PC. Understanding dentures: instructions to patients. J Colo Dent Assoc. 1981;59:4–8.

CAS   PubMed   Google Scholar  

Fleming TR, DeMets DL. Surrogate end points in clinical trials: are we being misled? Ann Intern Med. 1996;125:605–13.

Fronteira I. How to design a (good) epidemiological observational study: epidemiological research protocol at a glance. Acta Medica Port. 2013;26:731–6.

Ghooi RB, Divekar D. Insurance in clinical research. Perspect Clin Res. 2014;5:145–50.

Gilmore SJ. Evaluating statistics in clinical trials: making the unintelligible intelligible. Australas J Dermatol. 2008;49:177–84; quiz 185–176.

Huang Y, Gilbert PB. Comparing biomarkers as principal surrogate endpoints. Biometrics. 2011;67:1442–51.

Kanji S. Turning your research idea into a proposal worth funding. Can J Hosp Pharm. 2015;68:458–64.

O’Brien K, Wright J. How to write a protocol. J Orthod. 2002;29:58–61.

Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. ACP J Club. 1995;123:A12–3.

Rosenthal R, Schafer J, Briel M, Bucher HC, Oertli D, Dell-Kuster S. How to write a surgical clinical research protocol: literature review and practical guide. Am J Surg. 2014;207:299–312.

Sade RM, Akins CW, Weisel RD. Managing conflicts of interest. J Thorac Cardiovasc Surg. 2015;149:971–2.

Skelly AC, Dettori JR, Brodt ED. Assessing bias: the importance of considering confounding. Evid Based Spine Care J. 2012;3:9–12.

Van Spall HG, Toren A, Kiss A, Fowler RA. Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. JAMA. 2007;297:1233–40.

Vandenbroucke JP, von Elm E, Altman DG, Gotzsche PC, Mulrow CD, Pocock SJ, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. PLoS Med. 2007;4:e297.

Vavken P, Culen G, Dorotka R. Management of confounding in controlled orthopaedic trials: a cross-sectional study. Clin Orthop Relat Res. 2008;466:985–9.

Warren MD. Aide-memoire for preparing a protocol. Br Med J. 1978;1:1195–6.

Download references

Author information

Authors and affiliations.

Department of Orthopaedic Surgery and Traumatology, Kantonsspital Baselland (Bruderholz, Liestal, Laufen), Bruderholz, Switzerland

Lukas B. Moser & Michael T. Hirschmann

University of Basel, Basel, Switzerland

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Michael T. Hirschmann .

Editor information

Editors and affiliations.

UPMC Rooney Sports Complex, University of Pittsburgh, Pittsburgh, PA, USA

Volker Musahl

Department of Orthopaedics, Sahlgrenska Academy, Gothenburg University, Sahlgrenska University Hospital, Gothenburg, Sweden

Jón Karlsson

Department of Orthopaedic Surgery and Traumatology, Kantonsspital Baselland (Bruderholz, Laufen und Liestal), Bruderholz, Switzerland

Michael T. Hirschmann

McMaster University, Hamilton, ON, Canada

Olufemi R. Ayeni

Hospital for Special Surgery, New York, NY, USA

Robert G. Marx

Department of Orthopaedic Surgery, NorthShore University HealthSystem, Evanston, IL, USA

Jason L. Koh

Institute for Medical Science in Sports, Osaka Health Science University, Osaka, Japan

Norimasa Nakamura

Rights and permissions

Reprints and permissions

Copyright information

© 2019 ISAKOS

About this chapter

Cite this chapter.

Moser, L.B., Hirschmann, M.T. (2019). How to Write a Study Protocol. In: Musahl, V., et al. Basic Methods Handbook for Clinical Orthopaedic Research. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-58254-1_8

Download citation

DOI : https://doi.org/10.1007/978-3-662-58254-1_8

Published : 02 February 2019

Publisher Name : Springer, Berlin, Heidelberg

Print ISBN : 978-3-662-58253-4

Online ISBN : 978-3-662-58254-1

eBook Packages : Medicine Medicine (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is a Case Study? | Definition, Examples & Methods

What Is a Case Study? | Definition, Examples & Methods

Published on May 8, 2019 by Shona McCombes . Revised on November 20, 2023.

A case study is a detailed study of a specific subject, such as a person, group, place, event, organization, or phenomenon. Case studies are commonly used in social, educational, clinical, and business research.

A case study research design usually involves qualitative methods , but quantitative methods are sometimes also used. Case studies are good for describing , comparing, evaluating and understanding different aspects of a research problem .

Table of contents

When to do a case study, step 1: select a case, step 2: build a theoretical framework, step 3: collect your data, step 4: describe and analyze the case, other interesting articles.

A case study is an appropriate research design when you want to gain concrete, contextual, in-depth knowledge about a specific real-world subject. It allows you to explore the key characteristics, meanings, and implications of the case.

Case studies are often a good choice in a thesis or dissertation . They keep your project focused and manageable when you don’t have the time or resources to do large-scale research.

You might use just one complex case study where you explore a single subject in depth, or conduct multiple case studies to compare and illuminate different aspects of your research problem.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Once you have developed your problem statement and research questions , you should be ready to choose the specific case that you want to focus on. A good case study should have the potential to:

• Provide new or unexpected insights into the subject
• Challenge or complicate existing assumptions and theories
• Propose practical courses of action to resolve a problem
• Open up new directions for future research

TipIf your research is more practical in nature and aims to simultaneously investigate an issue as you solve it, consider conducting action research instead.

Unlike quantitative or experimental research , a strong case study does not require a random or representative sample. In fact, case studies often deliberately focus on unusual, neglected, or outlying cases which may shed new light on the research problem.

Example of an outlying case studyIn the 1960s the town of Roseto, Pennsylvania was discovered to have extremely low rates of heart disease compared to the US average. It became an important case study for understanding previously neglected causes of heart disease.

However, you can also choose a more common or representative case to exemplify a particular category, experience or phenomenon.

Example of a representative case studyIn the 1920s, two sociologists used Muncie, Indiana as a case study of a typical American city that supposedly exemplified the changing culture of the US at the time.

While case studies focus more on concrete details than general theories, they should usually have some connection with theory in the field. This way the case study is not just an isolated description, but is integrated into existing knowledge about the topic. It might aim to:

• Exemplify a theory by showing how it explains the case under investigation
• Expand on a theory by uncovering new concepts and ideas that need to be incorporated
• Challenge a theory by exploring an outlier case that doesn’t fit with established assumptions

To ensure that your analysis of the case has a solid academic grounding, you should conduct a literature review of sources related to the topic and develop a theoretical framework . This means identifying key concepts and theories to guide your analysis and interpretation.

There are many different research methods you can use to collect data on your subject. Case studies tend to focus on qualitative data using methods such as interviews , observations , and analysis of primary and secondary sources (e.g., newspaper articles, photographs, official records). Sometimes a case study will also collect quantitative data.

Example of a mixed methods case studyFor a case study of a wind farm development in a rural area, you could collect quantitative data on employment rates and business revenue, collect qualitative data on local people’s perceptions and experiences, and analyze local and national media coverage of the development.

The aim is to gain as thorough an understanding as possible of the case and its context.

Prevent plagiarism. Run a free check.

In writing up the case study, you need to bring together all the relevant aspects to give as complete a picture as possible of the subject.

How you report your findings depends on the type of research you are doing. Some case studies are structured like a standard scientific paper or thesis , with separate sections or chapters for the methods , results and discussion .

Others are written in a more narrative style, aiming to explore the case from various angles and analyze its meanings and implications (for example, by using textual analysis or discourse analysis ).

In all cases, though, make sure to give contextual details about the case, connect it back to the literature and theory, and discuss how it fits into wider patterns or debates.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Ecological validity

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 20). What Is a Case Study? | Definition, Examples & Methods. Scribbr. Retrieved April 6, 2024, from https://www.scribbr.com/methodology/case-study/

Is this article helpful?

Shona McCombes

Other students also liked, primary vs. secondary sources | difference & examples, what is a theoretical framework | guide to organizing, what is action research | definition & examples, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Write an Error-free Research Protocol As Recommended by WHO: 21 Elements You Shouldn’t Miss!

Principal Investigator: Did you draft the research protocol?

Student: Not yet. I have too many questions about it. Why is it important to write a research protocol? Is it similar to research proposal? What should I include in it? How should I structure it? Is there a specific format?

Researchers at an early stage fall short in understanding the purpose and importance of some supplementary documents, let alone how to write them. Let’s better your understanding of writing an acceptance-worthy research protocol.

Table of Contents

What Is Research Protocol?

The research protocol is a document that describes the background, rationale, objective(s), design, methodology, statistical considerations and organization of a clinical trial. It is a document that outlines the clinical research study plan. Furthermore, the research protocol should be designed to provide a satisfactory answer to the research question. The protocol in effect is the cookbook for conducting your study

Why Is Research Protocol Important?

In clinical research, the research protocol is of paramount importance. It forms the basis of a clinical investigation. It ensures the safety of the clinical trial subjects and integrity of the data collected. Serving as a binding document, the research protocol states what you are—and you are not—allowed to study as part of the trial. Furthermore, it is also considered to be the most important document in your application with your Institution’s Review Board (IRB).

It is written with the contributions and inputs from a medical expert, a statistician, pharmacokinetics expert, the clinical research coordinator, and the project manager to ensure all aspects of the study are covered in the final document.

Is Research Protocol Same As Research Proposal?

Often misinterpreted, research protocol is not similar to research proposal. Here are some significant points of difference between a research protocol and a research proposal:

What Are the Elements/Sections of a Research Protocol?

According to Good Clinical Practice guidelines laid by WHO, a research protocol should include the following:

1. General Information

• Protocol title, protocol identifying number (if any), and date.
• Name and address of the funder.
• Name(s) and contact details of the investigator(s) responsible for conducting the research, the research site(s).
• Responsibilities of each investigator.
• Name(s) and address(es) of the clinical laboratory(ies), other medical and/or technical department(s) and/or institutions involved in the research.

2. Rationale & Background Information

• The rationale and background information provides specific reasons for conducting the research in light of pertinent knowledge about the research topic.
• It is a statement that includes the problem that is the basis of the project, the cause of the research problem, and its possible solutions.
• It should be supported with a brief description of the most relevant literatures published on the research topic.

3. Study Objectives

• The study objectives mentioned in the research proposal states what the investigators hope to accomplish. The research is planned based on this section.
• The research proposal objectives should be simple, clear, specific, and stated prior to conducting the research.
• It could be divided into primary and secondary objectives based on their relativity to the research problem and its solution.

4. Study Design

• The study design justifies the scientific integrity and credibility of the research study.
• The study design should include information on the type of study, the research population or the sampling frame, participation criteria (inclusion, exclusion, and withdrawal), and the expected duration of the study.

5. Methodology

• The methodology section is the most critical section of the research protocol.
• It should include detailed information on the interventions to be made, procedures to be used, measurements to be taken, observations to be made, laboratory investigations to be done, etc.
• The methodology should be standardized and clearly defined if multiple sites are engaged in a specified protocol.

6. Safety Considerations

• The safety of participants is a top-tier priority while conducting clinical research .
• Safety aspects of the research should be scrutinized and provided in the research protocol.

7. Follow-up

• The research protocol clearly indicate of what follow up will be provided to the participating subjects.
• It must also include the duration of the follow-up.

8. Data Management and Statistical Analysis

• The research protocol should include information on how the data will be managed, including data handling and coding for computer analysis, monitoring and verification.
• It should clearly outline the statistical methods proposed to be used for the analysis of data.
• For qualitative approaches, specify in detail how the data will be analysed.

9. Quality Assurance

• The research protocol should clearly describe the quality control and quality assurance system.
• These include GCP, follow up by clinical monitors, DSMB, data management, etc.

10. Expected Outcomes of the Study

• This section indicates how the study will contribute to the advancement of current knowledge, how the results will be utilized beyond publications.
• It must mention how the study will affect health care, health systems, or health policies.

11. Dissemination of Results and Publication Policy

• The research protocol should specify not only how the results will be disseminated in the scientific media, but also to the community and/or the participants, the policy makers, etc.
• The publication policy should be clearly discussed as to who will be mentioned as contributors, who will be acknowledged, etc.

12. Duration of the Project

• The protocol should clearly mention the time likely to be taken for completion of each phase of the project.
• Furthermore a detailed timeline for each activity to be undertaken should also be provided.

13. Anticipated Problems

• The investigators may face some difficulties while conducting the clinical research. This section must include all anticipated problems in successfully completing their projects.
• Furthermore, it should also provide possible solutions to deal with these difficulties.

14. Project Management

• This section includes detailed specifications of the role and responsibility of each investigator of the team.
• Everyone involved in the research project must be mentioned here along with the specific duties they have performed in completing the research.
• The research protocol should also describe the ethical considerations relating to the study.
• It should not only be limited to providing ethics approval, but also the issues that are likely to raise ethical concerns.
• Additionally, the ethics section must also describe how the investigator(s) plan to obtain informed consent from the research participants.
• This section should include a detailed commodity-wise and service-wise breakdown of the requested funds.
• It should also include justification of utilization of each listed item.

17. Supplementary Support for the Project

• This section should include information about the received funding and other anticipated funding for the specific project.

18. Collaboration With Other Researchers or Institutions

• Every researcher or institute that has been a part of the research project must be mentioned in detail in this section of the research protocol.

19. Curriculum Vitae of All Investigators

• The CVs of the principal investigator along with all the co-investigators should be attached with the research protocol.
• Ideally, each CV should be limited to one page only, unless a full-length CV is requested.

20. Other Research Activities of Investigators

• A list of all current research projects being conducted by all investigators must be listed here.

21. References

• All relevant references should be mentioned and cited accurately in this section to avoid plagiarism.

How Do You Write a Research Protocol? (Research Protocol Example)

Main Investigator    

Number of Involved Centers (for multi-centric studies)

Indicate the reference center

Title of the Study

Protocol ID (acronym)

Keywords (up to 7 specific keywords)

Study Design

Mono-centric/multi-centric

Perspective/retrospective

Controlled/uncontrolled

Open-label/single-blinded or double-blinded

Randomized/non-randomized

n parallel branches/n overlapped branches

Experimental/observational

Endpoints (main primary and secondary endpoints to be listed)

Expected Results                                                

Analyzed Criteria

Main variables/endpoints of the primary analysis

Main variables/endpoints of the secondary analysis

Safety variables

Health Economy (if applicable)

Visits and Examinations

Therapeutic plan and goals

Visits/controls schedule (also with graphics)

Comparison to treatment products (if applicable)

Dose and dosage for the study duration (if applicable)

Formulation and power of the studied drugs (if applicable)

Method of administration of the studied drugs (if applicable)

Informed Consent

Study Population

Short description of the main inclusion, exclusion, and withdrawal criteria

Sample Size

Estimated Duration of the Study

Safety Advisory

Classification Needed

Requested Funds

Additional Features (based on study objectives)

Click Here to Download the Research Protocol Example/Template

Be prepared to conduct your clinical research by writing a detailed research protocol. It is as easy as mentioned in this article. Follow the aforementioned path and write an impactful research protocol. All the best!

Clear as template! Please, I need your help to shape me an authentic PROTOCOL RESEARCH on this theme: Using the competency-based approach to foster EFL post beginner learners’ writing ability: the case of Benin context. I’m about to start studies for a master degree. Please help! Thanks for your collaboration. God bless.

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

• Publishing Research
• Reporting Research

How to Optimize Your Research Process: A step-by-step guide

For researchers across disciplines, the path to uncovering novel findings and insights is often filled…

• Industry News
• Trending Now

Breaking Barriers: Sony and Nature unveil “Women in Technology Award”

Sony Group Corporation and the prestigious scientific journal Nature have collaborated to launch the inaugural…

Achieving Research Excellence: Checklist for good research practices

Academia is built on the foundation of trustworthy and high-quality research, supported by the pillars…

• Promoting Research

Plain Language Summary — Communicating your research to bridge the academic-lay gap

Science can be complex, but does that mean it should not be accessible to the…

Science under Surveillance: Journals adopt advanced AI to uncover image manipulation

Journals are increasingly turning to cutting-edge AI tools to uncover deceitful images published in manuscripts.…

Choosing the Right Analytical Approach: Thematic analysis vs. content analysis for…

Research Recommendations – Guiding policy-makers for evidence-based decision making

Demystifying the Role of Confounding Variables in Research

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

What should universities' stance be on AI tools in research and academic writing?

• [email protected]
• Connecting and sharing with us

• Entrepreneurship
• Growth of firm
• Sales Management
• Retail Management
• Import – Export
• International Business
• Project Management
• Production Management
• Quality Management
• Logistics Management
• Supply Chain Management
• Human Resource Management
• Organizational Culture
• Information System Management
• Corporate Finance
• Stock Market
• Office Management
• Theory of the Firm
• Management Science
• Microeconomics
• Research Process
• Experimental Research
• Research Philosophy
• Management Research
• Writing a thesis
• Writing a paper
• Literature Review
• Action Research
• Qualitative Content Analysis
• Observation
• Phenomenology
• Statistics and Econometrics
• Questionnaire Survey
• Quantitative Content Analysis
• Meta Analysis

The Case Study Protocol

A case study protocol has only one thing in common with a survey question­naire: Both are directed at a single data point—either a single case (even if the case is part of a larger, multiple-case study) or a single respondent.

Beyond this similarity are major differences. The protocol is more than a questionnaire or instrument. First, the protocol contains the instrument but also contains the procedures and general rales to be followed in using the pro­tocol. Second, the protocol is directed at an entirely different party than that of a survey questionnaire, explained below. Third, having a case study protocol is desirable under all circumstances, but it is essential if you are doing a mul­tiple-case study.

The protocol is a major way of increasing the reliability of case study research and is intended to guide the investigator in carrying out the data col­lection from a single case (again, even if the single case is one of several in a multiple-case study). Figure 3.2 gives a table of contents from an illustrative protocol, which was used in a study of innovative law enforcement practices supported by federal funds. The practices had been defined earlier through a careful screening process (see later discussion in this chapter for more detail on “screening case study nominations”). Furthermore, because data were to be collected from 18 such cases as part of a multiple-case study, the information about any given case could not be collected in great depth, and thus the number of the case study questions was minimal.

As a general matter, a case study protocol should have the following sections:

• an overview of the case study project (project objectives and auspices, case study issues, and relevant readings about the topic being investigated),
• field procedures (presentation of credentials, access to the case study “sites,” language pertaining to the protection of human subjects, sources of data, and procedural reminders),
• case study questions (the specific questions that the case study investigator must keep in mind in collecting data, “table shells” for specific arrays of data, and the potential sources of information for answering each question—see Figure 3.3 for an example), and
• a guide for the case study report (outline, format for the data, use and presenta­tion of other documentation, and bibliographical information).

A quick glance at these topics will indicate why the protocol is so important. First, it keeps you targeted on the topic of the case study. Second, preparing the protocol forces you to anticipate several problems, including the way that the case study reports are to be completed. This means, for instance, that you will have to identify the audience for your case study report even before you have conducted your case study. Such forethought will help to avoid mismatches in the long run.

The table of contents of the illustrative protocol in Figure 3.2 reveals another important feature of the case study report: In this instance, the desired report starts by calling for a description of the innovative practice being studied (see item Cl in Figure 3.2)—and only later covers the agency context and history pertaining to the practice (see item C4). This choice reflects the fact that most investigators write too extensively on history and background conditions. While these are important, the description of the subject of the study—the innovative practice—needs more attention.

Each section of the protocol is discussed next.

1. Overview of the Case Study Project

The overview should cover the background information about the project, the substantive issues being investigated, and the relevant readings about the issues.

As for background information, every project has its own context and perspective. Some projects, for instance, are funded by government agencies having a general mission and clientele that need to be remembered in con­ducting the research. Other projects have broader theoretical concerns or are offshoots, of earlier research studies. Whatever the situation, this type of back­ground information, in summary form, belongs in the overview section.

A procedural element of this background section is a statement about the project which you can present to anyone who may want to know about the pro­ject, its purpose, and the people involved in conducting and sponsoring the project. This statement can even be accompanied by a letter of introduction, to be sent to all major interviewees and organizations that may be the subject of study. (See Figure 3.4 for an illustrative letter.) The bulk of the overview, how­ever, should be devoted to the substantive issues being investigated. This may include the rationale for selecting the case(s), the propositions or hypotheses being examined, and the broader theoretical or policy relevance of the inquiry. For all of these topics, relevant readings should be cited, and the essential read­ing materials should be made available to everyone on the case study team.

A good overview will communicate to the informed reader (that is, some­one familiar with the general topic of inquiry) the case study’s purpose and set­ting. Some of the materials (such as a summary describing the project) may be needed for other purposes anyway, so that writing the overview should be seen as a doubly worthwhile activity. In the same vein, a well-conceived overview even may later form the basis for the background and introduction to the final case study report.

2. Field Procedures

Chapter 1 has previously defined case studies as studies of events within their real-life context. This has important implications for defining and design­ing the case study, which have been discussed in Chapters 1 and 2.

For data collection, however, this characteristic of case studies also raises an important issue, for which properly designed field procedures are essential. You will be collecting data from people and institutions in their everyday situations, not within the controlled confines of a laboratory, the sanctity of a library, or the structured limitations of a survey questionnaire. In a case study, you must there­fore learn to integrate real-world events with the needs of the data collection plan. In this sense, you do not have the control over the data collection environment as others might have in using the other research methods discussed in Chapter 1.

Note that in a laboratory experiment, human “subjects” are solicited to enter into the laboratory—an environment controlled nearly entirely by the research investigator. The subject, within ethical and physical constraints, must follow the investigator’s instructions, which carefully prescribe the desired behavior. Similarly, the human “respondent” to a survey questionnaire cannot deviate from the agenda set by the questions. Therefore, the respondent’s behavior also is con­strained by the ground rules of the investigator. Naturally, the subject or respon­dent who does not wish to follow the prescribed behaviors may freely drop out of the experiment or survey. Finally, in the historical archive, pertinent documents may not always be available, but the investigator can inspect what exists at his or her own pace and at a time convenient to her or his schedule. In all three situa­tions, the research investigator closely controls the formal data collection activity.

Doing case studies involves an entirely different situation. For interviewing key persons, you must cater to the interviewee’s schedule and availability, not your own. The nature of the interview is much more open-ended, and an interviewee may not necessarily cooperate fully in sticking to your line of questions. Similarly, in making observations of real-life activities, you are intruding into the world of the subject being studied rather than the reverse; under these conditions, you are the one who may have to make special arrangements, to be able to act as an observer (or even as a participant- observer). As a result, your behavior—and not that of the subject or respon­dent—is the one likely to be constrained.

This contrasting process of doing data collection leads to the need to have explicit and well-planned field procedures encompassing guidelines for “cop­ing” behaviors. Imagine, for instance, sending a youngster to camp; because you do not know what to expect, the best preparation is to have the resources to be prepared. Case study field procedures should be the same way.

With the preceding orientation in mind, the field procedures of the protocol need to emphasize the major tasks in collecting data, including

• gaining access to key organizations or interviewees;
• having sufficient resources while in the field—including a personal computer, writing instruments, paper, paper clips, and a preestablished, quiet place to write notes privately;
• developing a procedure for calling for assistance and guidance, if needed, from other case study investigators or colleagues;
• making a clear schedule of the data collection activities that are expected to be completed within specified periods of time; and
• providing for unanticipated events, including changes in the availability of interview­ees as well as changes in the mood and motivation of the case study investigator.

These are the types of topics that can be included in the field procedures sec­tion of the protocol. Depending upon the type of study being done, the specific procedures will vary.

The more operational these procedures are, the better. To take but one minor issue as an example, case study data collection frequently results in the accu­mulation of numerous documents at the field site. The burden of carrying such bulky documents can be reduced by two procedures. First, the case study team may have had the foresight to bring large, prelabeled envelopes, to mail the documents back to the office rather than carry them. Second, field time may have been set aside for perusing the documents and then going to a local copier facility and copying only the few relevant pages of each document—and then returning the original documents to the informants at the field site. These and other operational details can enhance the overall quality and efficiency of case study data collection.

A final part of this portion of the protocol should carefully describe the procedures for protecting human subjects. First, the protocol should repeat the rationale for the IRB-approved field procedures. Then, the protocol should include the “scripted” words or instructions for the team to use in obtaining informed consent or otherwise informing case study interviewees and other participants of the risks and conditions associated with the research.

3. Case Study Questions

The heart of the protocol is a set of substantive questions reflecting your actual line of inquiry. Some people may consider this part of the protocol to be the case study “instrument.” However, two characteristics distinguish case study questions from those in a survey instrument. (Refer back to Figure 3.3 for an illustrative question from a study of a school program; the complete protocol included dozens of such questions.)

General orientation of questions. First, the questions are posed to you, the investigator, not to an interviewee. In this sense, the protocol is directed at an entirely different party than a survey instrument. The protocol’s questions, in essence, are your reminders regarding the information that needs to be col­lected, and why. In some instances, the specific questions also may serve as prompts in asking questions during a case study interview. However, the main purpose of the protocol’s questions is to keep the investigator on track as data collection proceeds.

Each question should be accompanied by a list of likely sources of evidence. Such sources may include the names of individual interviewees, documents, or observations. This crosswalk between the questions of interest and the likely sources of evidence is extremely helpful in collecting case study data. Before arriving on the case study scene, for instance, a case study investigator can quickly review the major questions that the data collection should cover.

(Again, these questions form the structure of the inquiry and are not intended as the literal questions to be asked of any given interviewee.)

Levels of questions. Second, the questions in the case study protocol should distinguish clearly among different types or levels of questions. The poten­tially relevant questions can, remarkably, occur at any of five levels:

Level 1: questions asked of specific interviewees;

Level 2: questions asked of the individual case (these are the questions in the case study protocol to be answered by the investigator during a single case, even when the single case is part of a larger, multiple-case study);

Level 3: questions asked of the pattern of findings across multiple cases;

Level 4: questions asked of an entire study—for example, calling on information beyond the case study evidence and including other literature or published data that may have been reviewed; and

Level 5: normative questions about policy recommendations and conclusions, going beyond the narrow scope of the study.

Of these five levels, you should concentrate heavily on Level 2 for the case study protocol.

The difference between Level 1 and Level 2 questions is highly significant. The two types of questions are most commonly confused because investigators think that their questions of inquiry (Level 2) are synonymous with the spe­cific questions they will ask in the field (Level 1). To disentangle these two levels in your own mind, think again about a detective, especially a wily one. The detective has in mind what the course of events in a crime might have been (Level 2), but the actual questions posed to any witness or suspect (Level 1) do not necessarily betray the detective’s thinking. The verbal line of inquiry is different from the mental line of inquiry, and this is the difference between Level 1 and Level 2 questions. For the case study protocol, explicitly articu­lating the Level 2 questions is therefore of much greater importance than any attempt to identify the Level 1 questions.

In the field, keeping in mind the Level 2 questions while simultaneously articulating Level 1 questions in conversing with an interviewee is not easy. In a like manner, you can lose sight of your Level 2 questions when examin­ing a detailed document that will become part of the case study evidence (the common revelation occurs when you ask yourself, “Why am I reading this document?”). To overcome these problems, successful participation in the earlier seminar training helps. Remember that being a “senior” investigator means maintaining a working knowledge of the entire case study inquiry. The (Level 2) questions in the case study protocol embody this inquiry.

The other levels also should be understood clearly. A cross-case question, for instance (Level 3), may be whether the larger school districts among your cases are more responsive than smaller school districts or whether complex bureaucratic structures make the larger districts more cumbersome and less responsive. However, this Level 3 question should not be part of the protocol for collecting data from the single case, because the single case only can address the responsiveness of a single school district. The Level 3 question cannot be addressed until the data from all the single cases (in a multiple-case study) are examined. Thus, only the multiple-case analysis can cover Level 3 questions. Similarly, the questions at Levels 4 and 5 also go well beyond any individual case study, and you should note this limitation if you include such questions in the case study protocol. Remember: The protocol is for the data collection from a single case (even when part of a multiple-case study) and is not intended to serve the entire project.

Undesired confusion between unit of data collection and unit of analysis. Related to the distinction between Level 1 and Level 2 questions, a more sub­tle and serious problem can arise in articulating the questions in the case study protocol. The questions should cater to the unit of analysis of the case study, which may be at a different level from the unit of data collection of the case study. Confusion will occur if, under these circumstances, the data collection process leads to an (undesirable) distortion of the unit of analysis.

The common confusion begins because the data collection sources may be individual people (e.g., interviews with individuals), whereas the unit of analy­sis of your case study may be a collective (e.g., the organization to which the individual belongs)—a frequent design when the case study is about an orga­nization, community, or social group. Even though your data collection may have to rely heavily on information from individual interviewees, your con­clusions cannot be based entirely on interviews as a source of information (you would then have collected information about individuals’ reports about the organization, not necessarily about organizational events as they actually had occurred). In this example, the protocol questions therefore need to be about the organization, not the individual.

However, the reverse situation also can be true. Your case study may be about an individual, but the sources of information can include archival records (e.g., personnel files or student records) from an organization. In this situation, you also would want to avoid basing your conclusions about the individual from the organizational sources of information only. In this example, the protocol ques­tions therefore need to be about the individual, not the organization.

Figure 3.5 illustrates these two situations, where the unit of analysis for the case study is different from the unit of data collection.

Other data collection devices. The protocol questions also can include empty “table shells” (for more detail, see Miles & Huberman, 1994). These are the outlines of a table, defining precisely the “rows” and “columns” of a data array—but in the absence of having the actual data. In this sense, the table shell indicates the data to be collected, and your job is to collect the data called forth by the table. Such table shells help in several ways. First, the table shells force you to identify exactly what data are being sought. Second, they ensure that parallel information will be collected at different sites, where a multiple- case design is being used. Finally, the table shells aid in understanding what will be done with the data once they have been collected.

4. Guide for the Case Study Report

This element is generally missing in most case study plans. Investigators neglect to think about the outline, format, or audience for the case study report until after the data have been collected. Yet, some planning at this preparatory stage—admittedly out of sequence in the typical conduct of most research— means that a tentative outline can (and should) appear in the case study protocol. (Such planning accounts for the arrow between “prepare” and “share” in the figure at the outset of this chapter.)

Again, one reason for the traditional, linear sequence is related to practices with other research methods. One does not worry about the report from an exper­iment until after the experiment has been completed, because the format of the report and its likely audience already have been dictated by the conventional for­mats of academic journals. Most reports of experiments follow a similar outline: the posing of the research questions and hypotheses; a description of the research design, apparatus, and data collection procedures; the presentation of the data collected; the analysis of the data; and a discussion of findings and conclusions.

Unfortunately, case study reports do not have such a uniformly acceptable outline. Nor, in many instances, do case study reports end up in journals (Feagin et al., 1991, pp. 269-273). For this reason, each investigator must be concerned, throughout the conduct of a case study, with the design of the final case study report. The problem is not easy to deal with.

In addition, the protocol also can indicate the extent of documentation for the case study report. Properly done, the data collection is likely to lead to large amounts of documentary evidence, in the form of published reports, publica­tions, memoranda, and other documents collected about the case. What is to be done with this documentation, for later presentation? In most studies, the docu­ments are filed away and seldom retrieved. Yet, this documentation is an impor­tant part of the “database” for a case study (see Chapter 4) and should not be ignored until after the case study has been completed. One possibility is to have the case study report include an annotated bibliography in which each of the available documents is itemized. The annotations would help a reader (or the investigator, at some later date) to know which documents might be relevant for further inquiry.

In summary, to the extent possible, the basic outline of the case study report should be part of the protocol. This will facilitate the collection of relevant data, in the appropriate format, and will reduce the possibility that a return visit to the case study site will be necessary. At the same time, the existence of such an outline should not imply rigid adherence to a predesigned protocol. In fact, case study plans can change as a result of the initial data collection, and you are encouraged to consider these flexibilities—if used properly and with­out bias—to be an advantage of the case study method.

EXERCISE 3.4 Developing a Case Study Protocol

Select some phenomenon in need of explanation from the everyday life of your university or school (past or present). Illustrative topics might be, for example, why the university or school changed some policy or how it makes decisions about its curriculum requirements. For these illustrative topics (or some topics of your own choosing), design a case study protocol to collect the information needed to make an adequate explanation. What would be your main research questions or propositions? What specific sources of data would you seek (e.g., persons to be interviewed, documents to be sought, and field observations to be made)? Would your protocol be sufficient in guiding you through the entire process of doing your case study?

Source: Yin K Robert (2008), Case Study Research Designs and Methods , SAGE Publications, Inc; 4th edition.

13 Aug 2021

23 Oct 2019

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Username or email address  *

Password  *

Log in Remember me

Lost your password?

Alternative Quality Management Systems for Highway Construction (2015)

Chapter: appendix c: case study protocol.

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

APPENDIX C: CASE STUDY PROTOCOL C.1 Overview of Case Study Background Information Delivering highway projects using alternative project delivery methods demands a shift in the traditional agency quality assurance (QA) and quality control (QC) programs to accommodate the faster pace of design and construction as well as the redistribution of responsibilities among project stakeholders. Thus, the objectives of this research are to:  Identify and understand alternative quality management systems  Develop guidelines for their use in highway construction projects Alternative project delivery in highway construction often requires the application of alternative quality management systems that emphasize contractor quality control and quality assurance. These new systems allow owners to have confidence through a verification of contractor quality system processes. They also permit state transportation agencies (STAs) to satisfy due diligence requirements for federal- aid highway projects. For example, the International Organization for Standardization (ISO) 9001 quality management system regulates quality management at all levels from material suppliers through the contractors to owners. It requires a formal project performance evaluation after completion and uses that information to publish contractor performance ratings, which can then be used for future contractor prequalification. The U. S. Army Corps of Engineers’ quality management system relies on detailed guide specifications and rigorous on-site testing by contractors. The Corps has used alternative project delivery on a routine basis for over thirty years on a wide variety of heavy civil projects that include roads and bridges, and as a result, furnishes an excellent analog from which to draw lessons learned and best practices that apply to highway design and construction. Research is needed to provide guidance on the use of alternative quality management systems for highway construction projects using alternative delivery methods. This research needs to address the major quality issues associated with these methods including accelerated project timelines and the change of the designer-of-record’s (DOR) contractual relationships with the owner to permit the required level of integration with the construction contractor. Issues of quality are further complicated by the addition of private funding of public projects in public-private partnerships (PPP). On these projects, an argument can be made that since the concessionaire is at risk for project performance the public agency has few if any reasons to involve itself in the quality management process. From current and past projects, there exists a limited but rapidly expanding body of experience associated with alternate methods of assuring quality. The purpose of this research is to bring together this relatively new body of experience and summarize it in one easily accessible reference treating the subject of QA in alternative projects. The case studies for this research will be used to learn how existing projects have managed quality in light of non-traditional delivery methods. After analyzing and comparing the various case studies, the information gathered will be condensed into working theories and used to modify what the authors of this research are calling the Integrated Quality Management Model (IQ2M) shown in Figure C1. The model was developed to be generic to all forms of project delivery and furnish a foundation for assigning quality management responsibilities between the owner, the designer, and the constructor. As such, it acts as a framework to structure the analysis of other 167

alternative project delivery quality systems that are common in highway construction and will be used in that manner in this research. Figure C1 – Integrated Quality Management Model (IQ2M) (adapted from Synthesis 376 (Gransberg and Molenaar 2008)) Relevant Definitions Across the highway construction and engineering industry, terms relating to quality often have multiple meanings that in some cases overlap with one another and in others supersede each other. To prevent confusion among several vital terms important to this study, the following definitions have been provided. These definitions are in accordance with the most recent issuance of the TRB Circular Glossary of Highway Quality Assurance Terms E-C137 and the NCHRP Synthesis 376. • Quality: (1) The degree of excellence of a product or service. (2) The degree to which a product or service satisfies the needs of a specific customer. (3) The degree to which a product or service conforms to a given requirement. • Quality Assurance (QA): All those planned and systematic actions necessary to provide confidence that a product or facility will perform satisfactorily in service. [QA addresses the overall problem of obtaining the quality of a service, product, or facility in the most efficient, economical, and satisfactory manner possible. Within this broad context, QA involves continued evaluation of the activities of planning, design, development of plans and specifications, advertising and awarding of contracts, construction, and maintenance, and the interactions of these activities.] TRB E-C074. 168

• Quality Control (QC): Also called process control. Those QA actions and considerations necessary to assess and adjust production and construction processes, so as to control the level of quality being produced in the end product. TRB E-C074. • Quality Management (QM): The overarching system of policies and procedures that govern the performance of QA and QC activities. The totality of the effort to ensure quality in design and/or construction. • Design-Bid-Build (DBB): A project delivery method where the design is completed either by in- house professional engineering staff or a design consultant before the construction contract is advertised. Also called the “traditional method.” • Design-Build (DB): A project delivery method where both the design and the construction of the project are simultaneously awarded to a single entity. • Construction Manager-General Contractor (CMGC): A project delivery method where the contractor is selected during the design process and makes input to the design via constructability, cost engineering, and value analysis reviews. Once the design is complete, the same entity builds the projects as the general contractor. CMGC assumes that the contractor will self-perform a significant amount of the construction work. • Construction Manager-at-Risk (CMR): A project delivery method similar to CMGC, but where the CM does not self-perform any of the construction work. • Public Private Partnership (P3): A project delivery method where the agency contracts with a concessionaire organization to design, build, finance and operate an infrastructure facility for a defined extended period of time. • Design deliverable: A product produced by the design-builder’s design team that is submitted for review to the agency (i.e. design packages, construction documents, etc.). • Construction deliverable: A product produced by the design-builder’s construction team that is submitted for review to the agency (shop drawings, product submittals, etc.). Statement of Purpose The primary research objectives and research questions for this project are as follows: Objectives • Document and categorize current practices and applications of Quality Management Systems (both traditional and alternative) in highway construction for all project delivery methods • Explore how highway construction projects of all project delivery methods are effectively applying alternate quality management systems. (developing and implementing quality management systems) • Identify benefits and limitations of the approaches 169

• Explore how to implement and apply quality management system for all methods of project delivery • Produce a guidebook that will match appropriate quality management systems to selected alternative delivery methods o Describes the quality systems in the I2QM model o Discusses the barriers to each system o Gives guidance for individual roles in development and adoption of alt. quality management models in their agency • Produce a research report that addresses the implications of adopting the guidelines and the barriers to implementation Research Questions 1. What is the fundamental definition of quality and what is the underlying purpose of a “quality program?” 2. How are projects using alternative delivery methods currently applying quality management systems? 3. What are the advantages and disadvantages to the contractor and the owner of alternative quality management systems relating to various project delivery alternatives? 4. What changes must be made to the baseline quality management system to adapt to evolving project delivery methods? Relevant Readings The protocol is based largely on the following documents and research reports.  NCHRP Project 10-83 Proposal  Coding Structure for NCHRP Project 10-83  TRB Circular E-C137 Glossary of Highway Quality Assurance Terms  NCHRP Synthesis 376  NCHRP Synthesis 40-02 170

C.2 Field Procedures Project Researchers The following is a list of the project investigators and their contact information.  Keith R. Molenaar, Ph.D. – Principal Investigator K. Stanton Lewis Chair and Associate Professor Construction Engineering and Management Program Department of Civil, Environmental, and Architectural Engineering University of Colorado at Boulder Campus Box 428, ECOT 643 Boulder, Colorado 80309-0428 Telephone: 303-735-4276 Facsimile: 303-492-7317 E-mail: [email protected]  Douglas D. Gransberg, Ph.D., P.E. – Co-Principal Investigator Donald and Sharon Greenwood Chair and Professor of Construction Engineering Construction Engineering and Management Program Department of Civil, Construction, & Environmental Engineering Iowa State University 456 Town Engineering Ames, IA 50011 Telephone: 515-294-4148 E-mail: [email protected]  David N. Sillars, Ph.D. – Co-Principal Investigator R.C Wilson Chair and Associate Professor Construction Engineering and Management Program Civil and Construction Engineering Oregon State University 220 Owen Hall Corvallis, Oregon 97331 Telephone: 541 737-8058 Fax: 541 737-3300 Email: [email protected]  Elizabeth R. Kraft Graduate Student Construction Engineering and Management Program Department of Civil, Environmental, and Architectural Engineering University of Colorado at Boulder Campus Box 428 Boulder, Colorado 80309-0428 171

Telephone: Facsimile: Email: [email protected]  Nickie West Graduate Student Construction Engineering and Management Program Department of Civil, Construction, & Environmental Engineering Iowa State University 456 Town Engineering Ames, IA 50011 Telephone: Facsimile: E-mail: [email protected]  Landon S. Harman Graduate Student Construction Engineering and Management Program Civil and Construction Engineering Oregon State University 220 Owen Hall Corvallis, Oregon 97331 Telephone: Email: [email protected] Case Study delegation Note: The information in this section will not be available until after potential case studies have been identified and selected for study. When this information is available, this section will list who will be contacting each case study, what the scope of their questions/role will be, and who will be following up to garner additional information or thank participants for their time and effort. Case Study Identification and Schedule Case study project selection criteria In the original research proposal, it was stated that “a concerted effort will be made to select case study projects from transportation agencies that have mature experience with at least two different project delivery methods.” A total of 6 – 12 case studies will be performed with at least two of the case studies coming from a non-STA (USACE or the FTA). Additionally 2 pilot studies will be conducted and reviewed prior to the remaining studies to validate the case study protocol and data coding and to ensure that the research objectives will be met by the data collected. Potential case studies will be identified in Task 1 of the research as a part of the initial survey that aims to identify alternative quality management systems currently in use. The results of this survey will be used to populate an initial list of potential case studies. Case study project selection protocol will involve three tiers of project information that must be present to move a potential case study project into the 172

final list of candidates. The case study candidates will be forwarded to the NCHRP Panel for approval. The tiers are as follows: 1. Project Factor Information: This tier seeks to create a uniform set of data points for every case study project to ensure that trends or disconnects found during analysis can be uniformly mapped across the entire case study population (Yin 2008). Examples of this information are project location, size, major type of construction, initial and final budget amounts, initial and final delivery periods, delivery method, and other factors as required. 2. Project Quality Factor Information: This tier seeks to explicitly define the precise details of the system used to manage quality across the case study project’s life cycle. Examples of this information are quality plans, quality organization composition, use of consultants for independent technical review or independent assurance/oversight, quality audits, division of quality management responsibility between the various stakeholder in the project and other factors as required. 3. Project Performance Information: This tier seeks to measure, if possible, the success of the quality management system employed in each project. It will use to the extent possible the stipulated performance metrics for scope, schedule, cost, quality and risk that were created for each case study project and will attempt to back-calculate performance metrics for common areas in all the projects to furnish a means of comparison and to identify and quantify the magnitude of the trends and disconnects in the case study project population. In addition to the above, a concerted effort will be made to select case study projects from transportation agencies that have mature experience with at least two different project delivery methods. The FHWA Report to Congress on Design-Build Effectiveness (2006) identified more than 30 state STAs that had been authorized to delivery design-build projects under SEP-14 authority. However, of that sample 12 had not completed a single design-build project, another eight had only completed one design-build project, and only six had finished more than five design-build projects. Thus, at that point in time, for this project delivery method the quality management systems of only six STAs could have been impacted by multiple project experiences. Since that time additional experience has been gained and NCHRP Synthesis 376 (2008) reported that four STAs had completed five to 10 design-build projects and nine had completed greater than 10. Thus, depending on the final criterion for design-build experience, as many as 13 STAs will have potential case study design-build projects that had quality management systems that were tailored for DB project delivery. That is not the case in construction management at risk. NCHRP Synthesis 40-02 (2010) reported that only Florida and Utah have completed more than a single construction management at risk project. Thus, case study projects with construction management at risk tailored quality management systems will have to come from those two STAs or from another mode of transportation such as transit. To summarize, the primary criterion for case study project selection will be the requirement to have come from an agency that has sufficient experience with a given project delivery method that the potential exists that the project was designed and built using a quality management system that was modified from the baseline design-bid-build system based on actual experience, using the USACE cyclical quality management system where quality management experience is fed back into the quality planning process to continuously improve the performance of the system itself. 173

This list of potential case studies created from the previously mentioned protocol will be supplemented by the Industry Advisory Board and the investigators’ industry contacts. The following three pieces of information will have been collected for each case study as a part of the survey in Task 1: (1) name and location of the project; (2) description of the project delivery, procurement, and contracting method in use; and (3) description of the quality management system. The goal for selecting the case studies will be to generate a cross section of cases that allow for analysis of the advantages and disadvantages of the quality management systems across the various project delivery method characteristics. To ensure that this goal is met, the following criteria will be placed on the case study selection will include: • Quality management systems for design and construction; • Quality assurance by all parties and independent auditors; • Project quality assurance including independent verification and independent acceptance; • Project delivery methods including design-build, construction manager-at-risk, and PPP; • Procurement methods including best value, A+B, and qualifications-based selection; and • Payment methods including incentives, lump sum, and guaranteed maximum price. Case study informant selection Once a case study project has been selected, several members of the team directly associated with creating and implementing the quality management plan will need to be interviewed. While many people are responsible for ensuring quality on a project during its lifecycle from conception through construction, we will seek to speak with – at a minimum – enough project team members to fully satisfy the research objectives and goals. This may include speaking with representatives from the owner’s, designer’s, and contractor’s project team to develop a full picture of the quality management systems utilized on a project and their relation to each other. Potential interviewees include the following:  Agency project manager, contracting manager, quality manager, etc.  Project design manager, construction manager, design quality manager, construction quality manager, etc.  Designer quality manager, project manager  Contractor preconstruction manager, quality manager, project manager  Third party quality assurance/quality control inspectors 174

Case Study Basic Data and Research Delegation Tables # Case Study Name Location Organization Contact (Information) 1 2 3 4 5 6 7 8 # Case Study Name Contacted? Lead Investigato r Interview Type Intervie w Date Follow-up Date 1 2 3 4 5 6 7 8 175

# Case Study Name Materials/Documents Received 1 2 3 4 5 6 7 8 C.3 Requested Documents The following is a list of documents that will be requested. However, successful completion of the research study does not require that each of the documents is collected. A subset of these documents will likely be collected depending upon the unique attributes of the project being studied. • Project RFP/RFQ • Project RFP/RFQ Response • Project Quality Management Plan • Project Design Quality Plan • Project Construction Quality Plan • Agency/Company Quality Plan • Project Quality Organizational Chart • Copy of the contract with the engineer/contractor/consultant • Organizational document that outlines its approach to quality assurance on project Case Study Questions This section seeks to formalize the interview questions asked of each case study to allow for easier comparison and analysis between case studies later on. Each interview will be unique and largely guided by the case study participants. To draw out the pertinent information, specific questions may be needed while some of the questions listed here may not be relevant. In modifying the protocol questions, generality should still be maintained so that the results can still be readily compared and categorized according to the coding structure. 176

Questionnaire The purpose of the questionnaire is to identify how state highway agencies (SHA) have implemented alternative QA programs and from that baseline, identify commonly used practices for dissemination and use by SHAs that intend to implement alternative procurement on future projects or alternative QA methods in their current program. The questionnaire consists of closed ended questions which will allow the researchers to perform a quantitative analysis. The data gathered by the questionnaire will be used to validate the case study findings. Ideally this questionnaire will be completed by the respondent prior to the interview. If it is not completed prior to the interview then it will be requested that the respondent complete it during the interview. The questionnaire is included in APPENDIX B. Background and Overall Quality Questions Background  Background information to keep track of who we spoke with on each project  Relevant prior experience used to potentially weight their opinions in later comparison 1. Name, occupation, employer 2. What are you current duties, especially related to QM, QA, QC? 3. Have you held any positions prior to your current position related to QM, QA, or QC? If so, please briefly list that information. 4. How long have you held your current position? 5. Have you worked on projects of different project delivery methods? If so specifically what project delivery methods do you have experience with? 6. How many years of experience do you with projects using baseline/DBB quality systems? 7. Name and location of the project for which you are answering project-specific questions Overall Understanding of Quality  Examines the overall understanding of quality and the informants attitudes towards quality  Establishes a baseline of traditional/DBB quality for comparison 8. How do you define project quality? 9. What is your understanding of the differences between quality management, quality assurance, and quality control? 10. In your experience, how has quality management been approached on a traditional/DBB project?  Project by project basis?  During the RFP process?  Dictated by owner agency? If so, how is it dictated?? Specifications, performance… 11. What are the critical elements/milestones of a quality management plan on a traditional/DBB project? 12. How are the quality roles and responsibilities divided on a traditional/DBB project? 13. In your experience, what quality systems/procedures have been successful on traditional/DBB projects? 177

14. What are some characteristics of a successful quality management plan, regardless of the project delivery method, contracting method, or procurement method? Organizational Questions  Provides the needed categories and statistics (from the coding structure) to later sort and group the enterprise level QM information we receive. As not everyone selected for an interview will have answered the survey, these questions are not redundant Please answer the following questions related to quality management at different phases of a project for the organization you work for as a whole. Please be as thorough as possible when discussing what quality management systems may exist and be utilized within your organization. 15. Of the projects your organization designs/builds/owns what percent of the projects are managed using the following delivery methods:  Design-bid-build (DBB): ______  Design-build (DB): _______  Construction Manager/General Contractor (CM/GC): _______  Public-Private-Partnership (PPP): _______  Other (please list): _______ 16. Of the projects your organization designs/builds/owns what percent of the projects are procured using the following procurement methods:  Cost: ______  Best-value: _______  Qualifications: _______  Design: _________ 17. Of the projects your organization designs/builds/owns what percent of the projects utilize the following contract payment methods:  Payment type: ________  Incentive type: ________ 18. If you work for a STA, what due diligence requirements must your organization meet in your state for quality assurance purposes on public projects?  Please list applicable laws, reporting requirements, or processes required in your state  Can you utilize new or alternative quality management practices that produce equal or superior quality projects to meet these requirements? If not, what legal barriers prevent you from doing so?  If so, what steps are required to do this? Has this been done before? Please describe if it has. 19. Does your organization perform any in-house design work?  On what percent of your projects do you perform in-house design?  Do you have formal quality assurance systems in place for this process?  If so, please describe this process. 20. Does your organization self-perform any construction work?  On what percent of your projects do you self-perform some amount of construction?  Do you have formal quality assurance systems in place for this process?  If so, what are they? 178

Design Phase QM  Examines design QM and is broken up into components according to the coding structure to allow for easier comparison between case studies later on  How is design quality management managed on all project delivery types whether design is kept in-house or out of house 21. Does your organization have any design quality assurance systems in place?  If so, please describe the systems, policies, procedures, techniques, or standards used to ensure the quality of the designs you produce or receive. Can you provide us with an electronic or hard copy of these systems, policies, procedures, techniques, or standards?  Have you found these systems to be effective at producing quality designs? If yes, what makes them superior to other systems you are familiar with or function well? If no, what changes would you suggest to increase their efficacy?  Are there any additional procedures that would further improve the quality of designs you produce or receive? If so, please describe them. 22. Does your organization have any design quality control systems in place?  If so, please describe the systems, policies, procedures, techniques, or standards used for quality control of the designs you produce or receive. Can you provide us with an electronic or hard copy of these systems, policies, procedures, techniques, or standards? 23. Does your organization utilize any form of peer review or 3rd party/independent design quality assurance?  If so, please describe how this is conducted and how the results are utilized to improve the quality of the original design.  If this process is formalized, can you provide us with any documents detailing it? Construction Phase QM  Examines construction QM from a STA’s point of view and is broken up into components according to the coding structure to allow for easier comparison between case studies later on  The primary focus of many QM plans and where the bulk of the information gathered at an enterprise level may be found 24. How does your organization perform construction quality assurance? Who has primary responsibility?  Does the process change based on the project type, contract style, or delivery method? If so, how is the process tailored? Who decides which method will be used?  Has your organization experimented with new quality assurance methods? Are you actively implementing any now? If so, what are they and how have you evaluated their efficacy?  Has your organization modified traditional quality assurance processes to make them applicable to projects with alternative delivery methods? If so, how have you modified them? 25. Who is responsible for construction quality control on projects your organization is a part of?  What is the process used for quality control? If this process changes due to project type, contract style, or delivery method, how does it change?  Do you still utilize traditional quality control methods such as control charts?  Has your organization modified traditional quality control processes to make them applicable to projects with alternative delivery methods? If so, how have you modified them? 179

 Has your organization experimented with new quality control methods? Are you actively implementing any now? If so, what are they and how have you evaluated their efficacy? 26. What role does independent assurance (IA) play in your projects?  Which parties – owner, designer, or builder – utilize IA to ensure a quality product? 27. How is owner verification testing used in your traditional projects?  What role does it have in your projects?  What statistical tests are used to verify the contractor’s results?  Who conducts the testing the owner or a third party?  Is it used on your non-traditional projects as well? If so, what function does it serve on those projects? 28. How is owner acceptance testing utilized on this project?  Is this seen as sufficient justification of quality construction to ensure final payment? How are the results used?  Who conducts the testing, the owner or a third party?  If it is used on non-traditional projects, what function does it serve? 29. Does your organization make use of or offer any form of post-construction quality assurance? Please describe this if you can.  Is this part of a warranty or of an operate-and-maintain contract? If not, what is it?  If you are engaged in contracts with warranties or operate-and-maintain clauses, how does this impact your quality management model before and during construction? Project Specific Questions Please answer the following questions for the specific project you were contacted about. While not every question may apply to your project, please be as thorough as possible. If you have additional information regarding the quality management process on you project that you would like to provide, please feel free to add that below or contact us directly. Procurement and delivery method  Used to categorize the case studies for later comparison, these identifying questions will help focus future analysis  Seeks to explore the relationship (if one exists) between the delivery method selected and the QM techniques used 30. What delivery method is being/was used for this project (DBB, DB, CM/GC, PPP, etc.)? 31. What procurement method (cost, best-value, qualifications based, design based, A + B, multiparameter bidding, etc.) was used to select the designer/builder/concessionaire on this project? 32. Was a prequalification process used? If so, please describe that process.  If a prequalification process was used, were interested parties required to submit a quality management plan (QMP)? Were they required to identify a dedicated quality manager? If you still have the QMPs from the initial solicitations, can you provide us with copies? 33. What requirements related to Quality were included in the RFP? 180

 Submittal of a QMP prior to award, qualifications of the quality staff on the project, submittal of a QMP after award, etc. 34. Were either the procurement or delivery methods selected partially or in whole because of their effect on quality management? If so, why were the particular methods selected? 35. What contract payment approach was selected for the project, payment or incentive?  What effect did this have/is this having on quality management on the project? Quality Management Plan  Determine roles and responsibilities for creating the overall quality management plan  Determine the purpose/requirements of the quality management plan  Determine how the Quality Management plan was developed and approved 36. How was the Quality Management Plan (QMP) created?  Was it a stock plan modified for the project? Was it created from scratch? 37. Who was responsible for creating the Quality Management Plan? 38. How was the overall quality management hierarchy created for this project?  What is the overall quality management hierarchy? Can you provide us with a copy of the org chart? 39. What were the Agency’s quality objectives/requirements for this project and how were these communicated? 40. Describe the QMP approval process. 41. How did the QMP differ from a traditional DBB project QMP?  Why did this project’s QMP need to be different from a traditional DBB? 42. How was the QMP development process different from a traditional DBB project?  Why was this projects QMP development process different from a traditional DBB QMP development? 43. How was this QMP and its development process successful?  What about the QMP and its development process could be improved for the next project? 44. What other management plans were required for this project and what were the quality roles and responsibilities associated?  Design, construction, environmental, traffic, etc… 45. Describe how the quality management on this project is different from a traditional DBB project  What procedures/processes/systems were different?  What project factors required the design quality management plan to be different from the traditional (project delivery, project complexity, funding), Design phase QM  Details the specific design QM techniques used on this project as broken down by the coding structure and gathers any additional information related to this process 181

 Seeks to understand the relationship between the techniques used on the project and those recommended at the agency-wide level  Seeks to understand how the design QM was developed and approved and requests the needed documents 46. How was the design Quality Management Plan (QMP) created and by whom?  What requirements was it based on? 47. What was your organization’s role/responsibilities in developing and implementing the design QMP on this project? 48. What was the project hierarchy (org chart) for this phase of the project? Who was responsible for design QA/QC on this project?  Can we get a copy of that chart? 49. What was the approval process for the Design QM? 50. How were the Agency’s design quality requirements communicated to the Designer?  By RFP, design guidelines, performance specifications, etc. 51. Describe how the design quality management on this project is different from a traditional DBB project  What procedures/processes/systems were different?  What project factors required the design quality management plan to be different from the traditional (project delivery, project complexity, funding), 52. What was the basic premise of the design QMP for this project?  Over the shoulder reviews, spot checks, design checks at certain milestones, etc. 53. Was quality assurance incorporated in the design phase of this project? If so, how was it implemented on this project?  Who was responsible for QA, the owner or designer?  Was a formal plan drafted detailing how design QA would be considered? If so, can you provide us with a copy of that report? 54. Were design quality control processes utilized on this project? If so, how?  Are there documents outlining these processes? If so, can you provide us with a copy? 55. Was a dedicated design quality manager assigned by the owner, designer, CM, or concessionaire? What were their responsibilities? What authority did they have to make changes to the design or QA processes? 56. Was a peer review or independent assurance component included in design phase quality management?  If so, what was its role and how did it impact the final design? 57. What problems were discovered and corrected through the design phase quality management process?  What problems were not discovered until the construction phase of the project? Could these have been avoided with a more robust design quality management process? If so, what changes would need to be made to your process to catch these problems in the future? 182

58. Was the design phase QMP used on this project a standardized system used on most or all of your organization’s projects? If not, was it tailored to meet the needs of this project from an existing process or created from scratch for this project?  How much time/what resources did creating the design QMP for this project require?  If an existing design QMP was modified for this project, can you provide copies of both for comparison? 59. How effective was the design QMP on this project?  How could it have been improved? Construction Phase QM  Details the specific construction QM techniques used on this project as broken down by the coding structure and gathers any additional information related to this process  Seeks to understand the relationship between the techniques used on the project and those recommended at the agency-wide level  Seeks to understand how the design QM was developed and approved and requests the needed documents 60. How was the Construction Quality Management Plan (QMP) created and by whom? 61. How much of the quality systems were developed during the RFP/contracting phase?  Was a quality plan required at RFP submittal? after award?  Was the quality plan developed collaboratively? 62. What was the approval process for the construction QMP? 63. How were the Agency’s design quality requirements communicated to those crafting the construction QMP?  By RFP, design guidelines, performance specifications, etc. 64. How were this project’s QM, QA, and QC systems different from a project using traditional DBB?  Roles/responsibilities, liabilities, etc.  What project factors required the design quality management plan to be different from the traditional (project delivery, project complexity, funding), 65. Was there a dedicated quality manager for this project? Who employed the manager, the owner, builder, designer, concessionaire, etc.?  What responsibility and authority to make changes did this manager have? 66. How was construction quality assurance put into place on this project? 67. What role did construction quality control play on this project? If a problem was discovered in the QC process, were changes made quickly enough to prevent future problems?  What facilitated or hindered this rapid communication?  How was QC implemented on this project? 68. Was owner verification testing used on this project to check the contractor’s QC process?  Who performed the testing, the owner or a 3rd party?  Were significant discrepancies discovered on this project? If so, did the QMP provide an effective way to deal with these? How? 69. Was owner acceptance testing included on this project before final payment? 183

 Who performed this testing?  Were significant issues discovered?  How were they handled? Were all parties satisfied with the outcome? 70. Was independent assurance included as part of the construction QMP on this project?  If so, what role did it play in quality management on the project? 71. What other features/systems were parts of the QMP during the construction phase of this project?  What additional features could have prevented quality issues from arising on this project? 72. How was QM in the construction phase affected by the contract delivery method?  By the procurement method? 73. Can you provide a copy of the construction QMP for this project?  Did you modify an existing QMP for the construction phase of this project or develop one from scratch? If you modified an existing plan, can you provide copies of both for comparison? 74. What challenges did the QMP on this project face and how were they overcome? 75. Overall, were you satisfied with the quality management of the construction phase of this project?  Which particular aspects of the QMP were you especially pleased with?  If not, what would you change regarding quality management for your next project? 76. Based on your experience, would any of the quality techniques/systems used on this project be beneficial if applied to a traditional/DBB project? Post-Construction QM  Explores whether a formal policy for QM relating to the post-construction period exists and if such a policy exists, seeks to understand its relationship to the QMP implemented during design and construction of the project 77. Did/does this project include any warranty or operate-and-maintain provisions in the contract? If no, skip the rest of this section. If yes, please answer the following questions.  How did the inclusion of these provisions affect the QMP during construction?  When compared with other projects NOT having these provisions, how did these provisions affect the overall quality of the project? Additional Questions  This section is designed to allow for further refinement of the interview (in the pilot study phase)  Offers the participant the opportunity to add any additional information he/she believes is relevant to our study that we may have overlooked or not asked about for lack of project specific knowledge 78. As a case study participant, how easy to understand did you find the interview questions? Were they straight forward? Did you struggle to understand the intent of some questions? In addition to any answers to these questions, please provide a rating from 1 to 10 (10=easy to understand, 1=so hard to understand it was difficult to finish). 184

 Which questions did you find especially difficult to understand or answer? 79. Did the interview seem repetitive to you? If so, what sections/questions seemed to contain overlapping material? 80. Was the interview burdensome to complete? Were you able to answer each question to the extent of your knowledge without fatigue? 81. Given our project objectives, what additional information can you provide that would help us to better understand alternative quality management? What other QM processes, procedures, or ideas are you aware of (for any phase of a project) that you have not shared with us thus far or have been unable to utilize first-hand? 82. Given our project objectives, are there additional questions that you feel we should be asking? If so, what questions would you suggest? 83. Do you know of any other co-workers or industry peers with a position related to quality management that would be interested in speaking with us? If so, can you provide their contact information? 84. Are you aware of any projects that have utilized or are utilizing excellent quality management procedures on a project with a non-traditional delivery method (DB, CM/GC, PPP, etc.)? If so, can you provide us with the name, location, and any other pertinent information for the project? C.4 Data Analysis • The complete data analysis plan is described in the project Work Plan. The main points of the analysis include: • Advantages and disadvantages to each system from the agency’s and the designer’s/constructor’s point of view; • Identification of trends and common finding between the lit review, survey and case studies; • Triangulate the common findings from these three sources of data to arrive at valid conclusions; • Case studies will be summarized individually in the lens of the IQ2M model; • Using literature review and survey information, compare key attributes of the baseline approach to key attributes in the IQ2M models. (design quality is a gap, but rigorous comparison of construction quality control, construction quality assurance and independent audit procedures will be made); • Individual findings will analyzed across the cases using pattern matching techniques; and • Comparison to baseline quality management system approaches. 185

C.5 Case Study Contact Flowchart #1 Phone •Call contacts for identified case studies and secure their agreement to act as a champion for the case study process • Set up a date and time for the case study interview during this contact #2 Letter • Send letter #1 from the case study protocol to the contact confirming their agreement to act as a champion •This letter should include a brief outline of our project goals and documents we will request #3 Letter • Send letter #2 from the protocol and the questionnaire to participant and ask them to fill it out before the interview if possible •This letter should also include a list of the documents we will be requesting, ask them to collect them ahead of time if possible #4 In-Person •Conduct the case study interview at the agreed time and date •Before leaving the interview, be sure that at a minimum: the questionnaire has been filled out and the needed documents have been collected #5 Letter • Send letter #3 from the protocol thanking the participant for their time and assistance and offer to share the results of the research with them when it is finished 186

C.6 Sample Letters Letter #1 MEMORANDUM DATE TO: Survey Participant FROM: Keith Molenaar Principal Investigator SUBJECT: NCHRP 10-83 Case Study Thank you for agreeing to participate in the NCHRP 10-83 Research Project case study concerning alternative quality management procedures for highway construction projects utilizing non-traditional contracts. We have enclosed some brief background information about the research project, its objectives, goals, and methods. We are currently scheduled to (meet with/call) you on (insert day/month) at (insert time) to conduct our interview. If for some reason this no longer works for you, please contact me as soon as possible to reschedule. Before our interview, we will send you a questionnaire related to the topics we would like to cover in the interview. Please review the questionnaire prior to the interview to become acquainted with the nature of the questions that we will be discussing. I’ve attached a brief outline of our research interests along with a list of documents related to the project that we would like to collect. If you have any questions, please feel free to contact me by telephone at 303-735-4276 or by email at [email protected]. Regards, Keith Molenaar Requested Documents Below is a list of documents we would like to collect regarding the project this case study is focusing on. While your project may not have all of the listed documents, we need to obtain a copy of any of the documents included on your project as well as any additional documents not listed that you believe are relevant to our research objectives as outlined below. While we would prefer (hard/electronic) copies, 187

either will suffice. Additionally, are there any online resources we can examine (FTP sites, project websites, etc.) before the interview to familiarize ourselves with the project? • Copy of the contract with the engineer/ contractor/ consultant • Organizational document that outlines its approach to quality assurance on project • Project RFP/RFQ • Project Quality Organizational Chart • Project RFP/RFQ response • Project Construction quality plan • Project Quality Management Plan • Agency/Company quality plan • Project design quality Plan • Project Background The NCHRP 10-83 research project has four primary research objectives. They are to: • Document and categorize current practices and applications of Quality Management Systems (both traditional and alternative) in highway construction for all project delivery methods • Explore how highway construction projects of all project delivery methods are effectively applying alternate quality management systems. (developing and implementing quality management systems) • Identify benefits and limitations of the approaches • Explore how to implement and apply quality management system for all methods of project delivery In the course of meeting those objectives, we will seek to answer these four questions: 1. What is the fundamental definition of quality and what is the underlying purpose of a “quality program?” 2. How are projects using alternative delivery methods currently applying quality management systems? 3. What are the advantages and disadvantages to the contractor and the owner of alternative quality management systems relating to various project delivery alternatives? 4. What changes must be made to the baseline quality management system to adapt to evolving project delivery methods? To answer these questions, we are performing several in-depth project case studies, covering both traditional and alternative delivery methods. The case studies will be used to learn how existing projects have managed quality in light of non-traditional delivery methods. After analyzing and comparing the various case studies, the information gathered will be condensed into working theories, used to modify what we are calling the Integrated Quality Management Model (IQ2M), and ultimately used to prepare a final research report for the NCHRP and a set of guidelines for when implementing certain AQM systems may be useful in light of other project characteristics. 188

Letter #2 MEMORANDUM April 17, 2015 TO: Survey Participant FROM: Keith Molenaar Principal Investigator SUBJECT: NCHRP 10-83 Case Study Questionnaire Thank you again for your assistance with this research effort investigating alternative quality management procedures for highway construction projects utilizing non-traditional contracts. Enclosed is a questionnaire that touches on many of the topics we would like to cover in our in-depth interview. The questionnaire will be used as a baseline to generate easily comparable data across all of the different case studies we will be performing. It will be a starting point for our discussion and we would greatly appreciate it if you would take the time to look it over and fill it out to the best of your ability before our scheduled interview on (insert day/month) at (insert time). Also enclosed is a list of the documentation we are seeking for this case-study. While we would prefer (hard/electronic) copies, either will suffice. If this information is available on a website for FTP site, please let us know so that we may familiarize ourselves with the information ahead of time. If you have any questions, please feel free to contact me by telephone at 303-735-4276 or by email at [email protected]. Regards, Keith Molenaar 189

Letter #3 MEMORANDUM April 17, 2015 TO: Survey Participant FROM: Keith Molenaar Principal Investigator SUBJECT: NCHRP 10-83 Case Study Follow-Up Thank you for your participation in the NCHRP 10-83 interview process. We recognize that this process is time consuming and very much appreciate your assistance in helping us better understand alternative quality systems in this industry. Your insight and experience with this project will be invaluable as we compare it with projects from across the country and try to develop a better understanding of which systems work well and which do not. The research report and guidelines will not be finished until the summer of 2012, but we would be happy to share the research results with you then if you would like. Again, thank you for your time! If you have any questions, please feel free to contact me by telephone at 303-735-4276 or by email at [email protected]. Regards, Keith Molenaar 190

C.7 Questionnaire INTRODUCTION/BACKGROUND: The purpose of this questionnaire is to identify how state highway agencies (SHA) have implemented alternative QA programs and from that baseline, identify commonly used practices for dissemination and use by SHAs that intend to implement alternative procurement on future projects or alternative QA methods in their current program. DEFINITIONS: The research will use TRB Circular E-C074, Glossary of Highway Quality Assurance Terms to standardize its terminology. The following are terms that must be carefully understood to properly complete this survey. Quality: (1) The degree of excellence of a product or service. (2) The degree to which a product or service satisfies the needs of a specific customer. (3) The degree to which a product or service conforms with a given requirement. Quality Assurance (QA): All those planned and systematic actions necessary to provide confidence that a product or facility will perform satisfactorily in service. [QA addresses the overall problem of obtaining the quality of a service, product, or facility in the most efficient, economical, and satisfactory manner possible. Within this broad context, QA involves continued evaluation of the activities of planning, design, development of plans and specifications, advertising and awarding of contracts, construction, and maintenance, and the interactions of these activities.] TRB E-C074. Quality Control (QC): Also called process control. Those QA actions and considerations necessary to assess and adjust production and construction processes, so as to control the level of quality being produced in the end product. TRB E-C074. Quality Management (QM): The overarching system of policies and procedures that govern the performance of QA and QC activities. The totality of the effort to ensure quality in design and/or construction. Design-Bid-Build (DBB): A project delivery method where the design is completed either by in-house professional engineering staff or a design consultant before the construction contract is advertised. Also called the “traditional method.” Design-Build (DB): A project delivery method where both the design and the construction of the project are simultaneously awarded to a single entity. Construction Manager-General Contractor (CMGC): A project delivery method where the contractor is selected during the design process and makes input to the design via constructability, cost engineering, and value analysis reviews. Once the design is complete, the same entity builds the projects as the general contractor. CMGC assumes that the contractor will self-perform a significant amount of the construction work. Construction Manager-at-Risk (CMR): A project delivery method similar to CMGC, but where the CM does not self-perform any of the construction work. Public Private Partnership (P3): A project delivery method where the agency contracts with a concessionaire organization to design, build, finance and operate an infrastructure facility for a defined extended period of time. 191

Design deliverable: A product produced by the design-builder’s design team that is submitted for review to the agency (i.e. design packages, construction documents, etc.). Construction deliverable: A product produced by the design-builder’s construction team that is submitted for review to the agency (shop drawings, product submittals, etc.). Requested Documents Below is a list of documents we would like to collect regarding the project this case study is focusing on. While your project may not have all of the listed documents, we need to obtain a copy of any of the documents included on your project as well as any additional documents not listed that you believe are relevant to our research objectives as outlined below. While we would prefer electronic copies if available, but either will suffice. • Copy of the contract with the engineer/ contractor/ consultant • Organizational document that outlines its approach to quality assurance on project • Project RFP/RFQ • Project Quality Organizational Chart • Project RFP/RFQ response • Project Construction quality plan • Project Quality Management Plan • Agency/Company quality plan • Project design quality Plan • Additionally, are there any online resources we can examine (FTP sites, project websites, etc.) before the interview to familiarize ourselves with the project? General Information -STA: 7. US state in which the respondent is employed: 8. Name of Agency: 9. Names and groups/sections of interviewees: 10. Please check the appropriate boxes for your agency’s project delivery program (PDM). Project Delivery Method Legislative/Legal Authority Number of years of experience with PDM DBB ☐NA; ☐Pilot projects only; ☐General authorization ☐NA; ☐1-5; ☐5-10;☐ > 10 CMGC ☐NA; ☐Pilot projects only; ☐General authorization ☐NA; ☐1-5; ☐5-10;☐ > 10 DB ☐NA; ☐Pilot projects only; ☐General authorization ☐NA; ☐1-5; ☐5-10;☐ > 10 P3 ☐NA; ☐Pilot projects only; ☐General authorization ☐NA; ☐1-5; ☐5-10;☐ > 10 Other ☐NA; ☐Pilot projects only; ☐General authorization ☐NA; ☐1-5; ☐5-10;☐ > 10 11. Are your Quality Management systems different between Project Delivery Methods? ☐Yes ☐No 192

12. What is the approximate proportion of in-house design versus outsourced design services? In-house design services - Click here to enter text.% Outsourced design services - Click here to enter text. % Administrative prequalification: A set of procedures and accompanying forms/documentation that must be followed by a designer or construction contractor to qualify to submit bids on construction projects using traditional project delivery. Performance based prequalification: A set of procedures and back-up documents that must be followed by a designer or construction contractor to qualify to submit a bid on a construction project based on quality, past performance, safety, specialized technical capability, project-specific work experience, key personnel, and other factors. 3. Does your agency use a prequalification program for design firms? ☐Yes ☐No If the answer to the previous question is YES, please answer for your agency’s program to prequalify design firms. Designer prequalification program factors Prequalification Type Administrative Performance Based Prequalification required for all projects ☐ ☐ Prequalification required for selected projects only ☐ ☐ Prequalification standards are the same for all projects ☐ ☐ Prequalification standards are different by project class ☐ ☐ 4. Does your agency use a prequalification program for construction contractors? ☐Yes ☐No If the answer to the previous question is YES, please answer for your agency’s construction contractor prequalification program. Construction prequalification program factors Prequalification Type Administrative Performance Based Prequalification required for all projects ☐ ☐ Prequalification required for selected projects only ☐ ☐ Prequalification standards are the same for all projects ☐ ☐ Prequalification standards are different by project class ☐ ☐ Case Study Project Information and Data 1. Project Name and location: 2. Project scope of work: 193

3. Original Total Awarded Value of project: $ Final Total Value of project: $ 4. Date preliminary design contract awarded: Date project advertised: 5. Date final design contract awarded: Date construction contract awarded: [Note: same if DB] 6. Original Project Delivery Period (including design) Final Project Delivery Period (including design) Explanatory notes: 7. Project delivery method used on this project: Design-Bid- Build CM-at- Risk Design- Build P3 Please explain what effect this choice had on the overall quality of the project: 8. Which of the following were reasons why your agency selected the delivery method used for this project? Check all that apply. Reduce/compress/accelerate project delivery period Establish project budget at an early stage of design development Get early construction contractor involvement Encourage innovation Facilitate Value Engineering Encourage price competition (bidding process) Compete different design solutions through the proposal process Redistribute risk Complex project requirements Flexibility needs during construction phase Reduce life cycle costs Provide mechanism for follow-on operations and/or maintenance 194

Innovative financing Other: Explain 9. Which of the above was the single most significant reason for the delivery method decision on this project? 10. Please explain the process that led you to the choice of the project delivery system for this project . Case Study Project Quality Management Policy/Procedures Information: The following questions will break up the quality management process into the following four phases: ♦ Procurement phase: Actions taken regarding the quality management process that are reflected in the agency’s contractor prequalification requirements and/or solicitation documentation such as in the Invitation for Bids (IFB), Request for Qualifications (RFQ) and the Request for Proposals (RFP). ♦ Design Phase (in-house): Actions taken after approval to start design work regarding ensuring the quality of the design deliverables as well as that the final design complies with contractual requirements. OR ♦ Design Phase (out-source): Actions taken after design contract award regarding ensuring the quality of the design deliverables as well as that the final design complies with contractual requirements. ♦ Construction Phase: Actions taken after contract award regarding the quality of the final constructed product to ensure that it complies with both the completed design and other contractual requirements. The research team understands that the term “Approval” has a variety of slightly different meanings from state to state. It is used here to indicate the process by which the agency indicates that it is satisfied with the quality of the design or construction deliverable and is willing to make payment for satisfactory completion of that task if asked. Procurement phase: 1. What procurement method was used to select the designer, builder, or concessionaire? (low bid, best-value, qualifications based selection, etc.) 195

2. Please answer for the case study project. If your process was conducted in more than one way, please answer for the most prevalent set of procedures. Do your project advertising/solicitation documents (i.e. IFB, RFQ, RFP, etc.) contain the following? Required proposal/ bid package submittal? If YES: Is it evaluated to make the award decision? If NO: Is it a required submittal after contract award? Yes No Yes No Yes No Qualifications of the Design Quality Manager ☐ ☐ ☐ ☐ ☐ ☐ Qualifications of the Construction Quality Manager ☐ ☐ ☐ ☐ ☐ ☐ Qualifications of other Quality Management Personnel (design reviewers, construction inspectors, technicians, etc.) ☐ ☐ ☐ ☐ ☐ ☐ Design quality management plan ☐ ☐ ☐ ☐ ☐ ☐ Design quality assurance plan ☐ ☐ ☐ ☐ ☐ ☐ Design quality control plan ☐ ☐ ☐ ☐ ☐ ☐ Construction quality management plan ☐ ☐ ☐ ☐ ☐ ☐ Construction quality assurance plan ☐ ☐ ☐ ☐ ☐ ☐ Construction quality control plan ☐ ☐ ☐ ☐ ☐ ☐ Quality management roles and responsibilities ☐ ☐ ☐ ☐ ☐ ☐ Design criteria checklists ☐ ☐ ☐ ☐ ☐ ☐ Construction testing matrix ☐ ☐ ☐ ☐ ☐ ☐ Quality-based incentive/disincentive features ☐ ☐ ☐ ☐ ☐ ☐ Warranties ☐ ☐ ☐ ☐ ☐ ☐ Optional warranties ☐ ☐ ☐ ☐ ☐ ☐ 3. Was your procurement method selected in part because of its effect on quality management? If so, what effect did the method you chose have on overall project quality? Design Phase: 4. Did this project have a formal design quality assurance program for any design performed in- house? ☐Yes ☐No 5. Did this project have a formal design quality assurance program for design performed by design consultants? ☐Yes ☐No 6. Did this project combine in-house design services with projects delivered by alternative methods (CMGC, DB, P3)? ☐Yes ☐No 196

For this project who performed the following design quality management tasks? (Check all that apply) Does not apply Agency design staff Agency project manage- ment staff Project design consult- ant Project con- struction staff in CMGC, DB, P3 Indepen- dent quality consultan t Other Please specify below Technical review of design deliverables ☐ ☐ ☐ ☐ ☐ ☐ ☐ Checking of design calculations ☐ ☐ ☐ ☐ ☐ ☐ ☐ Checking of quantities ☐ ☐ ☐ ☐ ☐ ☐ ☐ Acceptance of design deliverables ☐ ☐ ☐ ☐ ☐ ☐ ☐ Review of specifications ☐ ☐ ☐ ☐ ☐ ☐ ☐ Approval of final construction plans & other design documents ☐ ☐ ☐ ☐ ☐ ☐ ☐ Approval of progress payments for design progress ☐ ☐ ☐ ☐ ☐ ☐ ☐ Approval of post-award design QM/QA/QC plans ☐ ☐ ☐ ☐ ☐ ☐ ☐ Construction Phase: For this project who performed the following construction quality management tasks? (Check all that apply) Does not apply Agency design staff Agency project manage- ment staff Project design consult- ant Project con- struction staff in CMGC, DB, P3 Indepen- dent quality consultan t Other Please specify below Technical review of construction shop drawings ☐ ☐ ☐ ☐ ☐ ☐ ☐ Technical review of construction material submittals ☐ ☐ ☐ ☐ ☐ ☐ ☐ Checking of pay quantities ☐ ☐ ☐ ☐ ☐ ☐ ☐ Routine construction inspection ☐ ☐ ☐ ☐ ☐ ☐ ☐ Quality control testing ☐ ☐ ☐ ☐ ☐ ☐ ☐ Verification testing ☐ ☐ ☐ ☐ ☐ ☐ ☐ Acceptance testing ☐ ☐ ☐ ☐ ☐ ☐ ☐ Approval of progress payments for construction progress ☐ ☐ ☐ ☐ ☐ ☐ ☐ Approval of construction post-award QM/QA/QC plans ☐ ☐ ☐ ☐ ☐ ☐ ☐ Report of nonconforming work or punchlist. ☐ ☐ ☐ ☐ ☐ ☐ ☐ Quality Management Planning: Please answer the following questions about the project based on your experience. 12. Are the design QM plans used on this project different from the QM plans used on traditional design projects? 197

☐No ☐ Yes If yes, what is the major difference? Click here to enter text. 13. Are the construction QM plans used on this project different from the QM plans used on traditional DBB construction projects? ☐No ☐ Yes If yes, what is the major difference? Click here to enter text. 14. Did the agency mandate the use of standard agency specifications? ☐No ☐ Yes 15. Did the agency mandate the use of standard agency design details? ☐No ☐ Yes 16. Did the agency mandate the use of standard agency construction means and/or methods? ☐No ☐ Yes If no, what was required in their place? 17. Did the agency mandate a specific set of qualifications for the quality management staff of design consultants and construction contractors on this project? ☐No ☐ Yes If yes, what are those qualifications? Click here to enter text. 18. Did your agency utilize contractor quality assurance test results for acceptance on this project? ☐Yes. ☐No. Quality Management Procedures: 2. Do you think that the agency held the CMGC/design-builder/P3 concessionaire’s design/construction quality management staff to a higher standard of care than it sets for its internal staff? ☐Yes ☐No 198

Comments? Click here to enter text. 3. Does your organization have a document that outlines its approach to quality assurance on project? ☐Yes ☐No If yes, was it used on this project? If no, what was used in its place? 4. Does your agency use an approach to quality management that is substantially different than that used by other agencies and might be considered an “alternative QM system”? (i.e. statistical analysis of material test reports that eliminates the need for agency acceptance testing) ☐Yes ☐No Describe: 5. Which of the below best describes your agency’s approach to QA on this project? Interviewer: select appropriate delivery method DBB CMGC DB P3 ☐Design consultant primarily responsible for QA/Agency audits consultant program ☐Contractor primarily responsible for QA/Agency audits contractor program ☐Agency retains traditional QA roles ☐Agency retains an independent party to perform QA roles ☐Agency uses two or more of the above depending on the project ☐None of the above ☐Design consultant primarily responsible for QA/Agency audits consultant program ☐Contractor primarily responsible for QA/Agency audits designers and CMGC’s program ☐Agency retains traditional QA roles ☐Agency retains an independent party to perform QA roles ☐Agency uses two or more of the above depending on the project ☐None of the above ☐Design-builder primarily responsible for QA/Agency audits design-builder’s program ☐Agency retains traditional QA roles ☐Agency retains an independent party to perform QA roles ☐Agency uses two or more of the above depending on the project ☐None of the above ☐Concessionaire primarily responsible for QA/Agency audits concessionaire’s program ☐Agency retains traditional QA roles ☐Agency retains an independent party to perform QA roles ☐Agency uses two or more of the above depending on the project ☐None of the above If “None of the above” was selected, please describe the approach that was used instead: 6. What was the biggest quality challenge in the procurement phase? 199

7. What was the biggest quality challenge in the design phase? 8. What was the biggest quality challenge in the construction phase? 9. Please rate the following factors for their impact on the quality of this project: Factor Very High Impact High Impact Some Impact Slight Impact No Impact Qualifications of agency design staff ☐ ☐ ☐ ☐ ☐ Qualifications of agency project management staff ☐ ☐ ☐ ☐ ☐ Qualifications of agency construction staff ☐ ☐ ☐ ☐ ☐ Qualifications of the design consultant’s staff ☐ ☐ ☐ ☐ ☐ Design consultant’s past project experience ☐ ☐ ☐ ☐ ☐ Qualifications of the construction contractor’s staff ☐ ☐ ☐ ☐ ☐ Construction contractor’s past project experience ☐ ☐ ☐ ☐ ☐ Submittal of Quality management plans prior to work start ☐ ☐ ☐ ☐ ☐ Level of agency involvement in the QM process ☐ ☐ ☐ ☐ ☐ Use of agency specifications and/or design details ☐ ☐ ☐ ☐ ☐ Level of detail expressed in the procurement documents (IFB/RFQ/RFP) ☐ ☐ ☐ ☐ ☐ Use of manuals, standards and specifications developed for DBB type projects ☐ ☐ ☐ ☐ ☐ Allowing flexibility in choice of design standards and construction specifications ☐ ☐ ☐ ☐ ☐ Use of performance criteria/specifications ☐ ☐ ☐ ☐ ☐ Detailed design criteria ☐ ☐ ☐ ☐ ☐ Warranty provisions ☐ ☐ ☐ ☐ ☐ Incentive/disincentive provisions ☐ ☐ ☐ ☐ ☐ Follow-on maintenance provisions ☐ ☐ ☐ ☐ ☐ Innovative financing (PPP/concession) ☐ ☐ ☐ ☐ ☐ 200

C.8 Case Study Selection Matrix Case Study Selection Matrix Delivery Method Procurement Method # Case Study Name State Est. Value (millions) Pilot Design- Bid-Build Design- Build CM/GC PPP No Prequalifications Designer Prequalification Contractor Prequalification 1 Hastings River Bridge MN $ ☐ ☐ ☐ ☐ ☐ ☐ ☐ ☐ 2 Willamette River Bridge OR $ ☐ ☐ ☐ ☐ ☐ ☐ ☐ ☐ 3 $ ☐ ☐ ☐ ☐ ☐ ☐ ☐ ☐ 4 $ ☐ ☐ ☐ ☐ ☐ ☐ ☐ ☐ 5 $ ☐ ☐ ☐ ☐ ☐ ☐ ☐ ☐ 6 $ ☐ ☐ ☐ ☐ ☐ ☐ ☐ ☐ 7 $ ☐ ☐ ☐ ☐ ☐ ☐ ☐ ☐ 8 $ ☐ ☐ ☐ ☐ ☐ ☐ ☐ ☐

TRB’s National Cooperative Highway Research Program (NCHRP) Web-Only Document 212: Alternative Quality Management Systems for Highway Construction documents the research process, data collection and analysis used to develop NCHRP Report 808: Guidebook on Alternative Quality Management Systems for Highway Construction .

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Clin Transl Sci
• v.5(1); 2021

A protocol for retrospective translational science case studies of health interventions

Sara e. dodson.

1 Office of Science Policy and Planning, National Institute of Neurological Diseases and Stroke, National Institutes of Health, Department of Health and Human Services, Bethesda, MD, USA

2 Office of Evaluation, Performance and Reporting, Division of Program Coordination, Planning, and Strategic Initiatives, National Institutes of Health, Department of Health and Human Services, Bethesda, MD, USA

Linda Scholl

3 Office of Applied Scholarship and Education Science, Mayo Clinic College of Medicine and Science, Rochester, MN, USA

Clara M. Pelfrey

4 Clinical and Translational Science Collaborative, Case Western Reserve University, Cleveland, OH, USA

William M. Trochim

5 Clinical and Translational Science Center, Weill Cornell Medicine, Cornell University, New York, NY, USA

Associated Data

For supplementary material accompanying this paper visit https://doi.org/10.1017/cts.2020.514.

The critical processes driving successful research translation remain understudied. We describe a mixed-method case study protocol for analyzing translational research that has led to the successful development and implementation of innovative health interventions. An overarching goal of these case studies is to describe systematically the chain of events between basic, fundamental scientific discoveries and the adoption of evidence-based health applications, including description of varied, long-term impacts. The case study approach isolates many of the key factors that enable the successful translation of research into practice and provides compelling evidence connecting the intervention to measurable changes in health and medical practice, public health outcomes, and other broader societal impacts. The goal of disseminating this protocol is to systematize a rigorous approach, which can enhance reproducibility, promote the development of a large collection of comparable studies, and enable cross-case analyses. This approach, an application of the “science of translational science,” will lead to a better understanding of key research process markers, timelines, and potential points of leverage for intervention that may help facilitate decisions, processes, and policies to speed the sustainable translational process. Case studies are effective communication vehicles to demonstrate both accountability and the impacts of the public’s investment in research.

Introduction

The science of translational science seeks to understand the scientific and operational principles underlying each step of the translational research process [ 1 ]. While the translational process is not linear, several distinct phases of research are typically operationalized, for example, basic, preclinical, clinical, clinical implementation, and public health research, with critical translation efforts required to move knowledge between each phase. To systematically assess the complex translational process, several promising formative and summative research evaluation approaches, including quantitative, qualitative, and mixed methodologies, have been developed in recent years [ 2 , 3 ]. Case studies frequently are used as tools for research evaluation because they provide a rigorous way to explain understudied practices, and they are an effective mechanism for identifying long-term outcomes of scientific research [ 4 – 7 ]. In addition, researchers who study the key processes and outcomes of scientific endeavors are continually refining frameworks for assessing scientific research impact [ 6 , 8 – 14 ].

A systematic translational science case study approach is currently lacking. This article fills this gap by providing a specific protocol for conducting case studies to evaluate the translational research processes underlying the development of successful health interventions. This protocol allows researchers to apply a common approach and generate comparable insights. The authors recommend Robert Yin’s textbook “Case Study Research and Applications: Design and Methods” for a full exploration of the theoretical foundations of the case study methodology [ 15 ]. Yin describes how case studies are an excellent evaluation tool as they allow for the combined use of qualitative and quantitative data, providing an in-depth examination of the factors that contributed to the success of specific research activities, as well as factors that may hinder success [ 15 – 18 ]. The process of conducting case studies requires an open and flexible approach that is driven by the unique case being studied. Case studies can capture a wide variety of impacts, including the unexpected, and can provide context about the evolution of research that may not be apparent in a review of outcomes. Case studies are particularly valuable in describing whether and how certain activities and contributors were pivotal in advancing science and improving public health outcomes [ 6 , 19 ]. Translational case study researchers often face the challenge of keeping the case focused on a specific and time-bound translational intervention and its evolution. Case study researchers must continually make decisions about which elements are and are not fundamental to the story.

Case studies, like all methodologies, have limitations. Results from a single case may not be generalizable [ 18 ]. Our case study protocol is not meant to cover the full scope of a research program, but rather to capture the central elements in the discovery and development of a specific health intervention. As such, it may potentially filter out aspects of a larger field of research in which the case is situated. In addition, case studies are data-intensive, time-consuming, require expert input to highlight the most important factors, and are susceptible to subjective interpretation [ 20 ]. Despite such limitations, case studies are arguably the most comprehensive way to study complex systems.

The sections below provide guidance on selecting cases for study; the key elements, themes, and analyses that are needed to develop the cases; as well as methods and data sources useful in conducting case studies. We encourage adoption of this protocol by diverse researchers working in any number of fields who focus on understanding the scientific process and research outcomes. The case study approach is most effective when findings are made accessible to broad communities, in particular because successful translation is often impeded by a lack of common vernacular between research disciplines, practice communities, and policymakers. Ultimately, the goal is to develop a collection of comparably-conducted case studies, enabling cross-case analyses that could inform the “science of translational science” to address questions such as: What processes tend to drive translation forward and in which contexts? What challenges can be anticipated with particular types of research studies? How can such challenges be addressed early to avoid delays in successful implementation? How can resources from research institutions and funders be directed to maximally support translational research efforts?

Translational Case Study Protocol

Case selection.

Appropriate case types include evidence-based interventions which have generated discernible health impacts, such as a specific technology, diagnostic, preventive, drug, device, biologic, behavioral intervention, or other treatment strategy. Cases should be examples of successful translation across the full continuum of research to practice, where the generation of knowledge falls within a definable range between inception of the intervention and its impacts . The selected intervention should be currently in use in medical or public health settings and there should be evidence that the intervention improves health outcomes, increases life expectancy, and/or improves individuals’ quality of life.

Case studies of research translation that have not progressed yet to impact, but which show strong potential for future impact, are also valuable, as are studies that examine “unsuccessful” aspects of research translation. However, unsuccessful research is much harder to study because there are very few negative studies published, and researchers are reluctant to highlight their failures. While this protocol may be adapted to the study of partial and not-yet-successful research translation efforts, the focus here is on the assessment of interventions that have been successfully implemented into practice.

Case Study Elements

The case study consists of two central elements: (1) a detailed timeline of the major events and milestones that marked the translational progress, and (2) a broader narrative that describes how and why such progress happened. The timeline and narrative should complement each other and contain overlapping content. Both should include detailed documentation of sources that support the central elements. A more detailed description of the translational case study elements is presented in Appendix A (in Supplementary Material).

The case study timeline

Case studies should include a timeline, or multiple interconnected timelines where warranted, which serves as a key graphic for organizing and communicating the case’s central information. The timeline is a universally understood device for visualizing temporal translational progression, for example, “distance” between milestones and complex cause and effect relationships. The timeline progresses in phases along one or more pathways and is punctuated by multiple milestones that help to anchor the case study’s chronological story. While timelines are typically linear, we recognize that the translational process often moves backward and forward through different phases and may have parallel storylines. A timeline is particularly effective for describing such parallel storylines and illustrating critical points of convergence and divergence. Organizing elements of the timeline include the following:

• 1. Start and end points of case study : Translational case studies link the chain of evidence from scientific observations to verifiable impacts of the intervention on health. The discreet start and end points of the case should be identified and a rationale for why those points were chosen should be presented. See Appendix A for a detailed discussion and guiding questions on how to select appropriate start and end points. Briefly, selecting the most relevant and appropriate start and end points for any translational case will be subjective and may be challenging. The start point is typically defined as the inception of the particular innovation being described in the case study and its association with a “target” such as a disease or diagnosis. Discussion of the start point chosen for the case study may point to foundational research knowledge that was essential for conceptualizing an effective intervention, perhaps going back decades or more in the research literature. The case study should address how far into practice the intervention has gone and the end point should represent a concrete outcome that has taken place in medical or public health practice. Outcomes may include how the intervention was implemented or otherwise “packaged” for scaling up. If the intervention is a drug, device, or biologic, evidence of adoption into practice should be included, if available. The development and adoption of an intervention in clinical and community settings may continue to evolve far beyond the chosen end point of the case study. When relevant, the start and end points need to be described in the larger context of scientific progress and may require additional relevant historical, social, and political context.
• 2. Progress markers/milestones : Markers or milestones are integrally related to key events that occur during the translational process, including the start and end points of study [ 21 ]. Markers are anchored on specific dates and can be represented as points or intervals on a timeline. Markers should be chosen for their ability to help tell the story of the development and translation of the intervention. Different types of markers include the following: (1) major inputs – resources, human and intellectual capital, and so on; (2) key activities and events – major meetings, formation of a collaboration or partnership, serendipitous events, interim research milestones, and so on; and (3) major products or outputs – presentations, publications, clinical trials results, drug approvals, markers of commercialization, changes in practice, changes in public health measures, and any other evidence of adoption of the intervention into practice.
• 3. Translational phases : It is useful to group a complex translational timeline visually into general research phases. Numerous similar multiphase schemes of translation have been proposed, but there is currently no universally accepted typology [ 21 ]. One model for these case studies is the National Institutes of Health (NIH) National Center for Advancing Translational Sciences’ translational phase model [ 22 ], which includes the following research phases: basic, preclinical, clinical, clinical implementation, and public health. Another useful translational research framework comes from the NIH’s National Institute of Environmental Health Sciences (NIEHS) [ 23 ]. The NIEHS framework was developed specifically to aid researchers in describing the evolution of their translational research in the area of environmental health. See Appendix B (in Supplementary Material) for suggested definitions and parameters for delineating research phases. Depending on the case, it may be necessary to apply a different translational model; however, whatever schema is used, it should clearly identify the markers and milestones that distinguish each phase. Well-written case studies should help reveal where there are intersecting points and gray areas between the discrete phases.

The case study narrative

The second central element of the case study is the narrative, which provides a coherent summary that moves the reader through the translational science process, describing the major actors, themes, forces, pivotal events, and advances that influenced the translational process. The narrative should focus on describing how and why the intervention developed as it did, how and why the markers/milestones were achieved, as well as what challenges were encountered and how those challenges were addressed. These drivers of translation may arise directly from key documents and/or interviews with the central researchers and stakeholders (e.g., funders, community advocates, or practitioners). They may also arise indirectly through an analysis of the information gathered throughout the course of researching the case study. Given the interdisciplinary nature of translational research, the narrative should avoid discipline-specific jargon and instead should use easily understood language. In addition, case study authors may want to consider writing the narrative for different target audiences, including a lay audience (see section “Formats of Finalized Case Study Materials” for a broader discussion of case study audiences and formats).

The case study narrative key elements

• 1. Health problem and relevance of the intervention : The case study narrative should begin with: (a) background on the relevant disease(s), disorder(s), or public health challenge(s), including some measure(s) of burden to help communicate the scope of the problem; (b) a description of the intervention; (c) relevant historical, social, and political context; and (d) a brief summary of the impact.
• 2. Key events: The key events are the scaffolding of the case study narrative and often correspond with the timeline progress markers/milestones, including the start and end points. They constitute the heart of the chronological story, describing the sequence of integral events.
• 3. Key people and partnerships : Over the course of translating an intervention from inception to impact, there are many individuals and groups who play important roles in the research progress. In a case study, determining which key actors are discussed, and why, requires careful judgment and should be backed by objective evidence. The case study should highlight individuals and sectors across the health research and practice ecosystem. This should include the central researchers and teams as well as those who were integral in disseminating and implementing the intervention, in the commercial or nonprofit development of the intervention, and in enabling broad uptake and adoption of the intervention. In addition, there should be a description of how and why different individuals collaborated with each other and what role those collaborations played in the development and implementation of the intervention. Collaborative relationships are often influenced by surrounding organizational culture(s) in ways that may be conducive or disruptive to the success of collaborative research endeavors. Where relevant, consider examining the characteristics of the organizational climate that helped create and support key collaborations.
• 4. Other influencing factors: There are many other factors that can influence a translational research process. Case studies should include descriptions of major facilitators and barriers, both expected and unexpected. Facilitators may include critical support and infrastructure; influential policies; transformative technologies, tools, and techniques; and knowledge or strategies borrowed from tangential lines of research. Major barriers or challenges should also be described, including failed or abandoned research directions, and how those difficulties were overcome. In addition, there may be critical contextual factors that influenced translational progress, such as historical, political, and other social events or changes. Well-designed interview questions are particularly useful to draw out influential factors that may not be obvious to those centrally involved in the research nor readily apparent from records and other archival materials.

Impacts will rarely be fully attributable to the case study; in most cases, they will be influenced by many additional moderating factors not covered in the study. Therefore, case study researchers should avoid “over-crediting” their findings, and should provide compelling evidence that the central factors identified have played a critical role.

• 6. Further developments : Case study narratives should conclude with a description of how the research, dissemination, and/or implementation is currently progressing (or could progress); analysis of the remaining knowledge gaps and work that still needs to be done; and/or any important postscripts to the case study.

Translational Case Study Methodology

The methodology for conducting a case study is an iterative process that progressively fills in the case details until no significant additional factors emerge. The methodology described here builds on research approaches used by social scientists, political scientists, historians, and even criminal investigators and investigative journalists. These approaches involve skills that include objectivity, analytical skills, interviewing skills, using mixed-methods, doing literature searches, consulting multiple data sources, and constructing a narrative. The list of methodological steps provided below is not intended to be strictly linear; steps can be revisited as information accumulates. For example, while most case studies will begin with defining the start and end points, these are likely to be revised over time as new information and insights arise. This iterative process allows the timeline to be a key methodological tool to tell the narrative and to identify remaining knowledge gaps.

Methodological steps may include the following:

• a. background on health issue/disease being addressed;
• b. background on key researchers and research team(s);
• c. information about the development, testing, and implementation of the intervention, including key process markers (grants, FDA approvals, clinical trials, patents, publications, research syntheses/meta-analyses, recommendations/guidelines); and
• d. evidence of accrued or potential impacts. Useful information gathering approaches include web searches (including websites maintained by research funders, news media, researchers, industry, health/patient advocacy organizations, professional societies, and so on), literature searches, and other database searches (e.g., for relevant grants, patents, clinical trials, population health data).
• 2. Create an initial timeline, identifying start and end points and chronologically mapping the progress markers in the translational research process.
• 3. Identify initial gaps in data/information.
• 4. Identify an initial list of key stakeholders, including individuals responsible for translational research progress, as well as others who may have strong historical perspective and subject matter knowledge.
• 5. Conduct semistructured interviews with selected stakeholders.
• 6. Continue gathering data until no significant additional details or factors emerge.
• 7. Return case study analyses back to interviewees/key stakeholders to validate and ensure accuracy and completeness.

Retrospective translational case study reporting template outline, which can be used as a guide when writing up a case study.

Data analysis will use a variety of appropriate quantitative and qualitative techniques, which may include:

Record review : A variety of records can be used to stitch together the timeline and narrative, including primary and secondary research literature; grant records; press releases and other media items; policy statements; legal and regulatory documents; program and service development announcements; clinical trials; changes in clinical practice guidelines; FDA approvals; patent records; and health service research findings. Often, research and review articles written by the developers of the intervention can be a valuable resource. However, care should be taken to avoid biasing the story toward certain research teams over other potentially pivotal contributors.

Interviews with key researchers and other stakeholders are critical for the verification of timeline and story, the identification of central themes and key contextual factors such as facilitators and barriers, and for filling in gaps in the story. Relevant stakeholders include research investigators and trainees, organizational leaders, health and medical practitioners, community health advocates, and patients. The process of conducting semistructured interviews is at the heart of the case study. Wherever possible, it is helpful to develop multiple independent corroborating interviews. Most stakeholders will agree about what the key markers/milestones were; however, there may be competing and irreconcilable stories about how the case evolved. In that situation, alternative stories should be noted rather than forced into a single coherent interpretation. See Appendix C (in Supplementary Material) for key themes to guide the interview and sample interview questions; however, interview guides should be tailored to the case and interviewee. In particular, question probes can be very helpful in encouraging interviewees to place their efforts, understanding, and opinions within the broader context.

Bibliographic, bibliometric, and grant portfolio analyses should be used as a rigorous and data-driven approach to identify and validate translational research milestones, central researchers and research funders, levels of research funding, and influential research collaborations. Case study researchers should consider applying these approaches to identify different inputs (such as funding, time, human capital, research infrastructure, equipment/technology, other research resources, and/or partners) as well as different outputs (knowledge generated, patents generated, etc.) that were critical for development of long-term outcomes. A detailed examination of research grant records can provide valuable information on key resources and pivotal research funders. Both the NIH RePORTER webtool [ 25 ] and the Federal RePORTER webtool are searchable databases of scientific awards from several federal agencies [ 25 , 26 ].

Relevant to measuring research outputs, bibliometric approaches include several techniques for assessing the quantity, extent of dissemination, and content of publications and patents [ 2 ]. For example, bibliographic analysis from an identified clinical trial, grant, or seminal publication can be used to indicate the number of times an article has been cited and in what topic area. The pattern of citations can show the influence of key publications and help provide evidence of knowledge links over time. Network analyses of publication citations can reveal key researchers/research teams, findings, and collaborations. A valuable free bibliometric tool for conducting translational case studies is the “Translational Module” in iCite, a machine learning model that tracks the flow of knowledge into clinical medicine [ 27 , 28 ]. (Details on websites for searching NIH/Federal grants, patents, and bibliometric sources are provided in Appendix B.)

Review and analysis of health data should be performed where possible, including review of primary and secondary literature. For example, population health publications and databases can be used to describe changes in disease incidence and severity. In addition, healthcare utilization data may exist (e.g., from enterprise healthcare databases), providing information on health and medical spending. Analyzing federal, state, and health organization policies can help determine how the case may have influenced policy changes.

See Appendix B for a collection of useful data resources organized by research and practice phase as well as type of impact.

Formats of Finalized Case Study Materials

While the gold-standard for disseminating case study results is publishing in a peer-reviewed research journal, other formats should be considered for dissemination to wider audiences. For example, the NIH has posted several case studies on the Web in an effort to help communicate the value of biomedical research to the general public [ 29 ]. These web materials include brief summaries of the broader case study narrative as well as graphics (e.g., stylized timelines, figures, infographics), and detailed documentation and supplemental materials with further information.

There are several potential audiences for these case studies, including various research communities, research funders, health practitioners, domestic and international research policymakers, patient communities, and the general public. While it is encouraged to construct case studies for diverse audiences, findings can also be packaged in a variety of formats aimed at specific audiences. Each of these groups may have different needs, value different aspects of health research, and respond to different types of approaches for disseminating information.

Future Developments Toward Enhancing the Science of Translational Science

This protocol is intended to be an evolving document, which can be updated and refined as exemplary practices in the science of translational science emerge. This is one of several complementary efforts to enhance the identification of translational science success factors and provide a sustainable and useful framework for both the scholarly and practical study of translational science. Together with this protocol, we encourage the development of (1) appropriate publication outlets, (2) archiving strategies, and (3) coding schemes, to ensure high-quality work that is accessible for further research, including comparative studies.

• Publication outlets and review : We encourage the development of a special category of publication – the translational research case study – that could be included on an ongoing basis in appropriate journals. We believe a recognized publication mechanism would go a long way toward enhancing the consistency, quality, and accessibility of such reports. Independent peer review of such publications would help assure quality. Conducting case studies is time consuming; therefore, a recognized publication type would help overcome that barrier by providing professional and academic value in the form of creditable publications.
• Coding : Developing a coding scheme for translational research case studies would enhance their subsequent retrieval and meta-analyses. Each case study could be coded on a number of standard variables, including classification of the type of intervention, disease/disorder/public health research area, populations affected, key markers/milestones, key themes identified, outcomes achieved, and the translational stages covered by the case. No such classification system currently exists, and we should develop one inductively after accumulating a sufficient number of cases. A simple coding/classification scheme would be desirable to include as part of the case study report. Such a scheme would resemble the classifications used to store clinical trials in www.clinicaltrials.gov , such as the Medical Subject Headings (MeSH) coding scheme managed through the National Library of Medicine [ 30 ].
• Archiving : A central archiving repository for translational case studies would provide (1) a certifying mechanism for the quality of cases archived; (2) a motivational mechanism to encourage researchers to contribute to the literature of case studies; and (3) a database to support meta-analyses that could lead to broader generalizations about the factors that influence successful translational research. This repository would include rich meta-data on each study (e.g., coding data described above). Over time, such an archive would help to establish a body of comparable studies supporting research on translation and enable valuable cross-case analyses.

Conclusions

This protocol provides a systematic approach to study the multifaceted processes of translating research findings into practice. Researchers from diverse backgrounds can follow this protocol to analyze how scientific knowledge is translated into effective health interventions, identifying critical factors that enhance or impede progress along the way. This framework can be used to assess the long-range impacts of successful translational science efforts as well as examples of incomplete implementation. The combined analysis of successes and failures should inform funding agencies regarding designing grant mechanisms and investing in future translational research.

With this protocol, we hope to generate excitement for the broad conduct of translational science case studies. Dissemination of this protocol is a first step toward enabling a novel, coordinated approach for this application of the science of translational science. While rigorous, well-researched case studies are individually valuable, the most exciting use of this protocol lies in cross-case analyses, to identify leverage points and exemplary practices, as well as theories of change developed for further empirical testing. To realize this potential, several complementary efforts should be pursued, including the establishment of dedicated publication outlets, coding schemas, and a case study archive. All of this will take time to realize fully, but the long-term payoffs will be worth the effort when we can demonstrate that insights from these case studies can be used to enhance translational research and speed the delivery of life-saving medical and health interventions.

Acknowledgements

Thanks to Laura Hogan, Science Editor, at the Institute for Clinical and Translational Research at the University of Wisconsin-Madison for her thorough edit of the final manuscript draft. Thanks to Arthur E. Blank, Associate Professor Emeritus at Albert Einstein College of Medicine, for the idea and motivation to begin a translational science case studies group among CTSA evaluators. Several individuals contributed to the design of NIH’s case study methodology and conducted case studies to test, refine, and demonstrate the approach, including David Kosub, PhD (NIH), Elizabeth Baden, PhD (NICHD), Kristine Alexander, PhD (Cambia Health Solutions), Joel Baumgart, PhD (Emory University), and Peter Reczek, PhD (Standards Coordinating Body for Regenerative Medicine). Substantive review and comment on the manuscript was provided by Meredith D. Temple-O’Connor, PhD (NCATS), Kristianna Pettibone, PhD (NIEHS), Sue Hamman, PhD (NIDCR), Bob Zalutsky, PhD (NINDS), and Paul Scott, PhD (NINDS). In particular, we thank Marina Volkov, PhD (NIH) and David Bochner, PhD (NIDA) who contributed substantially to the case study design and provided critical comments on the draft manuscript.

Grant numbers: UL1 TR002548 (CMP), Clinical and Translational Science Collaborative (CTSC) of Cleveland, Case Western Reserve University. UL1TR002373 (LMS), Institute for Clinical and Translational Research (ICTR) at University of Wisconsin, Madison. UL1TR000457 (WMT) Clinical and Translational Science Center (CTSC) of Weill Cornell Medicine.

Supplementary material

Disclosures.

The authors have no conflicts of interest to declare.