what is a document review in research

Document Analysis - How to Analyze Text Data for Research

Introduction

What is document analysis, where is document analysis used, how to perform document analysis, what is text analysis, atlas.ti as text analysis software.

In qualitative research , you can collect primary data through surveys , observations , or interviews , to name a few examples. In addition, you can rely on document analysis when the data already exists in secondary sources like books, public reports, or other archival records that are relevant to your research inquiry.

In this article, we will look at the role of document analysis, the relationship between document analysis and text analysis, and how text analysis software like ATLAS.ti can help you conduct qualitative research.

Document analysis is a systematic procedure used in qualitative research to review and interpret the information embedded in written materials. These materials, often referred to as “documents,” can encompass a wide range of physical and digital sources, such as newspapers, diaries, letters, policy documents, contracts, reports, transcripts, and many others.

At its core, document analysis involves critically examining these sources to gather insightful data and understand the context in which they were created. Research can perform sentiment analysis , text mining, and text categorization, to name a few methods. The goal is not just to derive facts from the documents, but also to understand the underlying nuances, motivations, and perspectives that they represent. For instance, a historical researcher may examine old letters not just to get a chronological account of events, but also to understand the emotions, beliefs, and values of people during that era.

Benefits of document analysis

There are several advantages to using document analysis in research:

• Authenticity : Since documents are typically created for purposes other than research, they can offer an unobtrusive and genuine insight into the topic at hand, without the potential biases introduced by direct observation or interviews.
• Availability : Documents, especially those in the public domain, are widely accessible, making it easier for researchers to source information.
• Cost-effectiveness : As these documents already exist, researchers can save time and resources compared to other data collection methods.

However, document analysis is not without challenges. One must ensure the documents are authentic and reliable. Furthermore, the researcher must be adept at discerning between objective facts and subjective interpretations present in the document.

Document analysis is a versatile method in qualitative research that offers a lens into the intricate layers of meaning, context, and perspective found within textual materials. Through careful and systematic examination, it unveils the richness and depth of the information housed in documents, providing a unique dimension to research findings.

Document analysis is employed in a myriad of sectors, serving various purposes to generate actionable insights. Whether it's understanding customer sentiments or gleaning insights from historical records, this method offers valuable information. Here are some examples of how document analysis is applied.

Analyzing surveys and their responses

A common use of document analysis in the business world revolves around customer surveys . These surveys are designed to collect data on the customer experience, seeking to understand how products or services meet or fall short of customer expectations.

By analyzing customer survey responses , companies can identify areas of improvement, gauge satisfaction levels, and make informed decisions to enhance the customer experience. Even if customer service teams designed a survey for a specific purpose, text analytics of the responses can focus on different angles to gather insights for new research questions.

Examining customer feedback through social media posts

In today's digital age, social media is a goldmine of customer feedback. Customers frequently share their experiences, both positive and negative, on platforms like Twitter, Facebook, and Instagram.

Through document analysis of social media posts, companies can get a real-time pulse of their customer sentiments. This not only helps in immediate issue resolution but also in shaping product or service strategies to align with customer preferences.

Interpreting customer support tickets

Another rich source of data is customer support tickets. These tickets often contain detailed descriptions of issues faced by customers, their frustrations, or sometimes their appreciation for assistance received.

By employing document analysis on these tickets, businesses can detect patterns, identify recurring issues, and work towards streamlining their support processes. This ensures a smoother and more satisfying customer experience.

Historical research and social studies

Beyond the world of business, document analysis plays a pivotal role in historical and social research. Scholars analyze old manuscripts, letters, and other archival materials to construct a narrative of past events, cultures, and civilizations.

As a result, document analysis is an ideal method for historical research since generating new data is less feasible than turning to existing sources for analysis. Researchers can not only examine historical narratives but also how those narratives were constructed in their own time.

Turn to ATLAS.ti for your data analysis needs

Try out our powerful data analysis tools with a free trial to make the most out of your data today.

Performing document analysis is a structured process that ensures researchers can derive meaningful, qualitative insights by organizing source material into structured data . Here's a brief outline of the process:

• Define the research question
• Choose relevant documents
• Prepare and organize the documents
• Begin initial review and coding
• Analyze and interpret the data
• Present findings and draw conclusions

The process in detail

Before diving into the documents, it's crucial to have a clear research question or objective. This serves as the foundation for the entire analysis and guides the selection and review of documents. A well-defined question will focus the research, ensuring that the document analysis is targeted and relevant.

The next step is to identify and select documents that align with the research question. It's vital to ensure that these documents are credible, reliable, and pertinent to the research inquiry. The chosen materials can vary from official reports, personal diaries, to digital resources like social media data , depending on the nature of the research.

Once the documents are selected, they need to be organized in a manner that facilitates smooth analysis. This could mean categorizing documents by themes, chronology, or source types. Digital tools and data analysis software , such as ATLAS.ti, can assist in this phase, making the organization more efficient and helping researchers locate specific data when needed.

With everything in place, the researcher starts an initial review of the documents. During this phase, the emphasis is on identifying patterns, themes, or specific information relevant to the research question.

Coding involves assigning labels or tags to sections of the text to categorize the information. This step is iterative, and codes can be refined as the researcher delves deeper.

After coding, interesting patterns across codes can be analyzed. Here, researchers seek to draw meaningful connections between codes, identify overarching themes, and interpret the data in the context of the research question .

This is where the hidden insights and deeper understanding emerge, as researchers juxtapose various pieces of information and infer meaning from them.

Finally, after the intensive process of document analysis, the researcher consolidates their findings, crafting a narrative or report that presents the results. This might also involve visual representations like charts or graphs, especially when demonstrating patterns or trends.

Drawing conclusions involves synthesizing the insights gained from the analysis and offering answers or perspectives in relation to the original research question.

Ultimately, document analysis is a meticulous and iterative procedure. But with a clear plan and systematic approach, it becomes a potent tool in the researcher's arsenal, allowing them to uncover profound insights from textual data.

Text analysis, often referenced alongside document analysis, is a method that focuses on extracting meaningful information from textual data. While document analysis revolves around reviewing and interpreting data from various sources, text analysis hones in on the intricate details within these documents, enabling a deeper understanding. Both these methods are vital in fields such as linguistics, literature, social sciences, and business analytics.

In the context of document analysis, text analysis emerges as a nuanced exploration of the textual content. After documents have been sourced, be it from books, articles, social networks, or any other medium, they undergo a preprocessing phase. Here, irrelevant information is eliminated, errors are rectified, and the text may be translated or converted to ensure uniformity.

This cleaned text is then tokenized into smaller units like words or phrases, facilitating a granular review. Techniques specific to text analysis, such as topic modeling to determine discussed subjects or pattern recognition to identify trends, are applied.

The derived insights can be visualized using tools like graphs or charts, offering a clearer understanding of the content's depth. Interpretation follows, allowing researchers to draw actionable insights or theoretical conclusions based on both the broader document context and the specific text analysis.

Merging text analysis with document analysis presents unique challenges. With the proliferation of digital content, managing vast data sets becomes a significant hurdle. The inherent variability of language, laden with cultural nuances, idioms, and sometimes sarcasm, can make precise interpretation elusive.

Many text analysis tools exist that can facilitate the analytical process. ATLAS.ti offers a well-rounded, useful solution as a text analytics software . In this section, we'll highlight some of the tools that can help you conduct document analysis.

Word Frequencies

A word cloud can be a powerful text analytics tool to understand the nature of human language as it pertains to a particular context. Researchers can perform text mining on their unstructured text data to get a sense of what is being discussed. The Word Frequencies tool can also parse out specific parts of speech, facilitating more granular text extraction.

Sentiment Analysis

The Sentiment Analysis tool employs natural language processing (NLP) and machine learning to analyze text based on sentiment and facilitate natural language understanding. This is important for tasks such as, for example, analyzing customer reviews and assessing customer satisfaction, because you can quickly categorize large numbers of customer data records by their positive or negative sentiment.

AI Coding relies on massive amounts of training data to interpret text and automatically code large amounts of qualitative data. Rather than read each and every document line by line, you can turn to AI Coding to process your data and devote time to the more essential tasks of analysis such as critical reflection and interpretation.

These text analytics tools can be a powerful complement to research. When you're conducting document analysis to understand the meaning of text, AI Coding can help with providing a code structure or organization of data that helps to identify deeper insights.

AI Summaries

Dealing with large numbers of discrete documents can be a daunting task if done manually, especially if each document in your data set is lengthy and complicated. Simplifying the meaning of documents down to their essential insights can help researchers identify patterns in the data.

AI Summaries fills this role by using natural language processing algorithms to simplify data to its salient points. Text generated by AI Summaries are stored in memos attached to documents to illustrate pathways to coding and analysis or to highlight how the data conveys meaning.

Take advantage of ATLAS.ti's analysis tools with a free trial

Let our powerful data analysis interface make the most out of your data. Download a free trial today.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Health Policy Plan
• v.35(10); 2020 Dec

Document analysis in health policy research: the READ approach

Sarah l dalglish.

1 Department of International Health, Johns Hopkins School of Public Health, 615 N. Wolfe St, Baltimore, MD 21205, USA

2 Institute for Global Health, University College London, Institute for Global Health 3rd floor, 30 Guilford Street, London WC1N 1EH, UK

Hina Khalid

3 School of Humanities and Social Sciences, Information Technology University, Arfa Software Technology Park, Ferozepur Road, Lahore 54000, Pakistan

Shannon A McMahon

4 Heidelberg Institute of Global Health, Medical Faculty and University Hospital, University of Heidelberg, Im Neuenheimer Feld 130/3, 69120 Heidelberg, Germany

Associated Data

Document analysis is one of the most commonly used and powerful methods in health policy research. While existing qualitative research manuals offer direction for conducting document analysis, there has been little specific discussion about how to use this method to understand and analyse health policy. Drawing on guidance from other disciplines and our own research experience, we present a systematic approach for document analysis in health policy research called the READ approach: (1) ready your materials, (2) extract data, (3) analyse data and (4) distil your findings. We provide practical advice on each step, with consideration of epistemological and theoretical issues such as the socially constructed nature of documents and their role in modern bureaucracies. We provide examples of document analysis from two case studies from our work in Pakistan and Niger in which documents provided critical insight and advanced empirical and theoretical understanding of a health policy issue. Coding tools for each case study are included as Supplementary Files to inspire and guide future research. These case studies illustrate the value of rigorous document analysis to understand policy content and processes and discourse around policy, in ways that are either not possible using other methods, or greatly enrich other methods such as in-depth interviews and observation. Given the central nature of documents to health policy research and importance of reading them critically, the READ approach provides practical guidance on gaining the most out of documents and ensuring rigour in document analysis.

Key Messages

• Rigour in qualitative research is judged partly by the use of deliberate, systematic procedures; however, little specific guidance is available for analysing documents, a nonetheless common method in health policy research.
• Document analysis is useful for understanding policy content across time and geographies, documenting processes, triangulating with interviews and other sources of data, understanding how information and ideas are presented formally, and understanding issue framing, among other purposes.
• The READ (Ready materials, Extract data, Analyse data, Distil) approach provides a step-by-step guide to conducting document analysis for qualitative policy research.
• The READ approach can be adapted to different purposes and types of research, two examples of which are presented in this article, with sample tools in the Supplementary Materials .

Introduction

Document analysis (also called document review) is one of the most commonly used methods in health policy research; it is nearly impossible to conduct policy research without it. Writing in early 20th century, Weber (2015) identified the importance of formal, written documents as a key characteristic of the bureaucracies by which modern societies function, including in public health. Accordingly, critical social research has a long tradition of documentary review: Marx analysed official reports, laws, statues, census reports and newspapers and periodicals over a nearly 50-year period to come to his world-altering conclusions ( Harvey, 1990 ). Yet in much of social science research, ‘documents are placed at the margins of consideration,’ with privilege given to the spoken word via methods such as interviews, possibly due to the fact that many qualitative methods were developed in the anthropological tradition to study mainly pre-literate societies ( Prior, 2003 ). To date, little specific guidance is available to help health policy researchers make the most of these wells of information.

The term ‘documents’ is defined here broadly, following Prior, as physical or virtual artefacts designed by creators, for users, to function within a particular setting ( Prior, 2003 ). Documents exist not as standalone objects of study but must be understood in the social web of meaning within which they are produced and consumed. For example, some analysts distinguish between public documents (produced in the context of public sector activities), private documents (from business and civil society) and personal documents (created by or for individuals, and generally not meant for public consumption) ( Mogalakwe, 2009 ). Documents can be used in a number of ways throughout the research process ( Bowen, 2009 ). In the planning or study design phase, they can be used to gather background information and help refine the research question. Documents can also be used to spark ideas for disseminating research once it is complete, by observing the ways those who will use the research speak to and communicate ideas with one another.

Documents can also be used during data collection and analysis to help answer research questions. Recent health policy research shows that this can be done in at least four ways. Frequently, policy documents are reviewed to describe the content or categorize the approaches to specific health problems in existing policies, as in reviews of the composition of drowning prevention resources in the United States or policy responses to foetal alcohol spectrum disorder in South Africa ( Katchmarchi et al. , 2018 ; Adebiyi et al. , 2019 ). In other cases, non-policy documents are used to examine the implementation of health policies in real-world settings, as in a review of web sources and newspapers analysing the functioning of community health councils in New Zealand ( Gurung et al. , 2020 ). Perhaps less frequently, document analysis is used to analyse policy processes, as in an assessment of multi-sectoral planning process for nutrition in Burkina Faso ( Ouedraogo et al. , 2020 ). Finally, and most broadly, document analysis can be used to inform new policies, as in one study that assessed cigarette sticks as communication and branding ‘documents,’ to suggest avenues for further regulation and tobacco control activities ( Smith et al. , 2017 ).

This practice paper provides an overarching method for conducting document analysis, which can be adapted to a multitude of research questions and topics. Document analysis is used in most or all policy studies; the aim of this article is to provide a systematized method that will enhance procedural rigour. We provide an overview of document analysis, drawing on guidance from disciplines adjacent to public health, introduce the ‘READ’ approach to document analysis and provide two short case studies demonstrating how document analysis can be applied.

What is document analysis?

Document analysis is a systematic procedure for reviewing or evaluating documents, which can be used to provide context, generate questions, supplement other types of research data, track change over time and corroborate other sources ( Bowen, 2009 ). In one commonly cited approach in social research, Bowen recommends first skimming the documents to get an overview, then reading to identify relevant categories of analysis for the overall set of documents and finally interpreting the body of documents ( Bowen, 2009 ). Document analysis can include both quantitative and qualitative components: the approach presented here can be used with either set of methods, but we emphasize qualitative ones, which are more adapted to the socially constructed meaning-making inherent to collaborative exercises such as policymaking.

The study of documents as a research method is common to a number of social science disciplines—yet in many of these fields, including sociology ( Mogalakwe, 2009 ), anthropology ( Prior, 2003 ) and political science ( Wesley, 2010 ), document-based research is described as ill-considered and underutilized. Unsurprisingly, textual analysis is perhaps most developed in fields such as media studies, cultural studies and literary theory, all disciplines that recognize documents as ‘social facts’ that are created, consumed, shared and utilized in socially organized ways ( Atkinson and Coffey, 1997 ). Documents exist within social ‘fields of action,’ a term used to designate the environments within which individuals and groups interact. Documents are therefore not mere records of social life, but integral parts of it—and indeed can become agents in their own right ( Prior, 2003 ). Powerful entities also manipulate the nature and content of knowledge; therefore, gaps in available information must be understood as reflecting and potentially reinforcing societal power relations ( Bryman and Burgess, 1994 ).

Document analysis, like any research method, can be subject to concerns regarding validity, reliability, authenticity, motivated authorship, lack of representativity and so on. However, these can be mitigated or avoided using standard techniques to enhance qualitative rigour, such as triangulation (within documents and across methods and theoretical perspectives), ensuring adequate sample size or ‘engagement’ with the documents, member checking, peer debriefing and so on ( Maxwell, 2005 ).

Document analysis can be used as a standalone method, e.g. to analyse the contents of specific types of policy as they evolve over time and differ across geographies, but document analysis can also be powerfully combined with other types of methods to cross-validate (i.e. triangulate) and deepen the value of concurrent methods. As one guide to public policy research puts it, ‘almost all likely sources of information, data, and ideas fall into two general types: documents and people’ ( Bardach and Patashnik, 2015 ). Thus, researchers can ask interviewees to address questions that arise from policy documents and point the way to useful new documents. Bardach and Patashnik suggest alternating between documents and interviews as sources as information, as one tends to lead to the other, such as by scanning interviewees’ bookshelves and papers for titles and author names ( Bardach and Patashnik, 2015 ). Depending on your research questions, document analysis can be used in combination with different types of interviews ( Berner-Rodoreda et al. , 2018 ), observation ( Harvey, 2018 ), and quantitative analyses, among other common methods in policy research.

The READ approach

The READ approach to document analysis is a systematic procedure for collecting documents and gaining information from them in the context of health policy studies at any level (global, national, local, etc.). The steps consist of: (1) ready your materials, (2) extract data, (3) analyse data and (4) distil your findings. We describe each of these steps in turn.

Step 1. Ready your materials

At the outset, researchers must set parameters in terms of the nature and number (approximately) of documents they plan to analyse, based on the research question. How much time will you allocate to the document analysis, and what is the scope of your research question? Depending on the answers to these questions, criteria should be established around (1) the topic (a particular policy, programme, or health issue, narrowly defined according to the research question); (2) dates of inclusion (whether taking the long view of several decades, or zooming in on a specific event or period in time); and (3) an indicative list of places to search for documents (possibilities include databases such as Ministry archives; LexisNexis or other databases; online searches; and particularly interview subjects). For difficult-to-obtain working documents or otherwise non-public items, bringing a flash drive to interviews is one of the best ways to gain access to valuable documents.

For research focusing on a single policy or programme, you may review only a handful of documents. However, if you are looking at multiple policies, health issues, or contexts, or reviewing shorter documents (such as newspaper articles), you may look at hundreds, or even thousands of documents. When considering the number of documents you will analyse, you should make notes on the type of information you plan to extract from documents—i.e. what it is you hope to learn, and how this will help answer your research question(s). The initial criteria—and the data you seek to extract from documents—will likely evolve over the course of the research, as it becomes clear whether they will yield too few documents and information (a rare outcome), far too many documents and too much information (a much more common outcome) or documents that fail to address the research question; however, it is important to have a starting point to guide the search. If you find that the documents you need are unavailable, you may need to reassess your research questions or consider other methods of inquiry. If you have too many documents, you can either analyse a subset of these ( Panel 1 ) or adopt more stringent inclusion criteria.

Exploring the framing of diseases in Pakistani media

In Table 1 , we present a non-exhaustive list of the types of documents that can be included in document analyses of health policy issues. In most cases, this will mean written sources (policies, reports, articles). The types of documents to be analysed will vary by study and according to the research question, although in many cases, it will be useful to consult a mix of formal documents (such as official policies, laws or strategies), ‘gray literature’ (organizational materials such as reports, evaluations and white papers produced outside formal publication channels) and, whenever possible, informal or working documents (such as meeting notes, PowerPoint presentations and memoranda). These latter in particular can provide rich veins of insight into how policy actors are thinking through the issues under study, particularly for the lucky researcher who obtains working documents with ‘Track Changes.’ How you prioritize documents will depend on your research question: you may prioritize official policy documents if you are studying policy content, or you may prioritize informal documents if you are studying policy process.

Types of documents that can be consulted in studies of health policy

During this initial preparatory phase, we also recommend devising a file-naming system for your documents (e.g. Author.Date.Topic.Institution.PDF), so that documents can be easily retrieved throughout the research process. After extracting data and processing your documents the first time around, you will likely have additional ‘questions’ to ask your documents and need to consult them again. For this reason, it is important to clearly name source files and link filenames to the data that you are extracting (see sample naming conventions in the Supplementary Materials ).

Step 2. Extract data

Data can be extracted in a number of ways, and the method you select for doing so will depend on your research question and the nature of your documents. One simple way is to use an Excel spreadsheet where each row is a document and each column is a category of information you are seeking to extract, from more basic data such as the document title, author and date, to theoretical or conceptual categories deriving from your research question, operating theory or analytical framework (Panel 2). Documents can also be imported into thematic coding software such as Atlas.ti or NVivo, and data extracted that way. Alternatively, if the research question focuses on process, documents can be used to compile a timeline of events, to trace processes across time. Ask yourself, how can I organize these data in the most coherent manner? What are my priority categories? We have included two different examples of data extraction tools in the Supplementary Materials to this article to spark ideas.

Case study Documents tell part of the story in Niger

Document analyses are first and foremost exercises in close reading: documents should be read thoroughly, from start to finish, including annexes, which may seem tedious but which sometimes produce golden nuggets of information. Read for overall meaning as you extract specific data related to your research question. As you go along, you will begin to have ideas or build working theories about what you are learning and observing in the data. We suggest capturing these emerging theories in extended notes or ‘memos,’ as used in Grounded Theory methodology ( Charmaz, 2006 ); these can be useful analytical units in themselves and can also provide a basis for later report and article writing.

As you read more documents, you may find that your data extraction tool needs to be modified to capture all the relevant information (or to avoid wasting time capturing irrelevant information). This may require you to go back and seek information in documents you have already read and processed, which will be greatly facilitated by a coherent file-naming system. It is also useful to keep notes on other documents that are mentioned that should be tracked down (sometimes you can write the author for help). As a general rule, we suggest being parsimonious when selecting initial categories to extract from data. Simply reading the documents takes significant time in and of itself—make sure you think about how, exactly, the specific data you are extracting will be used and how it goes towards answering your research questions.

Step 3. Analyse data

As in all types of qualitative research, data collection and analysis are iterative and characterized by emergent design, meaning that developing findings continually inform whether and how to obtain and interpret data ( Creswell, 2013 ). In practice, this means that during the data extraction phase, the researcher is already analysing data and forming initial theories—as well as potentially modifying document selection criteria. However, only when data extraction is complete can one see the full picture. For example, are there any documents that you would have expected to find, but did not? Why do you think they might be missing? Are there temporal trends (i.e. similarities, differences or evolutions that stand out when documents are ordered chronologically)? What else do you notice? We provide a list of overarching questions you should think about when viewing your body of document as a whole ( Table 2 ).

Questions to ask your overall body of documents

HIV and viral hepatitis articles by main frames (%). Note: The percentage of articles is calculated by dividing the number of articles appearing in each frame for viral hepatitis and HIV by the respectivenumber of sampled articles for each disease (N = 137 for HIV; N = 117 for hepatitis). Time frame: 1 January 2006 to 30 September 2016

Representations of progress toward Millennium Development Goal 4 in Nigerien policy documents. Sources: clockwise from upper left: ( WHO 2006 ); ( Institut National de la Statistique 2010 ); ( Ministè re de la Santé Publique 2010 ); ( Unicef 2010 )

In addition to the meaning-making processes you are already engaged in during the data extraction process, in most cases, it will be useful to apply specific analysis methodologies to the overall corpus of your documents, such as policy analysis ( Buse et al. , 2005 ). An array of analysis methodologies can be used, both quantitative and qualitative, including case study methodology, thematic content analysis, discourse analysis, framework analysis and process tracing, which may require differing levels of familiarity and skills to apply (we highlight a few of these in the case studies below). Analysis can also be structured according to theoretical approaches. When it comes to analysing policies, process tracing can be particularly useful to combine multiple sources of information, establish a chronicle of events and reveal political and social processes, so as to create a narrative of the policy cycle ( Yin, 1994 ; Shiffman et al. , 2004 ). Practically, you will also want to take a holistic view of the documents’ ‘answers’ to the questions or analysis categories you applied during the data extraction phase. Overall, what did the documents ‘say’ about these thematic categories? What variation did you find within and between documents, and along which axes? Answers to these questions are best recorded by developing notes or memos, which again will come in handy as you write up your results.

As with all qualitative research, you will want to consider your own positionality towards the documents (and their sources and authors); it may be helpful to keep a ‘reflexivity’ memo documenting how your personal characteristics or pre-standing views might influence your analysis ( Watt, 2007 ).

Step 4. Distil your findings

You will know when you have completed your document review when one of the three things happens: (1) completeness (you feel satisfied you have obtained every document fitting your criteria—this is rare), (2) out of time (this means you should have used more specific criteria), and (3) saturation (you fully or sufficiently understand the phenomenon you are studying). In all cases, you should strive to make the third situation the reason for ending your document review, though this will not always mean you will have read and analysed every document fitting your criteria—just enough documents to feel confident you have found good answers to your research questions.

Now it is time to refine your findings. During the extraction phase, you did the equivalent of walking along the beach, noticing the beautiful shells, driftwood and sea glass, and picking them up along the way. During the analysis phase, you started sorting these items into different buckets (your analysis categories) and building increasingly detailed collections. Now you have returned home from the beach, and it is time to clean your objects, rinse them of sand and preserve only the best specimens for presentation. To do this, you can return to your memos, refine them, illustrate them with graphics and quotes and fill in any incomplete areas. It can also be illuminating to look across different strands of work: e.g. how did the content, style, authorship, or tone of arguments evolve over time? Can you illustrate which words, concepts or phrases were used by authors or author groups?

Results will often first be grouped by theoretical or analytic category, or presented as a policy narrative, interweaving strands from other methods you may have used (interviews, observation, etc.). It can also be helpful to create conceptual charts and graphs, especially as this corresponds to your analytical framework (Panels 1 and 2). If you have been keeping a timeline of events, you can seek out any missing information from other sources. Finally, ask yourself how the validity of your findings checks against what you have learned using other methods. The final products of the distillation process will vary by research study, but they will invariably allow you to state your findings relative to your research questions and to draw policy-relevant conclusions.

Document analysis is an essential component of health policy research—it is also relatively convenient and can be low cost. Using an organized system of analysis enhances the document analysis’s procedural rigour, allows for a fuller understanding of policy process and content and enhances the effectiveness of other methods such as interviews and non-participant observation. We propose the READ approach as a systematic method for interrogating documents and extracting study-relevant data that is flexible enough to accommodate many types of research questions. We hope that this article encourages discussion about how to make best use of data from documents when researching health policy questions.

Supplementary Data

Supplementary data are available at Health Policy and Planning online.

Supplementary Material

Czaa064_supplementary_data, acknowledgements.

The data extraction tool in the Supplementary Materials for the iCCM case study (Panel 2) was conceived of by the research team for the multi-country study ‘Policy Analysis of Community Case Management for Childhood and Newborn Illnesses’. The authors thank Sara Bennett and Daniela Rodriguez for granting permission to publish this tool. S.M. was supported by The Olympia-Morata-Programme of Heidelberg University. The funders had no role in the decision to publish, or preparation of the manuscript. The content is the responsibility of the authors and does not necessarily represent the views of any funder.

Conflict of interest statement . None declared.

Ethical approval. No ethical approval was required for this study.

• Abdelmutti N, Hoffman-Goetz L.  2009. Risk messages about HPV, cervical cancer, and the HPV vaccine Gardasil: a content analysis of Canadian and U.S. national newspaper articles . Women & Health   49 : 422–40. [ PubMed ] [ Google Scholar ]
• Adebiyi BO, Mukumbang FC, Beytell A-M.  2019. To what extent is fetal alcohol spectrum disorder considered in policy-related documents in South Africa? A document review . Health Research Policy and Systems   17 : [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Atkinson PA, Coffey A.  1997. Analysing documentary realities In: Silverman D (ed). Qualitative Research: Theory, Method and Practice . London: SAGE. [ Google Scholar ]
• Bardach E, Patashnik EM.  2015. Practical Guide for Policy Analysis: The Eightfold Path to More Effective Problem Solving . Los Angeles: SAGE. [ Google Scholar ]
• Bennett S, Dalglish SL, Juma PA, Rodríguez DC.  2015. Altogether now… understanding the role of international organizations in iCCM policy transfer . Health Policy and Planning   30 : ii26–35. [ PubMed ] [ Google Scholar ]
• Berner-Rodoreda A, Bärnighausen T, Kennedy C  et al.  2018. From doxastic to epistemic: a typology and critique of qualitative interview styles . Qualitative Inquiry   26 : 291–305. 1077800418810724. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Bowen GA.  2009. Document analysis as a qualitative research method . Qualitative Research Journal   9 : 27–40. [ Google Scholar ]
• Bryman A.  1994. Analyzing Qualitative Data .
• Buse K, Mays N, Walt G.  2005. Making Health Policy . New York: Open University Press. [ Google Scholar ]
• Charmaz K.  2006. Constructing Grounded Theory: A Practical Guide through Qualitative Analysis . London: SAGE. [ PubMed ] [ Google Scholar ]
• Claassen L, Smid T, Woudenberg F, Timmermans DRM.  2012. Media coverage on electromagnetic fields and health: content analysis of Dutch newspaper articles and websites . Health, Risk & Society   14 : 681–96. [ Google Scholar ]
• Creswell JW.  2013. Qualitative Inquiry and Research Design . Thousand Oaks, CA: SAGE. [ Google Scholar ]
• Dalglish SL, Rodríguez DC, Harouna A, Surkan PJ.  2017. Knowledge and power in policy-making for child survival in Niger . Social Science & Medicine   177 : 150–7. [ PubMed ] [ Google Scholar ]
• Dalglish SL, Surkan PJ, Diarra A, Harouna A, Bennett S.  2015. Power and pro-poor policies: the case of iCCM in Niger . Health Policy and Planning   30 : ii84–94. [ PubMed ] [ Google Scholar ]
• Entman RM.  1993. Framing: toward clarification of a fractured paradigm . Journal of Communication   43 : 51–8. [ Google Scholar ]
• Fournier G, Djermakoye IA.  1975. Village health teams in Niger (Maradi Department) In: Newell KW (ed). Health by the People . Geneva: WHO. [ Google Scholar ]
• Gurung G, Derrett S, Gauld R.  2020. The role and functions of community health councils in New Zealand’s health system: a document analysis . The New Zealand Medical Journal   133 : 70–82. [ PubMed ] [ Google Scholar ]
• Harvey L.  1990. Critical Social Research . London: Unwin Hyman. [ Google Scholar ]
• Harvey SA.  2018. Observe before you leap: why observation provides critical insights for formative research and intervention design that you’ll never get from focus groups, interviews, or KAP surveys . Global Health: Science and Practice   6 : 299–316. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Institut National de la Statistique. 2010. Rapport National sur les Progrès vers l'atteinte des Objectifs du Millénaire pour le Développement. Niamey, Niger: INS. [ Google Scholar ]
• Kamarulzaman A.  2013. Fighting the HIV epidemic in the Islamic world . Lancet   381 : 2058–60. [ PubMed ] [ Google Scholar ]
• Katchmarchi AB, Taliaferro AR, Kipfer HJ.  2018. A document analysis of drowning prevention education resources in the United States . International Journal of Injury Control and Safety Promotion   25 : 78–84. [ PubMed ] [ Google Scholar ]
• Krippendorff K.  2004. Content Analysis: An Introduction to Its Methodology . SAGE. [ Google Scholar ]
• Marten R.  2019. How states exerted power to create the Millennium Development Goals and how this shaped the global health agenda: lessons for the sustainable development goals and the future of global health . Global Public Health   14 : 584–99. [ PubMed ] [ Google Scholar ]
• Maxwell JA.  2005. Qualitative Research Design: An Interactive Approach , 2 nd edn. Thousand Oaks, CA: Sage Publications. [ Google Scholar ]
• Mayring P.  2004. Qualitative Content Analysis . In: Flick U, von Kardorff E, Steinke I (eds).   A Companion to Qualitative Research SAGE. [ Google Scholar ]
• Ministère de la Santé Publique. 2010. Enquête nationale sur la survie des enfants de 0 à 59 mois et la mortalité au Niger 2010. Niamey, Niger: MSP. [ Google Scholar ]
• Mogalakwe M.  2009. The documentary research method—using documentary sources in social research . Eastern Africa Social Science Research Review   25 : 43–58. [ Google Scholar ]
• Nelkin D.  1991. AIDS and the news media . The Milbank Quarterly   69 : 293–307. [ PubMed ] [ Google Scholar ]
• Ouedraogo O, Doudou MH, Drabo KM  et al.  2020. Policy overview of the multisectoral nutrition planning process: the progress, challenges, and lessons learned from Burkina Faso . The International Journal of Health Planning and Management   35 : 120–39. [ PubMed ] [ Google Scholar ]
• Prior L.  2003. Using Documents in Social Research . London: SAGE. [ Google Scholar ]
• Shiffman J, Stanton C, Salazar AP.  2004. The emergence of political priority for safe motherhood in Honduras . Health Policy and Planning   19 : 380–90. [ PubMed ] [ Google Scholar ]
• Smith KC, Washington C, Welding K  et al.  2017. Cigarette stick as valuable communicative real estate: a content analysis of cigarettes from 14 low-income and middle-income countries . Tobacco Control   26 : 604–7. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Strömbäck J, Dimitrova DV.  2011. Mediatization and media interventionism: a comparative analysis of Sweden and the United States . The International Journal of Press/Politics   16 : 30–49. [ Google Scholar ]
• UNICEF. 2010. Maternal, Newborn & Child Surival Profile. Niamey, Niger: UNICEF . [ Google Scholar ]
• Watt D.  2007. On becoming a qualitative researcher: the value of reflexivity . Qualitative Report   12 : 82–101. [ Google Scholar ]
• Weber M.  2015. Bureaucracy In: Waters T, Waters D (eds). Rationalism and Modern Society: New Translations on Politics, Bureaucracy, and Social Stratification . London: Palgrave MacMillan. [ Google Scholar ]
• Wesley JJ.  2010. Qualitative Document Analysis in Political Science.
• World Health Organization. 2006. Country Health System Fact Sheet 2006: Niger. Niamey, Niger: WHO.
• Yin R.  1994. Case Study Research: Design and Methods . Thousand Oaks, CA: Sage. [ Google Scholar ]

To read this content please select one of the options below:

Please note you do not have access to teaching notes, document analysis as a qualitative research method.

Qualitative Research Journal

ISSN : 1443-9883

Article publication date: 3 August 2009

This article examines the function of documents as a data source in qualitative research and discusses document analysis procedure in the context of actual research experiences. Targeted to research novices, the article takes a nuts‐and‐bolts approach to document analysis. It describes the nature and forms of documents, outlines the advantages and limitations of document analysis, and offers specific examples of the use of documents in the research process. The application of document analysis to a grounded theory study is illustrated.

• Content analysis
• Grounded theory
• Thematic analysis
• Triangulation

Bowen, G.A. (2009), "Document Analysis as a Qualitative Research Method", Qualitative Research Journal , Vol. 9 No. 2, pp. 27-40. https://doi.org/10.3316/QRJ0902027

Emerald Group Publishing Limited

Copyright © 2009, Emerald Group Publishing Limited

Related articles

We’re listening — tell us what you think, something didn’t work….

Report bugs here

All feedback is valuable

Please share your general feedback

Join us on our journey

Platform update page.

Visit emeraldpublishing.com/platformupdate to discover the latest news and updates

Questions & More Information

Answers to the most commonly asked questions here

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Systematic Review | Definition, Example, & Guide

Systematic Review | Definition, Example & Guide

Published on June 15, 2022 by Shaun Turney . Revised on November 20, 2023.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, other interesting articles, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:

• Formulate a research question
• Develop a protocol
• Search for all relevant studies
• Apply the selection criteria
• Extract the data
• Synthesize the data
• Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

• A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
• If you’re doing a systematic review on your own (e.g., for a research paper or thesis ), you should take appropriate measures to ensure the validity and reliability of your research.
• Access to databases and journal archives. Often, your educational institution provides you with access.
• Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
• Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

• They minimize research bias by considering all available evidence and evaluating each study for bias.
• Their methods are transparent , so they can be scrutinized by others.
• They’re thorough : they summarize all available evidence.
• They can be replicated and updated by others.

Systematic reviews also have a few cons .

• They’re time-consuming .
• They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

• Allow you to more effectively communicate your research to other researchers and practitioners
• Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

• Population(s) or problem(s)
• Intervention(s)
• Comparison(s)

You can rearrange these four components to write your research question:

• What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .

• Type of study design(s)
• The population of patients with eczema
• The intervention of probiotics
• In comparison to no treatment, placebo , or non-probiotic treatment
• The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
• Randomized control trials, a type of study design

Their research question was:

• What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

• Background information : Provide the context of the research question, including why it’s important.
• Research objective (s) : Rephrase your research question as an objective.
• Selection criteria: State how you’ll decide which studies to include or exclude from your review.
• Search strategy: Discuss your plan for finding studies.
• Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

• Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
• Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
• Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of gray literature.
• Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

• Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
• Handsearch: Conference proceedings and reference lists of articles
• Gray literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
• Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

• Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
• Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

• Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
• Your judgment of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

• Narrative ( qualitative ): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
• Quantitative : Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

• Abstract : A summary of the review
• Introduction : Including the rationale and objectives
• Methods : Including the selection criteria, search method, data extraction method, and synthesis method
• Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
• Discussion : Including interpretation of the results and limitations of the review
• Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

In their report, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema. Note Generative AI tools like ChatGPT can be useful at various stages of the writing and research process and can help you to write your systematic review. However, we strongly advise against trying to pass AI-generated text off as your own work.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Prospective cohort study

Research bias

• Implicit bias
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic
• Social desirability bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Turney, S. (2023, November 20). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved April 15, 2024, from https://www.scribbr.com/methodology/systematic-review/

Is this article helpful?

Shaun Turney

Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• Knowledge Management
• Knowledge base Software
• Technical Documentation
• API Documentation
• Customer Support
• Best Practices
• Product Updates
• Product Engineering

Need an awesome Knowledge base?

We'll show you how to build a knowledge base (public or private) in minutes.

The Documentation Review Process: A Practical Guide

Category: Technical Documentation

Last updated on Mar 17, 2023

When you’re working on a project and have to conduct a documentation review, you might be confused about what exactly this process involves, who are the stakeholders, and the what the outcome of the review process might be. In this blog piece let us scratch the surface of the Document review process and its vitality to quality content.

Table of Contents

Definition of document review, different stages of document review, purpose of the documentation review, who conducts document reviews, what to look for during a documentation review, decide whether to run a peer review, how to conduct a document review, create custom workflow to facilitate peer review.

Documentation review is a process where the document goes through one or multiple stages of review and reviewers and the feedback gets amended in the document. The outcome of the document review process is that it enhances the accuracy and quality of the document.

Documentation review is one of the most time-consuming activities regarding end-user documentation. It requires much concentration and focuses since it’s not as simple as reading through the document and spotting grammatical errors or typos.

In fact, it’s much more complicated than that and requires scrutiny of every word, heading, sentence, and paragraph to ensure that there are no logical inconsistencies or gaps in information. You need to understand the structure of the document, identify the primary audience, evaluate its readability, identify any truncated sentences or sections that should be expanded, etc.

Peer review

– The document is reviewed by a relevant to the core content of the document

Editor review

– The document is reviewed by a senior writer, or an editor reviews the document’s for overall quality

Stakeholder review

– The document is reviewed by the Subject matter expert or the end users

Compliance review

– In case the document has to adhere to regulations and compliances set either by the industry, governing body, or the company the document is put through this review as well.

When you’re conducting a documentation review, the first thing you need to do is determine or understand the purpose of the review. This will determine the criteria you use to evaluate the document during the review and the length of the review.

For example, suppose you’re conducting a review of an internal employee training document. In that case, your primary goal will be to ensure that the document is accurate and that the information is presented in a way that’s both easy to understand and accessible to the employee. The outcome of the document should be that the employee is well-informed of the context after reading the document.

On the other hand, if you’re conducting a user manual review, your primary goal will be to ensure that the document conveys accurate, complete, and consistent information about the product or service to the end user.

For example , Let’s assume you are working on a user manual for an HR portal. You are reviewing an article on how users can reset their login password. In that case, you need to ensure the article contains the complete application process flow leading up to the Login screen, reset password button, and what happens after you have initiated the reset. Also, you must ensure the consistency of the business terms in the article.

Document review is done mostly by multiple reviewers and sometimes by a sole reviewer. The reviewers can be both in-house or independently hired reviewers. As far as personnel is concerned in the document review they can be other writers, Subject matter experts (SME), Editors, SEO personnel, or anyone relevant to the document for that matter.

Subject Matter Expert Review

A Subject Matter Expert (SME) can be anybody from your Developer, Engineer, Support manager, Analyst, etc., who has a comprehensive understanding of the core concept of your document.

For example , In a SaaS company, the developer can be considered a Subject matter expert as they develop the feature that the technical writer documents.

The role of the SME in the review process is crucial as they scope out all the technical gaps and inconsistencies in the document. It is not always the case that the document writer would be an expert on the technicalities of a feature compared to technical personnel working on the same product. Hence, Subject matter experts and Technical writers should work hand-in-hand on technical documents or product documentation .

When conducting a documentation review, you need to look for several things, including the quality of the writing, the structure of the document, the context provided, the flow of content, and the readability level of the document.

Let’s discuss these aspects in detail.

Quality of the writing – Whether you’re reviewing a user manual or training documentation , the quality of the writing is the first thing to look for when conducting a review. You must ensure that the document is free of grammar, punctuation, and spelling errors. If the document is riddled with these kinds of errors, it will reflect poorly on your organization, especially if it is an end-user document meant for customers. One suggestion to aid you with the review is enforcing a custom or available Document style guide. You can also use third-party grammar check tools such as Grammarly to a certain extent.

Structure of the document – The structure of the document refers to how the information has been organized. If the document has a table of contents, you must review the table of contents and make a note of any inconsistencies. If the document does not have a table of contents, you must identify the primary sections and provide a summary of each section. Usage of the Heading tags (H1, H2, H3…) is highly recommended. Also, the document should follow a set document pattern, the most common being Introduction-Problem Statement-Resolution-Conclusion-References.

For example, Document360 editors (Markdown and WYSIWYG) lets you add Heading tags such as H2, H3, and H4 in your article. The platform also has the ability to autogenerate Table of Contents (TOC) based on the Heading tags provided in the article.

For example, in a product comparison document, a table explaining the differences between various products in the ERP platform should have enough detail to make the reader understand the differences and get insight into each product’s unique selling proposition (USP).

Flow of sentences – You also need to look for the flow of sentences. In other words, you need to ensure that each sentence flows easily into the next one and that there are no abrupt or truncated sentences. If a sentence ends abruptly in the middle of a thought, this will confuse the reader. At the same time, you also have to be cautious to keep your sentences concise and complete at the same time.

Readability level of the document – You also need to look for the readability level of the document. This means that you need to evaluate the document according to the Flesch Reading Ease score. The lower the score, the more difficult the document is to read. Many CMS (Content management system) platforms provide an option or integration to measure the document’s readability.

For example, Document360 provides a Health check metric feature where authors can get insights on an article’s SEO health and Readability metrics. This feature assesses metrics such as Sentence/paragraph length, sub-heading distribution, and Readability score.

Also Read: Tips to keep a healthy knowledge base

Schedule a demo with one of our experts to take a deeper dive into Document360

When conducting a documentation review, you must also decide whether to run a peer review. In other words, you must choose whether to have independent or internal team reviewers proof the documentation.

It would be best if you considered several factors when deciding whether to run a peer review, including the nature of the documentation, size, and importance.

If the documentation is extremely important, it would be worthwhile to spend the extra money to hire an independent reviewer. You can also decide to go in for in-house peer reviews for different aspects of the document.

If you decide to hire an independent reviewer to conduct a peer review of your documentation, you must do so fairly and transparently. You must provide detailed instructions about what needs to be reviewed and what should be done during the review process. You can also provide feedback from previous reviews to give the reviewer a clear idea of what needs to be done during the review process.

Before you hire the reviewer, you must have a clear understanding of the deliverables from the reviewer. You must also agree to the terms and the timeline for review completion. It would be best if you also were transparent about the purpose of the review and the criteria used to evaluate the review.

If you decide to run an in-house or internal review, you can apply similar criteria, but you can add multiple levels of review and include multiple personnel if required. This will reflect in the review being optimal and guarantee you high-quality documentation.

Also Read: How to Create Document and Workflow for Business Process

At Document360, all our documentation goes through 5 levels of peer review before publication.

• Draft – Self-review based on a style guide
• L1 review – Peer writer review
• L2 review – Engineering team review
• L3 review – Editor review
• L4 review – SEO aspect and personnel review

We use the ‘ Workflow ‘ feature in the Document360 product to help us manage the peer review process. This allows us to add multiple levels of review, assign reviewers, record feedback, set due dates, and track revisions. The different phases of the review can also be custom set by the Document360 project owner or admin.  

Want to find out how Document360’s  Workflow feature helps to streamline the process? Check out the video below:

Once the article is drafted by the Author/Document writer, they use the assign feature in the ‘Workflow assignment’ option to assign different users accounts in Document360 for each of the review phases.

Additionally, they can also set due dates for each of the review. This will trigger a notification to the assignee/peer reviewer with information about the article name, assigner, and due date.

The peer reviewer then reviews the article for a specific aspect like grammar or technicality. Once the review is complete the peer reviewer can add the comments for the article in the ‘Workflow status’ section or use the ‘Discussions’ board in Article settings.

This helps us maintain a record of the peer reviews the article has undergone and the comments/feedback received from those reviews.

An intuitive technical documentation software to easily add your content and integrate it with any application. Give Document360 a try!

Wrapping Up

The document review process’s importance is on par with the drafting of the content itself. A new perspective is sometimes what is lacking from specific documents. The document review process ensures that if the document needs to be kept in a style guide’s lane, that also happens. A documentation review is also important because it can help improve the final product’s quality. If the documentation has been reviewed and edited beforehand, there is a lower chance that minor errors will be missed and made into the final product.

Also, taking into account the future of technical documentation and its sustainability, the quality of documents achieved using the document review process is very important. With the penetration of Artificial Intelligence in customer service, such as Voice assistants, quality support documentation can make a whole world of difference when fulfilling customer needs and queries.

The final thought is that the document review process and the framework to facilitate the review are impeccable.

Frequently Asked Questions

What are the types of documentation review.

Following completion of the document, it is subjected to the following reviews: peer reviews by colleagues, technical reviews by experts, editorial reviews by colleagues or editors, and managerial reviews by supervisors.

What are the 4 C’s of documentation?

The CSI 4-C’s, Clear/Concise/Correct/Complete helps improve the communication of your construction documents.

What is the purpose of document review?

There are two purposes of a document review. The first step is to determine which internal papers are relevant to a particular case. The second is for attorneys to fulfil their legal obligations when an opposing party requests essential litigation papers.

Dec 22, 2022

Access actionable resources on technical documentation

By signing up, you agree to our Terms , Policy and GDPR

Share this Article

Related Articles

Thank you for subscribing.

You will be receiving an email from us shortly.

Fill in the details to view the report

Document Review Process: Restructuring Creative Workflows

Documentation is a crucial and labor-intensive task that requires a lot of focus and attention. The document review process is an integral part of the documentation workflow to ensure the quality and accuracy of organizational data. Sometimes an orthographic pitfall can be embarrassing enough to cost you your business reputation. 

An impeccable document review process can save your day from any technical errors or discrepancies over intended purposes. Embracing new technological software in the documentation workflow can enhance document quality and help utilize time more productively. This blog explores what the review process is and how vital it is to enhance document quality.

What is Documentation Review?  

The document review process involves a series of steps that have a unified focus on ensuring the quality and accuracy of end-user documents. Document review is commonly known to be just skimming and spotting grammatical errors to make a document error-free. But the real meaning of documentation review i s much more than spotting errors.  

The reviewing process not only involves eliminating typos and grammatical mistakes but also pays heed to logical mismatches, poor readability, and information gaps. The documentation review process should also focus on the document structure and style formats. Thus, the right review process ensures that a particular document adheres to the defined style guide and format, making it a high-quality one.

Although reviewing documents starts at the drafting stage itself, it thoroughly checks the document reviewer to make sure that it is ready to use/publish. Document review helps fulfill the intended perspective of the document and satisfy the end user’s needs and queries. An exquisite document review never compromises the content quality and will never let the wow factor of the document diminish.

Why is the Document Review Process Necessary?

Document review is important for several reasons that are vital for effective and quality documentation. Document review is an important process that helps ensure accuracy, identify potential issues, ensure compliance, save costs, and mitigate risk. This helps everyone involved know the importance of the review process.

Planning the review cycles with the right reviewer will help organizations determine what is needed for the documentation review and approval process and decide on the expected results. This helps businesses identify the right solution, cut down on exorbitant documentation costs, and manage the associated time efficiently.

A thorough review process is necessary as it can help documentation with the following:

• Ensure Accuracy: Reviewing documents ensures that the information contained within them is accurate and free of errors. This is particularly important in legal and medical documents, where inaccuracies can have serious consequences.
• Identify Potential Issues: Reviewing documents can help identify potential issues in advance, allowing for proactive measures to be taken. For example, a legal document review may identify clauses that could be problematic, allowing for adjustments to be made before signing.
• Ensure Compliance: Many industries are subject to regulatory compliance requirements that require document reviews to ensure adherence to specific requirements.
• Save Costs: Document reviews can identify duplicate or unnecessary information, leading to cost savings through streamlined processes and reduced document storage needs.
• Mitigate Risk: Document reviews can help mitigate risks that emerge from legal disputes by identifying potential regulatory claims before they become larger problems.

Types of Document Review Process  

Reviewing documents is a complex process, and being aware of the complexities will help you avoid common pitfalls. Knowing the review document type, the review purpose, and its importance helps you determine the required evaluation or review stages.

You should proactively decide on the reviewer groups and time slots that are to be set up in the workflow.  Deciding how many reviewers are required and whether they should be internal or external teams ensures the feasibility of the process. This will ensure that the review is done well and provide you with documentation of the highest caliber.

The process of document review falls into the following categories:

• Legal document review
• Regulatory document review
• Technical document review
• Business document review
• Manuscript review

Legal Document Review Process:

Legal documents need to be reviewed with more responsibility since they hold more sensitive and confidential data. Reviewing legal documents involves two distinct steps. Identifying the pertinent documents and evaluating the same in case of disputes that require document redaction (fully or partially) from other parties or the production team due to their confidentiality.

Importance:

• The goal of document review is to make sure that the document meets the accuracy standards and adheres to the regulations without taking a call on data security.
• Document review is an inevitable thing in legal documentation as per GDPR and CCPA (regulatory authorities), which require legal policies, notices, contracts, or any other form of legal documents to be documented in plain language to enhance user readability
• Simple language in legal documents can help customers understand the document and be more informed about their stance or decisions. 

“I have read and agreed to the terms” is the “biggest lie on the web”, claims the site Terms of Service; Didn’t Read (ToS; DR). – This statement, in a way, puts forth that poor readability in such documents hinders user engagement. 

Legal teams collect huge volumes of the dataset and analyze it to understand and find information. Based on the facts and evidence, legal teams create judicial theories and manage them to identify the key witnesses in a case. The legal teams cull the collected data to find the right claims to be deposed. This in turn has put pressure on the legal teams to find automated solutions to narrow down these data volumes and streamline their management. 

Who Does The Review?

Since there are several types of document reviews, who does the review depends on the nature of the documents being reviewed and the purpose of the review. Document reviews can be conducted by a variety of professionals who have expertise in reviewing and analyzing documents. It can be a technical expert, Subject Matter Expert (SME), technical documentation writer, editor, SEO analyst, support engineer, or anyone relevant to the document for that matter. Be it a legal document or contract, regulatory compliance, financial reports, marketing materials, or manuscripts, a thorough review is necessary to improve clarity, coherence, accuracy, and relevance. 

Reviewing legal documents by available subject matter experts is risky enough to lose the originality of the documentation. So, it is of the utmost importance to seek advice from legal experts while reviewing a legal document . They help predict the confidentiality of the documents and take the necessary steps. 

The review team may decide which papers should have access to their contents completely or partly redacted or designated as private to prevent their exposure. To keep track of such situations, the team may also create privilege and/or redaction records.

Also Read: How to Get Your Documents Approved Faster?

Stages of Document Review Process

The documentation review process has different stages that validate the information in end-user documents. Reviewing a document through different stages deters the end-user from acquiring unintended information/knowledge. Inputs from past evaluations will help the reviewer have a clear understanding of what needs to be done throughout the document review process .

Several factors like review document size, importance, structure, and purpose influence review stages. Multi-stage reviews reflect and reward you with high-quality documentation.

• Self-Review: This check by the reviewer is essential to avoid any grammatical or orthographic mistakes. It can be termed the initial stage of review and is a must-have check for any type of document review.
• Peer Review: The relevancy of the content is reviewed here. Any kind of irrelevant content can end up changing the entire tone of the document. Peer reviewers watch for the content structure and the intended flow. Some documents will require peer review, while others won’t. It is the responsibility of the documenter to decide on peer review.
• Editor Review: This review is done by the seniors or editors to enhance the quality. Document standards and a check for style formats are done at this stage.
• Stakeholder Review:  This review is done by the end-user or the SEO experts. This review stage is done before the publishing stage, or reaching the end user. This stage primarily focuses on enhancing the user experience.
• Compliance Review: This review looks at the regulations and their compliance with the legal claims.  This is a mandatory stage while reviewing legal documents. Any misleading information in such legal papers can create judicial disputes, so it requires much attention while reviewing. 

Steps For the Seamless Document Review Process  

Documentation review is a collective and strategic process that requires prior planning and preparation. Document review is considered complicated not just because it requires more focus and attention, but also because the strategy involved and the optimization of the entire process require more attention to detail.

Specifying what has to be evaluated and what should be done throughout the review process will make the process of reviewing documents more seamless. You must have a clear knowledge of the reviewer’s deliverables involved in the review.

Clear review objectives and evaluation standards will be the game-changer here. Usually, the documentation workflow cycle involves stages of planning, pre-review preparation, review, correction, validation, and approval.

This stage involves identifying the purpose of the document review , determining the scope of the review, establishing review objectives, and defining review standards and protocols. 

Preparation:

This stage involves setting up the document review system, gathering all relevant documents and data, reviewing the relevant laws and regulations, preparing review guidelines and checklists, and establishing a review timeline. 

This stage involves the actual review of documents, which may include reading, analyzing, categorizing, coding, and tagging documents as per the guidelines and checklists prepared during the preparation stage. 

Quality control:

This stage involves checking the quality and accuracy of the review work conducted during the review stage and verifying that all relevant documents have been reviewed according to the established procedures and guidelines. 

Validation and Approval:

This stage involves preparing and submitting a report summarizing the findings from the review, including any identified issues or concerns, recommendations for improvements, and any remedial actions taken or proposed. 

This stage involves tracking and monitoring the implementation of the recommendations and remedial actions identified in the report. It is not necessary in case the document is published or at the end-user level. 

These steps make the documentation review process more efficient and make workflows more cost-effective. Knowledge of review goals and accomplishing what should be done ensures the readiness to provide high-quality documentation. Not everyone involved in the review process will necessarily agree with the goals and objectives.

Creating a customized workflow will be a major step toward achieving such readiness. As far as cost reduction is a vital growth element for the business bottom line, in-house documentation review software will be a stress-free and cost-effective option. In such instances, investing in the right documentation workflow software is a wise choice.

End-to-end workflow automation

Build fully-customizable, no code process workflows in a jiffy., what to look for during a documentation review: making the review a win-win  .

The goal of a documentation review is to make the document more accurate, complete, consistent, clear, compliant, accessible, and timely. A few criteria to look for when reviewing a document are:

• Accuracy: Ensure that the information provided in the documentation is correct, complete, and up-to-date. Providing reliable information helps build fair trust relationships among your customers.
• Completeness: Check to see whether all the necessary information has been included and if nothing is missing. The organized flow of sentences drives the tone of the document and makes it structured. This boosts the user’s readability, resulting in enhanced quality.
• Consistency: Ensure that the documentation is consistent in terms of formatting, language, style, and tone. Adherence to such specific aspects in a consistent manner will help reach your intended audience with ease.
• Format & Clarity: Check to see if the documentation is organized in a logical and easy-to-understand format. Ensure that the language used in the documentation is clear, concise, and easy to understand.
• Compliance: Check to see if the documentation complies with relevant standards, regulations, and policies. This check is mandatory when you review legal claims, finance documents, and contracts.
• Accessibility: Ensure that the documentation is accessible, i.e.,  it can be viewed and understood by its intended audience. Using custom review software can aid easy accessibility of the document to the intended audience and restrict access in cases of shared databases.
• Timeliness: Check to see if the documentation is delivered on time and update it regularly to keep it relevant and useful. Updation is a vital part of the documentation as it vanguard users in various ways. 

Being transparent about the goal of the review, the review kind, what’s to be done while reviewing documents, and available review tools gets you there. Reviewing documents for privilege, confidentiality, and relevance will never let your sensitive data lose its security or quality standards. 

How to Identify The Right Documentation Software

Choosing a document review software can be a daunting task as there are many options available on the market. Knowing the unique features and technological functions of the tool will be the deciding factor. Here are some queries that will help you choose the best document review software:

• Purpose: Determine why you need document review software and what features you are looking for. Do you need it for legal review or regulatory compliance purposes?
• Ease of use: Is the software easy to use and understand? Is it user-friendly? Does it have a user-friendly interface?
• Security: Never sacrifice your stand on security. Look for software that has data encryption and multi-factor authentication. Ensure that the software protects your data and files.
• Collaboration: Does the software allow multiple users to collaborate on a single document? Can they view, comment, and edit documents in real time?
• Cost-effectiveness: Consider the price of the software and whether it fits within your budget. Look for software that offers a free trial period or has a demo version available.
• Customer support: Look for software that offers excellent customer support or resources such as tutorials, FAQs, and community forums.
• Compatibility with the trend: Ensure that the software is compatible with your existing systems or software. Invest in advanced cloud-based software to propel your documentation workflow without any hurdles.

Businesses that value data security and a quicker turnaround consider investing in advanced software which has high cost and security advantages. Automated solutions like Cflow foster better streamlining of business procedures while maintaining better document control throughout all stages of the review process.                                                                         

Final Thoughts:  

As businesses thrive, all the key business processes are up for expansion and improvement.  And whenever processes expand their scope, the data volumes handled by them also increase. The solutions that scale will be the realistic option to invest in to achieve sustainable document management. Rising organizational pressure to reduce ad-hoc costs should not impact spending on document review solutions .

Be wise to choose a solution like Cflow that scales to bring routine document review in-house. This facilitates your business’s ability to maximize the utilization of resources, thereby offering huge savings and security benefits. Modern documentation tools help organizations pivot their spending to more complex and high-risk areas of productivity.

Level up your review process with Cflow – Experience frictionless document management by signing up today !

Thanks for reading till the end. Here are 3 ways we can help you automate your business:

Do better workflow automation with Cflow

Create workflows with multiple steps, parallel reviewals. auto approvals, public forms, etc. to save time and cost.

Try Cflow for free

Talk to a workflow expert

Get a 30-min. free consultation with our Workflow expert to optimize your daily tasks.

Book a demo

Get smarter with our workflow resources

Explore our workflow automation blogs, ebooks, and other resources to master workflow automation.

Check out our blogs

What would you like to do next?​

Automate your workflows with our Cflow experts.​

Get Your Workflows Automated for Free!

Business Name

Business Email

Good time to call

Describe your requirements (If any)

By submitting this form, you agree to our terms of service and privacy policy .

All You Need to Know about Business Process Integration

Procure to pay solutions – the way forward for the procurement function, you may also like, workflow vs process builder: which one your company needs, understanding the budget approval workflow, streamline your purchase order system with cloud-based workflow automation, procurement.

Crafting compelling content that captivates and converts your online audience.

IT Operations

Expand online reach and authority through strategic, quality-focused link building.

HR & Admin

Elevate online presence with data-driven Keyword Research for precise targeting.

Bringing your vision to life with cutting-edge design and seamless functionality.

Sales & Marketing

Get started with ready to use workflow templates from Cflow library.

• Workflow Templates

Advertisement

Toward a framework for selecting indicators of measuring sustainability and circular economy in the agri-food sector: a systematic literature review

• LIFE CYCLE SUSTAINABILITY ASSESSMENT
• Published: 02 March 2022

Cite this article

• Cecilia Silvestri   ORCID: orcid.org/0000-0003-2528-601X 1 ,
• Luca Silvestri   ORCID: orcid.org/0000-0002-6754-899X 2 ,
• Michela Piccarozzi   ORCID: orcid.org/0000-0001-9717-9462 1 &
• Alessandro Ruggieri 1  

2865 Accesses

11 Citations

9 Altmetric

Explore all metrics

A Correction to this article was published on 24 March 2022

This article has been updated

The implementation of sustainability and circular economy (CE) models in agri-food production can promote resource efficiency, reduce environmental burdens, and ensure improved and socially responsible systems. In this context, indicators for the measurement of sustainability play a crucial role. Indicators can measure CE strategies aimed to preserve functions, products, components, materials, or embodied energy. Although there is broad literature describing sustainability and CE indicators, no study offers such a comprehensive framework of indicators for measuring sustainability and CE in the agri-food sector.

Starting from this central research gap, a systematic literature review has been developed to measure the sustainability in the agri-food sector and, based on these findings, to understand how indicators are used and for which specific purposes.

The analysis of the results allowed us to classify the sample of articles in three main clusters (“Assessment-LCA,” “Best practice,” and “Decision-making”) and has shown increasing attention to the three pillars of sustainability (triple bottom line). In this context, an integrated approach of indicators (environmental, social, and economic) offers the best solution to ensure an easier transition to sustainability.

Conclusions

The sample analysis facilitated the identification of new categories of impact that deserve attention, such as the cooperation among stakeholders in the supply chain and eco-innovation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Source: Authors’ elaboration. Notes: The graph shows the temporal distribution of the articles under analysis

Source: Authors’ elaborations. Notes: The graph shows the time distribution of articles from the three major journals

Source: Authors’ elaboration. Notes: The graph shows the composition of the sample according to the three clusters identified by the analysis

Source: Authors’ elaboration. Notes: The graph shows the distribution of articles over time by cluster

Source: Authors’ elaboration. Notes: The graph shows the network visualization

Source: Authors’ elaboration. Notes: The graph shows the overlay visualization

Source: Authors’ elaboration. Notes: The graph shows the classification of articles by scientific field

Source: Authors’ elaboration. Notes: Article classification based on their cluster to which they belong and scientific field

Source: Authors’ elaboration

Source: Authors’ elaboration. Notes: The graph shows the distribution of items over time based on TBL

Source: Authors’ elaboration. Notes: The graph shows the Pareto diagram highlighting the most used indicators in literature for measuring sustainability in the agri-food sector

Source: Authors’ elaboration. Notes: The graph shows the distribution over time of articles divided into conceptual and empirical

Source: Authors’ elaboration. Notes: The graph shows the classification of articles, divided into conceptual and empirical, in-depth analysis

Source: Authors’ elaboration. Notes: The graph shows the geographical distribution of the authors

Source: Authors’ elaboration. Notes: The graph shows the distribution of authors according to the continent from which they originate

Source: Authors’ elaboration. Notes: The graph shows the time distribution of publication of authors according to the continent from which they originate

Source: Authors’ elaboration. Notes: Sustainability measurement indicators and impact categories of LCA, S-LCA, and LCC tools should be integrated in order to provide stakeholders with best practices as guidelines and tools to support both decision-making and measurement, according to the circular economy approach

Similar content being viewed by others

Common Methods and Sustainability Indicators

Transition heuristic frameworks in research on agro-food sustainability transitions

Hamid El Bilali

Research on agro-food sustainability transitions: where are food security and nutrition?

Change history, 24 march 2022.

A Correction to this paper has been published: https://doi.org/10.1007/s11367-022-02038-9

Acero AP, Rodriguez C, Ciroth A (2017) LCIA methods: impact assessment methods in life cycle assessment and their impact categories. Version 1.5.6. Green Delta 1–23

Accorsi R, Versari L, Manzini R (2015) Glass vs. plastic: Life cycle assessment of extra-virgin olive oil bottles across global supply chains. Sustain 7:2818–2840. https://doi.org/10.3390/su7032818

Adjei-Bamfo P, Maloreh-Nyamekye T, Ahenkan A (2019) The role of e-government in sustainable public procurement in developing countries: a systematic literature review. Resour Conserv Recycl 142:189–203. https://doi.org/10.1016/j.resconrec.2018.12.001

Article   Google Scholar  

Aivazidou E, Tsolakis N, Vlachos D, Iakovou E (2015) Water footprint management policies for agrifood supply chains: a critical taxonomy and a system dynamics modelling approach. Chem Eng Trans 43:115–120. https://doi.org/10.3303/CET1543020

Alhaddi H (2015) Triple bottom line and sustainability: a literature review. Bus Manag Stud 1:6–10

Allaoui H, Guo Y, Sarkis J (2019) Decision support for collaboration planning in sustainable supply chains. J Clean Prod 229:761–774. https://doi.org/10.1016/j.jclepro.2019.04.367

Alshqaqeeq F, Amin Esmaeili M, Overcash M, Twomey J (2020) Quantifying hospital services by carbon footprint: a systematic literature review of patient care alternatives. Resour Conserv Recycl 154:104560. https://doi.org/10.1016/j.resconrec.2019.104560

Anwar F, Chaudhry FN, Nazeer S et al (2016) Causes of ozone layer depletion and its effects on human: review. Atmos Clim Sci 06:129–134. https://doi.org/10.4236/acs.2016.61011

Aquilani B, Silvestri C, Ruggieri A (2016). A Systematic Literature Review on Total Quality Management Critical Success Factors and the Identification of New Avenues of Research. https://doi.org/10.1108/TQM-01-2016-0003

Aramyan L, Hoste R, Van Den Broek W et al (2011) Towards sustainable food production: a scenario study of the European pork sector. J Chain Netw Sci 11:177–189. https://doi.org/10.3920/JCNS2011.Qpork8

Arfini F, Antonioli F, Cozzi E et al (2019) Sustainability, innovation and rural development: the case of Parmigiano-Reggiano PDO. Sustain 11:1–17. https://doi.org/10.3390/su11184978

Assembly UG (2005) Resolution adopted by the general assembly. New York, NY

Avilés-Palacios C, Rodríguez-Olalla A (2021) The sustainability of waste management models in circular economies. Sustain 13:1–19. https://doi.org/10.3390/su13137105

Azevedo SG, Silva ME, Matias JCO, Dias GP (2018) The influence of collaboration initiatives on the sustainability of the cashew supply chain. Sustain 10:1–29. https://doi.org/10.3390/su10062075

Bajaj S, Garg R, Sethi M (2016) Total quality management: a critical literature review using Pareto analysis. Int J Product Perform Manag 67:128–154

Banasik A, Kanellopoulos A, Bloemhof-Ruwaard JM, Claassen GDH (2019) Accounting for uncertainty in eco-efficient agri-food supply chains: a case study for mushroom production planning. J Clean Prod 216:249–256. https://doi.org/10.1016/j.jclepro.2019.01.153

Barth H, Ulvenblad PO, Ulvenblad P (2017) Towards a conceptual framework of sustainable business model innovation in the agri-food sector: a systematic literature review. Sustain 9. https://doi.org/10.3390/su9091620

Bastas A, Liyanage K (2018) Sustainable supply chain quality management: a systematic review

Beckerman W (1992) Economic growth and the environment: whose growth? Whose environment? World Dev 20:481–496. https://doi.org/10.1016/0305-750X(92)90038-W

Belaud JP, Prioux N, Vialle C, Sablayrolles C (2019) Big data for agri-food 4.0: application to sustainability management for by-products supply chain. Comput Ind 111:41–50. https://doi.org/10.1016/j.compind.2019.06.006

Bele B, Norderhaug A, Sickel H (2018) Localized agri-food systems and biodiversity. Agric 8. https://doi.org/10.3390/agriculture8020022

Bilali H El, Calabrese G, Iannetta M et al (2020) Environmental sustainability of typical agro-food products: a scientifically sound and user friendly approach. New Medit 19:69–83. https://doi.org/10.30682/nm2002e

Blanc S, Massaglia S, Brun F et al (2019) Use of bio-based plastics in the fruit supply chain: an integrated approach to assess environmental, economic, and social sustainability. Sustain 11. https://doi.org/10.3390/su11092475

Bloemhof JM, van der Vorst JGAJ, Bastl M, Allaoui H (2015) Sustainability assessment of food chain logistics. Int J Logist Res Appl 18:101–117. https://doi.org/10.1080/13675567.2015.1015508

Bonisoli L, Galdeano-Gómez E, Piedra-Muñoz L (2018) Deconstructing criteria and assessment tools to build agri-sustainability indicators and support farmers’ decision-making process. J Clean Prod 182:1080–1094. https://doi.org/10.1016/j.jclepro.2018.02.055

Bonisoli L, Galdeano-Gómez E, Piedra-Muñoz L, Pérez-Mesa JC (2019) Benchmarking agri-food sustainability certifications: evidences from applying SAFA in the Ecuadorian banana agri-system. J Clean Prod 236. https://doi.org/10.1016/j.jclepro.2019.07.054

Bornmann L, Haunschild R, Hug SE (2018) Visualizing the context of citations referencing papers published by Eugene Garfield: a new type of keyword co-occurrence analysis. Scientometrics 114:427–437. https://doi.org/10.1007/s11192-017-2591-8

Boulding KE (1966) The economics of the coming spaceship earth. New York, 1-17

Bracquené E, Dewulf W, Duflou JR (2020) Measuring the performance of more circular complex product supply chains. Resour Conserv Recycl 154:104608. https://doi.org/10.1016/j.resconrec.2019.104608

Burck J, Hagen U, Bals C et al (2021) Climate Change Performance Index

Calisto Friant M, Vermeulen WJV, Salomone R (2020) A typology of circular economy discourses: navigating the diverse visions of a contested paradigm. Resour Conserv Recycl 161:104917. https://doi.org/10.1016/j.resconrec.2020.104917

Campbell BM, Beare DJ, Bennett EM et al (2017) Agriculture production as a major driver of the earth system exceeding planetary boundaries. Ecol Soc 22. https://doi.org/10.5751/ES-09595-220408

Capitanio F, Coppola A, Pascucci S (2010) Product and process innovation in the Italian food industry. Agribusiness 26:503–518. https://doi.org/10.1002/agr.20239

Caputo P, Zagarella F, Cusenza MA et al (2020) Energy-environmental assessment of the UIA-OpenAgri case study as urban regeneration project through agriculture. Sci Total Environ 729:138819. https://doi.org/10.1016/j.scitotenv.2020.138819

Article   CAS   Google Scholar  

Chabowski BR, Mena JA, Gonzalez-Padron TL (2011) The structure of sustainability research in marketing, 1958–2008: a basis for future research opportunities. J Acad Mark Sci 39:55–70. https://doi.org/10.1007/s11747-010-0212-7

Chadegani AA, Salehi H, Yunus M et al (2017) A comparison between two main academic literature collections : Web of Science and Scopus databases. Asian Soc Sci 9:18–26. https://doi.org/10.5539/ass.v9n5p18

Chams N, Guesmi B, Gil JM (2020) Beyond scientific contribution: assessment of the societal impact of research and innovation to build a sustainable agri-food sector. J Environ Manage 264. https://doi.org/10.1016/j.jenvman.2020.110455

Chandrakumar C, McLaren SJ, Jayamaha NP, Ramilan T (2019) Absolute sustainability-based life cycle assessment (ASLCA): a benchmarking approach to operate agri-food systems within the 2°C global carbon budget. J Ind Ecol 23:906–917. https://doi.org/10.1111/jiec.12830

Chaparro-Africano AM (2019) Toward generating sustainability indicators for agroecological markets. Agroecol Sustain Food Syst 43:40–66. https://doi.org/10.1080/21683565.2019.1566192

Colicchia C, Strozzi F (2012) Supply chain risk management: a new methodology for a systematic literature review

Conca L, Manta F, Morrone D, Toma P (2021) The impact of direct environmental, social, and governance reporting: empirical evidence in European-listed companies in the agri-food sector. Bus Strateg Environ 30:1080–1093. https://doi.org/10.1002/bse.2672

Coppola A, Ianuario S, Romano S, Viccaro M (2020) Corporate social responsibility in agri-food firms: the relationship between CSR actions and firm’s performance. AIMS Environ Sci 7:542–558. https://doi.org/10.3934/environsci.2020034

Corona B, Shen L, Reike D et al (2019) Towards sustainable development through the circular economy—a review and critical assessment on current circularity metrics. Resour Conserv Recycl 151:104498. https://doi.org/10.1016/j.resconrec.2019.104498

Correia MS (2019) Sustainability: An overview of the triple bottom line and sustainability implementation. Int J Strateg Eng 2:29–38.  https://doi.org/10.4018/IJoSE.2019010103

Coteur I, Marchand F, Debruyne L, Lauwers L (2019) Structuring the myriad of sustainability assessments in agri-food systems: a case in Flanders. J Clean Prod 209:472–480. https://doi.org/10.1016/j.jclepro.2018.10.066

CREA (2020) L’agricoltura italiana conta 2019

Crenna E, Sala S, Polce C, Collina E (2017) Pollinators in life cycle assessment: towards a framework for impact assessment. J Clean Prod 140:525–536. https://doi.org/10.1016/j.jclepro.2016.02.058

D’Eusanio M, Serreli M, Zamagni A, Petti L (2018) Assessment of social dimension of a jar of honey: a methodological outline. J Clean Prod 199:503–517. https://doi.org/10.1016/j.jclepro.2018.07.157

Dania WAP, Xing K, Amer Y (2018) Collaboration behavioural factors for sustainable agri-food supply chains: a systematic review. J Clean Prod 186:851–864

De Pascale A, Arbolino R, Szopik-Depczyńska K et al (2021) A systematic review for measuring circular economy: the 61 indicators. J Clean Prod 281. https://doi.org/10.1016/j.jclepro.2020.124942

De Schoenmakere M, Gillabel J (2017) Circular by design: products in the circular economy

Del Borghi A, Gallo M, Strazza C, Del Borghi M (2014) An evaluation of environmental sustainability in the food industry through life cycle assessment: the case study of tomato products supply chain. J Clean Prod 78:121–130. https://doi.org/10.1016/j.jclepro.2014.04.083

Del Borghi A, Strazza C, Magrassi F et al (2018) Life cycle assessment for eco-design of product–package systems in the food industry—the case of legumes. Sustain Prod Consum 13:24–36. https://doi.org/10.1016/j.spc.2017.11.001

Denyer D, Tranfield D (2009) Producing a systematic review. In: Buchanan B (ed) The sage handbook of organization research methods. Sage Publications Ltd, Cornwall, pp 671–689

Google Scholar  

Dietz T, Grabs J, Chong AE (2019) Mainstreamed voluntary sustainability standards and their effectiveness: evidence from the Honduran coffee sector. Regul Gov. https://doi.org/10.1111/rego.12239

Dixon-Woods M (2011) Using framework-based synthesis for conducting reviews of qualitative studies. BMC Med 9:9–10. https://doi.org/10.1186/1741-7015-9-39

do Canto NR, Bossle MB, Marques L, Dutra M, (2020) Supply chain collaboration for sustainability: a qualitative investigation of food supply chains in Brazil. Manag Environ Qual an Int J. https://doi.org/10.1108/MEQ-12-2019-0275

dos Santos RR, Guarnieri P (2020) Social gains for artisanal agroindustrial producers induced by cooperation and collaboration in agri-food supply chain. Soc Responsib J. https://doi.org/10.1108/SRJ-09-2019-0323

Doukidis GI, Matopoulos A, Vlachopoulou M, Manthou V, Manos B (2007) A conceptual framework for supply chain collaboration: empirical evidence from the agri‐food industry. Supply Chain Manag an Int Journal 12:177–186. https://doi.org/10.1108/13598540710742491

Durach CF, Kembro J, Wieland A (2017) A new paradigm for systematic literature reviews in supply chain management. J Supply Chain Manag 53:67–85. https://doi.org/10.1111/jscm.12145

Durán-Sánchez A, Álvarez-García J, Río-Rama D, De la Cruz M (2018) Sustainable water resources management: a bibliometric overview. Water 10:1–19. https://doi.org/10.3390/w10091191

Duru M, Therond O (2015) Livestock system sustainability and resilience in intensive production zones: which form of ecological modernization? Reg Environ Chang 15:1651–1665. https://doi.org/10.1007/s10113-014-0722-9

Edison Fondazione (2019) Le eccellenze agricole italiane. I primati europei e mondiali dell’Italia nei prodotti vegetali. Milan (IT)

Ehrenfeld JR (2005) The roots of sustainability. MIT Sloan Manag Rev 46(2)46:23–25

Elia V, Gnoni MG, Tornese F (2017) Measuring circular economy strategies through index methods: a critical analysis. J Clean Prod 142:2741–2751. https://doi.org/10.1016/j.jclepro.2016.10.196

Elkington J (1997) Cannibals with forks : the triple bottom line of 21st century business. Capstone, Oxford

Esposito B, Sessa MR, Sica D, Malandrino O (2020) Towards circular economy in the agri-food sector. A systematic literature review. Sustain 12. https://doi.org/10.3390/SU12187401

European Commission (2018) Agri-food trade in 2018

European Commission (2019) Monitoring EU agri-food trade: development until September 2019

Eurostat (2018) Small and large farms in the EU - statistics from the farm structure survey

FAO (2011) Biodiversity for food and agriculture. Italy, Rome

FAO (2012) Energy-smart food at FAO: an overview. Italy, Rome

FAO (2014) Food wastage footprint: fool cost-accounting

FAO (2016) The state of food and agriculture climate change, agriculture and food security. Italy, Rome

FAO (2017) The future of food and agriculture: trends and challenges. Italy, Rome

FAO (2020) The state of food security and nutrition in the world. Transforming Food Systems for Affordable Healthy Diets. Rome, Italy

Fassio F, Tecco N (2019) Circular economy for food: a systemic interpretation of 40 case histories in the food system in their relationships with SDGs. Systems 7:43. https://doi.org/10.3390/systems7030043

Fathollahi A, Coupe SJ (2021) Life cycle assessment (LCA) and life cycle costing (LCC) of road drainage systems for sustainability evaluation: quantifying the contribution of different life cycle phases. Sci Total Environ 776:145937. https://doi.org/10.1016/j.scitotenv.2021.145937

Ferreira VJ, Arnal ÁJ, Royo P et al (2019) Energy and resource efficiency of electroporation-assisted extraction as an emerging technology towards a sustainable bio-economy in the agri-food sector. J Clean Prod 233:1123–1132. https://doi.org/10.1016/j.jclepro.2019.06.030

Fiksel J (2006) A framework for sustainable remediation. JOM 8:15–22. https://doi.org/10.1021/es202595w

Flick U (2014) An introduction to qualitative research

Franciosi C, Voisin A, Miranda S et al (2020) Measuring maintenance impacts on sustainability of manufacturing industries : from a systematic literature review to a framework proposal. J Clean Prod 260:1–19. https://doi.org/10.1016/j.jclepro.2020.121065

Gaitán-Cremaschi D, Meuwissen MPM, Oude AGJML (2017) Total factor productivity: a framework for measuring agri-food supply chain performance towards sustainability. Appl Econ Perspect Policy 39:259–285. https://doi.org/10.1093/aepp/ppw008

Galdeano-Gómez E, Zepeda-Zepeda JA, Piedra-Muñoz L, Vega-López LL (2017) Family farm’s features influencing socio-economic sustainability: an analysis of the agri-food sector in southeast Spain. New Medit 16:50–61

Gallopín G, Herrero LMJ, Rocuts A (2014) Conceptual frameworks and visual interpretations of sustainability. Int J Sustain Dev 17:298–326. https://doi.org/10.1504/IJSD.2014.064183

Gallopín GC (2003) Sostenibilidad y desarrollo sostenible: un enfoque sistémico. Cepal, LATIN AMERICA

Garnett T (2013) Food sustainability: problems, perspectives and solutions. Proc Nutr Soc 72:29–39. https://doi.org/10.1017/S0029665112002947

Garofalo P, D’Andrea L, Tomaiuolo M et al (2017) Environmental sustainability of agri-food supply chains in Italy: the case of the whole-peeled tomato production under life cycle assessment methodology. J Food Eng 200:1–12. https://doi.org/10.1016/j.jfoodeng.2016.12.007

Gava O, Bartolini F, Venturi F et al (2018) A reflection of the use of the life cycle assessment tool for agri-food sustainability. Sustain 11. https://doi.org/10.3390/su11010071

Gazzola P, Querci E (2017) The connection between the quality of life and sustainable ecological development. Eur Sci J 7881:1857–7431

Geissdoerfer M, Savaget P, Bocken N, Hultink EJ (2017) The circular economy – a new sustainability paradigm ? The circular economy – a new sustainability paradigm ? J Clean Prod 143:757–768. https://doi.org/10.1016/j.jclepro.2016.12.048

Georgescu-Roegen N (1971) The entropy low and the economic process. Harward University Press, Cambridge Mass

Book   Google Scholar  

Gerbens-Leenes PW, Moll HC, Schoot Uiterkamp AJM (2003) Design and development of a measuring method for environmental sustainability in food production systems. Ecol Econ 46:231–248. https://doi.org/10.1016/S0921-8009(03)00140-X

Gésan-Guiziou G, Alaphilippe A, Aubin J et al (2020) Diversity and potentiality of multi-criteria decision analysis methods for agri-food research. Agron Sustain Dev 40. https://doi.org/10.1007/s13593-020-00650-3

Ghisellini P, Cialani C, Ulgiati S (2016) A review on circular economy: the expected transition to a balanced interplay of environmental and economic systems. J Clean Prod 114:11–32. https://doi.org/10.1016/j.jclepro.2015.09.007

Godoy-Durán Á, Galdeano- Gómez E, Pérez-Mesa JC, Piedra-Muñoz L (2017) Assessing eco-efficiency and the determinants of horticultural family-farming in southeast Spain. J Environ Manage 204:594–604. https://doi.org/10.1016/j.jenvman.2017.09.037

Gold S, Kunz N, Reiner G (2017) Sustainable global agrifood supply chains: exploring the barriers. J Ind Ecol 21:249–260. https://doi.org/10.1111/jiec.12440

Goucher L, Bruce R, Cameron DD et al (2017) The environmental impact of fertilizer embodied in a wheat-to-bread supply chain. Nat Plants 3:1–5. https://doi.org/10.1038/nplants.2017.12

Green A, Nemecek T, Chaudhary A, Mathys A (2020) Assessing nutritional, health, and environmental sustainability dimensions of agri-food production. Glob Food Sec 26:100406. https://doi.org/10.1016/j.gfs.2020.100406

Guinée JB, Heijungs R, Huppes G et al (2011) Life cycle assessment: past, present, and future †. Environ Sci Technol 45:90–96. https://doi.org/10.1021/es101316v

Guiomar N, Godinho S, Pinto-Correia T et al (2018) Typology and distribution of small farms in Europe: towards a better picture. Land Use Policy 75:784–798. https://doi.org/10.1016/j.landusepol.2018.04.012

Gunasekaran A, Patel C, McGaughey RE (2004) A framework for supply chain performance measurement. Int J Prod Econ 87:333–347. https://doi.org/10.1016/j.ijpe.2003.08.003

Gunasekaran A, Patel C, Tirtiroglu E (2001) Performance measures and metrics in a supply chain environment. Int J Oper Prod Manag 21:71–87. https://doi.org/10.1108/01443570110358468

Hamam M, Chinnici G, Di Vita G et al (2021) Circular economy models in agro-food systems: a review. Sustain 13

Harun SN, Hanafiah MM, Aziz NIHA (2021) An LCA-based environmental performance of rice production for developing a sustainable agri-food system in Malaysia. Environ Manage 67:146–161. https://doi.org/10.1007/s00267-020-01365-7

Harvey M, Pilgrim S (2011) The new competition for land: food, energy, and climate change. Food Policy 36:S40–S51. https://doi.org/10.1016/j.foodpol.2010.11.009

Hawkes C, Ruel MT (2006) Understanding the links between agriculture and health. DC: International Food Policy Research Institute. Washington, USA

Hellweg S, Milà i Canals L (2014) Emerging approaches, challenges and opportunities in life cycle assessment. Science (80)344:1109LP–1113. https://doi.org/10.1126/science.1248361

Higgins V, Dibden J, Cocklin C (2015) Private agri-food governance and greenhouse gas abatement: constructing a corporate carbon economy. Geoforum 66:75–84. https://doi.org/10.1016/j.geoforum.2015.09.012

Hill T (1995) Manufacturing strategy: text and cases., Macmillan

Hjeresen DD, Gonzales R (2020) Green chemistry promote sustainable agriculture?The rewards are higher yields and less environmental contamination. Environemental Sci Techonology 103–107

Horne R, Grant T, Verghese K (2009) Life cycle assessment: principles, practice, and prospects. Csiro Publishing, Collingwood, Australia

Horton P, Koh L, Guang VS (2016) An integrated theoretical framework to enhance resource efficiency, sustainability and human health in agri-food systems. J Clean Prod 120:164–169. https://doi.org/10.1016/j.jclepro.2015.08.092

Hospido A, Davis J, Berlin J, Sonesson U (2010) A review of methodological issues affecting LCA of novel food products. Int J Life Cycle Assess 15:44–52. https://doi.org/10.1007/s11367-009-0130-4

Huffman T, Liu J, Green M et al (2015) Improving and evaluating the soil cover indicator for agricultural land in Canada. Ecol Indic 48:272–281. https://doi.org/10.1016/j.ecolind.2014.07.008

Ilbery B, Maye D (2005) Food supply chains and sustainability: evidence from specialist food producers in the Scottish/English borders. Land Use Policy 22:331–344. https://doi.org/10.1016/j.landusepol.2004.06.002

Ingrao C, Faccilongo N, Valenti F et al (2019) Tomato puree in the Mediterranean region: an environmental life cycle assessment, based upon data surveyed at the supply chain level. J Clean Prod 233:292–313. https://doi.org/10.1016/j.jclepro.2019.06.056

Iocola I, Angevin F, Bockstaller C et al (2020) An actor-oriented multi-criteria assessment framework to support a transition towards sustainable agricultural systems based on crop diversification. Sustain 12. https://doi.org/10.3390/su12135434

Irabien A, Darton RC (2016) Energy–water–food nexus in the Spanish greenhouse tomato production. Clean Technol Environ Policy 18:1307–1316. https://doi.org/10.1007/s10098-015-1076-9

ISO 14040:2006 (2006) Environmental management — life cycle assessment — principles and framework

ISO 14044:2006 (2006) Environmental management — life cycle assessment — requirements and guidelines

ISO 15392:2008 (2008) Sustainability in building construction–general principles

Istat (2019) Andamento dell’economia agricola

Jaakkola E (2020) Designing conceptual articles : four approaches. AMS Rev 1–9. https://doi.org/10.1007/s13162-020-00161-0

Jin R, Yuan H, Chen Q (2019) Science mapping approach to assisting the review of construction and demolition waste management research published between 2009 and 2018. Resour Conserv Recycl 140:175–188. https://doi.org/10.1016/j.resconrec.2018.09.029

Johnston P, Everard M, Santillo D, Robèrt KH (2007) Reclaiming the definition of sustainability. Environ Sci Pollut Res Int 14:60–66. https://doi.org/10.1065/espr2007.01.375

Jorgensen SE, Burkhard B, Müller F (2013) Twenty volumes of ecological indicators-an accounting short review. Ecol Indic 28:4–9. https://doi.org/10.1016/j.ecolind.2012.12.018

Joshi S, Sharma M, Kler R (2020) Modeling circular economy dimensions in agri-tourism clusters: sustainable performance and future research directions. Int J Math Eng Manag Sci 5:1046–1061. https://doi.org/10.33889/IJMEMS.2020.5.6.080

Kamilaris A, Gao F, Prenafeta-Boldu FX, Ali MI (2017) Agri-IoT: a semantic framework for Internet of Things-enabled smart farming applications. In: 2016 IEEE 3rd World Forum on Internet of Things, WF-IoT 2016. pp 442–447

Karuppusami G, Gandhinathan R (2006) Pareto analysis of critical success factors of total quality management: a literature review and analysis. TQM Mag 18:372–385. https://doi.org/10.1108/09544780610671048

Kates RW, Parris TM, Leiserowitz AA (2005) What is sustainable development? Goals, indicators, values, and practice. Environ Sci Policy Sustain Dev 47:8–21. https://doi.org/10.1080/00139157.2005.10524444

Khounani Z, Hosseinzadeh-Bandbafha H, Moustakas K et al (2021) Environmental life cycle assessment of different biorefinery platforms valorizing olive wastes to biofuel, phosphate salts, natural antioxidant, and an oxygenated fuel additive (triacetin). J Clean Prod 278:123916. https://doi.org/10.1016/j.jclepro.2020.123916

Kitchenham B, Charters S (2007) Guidelines for performing systematic literature reviews in software engineering version 2.3. Engineering 45. https://doi.org/10.1145/1134285.1134500

Korhonen J, Nuur C, Feldmann A, Birkie SE (2018) Circular economy as an essentially contested concept. J Clean Prod 175:544–552. https://doi.org/10.1016/j.jclepro.2017.12.111

Kuisma M, Kahiluoto H (2017) Biotic resource loss beyond food waste: agriculture leaks worst. Resour Conserv Recycl 124:129–140. https://doi.org/10.1016/j.resconrec.2017.04.008

Laso J, Hoehn D, Margallo M et al (2018) Assessing energy and environmental efficiency of the Spanish agri-food system using the LCA/DEA methodology. Energies 11. https://doi.org/10.3390/en11123395

Lee KM (2007) So What is the “triple bottom line”? Int J Divers Organ Communities Nations Annu Rev 6:67–72. https://doi.org/10.18848/1447-9532/cgp/v06i06/39283

Lehmann RJ, Hermansen JE, Fritz M et al (2011) Information services for European pork chains - closing gaps in information infrastructures. Comput Electron Agric 79:125–136. https://doi.org/10.1016/j.compag.2011.09.002

León-Bravo V, Caniato F, Caridi M, Johnsen T (2017) Collaboration for sustainability in the food supply chain: a multi-stage study in Italy. Sustainability 9:1253

Lepage A (2009) The quality of life as attribute of sustainability. TQM J 21:105–115. https://doi.org/10.1108/17542730910938119

Li CZ, Zhao Y, Xiao B et al (2020) Research trend of the application of information technologies in construction and demolition waste management. J Clean Prod 263. https://doi.org/10.1016/j.jclepro.2020.121458

Lo Giudice A, Mbohwa C, Clasadonte MT, Ingrao C (2014) Life cycle assessment interpretation and improvement of the Sicilian artichokes production. Int J Environ Res 8:305–316. https://doi.org/10.22059/ijer.2014.721

Lueddeckens S, Saling P, Guenther E (2020) Temporal issues in life cycle assessment—a systematic review. Int J Life Cycle Assess 25:1385–1401. https://doi.org/10.1007/s11367-020-01757-1

Luo J, Ji C, Qiu C, Jia F (2018) Agri-food supply chain management: bibliometric and content analyses. Sustain 10. https://doi.org/10.3390/su10051573

Lynch J, Donnellan T, Finn JA et al (2019) Potential development of Irish agricultural sustainability indicators for current and future policy evaluation needs. J Environ Manage 230:434–445. https://doi.org/10.1016/j.jenvman.2018.09.070

MacArthur E (2013) Towards the circular economy. J Ind Ecol 2:23–44

MacArthur E (2017) Delivering the circular economy a toolkit for policymakers, The Ellen MacArthur Foundation

MacInnis DJ (2011) A framework for conceptual. J Mark 75:136–154. https://doi.org/10.1509/jmkg.75.4.136

Mangla SK, Luthra S, Rich N et al (2018) Enablers to implement sustainable initiatives in agri-food supply chains. Int J Prod Econ 203:379–393. https://doi.org/10.1016/j.ijpe.2018.07.012

Marotta G, Nazzaro C, Stanco M (2017) How the social responsibility creates value: models of innovation in Italian pasta industry. Int J Glob Small Bus 9:144–167. https://doi.org/10.1504/IJGSB.2017.088923

Martucci O, Arcese G, Montauti C, Acampora A (2019) Social aspects in the wine sector: comparison between social life cycle assessment and VIVA sustainable wine project indicators. Resources 8. https://doi.org/10.3390/resources8020069

Mayring P (2004) Forum : Qualitative social research Sozialforschung 2. History of content analysis. A Companion to Qual Res 1:159–176

McKelvey B (2002) Managing coevolutionary dynamics. In: 18th EGOS Conference. Barcelona, Spain, pp 1–21

McMichael AJ, Butler CD, Folke C (2003) New visions for addressing sustainability. Science (80- ) 302:1191–1920

Mehmood A, Ahmed S, Viza E et al (2021) Drivers and barriers towards circular economy in agri-food supply chain: a review. Bus Strateg Dev 1–17. https://doi.org/10.1002/bsd2.171

Mella P, Gazzola P (2011) Sustainability and quality of life: the development model. In: Kapounek S (ed) Enterprise and competitive environment. Mendel University: Brno, Czechia. 542–551

Merli R, Preziosi M, Acampora A (2018) How do scholars approach the circular economy ? A systematic literature review. J Clean Prod 178:703–722. https://doi.org/10.1016/j.jclepro.2017.12.112

Merli R, Preziosi M, Acampora A et al (2020) Recycled fibers in reinforced concrete: a systematic literature review. J Clean Prod 248:119207. https://doi.org/10.1016/j.jclepro.2019.119207

Miglietta PP, Morrone D (2018) Managing water sustainability: virtual water flows and economic water productivity assessment of the wine trade between Italy and the Balkans. Sustain 10. https://doi.org/10.3390/su10020543

Mitchell MGE, Chan KMA, Newlands NK, Ramankutty N (2020) Spatial correlations don’t predict changes in agricultural ecosystem services: a Canada-wide case study. Front Sustain Food Syst 4:1–17. https://doi.org/10.3389/fsufs.2020.539892

Moraga G, Huysveld S, Mathieux F et al (2019) Circular economy indicators: what do they measure?. Resour Conserv Recycl 146:452–461. https://doi.org/10.1016/j.resconrec.2019.03.045

Morrissey JE, Dunphy NP (2015) Towards sustainable agri-food systems: the role of integrated sustainability and value assessment across the supply-chain. Int J Soc Ecol Sustain Dev 6:41–58. https://doi.org/10.4018/IJSESD.2015070104

Moser G (2009) Quality of life and sustainability: toward person-environment congruity. J Environ Psychol 29:351–357. https://doi.org/10.1016/j.jenvp.2009.02.002

Muijs D (2010) Doing quantitative research in education with SPSS. London

Muller MF, Esmanioto F, Huber N, Loures ER (2019) A systematic literature review of interoperability in the green Building Information Modeling lifecycle. J Clean Prod 223:397–412. https://doi.org/10.1016/j.jclepro.2019.03.114

Muradin M, Joachimiak-Lechman K, Foltynowicz Z (2018) Evaluation of eco-efficiency of two alternative agricultural biogas plants. Appl Sci 8. https://doi.org/10.3390/app8112083

Naseer MA, ur R, Ashfaq M, Hassan S, et al (2019) Critical issues at the upstream level in sustainable supply chain management of agri-food industries: evidence from Pakistan’s citrus industry. Sustain 11:1–19. https://doi.org/10.3390/su11051326

Nattassha R, Handayati Y, Simatupang TM, Siallagan M (2020) Understanding circular economy implementation in the agri-food supply chain: the case of an Indonesian organic fertiliser producer. Agric Food Secur 9:1–16. https://doi.org/10.1186/s40066-020-00264-8

Nazari-Sharabian M, Ahmad S, Karakouzian M (2018) Climate change and eutrophication: a short review. Eng Technol Appl Sci Res 8:3668–3672. https://doi.org/10.5281/zenodo.2532694

Nazir N (2017) Understanding life cycle thinking and its practical application to agri-food system. Int J Adv Sci Eng Inf Technol 7:1861–1870. https://doi.org/10.18517/ijaseit.7.5.3578

Negra C, Remans R, Attwood S et al (2020) Sustainable agri-food investments require multi-sector co-development of decision tools. Ecol Indic 110:105851. https://doi.org/10.1016/j.ecolind.2019.105851

Newsham KK, Robinson SA (2009) Responses of plants in polar regions to UVB exposure: a meta-analysis. Glob Chang Biol 15:2574–2589. https://doi.org/10.1111/j.1365-2486.2009.01944.x

Niemeijer D, de Groot RS (2008) A conceptual framework for selecting environmental indicator sets. Ecol Indic 8:14–25. https://doi.org/10.1016/j.ecolind.2006.11.012

Niero M, Kalbar PP (2019) Coupling material circularity indicators and life cycle based indicators: a proposal to advance the assessment of circular economy strategies at the product level. Resour Conserv Recycl 140:305–312. https://doi.org/10.1016/j.resconrec.2018.10.002

Nikolaou IE, Tsagarakis KP (2021) An introduction to circular economy and sustainability: some existing lessons and future directions. Sustain Prod Consum 28:600–609. https://doi.org/10.1016/j.spc.2021.06.017

Notarnicola B, Hayashi K, Curran MA, Huisingh D (2012) Progress in working towards a more sustainable agri-food industry. J Clean Prod 28:1–8. https://doi.org/10.1016/j.jclepro.2012.02.007

Notarnicola B, Tassielli G, Renzulli PA, Monforti F (2017) Energy flows and greenhouses gases of EU (European Union) national breads using an LCA (life cycle assessment) approach. J Clean Prod 140:455–469. https://doi.org/10.1016/j.jclepro.2016.05.150

Opferkuch K, Caeiro S, Salomone R, Ramos TB (2021) Circular economy in corporate sustainability reporting: a review of organisational approaches. Bus Strateg Environ 1–22. https://doi.org/10.1002/bse.2854

Padilla-Rivera A, do Carmo BBT, Arcese G, Merveille N, (2021) Social circular economy indicators: selection through fuzzy delphi method. Sustain Prod Consum 26:101–110. https://doi.org/10.1016/j.spc.2020.09.015

Pagotto M, Halog A (2016) Towards a circular economy in Australian agri-food industry: an application of input-output oriented approaches for analyzing resource efficiency and competitiveness potential. J Ind Ecol 20:1176–1186. https://doi.org/10.1111/jiec.12373

Parent G, Lavallée S (2011) LCA potentials and limits within a sustainable agri-food statutory framework. Global food insecurity. Springer, Netherlands, Dordrecht, pp 161–171

Chapter   Google Scholar  

Pattey E, Qiu G (2012) Trends in primary particulate matter emissions from Canadian agriculture. J Air Waste Manag Assoc 62:737–747. https://doi.org/10.1080/10962247.2012.672058

Pauliuk S (2018) Critical appraisal of the circular economy standard BS 8001:2017 and a dashboard of quantitative system indicators for its implementation in organizations. Resour Conserv Recycl 129:81–92. https://doi.org/10.1016/j.resconrec.2017.10.019

Peano C, Migliorini P, Sottile F (2014) A methodology for the sustainability assessment of agri-food systems: an application to the slow food presidia project. Ecol Soc 19. https://doi.org/10.5751/ES-06972-190424

Peano C, Tecco N, Dansero E et al (2015) Evaluating the sustainability in complex agri-food systems: the SAEMETH framework. Sustain 7:6721–6741. https://doi.org/10.3390/su7066721

Pearce DW, Turner RK (1990) Economics of natural resources and the environment. Harvester Wheatsheaf, Hemel Hempstead, Herts

Pelletier N (2018) Social sustainability assessment of Canadian egg production facilities: methods, analysis, and recommendations. Sustain 10:1–17. https://doi.org/10.3390/su10051601

Peña C, Civit B, Gallego-Schmid A et al (2021) Using life cycle assessment to achieve a circular economy. Int J Life Cycle Assess 26:215–220. https://doi.org/10.1007/s11367-020-01856-z

Perez Neira D (2016) Energy sustainability of Ecuadorian cacao export and its contribution to climate change. A case study through product life cycle assessment. J Clean Prod 112:2560–2568. https://doi.org/10.1016/j.jclepro.2015.11.003

Pérez-Neira D, Grollmus-Venegas A (2018) Life-cycle energy assessment and carbon footprint of peri-urban horticulture. A comparative case study of local food systems in Spain. Landsc Urban Plan 172:60–68. https://doi.org/10.1016/j.landurbplan.2018.01.001

Pérez-Pons ME, Plaza-Hernández M, Alonso RS et al (2021) Increasing profitability and monitoring environmental performance: a case study in the agri-food industry through an edge-iot platform. Sustain 13:1–16. https://doi.org/10.3390/su13010283

Petti L, Serreli M, Di Cesare S (2018) Systematic literature review in social life cycle assessment. Int J Life Cycle Assess 23:422–431. https://doi.org/10.1007/s11367-016-1135-4

Pieroni MPP, McAloone TC, Pigosso DCA (2019) Business model innovation for circular economy and sustainability: a review of approaches. J Clean Prod 215:198–216. https://doi.org/10.1016/j.jclepro.2019.01.036

Polit DF, Beck CT (2004) Nursing research: principles and methods. Lippincott Williams & Wilkins, Philadelphia, PA

Porkka M, Gerten D, Schaphoff S et al (2016) Causes and trends of water scarcity in food production. Environ Res Lett 11:015001. https://doi.org/10.1088/1748-9326/11/1/015001

Prajapati H, Kant R, Shankar R (2019) Bequeath life to death: state-of-art review on reverse logistics. J Clean Prod 211:503–520. https://doi.org/10.1016/j.jclepro.2018.11.187

Priyadarshini P, Abhilash PC (2020) Policy recommendations for enabling transition towards sustainable agriculture in India. Land Use Policy 96:104718. https://doi.org/10.1016/j.landusepol.2020.104718

Pronti A, Coccia M (2020) Multicriteria analysis of the sustainability performance between agroecological and conventional coffee farms in the East Region of Minas Gerais (Brazil). Renew Agric Food Syst. https://doi.org/10.1017/S1742170520000332

Rabadán A, González-Moreno A, Sáez-Martínez FJ (2019) Improving firms’ performance and sustainability: the case of eco-innovation in the agri-food industry. Sustain 11. https://doi.org/10.3390/su11205590

Raut RD, Luthra S, Narkhede BE et al (2019) Examining the performance oriented indicators for implementing green management practices in the Indian agro sector. J Clean Prod 215:926–943. https://doi.org/10.1016/j.jclepro.2019.01.139

Recanati F, Marveggio D, Dotelli G (2018) From beans to bar: a life cycle assessment towards sustainable chocolate supply chain. Sci Total Environ 613–614:1013–1023. https://doi.org/10.1016/j.scitotenv.2017.09.187

Redclift M (2005) Sustainable development (1987–2005): an oxymoron comes of age. Sustain Dev 13:212–227. https://doi.org/10.1002/sd.281

Rezaei M, Soheilifard F, Keshvari A (2021) Impact of agrochemical emission models on the environmental assessment of paddy rice production using life cycle assessment approach. Energy Sources. Part A Recover Util Environ Eff 1–16

Rigamonti L, Mancini E (2021) Life cycle assessment and circularity indicators. Int J Life Cycle Assess. https://doi.org/10.1007/s11367-021-01966-2

Risku-Norja H, Mäenpää I (2007) MFA model to assess economic and environmental consequences of food production and consumption. Ecol Econ 60:700–711. https://doi.org/10.1016/j.ecolecon.2006.05.001

Ritzén S, Sandström GÖ (2017) Barriers to the circular economy – integration of perspectives and domains. Procedia CIRP 64:7–12. https://doi.org/10.1016/j.procir.2017.03.005

Rockström J, Steffen W, Noone K et al (2009) A safe operating space for humanity. Nature 461:472–475. https://doi.org/10.1038/461472a

Roos Lindgreen E, Mondello G, Salomone R et al (2021) Exploring the effectiveness of grey literature indicators and life cycle assessment in assessing circular economy at the micro level: a comparative analysis. Int J Life Cycle Assess. https://doi.org/10.1007/s11367-021-01972-4

Roselli L, Casieri A, De Gennaro BC et al (2020) Environmental and economic sustainability of table grape production in Italy. Sustain 12.  https://doi.org/10.3390/su12093670

Ross RB, Pandey V, Ross KL (2015) Sustainability and strategy in U.S. agri-food firms: an assessment of current practices. Int Food Agribus Manag Rev 18:17–48

Royo P, Ferreira VJ, López-Sabirón AM, Ferreira G. (2016) Hybrid diagnosis to characterise the energy and environmental enhancement of photovoltaic modules using smart materials. Energy 101:174–189. https://doi.org/10.1016/j.energy.2016.01.101

Ruggerio CA (2021) Sustainability and sustainable development: a review of principles and definitions. Sci Total Environ 786:147481. https://doi.org/10.1016/j.scitotenv.2021.147481

Ruiz-Almeida A, Rivera-Ferre MG (2019) Internationally-based indicators to measure agri-food systems sustainability using food sovereignty as a conceptual framework. Food Secur 11:1321–1337. https://doi.org/10.1007/s12571-019-00964-5

Ryan M, Hennessy T, Buckley C et al (2016) Developing farm-level sustainability indicators for Ireland using the Teagasc National Farm Survey. Irish J Agric Food Res 55:112–125. https://doi.org/10.1515/ijafr-2016-0011

Saade MRM, Yahia A, Amor B (2020) How has LCA been applied to 3D printing ? A systematic literature review and recommendations for future studies. J Clean Prod 244:118803. https://doi.org/10.1016/j.jclepro.2019.118803

Saitone TL, Sexton RJ (2017) Agri-food supply chain: evolution and performance with conflicting consumer and societal demands. Eur Rev Agric Econ 44:634–657. https://doi.org/10.1093/erae/jbx003

Salim N, Ab Rahman MN, Abd Wahab D (2019) A systematic literature review of internal capabilities for enhancing eco-innovation performance of manufacturing firms. J Clean Prod 209:1445–1460. https://doi.org/10.1016/j.jclepro.2018.11.105

Salimi N (2021) Circular economy in agri-food systems BT - strategic decision making for sustainable management of industrial networks. In: International S (ed) Rezaei J. Publishing, Cham, pp 57–70

Salomone R, Ioppolo G (2012) Environmental impacts of olive oil production: a life cycle assessment case study in the province of Messina (Sicily). J Clean Prod 28:88–100. https://doi.org/10.1016/j.jclepro.2011.10.004

Sánchez AD, Río DMDLC, García JÁ (2017) Bibliometric analysis of publications on wine tourism in the databases Scopus and WoS. Eur Res Manag Bus Econ 23:8–15. https://doi.org/10.1016/j.iedeen.2016.02.001

Saputri VHL, Sutopo W, Hisjam M, Ma’aram A (2019) Sustainable agri-food supply chain performance measurement model for GMO and non-GMO using data envelopment analysis method. Appl Sci 9. https://doi.org/10.3390/app9061199

Sassanelli C, Rosa P, Rocca R, Terzi S (2019) Circular economy performance assessment methods : a systematic literature review. J Clean Prod 229:440–453. https://doi.org/10.1016/j.jclepro.2019.05.019

Schiefer S, Gonzalez C, Flanigan S (2015) More than just a factor in transition processes? The role of collaboration in agriculture. In: Sutherland LA, Darnhofer I, Wilson GA, Zagata L (eds) Transition pathways towards sustainability in agriculture: case studies from Europe, CPI Group. Croydon, UK, pp. 83

Seuring S, Muller M (2008) From a literature review to a conceptual framework for sustainable supply chain management. J Clean Prod 16:1699–1710. https://doi.org/10.1016/j.jclepro.2008.04.020

Silvestri C, Silvestri L, Forcina A, et al (2021) Green chemistry contribution towards more equitable global sustainability and greater circular economy: A systematic literature review. J Clean Prod 294. https://doi.org/10.1016/j.jclepro.2021.126137

Smetana S, Schmitt E, Mathys A (2019) Sustainable use of Hermetia illucens insect biomass for feed and food: attributional and consequential life cycle assessment. Resour Conserv Recycl 144:285–296. https://doi.org/10.1016/j.resconrec.2019.01.042

Sonesson U, Berlin J, Ziegler F (2010) Environmental assessment and management in the food industry: life cycle assessment and related approaches. Woodhead Publishing, Cambridge

Soussana JF (2014) Research priorities for sustainable agri-food systems and life cycle assessment. J Clean Prod 73:19–23. https://doi.org/10.1016/j.jclepro.2014.02.061

Soylu A, Oruç C, Turkay M et al (2006) Synergy analysis of collaborative supply chain management in energy systems using multi-period MILP. Eur J Oper Res 174:387–403. https://doi.org/10.1016/j.ejor.2005.02.042

Spaiser V, Ranganathan S, Swain RB, Sumpter DJ (2017) The sustainable development oxymoron: quantifying and modelling the incompatibility of sustainable development goals. Int J Sustain Dev World Ecol 24:457–470. https://doi.org/10.1080/13504509.2016.1235624

Stewart R, Niero M (2018) Circular economy in corporate sustainability strategies: a review of corporate sustainability reports in the fast-moving consumer goods sector. Bus Strateg Environ 27:1005–1022. https://doi.org/10.1002/bse.2048

Stillitano T, Spada E, Iofrida N et al (2021) Sustainable agri-food processes and circular economy pathways in a life cycle perspective: state of the art of applicative research. Sustain 13:1–29. https://doi.org/10.3390/su13052472

Stone J, Rahimifard S (2018) Resilience in agri-food supply chains: a critical analysis of the literature and synthesis of a novel framework. Supply Chain Manag 23:207–238. https://doi.org/10.1108/SCM-06-2017-0201

Strazza C, Del Borghi A, Gallo M, Del Borghi M (2011) Resource productivity enhancement as means for promoting cleaner production: analysis of co-incineration in cement plants through a life cycle approach. J Clean Prod 19:1615–1621. https://doi.org/10.1016/j.jclepro.2011.05.014

Su B, Heshmati A, Geng Y, Yu X (2013) A review of the circular economy in China: moving from rhetoric to implementation. J Clean Prod 42:215–227. https://doi.org/10.1016/j.jclepro.2012.11.020

Suárez-Eiroa B, Fernández E, Méndez-Martínez G, Soto-Oñate D (2019) Operational principles of circular economy for sustainable development: linking theory and practice. J Clean Prod 214:952–961. https://doi.org/10.1016/j.jclepro.2018.12.271

Svensson G, Wagner B (2015) Implementing and managing economic, social and environmental efforts of business sustainability. Manag Environ Qual an Int Journal 26:195–213. https://doi.org/10.1108/MEQ-09-2013-0099

Tasca AL, Nessi S, Rigamonti L (2017) Environmental sustainability of agri-food supply chains: an LCA comparison between two alternative forms of production and distribution of endive in northern Italy. J Clean Prod 140:725–741. https://doi.org/10.1016/j.jclepro.2016.06.170

Tassielli G, Notarnicola B, Renzulli PA, Arcese G (2018) Environmental life cycle assessment of fresh and processed sweet cherries in southern Italy. J Clean Prod 171:184–197. https://doi.org/10.1016/j.jclepro.2017.09.227

Teixeira R, Pax S (2011) A survey of life cycle assessment practitioners with a focus on the agri-food sector. J Ind Ecol 15:817–820. https://doi.org/10.1111/j.1530-9290.2011.00421.x

Tobergte DR, Curtis S (2013) ILCD Handbook. J Chem Info Model. https://doi.org/10.278/33030

Tortorella MM, Di Leo S, Cosmi C et al (2020) A methodological integrated approach to analyse climate change effects in agri-food sector: the TIMES water-energy-food module. Int J Environ Res Public Health 17:1–21. https://doi.org/10.3390/ijerph17217703

Tranfield D, Denyer D, Smart P (2003) Towards a methodology for developing evidenceinformed management knowledge by means of systematic review. Br J Manag 14:207–222

Trivellas P, Malindretos G, Reklitis P (2020) Implications of green logistics management on sustainable business and supply chain performance: evidence from a survey in the greek agri-food sector. Sustain 12:1–29. https://doi.org/10.3390/su122410515

Tsangas M, Gavriel I, Doula M et al (2020) Life cycle analysis in the framework of agricultural strategic development planning in the Balkan region. Sustain 12:1–15. https://doi.org/10.3390/su12051813

Ülgen VS, Björklund M, Simm N (2019) Inter-organizational supply chain interaction for sustainability : a systematic literature review.

UNEP S (2020) Guidelines for social life cycle assessment of products and organizations 2020.

UNEP/SETAC (2009) United Nations Environment Programme-society of Environmental Toxicology and Chemistry. Guidelines for social life cycle assessment of products. France

United Nations (2011) Guiding principles on business and human rights. Implementing the United Nations “protect, respect and remedy” framework

United Nations (2015) Transforming our world: the 2030 agenda for sustainable development. sustainabledevelopment.un.org

Van Asselt ED, Van Bussel LGJ, Van Der Voet H et al (2014) A protocol for evaluating the sustainability of agri-food production systems - a case study on potato production in peri-urban agriculture in the Netherlands. Ecol Indic 43:315–321. https://doi.org/10.1016/j.ecolind.2014.02.027

Van der Ploeg JD (2014) Peasant-driven agricultural growth and food sovereignty. J Peasant Stud 41:999–1030. https://doi.org/10.1080/03066150.2013.876997

van Eck NJ, Waltman L (2010) Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics 84:523–538. https://doi.org/10.1007/s11192-009-0146-3

Van Eck NJ, Waltman L (2019) Manual for VOSviwer version 1.6.10. CWTS Meaningful metrics 1–53

Vasa L, Angeloska A, Trendov NM (2017) Comparative analysis of circular agriculture development in selected Western Balkan countries based on sustainable performance indicators. Econ Ann 168:44–47. https://doi.org/10.21003/ea.V168-09

Verdecho MJ, Alarcón-Valero F, Pérez-Perales D et al (2020) A methodology to select suppliers to increase sustainability within supply chains. Cent Eur J Oper Res. https://doi.org/10.1007/s10100-019-00668-3

Vergine P, Salerno C, Libutti A et al (2017) Closing the water cycle in the agro-industrial sector by reusing treated wastewater for irrigation. J Clean Prod 164:587–596. https://doi.org/10.1016/j.jclepro.2017.06.239

WCED (1987) Our common future - call for action

Webster K (2013) What might we say about a circular economy? Some temptations to avoid if possible. World Futures 69:542–554

Wheaton E, Kulshreshtha S (2013) Agriculture and climate change: implications for environmental sustainability indicators. WIT Trans Ecol Environ 175:99–110. https://doi.org/10.2495/ECO130091

Wijewickrama MKCS, Chileshe N, Rameezdeen R, Ochoa JJ (2021) Information sharing in reverse logistics supply chain of demolition waste: a systematic literature review. J Clean Prod 280:124359. https://doi.org/10.1016/j.jclepro.2020.124359

Woodhouse A, Davis J, Pénicaud C, Östergren K (2018) Sustainability checklist in support of the design of food processing. Sustain Prod Consum 16:110–120. https://doi.org/10.1016/j.spc.2018.06.008

Wu R, Yang D, Chen J (2014) Social Life Cycle Assessment Revisited Sustain 6:4200–4226. https://doi.org/10.3390/su6074200

Yadav S, Luthra S, Garg D (2021) Modelling Internet of things (IoT)-driven global sustainability in multi-tier agri-food supply chain under natural epidemic outbreaks. Environ Sci Pollut Res 16633–16654. https://doi.org/10.1007/s11356-020-11676-1

Yee FM, Shaharudin MR, Ma G et al (2021) Green purchasing capabilities and practices towards Firm’s triple bottom line in Malaysia. J Clean Prod 307:127268. https://doi.org/10.1016/j.jclepro.2021.127268

Yigitcanlar T (2010) Rethinking sustainable development: urban management, engineering, and design. IGI Global

Zamagni A, Amerighi O, Buttol P (2011) Strengths or bias in social LCA? Int J Life Cycle Assess 16:596–598. https://doi.org/10.1007/s11367-011-0309-3

Download references

Author information

Authors and affiliations.

Department of Economy, Engineering, Society and Business Organization, University of “Tuscia, ” Via del Paradiso 47, 01100, Viterbo, VT, Italy

Cecilia Silvestri, Michela Piccarozzi & Alessandro Ruggieri

Department of Engineering, University of Rome “Niccolò Cusano, ” Via Don Carlo Gnocchi, 3, 00166, Rome, Italy

Luca Silvestri

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Cecilia Silvestri .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Communicated by Monia Niero

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original online version of this article was revised: a number of ill-placed paragraph headings were removed and the source indication "Authors' elaborations" was added to Tables 1-3.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 31 KB)

Rights and permissions.

Reprints and permissions

About this article

Silvestri, C., Silvestri, L., Piccarozzi, M. et al. Toward a framework for selecting indicators of measuring sustainability and circular economy in the agri-food sector: a systematic literature review. Int J Life Cycle Assess (2022). https://doi.org/10.1007/s11367-022-02032-1

Download citation

Received : 15 June 2021

Accepted : 16 February 2022

Published : 02 March 2022

DOI : https://doi.org/10.1007/s11367-022-02032-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Agri-food sector
• Sustainability
• Circular economy
• Triple bottom line
• Life cycle assessment
• Find a journal
• Publish with us
• Track your research

• Community & Quality of Life

MSU approves $335M medical research center in Detroit

Officials also weighed in on concerns of MSU’s investments in Israel and the release of university documents regarding its investigation of Larry Nassar to state law enforcement.

A rendering of the Henry Ford Health and Michigan State University Health Sciences Research Center in Detroit.

Michigan State University’s Board of Trustees approved the construction of a new biomedical research center in Detroit at its meeting on Friday. Officials also weighed in on concerns of MSU’s investments in Israel and the release of university documents regarding its investigation of Larry Nassar to state law enforcement.

During its last meeting of the semester, the board unanimously voted to spend $335 million to build the 335,000-square-foot medical facility in Detroit’s New Center neighborhood.

The research center , a cornerstone of a 30-year agreement signed in 2021 between Henry Ford and MSU, is poised to become the university’s largest research facility yet.

“With the approval of the Henry Ford Health and Michigan State University Research Center, we are setting the stage for a future where health care innovation, education and community involvement come together,” said MSU Board of Trustees Chair Dan Kelly.

“This investment pushes the boundary of medical knowledge and signifies a commitment to advancing biomedical research and MSU’s commitment to the people of Detroit and the broader Michigan community.”

During his inaugural board meeting, MSU President Kevin Guskiewicz praised the board for creating opportunities to grow the university’s research in cancer, neuroscience, immunology and public health.

“As Spartans, collaborating to advance the common good is in our DNA, and we look forward to working with our partners to address health equity and other grand challenges of our time,” said Guskiewicz.

Construction on the research center is expected to begin in mid-May with an anticipated opening in 2027.

The research center is part of Henry Ford’s $3 billion “ Future of Health ” project that includes a new hospital tower as well as three mixed-use, mixed-income residential buildings. Last month, the Detroit City Council approved $296 million in state and local tax incentives over 35 years for the project.

MSU’s review of investments

At the board meeting, trustees said they are continuing to review the school’s foreign investment portfolio but declined to restrict its financial holdings in Israel.

“Divestment would conflict with stewarding the institution’s financial health would increase investment risk and limit returns and jeopardize the assurance that resources will continue to be available now and for future generations,” said MSU Trustee Sandy Pierce.

The announcement follows months of student and faculty outcry amid the country’s invasion and bombardment of Gaza. Many have called on MSU to divest from funds that support and profit from Israel’s military campaigns.

“This is a plausible genocide targeting a captive civilian population,” Jennifer Goett, an associate professor at MSU’s James Madison College, told trustees during the meeting. “Our families’ horror and trauma are only amplified knowing that MSU our employer invests in cities that support and profit from the killing of my husband’s relatives.”

MSU’s review of its investment portfolio is expected to be concluded at the end of May.

Following the board meeting, a group of about 100 pro-Palestinian protesters gathered at Cowles House, the on-campus home of the MSU president, to call for the university’s divestment from Israel.

MSU releases Larry Nassar documents to Attorney General

During the board meeting, MSU President Guskiewicz announced that the university is completing the transfer of documents related to its investigation of Larry Nassar to the Michigan Attorney General’s Office.

Despite cooperating with the long-standing request from state law enforcement, MSU officials are not planning to release the documents to the public for now.

“We have been advised until the investigation is completed, that those documents will remain with the AG and they will eventually at the appropriate time be turned over,” Guskiewicz said.

In 2018, Attorney General Dana Nessel asked MSU to share documents for the state’s investigation into how the university handled complaints of the disgraced doctor’s sexual abuse. Nessel eventually closed the case in 2021 due to a lack of cooperation from the university. She renewed that request in April 2023 amid new membership and leadership on the Board of Trustees.

MSU agreed to transfer the Nassar documents in December and began doing so last month.

Trusted, accurate, up-to-date.

WDET strives to make our journalism accessible to everyone. As a public media institution, we maintain our journalistic integrity through independent support from readers like you. If you value WDET as your source of news, music and conversation, please make a gift today .

Donate today »

View all posts