what is literature cited in research

• Langson Library
• Science Library
• Grunigen Medical Library
• Law Library
• Connect From Off-Campus
• Accessibility
• Gateway Study Center

Email this link

Writing a scientific paper.

• Writing a lab report
• INTRODUCTION

Literature Cited Section

Guides from other schools, citation styles & writing guides, "literature cited checklist" from: how to write a good scientific paper. chris a. mack. spie. 2018..

• Bibliography of guides to scientific writing and presenting
• Peer Review
• Presentations
• Lab Report Writing Guides on the Web

This is the last section of the paper. Here you should provide an alphabetical listing of all the published work you cited in the text of the paper. This does not mean every article you found in your research; only include the works you actually cited in the text of your paper. A standard format is used both to cite literature in the text and to list these studies in the Literature Cited section.  Hypothetical examples of the format used in the journal Ecology are below:     Djorjevic, M., D.W. Gabriel and B.G. Rolfe. 1987. Rhizobium: Refined parasite of legumes. Annual Review of Phytopathology 25: 145-168.     Jones, I. J. and B. J. Green. 1963. Inhibitory agents in walnut trees. Plant Physiology 70:101-152.     MacArthur, R.H. and E.O. Wilson. 1967. The Theory of Island Biogeography. Princeton University Press, Princeton, N.J.     Smith, E. A. 1949. Allelopathy in walnuts. American Journal of Botany 35:1066-1071. Here is a dissection of the first entry, in the format for Ecology :       Firstauthor, M., D.W. Secondauthor and B.G. Thirdauthor. Year. Article title with only the first letter capitalized. Journal Article Title with Important Words in Caps  volume#(issue# if there is one): firstpage-lastpage. Notice some of the following details:       - the list is alphabetized;     - no first or middle names are listed (the author's first and middle initials are used instead);     - only the first word in the title of the journal article (except for proper nouns) is capitalized;     - different journals use different styles for Literature Cited sections.   You should pay careful attention to details of formatting when you write your own Literature Cited section. For papers published in journals you must provide the date, title, journal name, volume number, and page numbers. For books you need the publication date, title, publisher, and place of publication.

• Bates College Guide to Citing Sources
• American Psychological Association (APA) style A guide to formatting papers using APA from Purdue University.
• APA 2007 Revision of Citation Styles An online revision of the information presented in the fifth edition of the Publication Manual of the American Psychological Association . more... less... This guide serves a resource for citation styles and uniform means of referencing authoritative works.
• APA Documentation (University of Wisconsin-Madison) A quick resource for citing references in papers using the 5th edition of the Publication Manual of the American Psychological Association (2001). Provided by The Writing Center at the University of Wisconsin-Madison.
• American Anthropological Association Style Manual Prepared for and preferred by the American Anthropological Association (AAA) using the Chicago Manual of Style. Citation examples listed from pages 10-14. Also recommend consulting the Chicago Manual of Style Online.

• Elements of Style This classic work by William Strunk is intended for use in which the practice of composition is combined with the study of literature. It gives the main requirements of plain English style and concentrates on the rules of usage most often abused.
• IEEE Editorial Style Manual This link will take you to a downloadable version of the IEEE Editorial Style Manual.
• Modern Language Association (MLA) style
• Purdue OWL (Online Writing Lab) Easy-to-use site that provides information and examples for using the American Psychological Association (APA) citation and format style and the Modern Language Association (MLA) citation and format style. Also included information about the Chicago Manual of Style (16th ed.)
• Include citations that provide sufficient context to allow for critical analysis of this
• work by others.
• Include citations that give the reader sources of background and related material so
• that the current work can be understood by the target audience.
• Include citations that provide examples of alternate ideas, data, or conclusions to
• compare and contrast with this work, if they exist. Do not exclude contrary evidence.
• Include citations that acknowledge and give credit to sources relied upon for this
• Are the citations up to date, referencing that latest work on this topic?
• It is the job of the authors to verify the accuracy of the references.
• Avoid: spurious citations (citations that are not needed but are included anyway);

biased citations (references added or omitted for reasons other than meeting the above goals of citations); excessive self-cites (citations to one’s own work). 

• << Previous: DISCUSSION
• Next: Bibliography of guides to scientific writing and presenting >>
• Last Updated: Aug 4, 2023 9:33 AM
• URL: https://guides.lib.uci.edu/scientificwriting

Off-campus? Please use the Software VPN and choose the group UCIFull to access licensed content. For more information, please Click here

Software VPN is not available for guests, so they may not have access to some content when connecting from off-campus.

Citing sources: Overview

• Citation style guides

Manage your references

Use these tools to help you organize and cite your references:

• Citation Management and Writing Tools

If you have questions after consulting this guide about how to cite, please contact your advisor/professor or the writing and communication center .

Why citing is important

It's important to cite sources you used in your research for several reasons:

• To show your reader you've done proper research by listing sources you used to get your information
• To be a responsible scholar by giving credit to other researchers and acknowledging their ideas
• To avoid plagiarism by quoting words and ideas used by other authors
• To allow your reader to track down the sources you used by citing them accurately in your paper by way of footnotes, a bibliography or reference list

About citations

Citing a source means that you show, within the body of your text, that you took words, ideas, figures, images, etc. from another place.

Citations are a short way to uniquely identify a published work (e.g. book, article, chapter, web site).  They are found in bibliographies and reference lists and are also collected in article and book databases.

Citations consist of standard elements, and contain all the information necessary to identify and track down publications, including:

• author name(s)
• titles of books, articles, and journals
• date of publication
• page numbers
• volume and issue numbers (for articles)

Citations may look different, depending on what is being cited and which style was used to create them. Choose an appropriate style guide for your needs.  Here is an example of an article citation using four different citation styles.  Notice the common elements as mentioned above:

Author - R. Langer

Article Title - New Methods of Drug Delivery

Source Title - Science

Volume and issue - Vol 249, issue 4976

Publication Date - 1990

Page numbers - 1527-1533

American Chemical Society (ACS) style:

Langer, R. New Methods of Drug Delivery. Science 1990 , 249 , 1527-1533.

IEEE Style:

R. Langer, " New Methods of Drug Delivery," Science , vol. 249 , pp. 1527-1533 , SEP 28, 1990 .

American Psychological Association   (APA) style:

Langer, R. (1990) . New methods of drug delivery. Science , 249 (4976), 1527-1533.

Modern Language Association (MLA) style:

Langer, R. " New Methods of Drug Delivery." Science 249.4976 (1990) : 1527-33.

What to cite

You must cite:

• Facts, figures, ideas, or other information that is not common knowledge

Publications that must be cited include:  books, book chapters, articles, web pages, theses, etc.

Another person's exact words should be quoted and cited to show proper credit 

When in doubt, be safe and cite your source!

Avoiding plagiarism

Plagiarism occurs when you borrow another's words (or ideas) and do not acknowledge that you have done so. In this culture, we consider our words and ideas intellectual property; like a car or any other possession, we believe our words belong to us and cannot be used without our permission.

Plagiarism is a very serious offense. If it is found that you have plagiarized -- deliberately or inadvertently -- you may face serious consequences. In some instances, plagiarism has meant that students have had to leave the institutions where they were studying.

The best way to avoid plagiarism is to cite your sources - both within the body of your paper and in a bibliography of sources you used at the end of your paper.

Some useful links about plagiarism:

• MIT Academic Integrity Overview on citing sources and avoiding plagiarism at MIT.
• Avoiding Plagiarism From the MIT Writing and Communication Center.
• Plagiarism: What It is and How to Recognize and Avoid It From Indiana University's Writing Tutorial Services.
• Plagiarism- Overview A resource from Purdue University.
• Next: Citation style guides >>
• Last Updated: Jan 16, 2024 7:02 AM
• URL: https://libguides.mit.edu/citing

• Directories
• What are citations and why should I use them?
• When should I use a citation?
• Why are there so many citation styles?
• Which citation style should I use?
• Chicago Notes Style
• Chicago Author-Date Style
• AMA Style (medicine)
• Bluebook (law)
• Additional Citation Styles
• Built-in Citation Tools
• Quick Citation Generators
• Citation Management Software
• Start Your Research
• Research Guides
• University of Washington Libraries
• Library Guides
• UW Libraries
• Citing Sources

Citing Sources: What are citations and why should I use them?

What is a citation.

Citations are a way of giving credit when certain material in your work came from another source. It also gives your readers the information necessary to find that source again-- it provides an important roadmap to your research process. Whenever you use sources such as books, journals or websites in your research, you must give credit to the original author by citing the source. 

Why do researchers cite?

Scholarship is a conversation  and scholars use citations not only to  give credit  to original creators and thinkers, but also to  add strength and authority  to their own work.  By citing their sources, scholars are  placing their work in a specific context  to show where they “fit” within the larger conversation.  Citations are also a great way to  leave a trail  intended to help others who may want to explore the conversation or use the sources in their own work.

In short, citations

(1) give credit

(2) add strength and authority to your work

(3) place your work in a specific context

(4) leave a trail for other scholars

"Good citations should reveal your sources, not conceal them. They should honeslty reflect the research you conducted." (Lipson 4)

Lipson, Charles. "Why Cite?"  Cite Right: A Quick Guide to Citation Styles--MLA, APA, Chicago, the Sciences, Professions, and More . Chicago: U of Chicago, 2006. Print.

What does a citation look like?

Different subject disciplines call for citation information to be written in very specific order, capitalization, and punctuation. There are therefore many different style formats. Three popular citation formats are MLA Style (for humanities articles) and APA or Chicago (for social sciences articles).

MLA style (print journal article):  

Whisenant, Warren A. "How Women Have Fared as Interscholastic Athletic Administrators Since the Passage of Title IX." Sex Roles Vol. 49.3 (2003): 179-182.

APA style (print journal article):

Whisenant, W. A. (2003) How Women Have Fared as Interscholastic Athletic Administrators Since the Passage of Title IX. Sex Roles , 49 (3), 179-182.

Chicago style (print journal article):

Whisenant, Warren A. "How Women Have Fared as Interscholastic Athletic Administrators Since the Passage of Title IX." Sex Roles 49, no. 3 (2003): 179-182.

No matter which style you use, all citations require the same basic information:

• Author or Creator
• Container (e.g., Journal or magazine, website, edited book)
• Date of creation or publication
• Publisher 

You are most likely to have easy access to all of your citation information when you find it in the first place. Take note of this information up front, and it will be much easier to cite it effectively later.

• << Previous: Basics of Citing
• Next: When should I use a citation? >>
• Last Updated: May 1, 2024 12:48 PM
• URL: https://guides.lib.uw.edu/research/citations

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

University of Lethbridge

Science Toolkit

Citations and the Literature Cited Section

Whenever you use facts or ideas taken from others, it is essential that you give credit to the original author. Failing to do so is plagiarism, and can result in severe penalties for students (or for any scientist). See Part 4 of the Calendar for more details on the University of Lethbridge’s plagiarism policy.

Scientists give credit for borrowed ideas and information through short citations in the text of papers, combined with a detailed reference in the Literature Cited section at the end of the paper. This not only gives credit to the original author, it provides support for the arguments being presented (by showing they are based on legitimate previous research), and allows readers to find the original papers being cited and check that the information has been duplicated accurately.

Keep in mind that citations aren’t limited to direct quotes (which should be used sparingly if at all in research papers). Citations should be used whenever you use information which is not common knowledge, or derived from your own experience or imagination.

How do I cite? Most scientific journals cite using the Name-Year system. In the text of your paper insert a short bracketed reference to the last name of the author and year of publication. For example if you are deriving information from a paper by J.D. Smith, and published in 1987, you would include (Smith 1987) in the text, as close as possible to the information being derived from Smith’s paper. If there are two authors, list both, as in (Smith and Weston 1976). Never change the order of the names from that presented in the original paper. If there are three or more authors, list only the first, followed by et al., as in (Smith et al. 1985). Et al. is written in italic font (or underlined) because it is an abbreviation of the Latin phrase “et alia” meaning “and the others.” If you have two papers by the same author(s) and from the same year, you should add letters to keep them separate, as in (Smith 1987a, Smith 1987b). Notice that several citations can be included in the same set of parentheses. You can also use your citation as the subject of the sentence. In this case, only the year is placed in parentheses. Below are a few examples of sentences containing citations.

Universal affirmatives may be only partially converted (Cleese 1971).

The arrival of the Spanish inquisition was almost always unexpected (Palin and Chapman 1972).

Instances of tri-gluteus syndrome are rare but have been documented (Cleese et al. 1973).

Jones and Palin (1975) found that African swallows could carry heavier loads than their European counterparts.

How often should I cite? Adding a citation to every sentence can make a paper cumbersome and harder to read. When a series of ideas in a paragraph all come from the same source, it is sometimes possible to use a single citation for all. But your first priority must be to make it clear to your reader exactly what information is being cited. If in doubt, include a citation.

How do I prepare a Literature Cited section? Your literature cited section provides detailed information on every paper cited in the text of your paper, but does not include any references not cited. It is not a bibliography of every book you consulted during your research. It only includes those from which you drew specific ideas or information.

The Literature Cited section comes at the end of your paper and is organized alphabetically by the last name of the first author. (The order of authors often ranks the role they played in the research and writing, and should never be changed.) Other information to be included is:

Initials of each author. The year of publication. The title of the paper (or book). The title of the journal (for a journal article). The volume of the journal (for a journal article). The page numbers in the journal (for a journal article). Publishing company (for a book). Place of publication (for a book). A typical Literature Cited entry for a journal article would look like this:

Beadle, G.W. and Tatum, E.L. 1942. Genetic control of a biochemical pathway in Neurospora. Proc. Natl. Acad. Sci. USA 27:499-506.

And an entry for a book would look like this:

Dawkins, R. 1989. The Selfish Gene: New Edition. Oxford University Press, Oxford.

They each provide all the information needed to go to the library, and find the article or book in question. Notice that the title of the journal is abbreviated. This is common practice, and can be frustrating for students. But you will quickly learn the abbreviations used for journals in your chosen field. Most are fairly intuitive.

What about style? Unfortunately, there is no universal style in use for citations. Fortunately, the differences are small, and typically involve relatively minor issues such as whether all words in the title are capitalized, or whether journal titles are written in italic font. The best solution is to find a paper in the journal you are submitting your paper to, or that you have been told by an instructor to use as a template. Photocopy a few pages from Lit. Cited sections, and use them as a style guide. This can also serve as a guide to proper format for types of reference not covered here.

What about WWW sites? No standardized format has yet been adopted for citing information available on the worldwide web. The difficulty with a web address is that it is not permanent. Once a book is published, it should be available indefinitely (although possibly difficult to locate), and the text will never change. A document published on the web may change daily, and can disappear without notice. When citing information found on the web, emphasis should be placed on author and publisher rather than simply the web address.

What about numbered citations? A few science journals use numbered citations (formally called the Citation-Sequence system) rather than the Name-Year system described above. Here the borrowed information is identified by a number placed in the text. The first identifier in the text becomes one, the second two, and so on. Usually this number is included as a superscript at the end of a sentence, but in some journals it is placed in parentheses. The detailed information on each reference is included in a list of endnotes rather than a Lit. Cited section. The main difference is that the list is organized by number rather than alphabetically. Numbers are used as a space-saving device, but are not as easy for the reader to follow as standard citations. Use the Citation-Sequence system in a paper only if you are submitting it to a journal which requires this format, or you have been specifically asked to use this format by an instructor.

Conduct a literature review

What is a literature review.

A literature review is a summary of the published work in a field of study. This can be a section of a larger paper or article, or can be the focus of an entire paper. Literature reviews show that you have examined the breadth of knowledge and can justify your thesis or research questions. They are also valuable tools for other researchers who need to find a summary of that field of knowledge.

Unlike an annotated bibliography, which is a list of sources with short descriptions, a literature review synthesizes sources into a summary that has a thesis or statement of purpose—stated or implied—at its core.

How do I write a literature review?

Step 1: define your research scope.

• What is the specific research question that your literature review helps to define?
• Are there a maximum or minimum number of sources that your review should include?

Ask us if you have questions about refining your topic, search methods, writing tips, or citation management.

Step 2: Identify the literature

Start by searching broadly. Literature for your review will typically be acquired through scholarly books, journal articles, and/or dissertations. Develop an understanding of what is out there, what terms are accurate and helpful, etc., and keep track of all of it with citation management tools . If you need help figuring out key terms and where to search, ask us .

Use citation searching to track how scholars interact with, and build upon, previous research:

• Mine the references cited section of each relevant source for additional key sources
• Use Google Scholar or Scopus to find other sources that have cited a particular work

Step 3: Critically analyze the literature

Key to your literature review is a critical analysis of the literature collected around your topic. The analysis will explore relationships, major themes, and any critical gaps in the research expressed in the work. Read and summarize each source with an eye toward analyzing authority, currency, coverage, methodology, and relationship to other works. The University of Toronto's Writing Center provides a comprehensive list of questions you can use to analyze your sources.

Step 4: Categorize your resources

Divide the available resources that pertain to your research into categories reflecting their roles in addressing your research question. Possible ways to categorize resources include organization by:

• methodology
• theoretical/philosophical approach

Regardless of the division, each category should be accompanied by thorough discussions and explanations of strengths and weaknesses, value to the overall survey, and comparisons with similar sources. You may have enough resources when:

• You've used multiple databases and other resources (web portals, repositories, etc.) to get a variety of perspectives on the research topic.
• The same citations are showing up in a variety of databases.

Additional resources

Undergraduate student resources.

• Literature Review Handout (University of North Carolina at Chapel Hill)
• Learn how to write a review of literature (University of Wisconsin-Madison)

Graduate student and faculty resources

• Information Research Strategies (University of Arizona)
• Literature Reviews: An Overview for Graduate Students (NC State University)
• Oliver, P. (2012). Succeeding with Your Literature Review: A Handbook for Students [ebook]
• Machi, L. A. & McEvoy, B. T. (2016). The Literature Review: Six Steps to Success

Graustein, J. S. (2012). How to Write an Exceptional Thesis or Dissertation: A Step-by-Step Guide from Proposal to Successful Defense [ebook]

Thomas, R. M. & Brubaker, D. L. (2008). Theses and Dissertations: A Guide to Planning, Research, and Writing

Boatwright Memorial Library

Citing sources research guide: literature reviews.

• Quick Citing in Databases
• Chicago/Turabian
• ASA & AAA (Soc/Ant)
• ACS (Chemistry)
• AP (Associated Press)
• APSA (Political Science)
• SBL (Society of Biblical Literature)
• Harvard Style
• MS Word Help
• DOIs (Digital Object Identifiers)
• Annotated Bibliographies
• Literature Reviews
• EndNote Web
• Citation exercises This link opens in a new window

Literature Reviews: Overview

This video from NCSU Libraries gives a helpful overview of literature reviews. Even though it says it's "for graduate students," the principles are the same for undergraduate students too!

Reading a Scholarly Article

• Reading a Scholarly Article or Literature Review Highlights sections of a scholarly article to identify structure of a literature review.
• Anatomy of a Scholarly Article (NCSU Libraries) Interactive tutorial that describes parts of a scholarly article typical of a Sciences or Social Sciences research article.
• Evaluating Information | Reading a Scholarly Article (Brown University Library) Provides examples and tips across disciplines for reading academic articles.
• Reading Academic Articles for Research [LIBRE Project] Gabriel Winer & Elizabeth Wadell (ASCCC Open Educational Resources Initiative (OERI))

Literature Review Examples

What is a Literature Review?

The literature review is a written explanation by you, the author, of the research already done on the topic, question or issue at hand. What do we know (or not know) about this issue/topic/question?

• A literature review provides a thorough background of the topic by giving your reader a guided overview of major findings and current gaps in what is known so far about the topic. 
• The literature review is not a list (like an annotated bibliography) -- it is a narrative helping your reader understand the topic and where you will "stand" in the debate between scholars regarding the interpretation of meaning and understanding why things happen. Your literature review  helps your reader start to see the "camps" or "sides" within a debate, plus who studies the topic and their arguments. 
• A good literature review should help the reader sense how you will answer your research question and should highlight the preceding arguments and evidence you think are most helpful in moving the topic forward.
• The purpose of the literature review is to dive into the existing debates on the topic to learn about the various schools of thought and arguments, using your research question as an anchor. If you find something that doesn't help answer your question, you don't have to read (or include) it. That's the power of the question format: it helps you filter what to read and include in your literature review, and what to ignore.

How Do I Start?

Essentially you will need to:

• Identify and evaluate relevant literature (books, journal articles, etc.) on your topic/question.
• Figure out how to classify what you've gathered. You could do this by schools of thought, different answers to a question, the authors' disciplinary approaches, the research methods used, or many other ways.
• Use those groupings to craft a narrative, or story, about the relevant literature on this topic. 
• Remember to cite your sources properly! 
• Research: Getting Started Visit this guide to learn more about finding and evaluating resources.
• Literature Review: Synthesizing Multiple Sources (IUPUI Writing Center) An in-depth guide on organizing and synthesizing what you've read into a literature review.
• Guide to Using a Synthesis Matrix (NCSU Writing and Speaking Tutorial Service) Overview of using a tool called a Synthesis Matrix to organize your literature review.
• Synthesis Matrix Template (VCU Libraries) A word document from VCU Libraries that will help you create your own Synthesis Matrix.

Additional Tutorials and Resources

• UR Writer's Web: Using Sources Guidance from the UR Writing Center on how to effectively use sources in your writing (which is what you're doing in your literature review!).
• Write a Literature Review (VCU Libraries) "Lit Reviews 101" with links to helpful tools and resources, including powerpoint slides from a literature review workshop.
• Literature Reviews (UNC Writing Center) Overview of the literature review process, including examples of different ways to organize a lit review.
• “Ten Simple Rules for Writing a Literature Review.” Pautasso, Marco. “Ten Simple Rules for Writing a Literature Review.” PLOS Computational Biology, vol. 9, no. 7, July 2013, p. e1003149.
• Writing the Literature Review Part I (University of Maryland University College) Video that explains more about what a literature review is and is not. Run time: 5:21.
• Writing the Literature Review Part II (University of Maryland University College) Video about organizing your sources and the writing process. Run time: 7:40.
• Writing a Literature Review (OWL @ Purdue)
• << Previous: Annotated Bibliographies
• Next: Zotero >>
• Last Updated: Mar 20, 2024 9:09 AM
• URL: https://libguides.richmond.edu/citingsources

Citation Styles

• Chicago Style
• Annotated Bibliographies

What is a Lit Review?

How to write a lit review.

• Video Introduction to Lit Reviews

Main Objectives

Examples of lit reviews, additional resources.

• Zotero (Citation Management)

What is a literature review?

• Either a complete piece of writing unto itself or a section of a larger piece of writing like a book or article
• A thorough and critical look at the information and perspectives that other experts and scholars have written about a specific topic
• A way to give historical perspective on an issue and show how other researchers have addressed a problem
• An analysis of sources based on your own perspective on the topic
• Based on the most pertinent and significant research conducted in the field, both new and old

• A descriptive list or collection of summaries of other research without synthesis or analysis
• An annotated bibliography
• A literary review (a brief, critical discussion about the merits and weaknesses of a literary work such as a play, novel or a book of poems)
• Exhaustive; the objective is not to list as many relevant books, articles, reports as possible
• To convey to your reader what knowledge and ideas have been established on a topic
• To explain what the strengths and weaknesses of that knowledge and those ideas might be
• To learn how others have defined and measured key concepts    
• To keep the writer/reader up to date with current developments and historical trends in a particular field or discipline
• To establish context for the argument explored in the rest of a paper
• To provide evidence that may be used to support your own findings
• To demonstrate your understanding and your ability to critically evaluate research in the field
• To suggest previously unused or underused methodologies, designs, and quantitative and qualitative strategies
• To identify gaps in previous studies and flawed methodologies and/or theoretical approaches in order to avoid replication of mistakes
• To help the researcher avoid repetition of earlier research
• To suggest unexplored populations
• To determine whether past studies agree or disagree and identify strengths and weaknesses on both sides of a controversy in the literature

• Choose a topic that is interesting to you; this makes the research and writing process more enjoyable and rewarding.
• For a literature review, you'll also want to make sure that the topic you choose is one that other researchers have explored before so that you'll be able to find plenty of relevant sources to review.

• Your research doesn't need to be exhaustive. Pay careful attention to bibliographies. Focus on the most frequently cited literature about your topic and literature from the best known scholars in your field. Ask yourself: "Does this source make a significant contribution to the understanding of my topic?"
• Reading other literature reviews from your field may help you get ideas for themes to look for in your research. You can usually find some of these through the library databases by adding literature review as a keyword in your search.
• Start with the most recent publications and work backwards. This way, you ensure you have the most current information, and it becomes easier to identify the most seminal earlier sources by reviewing the material that current researchers are citing.

The organization of your lit review should be determined based on what you'd like to highlight from your research. Here are a few suggestions:

• Chronology : Discuss literature in chronological order of its writing/publication to demonstrate a change in trends over time or to detail a history of controversy in the field or of developments in the understanding of your topic.  
• Theme: Group your sources by subject or theme to show the variety of angles from which your topic has been studied. This works well if, for example, your goal is to identify an angle or subtopic that has so far been overlooked by researchers.  
• Methodology: Grouping your sources by methodology (for example, dividing the literature into qualitative vs. quantitative studies or grouping sources according to the populations studied) is useful for illustrating an overlooked population, an unused or underused methodology, or a flawed experimental technique.

• Be selective. Highlight only the most important and relevant points from a source in your review.
• Use quotes sparingly. Short quotes can help to emphasize a point, but thorough analysis of language from each source is generally unnecessary in a literature review.
• Synthesize your sources. Your goal is not to make a list of summaries of each source but to show how the sources relate to one another and to your own work.
• Make sure that your own voice and perspective remains front and center. Don't rely too heavily on summary or paraphrasing. For each source, draw a conclusion about how it relates to your own work or to the other literature on your topic.
• Be objective. When you identify a disagreement in the literature, be sure to represent both sides. Don't exclude a source simply on the basis that it does not support your own research hypothesis.
• At the end of your lit review, make suggestions for future research. What subjects, populations, methodologies, or theoretical lenses warrant further exploration? What common flaws or biases did you identify that could be corrected in future studies?

• Double check that you've correctly cited each of the sources you've used in the citation style requested by your professor (APA, MLA, etc.) and that your lit review is formatted according to the guidelines for that style.

Your literature review should:

• Be focused on and organized around your topic.
• Synthesize your research into a summary of what is and is not known about your topic.
• Identify any gaps or areas of controversy in the literature related to your topic.
• Suggest questions that require further research.
• Have your voice and perspective at the forefront rather than merely summarizing others' work.
• Cyberbullying: How Physical Intimidation Influences the Way People are Bullied
• Use of Propofol and Emergence Agitation in Children
• Eternity and Immortality in Spinoza's 'Ethics'
• Literature Review Tutorials and Samples - Wilson Library at University of La Verne
• Literature Reviews: Introduction - University Library at Georgia State
• Literature Reviews - The Writing Center at UNC Chapel Hill
• Writing a Literature Review - Boston College Libraries
• Write a Literature Review - University Library at UC Santa Cruz
• << Previous: Annotated Bibliographies
• Next: Zotero (Citation Management) >>
• Last Updated: Jan 17, 2024 2:47 PM
• URL: https://researchguides.elac.edu/Citation

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Citation Resources

Literature Review: The What, Why and How-to Guide — Citation Resources

• Getting Started
• Introduction
• How to Pick a Topic
• Strategies to Find Sources
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Other Academic Writings

Plagarism - What is it and how to avoid it

From UConn’s Community Standards : “Academic misconduct is dishonest or unethical academic behavior that includes, but is not limited, to misrepresenting mastery in an academic area (e.g., cheating), failing to properly credit information, research or ideas to their rightful originators or representing such information, research or ideas as your own (e.g., plagiarism).” — University of Connecticut, Community Standards, Appendix A

The best way to avoid plagiarizing on your paper is to cite your sources using one of the many citations style used in academia. The Citation Guides and Management Tools Guide is your one stop shop to learn more about the most commonly used citation styles.

• Citation Styles and Management Tools Guide by Samuel Boss Last Updated Apr 9, 2024 2418 views this year

Purpose of Citations, When and What to Cite?

• OWL Purdue: Research and Citation Resources Explains in detail how, when, and why to use this citation style for both print and online sources, with an emphasis to the major citation styles: APA, MLA and Chicago.

There are four main reasons:

• To acknowledge the author(s) of the work that you used to write your paper.
• To provide context to your research and demonstrate that your paper is well-researched.
• To allow readers to find the original source and learn more about some aspect that you mentioned only briefly in the document.
• To enable further research by letting others discover what has already been explored and written about on a given topic.

What and When to Cite?

You should always cite other people's words, ideas and other intellectual property that you use in your papers or that influence your ideas. This includes but isn't limited to books, journal articles, web pages, reports, data, statistics, speeches, lectures, personal interviews, etc. You should cite whenever you:

• use a direct quote
• use facts or statistics that are relatively less known or relate directly to your argument.

Stable Links

A stable link is a web address that will consistently point to a specific information source such as an ebook, an article, a record in the catalog, a video, or a database. A stable link may also be called a permalink, document URL, persistent URL, or durable URL depending on the resource. You may also use a DOI (digital object identifier) found in many databases.

When citing online references your citation should look something like this:

Rivera Villegas, Carmen M. "La loca de la casa" de Marta Aponte Alsina: Reinvenciones romanticas de un canon fundacional.” Confluencia: Revista Hispanica de Cultura y Literatura , vol. 23, no. 1, 2007, p. 62, www.jstor.org/stable/27923253. Accessed 20 May 2009.

Rivera Villegas, Carmen M. "La loca de la casa" de Marta Aponte Alsina: Reinvenciones romanticas de un canon fundacional.” Confluencia: Revista Hispanica de Cultura y Literatura , vol. 23, no. 1, 2007, p. 62, JSTOR , doi:10.1353/mfs.1997.0056.

• << Previous: Writing Literature Review: Useful Sites
• Next: Other Academic Writings >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: May 15, 2024 9:53 AM
• URL: https://libguides.usc.edu/writingguide

• My Library Account
• Articles, Books & More
• Course Reserves
• Site Search
• Advanced Search
• Sac State Library
• Research Guides
• Biological & Environmental Sciences

Ecology Format

• Literature Cited/Bibliography
• In-text Citations
• Journal Article
• Government Document
• Tutorial for Ecology Format

General Hints

The Literature Cited section (bibliography) is found at the end of your paper and contains the complete reference for each of the in-text citations used in your paper. Generally, a citation includes the author(s), date, title and source of your publication.

• The entries in the Literature Cited section should be in one alphabetical listing by author
• Within each entry, list the authors in the order that they appear in the article you read
• Include as many initials for each author as you find on the first page of the article
• All words in the journal title should be spelled out completely
• Only capitalize the first word in the article title as well as proper names
• Type the title exactly as you see it in the journal article
• Do not include any informtion about the database that you used to access the article.
• If the journal is only available online and has no print eqivalent, include a DOI number
• If there are beween 2 and 12 authors, include all
• Italicize species names

Change Up: From Article to Lit Cited

Look at the image below to see how a citation is created from an article.

• << Previous: In-text Citations
• Next: Journal Article >>
• Last Updated: Jan 26, 2023 3:12 PM
• URL: https://csus.libguides.com/ecology

It is very important that you give credit to the source of any information or ideas that you present in your essay. The source of the information or ideas must be credited directly in the text and this is known as citing literature . Usually, the sources you cite will be your primary articles, popular article, books or texts. However, you should also cite any personal communications with researchers or instructors. Try to avoid direct quotes. These are appropriate when you wish to quote a pithy phrase that you cannot express better in your own words. In general, you should rephrase the information or ideas cited into your own words and cite the source appropriately. We suggest that you use the guidelines below in citing literature in your essay. However, you may use any format you wish as long as your citation style is accurate and consistent. We would prefer that you avoid footnotes, although you will not be penalized for them. Citing literature in the body of your essay: When the source has a single author:

Researching and writing for Economics students

4 literature review and citations/references.

Figure 4.1: Literature reviews and references

Your may have done a literature survey as part of your proposal. This will be incorporated into your dissertation, not left as separate stand-alone. Most economics papers include a literature review section, which may be a separate section, or incorporated into the paper’s introduction. (See organising for a standard format.)

Some disambiguation:

A ‘Literature survey’ paper: Some academic papers are called ‘literature surveys’. These try to summarise and discuss the existing work that has been done on a particular topic, and can be very useful. See, for example, works in The Journal of Economic Perspectives, the Journal of Economic Literature, the “Handbook of [XXX] Economics”

Many student projects and undergraduate dissertations are mainly literature surveys.

4.1 What is the point of a literature survey?

Your literature review should explain:

what has been done already to address your topic and related questions, putting your work in perspective, and

what techniques others have used, what are their strengths and weaknesses, and how might they be relevant tools for your own analysis.

Figure 4.2: Take notes on this as you read, and write them up.

4.2 What previous work is relevant?

Focus on literature that is relevant to your topic only.

But do not focus only on articles about your exact topic ! For example, if your paper is about the relative price of cars in the UK, you might cite papers (i) about the global automobile market, (ii) about the theory and evidence on competition in markets with similar features and (iii) using econometric techniques such as “hedonic regression” to estimate “price premia” in other markets and in other countries.

Consider: If you were Colchester a doctor and wanted to know whether a medicine would be effective for your patients, would you only consider medical studies that ran tests on Colchester residents, or would you consider more general national and international investigations?

4.3 What are “good” economics journal articles?

You should aim to read and cite peer-reviewed articles in reputable economics journals. (Journals in other fields such as Finance, Marketing and Political Science may also be useful.) These papers have a certain credibility as they have been checked by several referees and one or more editors before being published. (In fact, the publication process in Economics is extremely lengthy and difficult.)

Which journals are “reputable”? Economists spend a lot of time thinking about how to rank and compare journals (there are so many papers written about this topic that they someone could start a “Journal of Ranking Economics Journals”. For example, “ REPEC ” has one ranking, and SCIMAGO/SCOPUS has another one. You may want to focus on journals ranked in the top 100 or top 200 of these rankings. If you find it very interesting and relevant paper published somewhere that is ranked below this, is okay to cite it, but you may want to be a bit more skeptical of its findings.

Any journal you find on JSTOR is respectable, and if you look in the back of your textbooks, there will be references to articles in journals, most of which are decent.

You may also find unpublished “working papers”; these may also be useful as references. However, it is more difficult to evaluate the credibility of these, as they have not been through a process of peer review. However, if the author has published well and has a good reputation, it might be more likely that these are worth reading and citing.

Unpublished “working papers”

You may also find unpublished “working papers” or ‘mimeos’; these may also be useful as references. In fact, the publication process in Economics is so slow (six years from first working paper to publication is not uncommon) that not consulting working papers often means not being current.

However, it is more difficult to evaluate the credibility of this ‘grey literature’, as they have not been through a process of peer review. However, if the author has published well and has a good reputation, it might be more likely that these are worth reading and citing. Some working paper series are vetted, such as NBER; in terms of credibility, these might be seen as something in between a working paper and a publication.

Which of the following are “peer-reviewed articles in reputable economics journals”? Which of the following may be appropriate to cite in your literature review and in your final project? 8

Klein, G, J. (2011) “Cartel Destabilization and Leniency Programs – Empirical Evidence.” ZEW - Centre for European Economic Research Discussion Paper No. 10-107

Spencer, B. and Brander, J.A. (1983) “International R&D Rivalry and Industrial Strategy”, Review of Economic Studies Vol. 50, 707-722

Troisi, Jordan D., Andrew N. Christopher, and Pam Marek. “Materialism and money spending disposition as predictors of economic and personality variables.” North American Journal of Psychology 8.3 (2006): 421.

The Economist,. ‘Good, Bad And Ugly’. Web. 11 Apr. 2015. [accessed on…]

Mecaj, Arjola, and María Isabel González Bravo. “CSR Actions and Financial Distress: Do Firms Change Their CSR Behavior When Signals of Financial Distress Are Identified?.” Modern Economy 2014 (2014).

Universities, U. K. “Creating Prosperity: the role of higher education in driving the UK’s creative economy.” London Universities UK (2010).

4.4 How to find and access articles

You should be able to find and access all the relevant articles online. Leafing through bound volumes and photocopying should not be neededs. (Having been a student in the late 90’s and 2000’s, I wish I could get those hours back.)

Figure 4.3: The old way!

Good online tools include Jstor (jstor.org) and Google Scholar (scholar.google.co.uk). Your university should have access to Jstor, and Google is accessible to all (although the linked articles may require special access). You will usually have the ‘most access’ when logged into your university or library computing system.If you cannot access a paper, you may want to consult a reference librarian.

It is also ok, if you cannot access the journal article itself, to use the last working paper version (on Google scholar find this in the tab that says “all X versions”, where X is some number, and look for a PDF). However, authors do not always put up the most polished versions, although they should do to promote open-access. As a very last resort, you can e-mail the author and ask him or her to send you the paper.

When looking for references, try to find ones published in respected refereed economics journals (see above ).

4.5 Good starting points: Survey article, course notes, and textbooks

A “survey article” is a good place to start; this is a paper that is largely a categorization and discussion of previous work on a particular topic. You can often find such papers in journals such as

• the Journal of Economic Perspectives,
• the Journal of Economic Surveys,
• and the Journal of Economic Literature.

These will be useful as a “catalog” of papers to read and considers citing. They are also typically very readable and offer a decent introduction to the issue or the field.

It is also helpful to consult module (course) notes and syllabi from the relevant field. Do not only limit yourself to the ones at your own university; many of universities make their course materials publicly accessible online. These will not only typically contain reading lists with well-respected and useful references, they may also contain slides and other material that will help you better understand your topic and the relevant issues.

However, be careful not to take material from course notes without properly citing it. (Better yet, try to find the original paper that the course notes are referring to.)

Textbooks serve as another extremely useful jumping off point. Look through your own textbooks and other textbooks in the right fields. Textbooks draw from, and cite a range of relevant articles and papers. (You may also want to go back to textbooks when you are finding the articles you are reading too difficult. Textbooks may present a simpler version of the material presented in an article, and explain the concepts better.)

4.6 Backwards and forwards with references

When you find a useful paper, look for its “family.” You may want to go back to earlier, more fundamental references, by looking at the articles that this paper cited. See what is listed as “keywords” (these are usually given at the top of the paper), and “JEL codes”. Check what papers this paper cites, and check what other papers cited this paper. On Google scholar you can follow this with a link “Cited by…” below the listed article. “Related articles” is also a useful link.

4.7 Citations

Keep track of all references and citations

You may find it helpful to use software to help you manage your citations

A storage “database” of citations (e.g., Jabref, Zotero, Endnote, Mendeley); these interface well with Google Scholar and Jstor

An automatic “insert citation” and “insert bibliography” in your word processing software

Use a tool like Endnote to manage and insert the bibliographies, or use a bibliography manager software such as Zotero or Jabref,

Further discussion: Citation management tools

List of works cited

Put your list of references in alphabetical order by author’s last name (surname).

Include all articles and works that you cite in your paper; do not include any that you don’t cite.

Avoiding plagiarism and academic offenses**

Here is a definition of plagiarism

The main point is that you need to cite everything that is not your own work. Furthermore, be clear to distinguish what is your own work and your own language and what is from somewhere/someone else.

Why cite? Not just to give credit to others but to make it clear that the remaining uncited content is your own.

Here are some basic rules:

(Rephrased from University of Essex material, as seen in Department of Economics, EC100 Economics for Business Handbook 2017-18, https://www1.essex.ac.uk/economics/documents/EC100-Booklet_2017.pdf accessed on 20 July 2019, pp. 15-16)

Do not submit anything that is not your own work.

Never copy from friends.

Do not copy your own work or previously submitted work. (Caveat: If you are submitting a draft or a ‘literature review and project plan’ at an earlier stage, this can be incorporated into your final submission.

Don’t copy text directly into your work, unless:

• you put all passages in quotation marks: beginning with ’ and ending with ’, or clearly offset from the main text
• you cite the source of this text.

It is not sufficient merely to add a citation for the source of copied material following the copied material (typically the end of a paragraph). You must include the copied material in quotation marks. … Ignorance … is no defence.’ (ibid, pp. 15 )

(‘Ibid’ means ‘same as the previous citation’.)

Your university may use sophisticated plagiarism-detection software. Markers may also report if the paper looks suspect

Before final submission, they may ask you to go over your draft and sign that you understand the contents and you have demonstrated that the work is your own.

Not being in touch with your supervisor may put you under suspicion.

Your university may give a Viva Voce oral exam if your work is under suspicion. It is a cool-sounding word but probably something you want to avoid.

Your university may store your work in its our database, and can pursue disciplinary action, even after you have graduated.

Penalties may be severe, including failure with no opportunity to retake the module (course). You may even risk your degree!

Comprehension questions; answers in footnotes

True or false: “If you do not directly quote a paper you do not need to cite it” 9

You should read and cite a paper (choose all that are correct)… 10

• If it motivates ‘why your question is interesting’ and how it can be modeled economically
• Only if it asks the same question as your paper
• Only if it is dealing with the same country/industry/etc as you are addressing
• If it has any connection to your topic, question, or related matters
• If it answers a similar question as your paper
• If it uses and discusses techniques that inform those you are using

4.8 How to write about previous authors’ analysis and findings

Use the right terminology.

“Johnson et al. (2000) provide an analytical framework that sheds substantial doubt on that belief. When trying to obtain a correlation between institutional efficiency and wealth per capita, they are left with largely inconclusive results.”

They are not trying to “obtain a correlation”; they are trying to measure the relationship and test hypotheses.

“Findings”: Critically examine sources

Don’t take everything that is in print (or written online) as gospel truth. Be skeptical and carefully evaluate the arguments and evidence presented. Try to really survey what has been written, to consider the range of opinions and the preponderance of the evidence. You also need to be careful to distinguish between “real research” and propaganda or press releases.

The returns to higher education in Atlantis are extremely high. For the majority of Atlanian students a university degree has increased their lifetime income by over 50%, as reported in the “Benefits of Higher Education” report put out by the Association of Atlantian Universities (2016).

But don’t be harsh without explanation:

Smith (2014) found a return to education in Atlantis exceeding 50%. This result is unlikely to be true because the study was not a very good one.

“Findings:” “They Proved”

A theoretical economic model can not really prove anything about the real world; they typically rely on strong simplifying assumptions.

Through their economic model, they prove that as long as elites have incentives to invest in de facto power, through lobbying or corruption for example, they will invest as much as possible in order to gain favourable conditions in the future for their businesses.

In their two period model, which assumes \[details of key assumptions here\] , they find that when an elite Agent has an incentive to invest in de facto power, he invests a strictly positive amount, up to the point where marginal benefit equals marginal cost”

Empirical work does not “prove” anything (nor does it claim to).

It relies on statistical inference under specific assumptions, and an intuitive sense that evidence from one situation is likely to apply to other situations.

“As Smith et al (1999) proved using data from the 1910-1920 Scandanavian stock exchange, equity prices always increase in response to reductions in corporate tax rates.”

“Smith et al (199) estimated a VAR regression for a dynamic CAP model using data from the 1910-1920 Scandanavian stock exchange. They found a strongly statistically significant negative coefficient on corporate tax rates. This suggests that such taxes may have a negative effect on publicly traded securities. However, as their data was from a limited period with several simultaneous changes in policy, and their results are not robust to \[something here\] , further evidence is needed on this question.”

Use the language of classical 11 statistics:

Hypothesis testing, statistical significance, robustness checks, magnitudes of effects, confidence intervals.

Note that generalisation outside the data depends on an intuitive sense that evidence from one situation is likely to apply to other situations.

“Findings”: How do you (or the cited paper) claim to identify a causal relationship?

This policy was explained by Smith and Johnson (2002) in their research on subsidies and redistribution in higher education. Their results showed that people with higher degree have higher salaries and so pay higher taxes. Thus subsidizing higher education leads to a large social gain.

The results the student discusses seem to show an association between higher degrees and higher salaries. The student seems to imply that the education itself led to higher salaries. This has not been shown by the cited paper. Perhaps people who were able to get into higher education would earn higher salaries anyway. There are ways economists used to try to identify a “causal effect” (by the way, this widely used term is redundant as all effects must have a cause), but a mere association between two variables is not enough

As inflation was systematically lower during periods of recession, we see that too low a level of inflation increases unemployment.

Economists have long debated the nature of this “Phillips curve” relationship. There is much work trying to determine whether the association (to the extent it exists) is a causal one. We could not rule out reverse causality, or third factor that might cause changes in both variables.

4.9 …Stating empirical results

Don’t write: “I accept the null hypothesis.”

Do write: “The results fail to reject the null hypothesis, in spite of a large sample size and an estimate with small standard errors” (if this is the case)

Note: The question of what to infer from acceptance/rejection of null hypotheses is a complex difficult one in Classical (as opposed to Bayesian) statistics. This difficulty is in part philosophical: classical hypothesis testing is deductive , while inference is necessarily inductive.

4.10 What to report

You need to read this paper more clearly; it is not clear what they conclude nor what their evidence is.

4.11 Organising your literature review

A common marking comment:

These papers seem to be discussed in random order – you need some structure organising these papers thematically, by finding, by technique, or chronologically perhaps.

How should you organise it? In what order?

Thematically (usually better)

By method, by theoretical framework, by results or assumptions, by field

Chronologically (perhaps within themes)

Exercise: Compare how the literature review section is organized in papers you are reading.

Figure 4.4: Organising a set of references

Q: What sort of structure am I using in the above outline?

It may also be helpful to make a ‘table’ of the relevant literature, as in the figure below. This will help you get a sense of the methods and results, and how the papers relate, and how to assess the evidence. You may end up putting this in the actual paper.

Figure 4.5: Organisational table from Reinstein and Riener, 2012b

4.12 What if you have trouble reading and understanding a paper?

Consult a survey paper, textbook, or lecture notes that discuss this paper and this topic

Try to find an easier related paper

Ask your supervisor for help; if he or she can

Try to understand what you can; do not try to “fake it”

4.13 Some literature survey do’s and don’ts

Do not cite irrelevant literature.

Do not merely list all the papers you could find.

Discuss them, and their relevance to your paper.

What are their strengths and weaknesses? What techniques do they use, and what assumptions do they rely on? How do they relate to each other?

Use correct citation formats.

Try to find original sources (don’t just cite a web link).

Don’t just cut and paste from other sources. And make sure to attribute every source and every quote. Be clear: which part of your paper is your own work and what is cited from others? The penalties for plagiarism can be severe!

• Critically examine the sources, arguments, and methods

4.14 Comprehension questions: literature review

How to discuss empirical results: “Causal” estimation, e.g., with Instrumental Variables

Which is the best way to state it? 12

“As I prove in table 2, more lawyers lead to slower growth (as demonstrated by the regression analysis evidence).”

“Table 2 provides evidence that a high share of lawyers in a city’s population leads to slower growth.”

3.“Table 2 shows that a high share of lawyers in a city’s population is correlated with slower growth.”

Which is better? 13

• “However, when a set of observable determinants of city growth (such as Census Region growth) are accounted for, the estimate of this effect becomes less precise.”
• “In the correct regression I control for all determinants of city growth and find that there is no effect of lawyers on growth”

Stating empirical results: descriptive

“Using the US data from 1850-1950, I find that inflation is lower during periods of recession. This is statistically significant in a t-test [or whatever test] at the 99% level, and the difference is economically meaningful. This is consistent with the theory of …, which predicts that lower inflation increases unemployment. However, other explanations are possible, including reverse causality, and unmeasured covarying lags and trends.”

“I find a significantly lower level of inflation during periods of recession, and the difference is economically meaningful. This relationship is statistically significant and the data is accurately measured. Thus I find that inflation increases unemployment.”

Some tips on writing a good paper– relevant to literature reviews

• Answer the question
• Provide clear structure and signposting
• Demonstrate an ability for critical analysis
• Refer to your sources
• Produce a coherent, clear argument
• Take time to proofread for style and expresssion
• Source “Assignment Writing Skills EBS 3rd year 2012”"

Answer: only b is a ‘peer reviewed article in a reputable economics journal’. All of these might be useful to cite, however. ↩

False. You need to cite any content and ideas that are not your own. ↩

Answers: 1, 5, and 6. Note that 2 and 3 are too narrow criteria, and 4 is too broad. ↩

or Bayesian if you like ↩

The second one; if this is really causal evidence. ↩

The first one. There is no ‘correct regression’. It is also not really correct in classical statistics to ‘find no effect’. ↩

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

• PMC8474097.1 ; 2020 Dec 1
• PMC8474097.2 ; 2021 Aug 4
• ➤ PMC8474097.3; 2021 Sep 2

Using citation tracking for systematic literature searching - study protocol for a scoping review of methodological studies and a Delphi study

Julian hirt.

1 Department of Clinical Research, University Hospital Basel, Basel, Switzerland

2 International Graduate Academy, Institute of Health and Nursing Science, Medical Faculty, Martin Luther University Halle-Wittenberg, Halle (Saale), Germany

3 Institute of Applied Nursing Science, Department of Health, Eastern Switzerland University of Applied Sciences (formerly FHS St.Gallen), St.Gallen, Switzerland

Thomas Nordhausen

Christian appenzeller-herzog.

4 University Medical Library, University of Basel, Basel, Switzerland

Hannah Ewald

Associated data, underlying data.

No underlying data are associated with this article.

Reporting guidelines

Open Science Framework (OSF): PRISMA-P checklist for ‘Using citation tracking for systematic literature searching - study protocol for a scoping review of methodological studies and a Delphi study’, https://doi.org/10.17605/OSF.IO/7ETYD 23 .

Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Version Changes

Revised. amendments from version 2.

We added methodological details of the Delphi study and we now specify the number of rounds we expect to perform, the consensus rate, anonymity of participants, and expected non-response.

Peer Review Summary

Background: Up-to-date guidance on comprehensive study identification for systematic reviews is crucial. According to current recommendations, systematic searching should combine electronic database searching with supplementary search methods. One such supplementary search method is citation tracking. It aims at collecting directly and/or indirectly cited and citing references from "seed references". Tailored and evidence-guided recommendations concerning the use of citation tracking are strongly needed.

Objective: We intend to develop recommendations for the use of citation tracking in systematic literature searching for health-related topics. Our study will be guided by the following research questions: What is the benefit of citation tracking for systematic literature searching for health-related topics? Which methods, citation indexes, and other tools are used for citation tracking? What terminology is used for citation tracking methods?

Methods: Our study will have two parts: a scoping review and a Delphi study. The scoping review aims at identifying methodological studies on the benefit and use of citation tracking in systematic literature searching for health-related topics with no restrictions on study design, language, and publication date. We will perform database searching in MEDLINE (Ovid), CINAHL (EBSCOhost), Web of Science Core Collection, two information science databases, web searching, and contact experts in the field. Two reviewers will independently perform study selection. We will conduct direct backward and forward citation tracking on included articles. Data from included studies will be extracted using a prespecified extraction sheet and presented in both tabular and narrative form. The results of the scoping review will inform the subsequent Delphi study through which we aim to derive consensus recommendations for the future practice and research of citation tracking.

Introduction

Systematic reviews are considered to be of high clinical and methodological importance as they help to derive recommendations for health care practice and future research 1 – 3 . A comprehensive literature search that aims to identify the available evidence as completely as possible is the foundation of every systematic review 4 – 6 . Due to an ever-growing research volume, lack of universal terminology and indexation, as well as extensive time requirements for identifying studies in a systematic way, efficient search approaches are required 5 , 7 , 8 . According to current recommendations, systematic search approaches should include both electronic database searching and one or several supplementary search methods 9 . Potential supplementary search methods include citation tracking, contacting study authors or experts, handsearching, trial register searching, and web searching 10 . In this study, we focus on citation tracking.

Citation tracking is an umbrella term for multiple methods which directly or indirectly collect related references from so called "seed references". These seed references are usually eligible for inclusion into the review. Some may be known at the beginning of the review and others may emerge as eligible records following full-text screening 10 – 12 . The terminology used to describe the principles of citation tracking is non-uniform and heterogeneous 13 – 16 . Citation tracking methods are sub-categorized into direct and indirect citation tracking ( Figure 1a ). For direct citation tracking, the words "backward" and "forward" denote the directionality of tracking 13 , 17 , 18 . Backward citation tracking is the oldest form of citation tracking. It aims at identifying references cited by a seed reference - which can easily be achieved by checking the reference list. Terms like "footnote chasing" or "reference list searching" are synonyms 6 , 13 . In contrast, forward citation tracking or chaining aims at identifying citing references, i.e. references that cite a seed reference 19 . Indirect citation tracking describes the identification of (i) co-cited references or co-citations (i.e. other references cited by citing literature of a seed reference) and of (ii) co-citing references (i.e. publications sharing references with a seed reference) 11 , 20 . Direct and indirect citation relationships of references based on a seed reference are illustrated in Figure 1b . Both direct and indirect citation tracking may contain one or more layers of iteration. To this end, researchers may use newly retrieved, relevant references as new seed references.

1a : Hierarchical illustration of different citation tracking methods; 1b : Direct and indirect citation relationships of references based on a seed reference. A → B denotes A cites B. The horizontal axis denotes time, i.e. the chronology in which references were published relative to the seed reference: “Older” stands for references that were published before the seed reference, “Newer” stands for references that were published after the seed reference.

Direct backward citation tracking of cited references is currently the most common citation tracking method. However, recent guidance suggests that a combination of several methods (e.g., tracking cited, citing, co-cited and co-citing references) may be the most effective way to use citation tracking for systematic reviewing 10 . It is quite likely that the added value of any form of citation tracking is not the same for all systematic reviews. It rather depends on a variety of factors. For instance, citation tracking may be beneficial in research areas that require complex searches such as reviews of complex interventions, mixed-methods reviews, qualitative evidence syntheses, or reviews on public health topics. Furthermore, research areas without consistent terminology or with vocabulary overlaps with other fields, such as methodological topics, may also benefit from the use of citation tracking 20 , 21 . Hence, tailored and evidence-guided recommendations on the use of citation tracking are strongly needed. However, none of the current reviews on this topic has systematically identified available evidence on the use and benefit of citation tracking in the context of systematic literature searching 10 .

Therefore, the aim of our study is to develop recommendations for the use of citation tracking in systematic literature searching for health-related topics. The scoping review will be guided by the following three research questions which in turn will inform the Delphi study:

• What is the benefit of citation tracking for systematic literature searching for health-related topics?
• Which methods, citation indexes, and other tools are used for citation tracking?
• What terminology is used for citation tracking methods?

This protocol is reported according to the “Preferred Reporting Items for Systematic review and Meta-Analysis Protocols” (PRISMA-P) checklist 22 which we published on the Open Science Framework 23 . Our study will have two parts: a scoping review and a Delphi study. The scoping review has the objective to map the benefit and the use of citation tracking, or research gaps if the results are not sufficiently informative. The objective of the subsequent Delphi study is to derive consensus recommendations for future practice and research of citation tracking 24 – 26 . For the scoping review, we will use the framework by Arksey and O’Malley 26 and the “Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews” (PRISMA-ScR) 27 . For the Delphi study, we will follow the “Guidance on Conducting and REporting DElphi Studies” (CREDES) statement 28 .

Scoping review

Eligibility criteria. We will include any study with a focus on citation tracking as a means of evidence retrieval which exhibits one of the following criteria: benefit and/or effectiveness of (i) citation tracking in general; (ii) different methods of citation tracking (e.g., backward vs. forward, direct vs. indirect); or (iii) technical uses of citation tracking (e.g., comparing citation indexes and/or tools, e.g., Scopus vs. Web of Science, Oyster, Voyster). Eligible studies need to have a health-related context. There will be no restrictions on study design, language, and publication date.

We will exclude studies solely using citation tracking for evidence retrieval, e.g., a systematic review applying citation tracking as a supplementary search technique, or studies focussing on citation tracking as a means to explore network or citation impact (i.e. bibliometric analysis). Studies only assessing the benefit of combined search methods in which the isolated benefit of citation tracking cannot be extracted will also be excluded. Furthermore, we will exclude methodological guidelines without empirical investigations and other non-empirical publications like editorials, commentaries, letters and abstract-only publications. Table 1 illustrates our inclusion and exclusion criteria per domain.

Information sources. We will search MEDLINE via Ovid; CINAHL (Cumulative Index to Nursing and Allied Health Literature), LLISFT (Library Literature & Information Science Full Text) and LISTA (Library, Information Science & Technology Abstracts) via EBSCOhost, and the Web of Science Core Collection by using database-specific search strategies. Additionally, we will perform web searching via Google Scholar as well as direct forward and backward citation tracking of included studies. As some evidence suggests that one citation index may not be enough for this 29 , we will use Scopus, Web of Science, and Google Scholar for forward citation tracking. For backward citation tracking, we will use Scopus and, if seed references are not indexed in Scopus, we will manually extract the seed reference's reference list. We will iteratively repeat direct citation tracking on newly identified eligible references until no further eligible references will be identified. We will also contact librarians in the field of health sciences and information specialists through several mailing lists (Canadian Medical Libraries, Expertsearching, MEDIBIB-L/German-speaking medical librarians, and EAHIL-list) to ask for further studies.

Search strategy. Due to a lack of adequate index terms, our search strategy will be based on text words only. To determine frequently occurring terms for inclusion into the search strategy, we analysed keywords in the titles and abstracts of potentially relevant publications retrieved from preliminary searches and similar articles identified via PubMed by using various text mining tools ( PubMed Reminer , AntConc , Yale MeSH analyzer , Voyant , VOSviewer , Termine , Text analyzer ) 30 . We restricted some of our text words to the title field in order to avoid retrieving systematic reviews that used citation tracking.

All authors contributed to the development of search strategies. HE and CAH are information specialists with a professional background in research; JH and TN are researchers experienced in the development of search strategies. HE drafted the search strategy and JH peer-checked it.

Box 1 shows the final search for MEDLINE in Ovid syntax. To use the search in other databases, we will translate it by means of Polyglot Search Translator 31 . CAH will conduct the searches and eliminate duplicates using the Bramer method 32 . We will perform web searching in Google Scholar using search terms from our database search. We will document our search strategy according to PRISMA-S 33 .

Search strategy for MEDLINE via Ovid

(reference list OR reference lists OR ((reference OR references OR citation OR citations OR co-citation OR co-citations) ADJ3 (search OR searches OR searching OR searched OR screen OR screening OR chain OR chains OR chaining OR check OR checking OR checked OR chased OR chasing OR tracking OR tracked OR harvesting OR tool OR tools OR backward OR forward)) OR ((cited OR citing OR cocited OR cociting OR co-cited OR co-citing) ADJ3 (references OR reference)) OR citation discovery tool OR cocitation OR co-citation OR cocitations OR co-citations OR co-cited OR backward chaining OR forward chaining OR snowball sampling OR snowballing OR footnote chasing OR berry picking OR cross references OR cross referencing OR cross-references OR cross-referencing OR citation activity OR citation activities OR citation analysis OR citation analyses OR citation network OR citation networks OR citation relationship OR citation relationships).ti OR (((((strategy OR strategies OR method* OR literature OR evidence OR additional OR complementary OR supplementary) ADJ3 (find OR finding OR search* OR retriev*)) OR (database ADJ2 combin*)).ti) AND ((search OR searches OR searching OR searched).ab))

Data management. A bibliography management tool will be used to manage the number of reference retrievals throughout the study selection process. Furthermore, we will use specific tools for study selection that we describe below.

Selection of sources of evidence. After an initial calibration phase, that is screening 100 titles and abstracts separately and discussing divergent decisions (TN, JH, HE), two authors (JH, TN) will independently screen titles, abstracts, and full texts using Rayyan 34 . They will solve disagreements by third author arbitration (HE). To screen the results of the citation tracking step, we will consider ASReview , particularly if the number of references exceeds 1000. ASReview combines machine (deep) learning models on a set of eligible studies with active learning on manual selections during title-abstract screening to generate a relevancy-ranked abstract list and to save screening time. Should the tool prove to be beneficial for reducing the screening load, we will consider conducting a more sensitive database search at a later stage and screen additional results with ASReview.

Data charting process. We will pilot a prespecified data extraction sheet approved by consensus among the authors. We will extract bibliographic and geographic data, design- and study-specific data as well as results that answer our research questions. Since we expect heterogeneous studies in terms of aim, design, and methods, we aim for an iterative data extraction process. This will allow a flexible and study-specific data extraction process, e.g., by adding previously neglected data extraction items that might contribute to the overall body of knowledge to the data extraction form. In the final publication, we will provide a detailed overview of extracted data items. One author will extract data and a second author will peer-check the extraction. We will solve disagreements by third author arbitration.

Synthesis of results. One author (JH) will narratively summarise study characteristics and results. Depending on the results, we will also chart them graphically.

Delphi study

Design and rationale. A consensus multi-stage online Delphi procedure will be used to derive recommendations for the use of citation tracking in systematic literature searching for health-related topics 28 , 35 , 36 . A Delphi procedure will be chosen since the method enables to collect the perspectives of international experts on citation tracking, promote discussions on the topic as well as derive consensus recommendations for future practice and research. The Delphi study will entail several Delphi rounds (see below). The results of the scoping review will inform the initial Delphi round (see below for details). To distribute the Delphi rounds to the experts, we will use the web-based tool SosciSurvey 37 . The Delphi language will be English.

Expert panel. The recruitment of experts will be based on a stepwise approach. First, we will contact authors of pertinent articles identified during the literature search as well as experts from our professional networks. This "person-based" approach will help us to identify experts who authored papers, books, comments, and reviews in the field of citation tracking. We will ask the contacted persons to take part in the Delphi study. Second, we will identify and contact relevant national and international organisations as well as systematic review collaborations (e.g., Cochrane groups, Joanna Briggs Institute (JBI), Campbell Collaboration, National Academy of Medicine (NAM), expert information specialists, Evidence Synthesis International, and PRISMA-S working group). This "organisation-based approach" will allow us to reach experts in the field of literature retrieval methods who are potentially using citation tracking without necessarily being the authors of methodological studies (yet). By using this stepwise approach, we intend to recruit at least 15 experts.

Data collection. In online Delphi rounds, we will seek guidance on various aspects of citation tracking. For example, recommendations on the following aspects could be of particular interest:

• Uniform terminology for citation tracking methods
• Situations in which citation tracking should be applied
• Potential situations in which citation tracking can be used as a sole method of evidence retrieval
• Situations in which a particular citation tracking method or a combination thereof is likely to be most effective
• Situations in which further layers of iteration of citation tracking should be applied
• Necessity to use multiple citation indexes for citation tracking
• For indirect citation tracking, screening of selected records only and definition of their ranking and cut-off
• Reporting of citation tracking (complementing PRISMA-S 33 )
• Questions on citation tracking that currently cannot be answered and require more research

Based on the results of our scoping review, we will formulate draft recommendations for the first Delphi round. Experts will be invited to rate their agreement with the draft recommendations using a four-point Likert scale (strongly agree – agree – disagree – strongly disagree). If experts vote disagree/strongly disagree, they will be required to comment on their reasons and/or give constructive feedback. We consider a recommendation as consented when at least 75% of the experts agree/strongly agree. All other recommendations will be adapted for the next Delphi round. This adaptation will be based on the comments collected from the experts and, if necessary, on discussion via video conference.

There are items where we will not directly propose recommendations, e.g., if the results of our scoping review do not allow it or if there are several equally valid options (e.g., for terminology). In these cases, we will either ask the Delphi experts for their experiences and perspectives or let them vote on several options. We will use the resulting answers to formulate draft recommendations, which will be entered into the Delphi consensus process (see above). Therefore, our Delphi study may comprise qualitative and quantitative aspects.

We will limit the number of Delphi rounds to a maximum of four rounds. Should there be no consensus for any of the items by the end of the fourth round, we will report the results but not give any recommendations.

Expert assessments will be anonymous among experts but open to the study team. We expect a low non-response rate since experts' participation is indicative of their interest in our study.

To describe experts’ characteristics, we will collect sociodemographic data, i.e. professional education and background, current field of work as well as years of experience in literature searching and citation tracking. We expect that experts will invest around 30 to 90 minutes per Delphi round depending on the underpinning aim of the Delphi round as well as experts’ familiarity and experiences with the topic. For each Delphi round, we will schedule approximately three weeks for participation. Table 2 illustrates our reminder strategy within a Delphi round. We will pilot test and discuss our Delphi items with a person experienced in literature searching who is not an author and not involved in the Delphi study.

Note: Person-based approach: contacting authors of pertinent articles identified during the literature search as well as experts from authors’ professional networks; Organisation-based approach: contacting national and international organisations and systematic review collaborations.

Data analysis. We will use descriptive statistics for votes for which results are numeric or can be converted into numbers. For free text answers and statements of experts, we will use thematic categorisation 38 .

Ethical concerns. The online Delphi study will contain introductory information on our aims, the Delphi itself, data management and security. We do not expect vulnerability on the part of experts and with regard to the Swiss Human Research Act, our research does not concern human diseases and the structure and function of the human body 39 . We will therefore not apply for ethical approval of the Delphi study. Taking part in the Delphi study will indicate consent to participate. There will be no mandatory participation once an expert consented to participate. Experts will not receive an incentive for participation and may leave the process at any time.

Dissemination of results

Our dissemination strategy uses multiple ways to share our study results with academic stakeholders. The final scoping review and Delphi study will each be published in an international open access journal relevant in the field of information retrieval. Additionally, we will discuss our results with experts at national and international conferences (e.g., conference of the German Network for Evidence-based Medicine (EbM-Netzwerk), conference of the European Association for Health Information and Libraries (EAHIL), Cochrane Colloquium, Health Technology Assessment International (HTAi) conference). To inform about our study results and publications, we will use Twitter, ResearchGate, and mailing lists from relevant stakeholders such as Canadian Medical Libraries, Expertsearching, MEDIBIB-L/German-speaking medical librarians, and EAHIL-list.

Study status

We conducted the initial search for the scoping review in November 2020 and expect to complete the Delphi study in 2022.

Current study status: literature searches: yes; piloting of the study selection process: yes; formal screening of search results against eligibility criteria: yes; data extraction: no; data analysis: no.

Conclusions

Missing pertinent evidence might have an impact on the validity of systematic reviews and, consequently, on the quality of health care 40 , 41 . Therefore, authors of systematic reviews should conduct high quality literature searches aiming to detect all relevant evidence. Citation tracking may be an effective way to complement electronic database searches and to broaden the scope of possible findings. Therefore, our study intends to provide literature- and expert-based recommendations on the use of citation tracking for systematic literature searching. Although we solely focus on a health-related context, it is possible that some of the recommendations developed during this project may prove relevant also for other academic fields such as social or environmental sciences 9 , 42 . Finally, tailored and evidence-based recommendations concerning the use of citation tracking for systematic literature searching may guide future steps in semi-automated and automated literature retrieval methods 43 , 44 .

Data availability

[version 3; peer review: 2 approved]

Funding Statement

The author(s) declared that no grants were involved in supporting this work.

Reviewer response for version 3

David moher.

1 School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada

2 Centre for Journalology, Ottawa Hospital Research Institute, Ottawa, Canada

The authors have addressed the concerns I raised in my second review.

Is the study design appropriate for the research question?

Is the rationale for, and objectives of, the study clearly described?

Are sufficient details of the methods provided to allow replication by others?

Are the datasets clearly presented in a useable and accessible format?

Reviewer Expertise:

Systematic reviews; open science; reporting guidelines.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Reviewer response for version 2

Julie glanville.

1 Cochrane Information Retrieval Methods Group, London, UK

I have read the revised version of the study protocol and I am happy with the changes that have been made.

Not applicable

Information retrieval for evidence identification for systematic reviews.

I think the revisions, particularly around the scoping review, are much improved.

In terms of the Delphi study, I think details are still missing. For example, it is unclear how many rounds will be given. After round 1, what consensus rate (e.g., 75%) will be used to drop an item from subsequent rounds? Overall, I think more details are required for the proposed Delphi methods. My recommendation is to read these papers:

• Examples of Delphi studies: Vogel  et al ., 2019 1 , Santaguida  et al ., 2018 2 ;
• General methodological issues about Delphi: Okoli  et al ., 2004 3 , Diamond  et al ., 2014 4

One final minor detail. In the study status section of the paper, the authors state they started in November 2020. Is this correct?

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

University Hospital Basel, Switzerland

Dear David Moher

Thank you very much for your helpful comments and references. We revised our manuscript with respect to methodological details of the Delphi study and now specified the number of rounds we expect to perform, the consensus rate, anonymity of participants, and expected non-response.

Reviewer comment: I think the revisions, particularly around the scoping review, are much improved.

Examples of Delphi studies: Vogel et al., 2019 1, Santaguida et al., 2018 2;

General methodological issues about Delphi: Okoli et al., 2004 3, Diamond et al., 2014 4

Authors’ response: We added “Based on the results of our scoping review, we will formulate draft recommendations for the first Delphi round. Experts will be invited to rate their agreement with the draft recommendations using a four-point Likert scale (strongly agree – agree – disagree – strongly disagree). If experts vote disagree/strongly disagree, they will be required to comment on their reasons and/or give constructive feedback. We consider a recommendation as consented when at least 75% of the experts agree/strongly agree. All other recommendations will be adapted for the next Delphi round. This adaptation will be based on the comments collected from the experts and, if necessary, on discussion via video conference.

Expert assessments will be anonymous among experts but open to the study team. We expect a low non-response rate since experts' participation is indicative of their interest in our study.”

Authors’ response: Yes, we started to work on this scoping review in November by doing the initial searches. In the revised version, we reworded to be clearer on that issue: “We conducted the initial search for the scoping review in November 2020 and expect to complete the Delphi study in 2022.”

Reviewer response for version 1

The authors are proposing a new guidance on citation tracking. They will achieve this by completing a scoping review and subsequently a survey. The authors propose using appropriate methods (and reporting) for conducting the scoping review. I found the methods for the survey less clear and hope my comments/questions, below, are helpful in revised the protocol.

Questions/comments:

• A more conceptual question is to what degree this initiative overlaps and differs from PRISMA-S (Rethlefsen ML, Kirtley S, Waffenschmidt S, Ayala AP, Moher D, Page MJ, Koffel JB; PRISMA-S Group. PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Syst Rev. 2021 Jan 26;10(1):39. doi: 10.1186/s13643-020-01542-z) 1 .
• For example, item 5 of PRIMSA-S – Citation searching - states “Citation searching 5 Indicate whether cited references or citing references were examined, and describe any methods used for locating cited/citing references (e.g., browsing reference lists, using a citation index, setting up email alerts for references citing included studies).” I think this is important so as not to potentially confuse readers. Indeed, should the current initiative be seen as an implementation of PRISMA-S?
• I had more difficulty understanding the proposed survey. For me, much of the methods were missing. I understand the recruitment (well reported). I would recommend the authors include the PRISMA-S group unless there is a lot of overlap between PRISMA-S and the other groups mentioned.
• It was not clear to me how long the authors will make completing the survey– 15 minutes of 45 minutes? Similarly, will the respondents receive an incentive for completing the survey?
• Why ‘simply’ a survey rather than a Delphi (or modified) approach? Will the survey (or Delphi) be pilot tested for question and response option clarity and language?
• The data analysis describes cross tables. I assume the authors mean cross tabulations which can be more than descriptive. Will the authors be conducting Chi-square analysis or other analytical approaches of the cross tabulations (e.g., p values)?
• The ethical concerns section is likely jurisdiction specific. In my setting ethics would be required. Can the authors explicitly indicate whether ethics is required or not. The section is currently vague on this critical issue. 
• For the survey, do the authors have an estimated sample size they are aiming for. Similarly, do the authors have an estimated response rate they are aiming for?

We are grateful for your reviewer report and, in particular, for your helpful suggestions to derive evidence-based recommendations from our study. Below, we provide a detailed point-by-point response with changes that we implemented in version 2 of our manuscript.

You will see that in response to your comments, we replaced the planned survey with a planned Delphi study to address the original objective. This change also entails the reformulation of our research questions and the intent to publish the final scoping review and the results of the Delphi study separately.

We hope that these changes fully meet your concerns.

Sincerely yours,

Julian Hirt, Thomas Nordhausen, Christian Appenzeller-Herzog, and Hannah Ewald

Reviewer comment: A more conceptual question is to what degree this initiative overlaps and differs from PRISMA-S (Rethlefsen ML, Kirtley S, Waffenschmidt S, Ayala AP, Moher D, Page MJ, Koffel JB; PRISMA-S Group. PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Syst Rev. 2021 Jan 26;10(1):39. doi: 10.1186/s13643-020-01542-z).

Authors’ response: PRISMA-S is a reporting guideline for systematic literature searches. However, it is lacking on precise recommendations how to report citation tracking methods and results. By conducting a Delphi study with selected experts in the field (see our answer below for details), we aim at developing more detailed recommendations on adequately reporting citation tracking methods as part of systematic literature searching. This may complement recommendations on reporting given by the PRISMA-S group or could even be integrated in future versions. For the Delphi, we will invite the PRISMA-S group to best use synergies and experiences.

Reviewer comment: For example, item 5 of PRIMSA-S – Citation searching - states “Citation searching 5 Indicate whether cited references or citing references were examined, and describe any methods used for locating cited/citing references (e.g., browsing reference lists, using a citation index, setting up email alerts for references citing included studies).” I think this is important so as not to potentially confuse readers. Indeed, should the current initiative be seen as an implementation of PRISMA-S?

Authors’ response: See answer above. An important detail for the reporting of our study: since we will use citation tracking as part of our information retrieval, we will use PRISMA-S to guide our reporting of the scoping review. In the revised version, we reference it accordingly: “We will document our search strategy according to PRISMA-S”.

Reviewer comment: I had more difficulty understanding the proposed survey. For me, much of the methods were missing. I understand the recruitment (well reported). I would recommend the authors include the PRISMA-S group unless there is a lot of overlap between PRISMA-S and the other groups mentioned.

Authors’ response: As detailed further below, the survey will be replaced by a Delphi study. So, we replaced the survey methods with more detailed methods for the Delphi study. Concerning PRISMA-S, we agree with the reviewer and will ask the PRISMA-S working group to participate in our Delphi study (see below).

Reviewer comment: It was not clear to me how long the authors will make completing the survey– 15 minutes of 45 minutes? Similarly, will the respondents receive an incentive for completing the survey?

Authors’ response: Following your recommendation below, we now plan to conduct an expert Delphi study containing multiple Delphi rounds. We expect that experts will invest around 30 to 90 minutes per Delphi round depending on the underpinning aim of the Delphi round as well as experts’ familiarity and experiences with the topic. Participants will not receive an incentive for participation.

Reviewer comment: Why ‘simply’ a survey rather than a Delphi (or modified) approach? Will the survey (or Delphi) be pilot tested for question and response option clarity and language?

Authors’ response: This is an important point and we thank the reviewer for raising it. We agree that a Delphi study containing several Delphi rounds is suitable to collect the perspectives of international experts on citation tracking, promote discussions on the topic as well as to derive consensus recommendations for future practice and research on the use of citation tracking in systematic literature searching for health-related topics. We subjected our protocol to a major revision based on this point and now outline the methods for the Delphi. As stated in the revised text, we will pilot test and discuss our Delphi items with a person experienced in literature searching but who is not an author and not involved in the Delphi study.

Reviewer comment: The data analysis describes cross tables. I assume the authors mean cross tabulations which can be more than descriptive. Will the authors be conducting Chi-square analysis or other analytical approaches of the cross tabulations (e.g., p values)?

Authors’ response: We revised the corresponding section to address a suitable type of analysis for our data retrieved in Delphi rounds. For free text answers and statements of experts, we will use thematic categorisation. For votes whose results are available or can be converted into numbers, we will use descriptive statistics.

Reviewer comment: The ethical concerns section is likely jurisdiction specific. In my setting ethics would be required. Can the authors explicitly indicate whether ethics is required or not. The section is currently vague on this critical issue.

Authors’ response: We added details to the ethical concerns section. With regard to the Swiss Human Research Act, our research does not concern human diseases and the structure and function of the human body. We will therefore not apply for ethical approval of the expert Delphi study.

Reviewer comment: For the survey, do the authors have an estimated sample size they are aiming for. Similarly, do the authors have an estimated response rate they are aiming for?

Authors’ response: This answer concerns the recruitment of experts for the Delphi study, not the survey. We intend to recruit at least 15 participants by using our stepwise approach for recruitment, the person-based approach (contacting authors of pertinent articles identified during the literature search as well as experts from authors’ professional networks) and the organisation-based approach (contacting national and international organisations and systematic review collaborations).

The authors propose to conduct a scoping review of the use of citation tracking techniques to identify research evidence to inform systematic reviews and also to survey experts on their use of citation tracking.

My suggestions for action on the proposal are as follows:

'These references are usually eligible for inclusion into the review and some may be known at the beginning of the review and others may emerge as eligible records following the main database searches 10 – 12 .'

Then perhaps you may not need the sentence that follows the one I am commenting on.

• Introduction: 'The taxonomy used to describe the…'. I think it might be better to use a word such as  'terminology' rather than 'taxonomy'. 

Belter CW. A relevance ranking method for citation-based search results. Scientometrics. 2017;112(2):731-46. 1

Janssens A, Gwinn M, Brockman JE, Powell K, Goodman M. Novel citation-based search method for scientific literature: a validation study. BMC Med Res Methodol. 2020;20(1):25. 2

• Introduction: 'However, recent guidance suggests that combining several citation tracking methods (e.g. screening cited, citing, co-cited and co-citing references) may be the most effective way to use citation tracking for systematic reviewing. '  Please cite the guidance. 
• Introduction: ' It rather depends on a variety of factors. For instance, citation tracking may be especially beneficial in case of (i) complex searches (e.g. for reviews on public health topics), (ii) searching for health outcome measurement instruments, or (iii) research areas without consistent taxonomy, with vocabulary overlaps with other fields, or with a lack of index terms in databases (e.g. methodological topics)'.  These are all important issues and each deserves a bit more description so that readers can understand the differences. Again I think the word 'taxonomy' should be replaced with 'terminology'. 
• Introduction: current 'topical reviews' - suggest replacing with 'recent reviews on this topic'. Then please add in more citations so that 'reviews' can be supported with more than one reference.  
• Introduction: 'health-related systematic literature searching'.  This occurs three times and for each occurrence I suggest rewording as follows for clarity: 'literature searching for systematic reviews of health and health-related topics'. 
• Eligibility criteria. 'evidence retrieval method'.  Suggest that 'method' is not needed as the sentence has 'means'. 
• 'Eligible studies need to have a health-related context. Studies without an explicitly specified research context are also eligible. '  This seems contradictory.  Why are studies in education topics for example not useful if they are looking at the methods rather than the topic? Just including papers where the topic is not clear seems arbitrary when there may be much to learn from a paper even if it has an engineering topic, say.  Perhaps reword along the lines of 'although studies undertaken in the context of health-related topics are the main focus, studies of citation tracking in other literatures will also be eligible where the focus is on exploring the methods rather than the subject topic'. 
• Table 1. Publication Type cannot be 'Any' when there are publication types listed as ineligible. The authors should list eligible study types or use some sort of exception wording.
• Information sources. I am not sure 'free web searching via Google Scholar' is a helpful description, perhaps just say 'searches of Google Scholar' here and further down where it is mentioned again. 
• 'For citation tracking, we will use Scopus, as this database seems to cover the largest number of relevant citations for the purpose of our review 30 . '  There is quite a lot of research that reports that other resources such as Google Scholar or Microsoft Academic provide wider coverage - the authors may wish to consider searching some of the free resources as well (since Scopus alone is not going to find the largest number of results) or finding other reasons for the use of a Scopus only approach. 
• 'MEDBIB-L' - I think there may be a typo here and it is MEDLIB-L? I think it may also need to be corrected in the Dissemination section. 
• My understanding of the purpose of a protocol is that it should list what will happen and therefore the resources to be searched should be listed and there should be no examples which imply that other things may be added. So there should be no' e.g.'. 
• 'text mining approach' - the chapter describes many approaches - which one was used in this case? 
• 'parts of our textwords' - suggest 'some of our textwords'? 
• Box 1 - there is harmless redundancy in the search. For example co-citation is searched alone and in combination with other terms - it only needs to be searched alone.  
• Box 1 - with the focus on searching in the titles, the use of such close adjacency does not seem warranted - it would be more sensitive to use AND for title searching.
• Box 1 - it might be helpful in the absence of subject headings and the focus on searching mainly in titles to also search the author keywords field.
• Data charting process: I realise it is difficult to know what will happen but the authors description is not very clear about their plans 'We aim for an iterative data extraction process, but in the final publication, we will provide a detailed overview of extracted data items.'  What is an 'iterative data extraction process' - what might it look like and what does it involve?  
• Data collection 'preferred taxonomy' - again, I suggest the word you mean is 'terminology' not 'taxonomy'. 
• Data analysis.  'To analyse the survey data, we will apply descriptive statistics based on frequencies, percentages, and cross tables.' I think it may be 'cross tabulations' rather than 'cross tables'? 
• Dissemination of results - it might be useful to also to try to present results at conferences of systematic review experts - Cochrane Colloquium and the Health Technology Assessment International (HTAi) conferences. These organisations also have active information retrieval special interest groups. 
• Conclusions: 'Depending on the available study landscape' - this seems rather vague to me - it would help the reader if the authors could be more explicit about what they mean. 
• 'consequently, on the quality of clinical care' - I think it might not only be clinical care that could be impacted - it might be any aspect of health care depending on what is being reviewed e.g. a service, a policy, a new method of organising staff etc.
• 'in a health-related context may prove relevant also for other academic fields such as social or environmental sciences' - I think that much of what you find may be generalisable to other disciplines, so I suggest that you state this earlier as a possibility.

Dear Julie Glanville

We are grateful for your reviewer report and, in particular, for your helpful suggestions for improvement of our scoping review protocol. Below, we provide a detailed point-by-point response with changes that we implemented in version 2 of our manuscript.

Please note that in response to the comments by reviewer 2 (David Moher), we replaced the planned survey with a planned Delphi study to address the original objective. This change also entails the reformulation of our research questions and the intent to publish the final scoping review and the results of the Delphi study separately.

We hope our revisions now fully meet your concerns.

Reviewer comment: Introduction: 'These references are usually eligible for inclusion into the review and known at the start of the citation search10–12.' I think this could be reworded for clarity along the following lines:

'These references are usually eligible for inclusion into the review and some may be known at the beginning of the review and others may emerge as eligible records following the main database searches10–12.'

Authors’ response: We adapted this sentence and the subsequent one according to your suggestion.

Reviewer comment: Introduction: 'The taxonomy used to describe the…'. I think it might be better to use a word such as 'terminology' rather than 'taxonomy'.

Authors’ response: Thank you. We now use terminology instead of taxonomy throughout the manuscript.

Reviewer comment: Figure 1. The figure would be more helpful if it could show the relationships of the indirect citation tracking in the Figure 1b perhaps by colour coding. Please check diagrams in the following references which incorporate chronology into the citation picture as well as showing the indirect citation tracking clearly.

Belter CW. A relevance ranking method for citation-based search results. Scientometrics. 2017;112(2):731-46.1

Janssens A, Gwinn M, Brockman JE, Powell K, Goodman M. Novel citation-based search method for scientific literature: a validation study. BMC Med Res Methodol. 2020;20(1):25.2

Authors’ response: We have colour coded the 4 citation tracking techniques as suggested. We had already consulted the citation tracking figures of Belter et al . and Janssens. et al. amongst others. Furthermore, we have added text to the figure caption. We feel that our figure is now clearer with respect to chronology.

Reviewer comment: Introduction: 'However, recent guidance suggests that combining several citation tracking methods (e.g. screening cited, citing, co-cited and co-citing references) may be the most effective way to use citation tracking for systematic reviewing.' Please cite the guidance.

Authors’ response: We added a reference.

Reviewer comment: Introduction: ' It rather depends on a variety of factors. For instance, citation tracking may be especially beneficial in case of (i) complex searches (e.g. for reviews on public health topics), (ii) searching for health outcome measurement instruments, or (iii) research areas without consistent taxonomy, with vocabulary overlaps with other fields, or with a lack of index terms in databases (e.g. methodological topics)'. These are all important issues and each deserves a bit more description so that readers can understand the differences. Again I think the word 'taxonomy' should be replaced with 'terminology'.

Authors’ response: Thank you for this point! We have revised and elaborated this section.

Reviewer comment: Introduction: current 'topical reviews' - suggest replacing with 'recent reviews on this topic'. Then please add in more citations so that 'reviews' can be supported with more than one reference. 

Authors’ response: We adapted the sentence according to your suggestion, thank you.

Reviewer comment: Introduction: 'health-related systematic literature searching'. This occurs three times and for each occurrence I suggest rewording as follows for clarity: 'literature searching for systematic reviews of health and health-related topics'.

Authors’ response: Thank you for your proposition. We now use “systematic literature searching for health-related topics“.

Reviewer comment: Eligibility criteria. 'evidence retrieval method'. Suggest that 'method' is not needed as the sentence has 'means'.

Authors’ response: We deleted the word ‘method’.

Reviewer comment: 'Eligible studies need to have a health-related context. Studies without an explicitly specified research context are also eligible.' This seems contradictory. Why are studies in education topics for example not useful if they are looking at the methods rather than the topic? Just including papers where the topic is not clear seems arbitrary when there may be much to learn from a paper even if it has an engineering topic, say. Perhaps reword along the lines of 'although studies undertaken in the context of health-related topics are the main focus, studies of citation tracking in other literatures will also be eligible where the focus is on exploring the methods rather than the subject topic'.

Authors’ response: We agree that it is interesting to learn about citation tracking from other disciplines. However, we have decided to focus on studies in a health-related context because this enables us to (i) consider the context in which the studies were conducted and to (ii) specifically direct our conclusions and recommendations to the health context. Furthermore, we concentrate on health-related studies for practical reasons facing the high number of references that need screening. We deleted the sentence ‘Studies without an explicitly specified research context are also eligible’ to avoid misunderstandings.

On that note we also improved our exclusion criteria by adding this criterion: ‘any study only assessing the benefit of combined search methods (whereas the isolated benefit of citation tracking cannot be extracted)’.

Reviewer comment: Table 1. Publication Type cannot be 'Any' when there are publication types listed as ineligible. The authors should list eligible study types or use some sort of exception wording.

Authors’ response: We now use ‘Any reports of empirical studies’.

Reviewer comment: Information sources. I am not sure 'free web searching via Google Scholar' is a helpful description, perhaps just say 'searches of Google Scholar' here and further down where it is mentioned again.

Authors’ response: We now use ‘web searching’ instead of ‘free web searching’ throughout the manuscript.

Reviewer comment: 'For citation tracking, we will use Scopus, as this database seems to cover the largest number of relevant citations for the purpose of our review30.' There is quite a lot of research that reports that other resources such as Google Scholar or Microsoft Academic provide wider coverage - the authors may wish to consider searching some of the free resources as well (since Scopus alone is not going to find the largest number of results) or finding other reasons for the use of a Scopus only approach.

Authors’ response: We agree and now plan forward citation tracking using a triple approach: Scopus, Web of Science, and Google Scholar. We will iteratively repeat citation tracking on newly identified records to include until no further eligible references will be identified. We now describe this new approach in the methods.

Reviewer comment: 'MEDBIB-L' - I think there may be a typo here and it is MEDLIB-L? I think it may also need to be corrected in the Dissemination section.

Authors’ response: The correct name is MEDIBIB-L (the “I” was missing), a mailing list specific to libraries of medicine in the German-speaking area (library = Bibliothek; https://lists.uni-due.de/mailman/listinfo/medbib-l ).

Reviewer comment: My understanding of the purpose of a protocol is that it should list what will happen and therefore the resources to be searched should be listed and there should be no examples which imply that other things may be added. So there should be no' e.g.'.

Authors’ response: We deleted ‘e.g.’

Reviewer comment: 'text mining approach' - the chapter describes many approaches - which one was used in this case?

Authors’ response: We used several tools that were used to improve our search. We now list all the tools that we used during search drafting phase in our manuscript and here:

• PubMed PubReMiner: https://hgserver2.amc.nl/cgi-bin/miner/miner2.cgi
• AntConc 3.5.7 (Windows), developed by Laurence Anthony, Faculty of Science and Engineering, Waseda University, Japan
• Yale MeSH Analyzer: http://mesh.med.yale.edu/
• TerMine: http://www.nactem.ac.uk/software/termine/
• Text Analyzer: https://www.online-utility.org/text/analyzer.jsp
• Voyant: https://voyant-tools.org/
• VOSviewer: https://www.vosviewer.comrt

Reviewer comment: 'parts of our textwords' - suggest 'some of our textwords'?

Authors’ response: We agree, thank you.

Reviewer comment: Box 1 - there is harmless redundancy in the search. For example co-citation is searched alone and in combination with other terms - it only needs to be searched alone. 

Authors’ response: Yes, these are remnants of a previously more specific search and indeed now redundant. Since we already ran the searches by now, we would prefer to leave the search string exactly as we ran it for precise documentation. However, we noted this in our search files in case of a later update.

Reviewer comment: Box 1 - with the focus on searching in the titles, the use of such close adjacency does not seem warranted - it would be more sensitive to use AND for title searching.

Authors’ response: Again, this is very good input which we noted in our search files in case of a later update. Unfortunately, we already ran the searches by now and are done screening. We are, however, confident that any article that we missed due to the close adjacency will be retrieved through our extensive citation tracking, web searches or expert contacting.

Reviewer comment: Box 1 - it might be helpful in the absence of subject headings and the focus on searching mainly in titles to also search the author keywords field.

Authors’ response: Thank you for the sensible suggestion which we also noted in our search files in case of a later update.

Reviewer comment: Data charting process: I realise it is difficult to know what will happen but the authors description is not very clear about their plans 'We aim for an iterative data extraction process, but in the final publication, we will provide a detailed overview of extracted data items.' What is an 'iterative data extraction process' - what might it look like and what does it involve? 

Authors’ response: We revised this paragraph and provide more details: ‘Since we expect heterogeneous studies in terms of aim, design, and methods, we aim for an iterative data extraction process. This allows a flexible and study-specific data extraction process, e.g. by adding previously neglected data extraction items that might contribute to the overall body of knowledge to the data extraction form.’

Reviewer comment: Data collection 'preferred taxonomy' - again, I suggest the word you mean is 'terminology' not 'taxonomy'.

Authors’ response: Indeed, thank you.

Reviewer comment: Data analysis. 'To analyse the survey data, we will apply descriptive statistics based on frequencies, percentages, and cross tables.' I think it may be 'cross tabulations' rather than 'cross tables'?

Authors’ response: We revised this paragraph to address a suitable type of analysis for our data retrieved in Delphi rounds. For free text answers and statements of experts, we will use thematic categorisation. For votes whose results are available or can be converted into numbers, we will use descriptive statistics.

Reviewer comment: Dissemination of results - it might be useful to also to try to present results at conferences of systematic review experts - Cochrane Colloquium and the Health Technology Assessment International (HTAi) conferences. These organisations also have active information retrieval special interest groups.

Authors’ response: Thank you, we added these conferences to our list of potential conferences.

Reviewer comment: Conclusions: 'Depending on the available study landscape' - this seems rather vague to me - it would help the reader if the authors could be more explicit about what they mean.

Authors’ response: We deleted ‘Depending on the available study landscape’ to be more precise.

Reviewer comment: 'consequently, on the quality of clinical care' - I think it might not only be clinical care that could be impacted - it might be any aspect of health care depending on what is being reviewed e.g. a service, a policy, a new method of organising staff etc.

Authors’ response: We agree and revised the sentence to ‘health care’ instead of ‘clinical care’.

Reviewer comment: 'in a health-related context may prove relevant also for other academic fields such as social or environmental sciences' - I think that much of what you find may be generalisable to other disciplines, so I suggest that you state this earlier as a possibility.

Authors’ response: We agree that our results may indeed prove relevant to other disciplines. However, we decided to narrow our research efforts to the health-related fields, so we think this should be rather a part of the discussion/outlook than earlier in the manuscript. We changed the sentence to underline that aspect: “Although we solely focus on a health-related context, it is possible that some of the recommendations developed during this project may prove relevant also for other academic fields such as social or environmental sciences”.

Holman Library

• Library Instruction

ENGL 127 Research Writing: Social Sciences

What is scholarly literature.

• English 127
• Explore Research Topics
• Map out your Concept & Keywords
• Get an Overview with One Search
• Get Started with Reference
• Find Magazine and Newspaper Articles
• Find Multimedia: Documentary, TV, Radio, Podcasts
• Find Books/Book Chapters
• Find Professional/ Trade Articles
• Find Websites
• Find Legislation
• Find Statistics
• Find Editorial, Opinion & Argument
• Evaluating Sources
• Interlibrary Loan
• APA citations
• Find Help @ GRC
• Special Topic 20-21: the US Carceral System & Injustice This link opens in a new window
• Special Topic 21-22 One Book: So You Want to Talk about Race This link opens in a new window
• Special Topic 22-23: Environmental Justice This link opens in a new window
• What Makes Information Scholarly?
• What is Peer-Review and Why Does it Matter?
• Anatomy of a Scholarly Research Article in Social Sciences
• Compare: Scholarly vs Popular Article

Features of Scholarly Literature

• Written by scholars and subject experts
• Written for other scholars, professionals. and policy makers. Also read by student researchers
• It reflects a specific discipline or interdisciplinary field, such as sociology, history, gender studies, etc.
• Literature reviews, for example, give an overview of the body of research on a specific topic.
• Original research studies, on the other hand, present original research findings in a field.
• Theoretical articles offer original analysis.
• Research studies aim to be transparent. They outline the purpose of the study, the methodology, findings, and conclusions. One of the goals of a research study is that it be reproduceable so that its conclusions may be verified.
• Scholarship is usually, though not always, lengthy, and it engages with issues at a more substantial level than magazine articles.
• Scholarship in the social sciences will include an extensive list of References at the end of the work, and comprehensive in-text citations throughout for all claims made in the body of the article.
• Scholarship is published in scholarly journals and in books generally from university presses.
• The purpose of a body of scholarship on a subject is to advance our knowledge and understanding in that field.

What can you ask to determine if a source is scholarly?

When you are determining whether or not the article you found is a peer-reviewed article, you should consider the following.

(Click on image to enlarge) 

Also consider...

• Is the journal in which you found the article published or sponsored by a professional scholarly society, professional association, or university academic department? Does it describe itself as a peer-reviewed publication? (To know that, check the journal's website). 
• Did you find a citation for it in one of the  databases that includes scholarly publications? (Academic Search Complete, PsycINFO, etc.)?  Read the database description to see if it includes scholarly publications.
• In the database, did you limit your search to scholarly or peer-reviewed publications? (See video tutorial below for a demonstration.)
• Is the topic of the article narrowly focused and explored in depth?
• Is the article based on either original research or authorities in the field (as opposed to personal opinion)?
• Is the article written for readers with some prior knowledge of the subject?
• Introduction
• Theory or Background
• Literature review

What is the peer review process?

In academic publishing, the goal of peer review is to assess the quality of articles submitted for publication in a scholarly journal. Before an article is deemed appropriate to be published in a peer-reviewed journal, it must undergo the following process:

• The author of the article must submit it to the journal editor who forwards the article to experts in the field. Because the reviewers specialize in the same scholarly area as the author, they are considered the author’s peers (hence “peer review”).
• These impartial reviewers are charged with carefully evaluating the quality of the submitted manuscript.
• The peer reviewers check the manuscript for accuracy and assess the validity of the research methodology and procedures.
• If appropriate, they suggest revisions. If they find the article lacking in scholarly validity and rigor, they reject it.

Because a peer-reviewed journal will not publish articles that fail to meet the standards established for a given discipline, peer-reviewed articles that are accepted for publication exemplify the best research practices in a field.

Attribution: Much of the information about the  the peer-review process  was used with permission from the awesome librarians at the  Lloyd Sealy Library at the John Jay College of Criminal Justice .

Source: "Peer Review in 3 Minutes" by libncsu , is licensed under a Standard YouTube License.

Identifying common elements in a journal article

Look through the images and notes below to learn more about the various parts common to a scholarly article

The Citation information: Authors, Article title, Journal information and Abstract

• As shown in the image below, much of the citation information about the article appears on the first page. Here we see the title of journal, page numbers, and the publication date.
• Multiple authors are common in search and they usually include academic affiliations that are listed just near the authors' names, as shown in this image, or as a footnote at the bottom of the page. 
• Also on the first page, you can often see the abstract to the article, which is common.

The Abstract:

Read the abstract to determine if an article will be relevant, helpful, and comprehensible.

The abstract is more than a summary. It maps out the article with a brief overview of each section. The abstract may present the article's claim, purpose, methodology, and conclusions. 

In-Text Citations

• As show in the image of the article text below, you can see that in-text citations are common. This article also shows the contact information for the main author. 

(click on image to enlarge)

The Methodology:

If the article is a research study, the article will include the methodology and findings. 

The methodology lays out how the study was done. Check to see if the study was conducted with many or few subjects, over a long or short period of time, if subjects represent a broad or narrow cross-section of stakeholders, and if controls make results reliable.

The Results:

The results section presents the raw data gathered in the study. Seeing the data helps one determine if the conclusions are supported by the data.

Data set up for comparison - Tables and Charts:

The Discussion & Conclusion:

The References:

The references section of scholarly articles (and books) document the works upon, and sometimes against, which the current study was conducted. 

Use the references section as a research tool and "mine it" for relevant works you might look for.

Click on images to enlarge

Open the full articles linked below for a hands-on exploration of differences between popular and scholarly texts.

• A Balanced Life: How One Mom Finds Time for Work and Life Learning Magazine article by Vanessa Sheets, Natural Life, May/June 2009
• Well-balanced families?: A gendered analysis of work-life balance policies and work family practices

Video: How Library Stuff Works: Scholarly vs Popular Sources

Source: "How Library Stuff Works: Scholarly vs. Popular Sources" by McMaster Libraries , is licensed under a Standard YouTube License.

• << Previous: Find Scholarship
• Next: Evaluating Sources >>
• Last Updated: May 14, 2024 4:16 PM
• URL: https://libguides.greenriver.edu/engl127

Help | Advanced Search

Computer Science > Cryptography and Security

Title: large language models for cyber security: a systematic literature review.

Abstract: The rapid advancement of Large Language Models (LLMs) has opened up new opportunities for leveraging artificial intelligence in various domains, including cybersecurity. As the volume and sophistication of cyber threats continue to grow, there is an increasing need for intelligent systems that can automatically detect vulnerabilities, analyze malware, and respond to attacks. In this survey, we conduct a comprehensive review of the literature on the application of LLMs in cybersecurity (LLM4Security). By comprehensively collecting over 30K relevant papers and systematically analyzing 127 papers from top security and software engineering venues, we aim to provide a holistic view of how LLMs are being used to solve diverse problems across the cybersecurity domain. Through our analysis, we identify several key findings. First, we observe that LLMs are being applied to a wide range of cybersecurity tasks, including vulnerability detection, malware analysis, network intrusion detection, and phishing detection. Second, we find that the datasets used for training and evaluating LLMs in these tasks are often limited in size and diversity, highlighting the need for more comprehensive and representative datasets. Third, we identify several promising techniques for adapting LLMs to specific cybersecurity domains, such as fine-tuning, transfer learning, and domain-specific pre-training. Finally, we discuss the main challenges and opportunities for future research in LLM4Security, including the need for more interpretable and explainable models, the importance of addressing data privacy and security concerns, and the potential for leveraging LLMs for proactive defense and threat hunting. Overall, our survey provides a comprehensive overview of the current state-of-the-art in LLM4Security and identifies several promising directions for future research.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

• Search Menu
• Advance articles
• Author Interviews
• Research Curations
• Author Guidelines
• Open Access
• Submission Site
• Why Submit?
• About Journal of Consumer Research
• Editorial Board
• Advertising and Corporate Services
• Self-Archiving Policy
• Dispatch Dates
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Consumer research and value hierarchy, value hierarchy as inequality, a future research agenda, concluding remarks, data collection statement, author notes.

• < Previous

After 50 Years, It Is Time to Talk about Value Hierarchy and Inequality

• Article contents
• Figures & tables
• Supplementary Data

Lez E Trujillo-Torres, Benét DeBerry-Spence, Sonya A Grier, Søren Askegaard, After 50 Years, It Is Time to Talk about Value Hierarchy and Inequality, Journal of Consumer Research , Volume 51, Issue 1, June 2024, Pages 79–90, https://doi.org/10.1093/jcr/ucad040

• Permissions Icon Permissions

This research enriches the field’s perspective on value and argues that to seriously address inequality during the next 50 years, consumer research must explore processual aspects of value hierarchy and consider its relationship to inequality. Doing so recognizes the duality of structures as embodying outcomes and agency, as well as the need to view value not only as what it is but also as what it does. To begin to address limitations in the literature, we use empirical evidence from an investigation of the cancer care market from 1970 to 2021 to understand how value hierarchy shapes and manifests as inequality. This is conceptualized as: distribution of multilevel resources, consolidation of consumer power, stratification of consumer agency, and (de)credentialization of worthiness. Building on each of these, we discuss a research agenda for future JCR inquiries and introduce “value hierarchy as inequality” as a big-tent issue for consumer research.

During the past 50 years, the concept of value has had implications, directly or indirectly, for each and every Journal of Consumer Research ( JCR ) article published. This is because value, defined as a sense of benefit, meaning, or worth of marketplace entities, is critical to consumer decision-making ( Figueiredo and Scaraboto 2016 ; Zeithaml 1988 ). Marketplace entities can encompass concrete actors such as products, people, and institutions and also abstract ideas and causes. Certainly, then, value is a big-tent issue that is central to the development of new consumer behavior topics, given that it is at the core of “what” is being exchanged in a marketing context. But value also entails hierarchization. For example, as the cancer awareness marketing campaign in figure 1 illustrates, not all health consumption experiences are valued the same; rather, inequality permeates markets and many consumer experiences and is “a global issue that defines our time” ( United Nations 2023 ). We argue that to seriously address inequality over the next 50 years, consumer researchers must foreground the often ignored role, functioning, and impact of value hierarchy.

EXAMPLES OF PERCEIVED INEQUALITIES BETWEEN CANCER TYPES BY PANCREATIC CANCER PATIENTS

Source.— Pancreatic Cancer Action.

We define “value hierarchy” as a structure in which entities are layered according to a value rank or order ( Lenski 1984 ). Our understanding of value hierarchy is informed by structural perspectives of hierarchy and inequality found in organizational studies, in which hierarchy in everyday life plays a crucial role in the formation and reproduction of inequalities ( Bunderson et al 2016 ; Harrison and Klein 2007 ). In marketplaces, for example, consumers value brands, products, or experiences and establish value hierarchy among them ( Moreau, Lehmann, and Markman 2001 ). Inequalities are pronounced differences between marketplace entities ( Crockett and Grier 2021 ; DeBerry-Spence et al. 2023 ), which can limit privileges, opportunities, or resources; hinder market participation; support discriminatory practices; and lead to unequal outcomes ( Poole et al. 2021 ; Trujillo-Torres and DeBerry-Spence 2019 ). As such, hierarchy and inequalities mark imbalances in social arrangements ( Sewell 1992 ) as differences in the social valuation of groups ( Lamont 2012 ) and as moral justifications aligned with higher-order principles ( Boltanski and Thévenot 2006 ).

A structural perspective of value hierarchy is necessary to expose its foundational connections with inequality. Specifically, value hierarchy can result in and unfold as both centralized inequality ( Argote, Turner, and Fichman 1989 ; Harrison and Klein 2007 ) and steep inequality ( Bunderson et al. 2016 ). When centralized, inequality is embedded in the number and type of resources entities receive within a value hierarchy. For example, some consumers have historically received more social and material resources than others ( Poole et al. 2021 ). When steep, inequality is associated with the sheer depth or magnitude differences between entities within a value hierarchy ( Halevy et al. 2012 ; Leonard 1990 ). This is present in studies that note dramatic differences between consumers in power, status, and self-perception ( Bone, Christensen, and Williams 2014 ). These examples in the marketplace are consistent with studies’ findings that centralization and steepness are linked to negative implications. Both can manifest in “competition, differentiation, and (resentful) deviance among some [organizational] unit members” ( Harrison and Klein 2007 , 1206); in conflict ( Bendersky and Hays 2012 ; Bunderson et al. 2016 ); and in enduring discrimination of racial and ethnic groups ( Ray 2019 ). Taken as a whole, a structuralist perspective of hierarchy, then, provides a path for our work, which investigates value hierarchy as inequality. See web appendix A for key constructs summary.

We argue that marketers need to view value hierarchy as a process of inequality (i.e., a means to establish, sustain, and alter inequalities) and as “hierarchy resulting in inequality across valued characteristics or outcomes” ( Bunderson et al. 2016 , 1267). While a robust legacy that examines how consumers shape value in the marketplace exists in consumer research ( Gollnhofer, Weijo, and Schouten 2019 ; Nøjgaard 2023 ; Schau, Muñiz, and Arnould 2009 ; Zeithaml 1988 ), the emphasis is on inequality as solely an outcome of value hierarchy. Thus, theorizing value hierarchy as a process brings to the fore the processual aspects of its relationship to inequalities and moves the field forward to explore the structural relationships of value among entities . These can include the structures that arise and/or are sustained, the resources that are distributed, and the consequences and depth of inequality. Value hierarchy as inequality also recognizes the duality of structures as embodying outcomes and agency, and importantly, it addresses the need to view value not only as what it is but also as what it does. The relevance of this is perhaps obvious, given that value hierarchy implicates and links a multitude of market and social processes, such as legitimation, in which entities change value positions when these become (de)legitimized ( Humphreys 2010 ; Mimoun, Trujillo-Torres, and Sobande 2022 ), and categorization, in which value is assigned and sorted ( Moreau et al. 2001 ).

Our research, therefore, underscores the nascent theoretical conversation about consumers, value hierarchy, and inequality. We argue that value hierarchy shapes and manifests as inequality as distribution of multilevel resources, consolidation of consumer power, stratification of consumer agency, and (de)credentialization of worthiness. We draw empirical evidence demonstrating each of these elements from the context of the cancer care market and the experiences of people with breast, leukemia, or lung cancer from 1970 to 2021. This market, which represents approximately 7% (nearly $209 billion) of all U.S. health expenditures related to diseases ( National Cancer Institute 2020 ), is characterized by value hierarchy with marked inequalities in resources (public, governmental, and medical), product innovation, service provider distribution, and funding ( Wailoo 2010 ). The implications for health services consumption and related consumer experiences are indisputable. As such, it is an ideal context to illustrate value hierarchy as inequality. Details about our context, data collection, and analysis approaches appear in web appendices B and C . We use our conceptualization and the literature as building blocks to identify future research priorities for JCR inquiries.

While the literature on value is extensive, surprisingly, it has paid scant attention to the value hierarchies implicit in marketing and consumer practices, despite strong evidence of consumers’ power and capacity to establish, sustain, and alter value structures. Thus, a structural perspective of value hierarchy can complement and extend the historically prominent position that value and value creation practices have occupied in consumer research. In table 1 , we provide a brief synopsis of the consumer research literature vis-à-vis value hierarchies. For example, consumers, in groups and/or through consumer advocacy organizations, can place or rank entities at different value positions that not only change markets ( Nøjgaard 2023 ) but also influence the distributive process that dictates who gets what within these markets, such as attention, prestige, and power. Similarly, consumer movements can assign greater value to products by devising new value regimes and value flows that alter privilege positions among entities (e.g., wasted food vs. traditional supermarket food; Gollnhofer et al. 2019 ).

SELECT CONSUMER RESEARCH AND VALUE HIERARCHIES

Note.— Illustrative exemplars are not exhaustive and may be relevant to multiple consumer research areas.

Of note, our review reveals that value hierarchy is implicit in much of consumer research and marketing strategy, from specific theoretical perspectives, such as social marketing and segmenting, targeting, and positioning, to academic movements, such as transformative consumer research and Race in the Marketplace. Each of the theoretical areas in table 1 presents broad opportunities to expand the accumulated knowledge on value hierarchy. For example, the value perception and creation literature assigns and sorts value across entities ( Moreau et al. 2001 ), whereas the (de)stigmatization literature adds or removes (de)valuing characteristics ( Eichert and Luedicke 2022 ). Furthermore, value hierarchy is interconnected with core elements of marketing practices, such as segmenting, targeting, and positioning. Diaz Ruiz and Kjellberg (2020 , 431) show that through feral segmentation, consumers “coin customer categories” with differential worth. These consumer research studies are consistent with those that emphasize that value hierarchy is intertwined with “inequality in member power, status, or privilege” ( Bunderson et al. 2016 , 1266) and the perspective that markets create structures of inequality that involve schemas, rules, and social and material resources. This underscores the need to “better understand stability, change, and the institutionalization” ( Ray 2019 , 26) of inequality.

An implication for consumer research is that inequalities are not simply an epiphenomenon of value relationships (i.e., a by-product or tangentially related), but rather structurally co-constitutive of the positions that entities occupy within a value hierarchy. For example, inequalities can create significant drawbacks for consumers ( Trujillo-Torres and DeBerry-Spence 2023 ) and are associated with sociocultural factors such as “age, gender, race, ethnicity, migrant status and disability” ( United Nations 2023 ). These factors can lead to systemic injustices, such as racist and discriminatory practices ( Crockett and Grier 2021 ), or stigmatizing practices ( Wallendorf 2001 ). This is consistent with Ray’s (2019) observation that “racial inequality is not merely ‘in’ organizations but ‘of’ them (48)” and that, correspondingly, “cultural schemas [are] connected to social resources” (30). However, inequalities are not always detrimental. Pronounced differences can reflect consumers’ rankings of preferred products ( Moreau et al. 2001 ), brands ( Schau et al. 2009 ), or people ( Grier and Deshpandé 2001 ) or support competition and market creation ( Gollnhofer et al. 2019 ). Thus, a structural perspective of value hierarchy as inequality suggests that inequalities are not just stand-alone by-products of or distinct from structures and processes.

Taken as a whole, our research acknowledges the need to examine value hierarchy as outcomes and processes and highlights the structural relationships of value among entities. Such an examination can provide greater insight into “who gets what and why” ( Lenski 1984 , 1) and extend understanding of not only what value is but also what value does. Importantly, we make explicit the relationship between value hierarchy and inequalities and address the broader “moral imperative” ( United Nations 2023 ) related to inequality. Collectively, these considerations create a path for marketing scholarship to contribute to this global policy debate.

To begin to address the limitations in the literature, we conceptualize how value hierarchy shapes and manifests as inequality. Building on existing literature and drawing on 50 years’ worth of data from the cancer care market and consumer health care experiences (see table W5 for more examples), we conceptualize value hierarchy as the distribution of multilevel resources, consolidation of consumer power, stratification of consumer agency, and (de)credentialization of worthiness. These elements underscore the outcomes and processual aspects embedded in value hierarchy, consistent with structures as embodying outcomes and agency. Our work highlights the role of structure as a medium and an outcome ( Giddens 1984 ; Sewell 1992 ) and therefore acknowledges the duality of value hierarchy. While we detail each of these independently for analytical purposes, we note that they can co-occur and overlap.

Distribution of Multilevel Resources

The distribution of multilevel resources is when consumers influence the allocation of resources that are situated at multiple levels within a value hierarchy. This can involve consumers linking with a variety of actors to establish, sustain, or change their position within a hierarchy. Resources can vary from the intangible (e.g., social support, knowledge, skills, rules), to the material (e.g., capital, commodities), to any vehicle of power that consumers “[use] to gain, enhance, contest, or maintain” a certain position ( Ray 2019 , 31). As an outcome and a process, the distribution of multilevel resources implicates attributes and actions; that is, it both characterizes observed market inequalities and consumer experiences of resource differences and acts to sustain and (re)create value hierarchies. The distribution of these (i.e., what, to whom, and when) also shapes value hierarchy as inequality by implicating multiple levels of actors and resources, from the micro (e.g., consumer), to the meso (e.g., marketers, market), to the macro (e.g., government, legislation). For example, Huff, Humphreys, and Wilner (2021) show that the legitimacy of cannabis consumption, which implicates an improved position within a value hierarchy, depends on different types of consumers, producers, and the meta market, with the last including regulatory bodies, news media, and public opinion. These micro and macro hierarchical processes are also present in the Black Lives Matter movement, in which consumers marshal resources (e.g., hashtags, its name) to influence positions within a value hierarchy. As might be expected, then, if select entities receive resources (i.e., centralization) that are dramatically different from what others receive within a value hierarchy (i.e., steepness), structural dominance and inequalities are likely to arise over time.

Breast cancer activism illustrates value hierarchy as inequality vis-à-vis the distribution of multilevel resources in the cancer care market. In the 1990s, breast cancer activists, with help from political actors, instituted a new value hierarchy that directed resources (funding, attention) to breast cancer (the disease) and brought attention to and elevated in public discourse these patient experiences. In other words, activists created micro–macro linkages (e.g., Congress “lobby” days and rallies; web appendix C ) that resulted in outcomes such as the dramatic increase in funding in 1993 by the Clinton Administration for breast cancer research and a role in research funding oversight. These efforts also created inequalities in the cancer care market. For example, web appendix D shows that from 1993 to 2018, breast cancer was the highest U.S. government-funded cancer, far ahead of other cancers also with high rates of disease incidences and mortality. Breast cancer also led (followed by leukemia) across many measures of success (e.g., private donations, media coverage) and benefited from micro–meso linkages of consumers with marketers and other market institutions interested in supporting breast cancer and/or the experiences of patients through cause marketing ( Jain 2013 ). Indeed, in 2003, cancer awareness, driven by breast cancer, became the number one health problem in the United States in the national Gallup survey, reflecting another value hierarchy among various health concerns. By contrast, lung cancer had fewer resources, its patients were stigmatized, and patient advocacy was limited, reflecting prejudices against smoking ( Chapple, Ziebland, and McPherson 2004 ).

Consolidation of Consumer Power

The consolidation of consumer power is when consumers concurrently wield power and create outcomes from the mobilization of power. Acknowledging this duality recognizes the ability of consumers to successfully consolidate resources ( Sewell 1992 ) and to achieve certain outcomes. It also opens the door for greater examination of how consumers wield power to create “novel mechanisms” ( Ray 2019 , 35) that reproduce hierarchical relationships as inequality. After all, consumers create or support value hierarchies that differentially grant resources; those who are resource dominant (resource deprived) have significantly more (less) power. In other words, resource distribution involves centralization of power and steepness in power ( Sewell 1992 ), with obvious implications for markets and consumer experiences of inequality. For example, gentrification, in which lower-income residents are replaced with higher-income residents, reflects hierarchical differences with significant power implications for local commercial development and residents, including the power to create a sense of community and racial harmony ( Grier and Perry 2018 ).

Empirical evidence from the cancer care market shows consolidation and use of consumer power in consumer-driven product innovation by leukemia patients and activists. Such is the case of the Team in Training fundraising initiative, a program created in 1988 by a leukemia patient’s father, who raised $322,000 participating in the New York City marathon ( LLS 2023 ). This fundraising initiative showcases athletic humanitarians who self-sacrifice pursuing an athletic goal and a cancer cure, marshaling social and material resources to leukemia institutions, such as the Leukemia and Lymphoma Society (LLS), and leukemia patients. Initiatives such as these catapulted the power of the “patient hero” in public discourse. Not surprising, LLS became one of the most successful fundraising charities in the United States for private donations from 1970 to 2021, far above other cancer charities that represent diseases with higher mortality and incidence rates ( web appendix E ). Team in Training activities were also associated with extensive commercial solidarity and were key enablers of the power leveraged by leukemia organizations ( web appendix F ). As a result, the power of leukemia activism translated into the ability to foster commercially available treatment innovations for various leukemia types with private funding. Leukemia is only second to breast cancer in the number of Food and Drug Administration-approved drugs and treatments (75 and 108, respectively) ( web appendix G ). By contrast, lung cancer in third place has benefited from leukemia and breast cancer drug discoveries and research, showing that dominant entities can also uplift others. This suggests that value hierarchy as inequality may not be as steep among some types of cancer.

Stratification of Consumer Agency

Stratification of consumer agency entails consumers creating categories of entities with distinct levels of agency. This concerns central tenets of value hierarchy; that is, stratification implicates structure and agency and links value hierarchy as inequality that stems from the allocation of consumer agency. Agency involves deliberate action and exercise of subjectivity and free will in making decisions and communicating ( White and Wyn 1998 ), while structure is the organized rules and resources that actors create and reproduce in social interactions and that constitute a social system ( Giddens 1984 ). Agency differentials contribute to inequality processes through the maintenance, reproduction, or disruption of hierarchy. In this way, value hierarchy as inequality can typify the push and pull of structure and agency associated with Giddens’s (1984) theory of structuration. For example, Figueiredo et al. (2015) identify consumer agency as both constrained and fostered. They note that consumers in informal markets exert agency by transgressing the boundaries of formal economies but also acknowledge that respecting consumer agency is not always achievable. This suggests that consumers do not all have the same agency and that consumers’ agencies are not all valued the same.

We turn to the cancer market to empirically demonstrate the stratification of consumer agency. In the 1990s, disease-specific advocacy organizations such as the LLS actively shaped and reproduced notions of agency in leukemia patients, who often faced exclusion, stigma, and discrimination in school, employment, medical access, and other areas ( Wailoo 2010 ). For example, an LLS employee in a 1990 Chicago Tribune article argued for deliberate action and the exercise of free will: “Jeff is a survivor. Years ago, leukemia was a death sentence. But today many more people survive … we're winning the battle in a lot of cases” ( Ogintz 1990 ). Agency here refers to the ability of these patients to seek, access, and employ treatments successfully (e.g., bone marrow transplant, drugs) and communicate success. The active inclusion of leukemia patients in Team in Training activities such as marathons and triathlons complemented these rhetorical efforts. As a patient in remission stated: “You can do anything, if you want it badly enough” ( Willismaon 1994 ). These efforts have led to the reduction of inequalities for leukemia patients by enabling the use of products and technical innovations, reduced mortality, and increased public support (e.g., media coverage, private donations). However, tensions related to who benefits from agency allocations and which patients can enact agency remain. For example, lung cancer patients’ ability to seek treatments has been limited—truncated by changing social conditions (e.g., stigma against smoking, secondhand smoke danger), their perceived culpability, and nonsmoker victimization. Consequently, less public, scientific, and government attention has been given to the disease, patients, and advocacy organizations.

(De)Credentialization of Worthiness

The (de)credentialization of worthiness entails consumers having (or not having) credentials and the process of establishing (removing) credentials within a value hierarchy. In other words, (de)credentialization determines what is worthy within a value hierarchy. This focus is important because credentials and the process of credentialization implicate resources, power, agency, and, correspondingly, inequalities. Thus, credentials and credentialization can bear fewer inequalities (e.g., through inclusion) or more inequalities (e.g., through exclusion). For example, Ray (2019) notes that within organizations, whiteness has been an enduring credential for the allocation of resources, power, and agency to organizational actors, with negative implications (e.g., racism, discrimination) for non-white actors. In consumer research, various credentials of worthiness, from race to gender to class to sexual orientation, pertain to value hierarchy and relatedly inequality ( Mimoun et al. 2022 ).

(De)credentialization of worthiness occurs in the cancer care market. Until the late 2000s, the common credentials to obtain cancer treatments were survivability prospects (usually an early detected cancer) and considerations that often related to patients’ race and socioeconomic status ( Jain 2013 ). Patients with curable cancers and from middle-class and educated backgrounds ( Wailoo 2010 ) had more consumption opportunities than other cancer patients. Then, biomarker matching emerged as a new credential, aligning patients’ biological markers (e.g., proteins, cells, genes, nucleotides) with available therapeutic and preventive options. This decredentialization provided opportunities for patients who had previously experienced significant inequality to seek innovative treatments. This is the case for women with metastatic breast cancers, or “metavivors” whose cancer migrated from the breast to different organs or systems, who often feel excluded, as they are counternormative examples of “cured” early-stage survivors ( Jain 2013 ) and advocate for more attention to their disease and experiences. Though increasing these patients’ visibility, biomarker matching offers no guarantee of longevity (see web appendix H for metavivors’ photos), as only one-third of metastatic breast cancer patients live for at least 5 years, given the lack of treatments ( National Cancer Institute 2020 ). Biomarkers have also rendered visible the “previvors,” who are predisposed to breast and ovarian cancer due to BRCA1 and BRCA2 genes but who are not diagnosed with cancer. Their experiences with preventive cancer surgeries are often legitimized by public figures (e.g., Angelina Jolie) but can be minimized by actual cancer patients. Thus, biomarker matching has created a dynamic decredentialization of worthiness requirements in the cancer care market, removing some inequalities and revealing multiple hierarchies within the breast cancer patient community.

Building on our conceptualization of how value hierarchy both shapes and manifests as inequality as well as the literature, we discuss research areas for future JCR inquiries. Table 2 summarizes these efforts and highlights not only a variety of conceptual domains (consumer research areas) in which our conceptualization can find application but also multiple empirical contexts and questions that would benefit from further exploration.

FUTURE RESEARCH PRIORITIES AND QUESTIONS ON VALUE HIERARCHY AS INEQUALITY

Regarding the distribution of multilevel resources, we encourage the continued examination of how resource distribution can further enable or constrain inequalities in consumer actions, organizations, and outcomes within value hierarchy. Historically, consumer research on value has focused on the micro–meso linkages in markets ( Schau et al. 2009 ; Zeithaml 1988 ), despite evidence that macro-level institutions (e.g., governments, legislation; Humphreys 2010 ) and ideologies ( Giesler and Veresiu 2014 ) are key sources of legitimacy. Moreover, inequality, whether in the marketplace or society, is systemic and institutionalized ( Poole et al. 2021 ), which is evident in studies on race in the marketplace ( Trujillo-Torres and DeBerry-Spence 2023 ). We can therefore also leverage Ray’s (2019) tenets of racialized organizations for the study of consumer value hierarchies. For example, future examinations could investigate the factors that determine the roles of conflict, political considerations, allies, coalitions, and ideologies on the levels and boundaries of resource distribution within value hierarchies in consumer markets. Such research could enrich understanding of how value hierarchy affects the “stability and change” ( Ray 2019 , 30) of inequalities by “accumulating, managing, monopolizing, and apportioning … resources.”

In terms of the consolidation of consumer power, prior research has demonstrated the status games that consumers play to compensate for an underprivileged position in a static value hierarchy ( Rucker and Galinsky 2008 ), but also the variety of ways value orientations can shift through consumer agency. For example, Gollnhofer et al. (2019) showcase the collaboration of consumers with marketers to assign greater value to wasted food by devising alternative “flows of food to waste” that position wasted food as worthy of consumption ( Gollnhofer et al. 2019 , 461), thereby subverting one value hierarchy and (being in the process of) establishing another. Looking ahead, future consumer research on value hierarchy as inequality could investigate how consolidated and consolidating consumer power opens new avenues for consumer-driven innovation and/or collaboration ( Kjeldgaard et al. 2017 ). Scholars might also examine how consumers consolidate and deploy resources from governmental and commercial resources to, for example, resist changes that occur in neighborhoods following urban revitalization that prioritizes some consumers’ interests over others. Moreover, studies on consumer power across gentrifying markets in diverse contexts would be fruitful, as particular market configurations differ given competing ideologies, historical influences, and regulations.

Regarding the stratification of consumer agency, future research could benefit from a greater understanding of the tensions that underlie consumer agency and value hierarchy within markets, societies, and institutions and specifically the tensions that arise when consumers desire to enact agency but are either limited or restricted in their ability to do so. This is the case for “fatshionistas,” who are subject to inequality in terms of product options and consumption opportunities ( Scaraboto and Fischer 2013 ). These consumers exhibit agency by blogging and participating in the Fat Acceptance Movement, but their agency is bound within the logics of the fashion system. Conversely, Adkins and Ozanne (2005) demonstrate how low-literate consumers can mobilize “consumer literacy” through compensating coping skills. Here, value hierarchy is dominant but not hegemonic. Consumer research could also examine how these tensions manifest and, importantly, how they relate to the processes associated with the maintenance and/or reproduction of value hierarchy as inequality.

Future studies could also shed greater light on the stratification of consumer agency by examining periods of destabilization in consumption and markets. For example, the COVID-19 pandemic revealed the significance of long-standing health disparities between population groups ( DeBerry-Spence and Trujillo-Torres 2022 ), and thus, research could benefit from a societal-level understanding of value hierarchy, agency allocation, and resource distribution. This underscores the need to realize inequality in agency to identify novel intervention points for transformative consumer research efforts. Such a focus would enhance understanding of how “agency, motive, and action” relate “to resources and cultural schemas” ( Ray 2019 , 35).

Future research on (de)credentialization of worthiness would give the field a better understanding of the dynamism of value hierarchy as inequality, including its stability, change, and institutionalization. For example, Mimoun et al. (2022) identify hierarchical differences based on worthiness credentials that impeded (facilitated) fertility technology consumption. In their study, single women, women over the age of 40, and LGBTQ+ individuals defied the worthiness requirements of those who were considered “good parents”: young, married, heterosexual couples. Importantly, credentialization can be dynamic to reflect changes that alter conceptions of worthiness, such as a changing historical context, power relations, or new systems of thought ( Ray 2019 ). In Mimoun et al. (2022) , for example, developments such as higher rates of female participation in the workforce, higher earnings of women, and changing societal acceptance of sexual identity have decredentialized the old value hierarchy, legitimized consumer segments, and decreased inequalities in this market. Conversely, Nøjgaard (2023) examines the credentialization processes through which value hierarchies are established from “objectifying” value translation processes in a formalized consumer activism setting. Future studies could extend the dynamism of (de)credentialization of worthiness by investigating, in more depth, its intersection with other social processes, such as legitimation and stigmatization, and the transferability or inheritance of credentials over time. Other areas that have received scant research attention include how the valence or neutrality of credentials is (re)constructed and the role of (de)credentialization in market evolution and disruption. These inquiries could further delineate the processes and effects of value hierarchy.

Our research highlights the significance of inequality in markets and the relationship to value hierarchy. Of note, our understanding of value hierarchy infers a “valorization process,” as social hierarchies are not given but are necessarily the outcome of social processes. We introduce the concept “value hierarchy as inequality” and draw from the literature and empirical evidence to conceptualize and demonstrate how value hierarchy shapes and manifests as inequality. Doing so helps move the field forward to comprehend value hierarchy as both an outcome and a process and brings to the fore the structural relationships of value among entities. It also makes clear the need to view value not only as what it is but also as what it does. Our conceptualization is supported by more than 50 years of empirical evidence drawn from the cancer care market. Thus, our work provides a springboard for consumer behavior researchers to pursue more influential scholarship that addresses value hierarchies and their importance for inequality. It also aligns with the global recognition that inequalities are not inconsequential. Thus, we call for future JCR inquiries over the next 50 years to address value hierarchy as inequality.

The first author led the data collection efforts from 2015 to 2021. The archival data were collected from library databases, news media, blogs and social media, and other secondary sources. The authors also conducted observations. The data were analyzed jointly by all authors. The data are currently stored in a project directory on Research Box.

Lez E. Trujillo-Torres ( [email protected] ) is an assistant professor of marketing at the University of Illinois Chicago, Chicago, IL 60607, USA.

Benét DeBerry-Spence ( [email protected] ) is a professor of marketing at the University of Illinois Chicago, Chicago, IL 60607, USA.

Sonya A. Grier ( [email protected] ) is a professor at the Department of Marketing, American University, Washington, DC 20016, USA.

Søren Askegaard ( [email protected] ) is a professor at the Department of Business & Management and chair at Danish Institute for Advanced Study (DIAS), University of Southern Denmark, 5230 Odense M, Denmark.

The first two authors contributed equally to this work. This article is based on the first author’s dissertation, which was completed at the University of Illinois Chicago. The authors thank the editor, associate editor, and three reviewers for their support and constructive feedback. The authors report no conflicts of interest. Supplementary materials are included in the web appendix accompanying the online version of this article.

Aaker Jennifer L. , Brumbaugh Anne M. , Grier Sonya A. ( 2000 ), “ Nontarget Markets and Viewer Distinctiveness: The Impact of Target Marketing on Advertising Attitudes ,” Journal of Consumer Psychology , 9 ( 3 ), 127 – 40 .

Google Scholar

Adkins Natalie R. , Ozanne Julie L. ( 2005 ), “ The Low Literate Consumer ,” Journal of Consumer Research , 32 ( 1 ), 93 – 105 .

Argote Linda , Turner Marlene E. , Fichman Mark ( 1989 ), “ To Centralize or Not to Centralize: The Effects of Uncertainty and Threat on Group Structure and Performance ,” Organizational Behavior and Human Decision Processes , 43 ( 1 ), 58 – 74 .

Arsel Zeynep , Bean Jonathan ( 2013 ), “ Taste Regimes and Market-Mediated Practice ,” Journal of Consumer Research , 39 ( 5 ), 899 – 917 .

Askegaard Søren , Linnet Jeppe Trolle ( 2011 ), “ Towards an Epistemology of Consumer Culture Theory: Phenomenology and the Context of Context ,” Marketing Theory , 11 ( 4 ), 381 – 404 .

Baker Stacey Menzel , Azzari Courtney Nations , Thomas Meredith Rhoads , Bennett Aronté Marie ( 2020 ), “ When Does the Social Service Ecosystem Meet Consumption Needs? A Power–Justice–Access Model of Holistic Well-being from Recipients’ Perspectives ,” Journal of Public Policy & Marketing , 39 ( 2 ), 220 – 39 .

Bendersky Corinne , Hays Nicholas A. ( 2012 ), “ Status Conflict in Groups ,” Organization Science , 23 ( 2 ), 323 – 40 .

Bloom Paul N. ( 2009 ), “ Overcoming Consumption Constraints through Social Entrepreneurship ,” Journal of Public Policy & Marketing , 28 ( 1 ), 128 – 34 .

Boltanski Luc , Thévenot Laurent ( 2006 ), On Justification: Economies of Worth , Vol. 27 , Princeton, NJ : Princeton University Press .

Google Preview

Bone Sterling A. , Christensen Glenn L. , Williams Jerome D. ( 2014 ), “ Rejected, Shackled, and Alone: The Impact of Systemic Restricted Choice on Minority Consumers' Construction of Self ,” Journal of Consumer Research , 41 ( 2 ), 451 – 74 .

Bunderson J. Stuart , Van Der Vegt Gerben S. , Cantimur Yeliz , Rink Floor ( 2016 ), “ Different Views of Hierarchy and Why They Matter: Hierarchy as Inequality or as Cascading Influence ,” Academy of Management Journal , 59 ( 4 ), 1265 – 89 .

Chapple Alison , Ziebland Sue , McPherson Ann ( 2004 ), “ Stigma, Shame, and Blame Experienced by Patients with Lung Cancer: Qualitative Study ,” BMJ (Clinical Research ed.) , 328 ( 7454 ), 1470 .

Chatzidakis Andreas , Maclaran Pauline , Varman Rohit ( 2021 ), “ The Regeneration of Consumer Movement Solidarity ,” Journal of Consumer Research , 48 ( 2 ), 289 – 308 .

Crockett David , Grier Sonya A. , Williams Jacqueline A. ( 2003 ), “ Coping with Marketplace Discrimination: An Exploration of the Experiences of Black Men ,” Academy of Marketing Science Review , 4 ( 7 ), 1 – 21 .

Crockett David , Grier A. Sonya ( 2021 ), “ Race in the Marketplace and COVID-19 ,” Journal of Public Policy & Marketing , 40 ( 1 ), 89 – 91 .

DeBerry-Spence Benét , Ekpo Akon E. , Hogan Daniel ( 2019 ), “ Mobile Phone Visual Ethnography (MpVE): Bridging Transformative Photography and Mobile Phone Ethnography ,” Journal of Public Policy & Marketing , 38 ( 1 ), 81 – 95 .

DeBerry-Spence Benét , Trujillo-Torres Lez ( 2022 ), “ Don’t Give Us Death like This!’ Commemorating Death in the Age of COVID-19 ,” Journal of the Association for Consumer Research , 7 ( 1 ), 27 – 35 .

DeBerry-Spence Benét , Trujillo-Torres Lez E. , Sengupta Rumela , Matsumoto Kohei , Chen Jia ( 2023 ), “ Marketing’s Role in Promoting the Common Good: A Systematic Examination and an Agenda for Future Inquiry ,” Journal of Public Policy & Marketing , 42 ( 2 ), 95 – 114 .

Diaz Ruiz Carlos A. , Kjellberg Hans ( 2020 ), “ Feral Segmentation: How Cultural Intermediaries Perform Market Segmentation in the Wild ,” Marketing Theory , 20 ( 4 ), 429 – 57 .

Eichert Christian A. , Luedicke Marius K. ( 2022 ), “ Almost Equal: Consumption under Fragmented Stigma ,” Journal of Consumer Research , 49 ( 3 ), 409 – 29 .

Ekici Ahmet , Peterson Mark ( 2009 ), “ The Unique Relationship between Quality of Life and Consumer Trust in Market-Related Institutions among Financially Constrained Consumers in a Developing Country ,” Journal of Public Policy & Marketing , 28 ( 1 ), 56 – 70 .

Epp Amber M. , Price Linda L. ( 2010 ), “ The Storied Life of Singularized Objects: Forces of Agency and Network Transformation ,” Journal of Consumer Research , 36 ( 5 ), 820 – 37 .

Figueiredo Bernardo , Chelekis Jessica , DeBerry-Spence Benet , Fırat A. Fuat , Ger Güliz , Godefroit-Winkel Delphine , Kravets Olga , Moisander Johanna , Nuttavuthisit Krittinee , Peñaloza Lisa , Tadajewski Mark ( 2015 ), “ Developing Markets? Understanding the Role of Markets and Development at the Intersection of Macromarketing and Transformative Consumer Research (TCR) ,” Journal of Macromarketing , 35 ( 2 ), 257 – 71 .

Figueiredo Bernardo , Scaraboto Daiane ( 2016 ), “ The Systemic Creation of Value through Circulation in Collaborative Consumer Networks ,” Journal of Consumer Research , 43 ( 4 ), 509 – 33 .

Gau Roland , Ramirez Edward , Barua Maria E. , Gonzalez Ricardo ( 2014 ), “ Community-based Initiatives and Poverty Alleviation in Subsistence Marketplaces ,” Journal of Macromarketing , 34 ( 2 ), 160 – 70 .

Giddens Anthony ( 1984 ), The Constitution of Society: Outline of the Theory of Structuration , Berkeley : University of California Press .

Giesler Markus , Veresiu Ela ( 2014 ), “ Creating the Responsible Consumer: Moralistic Governance Regimes and Consumer Subjectivity ,” Journal of Consumer Research , 41 ( 3 ), 840 – 57 .

Gollnhofer Johanna F. , Weijo Henri A. , Schouten John W. ( 2019 ), “ Consumer Movements and Value Regimes: Fighting Food Waste in Germany by Building Alternative Object Pathways ,” Journal of Consumer Research , 46 ( 3 ), 460 – 82 .

Grier Sonya A. , Brumbaugh Anne M. ( 1999 ), “ Noticing Cultural Differences: Advertising Meanings Created by Target and Non-Target Markets ,” Journal of Advertising , 28 ( 1 ), 79 – 93 .

Grier Sonya A. , Kumanyika Shiriki K. ( 2008 ), “ The Context for Choice: Health Implications of Targeted Food and Beverage Marketing to African Americans ,” American Journal of Public Health , 98 ( 9 ), 1616 – 29 .

Grier Sonya A. , Schaller Tracey King ( 2020 ), “ Operating in a Constricted Space: Policy Actor Perceptions of Targeting to Address US Health Disparities ,” Journal of Public Policy & Marketing , 39 ( 1 ): 31 – 47 .

Grier Sonya , Bryant Carol A. ( 2005 ), “ Social Marketing in Public Health ,” Annual Review of Public Health , 26 , 319 – 39 .

Grier Sonya , Davis Brennan ( 2013 ), “ Are All Proximity Effects Created Equal? Fast Food near Schools and Body Weight among Diverse Adolescents ,” Journal of Public Policy & Marketing , 32 ( 1 ), 116 – 28 .

Grier Sonya A. , Deshpandé Rohit ( 2001 ), “ Social Dimensions of Consumer Distinctiveness: The Influence of Social Status on Group Identity and Advertising Persuasion ,” Journal of Marketing Research , 38 ( 2 ), 216 – 24 .

Grier Sonya A. , Perry Vanessa G. ( 2018 ), “ Dog Parks and Coffee Shops: Faux Diversity and Consumption in Gentrifying Neighborhoods ,” Journal of Public Policy & Marketing , 37 ( 1 ), 23 – 38 .

Halevy Nir , Chou Eileen Y. , Galinsky Adam D. , Murnighan J. Keith ( 2012 ), “ When Hierarchy Wins: Evidence from the National Basketball Association ,” Social Psychological and Personality Science , 3 ( 4 ), 398 – 406 .

Harrison David A. , Klein Katherine J ( 2007 ), “ What's the Difference? Diversity Constructs as Separation, Variety, or Disparity in Organizations ,” Academy of Management Review , 32 ( 4 ), 1199 – 228 .

Hein Wendy , Steinfield Laurel , Ourahmoune Nacima , Coleman Catherine A. , Zayer Linda Tuncay , Littlefield Jon ( 2016 ), “ Gender Justice and the Market: A Transformative Consumer Research Perspective ,” Journal of Public Policy & Marketing , 35 ( 2 ), 223 – 36 .

Huff Aimee Dinnin , Humphreys Ashlee , Wilner Sarah J. S. ( 2021 ), “ The Politicization of Objects: Meaning and Materiality in the U.S. Cannabis Market ,” Journal of Consumer Research , 48 ( 1 ), 22 – 50 .

Hui Sam , Krishnamurthy Parthasarathy , Kumar Shiv , Siddegowda Hareesha B. , Patel Prachi ( 2020 ), “ Understanding the Effectiveness of Peer Educator Outreach on Reducing Sexually Transmitted Infections: The Role of Prevention vs. Early Detection ,” Marketing Science , 39 ( 3 ), 500 – 15 .

Humphreys Ashlee ( 2010 ), “ Semiotic Structure and the Legitimation of Consumption Practices: The Case of Casino Gambling ,” Journal of Consumer Research , 37 ( 3 ), 490 – 510 .

Jagadale Sujit Raghunathrao , Kadirov Djavlonbek , Chakraborty Debojyoti ( 2018 ), “ Tackling the Subaltern Quandary: Marketing Systems of Dignity ,” Journal of Macromarketing , 38 ( 1 ), 91 – 111 .

Jagadale Sujit Raghunathrao , Kemper Joya ( 2022 ), “ ‘Give It Up!’: A Macro-Social Marketing Approach to India's Clean Cooking Fuel Access ,” Journal of Macromarketing , 42 ( 3 ), 433 – 53 .

Jain S. Lochlann ( 2013 ), Malignant: How Cancer Becomes Us , Berkely : University of California Press .

Johnson Guillaume D. , Thomas Kevin D. , Grier Sonya A. ( 2017 ), “ When the Burger Becomes Halal: A Critical Discourse Analysis of Privilege and Marketplace Inclusion ,” Consumption Markets & Culture , 20 ( 6 ), 497 – 522 .

Johnson Guillaume D. , Thomas Kevin D. , Harrison Anthony Kwame , Grier Sonya A. , eds. ( 2019 ), Race in the Marketplace: Crossing Critical Boundaries , Cham, Switzerland : Palgrave Macmillan .

Kjeldgaard Dannie , Askegaard Søren , Ørnstedt Rasmussen Jannick , Østergaard Per ( 2017 ), “ Consumers’ Collective Action in Market System Dynamics: A Case of Beer ,” Marketing Theory , 17 ( 1 ), 51 – 70 .

Kotler Philip , Roberto Ned , Leisner Tony ( 2006 ), “ Alleviating Poverty: A Macro/Micro Marketing Perspective ,” Journal of Macromarketing , 26 ( 2 ), 233 – 9 .

Kozinets Robert V. , Handelman Jay M. ( 2004 ), “ Adversaries of Consumption: Consumer Movements, Activism, and Ideology ,” Journal of Consumer Research , 31 ( 3 ), 691 – 704 .

Lamont Michèle ( 2012 ), “ Toward a Comparative Sociology of Valuation and Evaluation ,” Annual Review of Sociology , 38 ( 1 ), 201 – 21 .

Lenski Gerhard E. ( 1984 ), Power and Privilege: A Theory of Social Stratification , Chapel Hill : University of North Carolina Press .

Leonard Jonathan S. ( 1990 ), “ Executive Pay and Firm Performance ,” Industrial & Labor Relations Review , 43 , 13 – 29 .

LLS ( 2023 ), “Team in Training. The History: Beat Cancer to the Finish Line,” https://www.teamintraining.org/about/history , Last Accessed June 29, 2023.

Martin Kelly D. , Paul Hill Ronald ( 2012 ), “ Life Satisfaction, Self-Determination, and Consumption Adequacy at the Bottom of the Pyramid ,” Journal of Consumer Research , 38 ( 6 ), 1155 – 68 .

McMillan Tracie ( 2017 ), “What Do We Think Poverty Looks Like?” New York Times , July 8. Last Accessed June 30, 2023. https://nyti.ms/3KG80kS .

Mimoun Laetitia , Trujillo-Torres Lez , Sobande Francesca ( 2022 ), “ Social Emotions and the Legitimation of the Fertility Technology Market ,” Journal of Consumer Research , 48 ( 6 ), 1073 – 95 .

Mirabito Ann M. , Otnes Cele C. , Crosby Elizabeth , Wooten David B. , Machin Jane E. , Pullig Chris , Adkins Natalie Ross , Dunnett Susan et al.  ( 2016 ), “ The Stigma Turbine: A Theoretical Framework for Conceptualizing and Contextualizing Marketplace Stigma ,” Journal of Public Policy & Marketing , 35 ( Fall ), 170 – 84 .

Moreau C. Page , Lehmann Donald R. , Markman Arthur B. ( 2001 ), “ Entrenched Knowledge Structures and Consumer Response to New Products ,” Journal of Marketing Research , 38 ( 1 ), 14 – 29 .

National Cancer Institute ( 2020 ), “Cancer Statistics,” https://www.cancer.gov/about-cancer/understanding/statistics#:~:text=The%20rate%20of%20new%20cases,on%202013%E2%80%932017%20deaths) , Last Accessed June 29, 2023.

Nøjgaard Mikkel ( 2023 ), “ The Value-Translation Model of Consumer Activism: How Consumer Watchdog Organizations Change Markets ,” Journal of Consumer Research , 49 ( 6 ), 967 – 86 .

Olson KayLoni L. , Landers Jacob D. , Thaxton Tyler T. , Emery Charles F. ( 2019 ), “ The Pain of Weight-Related Stigma among Women with Overweight or Obesity ,” Stigma and Health , 4 ( 3 ), 243 .

Ogintz Eileen ( 1990 ), “Slice of Life,” Chicago Tribune , March 23, https://www.chicagotribune.com/news/ct-xpm-1990-05-23-9002110963-story.html , Last Accessed June 29, 2023.

Poole Sonja Martin , Grier Sonya A. , Thomas Kevin D. , Sobande Francesca , Ekpo Akon E. , Torres Lez Trujillo , Addington Lynn A. , Weekes-Laidlow Melinda , Henderson Geraldine Rosa ( 2021 ), “ Operationalizing Critical Race Theory in the Marketplace ,” Journal of Public Policy & Marketing , 40 ( 2 ), 126 – 42 .

Ray Victor ( 2019 ), “ A Theory of Racialized Organizations ,” American Sociological Review , 84 ( 1 ), 26 – 53 .

Rucker Derek D. , Galinsky Adam D. ( 2008 ), “ Desire to Acquire: Powerlessness and Compensatory Consumption ,” Journal of Consumer Research , 35 ( 2 ), 257 – 67 .

Scaraboto Daiane ( 2015 ), “ Selling, Sharing, and Everything in Between: The Hybrid Economies of Collaborative Networks ,” Journal of Consumer Research , 42 ( 1 ), 152 – 76 .

Scaraboto Daiane , Fischer Eileen ( 2013 ), “ Frustrated Fatshionistas: An Institutional Theory Perspective on Consumer Quests for Greater Choice in Mainstream Markets ,” Journal of Consumer Research , 39 ( 6 ), 1234 – 57 .

Schau Hope J. , Muñiz Albert Jr. , Arnould Eric J. ( 2009 ), “ How Brand Community Practices Create Value ,” Journal of Marketing , 73 ( 5 ), 30 – 51 .

Scott Maura L. , Nowlis Stephen M. , Mandel Naomi , Morales Andrea C. ( 2008 ), “ The Effects of Reduced Food Size and Package Size on the Consumption Behavior of Restrained and Unrestrained Eaters ,” Journal of Consumer Research , 35 ( 3 ), 391 – 405 .

Sewell William H. Jr. ( 1992 ), “ A Theory of Structure: Duality, Agency, and Transformation ,” American Journal of Sociology , 98 ( 1 ), 1 – 29 .

Sredl Katherine C. ( 2018 ), “ Gendered Market Subjectivity: Autonomy, Privilege, and Emotional Subjectivity in Normalizing Post-socialist Neoliberal Ideology ,” Consumption Markets & Culture , 21 ( 6 ), 532 – 53 .

Tadajewski Mark , Chelekis Jessica , DeBerry-Spence Benét , Figueiredo Bernardo , Kravets Olga , Nuttavuthisit Krittinee , Peñaloza Lisa , Moisander Johanna ( 2014 ), “ The discourses of marketing and development: Towards ‘critical transformative marketing research’ ,” Journal of Marketing Management , 30 ( 17–18 ), 1728 – 71 .

Tanner Jr. John F. , Tanner Emily C. ( 2020 ), “ Fairy Tales Don’t Come True: The Impact of Aspirational Distance on Teen Pregnancy Prevention Messages ,” Journal of Public Policy & Marketing , 39 ( 1 ), 15 – 30 .

Trujillo-Torres Lez , DeBerry-Spence Benét ( 2019 ), “ Consumer Valorization Strategies in Traumatic Extraordinary Experiences ,” Journal of the Academy of Marketing Science , 47 ( 3 ), 516 – 31 .

Trujillo-Torres Lez , DeBerry-Spence Benét ( 2023 ), “ In the Back of the Bus: Racialized High Risk Consumption and Sickle Disease ,” Journal of the Association for Consumer Research , 8 ( 1 ), 8 – 20 .

Umashankar Nita , Srinivasan Raji ( 2013 ), “ Designing Social Interventions to Improve Newcomer Adjustment: Insights from the Indian Sex Worker Community ,” Journal of Public Policy & Marketing , 32 ( 2 ), 271 – 85 .

United Nations ( 2023 ), “Inequality,” https://www.un.org/en/desa/inequality-%E2%80%93-defining-challenge-our-time , Last Accessed June 29, 2023.

Valor Carmen , Lloveras Javier , Papaoikonomou Eleni ( 2001 ), “ The Role of Emotion Discourse and Pathic Stigma in the Delegitimization of Consumer Practices ,” Journal of Consumer Research , 47 ( 5 ), 636 – 53 .

Viswanathan Madhu , Rosa José Antonio , Ruth Julie A. ( 2010 ), “ Exchanges in Marketing Systems: The Case of Subsistence Consumer–Merchants in Chennai, India ,” Journal of Marketing , 74 ( 3 ), 1 – 17 .

Viswanathan Madhubalan , Sridharan Srinivas , Gau Roland , Ritchie Robin ( 2009 ), “ Designing Marketplace Literacy Education in Resource-Constrained Contexts: Implications for Public Policy and Marketing ,” Journal of Public Policy & Marketing , 28 ( 1 ), 85 – 94 .

Wooliscroft Ben , Ganglmair-Wooliscroft Alexandra , Noone Abigayle ( 2014 ), “ The Hierarchy of Ethical Consumption Behavior: The Case of New Zealand ,” Journal of Macromarketing , 34 ( 1 ), 57 – 72 .

Wailoo Keith ( 2010 ), How Cancer Crossed the Color Line , New York : Oxford University Press .

Wallendorf Melanie ( 2001 ), “ Literally Literacy ,” Journal of Consumer Research , 27 ( 4 ), 505 – 11 .

White Rob , Wyn Johanna ( 1998 ), “ Youth Agency and Social Context ,” Journal of Sociology , 34 ( 3 ), 314 – 27 .

Willismaon Norma ( 1994 ), “Arapahoe Man Celebrates Remission of Leukemia,” Wind River News , May 31, https://www.proquest.com/docview/362732295/9F6CE909E6F94767PQ/1?accountid=14552 , Last Accessed June 29, 2023.

Zeithaml Valarie A. ( 1988 ), “ Consumer Perceptions of Price, Quality, and Value: A Means-End Model and Synthesis of Evidence ,” Journal of Marketing , 52 ( 3 ), 2 – 22 .

Supplementary data

Email alerts, citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1537-5277
• Print ISSN 0093-5301
• Copyright © 2024 Journal of Consumer Research Inc.
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

• Download PDF
• Share X Facebook Email LinkedIn
• Permissions

Burden of Mental Disorders and Suicide Attributable to Childhood Maltreatment

• 1 The Matilda Centre for Research in Mental Health and Substance Use, The University of Sydney, Sydney, New South Wales, Australia
• 2 Department of Clinical, Educational, and Health Psychology, Division of Psychology and Language Sciences, University College London, London, United Kingdom
• 3 Department of Epidemiology, Mailman School of Public Health, Columbia University, New York, New York

Question   What proportion of mental health conditions and burden in Australia is attributable to childhood maltreatment?

Findings   This meta-analysis found, after controlling for genetic and environmental confounding, that childhood maltreatment accounts for 21% to 41% of common mental health conditions in Australia, with the highest attributable proportion for suicide attempts and self-harm. More than 1.8 million cases of depressive, anxiety, and substance use disorders, 66 143 years of life lost, and 184 636 disability-adjusted life-years could be prevented if childhood maltreatment was eradicated in Australia.

Meaning   Efforts to prevent child maltreatment exposure have the potential to improve mental health at a population level in Australia.

Importance   The proportion of mental disorders and burden causally attributable to childhood maltreatment is unknown.

Objective   To determine the contribution of child maltreatment to mental health conditions in Australia, accounting for genetic and environmental confounding.

Design, Setting, and Participants   This meta-analysis involved an epidemiological assessment accounting for genetic and environmental confounding between maltreatment and mental health and 3 cross-sectional national surveys: the Australian Child Maltreatment Study (ACMS) 2023, National Study of Mental Health and Well-being 2020-2022, and Australian Burden of Disease Study 2023. Causal estimates were derived on the association between childhood maltreatment and mental health conditions from a meta-analysis of quasi-experimental studies. This was combined with the prevalence of maltreatment from the ACMS to calculate the population attributable fraction (PAF). The PAF was applied to the number and burden of mental health conditions in Australia, sourced from 2 population-based, nationally representative surveys of Australians aged 16 to 85 years, to generate the number and associated burden of mental disorders attributable to child maltreatment.

Exposure   Physical abuse, sexual abuse, emotional abuse, or neglect prior to age 18 years.

Main Outcomes and Measures   Proportion and number of cases, years of life lost, years lived with disability, and disability-adjusted life-years of mental health conditions (anxiety, depression, harmful alcohol and drug use, self-harm, and suicide attempt) attributable to childhood maltreatment.

Results   Meta-analytic estimates were generated from 34 studies and 54 646 participants and applied to prevalence estimates of childhood maltreatment generated from 8503 Australians. Childhood maltreatment accounted for a substantial proportion of mental health conditions, ranging from 21% (95% CI, 13%-28%) for depression to 41% (95% CI, 27%-54%) of suicide attempts. More than 1.8 million cases of depressive, anxiety, and substance use disorders could be prevented if childhood maltreatment was eradicated. Maltreatment accounted for 66 143 years of life lost (95% CI, 43 313-87 314), primarily through suicide, and 184 636 disability-adjusted life-years (95% CI, 109 321-252 887).

Conclusions and Relevance   This study provides the first estimates of the causal contribution of child maltreatment to mental health in Australia. Results highlight the urgency of preventing child maltreatment to reduce the population prevalence and burden of mental disorders.

Read More About

Grummitt L , Baldwin JR , Lafoa’i J , Keyes KM , Barrett EL. Burden of Mental Disorders and Suicide Attributable to Childhood Maltreatment. JAMA Psychiatry. Published online May 08, 2024. doi:10.1001/jamapsychiatry.2024.0804

Manage citations:

© 2024

Artificial Intelligence Resource Center

Psychiatry in JAMA : Read the Latest

Browse and subscribe to JAMA Network podcasts!

Others Also Liked

Select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing
• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts

U.S. halts funding to virus research organization linked to pandemic probes

Officials cited EcoHealth Alliance’s failure to monitor virus experiments in a Chinese lab before the coronavirus pandemic.

Federal health officials Wednesday suspended funding to a U.S. research organization linked to investigations about the novel coronavirus ’s origins, saying the move is “necessary to protect the public interest” given the organization’s failure to monitor virus experiments in a Chinese lab before the pandemic.

Federal officials also are seeking to block future funding to EcoHealth Alliance, which worked with the Wuhan Institute of Virology in China, citing new evidence about EcoHealth’s actions that emerged ahead of a contentious congressional hearing this month.

According to the Department of Health and Human Services, EcoHealth failed to adequately monitor and report on risky virus experiments at the Wuhan Institute of Virology, violating the terms of its federal grant and biosafety requirements. EcoHealth is now suspended from federal funding and will be potentially debarred , HHS said in its letter to the organization. Suspended and debarred organizations are ineligible to receive new federal contracts, grants and other types of funding from the United States.

EcoHealth Alliance has been the target of investigations since the earliest days of the pandemic, with the New York nonprofit facing accusations that its experiments with bat coronaviruses could have contributed to the development and spread of SARS-CoV-2, the virus linked to the deaths of millions of people around the globe.

President Donald Trump in April 2020 publicly questioned EcoHealth’s work in Wuhan — the city where the outbreak was first detected — and federal officials abruptly suspended a federal grant to the organization. That grant was restored last year over Republican objections. Federal officials have repeatedly pressed the organization to turn over documents, including a 2021 clash between officials and EcoHealth over unpublished data from a virus experiment involving mice.

EcoHealth and its defenders have said the organization is being scapegoated and that no evidence exists linking its work to the pandemic.

“We are confident that the bat coronavirus research conducted by EcoHealth Alliance and the Wuhan Institute of Virology could not have started the COVID-19 pandemic,” Peter Daszak, EcoHealth’s president, wrote last year in an email to The Washington Post.

EcoHealth on Wednesday said it would appeal federal officials’ decision to suspend funding.

Federal officials and scientists have said they are exploring the possibility that the coronavirus pandemic began from a natural spillover event or a leak from a laboratory. There is no evidence the Wuhan lab or any other research center possessed SARS-CoV-2 before the pandemic, and Daszak was among the most prominent voices in early 2020 arguing that the pandemic was probably sparked by a virus naturally spreading between animals and humans.

Members of Congress on Wednesday applauded the federal government’s decision to suspend funding.

Republicans this month issued a report that said EcoHealth and Daszak had acted with “contempt for the American people” and called for federal officials to cut funding to the organization. Democrats also released a report that concluded EcoHealth “did not cause the Covid-19 pandemic but did engage in questionable professional conduct” and asked whether the organization should continue to receive federal funds.

“EcoHealth’s immediate funding suspension and future debarment is not only a victory for the U.S. taxpayer, but also for American national security and the safety of citizens worldwide,” Rep. Brad Wenstrup (R-Ohio), who chairs the House panel investigating the coronavirus response, said in a statement.

“Every recipient of federal taxpayer funding has an obligation to meet the utmost standards of transparency and accountability to the American public,” Rep. Raul Ruiz (Calif.), the panel’s top Democrat, said in a statement. Ruiz cautioned that the decision to suspend funding to EcoHealth should not be regarded as evidence that the organization was linked to the pandemic.

Wenstrup, Ruiz and other members of their panel grilled Daszak on May 1 about EcoHealth’s work in China and its compliance with federal requirements, including why the organization waited nearly two years to submit a required report about the results of its research and activities leading up to the pandemic.

Investigators have also focused on a 2018 proposal by EcoHealth to experiment on bat coronaviruses, with Daszak suggesting that the work could be performed at the Wuhan Institute of Virology, according to documents first obtained by U.S. Right to Know, a nonprofit that has probed the origins of the virus. Federal officials at the Defense Advanced Research Projects Agency — known by the acronym DARPA — rejected the proposal, and EcoHealth officials have said they did not separately pursue the work detailed in that proposal, although the group collaborated on other research on bat coronaviruses in China.

HHS’s decision to cut funding to EcoHealth, citing the organization’s failures to inform federal officials of activities at the Wuhan lab, should raise doubts about whether EcoHealth’s broader claims are trustworthy, said Rep. Morgan Griffith (R-Va.), who chairs the House Energy and Commerce’s oversight panel and has investigated biosafety issues at laboratories.

“EcoHealth’s mismanagement, along with actions by the Wuhan Institute of Virology, have so clouded the issue, we may never know,” Griffith said in a statement. “But at the end of the day, I believe the evidence points to a lab leak as the most likely source.”

Scientists continue to debate the virus’s origins and call for more research into the work being done in China, including at a virtual roundtable on Tuesday convened by the Brookings Institution, a nonpartisan think tank.

“The strongest argument against a lab origin is that there’s no evidence that SARS-CoV-2 was present in the lab prior to the pandemic,” Jesse Bloom, a virologist at the Fred Hutchinson Cancer Center in Seattle, said at the Brookings event. But Bloom cautioned that “the argument is weakened” if there were active efforts to collect and modify coronaviruses at the Wuhan laboratory, given questions about whether China has been transparent about the lab’s work.

The federal government last year separately cut funding to the Wuhan Institute of Virology, saying the lab had failed to turn over key documents about its research.