research paper literature survey

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Dissertation
• What is a Literature Review? | Guide, Template, & Examples

What is a Literature Review? | Guide, Template, & Examples

Published on 22 February 2022 by Shona McCombes . Revised on 7 June 2022.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research.

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarise sources – it analyses, synthesises, and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

Why write a literature review, examples of literature reviews, step 1: search for relevant literature, step 2: evaluate and select sources, step 3: identify themes, debates and gaps, step 4: outline your literature review’s structure, step 5: write your literature review, frequently asked questions about literature reviews, introduction.

• Quick Run-through
• Step 1 & 2

When you write a dissertation or thesis, you will have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and scholarly context
• Develop a theoretical framework and methodology for your research
• Position yourself in relation to other researchers and theorists
• Show how your dissertation addresses a gap or contributes to a debate

You might also have to write a literature review as a stand-alone assignment. In this case, the purpose is to evaluate the current state of research and demonstrate your knowledge of scholarly debates around a topic.

The content will look slightly different in each case, but the process of conducting a literature review follows the same steps. We’ve written a step-by-step guide that you can follow below.

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Correct my document today

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research objectives and questions .

If you are writing a literature review as a stand-alone assignment, you will have to choose a focus and develop a central question to direct your search. Unlike a dissertation research question, this question has to be answerable without collecting original data. You should be able to answer it based only on a review of existing publications.

Make a list of keywords

Start by creating a list of keywords related to your research topic. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list if you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can use boolean operators to help narrow down your search:

Read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

To identify the most important publications on your topic, take note of recurring citations. If the same authors, books or articles keep appearing in your reading, make sure to seek them out.

You probably won’t be able to read absolutely everything that has been written on the topic – you’ll have to evaluate which sources are most relevant to your questions.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models and methods? Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• How does the publication contribute to your understanding of the topic? What are its key insights and arguments?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible, and make sure you read any landmark studies and major theories in your field of research.

You can find out how many times an article has been cited on Google Scholar – a high citation count means the article has been influential in the field, and should certainly be included in your literature review.

The scope of your review will depend on your topic and discipline: in the sciences you usually only review recent literature, but in the humanities you might take a long historical perspective (for example, to trace how a concept has changed in meaning over time).

Remember that you can use our template to summarise and evaluate sources you’re thinking about using!

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It’s important to keep track of your sources with references to avoid plagiarism . It can be helpful to make an annotated bibliography, where you compile full reference information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

You can use our free APA Reference Generator for quick, correct, consistent citations.

Prevent plagiarism, run a free check.

To begin organising your literature review’s argument and structure, you need to understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly-visual platforms like Instagram and Snapchat – this is a gap that you could address in your own research.

There are various approaches to organising the body of a literature review. You should have a rough idea of your strategy before you start writing.

Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarising sources in order.

Try to analyse patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organise your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text, your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

If you are writing the literature review as part of your dissertation or thesis, reiterate your central problem or research question and give a brief summary of the scholarly context. You can emphasise the timeliness of the topic (“many recent studies have focused on the problem of x”) or highlight a gap in the literature (“while there has been much research on x, few researchers have taken y into consideration”).

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, make sure to follow these tips:

• Summarise and synthesise: give an overview of the main points of each source and combine them into a coherent whole.
• Analyse and interpret: don’t just paraphrase other researchers – add your own interpretations, discussing the significance of findings in relation to the literature as a whole.
• Critically evaluate: mention the strengths and weaknesses of your sources.
• Write in well-structured paragraphs: use transitions and topic sentences to draw connections, comparisons and contrasts.

In the conclusion, you should summarise the key findings you have taken from the literature and emphasise their significance.

If the literature review is part of your dissertation or thesis, reiterate how your research addresses gaps and contributes new knowledge, or discuss how you have drawn on existing theories and methods to build a framework for your research. This can lead directly into your methodology section.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarise yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your  dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2022, June 07). What is a Literature Review? | Guide, Template, & Examples. Scribbr. Retrieved 22 April 2024, from https://www.scribbr.co.uk/thesis-dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, how to write a dissertation proposal | a step-by-step guide, what is a theoretical framework | a step-by-step guide, what is a research methodology | steps & tips.

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: Apr 24, 2024 10:51 AM
• URL: https://libguides.usc.edu/writingguide

• Resources Home 🏠
• Try SciSpace Copilot
• Search research papers
• Add Copilot Extension
• Try AI Detector
• Try Paraphraser
• Try Citation Generator
• April Papers
• June Papers
• July Papers

How To Write A Literature Review - A Complete Guide

Table of Contents

A literature review is much more than just another section in your research paper. It forms the very foundation of your research. It is a formal piece of writing where you analyze the existing theoretical framework, principles, and assumptions and use that as a base to shape your approach to the research question.

Curating and drafting a solid literature review section not only lends more credibility to your research paper but also makes your research tighter and better focused. But, writing literature reviews is a difficult task. It requires extensive reading, plus you have to consider market trends and technological and political changes, which tend to change in the blink of an eye.

Now streamline your literature review process with the help of SciSpace Copilot. With this AI research assistant, you can efficiently synthesize and analyze a vast amount of information, identify key themes and trends, and uncover gaps in the existing research. Get real-time explanations, summaries, and answers to your questions for the paper you're reviewing, making navigating and understanding the complex literature landscape easier.

In this comprehensive guide, we will explore everything from the definition of a literature review, its appropriate length, various types of literature reviews, and how to write one.

What is a literature review?

A literature review is a collation of survey, research, critical evaluation, and assessment of the existing literature in a preferred domain.

Eminent researcher and academic Arlene Fink, in her book Conducting Research Literature Reviews , defines it as the following:

“A literature review surveys books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated.

Literature reviews are designed to provide an overview of sources you have explored while researching a particular topic, and to demonstrate to your readers how your research fits within a larger field of study.”

Simply put, a literature review can be defined as a critical discussion of relevant pre-existing research around your research question and carving out a definitive place for your study in the existing body of knowledge. Literature reviews can be presented in multiple ways: a section of an article, the whole research paper itself, or a chapter of your thesis.

A literature review does function as a summary of sources, but it also allows you to analyze further, interpret, and examine the stated theories, methods, viewpoints, and, of course, the gaps in the existing content.

As an author, you can discuss and interpret the research question and its various aspects and debate your adopted methods to support the claim.

What is the purpose of a literature review?

A literature review is meant to help your readers understand the relevance of your research question and where it fits within the existing body of knowledge. As a researcher, you should use it to set the context, build your argument, and establish the need for your study.

What is the importance of a literature review?

The literature review is a critical part of research papers because it helps you:

• Gain an in-depth understanding of your research question and the surrounding area
• Convey that you have a thorough understanding of your research area and are up-to-date with the latest changes and advancements
• Establish how your research is connected or builds on the existing body of knowledge and how it could contribute to further research
• Elaborate on the validity and suitability of your theoretical framework and research methodology
• Identify and highlight gaps and shortcomings in the existing body of knowledge and how things need to change
• Convey to readers how your study is different or how it contributes to the research area

How long should a literature review be?

Ideally, the literature review should take up 15%-40% of the total length of your manuscript. So, if you have a 10,000-word research paper, the minimum word count could be 1500.

Your literature review format depends heavily on the kind of manuscript you are writing — an entire chapter in case of doctoral theses, a part of the introductory section in a research article, to a full-fledged review article that examines the previously published research on a topic.

Another determining factor is the type of research you are doing. The literature review section tends to be longer for secondary research projects than primary research projects.

What are the different types of literature reviews?

All literature reviews are not the same. There are a variety of possible approaches that you can take. It all depends on the type of research you are pursuing.

Here are the different types of literature reviews:

Argumentative review

It is called an argumentative review when you carefully present literature that only supports or counters a specific argument or premise to establish a viewpoint.

Integrative review

It is a type of literature review focused on building a comprehensive understanding of a topic by combining available theoretical frameworks and empirical evidence.

Methodological review

This approach delves into the ''how'' and the ''what" of the research question —  you cannot look at the outcome in isolation; you should also review the methodology used.

Systematic review

This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research and collect, report, and analyze data from the studies included in the review.

Meta-analysis review

Meta-analysis uses statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects than those derived from the individual studies included within a review.

Historical review

Historical literature reviews focus on examining research throughout a period, often starting with the first time an issue, concept, theory, or phenomenon emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and identify future research's likely directions.

Theoretical Review

This form aims to examine the corpus of theory accumulated regarding an issue, concept, theory, and phenomenon. The theoretical literature review helps to establish what theories exist, the relationships between them, the degree the existing approaches have been investigated, and to develop new hypotheses to be tested.

Scoping Review

The Scoping Review is often used at the beginning of an article, dissertation, or research proposal. It is conducted before the research to highlight gaps in the existing body of knowledge and explains why the project should be greenlit.

State-of-the-Art Review

The State-of-the-Art review is conducted periodically, focusing on the most recent research. It describes what is currently known, understood, or agreed upon regarding the research topic and highlights where there are still disagreements.

Can you use the first person in a literature review?

When writing literature reviews, you should avoid the usage of first-person pronouns. It means that instead of "I argue that" or "we argue that," the appropriate expression would be "this research paper argues that."

Do you need an abstract for a literature review?

Ideally, yes. It is always good to have a condensed summary that is self-contained and independent of the rest of your review. As for how to draft one, you can follow the same fundamental idea when preparing an abstract for a literature review. It should also include:

• The research topic and your motivation behind selecting it
• A one-sentence thesis statement
• An explanation of the kinds of literature featured in the review
• Summary of what you've learned
• Conclusions you drew from the literature you reviewed
• Potential implications and future scope for research

Here's an example of the abstract of a literature review

Is a literature review written in the past tense?

Yes, the literature review should ideally be written in the past tense. You should not use the present or future tense when writing one. The exceptions are when you have statements describing events that happened earlier than the literature you are reviewing or events that are currently occurring; then, you can use the past perfect or present perfect tenses.

How many sources for a literature review?

There are multiple approaches to deciding how many sources to include in a literature review section. The first approach would be to look level you are at as a researcher. For instance, a doctoral thesis might need 60+ sources. In contrast, you might only need to refer to 5-15 sources at the undergraduate level.

The second approach is based on the kind of literature review you are doing — whether it is merely a chapter of your paper or if it is a self-contained paper in itself. When it is just a chapter, sources should equal the total number of pages in your article's body. In the second scenario, you need at least three times as many sources as there are pages in your work.

Quick tips on how to write a literature review

To know how to write a literature review, you must clearly understand its impact and role in establishing your work as substantive research material.

You need to follow the below-mentioned steps, to write a literature review:

• Outline the purpose behind the literature review
• Search relevant literature
• Examine and assess the relevant resources
• Discover connections by drawing deep insights from the resources
• Structure planning to write a good literature review

1. Outline and identify the purpose of  a literature review

As a first step on how to write a literature review, you must know what the research question or topic is and what shape you want your literature review to take. Ensure you understand the research topic inside out, or else seek clarifications. You must be able to the answer below questions before you start:

• How many sources do I need to include?
• What kind of sources should I analyze?
• How much should I critically evaluate each source?
• Should I summarize, synthesize or offer a critique of the sources?
• Do I need to include any background information or definitions?

Additionally, you should know that the narrower your research topic is, the swifter it will be for you to restrict the number of sources to be analyzed.

2. Search relevant literature

Dig deeper into search engines to discover what has already been published around your chosen topic. Make sure you thoroughly go through appropriate reference sources like books, reports, journal articles, government docs, and web-based resources.

You must prepare a list of keywords and their different variations. You can start your search from any library’s catalog, provided you are an active member of that institution. The exact keywords can be extended to widen your research over other databases and academic search engines like:

• Google Scholar
• Microsoft Academic
• Science.gov

Besides, it is not advisable to go through every resource word by word. Alternatively, what you can do is you can start by reading the abstract and then decide whether that source is relevant to your research or not.

Additionally, you must spend surplus time assessing the quality and relevance of resources. It would help if you tried preparing a list of citations to ensure that there lies no repetition of authors, publications, or articles in the literature review.

3. Examine and assess the sources

It is nearly impossible for you to go through every detail in the research article. So rather than trying to fetch every detail, you have to analyze and decide which research sources resemble closest and appear relevant to your chosen domain.

While analyzing the sources, you should look to find out answers to questions like:

• What question or problem has the author been describing and debating?
• What is the definition of critical aspects?
• How well the theories, approach, and methodology have been explained?
• Whether the research theory used some conventional or new innovative approach?
• How relevant are the key findings of the work?
• In what ways does it relate to other sources on the same topic?
• What challenges does this research paper pose to the existing theory
• What are the possible contributions or benefits it adds to the subject domain?

Be always mindful that you refer only to credible and authentic resources. It would be best if you always take references from different publications to validate your theory.

Always keep track of important information or data you can present in your literature review right from the beginning. It will help steer your path from any threats of plagiarism and also make it easier to curate an annotated bibliography or reference section.

4. Discover connections

At this stage, you must start deciding on the argument and structure of your literature review. To accomplish this, you must discover and identify the relations and connections between various resources while drafting your abstract.

A few aspects that you should be aware of while writing a literature review include:

• Rise to prominence: Theories and methods that have gained reputation and supporters over time.
• Constant scrutiny: Concepts or theories that repeatedly went under examination.
• Contradictions and conflicts: Theories, both the supporting and the contradictory ones, for the research topic.
• Knowledge gaps: What exactly does it fail to address, and how to bridge them with further research?
• Influential resources: Significant research projects available that have been upheld as milestones or perhaps, something that can modify the current trends

Once you join the dots between various past research works, it will be easier for you to draw a conclusion and identify your contribution to the existing knowledge base.

5. Structure planning to write a good literature review

There exist different ways towards planning and executing the structure of a literature review. The format of a literature review varies and depends upon the length of the research.

Like any other research paper, the literature review format must contain three sections: introduction, body, and conclusion. The goals and objectives of the research question determine what goes inside these three sections.

Nevertheless, a good literature review can be structured according to the chronological, thematic, methodological, or theoretical framework approach.

Literature review samples

1. Standalone

2. As a section of a research paper

How SciSpace Discover makes literature review a breeze?

SciSpace Discover is a one-stop solution to do an effective literature search and get barrier-free access to scientific knowledge. It is an excellent repository where you can find millions of only peer-reviewed articles and full-text PDF files. Here’s more on how you can use it:

Find the right information

Find what you want quickly and easily with comprehensive search filters that let you narrow down papers according to PDF availability, year of publishing, document type, and affiliated institution. Moreover, you can sort the results based on the publishing date, citation count, and relevance.

Assess credibility of papers quickly

When doing the literature review, it is critical to establish the quality of your sources. They form the foundation of your research. SciSpace Discover helps you assess the quality of a source by providing an overview of its references, citations, and performance metrics.

Get the complete picture in no time

SciSpace Discover’s personalized suggestion engine helps you stay on course and get the complete picture of the topic from one place. Every time you visit an article page, it provides you links to related papers. Besides that, it helps you understand what’s trending, who are the top authors, and who are the leading publishers on a topic.

Make referring sources super easy

To ensure you don't lose track of your sources, you must start noting down your references when doing the literature review. SciSpace Discover makes this step effortless. Click the 'cite' button on an article page, and you will receive preloaded citation text in multiple styles — all you've to do is copy-paste it into your manuscript.

Final tips on how to write a literature review

A massive chunk of time and effort is required to write a good literature review. But, if you go about it systematically, you'll be able to save a ton of time and build a solid foundation for your research.

We hope this guide has helped you answer several key questions you have about writing literature reviews.

Would you like to explore SciSpace Discover and kick off your literature search right away? You can get started here .

Frequently Asked Questions (FAQs)

1. how to start a literature review.

• What questions do you want to answer?

• What sources do you need to answer these questions?

• What information do these sources contain?

• How can you use this information to answer your questions?

2. What to include in a literature review?

• A brief background of the problem or issue

• What has previously been done to address the problem or issue

• A description of what you will do in your project

• How this study will contribute to research on the subject

3. Why literature review is important?

The literature review is an important part of any research project because it allows the writer to look at previous studies on a topic and determine existing gaps in the literature, as well as what has already been done. It will also help them to choose the most appropriate method for their own study.

4. How to cite a literature review in APA format?

To cite a literature review in APA style, you need to provide the author's name, the title of the article, and the year of publication. For example: Patel, A. B., & Stokes, G. S. (2012). The relationship between personality and intelligence: A meta-analysis of longitudinal research. Personality and Individual Differences, 53(1), 16-21

5. What are the components of a literature review?

• A brief introduction to the topic, including its background and context. The introduction should also include a rationale for why the study is being conducted and what it will accomplish.

• A description of the methodologies used in the study. This can include information about data collection methods, sample size, and statistical analyses.

• A presentation of the findings in an organized format that helps readers follow along with the author's conclusions.

6. What are common errors in writing literature review?

• Not spending enough time to critically evaluate the relevance of resources, observations and conclusions.

• Totally relying on secondary data while ignoring primary data.

• Letting your personal bias seep into your interpretation of existing literature.

• No detailed explanation of the procedure to discover and identify an appropriate literature review.

7. What are the 5 C's of writing literature review?

• Cite - the sources you utilized and referenced in your research.

• Compare - existing arguments, hypotheses, methodologies, and conclusions found in the knowledge base.

• Contrast - the arguments, topics, methodologies, approaches, and disputes that may be found in the literature.

• Critique - the literature and describe the ideas and opinions you find more convincing and why.

• Connect - the various studies you reviewed in your research.

8. How many sources should a literature review have?

When it is just a chapter, sources should equal the total number of pages in your article's body. if it is a self-contained paper in itself, you need at least three times as many sources as there are pages in your work.

9. Can literature review have diagrams?

• To represent an abstract idea or concept

• To explain the steps of a process or procedure

• To help readers understand the relationships between different concepts

10. How old should sources be in a literature review?

Sources for a literature review should be as current as possible or not older than ten years. The only exception to this rule is if you are reviewing a historical topic and need to use older sources.

11. What are the types of literature review?

• Argumentative review

• Integrative review

• Methodological review

• Systematic review

• Meta-analysis review

• Historical review

• Theoretical review

• Scoping review

• State-of-the-Art review

12. Is a literature review mandatory?

Yes. Literature review is a mandatory part of any research project. It is a critical step in the process that allows you to establish the scope of your research, and provide a background for the rest of your work.

But before you go,

• Six Online Tools for Easy Literature Review
• Evaluating literature review: systematic vs. scoping reviews
• Systematic Approaches to a Successful Literature Review
• Writing Integrative Literature Reviews: Guidelines and Examples

You might also like

Consensus GPT vs. SciSpace GPT: Choose the Best GPT for Research

Literature Review and Theoretical Framework: Understanding the Differences

Types of Essays in Academic Writing - Quick Guide (2024)

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Literature Reviews

What this handout is about.

This handout will explain what literature reviews are and offer insights into the form and construction of literature reviews in the humanities, social sciences, and sciences.

Introduction

OK. You’ve got to write a literature review. You dust off a novel and a book of poetry, settle down in your chair, and get ready to issue a “thumbs up” or “thumbs down” as you leaf through the pages. “Literature review” done. Right?

Wrong! The “literature” of a literature review refers to any collection of materials on a topic, not necessarily the great literary texts of the world. “Literature” could be anything from a set of government pamphlets on British colonial methods in Africa to scholarly articles on the treatment of a torn ACL. And a review does not necessarily mean that your reader wants you to give your personal opinion on whether or not you liked these sources.

What is a literature review, then?

A literature review discusses published information in a particular subject area, and sometimes information in a particular subject area within a certain time period.

A literature review can be just a simple summary of the sources, but it usually has an organizational pattern and combines both summary and synthesis. A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information. It might give a new interpretation of old material or combine new with old interpretations. Or it might trace the intellectual progression of the field, including major debates. And depending on the situation, the literature review may evaluate the sources and advise the reader on the most pertinent or relevant.

But how is a literature review different from an academic research paper?

The main focus of an academic research paper is to develop a new argument, and a research paper is likely to contain a literature review as one of its parts. In a research paper, you use the literature as a foundation and as support for a new insight that you contribute. The focus of a literature review, however, is to summarize and synthesize the arguments and ideas of others without adding new contributions.

Why do we write literature reviews?

Literature reviews provide you with a handy guide to a particular topic. If you have limited time to conduct research, literature reviews can give you an overview or act as a stepping stone. For professionals, they are useful reports that keep them up to date with what is current in the field. For scholars, the depth and breadth of the literature review emphasizes the credibility of the writer in his or her field. Literature reviews also provide a solid background for a research paper’s investigation. Comprehensive knowledge of the literature of the field is essential to most research papers.

Who writes these things, anyway?

Literature reviews are written occasionally in the humanities, but mostly in the sciences and social sciences; in experiment and lab reports, they constitute a section of the paper. Sometimes a literature review is written as a paper in itself.

Let’s get to it! What should I do before writing the literature review?

If your assignment is not very specific, seek clarification from your instructor:

• Roughly how many sources should you include?
• What types of sources (books, journal articles, websites)?
• Should you summarize, synthesize, or critique your sources by discussing a common theme or issue?
• Should you evaluate your sources?
• Should you provide subheadings and other background information, such as definitions and/or a history?

Find models

Look for other literature reviews in your area of interest or in the discipline and read them to get a sense of the types of themes you might want to look for in your own research or ways to organize your final review. You can simply put the word “review” in your search engine along with your other topic terms to find articles of this type on the Internet or in an electronic database. The bibliography or reference section of sources you’ve already read are also excellent entry points into your own research.

Narrow your topic

There are hundreds or even thousands of articles and books on most areas of study. The narrower your topic, the easier it will be to limit the number of sources you need to read in order to get a good survey of the material. Your instructor will probably not expect you to read everything that’s out there on the topic, but you’ll make your job easier if you first limit your scope.

Keep in mind that UNC Libraries have research guides and to databases relevant to many fields of study. You can reach out to the subject librarian for a consultation: https://library.unc.edu/support/consultations/ .

And don’t forget to tap into your professor’s (or other professors’) knowledge in the field. Ask your professor questions such as: “If you had to read only one book from the 90’s on topic X, what would it be?” Questions such as this help you to find and determine quickly the most seminal pieces in the field.

Consider whether your sources are current

Some disciplines require that you use information that is as current as possible. In the sciences, for instance, treatments for medical problems are constantly changing according to the latest studies. Information even two years old could be obsolete. However, if you are writing a review in the humanities, history, or social sciences, a survey of the history of the literature may be what is needed, because what is important is how perspectives have changed through the years or within a certain time period. Try sorting through some other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to consider what is currently of interest to scholars in this field and what is not.

Strategies for writing the literature review

Find a focus.

A literature review, like a term paper, is usually organized around ideas, not the sources themselves as an annotated bibliography would be organized. This means that you will not just simply list your sources and go into detail about each one of them, one at a time. No. As you read widely but selectively in your topic area, consider instead what themes or issues connect your sources together. Do they present one or different solutions? Is there an aspect of the field that is missing? How well do they present the material and do they portray it according to an appropriate theory? Do they reveal a trend in the field? A raging debate? Pick one of these themes to focus the organization of your review.

Convey it to your reader

A literature review may not have a traditional thesis statement (one that makes an argument), but you do need to tell readers what to expect. Try writing a simple statement that lets the reader know what is your main organizing principle. Here are a couple of examples:

The current trend in treatment for congestive heart failure combines surgery and medicine. More and more cultural studies scholars are accepting popular media as a subject worthy of academic consideration.

Consider organization

You’ve got a focus, and you’ve stated it clearly and directly. Now what is the most effective way of presenting the information? What are the most important topics, subtopics, etc., that your review needs to include? And in what order should you present them? Develop an organization for your review at both a global and local level:

First, cover the basic categories

Just like most academic papers, literature reviews also must contain at least three basic elements: an introduction or background information section; the body of the review containing the discussion of sources; and, finally, a conclusion and/or recommendations section to end the paper. The following provides a brief description of the content of each:

• Introduction: Gives a quick idea of the topic of the literature review, such as the central theme or organizational pattern.
• Body: Contains your discussion of sources and is organized either chronologically, thematically, or methodologically (see below for more information on each).
• Conclusions/Recommendations: Discuss what you have drawn from reviewing literature so far. Where might the discussion proceed?

Organizing the body

Once you have the basic categories in place, then you must consider how you will present the sources themselves within the body of your paper. Create an organizational method to focus this section even further.

To help you come up with an overall organizational framework for your review, consider the following scenario:

You’ve decided to focus your literature review on materials dealing with sperm whales. This is because you’ve just finished reading Moby Dick, and you wonder if that whale’s portrayal is really real. You start with some articles about the physiology of sperm whales in biology journals written in the 1980’s. But these articles refer to some British biological studies performed on whales in the early 18th century. So you check those out. Then you look up a book written in 1968 with information on how sperm whales have been portrayed in other forms of art, such as in Alaskan poetry, in French painting, or on whale bone, as the whale hunters in the late 19th century used to do. This makes you wonder about American whaling methods during the time portrayed in Moby Dick, so you find some academic articles published in the last five years on how accurately Herman Melville portrayed the whaling scene in his novel.

Now consider some typical ways of organizing the sources into a review:

• Chronological: If your review follows the chronological method, you could write about the materials above according to when they were published. For instance, first you would talk about the British biological studies of the 18th century, then about Moby Dick, published in 1851, then the book on sperm whales in other art (1968), and finally the biology articles (1980s) and the recent articles on American whaling of the 19th century. But there is relatively no continuity among subjects here. And notice that even though the sources on sperm whales in other art and on American whaling are written recently, they are about other subjects/objects that were created much earlier. Thus, the review loses its chronological focus.
• By publication: Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on biological studies of sperm whales if the progression revealed a change in dissection practices of the researchers who wrote and/or conducted the studies.
• By trend: A better way to organize the above sources chronologically is to examine the sources under another trend, such as the history of whaling. Then your review would have subsections according to eras within this period. For instance, the review might examine whaling from pre-1600-1699, 1700-1799, and 1800-1899. Under this method, you would combine the recent studies on American whaling in the 19th century with Moby Dick itself in the 1800-1899 category, even though the authors wrote a century apart.
• Thematic: Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For instance, the sperm whale review could focus on the development of the harpoon for whale hunting. While the study focuses on one topic, harpoon technology, it will still be organized chronologically. The only difference here between a “chronological” and a “thematic” approach is what is emphasized the most: the development of the harpoon or the harpoon technology.But more authentic thematic reviews tend to break away from chronological order. For instance, a thematic review of material on sperm whales might examine how they are portrayed as “evil” in cultural documents. The subsections might include how they are personified, how their proportions are exaggerated, and their behaviors misunderstood. A review organized in this manner would shift between time periods within each section according to the point made.
• Methodological: A methodological approach differs from the two above in that the focusing factor usually does not have to do with the content of the material. Instead, it focuses on the “methods” of the researcher or writer. For the sperm whale project, one methodological approach would be to look at cultural differences between the portrayal of whales in American, British, and French art work. Or the review might focus on the economic impact of whaling on a community. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed. Once you’ve decided on the organizational method for the body of the review, the sections you need to include in the paper should be easy to figure out. They should arise out of your organizational strategy. In other words, a chronological review would have subsections for each vital time period. A thematic review would have subtopics based upon factors that relate to the theme or issue.

Sometimes, though, you might need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. Put in only what is necessary. Here are a few other sections you might want to consider:

• Current Situation: Information necessary to understand the topic or focus of the literature review.
• History: The chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Methods and/or Standards: The criteria you used to select the sources in your literature review or the way in which you present your information. For instance, you might explain that your review includes only peer-reviewed articles and journals.

Questions for Further Research: What questions about the field has the review sparked? How will you further your research as a result of the review?

Begin composing

Once you’ve settled on a general pattern of organization, you’re ready to write each section. There are a few guidelines you should follow during the writing stage as well. Here is a sample paragraph from a literature review about sexism and language to illuminate the following discussion:

However, other studies have shown that even gender-neutral antecedents are more likely to produce masculine images than feminine ones (Gastil, 1990). Hamilton (1988) asked students to complete sentences that required them to fill in pronouns that agreed with gender-neutral antecedents such as “writer,” “pedestrian,” and “persons.” The students were asked to describe any image they had when writing the sentence. Hamilton found that people imagined 3.3 men to each woman in the masculine “generic” condition and 1.5 men per woman in the unbiased condition. Thus, while ambient sexism accounted for some of the masculine bias, sexist language amplified the effect. (Source: Erika Falk and Jordan Mills, “Why Sexist Language Affects Persuasion: The Role of Homophily, Intended Audience, and Offense,” Women and Language19:2).

Use evidence

In the example above, the writers refer to several other sources when making their point. A literature review in this sense is just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence to show that what you are saying is valid.

Be selective

Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the review’s focus, whether it is thematic, methodological, or chronological.

Use quotes sparingly

Falk and Mills do not use any direct quotes. That is because the survey nature of the literature review does not allow for in-depth discussion or detailed quotes from the text. Some short quotes here and there are okay, though, if you want to emphasize a point, or if what the author said just cannot be rewritten in your own words. Notice that Falk and Mills do quote certain terms that were coined by the author, not common knowledge, or taken directly from the study. But if you find yourself wanting to put in more quotes, check with your instructor.

Summarize and synthesize

Remember to summarize and synthesize your sources within each paragraph as well as throughout the review. The authors here recapitulate important features of Hamilton’s study, but then synthesize it by rephrasing the study’s significance and relating it to their own work.

Keep your own voice

While the literature review presents others’ ideas, your voice (the writer’s) should remain front and center. Notice that Falk and Mills weave references to other sources into their own text, but they still maintain their own voice by starting and ending the paragraph with their own ideas and their own words. The sources support what Falk and Mills are saying.

Use caution when paraphrasing

When paraphrasing a source that is not your own, be sure to represent the author’s information or opinions accurately and in your own words. In the preceding example, Falk and Mills either directly refer in the text to the author of their source, such as Hamilton, or they provide ample notation in the text when the ideas they are mentioning are not their own, for example, Gastil’s. For more information, please see our handout on plagiarism .

Revise, revise, revise

Draft in hand? Now you’re ready to revise. Spending a lot of time revising is a wise idea, because your main objective is to present the material, not the argument. So check over your review again to make sure it follows the assignment and/or your outline. Then, just as you would for most other academic forms of writing, rewrite or rework the language of your review so that you’ve presented your information in the most concise manner possible. Be sure to use terminology familiar to your audience; get rid of unnecessary jargon or slang. Finally, double check that you’ve documented your sources and formatted the review appropriately for your discipline. For tips on the revising and editing process, see our handout on revising drafts .

Works consulted

We consulted these works while writing this handout. This is not a comprehensive list of resources on the handout’s topic, and we encourage you to do your own research to find additional publications. Please do not use this list as a model for the format of your own reference list, as it may not match the citation style you are using. For guidance on formatting citations, please see the UNC Libraries citation tutorial . We revise these tips periodically and welcome feedback.

Anson, Chris M., and Robert A. Schwegler. 2010. The Longman Handbook for Writers and Readers , 6th ed. New York: Longman.

Jones, Robert, Patrick Bizzaro, and Cynthia Selfe. 1997. The Harcourt Brace Guide to Writing in the Disciplines . New York: Harcourt Brace.

Lamb, Sandra E. 1998. How to Write It: A Complete Guide to Everything You’ll Ever Write . Berkeley: Ten Speed Press.

Rosen, Leonard J., and Laurence Behrens. 2003. The Allyn & Bacon Handbook , 5th ed. New York: Longman.

Troyka, Lynn Quittman, and Doug Hesse. 2016. Simon and Schuster Handbook for Writers , 11th ed. London: Pearson.

You may reproduce it for non-commercial use if you use the entire handout and attribute the source: The Writing Center, University of North Carolina at Chapel Hill

Make a Gift

Research Methods

• Getting Started
• Literature Review Research
• Research Design
• Research Design By Discipline
• SAGE Research Methods
• Teaching with SAGE Research Methods

Literature Review

• What is a Literature Review?
• What is NOT a Literature Review?
• Purposes of a Literature Review
• Types of Literature Reviews
• Literature Reviews vs. Systematic Reviews
• Systematic vs. Meta-Analysis

Literature Review  is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

Also, we can define a literature review as the collected body of scholarly works related to a topic:

• Summarizes and analyzes previous research relevant to a topic
• Includes scholarly books and articles published in academic journals
• Can be an specific scholarly paper or a section in a research paper

The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic

• Help gather ideas or information
• Keep up to date in current trends and findings
• Help develop new questions

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Helps focus your own research questions or problems
• Discovers relationships between research studies/ideas.
• Suggests unexplored ideas or populations
• Identifies major themes, concepts, and researchers on a topic.
• Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
• Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
• Indicates potential directions for future research.

All content in this section is from Literature Review Research from Old Dominion University 

Keep in mind the following, a literature review is NOT:

Not an essay 

Not an annotated bibliography  in which you summarize each article that you have reviewed.  A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.

Not a research paper   where you select resources to support one side of an issue versus another.  A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.

A literature review serves several purposes. For example, it

• provides thorough knowledge of previous studies; introduces seminal works.
• helps focus one’s own research topic.
• identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
• suggests previously unused or underused methodologies, designs, quantitative and qualitative strategies.
• identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
• helps the researcher avoid repetition of earlier research.
• suggests unexplored populations.
• determines whether past studies agree or disagree; identifies controversy in the literature.
• tests assumptions; may help counter preconceived ideas and remove unconscious bias.

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

Robinson, P. and Lowe, J. (2015),  Literature reviews vs systematic reviews.  Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393

What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California

Systematic review or meta-analysis?

A  systematic review  answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.

A  meta-analysis  is the use of statistical methods to summarize the results of these studies.

Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:

• clearly stated objectives with pre-defined eligibility criteria for studies
• explicit, reproducible methodology
• a systematic search that attempts to identify all studies
• assessment of the validity of the findings of the included studies (e.g. risk of bias)
• systematic presentation, and synthesis, of the characteristics and findings of the included studies

Not all systematic reviews contain meta-analysis. 

Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.  More information on meta-analyses can be found in  Cochrane Handbook, Chapter 9 .

A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies.  It is a systematic review that uses quantitative methods to synthesize and summarize the results.

An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings.  Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted.  In that case, an integrative review is an appropriate strategy. 

Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.

• << Previous: Getting Started
• Next: Research Design >>
• Last Updated: Aug 21, 2023 4:07 PM
• URL: https://guides.lib.udel.edu/researchmethods

• U.S. Locations
• UMGC Europe
• Learn Online
• Find Answers
• 855-655-8682
• Current Students

Online Guide to Writing and Research

The research process, explore more of umgc.

• Online Guide to Writing

Planning and Writing a Research Paper

Survey the Literature

After you have decided on a topic that you want to learn more about, you need to review the literature that has been written about your topic.  This is called performing a literature review.  

If you are a UMGC student, you could use the search box in the middle of the library homepage , and see what you find out about your topic.  If you are not a UMGC student, then you can use your local library catalog, or you could do an internet search with your topic terms in Google and see what you might find.  This is a learning and discovering time.  

Steps for Surveying the Literature

Review the literature:  Take a few hours or an evening to investigate the library for journals, scholarly books, and publications from credible sources to give you a general sense of the topic.  When you are researching and finding resources, you are answering the question of how to review literature. 

Make sure to keep a list of the resources that you find useful.  This preliminary research will prepare you to frame your research question.

Weed out sources:  This is where you determine if the sources that you found will help your writing or if you need to “weed” out or remove any that are not helpful to you.   

Refine your topic:  This is where you determine if you need to reframe your topic to utilize all of the sources that you found when digging in to your topic.  In this part of the process, you are answering the question of why we do literature reviews in the first place.  When you refine and make your topic more detailed and concise, you will have an easier time when sitting down and writing your paper.  

Key Takeaways

• This process, like writing in general, is recursive. 
• You may need to survey the literature a few more times as you begin writing your paper.

Mailing Address: 3501 University Blvd. East, Adelphi, MD 20783 This work is licensed under a  Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License . © 2022 UMGC. All links to external sites were verified at the time of publication. UMGC is not responsible for the validity or integrity of information located at external sites.

Table of Contents: Online Guide to Writing

Chapter 1: College Writing

How Does College Writing Differ from Workplace Writing?

What Is College Writing?

Why So Much Emphasis on Writing?

Chapter 2: The Writing Process

Doing Exploratory Research

Getting from Notes to Your Draft

Introduction

Prewriting - Techniques to Get Started - Mining Your Intuition

Prewriting: Targeting Your Audience

Prewriting: Techniques to Get Started

Prewriting: Understanding Your Assignment

Rewriting: Being Your Own Critic

Rewriting: Creating a Revision Strategy

Rewriting: Getting Feedback

Rewriting: The Final Draft

Techniques to Get Started - Outlining

Techniques to Get Started - Using Systematic Techniques

Thesis Statement and Controlling Idea

Writing: Getting from Notes to Your Draft - Freewriting

Writing: Getting from Notes to Your Draft - Summarizing Your Ideas

Writing: Outlining What You Will Write

Chapter 3: Thinking Strategies

A Word About Style, Voice, and Tone

A Word About Style, Voice, and Tone: Style Through Vocabulary and Diction

Critical Strategies and Writing

Critical Strategies and Writing: Analysis

Critical Strategies and Writing: Evaluation

Critical Strategies and Writing: Persuasion

Critical Strategies and Writing: Synthesis

Developing a Paper Using Strategies

Kinds of Assignments You Will Write

Patterns for Presenting Information

Patterns for Presenting Information: Critiques

Patterns for Presenting Information: Discussing Raw Data

Patterns for Presenting Information: General-to-Specific Pattern

Patterns for Presenting Information: Problem-Cause-Solution Pattern

Patterns for Presenting Information: Specific-to-General Pattern

Patterns for Presenting Information: Summaries and Abstracts

Supporting with Research and Examples

Writing Essay Examinations

Writing Essay Examinations: Make Your Answer Relevant and Complete

Writing Essay Examinations: Organize Thinking Before Writing

Writing Essay Examinations: Read and Understand the Question

Chapter 4: The Research Process

Planning and Writing a Research Paper: Ask a Research Question

Planning and Writing a Research Paper: Cite Sources

Planning and Writing a Research Paper: Collect Evidence

Planning and Writing a Research Paper: Decide Your Point of View, or Role, for Your Research

Planning and Writing a Research Paper: Draw Conclusions

Planning and Writing a Research Paper: Find a Topic and Get an Overview

Planning and Writing a Research Paper: Manage Your Resources

Planning and Writing a Research Paper: Outline

Planning and Writing a Research Paper: Survey the Literature

Planning and Writing a Research Paper: Work Your Sources into Your Research Writing

Research Resources: Where Are Research Resources Found? - Human Resources

Research Resources: What Are Research Resources?

Research Resources: Where Are Research Resources Found?

Research Resources: Where Are Research Resources Found? - Electronic Resources

Research Resources: Where Are Research Resources Found? - Print Resources

Structuring the Research Paper: Formal Research Structure

Structuring the Research Paper: Informal Research Structure

The Nature of Research

The Research Assignment: How Should Research Sources Be Evaluated?

The Research Assignment: When Is Research Needed?

The Research Assignment: Why Perform Research?

Chapter 5: Academic Integrity

Academic Integrity

Giving Credit to Sources

Giving Credit to Sources: Copyright Laws

Giving Credit to Sources: Documentation

Giving Credit to Sources: Style Guides

Integrating Sources

Practicing Academic Integrity

Practicing Academic Integrity: Keeping Accurate Records

Practicing Academic Integrity: Managing Source Material

Practicing Academic Integrity: Managing Source Material - Paraphrasing Your Source

Practicing Academic Integrity: Managing Source Material - Quoting Your Source

Practicing Academic Integrity: Managing Source Material - Summarizing Your Sources

Types of Documentation

Types of Documentation: Bibliographies and Source Lists

Types of Documentation: Citing World Wide Web Sources

Types of Documentation: In-Text or Parenthetical Citations

Types of Documentation: In-Text or Parenthetical Citations - APA Style

Types of Documentation: In-Text or Parenthetical Citations - CSE/CBE Style

Types of Documentation: In-Text or Parenthetical Citations - Chicago Style

Types of Documentation: In-Text or Parenthetical Citations - MLA Style

Types of Documentation: Note Citations

Chapter 6: Using Library Resources

Finding Library Resources

Chapter 7: Assessing Your Writing

How Is Writing Graded?

How Is Writing Graded?: A General Assessment Tool

The Draft Stage

The Draft Stage: The First Draft

The Draft Stage: The Revision Process and the Final Draft

The Draft Stage: Using Feedback

The Research Stage

Using Assessment to Improve Your Writing

Chapter 8: Other Frequently Assigned Papers

Reviews and Reaction Papers: Article and Book Reviews

Reviews and Reaction Papers: Reaction Papers

Writing Arguments

Writing Arguments: Adapting the Argument Structure

Writing Arguments: Purposes of Argument

Writing Arguments: References to Consult for Writing Arguments

Writing Arguments: Steps to Writing an Argument - Anticipate Active Opposition

Writing Arguments: Steps to Writing an Argument - Determine Your Organization

Writing Arguments: Steps to Writing an Argument - Develop Your Argument

Writing Arguments: Steps to Writing an Argument - Introduce Your Argument

Writing Arguments: Steps to Writing an Argument - State Your Thesis or Proposition

Writing Arguments: Steps to Writing an Argument - Write Your Conclusion

Writing Arguments: Types of Argument

Appendix A: Books to Help Improve Your Writing

Dictionaries

General Style Manuals

Researching on the Internet

Special Style Manuals

Writing Handbooks

Appendix B: Collaborative Writing and Peer Reviewing

Collaborative Writing: Assignments to Accompany the Group Project

Collaborative Writing: Informal Progress Report

Collaborative Writing: Issues to Resolve

Collaborative Writing: Methodology

Collaborative Writing: Peer Evaluation

Collaborative Writing: Tasks of Collaborative Writing Group Members

Collaborative Writing: Writing Plan

General Introduction

Peer Reviewing

Appendix C: Developing an Improvement Plan

Working with Your Instructor’s Comments and Grades

Appendix D: Writing Plan and Project Schedule

Devising a Writing Project Plan and Schedule

Reviewing Your Plan with Others

By using our website you agree to our use of cookies. Learn more about how we use cookies by reading our  Privacy Policy .

Faculty and researchers : We want to hear from you! We are launching a survey to learn more about your library collection needs for teaching, learning, and research. If you would like to participate, please complete the survey by May 17, 2024. Thank you for your participation!

• University of Massachusetts Lowell
• University Libraries

Survey Research: Design and Presentation

• Literature Review: Definition and Context
• Introduction to Survey Research Design
• Planning a Thesis Proposal
• Slides, Articles
• Evaluating Survey Results
• Related Library Databases

Literature Review for Grad Students in Education

• Library Guide: Literature Review

Introduction to Literature Review

If you cannot access the above video, you can watch it here

What is a Literature Review

  The purpose of an academic research paper is to express and document an original idea. Literature Review is one part of that process of writing a research paper. In a research paper, you use the literature as a starting point, a building block and as evidence of a new insight. The goal of the literature review is only to summarize and synthesize the arguments and ideas of others. You should not present your original idea.

The reading that you do as part of a literature review will answer one of two questions:

“What do we know about the subject of our study?” “Based on what we know, what conclusions can we draw about the research question?”

Notice that the conclusions to be drawn are about the research question , as opposed to a novel theory. 

The types of conclusions about your research question that you want to discover are: ❖ gaps in the knowledge on a subject area ❖ questions about your topic that remain unanswered ❖ areas of disagreement in your subject area that need to be settled.

Purpose of Literature Review?

There are a number of differing descriptions of the purpose of a literature review. Primarily it is a tool for

❖ researching the history of scholarly publication on a topic

❖ becoming aware of the scholarly debate within a topic

❖  a summary or restatement of conclusions from research which has been published

❖ synthesis or recombining, comparing and contrasting, the ideas of others.

❖ evaluate sources

❖ search for gaps

A literature review provides a comprehensive overview of a topic , supporting the fundamental purpose of a research paper, which is to present a new point of view or insight on a topic. The literature review supports the new insight. It does not present or argue for it.

Structure of Literature Review

• Choose a topic
• Find research
• Organize sources/notetaking
• Evaluate Sources
• Synthesize: think of this phase as a narrative . 

There are various ways of organizing the literature review process- if one of these seems closer to your purpose, try it out.

Different Types of Literature Sources

• << Previous: Planning a Thesis Proposal
• Next: Slides, Articles >>
• Last Updated: Jan 22, 2024 2:05 PM
• URL: https://libguides.uml.edu/rohland_surveys

• Search This Site All UCSD Sites Faculty/Staff Search Term
• Contact & Directions
• Climate Statement
• Cognitive Behavioral Neuroscience
• Cognitive Psychology
• Developmental Psychology
• Social Psychology
• Adjunct Faculty
• Non-Senate Instructors
• Researchers
• Psychology Grads
• Affiliated Grads
• New and Prospective Students
• Honors Program
• Experiential Learning
• Programs & Events
• Psi Chi / Psychology Club
• Prospective PhD Students
• Current PhD Students
• Area Brown Bags
• Colloquium Series
• Anderson Distinguished Lecture Series
• Speaker Videos
• Undergraduate Program
• Academic and Writing Resources

Writing Research Papers

• Writing a Literature Review

When writing a research paper on a specific topic, you will often need to include an overview of any prior research that has been conducted on that topic.  For example, if your research paper is describing an experiment on fear conditioning, then you will probably need to provide an overview of prior research on fear conditioning.  That overview is typically known as a literature review.  

Please note that a full-length literature review article may be suitable for fulfilling the requirements for the Psychology B.S. Degree Research Paper .  For further details, please check with your faculty advisor.

Different Types of Literature Reviews

Literature reviews come in many forms.  They can be part of a research paper, for example as part of the Introduction section.  They can be one chapter of a doctoral dissertation.  Literature reviews can also “stand alone” as separate articles by themselves.  For instance, some journals such as Annual Review of Psychology , Psychological Bulletin , and others typically publish full-length review articles.  Similarly, in courses at UCSD, you may be asked to write a research paper that is itself a literature review (such as, with an instructor’s permission, in fulfillment of the B.S. Degree Research Paper requirement). Alternatively, you may be expected to include a literature review as part of a larger research paper (such as part of an Honors Thesis). 

Literature reviews can be written using a variety of different styles.  These may differ in the way prior research is reviewed as well as the way in which the literature review is organized.  Examples of stylistic variations in literature reviews include: 

• Summarization of prior work vs. critical evaluation. In some cases, prior research is simply described and summarized; in other cases, the writer compares, contrasts, and may even critique prior research (for example, discusses their strengths and weaknesses).
• Chronological vs. categorical and other types of organization. In some cases, the literature review begins with the oldest research and advances until it concludes with the latest research.  In other cases, research is discussed by category (such as in groupings of closely related studies) without regard for chronological order.  In yet other cases, research is discussed in terms of opposing views (such as when different research studies or researchers disagree with one another).

Overall, all literature reviews, whether they are written as a part of a larger work or as separate articles unto themselves, have a common feature: they do not present new research; rather, they provide an overview of prior research on a specific topic . 

How to Write a Literature Review

When writing a literature review, it can be helpful to rely on the following steps.  Please note that these procedures are not necessarily only for writing a literature review that becomes part of a larger article; they can also be used for writing a full-length article that is itself a literature review (although such reviews are typically more detailed and exhaustive; for more information please refer to the Further Resources section of this page).

Steps for Writing a Literature Review

1. Identify and define the topic that you will be reviewing.

The topic, which is commonly a research question (or problem) of some kind, needs to be identified and defined as clearly as possible.  You need to have an idea of what you will be reviewing in order to effectively search for references and to write a coherent summary of the research on it.  At this stage it can be helpful to write down a description of the research question, area, or topic that you will be reviewing, as well as to identify any keywords that you will be using to search for relevant research.

2. Conduct a literature search.

Use a range of keywords to search databases such as PsycINFO and any others that may contain relevant articles.  You should focus on peer-reviewed, scholarly articles.  Published books may also be helpful, but keep in mind that peer-reviewed articles are widely considered to be the “gold standard” of scientific research.  Read through titles and abstracts, select and obtain articles (that is, download, copy, or print them out), and save your searches as needed.  For more information about this step, please see the Using Databases and Finding Scholarly References section of this website.

3. Read through the research that you have found and take notes.

Absorb as much information as you can.  Read through the articles and books that you have found, and as you do, take notes.  The notes should include anything that will be helpful in advancing your own thinking about the topic and in helping you write the literature review (such as key points, ideas, or even page numbers that index key information).  Some references may turn out to be more helpful than others; you may notice patterns or striking contrasts between different sources ; and some sources may refer to yet other sources of potential interest.  This is often the most time-consuming part of the review process.  However, it is also where you get to learn about the topic in great detail.  For more details about taking notes, please see the “Reading Sources and Taking Notes” section of the Finding Scholarly References page of this website.

4. Organize your notes and thoughts; create an outline.

At this stage, you are close to writing the review itself.  However, it is often helpful to first reflect on all the reading that you have done.  What patterns stand out?  Do the different sources converge on a consensus?  Or not?  What unresolved questions still remain?  You should look over your notes (it may also be helpful to reorganize them), and as you do, to think about how you will present this research in your literature review.  Are you going to summarize or critically evaluate?  Are you going to use a chronological or other type of organizational structure?  It can also be helpful to create an outline of how your literature review will be structured.

5. Write the literature review itself and edit and revise as needed.

The final stage involves writing.  When writing, keep in mind that literature reviews are generally characterized by a summary style in which prior research is described sufficiently to explain critical findings but does not include a high level of detail (if readers want to learn about all the specific details of a study, then they can look up the references that you cite and read the original articles themselves).  However, the degree of emphasis that is given to individual studies may vary (more or less detail may be warranted depending on how critical or unique a given study was).   After you have written a first draft, you should read it carefully and then edit and revise as needed.  You may need to repeat this process more than once.  It may be helpful to have another person read through your draft(s) and provide feedback.

6. Incorporate the literature review into your research paper draft.

After the literature review is complete, you should incorporate it into your research paper (if you are writing the review as one component of a larger paper).  Depending on the stage at which your paper is at, this may involve merging your literature review into a partially complete Introduction section, writing the rest of the paper around the literature review, or other processes.

Further Tips for Writing a Literature Review

Full-length literature reviews

• Many full-length literature review articles use a three-part structure: Introduction (where the topic is identified and any trends or major problems in the literature are introduced), Body (where the studies that comprise the literature on that topic are discussed), and Discussion or Conclusion (where major patterns and points are discussed and the general state of what is known about the topic is summarized)

Literature reviews as part of a larger paper

• An “express method” of writing a literature review for a research paper is as follows: first, write a one paragraph description of each article that you read. Second, choose how you will order all the paragraphs and combine them in one document.  Third, add transitions between the paragraphs, as well as an introductory and concluding paragraph. 1
• A literature review that is part of a larger research paper typically does not have to be exhaustive. Rather, it should contain most or all of the significant studies about a research topic but not tangential or loosely related ones. 2   Generally, literature reviews should be sufficient for the reader to understand the major issues and key findings about a research topic.  You may however need to confer with your instructor or editor to determine how comprehensive you need to be.

Benefits of Literature Reviews

By summarizing prior research on a topic, literature reviews have multiple benefits.  These include:

• Literature reviews help readers understand what is known about a topic without having to find and read through multiple sources.
• Literature reviews help “set the stage” for later reading about new research on a given topic (such as if they are placed in the Introduction of a larger research paper). In other words, they provide helpful background and context.
• Literature reviews can also help the writer learn about a given topic while in the process of preparing the review itself. In the act of research and writing the literature review, the writer gains expertise on the topic .

Downloadable Resources

• How to Write APA Style Research Papers (a comprehensive guide) [ PDF ]
• Tips for Writing APA Style Research Papers (a brief summary) [ PDF ]
• Example APA Style Research Paper (for B.S. Degree – literature review) [ PDF ]

Further Resources

How-To Videos     

• Writing Research Paper Videos
• UCSD Library Psychology Research Guide: Literature Reviews

External Resources

• Developing and Writing a Literature Review from N Carolina A&T State University
• Example of a Short Literature Review from York College CUNY
• How to Write a Review of Literature from UW-Madison
• Writing a Literature Review from UC Santa Cruz  
• Pautasso, M. (2013). Ten Simple Rules for Writing a Literature Review. PLoS Computational Biology, 9 (7), e1003149. doi : 1371/journal.pcbi.1003149

1 Ashton, W. Writing a short literature review . [PDF]     

2 carver, l. (2014).  writing the research paper [workshop]. , prepared by s. c. pan for ucsd psychology.

Back to top

• Research Paper Structure
• Formatting Research Papers
• Using Databases and Finding References
• What Types of References Are Appropriate?
• Evaluating References and Taking Notes
• Citing References
• Writing Process and Revising
• Improving Scientific Writing
• Academic Integrity and Avoiding Plagiarism
• Writing Research Papers Videos

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Korean Med Sci
• v.35(45); 2020 Nov 23

Reporting Survey Based Studies – a Primer for Authors

Prithvi sanjeevkumar gaur.

1 Smt. Kashibai Navale Medical College and General Hospital, Pune, India.

Olena Zimba

2 Department of Internal Medicine No. 2, Danylo Halytsky Lviv National Medical University, Lviv, Ukraine.

Vikas Agarwal

3 Department Clinical Immunology and Rheumatology, Sanjay Gandhi Postgraduate Institute of Medical Sciences, Lucknow, India.

Latika Gupta

Associated data.

The coronavirus disease 2019 (COVID-19) pandemic has led to a massive rise in survey-based research. The paucity of perspicuous guidelines for conducting surveys may pose a challenge to the conduct of ethical, valid and meticulous research. The aim of this paper is to guide authors aiming to publish in scholarly journals regarding the methods and means to carry out surveys for valid outcomes. The paper outlines the various aspects, from planning, execution and dissemination of surveys followed by the data analysis and choosing target journals. While providing a comprehensive understanding of the scenarios most conducive to carrying out a survey, the role of ethical approval, survey validation and pilot testing, this brief delves deeper into the survey designs, methods of dissemination, the ways to secure and maintain data anonymity, the various analytical approaches, the reporting techniques and the process of choosing the appropriate journal. Further, the authors analyze retracted survey-based studies and the reasons for the same. This review article intends to guide authors to improve the quality of survey-based research by describing the essential tools and means to do the same with the hope to improve the utility of such studies.

Graphical Abstract

INTRODUCTION

Surveys are the principal method used to address topics that require individual self-report about beliefs, knowledge, attitudes, opinions or satisfaction, which cannot be assessed using other approaches. 1 This research method allows information to be collected by asking a set of questions on a specific topic to a subset of people and generalizing the results to a larger population. Assessment of opinions in a valid and reliable way require clear, structured and precise reporting of results. This is possible with a survey based out of a meticulous design, followed by validation and pilot testing. 2 The aim of this opinion piece is to provide practical advice to conduct survey-based research. It details the ethical and methodological aspects to be undertaken while performing a survey, the online platforms available for distributing survey, and the implications of survey-based research.

Survey-based research is a means to obtain quick data, and such studies are relatively easy to conduct and analyse, and are cost-effective (under a majority of the circumstances). 3 These are also one of the most convenient methods of obtaining data about rare diseases. 4 With major technological advancements and improved global interconnectivity, especially during the coronavirus disease 2019 (COVID-19) pandemic, surveys have surpassed other means of research due to their distinctive advantage of a wider reach, including respondents from various parts of the world having diverse cultures and geographically disparate locations. Moreover, survey-based research allows flexibility to the investigator and respondent alike. 5 While the investigator(s) may tailor the survey dates and duration as per their availability, the respondents are allowed the convenience of responding to the survey at ease, in the comfort of their homes, and at a time when they can answer the questions with greater focus and to the best of their abilities. 6 Respondent biases inherent to environmental stressors can be significantly reduced by this approach. 5 It also allows responses across time-zones, which may be a major impediment to other forms of research or data-collection. This allows distant placement of the investigator from the respondents.

Various digital tools are now available for designing surveys ( Table 1 ). 7 Most of these are free with separate premium paid options. The analysis of data can be made simpler and cleaning process almost obsolete by minimising open-ended answer choices. 8 Close-ended answers makes data collection and analysis efficient, by generating an excel which can be directly accessed and analysed. 9 Minimizing the number of questions and making all questions mandatory can further aid this process by bringing uniformity to the responses and analysis simpler. Surveys are arguably also the most engaging form of research, conditional to the skill of the investigator.

Q/t = questions per typeform, A/m = answers per month, Q/s = questions per survey, A/s = answers per survey, NA = not applicable, NPS = net promoter score.

Data protection laws now mandate anonymity while collecting data for most surveys, particularly when they are exempt from ethical review. 10 , 11 Anonymization has the potential to reduce (or at times even eliminate) social desirability bias which gains particular relevance when targeting responses from socially isolated or vulnerable communities (e.g. LGBTQ and low socio-economic strata communities) or minority groups (religious, ethnic and medical) or controversial topics (drug abuse, using language editing software).

Moreover, surveys could be the primary methodology to explore a hypothesis until it evolves into a more sophisticated and partly validated idea after which it can be probed further in a systematic and structured manner using other research methods.

The aim of this paper is to reduce the incorrect reporting of surveys. The paper also intends to inform researchers of the various aspects of survey-based studies and the multiple points that need to be taken under consideration while conducting survey-based research.

SURVEYS IN THE COVID-19 PANDEMIC

The COVID-19 has led to a distinctive rise in survey-based research. 12 The need to socially distance amid widespread lockdowns reduced patient visits to the hospital and brought most other forms of research to a standstill in the early pandemic period. A large number of level-3 bio-safety laboratories are being engaged for research pertaining to COVID-19, thereby limiting the options to conduct laboratory-based research. 13 , 14 Therefore, surveys appear to be the most viable option for researchers to explore hypotheses related to the situation and its impact in such times. 15

LIMITATIONS WHILE CONDUCTING SURVEY-BASED RESEARCH

Designing a fine survey is an arduous task and requires skill even though clear guidelines are available in regard to the same. Survey design requires extensive thoughtfulness on the core questions (based on the hypothesis or the primary research question), with consideration of all possible answers, and the inclusion of open-ended options to allow recording other possibilities. A survey should be robust, in regard to the questions gathered and the answer choices available, it must be validated, and pilot tested. 16 The survey design may be supplanted with answer choices tailored for the convenience of the responder, to reduce the effort while making it more engaging. Survey dissemination and engagement of respondents also requires experience and skill. 17

Furthermore, the absence of an interviewer prevents us from gaining clarification on responses of open-ended questions if any. Internet surveys are also prone to survey fraud by erroneous reporting. Hence, anonymity of surveys is a boon and a bane. The sample sizes are skewed as it lacks representation of population absent on the Internet like the senile or the underprivileged. The illiterate population also lacks representation in survey-based research.

The “Enhancing the QUAlity and Transparency Of health Research” network (EQUATOR) provides two separate guidelines replete with checklists to ensure valid reporting of e-survey methodology. These include “The Checklist for Reporting Results of Internet E-Surveys” (CHERRIES) statement and “ The Journal of Medical Internet Research ” (JMIR) checklist.

COMMON TYPES OF SURVEY-BASED RESEARCH

From a clinician's standpoint, the common survey types include those centered around problems faced by the patients or physicians. 18 Surveys collecting the opinions of various clinicians on a debated clinical topic or feedback forms typically served after attending medical conferences or prescribing a new drug or trying a new method for a given procedure are also surveys. The formulation of clinical practice guidelines entails Delphi exercises using paper surveys, which are yet another form of survey-mediated research.

Size of the survey depends on its intent. They could be large or small surveys. Therefore, identification of the intent behind the survey is essential to allow the investigator to form a hypothesis and then explore it further. Large population-based or provider-based surveys are often done and generate mammoth data over the years. E.g. The National Health and Nutrition Examination Survey, The National Health Interview Survey and the National Ambulatory Medical Care Survey.

SCENARIOS FOR CONDUCTING SURVEY-BASED RESEARCH

Despite all said and done about the convenience of conducting survey-based research, it is prudent to conduct a feasibility check before embarking on one. Certain scenarios may be the key determinants in determining the fate of survey-based research ( Table 2 ).

ETHICS APPROVAL FOR SURVEY-BASED RESEARCH

Approval from the Institutional Review Board should be taken as per requirement according to the CHERRIES checklist. However, rules for approval are different as per the country or nation and therefore, local rules must be checked and followed. For instance, in India, the Indian Council of Medical Research released an article in 2017, stating that the concept of broad consent has been updated which is defined “consent for an unspecified range of future research subject to a few contents and/or process restrictions.” It talks about “the flexibility of Indian ethics committees to review a multicentric study proposal for research involving low or minimal risk, survey or studies using anonymized samples or data or low or minimal risk public health research.” The reporting of approvals received and applied for and the procedure of written, informed consent followed must be clear and transparent. 10 , 19

The use of incentives in surveys is also an ethical concern. 20 The different of incentives that can be used are monetary or non-monetary. Monetary incentives are usually discouraged as these may attract the wrong population due to the temptation of the monetary benefit. However, monetary incentives have been seen to make survey receive greater traction even though this is yet to proven. Monetary incentives are not only provided in terms of cash or cheque but also in the form of free articles, discount coupons, phone cards, e-money or cashback value. 21 These methods though tempting must be seldom used. If used, their use must be disclosed and justified in the report. The use of non-monetary incentives like a meeting with a famous personality or access to restricted and authorized areas. These can also help pique the interest of the respondents.

DESIGNING A SURVEY

As mentioned earlier, the design of a survey is reflective of the skill of the investigator curating it. 22 Survey builders can be used to design an efficient survey. These offer majority of the basic features needed to construct a survey, free of charge. Therefore, surveys can be designed from scratch, using pre-designed templates or by using previous survey designs as inspiration. Taking surveys could be made convenient by using the various aids available ( Table 1 ). Moreover, even the investigator should be mindful of the unintended response effects of ordering and context of survey questions. 23

Surveys using clear, unambiguous, simple and well-articulated language record precise answers. 24 A well-designed survey accounts for the culture, language and convenience of the target demographic. The age, region, country and occupation of the target population is also considered before constructing a survey. Consistency is maintained in the terms used in the survey and abbreviations are avoided to allow the respondents to have a clear understanding of the question being answered. Universal abbreviations or previously indexed abbreviations maintain the unambiguity of the survey.

Surveys beginning with broad, easy and non-specific questions as compared to sensitive, tedious and non-specific ones receive more accurate and complete answers. 25 Questionnaires designed such that the relatively tedious and long questions requiring the respondent to do some nit-picking are placed at the end improves the response rate of the survey. This prevents the respondent to be discouraged to answer the survey at the beginning itself and motivates the respondent to finish the survey at the end. All questions must provide a non-response option and all questions should be made mandatory to increase completeness of the survey. Questions can be framed in close-ended or open-ended fashion. However, close-ended questions are easier to analyze and are less tedious to answer by the respondent and therefore must be the main component in a survey. Open-ended questions have minimal use as they are tedious, take time to answer and require fine articulation of one's thoughts. Also, their minimal use is advocated because the interpretation of such answers requires dedication in terms of time and energy due to the diverse nature of the responses which is difficult to promise owing to the large sample sizes. 26 However, whenever the closed choices do not cover all probabilities, an open answer choice must be added. 27 , 28

Screening questions to meet certain criteria to gain access to the survey in cases where inclusion criteria need to be established to maintain authenticity of target demographic. Similarly, logic function can be used to apply an exclusion. This allows clean and clear record of responses and makes the job of an investigator easier. The respondents can or cannot have the option to return to the previous page or question to alter their answer as per the investigator's preference.

The range of responses received can be reduced in case of questions directed towards the feelings or opinions of people by using slider scales, or a Likert scale. 29 , 30 In questions having multiple answers, check boxes are efficient. When a large number of answers are possible, dropdown menus reduce the arduousness. 31 Matrix scales can be used to answer questions requiring grading or having a similar range of answers for multiple conditions. Maximum respondent participation and complete survey responses can be ensured by reducing the survey time. Quiz mode or weighted modes allow the respondent to shuffle between questions and allows scoring of quizzes and can be used to complement other weighted scoring systems. 32 A flowchart depicting a survey construct is presented as Fig. 1 .

Survey validation

Validation testing though tedious and meticulous, is worthy effort as the accuracy of a survey is determined by its validity. It is indicative of the of the sample of the survey and the specificity of the questions such that the data acquired is streamlined to answer the questions being posed or to determine a hypothesis. 33 , 34 Face validation determines the mannerism of construction of questions such that necessary data is collected. Content validation determines the relation of the topic being addressed and its related areas with the questions being asked. Internal validation makes sure that the questions being posed are directed towards the outcome of the survey. Finally, Test – retest validation determines the stability of questions over a period of time by testing the questionnaire twice and maintaining a time interval between the two tests. For surveys determining knowledge of respondents pertaining to a certain subject, it is advised to have a panel of experts for undertaking the validation process. 2 , 35

Reliability testing

If the questions in the survey are posed in a manner so as to elicit the same or similar response from the respondents irrespective of the language or construction of the question, the survey is said to be reliable. It is thereby, a marker of the consistency of the survey. This stands to be of considerable importance in knowledge-based researches where recall ability is tested by making the survey available for answering by the same participants at regular intervals. It can also be used to maintain authenticity of the survey, by varying the construction of the questions.

Designing a cover letter

A cover letter is the primary means of communication with the respondent, with the intent to introduce the respondent to the survey. A cover letter should include the purpose of the survey, details of those who are conducting it, including contact details in case clarifications are desired. It should also clearly depict the action required by the respondent. Data anonymization may be crucial to many respondents and is their right. This should be respected in a clear description of the data handling process while disseminating the survey. A good cover letter is the key to building trust with the respondent population and can be the forerunner to better response rates. Imparting a sense of purpose is vital to ideationally incentivize the respondent population. 36 , 37 Adding the credentials of the team conducting the survey may further aid the process. It is seen that an advance intimation of the survey prepares the respondents while improving their compliance.

The design of a cover letter needs much attention. It should be captivating, clear, precise and use a vocabulary and language specific to the target population for the survey. Active voice should be used to make a greater impact. Crowding of the details must be avoided. Using italics, bold fonts or underlining may be used to highlight critical information. the tone ought to be polite, respectful, and grateful in advance. The use of capital letters is at best avoided, as it is surrogate for shouting in verbal speech and may impart a bad taste.

The dates of the survey may be intimated, so the respondents may prepare themselves for taking it at a time conducive to them. While, emailing a closed group in a convenience sampled survey, using the name of the addressee may impart a customized experience and enhance trust building and possibly compliance. Appropriate use of salutations like Mr./Ms./Mrs. may be considered. Various portals such as SurveyMonkey allow the researchers to save an address list on the website. These may then be reached out using an embedded survey link from a verified email address to minimize bouncing back of emails.

The body of the cover letter must be short, crisp and not exceed 2–3 paragraphs under idea circumstances. Ernest efforts to protect confidentiality may go a long way in enhancing response rates. 38 While it is enticing to provide incentives to enhance response, these are best avoided. 38 , 39 In cases when indirect incentives are offered, such as provision of results of the survey, these may be clearly stated in the cover letter. Lastly, a formal closing note with the signatures of the lead investigator are welcome. 38 , 40

Designing questions

Well-constructed questionnaires are essentially the backbone of successful survey-based studies. With this type of research, the primary concern is the adequate promotion and dissemination of the questionnaire to the target population. The careful of selection of sample population, therefore, needs to be with minimal flaws. The method of conducting survey is an essential determinant of the response rate observed. 41 Broadly, surveys are of two types: closed and open. Depending on the sample population the method of conducting the survey must be determined.

Various doctors use their own patients as the target demographic, as it improves compliance. However, this is effective in surveys aiming towards a geographically specific, fairly common disease as the sample size needs to be adequate. Response bias can be identified by the data collected from respondent and non-respondent groups. 42 , 43 Therefore, to choose a target population whose database of baseline characteristics is already known is more efficacious. In cases of surveys focused on patients having a rare group of diseases, online surveys or e-surveys can be conducted. Data can also be gathered from the multiple national organizations and societies all over the world. 44 , 45 Computer generated random selection can be done from this data to choose participants and they can be reached out to using emails or social media platforms like WhatsApp and LinkedIn. In both these scenarios, closed questionnaires can be conducted. These have restricted access either through a URL link or through e-mail.

In surveys targeting an issue faced by a larger demographic (e.g. pandemics like the COVID-19, flu vaccines and socio-political scenarios), open surveys seem like the more viable option as they can be easily accessed by majority of the public and ensures large number of responses, thereby increasing the accuracy of the study. Survey length should be optimal to avoid poor response rates. 25 , 46

SURVEY DISSEMINATION

Uniform distribution of the survey ensures equitable opportunity to the entire target population to access the questionnaire and participate in it. While deciding the target demographic communities should be studied and the process of “lurking” is sometimes practiced. Multiple sampling methods are available ( Fig. 1 ). 47

Distribution of survey to the target demographic could be done using emails. Even though e-mails reach a large proportion of the target population, an unknown sender could be blocked, making the use of personal or a previously used email preferable for correspondence. Adding a cover letter along with the invite adds a personal touch and is hence, advisable. Some platforms allow the sender to link the survey portal with the sender's email after verifying it. Noteworthily, despite repeated email reminders, personal communication over the phone or instant messaging improved responses in the authors' experience. 48 , 49

Distribution of the survey over other social media platforms (SMPs, namely WhatsApp, Facebook, Instagram, Twitter, LinkedIn etc.) is also practiced. 50 , 51 , 52 Surveys distributed on every available platform ensures maximal outreach. 53 Other smartphone apps can also be used for wider survey dissemination. 50 , 54 It is important to be mindful of the target population while choosing the platform for dissemination of the survey as some SMPs such as WhatsApp are more popular in India, while others like WeChat are used more widely in China, and similarly Facebook among the European population. Professional accounts or popular social accounts can be used to promote and increase the outreach for a survey. 55 Incentives such as internet giveaways or meet and greets with their favorite social media influencer have been used to motivate people to participate.

However, social-media platforms do not allow calculation of the denominator of the target population, resulting in inability to gather the accurate response rate. Moreover, this method of collecting data may result in a respondent bias inherent to a community that has a greater online presence. 43 The inability to gather the demographics of the non-respondents (in a bid to identify and prove that they were no different from respondents) can be another challenge in convenience sampling, unlike in cohort-based studies.

Lastly, manually filling of surveys, over the telephone, by narrating the questions and answer choices to the respondents is used as the last-ditch resort to achieve a high desired response rate. 56 Studies reveal that surveys released on Mondays, Fridays, and Sundays receive more traction. Also, reminders set at regular intervals of time help receive more responses. Data collection can be improved in collaborative research by syncing surveys to fill out electronic case record forms. 57 , 58 , 59

Data anonymity refers to the protection of data received as a part of the survey. This data must be stored and handled in accordance with the patient privacy rights/privacy protection laws in reference to surveys. Ethically, the data must be received on a single source file handled by one individual. Sharing or publishing this data on any public platform is considered a breach of the patient's privacy. 11 In convenience sampled surveys conducted by e-mailing a predesignated group, the emails shall remain confidential, as inadvertent sharing of these as supplementary data in the manuscript may amount to a violation of the ethical standards. 60 A completely anonymized e-survey discourages collection of Internet protocol addresses in addition to other patient details such as names and emails.

Data anonymity gives the respondent the confidence to be candid and answer the survey without inhibitions. This is especially apparent in minority groups or communities facing societal bias (sex workers, transgenders, lower caste communities, women). Data anonymity aids in giving the respondents/participants respite regarding their privacy. As the respondents play a primary role in data collection, data anonymity plays a vital role in survey-based research.

DATA HANDLING OF SURVEYS

The data collected from the survey responses are compiled in a .xls, .csv or .xlxs format by the survey tool itself. The data can be viewed during the survey duration or after its completion. To ensure data anonymity, minimal number of people should have access to these results. The data should then be sifted through to invalidate false, incorrect or incomplete data. The relevant and complete data should then be analyzed qualitatively and quantitatively, as per the aim of the study. Statistical aids like pie charts, graphs and data tables can be used to report relative data.

ANALYSIS OF SURVEY DATA

Analysis of the responses recorded is done after the time made available to answer the survey is complete. This ensures that statistical and hypothetical conclusions are established after careful study of the entire database. Incomplete and complete answers can be used to make analysis conditional on the study. Survey-based studies require careful consideration of various aspects of the survey such as the time required to complete the survey. 61 Cut-off points in the time frame allow authentic answers to be recorded and analyzed as compared to disingenuous completed questionnaires. Methods of handling incomplete questionnaires and atypical timestamps must be pre-decided to maintain consistency. Since, surveys are the only way to reach people especially during the COVID-19 pandemic, disingenuous survey practices must not be followed as these will later be used to form a preliminary hypothesis.

REPORTING SURVEY-BASED RESEARCH

Reporting the survey-based research is by far the most challenging part of this method. A well-reported survey-based study is a comprehensive report covering all the aspects of conducting a survey-based research.

The design of the survey mentioning the target demographic, sample size, language, type, methodology of the survey and the inclusion-exclusion criteria followed comprises a descriptive report of a survey-based study. Details regarding the conduction of pilot-testing, validation testing, reliability testing and user-interface testing add value to the report and supports the data and analysis. Measures taken to prevent bias and ensure consistency and precision are key inclusions in a report. The report usually mentions approvals received, if any, along with the written, informed, consent taken from the participants to use the data received for research purposes. It also gives detailed accounts of the different distribution and promotional methods followed.

A detailed account of the data input and collection methods along with tools used to maintain the anonymity of the participants and the steps taken to ensure singular participation from individual respondents indicate a well-structured report. Descriptive information of the website used, visitors received and the externally influencing factors of the survey is included. Detailed reporting of the post-survey analysis including the number of analysts involved, data cleaning required, if any, statistical analysis done and the probable hypothesis concluded is a key feature of a well-reported survey-based research. Methods used to do statistical corrections, if used, should be included in the report. The EQUATOR network has two checklists, “The Checklist for Reporting Results of Internet E-Surveys” (CHERRIES) statement and “ The Journal of Medical Internet Research ” (JMIR) checklist, that can be utilized to construct a well-framed report. 62 , 63 Importantly, self-reporting of biases and errors avoids the carrying forward of false hypothesis as a basis of more advanced research. References should be cited using standard recommendations, and guided by the journal specifications. 64

CHOOSING A TARGET JOURNAL FOR SURVEY-BASED RESEARCH

Surveys can be published as original articles, brief reports or as a letter to the editor. Interestingly, most modern journals do not actively make mention of surveys in the instructions to the author. Thus, depending on the study design, the authors may choose the article category, cohort or case-control interview or survey-based study. It is prudent to mention the type of study in the title. Titles albeit not too long, should not exceed 10–12 words, and may feature the type of study design for clarity after a semicolon for greater citation potential.

While the choice of journal is largely based on the study subject and left to the authors discretion, it may be worthwhile exploring trends in a journal archive before proceeding with submission. 65 Although the article format is similar across most journals, specific rules relevant to the target journal may be followed for drafting the article structure before submission.

RETRACTION OF ARTICLES

Articles that are removed from the publication after being released are retracted articles. These are usually retracted when new discrepancies come to light regarding, the methodology followed, plagiarism, incorrect statistical analysis, inappropriate authorship, fake peer review, fake reporting and such. 66 A sufficient increase in such papers has been noticed. 67

We carried out a search of “surveys” on Retraction Watch on 31st August 2020 and received 81 search results published between November 2006 to June 2020, out of which 3 were repeated. Out of the 78 results, 37 (47.4%) articles were surveys, 23 (29.4%) showed as unknown types and 18 (23.2%) reported other types of research. ( Supplementary Table 1 ). Fig. 2 gives a detailed description of the causes of retraction of the surveys we found and its geographic distribution.

A good survey ought to be designed with a clear objective, the design being precise and focused with close-ended questions and all probabilities included. Use of rating scales, multiple choice questions and checkboxes and maintaining a logical question sequence engages the respondent while simplifying data entry and analysis for the investigator. Conducting pilot-testing is vital to identify and rectify deficiencies in the survey design and answer choices. The target demographic should be defined well, and invitations sent accordingly, with periodic reminders as appropriate. While reporting the survey, maintaining transparency in the methods employed and clearly stating the shortcomings and biases to prevent advocating an invalid hypothesis.

Disclosure: The authors have no potential conflicts of interest to disclose.

Author Contributions:

• Conceptualization: Gaur PS, Zimba O, Agarwal V, Gupta L.
• Visualization: Gaur PS, Zimba O, Agarwal V, Gupta L.
• Writing - original draft: Gaur PS, Gupta L.

SUPPLEMENTARY MATERIAL

Reporting survey based research

State-of-the-art on writing a literature review: An overview of types and components

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Research-paper recommender systems: a literature survey

• Published: 26 July 2015
• Volume 17 , pages 305–338, ( 2016 )

Cite this article

• Joeran Beel 1 ,
• Bela Gipp 2 ,
• Stefan Langer 3 &
• Corinna Breitinger 4  

19k Accesses

448 Citations

27 Altmetric

Explore all metrics

In the last 16 years, more than 200 research articles were published about research-paper recommender systems . We reviewed these articles and present some descriptive statistics in this paper, as well as a discussion about the major advancements and shortcomings and an overview of the most common recommendation concepts and approaches. We found that more than half of the recommendation approaches applied content-based filtering (55 %). Collaborative filtering was applied by only 18 % of the reviewed approaches, and graph-based recommendations by 16 %. Other recommendation concepts included stereotyping, item-centric recommendations, and hybrid recommendations. The content-based filtering approaches mainly utilized papers that the users had authored, tagged, browsed, or downloaded. TF-IDF was the most frequently applied weighting scheme. In addition to simple terms, n-grams, topics, and citations were utilized to model users’ information needs. Our review revealed some shortcomings of the current research. First, it remains unclear which recommendation concepts and approaches are the most promising. For instance, researchers reported different results on the performance of content-based and collaborative filtering. Sometimes content-based filtering performed better than collaborative filtering and sometimes it performed worse. We identified three potential reasons for the ambiguity of the results. (A) Several evaluations had limitations. They were based on strongly pruned datasets, few participants in user studies, or did not use appropriate baselines. (B) Some authors provided little information about their algorithms, which makes it difficult to re-implement the approaches. Consequently, researchers use different implementations of the same recommendations approaches, which might lead to variations in the results. (C) We speculated that minor variations in datasets, algorithms, or user populations inevitably lead to strong variations in the performance of the approaches. Hence, finding the most promising approaches is a challenge. As a second limitation, we noted that many authors neglected to take into account factors other than accuracy, for example overall user satisfaction. In addition, most approaches (81 %) neglected the user-modeling process and did not infer information automatically but let users provide keywords, text snippets, or a single paper as input. Information on runtime was provided for 10 % of the approaches. Finally, few research papers had an impact on research-paper recommender systems in practice. We also identified a lack of authority and long-term research interest in the field: 73 % of the authors published no more than one paper on research-paper recommender systems, and there was little cooperation among different co-author groups. We concluded that several actions could improve the research landscape: developing a common evaluation framework, agreement on the information to include in research papers, a stronger focus on non-accuracy aspects and user modeling, a platform for researchers to exchange information, and an open-source framework that bundles the available recommendation approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Artificial intelligence in E-Commerce: a bibliometric study and literature review

Recommender systems and their ethical challenges

Recommender Systems: Techniques, Applications, and Challenges

Some recommender systems also recommended “citations” but in our opinion, differences between recommending papers and citations are marginal, which is why we do not distinguish between these two terms in this paper.

http://citeseerx.ist.psu.edu .

http://scholar.google.com/scholar?sciupd=1&hl=en&as_sdt=0,5 .

http://www.ncbi.nlm.nih.gov/pubmed .

http://www.researchgate.net/ .

http://www.citeulike.org/ .

http://www.docear.org .

http://www.mendeley.com/ .

http://www.bibtip.com/ .

http://www.exlibrisgroup.com/category/bXUsageBasedServices .

http://refseer.ist.psu.edu/ .

http://theadvisor.osu.edu/ .

http://lab.cisti-icist.nrc-cnrc.gc.ca/Sarkanto/ .

http://www.um.org/conferences .

http://recsys.acm.org/ .

Recommendation frameworks such as LensKit or Mahout may also be helpful for researchers and developers, but frameworks are not the topic of this paper.

http://grouplens.org/datasets/movielens/ .

http://labrosa.ee.columbia.edu/millionsong/ .

http://www.kde.cs.uni-kassel.de/ws/dc13/ .

http://trec.nist.gov/data.html .

http://www.citeulike.org/faq/data.adp .

https://www.kde.cs.uni-kassel.de/bibsonomy/dumps/ .

http://csxstatic.ist.psu.edu/about/data .

http://www.comp.nus.edu.sg/~sugiyama/SchPaperRecData.html .

http://core.kmi.open.ac.uk/intro/data_dumps .

Relatedness between genes was retrieved from an external data source that maintained information about gene relatedness.

Attribute similarity was calculated, e.g., based on the number of pages.

http://grouplens.org/ .

The recommender systems of Mendeley, CiteULike, and CiteSeer are counted twice because they offer or offered two independent recommender systems.

We classified a recommender system as not actively maintained if no article was published or no changes were made to the system for a year.

ResearchGate also applied other recommender systems, e.g., for people or news, and it seems that these approaches are more sophisticated.

Median author count was three, maximum count eleven.

http://www.mymedialite.net/ .

http://lenskit.grouplens.org/ .

http://mahout.apache.org/ .

http://www.duineframework.org/ .

http://code.richrelevance.com/reclab-core/ .

http://easyrec.org/ .

http://ls13-www.cs.uni-dortmund.de/homepage/recommender101/index.shtml .

Bollacker, K.D., Lawrence, S., Giles, C.L.: CiteSeer: an autonomous web agent for automatic retrieval and identification of interesting publications. In: Proceedings of the 2nd international conference on Autonomous agents, pp. 116–123 (1998)

Google Scholar, Scholar Update: Making New Connections, Google Scholar Blog. http://googlescholar.blogspot.de/2012/08/scholar-updates-making-new-connections.html

Airoldi, E.M., Blei, D.M., Fienberg, S.E., Xing, E.P., Jaakkola, T.: Mixed membership stochastic block models for relational data with application to protein–protein interactions. In: Proceedings of the International Biometrics Society Annual Meeting, pp. 1–34 (2006)

Arnold, A., Cohen, W.W.: Information extraction as link prediction: using curated citation networks to improve gene detection. In: Proceedings of the 4th International Conference on Wireless Algorithms, Systems, and Applications, pp. 541–550 (2009)

Beel, J., Langer, S., Genzmehr, M.: Sponsored vs. Organic (Research Paper) Recommendations and the Impact of Labeling. In: Proceedings of the 17th International Conference on Theory and Practice of Digital Libraries (TPDL 2013), pp. 395–399 (2013)

Beel, J., Langer, S., Genzmehr, M., Nürnberger, A.: Persistence in Recommender Systems: Giving the Same Recommendations to the Same Users Multiple Times. In: Proceedings of the 17th International Conference on Theory and Practice of Digital Libraries (TPDL 2013), vol. 8092, pp. 390–394 (2013)

Beel, J., Langer, S., Genzmehr, M., Nürnberger, A.: Introducing Docear’s Research Paper Recommender System. In: Proceedings of the 13th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL’13), pp. 459–460 (2013)

Beel, J., Langer, S., Nürnberger, A., Genzmehr, M.: The Impact of Demographics (Age and Gender) and Other User Characteristics on Evaluating Recommender Systems. In: Proceedings of the 17th International Conference on Theory and Practice of Digital Libraries (TPDL 2013), pp. 400–404 (2013)

Böhm, W., Geyer-schulz, A., Hahsler, M., Jahn, M.: Repeat-Buying Theory and Its Application for Recommender Services. In: Proceedings of the 25th Annual Conference of the Gesellschaft für Klassifikation e.V., pp. 229–239 (2003)

Baez, M., Mirylenka, D., Parra, C.: Understanding and supporting search for scholarly knowledge. In: Proceeding of the 7th European Computer Science Summit, pp. 1–8 (2011)

Beel, J., Gipp, B., Langer, S., Genzmehr, M.: Docear: an academic literature suite for searching, organizing and creating academic literature. In: Proceedings of the 11th Annual International ACM/IEEE Joint Conference on Digital Libraries (JCDL), pp. 465–466 (2011)

Beel, J., Gipp, B., Mueller, C.: SciPlore MindMapping’—a tool for creating mind maps combined with PDF and reference management. D-Lib Mag. 15 (11) (2009)

Bethard, S., Jurafsky, D.: Who should I cite: learning literature search models from citation behavior. In: Proceedings of the 19th ACM international conference on Information and knowledge management, pp. 609–618 (2010)

Bogers, T., van den Bosch, A.: Recommending scientific articles using citeulike. In: Proceedings of the 2008 ACM conference on Recommender systems, pp. 287–290 (2008)

Bollen, J., Van de Sompel, H.: An architecture for the aggregation and analysis of scholarly usage data. In: Proceedings of the 6th ACM/IEEE-CS joint conference on Digital libraries, pp. 298–307 (2006)

CiteSeerX, T.: About RefSeer. http://refseer.ist.psu.edu/about (2012)

CiteULike: My Top Recommendations. Website http://www.citeulike.org/profile/username/recommendations (2011)

CiteULike: Science papers that interest you. Blog. http://blog.citeulike.org/?p=11 (2009)

CiteULike: Data from CiteULike’s new article recommender. Blog, http://blog.citeulike.org/?p=136 (2009)

Caragea, C., Silvescu, A., Mitra, P., Giles, C.L.: Can’t See the Forest for the Trees? A Citation Recommendation System. In: iConference 2013 Proceedings, pp. 849–851 (2013)

Chandrasekaran, K., Gauch, S., Lakkaraju, P., Luong, H.: Concept-based document recommendations for citeseer authors. In: Proceedings of the 5th international conference on Adaptive Hypermedia and Adaptive Web-Based Systems, pp. 83–92 (2008)

Choochaiwattana, W.: Usage of tagging for research paper recommendation. In: Proceedings of the 3rd International Conference on Advanced Computer Theory and Engineering (ICACTE), vol. 2, pp. 439–442 (2010)

Councill, I., Giles, C., Di Iorio, E., Gori, M., Maggini, M., Pucci, A.: Towards next generation CiteSeer: a flexible architecture for digital library deployment. In: Research and Advanced Technology for Digital Libraries, pp. 111–122 (2006)

Dong, R., Tokarchuk, L., Ma, A.: Digging Friendship: Paper Recommendation in Social Network. In: Proceedings of Networking and Electronic Commerce Research Conference (NAEC 2009), pp. 21–28 (2009)

ExLibris: bX Usage-Based Services transform your discovery experience!, Web page, http://www.exlibrisgroup.com/category/bXUsageBasedServices (2013)

Ekstrand, M.D., Kannan, P., Stemper, J.A., Butler, J.T., Konstan, J.A., Riedl, J.T.: Automatically building research reading lists. In: Proceedings of the 4th ACM conference on Recommender systems, pp. 159–166 (2010)

Erosheva, E., Fienberg, S., Lafferty, J.: Mixed-membership models of scientific publications. Proc. Natl. Acad. Sci. U. S. Am. 101 (Suppl 1), 5220–5227 (2004)

Article   Google Scholar  

Franke, M., Geyer-Schulz, A.: Using restricted random walks for library recommendations and knowledge space exploration. Int. J. Pattern Recognit. Artif. Intell. 21 (02), 355–373 (2007)

Ferrara, F., Pudota, N., Tasso, C.: A Keyphrase-Based Paper Recommender System. In: Proceedings of the IRCDL’11, pp. 14–25 (2011)

Geyer-Schulz, A., Hahsler, M.: Comparing two recommender algorithms with the help of recommendations by peers. In: Proceedings of the WEBKDD 2002—Mining Web Data for Discovering Usage Patterns and Profiles, pp. 137–158 (2003)

Geyer-Schulz, A., Hahsler, M.: Evaluation of recommender algorithms for an internet information broker based on simple association rules and on the repeat-buying theory. In: Proceedings of the 4th WebKDD Workshop: Web Mining for Usage Patterns and User Profiles, pp. 100–114 (2002)

Geyer-Schulz, A., Hahsler, M., Jahn, M.: A customer purchase incidence model applied to recommender services. In: Proceedings of the 3rd International Workshop on Mining Web Log Data Across All Customers Touch Points, pp. 25–47 (2002)

Geyer-Schulz, A., Hahsler, M., Jahn, M.: Recommendations for virtual universities from observed user behavior. In: Proceedings of the 24th Annual Conference of the Gesellschaft für Klassifikation e.V., pp. 273–280 (2002)

Geyer-Schulz, A., Hahsler, M., Jahn, M., Geyer, A.: Wissenschaftliche Recommendersysteme in Virtuellen Universitäten. In: Proceedings of the Symposiom of Unternehmen Hochschule, pp. 101–114 (2001)

Geyer-Schulz, A., Hahsler, M., Neumann, A., Thede, A.: An integration strategy for distributed recommender services in legacy library systems. In: Between Data Science and Applied Data Analysis. Springer, pp. 412–420 (2003)

Geyer-Schulz, A., Hahsler, M., Neumann, A., Thede, A.: Behavior-based recommender systems as value-added services for scientific libraries. Statistical Data Mining and Knowledge Discovery, pp. 433–454 (2003)

Geyer-Schulz, A., Hahsler, M., Thede, A.: Comparing Simple Association-Rules and Repeat-Buying Based Recommender Systems in a B2B Environment. In: Proceedings of the 26th Annual Conference of the Gesellschaft für Klassifikation e.V., pp. 421–429 (2003)

Geyer-Schulz, A., Neumann, A., Thede, A.: An architecture for behavior-based library recommender systems. Inf. Technol. Libr. 22 (4), 165–174 (2003)

MATH   Google Scholar  

Geyer-Schulz, A., Neumann, A., Thede, A.: Others also use: a robust recommender system for scientific libraries. In: Proceedings of the 7th European Conference on Digital Libraries, pp. 113–125 (2003)

Gillitzer, B.: Der Empfehlungsdienst BibTip - Ein flächendeckendes Angebot im Bibliotheksverbund Bayern. http://www.b-i-t-online.de/heft/2010-01/nachrichtenbeitrag3 . pp. 1–4 (2010)

Gottwald, S.: Recommender Systeme fuer den Einsatz in Bibliotheken/Survey on recommender systems. Konrad-Zuse-Zentrum für Informationstechnik Berlin, ZIB-Report 11–30 (2011)

Geyer-Schulz, A., Hahsler, M., Jahn, M.: Educational and scientific recommender systems: designing the information channels of the virtual university. Int. J. Eng. Educ. 17 (2), 153–163 (2001)

Google Scholar  

Giles, C.L., Bollacker, K.D., Lawrence, S.: CiteSeer: an automatic citation indexing system. In: Proceedings of the 3rd ACM conference on Digital libraries, pp. 89–98 (1998)

Gipp, B., Beel, J.: Citation proximity analysis (CPA)—a new approach for identifying related work based on co-citation analysis. In: Proceedings of the 12th international conference on Scientometrics and informetrics (ISSI’09), vol. 2, pp. 571–575 (2009)

Gipp, B., Beel, J., Hentschel, C.: Scienstein: a research paper recommender system. In: Proceedings of the international conference on Emerging trends in computing (ICETiC’09), pp. 309–315 (2009)

Gori, M., Pucci, A.: Research paper recommender systems: a random-walk based approach. In: Proceedings of the 2006 IEEE/WIC/ACM international conference on Web intelligence, pp. 778–781 (2006)

Henning, V., Reichelt, J.: Mendeley-a last. fm for research? In: Proceedings of the IEEE 4th international conference on eScience, pp. 327–328 (2008)

Hwang, S.-Y., Hsiung, W.-C., Yang, W.-S.: A prototype WWW literature recommendation system for digital libraries. Online Inf. Rev. 27 (3), 169–182 (2003)

He, J., Nie, J.-Y., Lu, Y., Zhao, W.X.: Position-aligned translation model for citation recommendation. In: Proceedings of the 19th international conference on String processing and information retrieval, pp. 251–263 (2012)

He, Q., Kifer, D., Pei, J., Mitra, P., Giles, C.L.: Citation recommendation without author supervision. In: Proceedings of the 4th ACM international conference on Web search and data mining, pp. 755–764 (2011)

He, Q., Pei, J., Kifer, D., Mitra, P., Giles, L.: Context-aware citation recommendation. In: Proceedings of the 19th international conference on World wide web, pp. 421–430 (2010)

Hess, C.: Trust-Based Recommendations in Multi-Layer Networks. IOS Press, Amsterdam (2008)

Hess, C.: Trust-based recommendations for publications: a multi-layer network approach. TCDL Bull. 2 (2), 190–201 (2006)

Hess, C., Stein, K., Schlieder, C.: Trust-enhanced visibility for personalized document recommendations. In: Proceedings of the 2006 ACM symposium on Applied computing, pp. 1865–1869 (2006)

Huang, S., Xue, G.R., Zhang, B.Y., Chen, Z., Yu, Y., Ma, W.Y.: Tssp: a reinforcement algorithm to find related papers. In: Proceedings of the IEEE/WIC/ACM international conference on Web intelligence (WI), pp. 117–123 (2004)

Huang, W., Kataria, S., Caragea, C., Mitra, P., Giles, C.L., Rokach, L.: Recommending citations: translating papers into references. In: Proceedings of the 21st ACM international conference on Information and knowledge management, pp. 1910–1914 (2012)

Huang, Z., Chung, W., Ong, T.H., Chen, H.: A graph-based recommender system for digital library. In: Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital libraries, pp. 65–73 (2002)

Jack, K.: Mendeley: recommendation systems for academic literature. Presentation at Technical University of Graz (TUG) (2012)

Jack, K.: Mendeley suggest: engineering a personalised article recommender system. Presentation at RecSysChallenge workshop 2012 (2012)

Jack, K.: Mahout becomes a researcher: large scale recommendations at Mendeley. Presentation at big data week conferences (2012)

Jiang, Y., Jia, A., Feng, Y., Zhao, D.: Recommending academic papers via users’ reading purposes. In: Proceedings of the 6th ACM conference on Recommender systems, pp. 241–244 (2012)

Jomsri, P., Sanguansintukul, S., Choochaiwattana, W.: A framework for tag-based research paper recommender system: an IR approach. In: Proceedings of the 24th international conference on Advanced information networking and applications (WAINA), pp. 103–108 (2010)

Kapoor, N., Chen, J., Butler, J.T., Fouty, G.C., Stemper, J.A., Riedl, J., Konstan, J.A.: Techlens: a researcher’s desktop. In: Proceedings of the 2007 ACM conference on Recommender systems, pp. 183–184 (2007)

Konstan, J.A., Kapoor, N., McNee, S.M., Butler, J.T.: Techlens: exploring the use of recommenders to support users of digital libraries. In: Proceedings of the coalition for networked information fall 2005 task force meeting, pp. 111–112 (2005)

Kataria, S., Mitra, P., Bhatia, S.: Utilizing context in generative bayesian models for linked corpus. In: Proceedings of the 24th AAAI conference on Artificial intelligence, pp. 1340–1345 (2010)

Kodakateri Pudhiyaveetil, A., Gauch, S., Luong, H., Eno, J.: Conceptual recommender system for CiteSeerX. In: Proceedings of the 3rd ACM conference on Recommender systems, pp. 241–244 (2009)

Kuberek, M., Mönnich, M.: Einsatz von Recommendersystemen in Bibliotheken Recommender systems in libraries. Presentation (2012)

Küçüktunç, O., Kaya, K., Saule, E., Catalyürek, U.V.: Fast recommendation on bibliographic networks. In: Proceedings of the IEEE/ACM international conference on Advances in social networks analysis and mining (ASONAM), pp. 480–487 (2012)

Küçüktunç, O., Kaya, K., Saule, E., Catalyürek, U.V.: Fast recommendation on bibliographic networks with sparse-matrix ordering and partitioning. Soc. Netw. Anal. Min. 3 (4), 1097–1111 (2013)

Küçüktunç, O., Saule, E., Kaya, K., Çatalyürek, Ü.V.: Result Diversification in automatic citation recommendation. In: Proceedings of the iConference workshop on Computational scientometrics: theory and applications, pp. 1–4 (2013)

Küçüktunç, O., Saule, E., Kaya, K., Çatalyürek, Ü.V.: Diversifying citation recommendations. arXiv preprint. arXiv:1209.5809 . pp. 1–19 (2012)

Küçüktunç, O., Saule, E., Kaya, K., Çatalyürek, Ü.V.: Recommendation on academic networks using direction aware citation analysis. arXiv preprint. arXiv:1205.1143 . pp. 1–10 (2012)

Küçüktunç, O., Saule, E., Kaya, K., Çatalyürek, Ü.V.: Direction awareness in citation recommendation. In: Proceedings of DBRank workshop in conjunction with VLDB’12. pp. 161–166 (2012)

Lao, N.: Efficient random walk inference with knowledge bases. PhD Thesis. The Carnegie Mellon University (2012)

Lao, N., Cohen, W.W.: Personalized reading recommendations for Saccharomyces genome database. Unpublished Paper. http://www.cs.cmu.edu/nlao/publication/2012/2012.dils.pdf . pp. 1–15 (2012)

Lao, N., Cohen, W.W.: Personalized reading recommendations for Saccharomyces genome database. Unpublished Poster. http://www.cs.cmu.edu/nlao/publication/2012/2012.dils.poster.portrat.pdf (2012)

Lao, N., Cohen, W. W.: Contextual recommendation with path constrained random walks. Unpublished. http://www.cs.cmu.edu/nlao/doc/2011.cikm.pdf . pp. 1–9 (2011)

Lakkaraju, P., Gauch, S., Speretta, M.: Document similarity based on concept tree distance. In: Proceedings of the 19th ACM conference on Hypertext and hypermedia, pp. 127–132 (2008)

Lao, N., Cohen, W.W.: Relational retrieval using a combination of path-constrained random walks. Mach. Learn. 81 (1), 53–67 (2010)

Article   MathSciNet   Google Scholar  

Lawrence, K.D.B.S.: A system for automatic personalized tracking of scientific literature on the web. In: Proceedings of the 4th ACM conference on Digital libraries, pp. 105–113 (1999)

Lawrence, S.R., Bollacker, K.D., Giles, C.L.: Autonomous citation indexing and literature browsing using citation context. U.S. Patent US 6,738,780 B2Summer-2004

Lawrence, S.R., Giles, C. L., Bollacker, K.D.: Autonomous citation indexing and literature browsing using citation context. U.S. Patent US 6,289,342 B1Nov-2001

Li, H., Councill, I., Lee, W.-C., Giles, C. L.: CiteSeerx: an architecture and web service design for an academic document search engine. In: Proceedings of the 15th international conference on World wide web, pp. 883–884 (2006)

Liang, Y., Li, Q., Qian, T.: Finding relevant papers based on citation relations. In: Proceedings of the 12th international conference on Web-age information management, pp. 403–414 (2011)

Lin, J., Wilbur, W.J.: PubMed related articles: a probabilistic topic-based model for content similarity. BMC Bioinform. 8 (1), 423–436 (2007)

Lu, Y., He, J., Shan, D., Yan, H.: Recommending citations with translation model. In: Proceedings of the 20th ACM international conference on Information and knowledge management, pp. 2017–2020 (2011)

McNee, S. M., Kapoor, N., Konstan, J.A.: Don’t look stupid: avoiding pitfalls when recommending research papers. In: Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work, pp. 171–180 (2006)

Middleton, S.E., Alani, H., De Roure, D.C.: Exploiting synergy between ontologies and recommender systems. In: Proceedings of the semantic web workshop, pp. 1–10 (2002)

Middleton, S.E., De Roure, D., Shadbolt, N.R.: Ontology-based recommender systems. In: Handbook on Ontologies, pp. 779–796, Springer, Berlin (2009)

Middleton, S.E., De Roure, D.C., Shadbolt, N.R.: Foxtrot recommender system: user profiling, ontologies and the World Wide Web. In: Proceedings of the WWW conference, pp. 1–3 (2002)

Middleton, S.E., De Roure, D.C., Shadbolt, N.R.: Capturing knowledge of user preferences: ontologies in recommender systems. In: Proceedings of the 1st international conference on Knowledge capture, pp. 100–107 (2001)

Mönnich, M., Spiering, M.: Adding value to the library catalog by implementing a recommendation system. D-Lib Mag. 14 (5), 4–11 (2008)

McNee, S.M., Albert, I., Cosley, D., Gopalkrishnan, P., Lam, S.K., Rashid, A.M., Konstan, J.A., Riedl, J.: On the recommending of citations for research papers. In: Proceedings of the ACM conference on Computer supported cooperative work, pp. 116–125 (2002)

Middleton, S.E., Shadbolt, N.R., De Roure, D.C.: Ontological user profiling in recommender systems. ACM Trans. Inf. Syst. (TOIS) 22 (1), 54–88 (2004)

Monnich, M., Spiering, M.: Einsatz von BibTip als Recommendersystem im Bibliothekskatalog. Bibliotheksdienst 42 (1), 54 (2008)

Naak, A.: Papyres: un système de gestion et de recommandation d’articles de recherche. Master Thesis. Université de Montréal (2009)

Neumann, A.W.: Recommender Systems for Information Providers. Springer, Berlin (2009)

Naak, A., Hage, H., Aimeur, E.: A multi-criteria collaborative filtering approach for research paper recommendation in papyres. In: Proceedings of the 4th international conference MCETECH, pp. 25–39 (2009)

Naak, A., Hage, H., Aimeur, E.: Papyres: a research paper management system. In: Proceedings of the 10th E-Commerce Technology Conference on Enterprise Computing, E-Commerce and E-Services, pp. 201–208 (2008)

Nallapati, R.M., Ahmed, A., Xing, E.P., Cohen, W.W.: Joint latent topic models for text and citations. In: Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 542–550 (2008)

Nascimento, C., Laender, A.H., da Silva, A.S., Gonçalves, M.A.: A source independent framework for research paper recommendation. In: Proceedings of the 11th annual international ACM/IEEE joint conference on Digital libraries, pp. 297–306 (2011)

Ozono, T., Goto, S., Fujimaki, N., Shintani, T.: P2p based knowledge source discovery on research support system papits. In: Proceedings of the 1st international joint conference on Autonomous agents and multiagent systems: part 1, pp. 49–50 (2002)

Ozono, T., Shintani, T.: P2P based information retrieval on research support system Papits. In: Proceedngs of the IASTED international conference on Artificial and computational intelligence, pp. 136–141 (2002)

Ozono, T., Shintani, T.: Paper classification for recommendation on research support system papits. IJCSNS Int. J. Comput. Sci. Netw. Secur. 6 , 17–23 (2006)

Ozono, T., Shintani, T., Ito, T., Hasegawa, T.: A feature selection for text categorization on research support system Papits. In: Proceedings of the 8th Pacific Rim international conference on Artificial intelligence, pp. 524–533 (2004)

Pennock, D.M., Horvitz, E., Lawrence, S., Giles, C.L.: Collaborative filtering by personality diagnosis: a hybrid memory-and model-based approach. In: Proceedings of the 16th conference on Uncertainty in artificial intelligence, pp. 473–480 (2000)

Petinot, Y., Giles, C.L., Bhatnagar, V., Teregowda, P.B., Han, H.: Enabling interoperability for autonomous digital libraries: an API to citeseer services. In: Digital Libraries, 2004. Proceedings of the 2004 joint ACM/IEEE conference on, pp. 372–373 (2004)

Petinot, Y., Giles, C.L., Bhatnagar, V., Teregowda, P.B., Han, H., Councill, I.: A service-oriented architecture for digital libraries. In: Proceedings of the 2nd international conference on Service oriented computing, pp. 263–268 (2004)

Pohl, S.: Using access data for paper recommendations on ArXiv. org. Master Thesis. Technical University of Darmstadt (2007)

Pohl, S., Radlinski, F., Joachims, T.: Recommending related papers based on digital library access records. In: Proceedings of the 7th ACM/IEEE-CS joint conference on Digital libraries, pp. 417–418 (2007)

Researchgate, T.: Researchgate recommender. http://www.researchgate.net/directory/publications/ (2011)

Rokach, L., Mitra, P., Kataria, S., Huang, W., Giles, L.: A supervised learning method for context-aware citation recommendation in a large corpus. In: Proceedings of the large-scale and distributed systems for information retrieval workshop (LSDS-IR), pp. 17–22 (2013)

Sarkanto: About the Sarkanto Recommender Demo. http://lab.cisti-icist.nrc-cnrc.gc.ca/Sarkanto/about.jsp (2013)

Strohman, T., Croft, W.B., Jensen, D.: Recommending citations for academic papers. In: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pp. 705–706 (2007)

Sugiyama, K., Kan, M.-Y.: Scholarly paper recommendation via user’s recent research interests. In: Proceedings of the 10th ACM/IEEE annual joint conference on Digital libraries (JCDL), pp. 29–38 (2010)

Thomas, D., Greenberg, A., Calarco, P.: Scholarly usage based recommendations: evaluating bX for a Consortium, Presentation. http://igelu.org/wp-content/uploads/2011/09/bx_igelu_presentation_updated_september-13.pdf (2011)

Torres, R., McNee, S.M., Abel, M., Konstan, J.A., Riedl, J.: Enhancing digital libraries with TechLens+. In: Proceedings of the 4th ACM/IEEE-CS joint conference on Digital libraries, 2004, pp. 228–236

Uchiyama, K., Nanba, H., Aizawa, A., Sagara, T.: OSUSUME: cross-lingual recommender system for research papers. In: Proceedings of the 2011 workshop on context-awareness in retrieval and recommendation, pp. 39–42 (2011)

Vellino, A.: A comparison between usage-based and citation-based methods for recommending scholarly research articles. Proc. Am. Soc. Inf. Sci. Technol. 47 (1), 1–2 (2010)

Vellino, A., Zeber, D.: A hybrid, multi-dimensional recommender for journal articles in a scientific digital library. In: Proceedings of the 2007 IEEE/WIC/ACM international conference on Web intelligence, pp. 111–114 (2007)

Wang, Y., Zhai, E., Hu, J., Chen, Z.: Claper: recommend classical papers to beginners. Seventh international conference on Fuzzy systems and knowledge discovery 6 , 2777–2781 (2010)

Watanabe, S., Ito, T., Ozono, T., Shintani, T.: A paper recommendation mechanism for the research support system papits. In: Proceedings of the international workshop on Data engineering issues in E-Commerce, pp. 71–80

Woodruff, A., Gossweiler, R., Pitkow, J., Chi, E.H., Card, S.K.: Enhancing a digital book with a reading recommender. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 153–160 (2000)

Yang, C., Wei, B., Wu, J., Zhang, Y., Zhang, L.: CARES: a ranking-oriented CADAL recommender system. In: Proceedings of the 9th ACM/IEEE-CS joint conference on Digital libraries, pp. 203–212 (2009)

Zarrinkalam, F., Kahani, M.: SemCiR—a citation recommendation system based on a novel semantic distance measure. Program: Electron. Libr. Inf. Syst. 47 (1), 92–112 (2013)

Zarrinkalam, F., Kahani, M.: A new metric for measuring relatedness of scientific papers based on non-textual features. Intell. Inf. Manag. 4 (4), 99–107 (2012)

Zhou, D., Zhu, S., Yu, K., Song, X., Tseng, B.L., Zha, H., Giles, C.L.: Learning multiple graphs for document recommendations. In: Proceedings of the 17th international conference on World Wide Web, pp. 141–150 (2008)

Avancini, H., Candela, L., Straccia, U.: Recommenders in a personalized, collaborative digital library environment. J. Intell. Inf. Syst. 28 (3), 253–283 (2007)

Agarwal, N., Haque, E., Liu, H., Parsons, L.: A subspace clustering framework for research group collaboration. Int. J. Inf. Technol. Web Eng. 1 (1), 35–58 (2006)

Agarwal, N., Haque, E., Liu, H., Parsons, L.: Research paper recommender systems: a subspace clustering approach. In: Proceedings of the 6th international conference on Advances in Web-Age Information Management (WAIM’05), pp. 475–491 (2005)

Bollen, J., Rocha, L.M.: An adaptive systems approach to the implementation and evaluation of digital library recommendation systems. In: Proceedings of the 4th European conference on Digital libraries, Springer, pp. 356–359 (2000)

Bancu, C., Dagadita, M., Dascalu, M., Dobre, C., Trausan-Matu, S., Florea, A.M.: ARSYS-article recommender system. In: Proceedings of the 14th international symposium on Symbolic and numeric algorithms for scientific computing, pp. 349–355 (2012)

Cazella, S.C., Alvares, L.O.C.: Combining data mining technique and users’ relevance opinion to build an efficient recommender system. Revista Tecnologia da Informação, UCB, 4(2) (2005)

Cazella, S., Alvares, L.: Modeling user’s opinion relevance to recommending research papers. In: Proceedings of the UMAP Conference, pp. 150–150 (2005)

Chirawatkul, P.: Structured Peer-to-Peer Search to Build a Bibliographic Paper Recommendation System. Saarland University, Saarland (2006)

Dattolo, A., Ferrara, F., Tasso, C.: Supporting personalized user concept spaces and recommendations for a publication sharing system. In: Proceedings of the 17th international conference on User modeling, adaptation, and personalization, pp. 325–330 (2009)

Daud, A.: Muhammad Akramand Rajpar Shaikh, A.H.: Scientific reference mining using semantic information through topic modeling. Res. J. Eng. Technol. 28 (2), 253–262 (2009)

Farooq, U., Ganoe, C.H., Carroll, J.M., Councill, I.G.: Lee Giles, C.: Design and evaluation of awareness mechanisms in CiteSeer. Inf. Process. Manag. 44 (2), 596–612 (2008)

Fernández, L., Sánchez, J.A., García, A.: Mibiblio: personal spaces in a digital library universe. In: Proceedings of the 5th ACM conference on Digital libraries, pp. 232–233 (2000)

Gross, T.: CYCLADES: a distributed system for virtual community support based on open archives. In: Proceedings of the 11th Euromicro Conference on Parallel, distributed and network-based orocessing, pp. 484–491 (2003)

Geisler, G., McArthur, D., Giersch, S.: Developing recommendation services for a digital library with uncertain and changing data. In: Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries, pp. 199–200 (2001)

Hong, K., Jeon, H., Jeon, C.: UserProfile-based personalized research paper recommendation system. In: Proceedings of the 8th international conference on Computing and networking technology, pp. 134–138 (2012)

Huang, Y.: Combining Social Networks and Content for Recommendation in a Literature Digital Library. National Sun Yat-Sen University, Taiwan (2007)

Kang, S., Cho, Y.: A novel personalized paper search system. In: Proceedings of the international conference on Intelligent computing, pp. 1257–1262 (2006)

Martin, G.H., Schockaert, S., Cornelis, C., Naessens, H.: Metadata impact on research paper similarity. In: 14th European Conference on Digital libraries, pp. 457–460 (2010)

Morales-del-Castillo, J.M., Peis, E., Herrera-Viedma, E.: A filtering and recommender system prototype for scholarly users of digital libraries. In: Proceedings of the Second World Summit on the Knowledge Society, Springer, pp. 108–117 (2009)

Mao, Y., Vassileva, J., Grassmann, W.: A system dynamics approach to study virtual communities. In: Proceedings of the 40th Annual Hawaii International Conference on System Sciences, pp. 178–197 (2007)

Matsatsinis, N.F., Lakiotaki, K., Delia, P.: A system based on multiple criteria analysis for scientific paper recommendation. In: Proceedings of the 11th Panhellenic Conference on Informatics, pp. 135–149 (2007)

Mishra, G.: Optimised research paper recommender system using social tagging. Int. J. Eng. Res. Appl. 2 (2), 1503–1507 (2012)

Nakagawa, A., Ito, T.: An implementation of a knowledge recommendation system based on similarity among users’ profiles. In: Proceedings of the 41st SICE annual conference, vol. 1, pp. 326–327 (2002)

Pan, C., Li, W.: Research paper recommendation with topic analysis. In: Proceedings of the international conference on Computer design and applications (ICCDA), pp. 264–268 (2010)

Popa, H.-E., Negru, V., Pop, D., Muscalagiu, I.: DL-AgentRecom-A multi-agent based recommendation system for scientific documents. In: Proceedings of the 10th international symposium on Symbolic and numeric algorithms for scientific computing, pp. 320–324 (2008)

Ratprasartporn, N., Ozsoyoglu, G.: Finding related papers in literature digital libraries. In: Proceedings of the 11th European Conference on Digital libraries, pp. 271–284 (2007)

Rocha, L.M.: TalkMine: a soft computing approach to adaptive knowledge recommendation. Stud. Fuzziness Soft Comput. 75 , 89–116 (2001)

Rocha, L.M.: Talkmine and the adaptive recommendation project. In: Proceedings of the fourth ACM conference on Digital libraries, pp. 242–243 (1999)

Stock, K., Robertson, A., Reitsma, F., Stojanovic, T., Bishr, M., Medyckyj-Scott, D., Ortmann, J.: eScience for Sea Science: a semantic scientific knowledge infrastructure for marine scientists. In: Proceedings of the 5th IEEE international conference on e-Science, pp. 110–117 (2009)

Straccia, U.: Cyclades: an open collaborative virtual archive environment. Poster ( http://www.ercim.eu/cyclades/cyclades-fs.pdf ) (2003)

Shaoping, Z.: ActiveCite: an interactive system for automatic citation suggestion. Master Thesis. National University of Singapore (2010)

Stock, K., Karasova, V., Robertson, A., Roger, G., Small, M., Bishr, M., Ortmann, J., Stojanovic, T., Reitsma, F., Korczynski, L., Brodaric, B., Gardner, Z.: Finding science with science: evaluating a domain and scientific ontology user interface for the discovery of scientific resources. Trans. GIS 1 , 1–28 (2013)

Tang, T.Y., McCalla, G.: Towards pedagogy-oriented paper recommendations and adaptive annotations for a web-based learning system. In: Knowledge representation and automated reasoning for E-Learning systems, pp. 72–80 (2003)

Tang, J., Zhang, J.: A discriminative approach to topic-based citation recommendation. Advances in Knowledge Discovery and Data Mining, pp. 572–579 (2009)

Tang, T., McCalla, G.: Utilizing artificial learners to help overcome the cold-start problem in a pedagogically-oriented paper recommendation system. In: Adaptive hypermedia and adaptive web-based systems, pp. 245–254 (2004)

Tang, T., McCalla, G.: Beyond learners’ interest: personalized paper recommendation based on their pedagogical features for an e-learning system. In: Proceedings of the 8th Pacific Rim international conference on Artificial intelligence, Springer, pp. 301–310 (2004)

Tang, T.Y., McCalla, G.: Mining implicit ratings for focused collaborative filtering for paper recommendations. In: Proceedings of the workshop on User and group models for web-based adaptive collaborative environments (2003)

Tang, T.Y., McCalla, G.: Smart recommendation for an evolving e-learning system. In: Proceedings of the workshop on Technologies for electronic documents for supporting learning, at the international conference on Artificial intelligence in education, pp. 699–710 (2003)

Tang, T.Y.: The design and study of pedagogical paper recommendation. PhD Thesis. University of Saskatchewan (2008)

Tang, T.Y., McCalla, G.: A multidimensional paper recommender: experiments and evaluations. Internet Comput. IEEE 13 (4), 34–41 (2009)

Tang, T.Y., McCalla, G.: The pedagogical value of papers: a collaborative-filtering based paper recommender. J. Digit. Inf. 10 (2), 1–12 (2009)

Tang, T.Y., McCalla, G.: On the pedagogically guided paper recommendation for an evolving web-based learning system. In: Proceedings of the FLAIRS Conference, pp. 86–91 (2004)

Tang, T.Y., McCalla, G.: The social affordance of a paper. In: Proceedings of the workshop of assessment of group and individual learning through intelligent visualization on the 13th international conference on Artificial intelligence in education, pp. 34–42 (2007)

Tang, X., Zeng, Q.: Keyword clustering for user interest profiling refinement within paper recommender systems. J. Syst. Softw. 85 (1), 87–101 (2012)

Vassileva, J.: Harnessing p2p power in the classroom. In: Proceedings of the conference on Intelligent tutoring systems, pp. 305–314 (2004)

Vassileva, J.: Supporting peer-to-peer user communities. In: Proceedings of the conference on the move to meaningful internet systems, pp. 230–247 (2002)

Vassileva, J., Detters, R., Geer, J., Maccalla, G., Bull, S., Kettel, L.: Lessons from deploying I-Help. In: Workshop on Multi-agent architectures for distributed learning environments. In: Proceedings of international conference on AI and Education, San Antonio, TX, pp. 3–11 (2001)

Vivacqua, A.S., Oliveira, J., de Souza, J.M.: i-ProSE: inferring user profiles in a scientific context. Comput. J. 52 (7), 789–798 (2009)

Weng, S.-S., Chang, H.-L.: Using ontology network analysis for research document recommendation. Expert Syst. Appl. 34 (3), 1857–1869 (2008)

Winoto, P., Tang, T.Y., McCalla, G.I.: Contexts in a paper recommendation system with collaborative filtering. Int. Rev. Res. Open Distance Learn. 13 (5), 56–75 (2012)

Wu, H., Hua, Y., Li, B., Pei, Y.: Enhancing citation recommendation with various evidences. In: Proceedings of the 9th international conference on Fuzzy systems and knowledge discovery (FSKD), pp. 1160–1165 (2012)

Xia, H., Li, J., Tang, J., Moens, M.-F.: Plink-LDA: using link as prior information in topic modeling. In: Proceedings of the conference on Database systems for advanced applications (DASFAA), pp. 213–227 (2012)

Yang, Q., Zhang, S., Feng, B.: Research on personalized recommendation system of scientific and technological periodical based on automatic summarization. In: Proceedings of the 1st international symposium on Information technologies and applications in education, pp. 34–39 (2007)

Yang, S.-Y., Hsu, C.-L.: A new ontology-supported and hybrid recommending information system for scholars. In: Proceedings of the 13th international conference on Network-based information systems (NBiS), pp. 379–384 (2010)

Yin, P., Zhang, M., Li, X.: Recommending scientific literatures in a collaborative tagging environment. In: Proceedings of the 10th international conference on Asian digital libraries, Springer, pp. 478–481 (2007)

Zarrinkalam, F., Kahani, M.: A multi-criteria hybrid citation recommendation system based on linked data. In: Proceedings of the 2nd international eConference on Computer and knowledge engineering, pp. 283–288 (2012)

Zhang, M., Wang, W., Li, X.: A paper recommender for scientific literatures based on semantic concept similarity. In: Proceedings of the international conference on Asian Digital Libraries, pp. 359–362 (2008)

Zhang, Z., Li, L.: A research paper recommender system based on spreading activation model. In: Proceedings of the 2nd international conference on Information Science and Engineering (ICISE), pp. 928–931 (2010)

Gottwald, S., Koch, T.: Recommender systems for libraries. In: Proceedings of the ACM international conference on Recommender systems, pp. 1–5 (2011)

Leong, S.: A survey of recommender systems for scientific papers. Presentation. http://www.liquidpub.org/mediawiki/upload/f/ff/RecommenderSystems.pdf (2012)

Smeaton, A.F., Callan, J.: Personalisation and recommender systems in digital libraries. Int. J. Digit. Libr. 5 (4), 299–308 (2005)

Alotaibi, S., Vassileva, J.: Trust-based recommendations for scientific papers based on the researcher’s current interest. In: Artificial Intelligence in Education, pp. 717–720 (2013)

Beel, J., Langer, S., Genzmehr, M., Gipp, B., Breitinger, C., Nürnberger, A.: Research paper recommender system evaluation: a quantitative literature survey. In: Proceedings of the Workshop on Reproducibility and Replication in Recommender Systems Evaluation (RepSys) at the ACM Recommender System Conference (RecSys), pp. 15–22 (2013)

Beel, J., Langer, S., Genzmehr, M., Gipp, B., Nürnberger, A.: A comparative analysis of offline and online evaluations and discussion of research paper recommender system evaluation. In: Proceedings of the Workshop on Reproducibility and Replication in Recommender Systems Evaluation (RepSys) at the ACM Recommender System Conference (RecSys), pp. 7–14 (2013)

Chen, C., Mao, C., Tang, Y., Chen, G., Zheng, J.: Personalized recommendation based on implicit social network of researchers. In: Joint international conference, ICPCA/SWS, pp. 97–107 (2013)

De Nart, D., Ferrara, F., Tasso, C.: Personalized access to scientific publications: from recommendation to explanation. In: Proceedings of the international conference on User modeling, adaptation, and personalization, pp. 296–301 (2013)

De Nart, D., Ferrara, F., Tasso, C.: RES: a personalized filtering tool for CiteSeerX queries based on keyphrase extraction. In: Proceedings of the international conference on User modeling, adaptation, and personalization (UMAP), pp. 341–343 (2013)

Franke, M., Geyer-Schulz, A., Neumann, A.: Building recommendations from random walks on library opac usage data. In: Data Analysis, Classification and the Forward Search, Springer, pp. 235–246 (2006)

Kim, S.: iScholar: a mobile research support system. PhD Thesis. University of Regina (2013)

Küçüktunç, O.: Result Diversication on Spatial, Multidimensional, Opinion, and Bibliographic Data. Ohio State University, Columbus (2013)

Küçüktunç, O., Saule, E., Kaya, K., Çatalyürek, Ü. V.: TheAdvisor: a webservice for academic recommendation. In: Proceedings of the 13th ACM/IEEE-CS joint conference on Digital libraries, pp. 433–434 (2013)

Küçüktunç, O., Saule, E., Kaya, K., Çatalyürek, Ü. V.: Towards a personalized, scalable, and exploratory academic recommendation service. In: Proceedings of the 2013 IEEE/ACM international conference on Advances in social networks analysis and mining, pp. 636–641 (2013)

Lai, Y., Zeng, J.: A cross-language personalized recommendation model in digital libraries. Electron. Libr. 31 (3), 164–277 (2013)

Li, Y., Yang, M., Zhang, Z.M.: Scientific articles recommendation. In: Proceedings of the 22nd ACM International conference on information and knowledge management, pp. 1147–1156 (2013)

Lee, J., Lee, K., Kim, J.G.: Personalized academic research paper recommendation system. ArXiv Preprint, vol. arXiv:1304.5457 . pp. 1–8 (2013)

Manouselis, N., Verbert, K.: Layered evaluation of multi-criteria collaborative filtering for scientific paper recommendation. Procedia Comput. Sci. 18 , 1189–1197 (2013)

Meng, F., Gao, D., Li, W., Sun, X., Hou, Y.: A unified graph model for personalized query-oriented reference paper recommendation. In: Proceedings of the 22nd ACM international conference on Conference on information and knowledge management, pp. 1509–1512 (2013)

Pera, M.S., Ng, Y.-K.: Exploiting the wisdom of social connections to make personalized recommendations on scholarly articles. J. Intell. Inf. Syst. 42 (3), 371–391 (2014)

Sugiyama, K., Kan, M.-Y.: Exploiting potential citation papers in scholarly paper recommendation. In: Proceedings of the 13th ACM/IEEE-CS joint conference on Digital libraries, pp. 153–162 (2013)

Sun, J., Ma, J., Liu, X., Liu, Z., Wang, G., Jiang, H., Silva, T.: A novel approach for personalized article recommendation in online scientific communities. In: Proceedings of the 46th Hawaii international conference on System sciences (HICSS) (2013)

Sun, J., Ma, J., Liu, Z., Miao, Y.: Leveraging content and connections for scientific article recommendation. Comput. J. 60–71 (2013)

Tian, G., Jing, L.: Recommending scientific articles using bi-relational graph-based iterative RWR. In: Proceedings of the 7th ACM conference on Recommender systems, pp. 399–402 (2013)

Vellino, A.: Usage-based vs. citation-based methods for recommending scholarly research articles. Arxiv, vol. arXiv:1303.7149 (2013)

Yan, R., Yan, H. et al.: Guess what you will cite: personalized citation recommendation based on users’s preference. In: Proceedings of the annual I&R training and education conference, pp. 428–439 (2013)

Yang, W.-S., Lin, Y.-R.: A task-focused literature recommender system for digital libraries. Online Inf. Rev. 37 (4), 581–601 (2013)

Yao, W., He, J., Huang, G., Cao, J., Zhang, Y.: Personalized recommendation on multi-layer context graph. In: Web Information Systems Engineering (WISE 2013), pp. 135–148 (2013)

Yu, L., Yang, J., Yang, D., Yang, X.: A decision support system for finding research topic based on paper recommendation. In: Proceedings of the Pacific Asia conference on Information systems (2013)

Zarrinkalam, F., Kahani, M.: Using semantic relations to improve quality of a citation recommendation system. Soft Comput. J. 1 (2), 36–45 (2013)

Zhang, Z.P., Li, L.N., Yu, H.Y.: A hybrid document recommender algorithm based on random walk. Appl. Mech. Mater. 2270 , 336–338 (2013)

Beel, J., Gipp, B.: Academic search engine spam and Google Scholar’s resilience against it. J. Electron. Publ. 13 (3) (2010)

Bar-Ilan, J.: Which h-index?—A comparison of WoS. Scopus Google Scholar Scientometr. 74 (2), 257–271 (2007)

MathSciNet   Google Scholar  

Noruzi, A.: Google Scholar: the new generation of citation indexes. Libri 55 (4), 170–180 (2005)

Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., Riedl, J.: GroupLens: an open architecture for collaborative filtering of netnews. In: Proceedings of the 1994 ACM conference on Computer supported cooperative work, pp. 175–186 (1994)

Melville, P., Mooney, R.J., Nagarajan, R.: Content-boosted collaborative filtering for improved recommendations. In: Proceedings of the National Conference on Artificial Intelligence, pp. 187–192 (2002)

Shi, Y., Larson, M., Hanjalic, A.: Collaborative filtering beyond the user-item matrix: a survey of the state of the art and future challenges. ACM Comput. Surv. 47(1), 3:1–3:45 (2014)

Gunawardana, A., Shani, G.: A survey of accuracy evaluation metrics of recommendation tasks. J. Mach. Learn. Res. 10 , 2935–2962 (2009)

MathSciNet   MATH   Google Scholar  

Herlocker, J.L., Konstan, J.A., Terveen, L.G., Riedl, J.T.: Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. (TOIS) 22 (1), 5–53 (2004)

Ge, M., Delgado-Battenfeld, C., Jannach, D.: Beyond accuracy: evaluating recommender systems by coverage and serendipity. In: Proceedings of the 4th ACM conference on Recommender systems, pp. 257–260 (2010)

Ritchie, A., Teufel, S., Robertson, S.: Using terms from citations for IR: some first results. In: Macdonald, C., Ounis, I., Plachouras, V., Ruthven, I., White, R.W. (eds.) Advances in Information Retrieval, pp. 211–221. Springer (2008)

Ritchie, A.: Citation context analysis for information retrieval. PhD Thesis. University of Cambridge (2008)

Dumais, S.T., Nielsen, J.: Automating the assignment of submitted manuscripts to reviewers. In: Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, pp. 233–244 (1992)

Wang, F., Shi, N., Chen, B.: A comprehensive survey of the reviewer assignment problem. Int. J. Inf. Technol. Decis. Mak. 9 (04), 645–668 (2010)

Article   MATH   Google Scholar  

Hirsch, J.E.: An index to quantify an individual’s scientific research output. Proc. Natl. Acad. Sci. U. S. Am. 102 (46), 16569 (2005)

Small, H.: Co-citation in the scientific literature: a new measure of the relationship between two documents. J. Am. Soc. Inf. Sci. 24 , 265–269 (1973)

Kessler, M.M.: Bibliographic coupling between scientific papers. Am. Documentation 14 , 10–25 (1963)

Zyczkowski, K.: Citation graph, weighted impact factors and performance indices. Scientometrics 85 (1), 301–315 (2010)

Fischer, G.: User modeling in human–computer interaction. User Model. User-Adapt. Interact. 11 (1), 65–86 (2001)

Eirinaki, M., Vazirgiannis, M.: Web mining for web personalization. ACM Trans. Internet Technol. (TOIT) 3 (1), 1–27 (2003)

Pierrakos, D., Paliouras, G., Papatheodorou, C., Spyropoulos, C.D.: Web usage mining as a tool for personalization: a survey. User Model. User-Adapt. Interact. 13 (4), 311–372 (2003)

Mooney, R.J., Roy, L.: Content-based book recommending using learning for text categorization. In: Proceedings of the 5th ACM conference on Digital libraries, pp. 195–204 (2000)

Brusilovsky, P., Farzan, R., Ahn, J.: Comprehensive personalized information access in an educational digital library. In: Digital Libraries, 2005. JCDL’05. In: Proceedings of the 5th ACM/IEEE-CS joint conference on, pp. 9–18 (2005)

Faensen, D., Faultstich, L., Schweppe, H., Hinze, A., Steidinger, A.: Hermes: a notification service for digital libraries. In: Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries, pp. 373–380 (2001)

Das, S., Mitra, P., Giles, C.L.: Similar researcher search’. In: Academic Environments. In: Proceedings of the JCDL’12, pp. 167–170 (2012)

Abu-Jbara, A., Radev, D.: Coherent citation-based summarization of scientific papers. In: Proceedings of the 49th annual meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 500–509 (2011)

Mohammad, S., Dorr, B., Egan, M., Hassan, A., Muthukrishan, P., Qazvinian, V., Radev, D., Zajic, D.: Using citations to generate surveys of scientific paradigms. In: Proceedings of human language technologies: the 2009 annual conference of the North American Chapter of the Association for Computational Linguistics, 2009, pp. 584–592

Teufel, S., Moens, M.: Summarizing scientific articles: experiments with relevance and rhetorical status. Comput. Linguist. 28 (4), 409–445 (2002)

Collins, L.M., Mane, K.K., Martinez, M.L., Hussell, J.A., Luce, R.E.: ScienceSifter: facilitating activity awareness in collaborative research groups through focused information feeds. In: 1st international conference on e-Science and grid computing, pp. 40–47 (2005)

Klamma, R., Cuong, P.M., Cao, Y.: You never walk alone: recommending academic events based on social network analysis. In: Zhou, J. (ed.) Complex Sciences, pp. 657–670. Springer (2009)

Yang, Z., Davison, B. D.: Venue recommendation: submitting your paper with style. In: Machine learning and applications (ICMLA), 2012 11th international conference on, vol. 1, pp. 681–686 (2012)

Oh, S., Lei, Z., Lee, W.-C., Mitra, P., Yen, J.: CV-PCR: a context-guided value-driven framework for patent citation recommendation. In: Proceedings of the 22nd ACM international conference on Conference on information and knowledge management, pp. 2291–2296 (2013)

Singhal, A., Kasturi, R., Sivakumar, V., Srivastava, J.: Leveraging web intelligence for finding interesting research datasets. In: Web intelligence (WI) and intelligent agent technologies (IAT), 2013 IEEE/WIC/ACM international joint conferences on, vol. 1, pp. 321–328 (2013)

Gipp, B., Beel, J.: Citation based plagiarism detection–a new approach to identify plagiarized work language independently. In: Proceedings of the 21st ACM conference on Hypertext and hypermedia, pp. 273–274 (2010)

Zhan, S., Byung-Ryul, A., Ki-Yol, E., Min-Koo, K., Jin-Pyung, K., Moon-Kyun, K. (2008) Plagiarism detection using the Levenshtein distance and Smith-Waterman algorithm. In: Proceedings of the 3rd international conference on Innovative computing information and control, pp. 569–569

Zini, M., Fabbri, M., Moneglia, M., Panunzi, A.: Plagiarism detection through multilevel text comparison. In: Proceedings of the 2nd conference on Automated production of cross media content for multi-channel distribution, pp. 181–185 (2006)

Ley, M., Reuther, P.: Maintaining an online bibliographical database: the problem of data quality, EGC’2006, Actes des sixièmes journées Extraction et Gestion des Connaissances, pp. 17–20 (2006)

Beel, J., Langer, S., Genzmehr, M., Müller, C.: Docears PDF inspector: title extraction from PDF files. In: Proceedings of the 13th ACM/IEEE-CS joint conference on Digital libraries (JCDL’13), pp. 443–444 (2013)

Beel, J., Gipp, B., Shaker, A., Friedrich, N.: SciPlore Xtract: extracting titles from scientific PDF documents by analyzing style information (Font Size). In: Research and Advanced Technology for Digital Libraries. Proceedings of the 14th European conference on Digital libraries (ECDL’10), vol. 6273, pp. 413–416 (2010)

Han, H., Giles, C.L., Manavoglu, E., Zha, H., Zhang, Z., Fox, E.A.: Automatic document metadata extraction using support vector machines. In: Proceedings of the 3rd ACM/IEEE-CS joint conference on Digital libraries, pp. 37–48 (2003)

Hu, Y., Li, H., Cao, Y., Teng, L., Meyerzon, D., Zheng, Q.: Automatic extraction of titles from general documents using machine learning. Inf. Process. Manag. 42 (5), 1276–1293 (2006)

Peng, F., McCallum, A.: Information extraction from research papers using conditional random fields. Inf. Process. Manag. 42 (4), 963–979 (2006)

Lawrence, S., Giles, C.L., Bollacker, K.D.: Autonomous citation matching. In: Proceedings of the 3rd annual conference on Autonomous agents, pp. 392–393 (1999)

Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds): Recommender Systems Handbook, pp. 1–35. Springer, Berlin (2011)

Schafer, J.B., Frankowski, D., Herlocker, J., Sen, S.: Collaborative filtering recommender systems. Lect. Notes Comput. Sci. 4321 , 291 (2007)

Rossi, P.H., Lipsey, M.W., Freeman, H.E.: Evaluation: A Aystematic Approach, 7th edn. Sage publications, Thousand Oaks (2004)

Gorrell, G., Ford, N., Madden, A., Holdridge, P., Eaglestone, B.: Countering method bias in questionnaire-based user studies. J. Documentation 67 (3), 507–524 (2011)

Leroy, G.: Designing User Studies in Informatics. Springer, Berlin (2011)

Book   Google Scholar  

Said, A., Tikk, D., Shi, Y., Larson, M., Stumpf, K., Cremonesi, P.: Recommender systems evaluation: a 3d benchmark. In: ACM RecSys 2012 workshop on Recommendation utility evaluation: beyond RMSE, Dublin, Ireland, pp. 21–23 (2012)

Cremonesi, P., Garzotto, F., Turrin, R.: Investigating the persuasion potential of recommender systems from a quality perspective: an empirical study. ACM Trans. Interact. Intell. Syst. (TiiS) 2 (2), 11 (2012)

Cremonesi, P., Garzotto, F., Negro, S., Papadopoulos, A.V., Turrin, R.: Looking for ‘good’ recommendations: a comparative evaluation of recommender systems. In: Human–computer interaction-INTERACT 2011, Springer, pp. 152–168 (2011)

Burns, C.A., Bush, F.R.: Marketing Research, 7th edn. Prentice Hall, Upper Saddle River (2013)

Loeppky, J.L., Sacks, J., Welch, W.J.: Choosing the sample size of a computer experiment: a practical guide. Technometrics 51 (4), 366–376 (2009)

Zheng, H., Wang, D., Zhang, Q., Li, H., Yang, T.: Do clicks measure recommendation relevancy?: an empirical user study. In: Proceedings of the 4th ACM conference on Recommender systems, pp. 249–252 (2010)

Konstan, J.A., Riedl, J.: Recommender systems: from algorithms to user experience. User Model. User-Adapt. Interact. 22 (1–2), 101–123 (2012)

Konstan, J.A., Riedl, J.: Recommender systems: from algorithms to user experience. User Model. User-Adapt. Interact. 22(1–2), 101–123 (2012)

Matejka, J., Li, W., Grossman, T., Fitzmaurice, G.: CommunityCommands: command recommendations for software applications. In: Proceedings of the 22nd annual ACM symposium on User interface software and technology, pp. 193–202 (2009)

Rashid, A.M., Albert, I., Cosley, D., Lam, S.K., McNee, S.M., Konstan, J.A., Riedl, J.: Getting to know you: learning new user preferences in recommender systems. In: Proceedings of the 7th international conference on Intelligent user interfaces, pp. 127–134 (2002)

Hersh, W., Turpin, A., Price, S., Chan, B., Kramer, D., Sacherek, L., Olson, D.: Do batch and user evaluations give the same results? In: Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pp. 17–24 (2000)

Hersh, W.R., Turpin, A., Sacherek, L., Olson, D., Price, S., Chan, B., Kraemer, D.: Further Analysis of whether batch and user evaluations give the same results with a question-answering task. In: Proceedings of the 9th Text REtrieval Conference (TREC 9) (2000)

Said, A.: Evaluating the accuracy and utility of recommender systems. PhD Thesis. Technische Universität Berlin (2013)

Turpin, A.H., Hersh, W.: Why batch and user evaluations do not give the same results. In: Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, pp. 225–231 (2001)

Jannach, D., Lerche, L., Gedikli, F., Bonnin, G.: What recommenders recommend—an analysis of accuracy, popularity, and sales diversity effects. In: User Modeling, Adaptation, and Personalization, Springer, pp. 25–37 (2013)

Knijnenburg, B.P., Willemsen, M.C., Gantner, Z., Soncu, H., Newell, C.: Explaining the user experience of recommender systems. User Model. User-Adapt. Interact. 22 (4–5), 441–504 (2012)

Jannach, D., Zanker, M., Ge, M., Gröning, M.: Recommender systems in computer science and information systems–a landscape of research. In: Proceedings of the 13th international conference, EC-Web, pp. 76–87 (2012)

Good, N., Schafer, J.B., Konstan, J.A., Borchers, A., Sarwar, B., Herlocker, J., Riedl, J.: Combining collaborative filtering with personal agents for better recommendations. In: Proceedings of the National Conference on Artificial Intelligence, pp. 439–446 (1999)

Palopoli, L., Rosaci, D., Sarné, G.M.: A multi-tiered recommender system architecture for supporting E-Commerce. In: Fortino, G., Badica, C., Malgeri, M., Unland, R. (eds.) Intelligent Distributed Computing VI, pp. 71–81. Springer (2013)

Lee, Y.-L., Huang, F.-H.: Recommender system architecture for adaptive green marketing. Expert Syst. Appl. 38 (8), 9696–9703 (2011)

Prieto, M.E., Menéndez, V.H., Segura, A.A., Vidal, C.L.: A recommender system architecture for instructional engineering. In: Emerging Technologies and Information Systems for the Knowledge Society, Springer, pp. 314–321 (2008)

Bhatia, S., Caragea, C., Chen, H.-H., Wu, J., Treeratpituk, P., Wu, Z., Khabsa, M., Mitra, P., Giles, C.L.: Specialized research datasets in the CiteSeerx digital library. D-Lib Mag. 18 (7/8) (2012)

Jack, K., Hristakeva, M., de Zuniga, R.G., Granitzer, M.: Mendeley’s open data for science and learning: a reply to the dataTEL challenge. Int. J. Technol. Enhanc. Learn. 4 (1/2), 31–46 (2012)

Breese, J.S., Heckerman, D., Kadie, C.: Empirical analysis of predictive algorithms for collaborative filtering. Microsoft Research, MSR-TR-98-12 (1998)

Karypis, G.: Evaluation of item-based top-n recommendation algorithms. In: Proceedings of the 10th international conference on Information and knowledge management, pp. 247–254 (2001)

Casadevall, A., Fang, F.C.: Reproducible science. Infect. Immun. 78 (12), 4972–4975 (2010)

Rehman, J.: Cancer research in crisis: are the drugs we count on based on bad science? http://www.salon.com/2013/09/01/is_cancer_research_facing_a_crisis/ (2013)

Drummond, C.: Replicability is not reproducibility: nor is it good science. In: Proceedings of the evaluation methods for MachineLearning Workshop at the 26th ICML (2009)

Al-Maskari, A., Sanderson, M., Clough, P.: The relationship between IR effectiveness measures and user satisfaction. In: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pp. 773–774 (2007)

Knijnenburg, B.P., Willemsen, M.C., Kobsa, A.: A pragmatic procedure to support the user-centric evaluation of recommender systems. In: Proceedings of the 5th ACM conference on Recommender systems, pp. 321–324 (2011)

Pu, P., Chen, L., Hu, R.: Evaluating recommender systems from the user’s perspective: survey of the state of the art. User Model. User-Adapt. Interact. 22 (4–5), 317–355 (2012)

Ekstrand, M.D., Ludwig, M., Konstan, J.A., Riedl, J.T.: Rethinking the recommender research ecosystem: reproducibility, openness, and LensKit. In: Proceedings of the 5th ACM conference on Recommender systems, pp. 133–140 (2011)

Konstan, J.A., Adomavicius, G.: Toward identification and adoption of best practices in algorithmic recommender systems research. In: Proceedings of the international workshop on Reproducibility and replication in recommender systems evaluation, pp. 23–28 (2013)

Burke, R.: Hybrid recommender systems: survey and experiments. User Model. User-Adapt. Interact. 12 (4), 331–370 (2002)

Perugini, S., Gonçalves, M.A., Fox, E.A.: Recommender systems research: a connection-centric survey. J. Intell. Inf. Syst. 23 (2), 107–143 (2004)

Torre, I.: Adaptive systems in the era of the semantic and social web, a survey. User Model. User-Adapt. Interact. 19 (5), 433–486 (2009)

Zanker, M., Jessenitschnig, M., Jannach, D., Gordea, S.: Comparing recommendation strategies in a commercial context. IEEE Intell. Syst. 22 (3), 69–73 (2007)

Rich, E.: User modeling via stereotypes. Cogn. Sci. 3 (4), 329–354 (1979)

Barla, M.: Towards social-based user modeling and personalization. Inf. Sci. Technol. Bull. ACM Slovakia 3 , 52–60 (2011)

Weber, I., Castillo, C.: The demographics of web search. In: Proceeding of the 33rd international ACM SIGIR conference on Research and development in information retrieval, pp. 523–530 (2010)

Mattioli, D.: On Orbitz, Mac users steered to pricier hotels. Wall Str. J. vol. http://online.wsj.com/news/articles/SB10001424052702304458604577488822667325882 (2012)

Beel, J.: Towards effective research-paper recommender systems and user modeling based on mind maps. PhD Thesis. Otto-von-Guericke Universität Magdeburg (2015)

Beel, J., Langer, S., Kapitsaki, G.M., Breitinger, C., Gipp, B.: Exploring the potential of user modeling based on mind maps. In: Proceedings of the 23rd conference on User modelling, adaptation and personalization (UMAP) (to appear) (2015)

Beel, J., Gipp, B., Wilde, E.: Academic search engine optimization (ASEO): optimizing scholarly literature for Google Scholar and Co. J. Sch. Publ. 41 (2), 176–190 (2010)

Paik, W., Yilmazel, S., Brown, E., Poulin, M., Dubon, S., Amice, C.: Applying natural language processing (nlp) based metadata extraction to automatically acquire user preferences. In: Proceedings of the 1st international conference on Knowledge capture, pp. 116–122 (2001)

Seroussi, Y.: Utilising user texts to improve recommendations. In: De Bra, P., Kobsa, A., Chin, D. (eds.) User Modeling, Adaptation, and Personalization, pp. 403–406. Springer, Berlin (2010)

Seroussi, Y., Zukerman, I., Bohnert, F.: Collaborative inference of sentiments from texts. In: De Bra, P., Kobsa, A., Chin, D. (eds.) User Modeling, Adaptation, and Personalization, pp. 195–206. Springer, Berlin (2010)

Esposito, F., Ferilli, S., Basile, T.M.A., Mauro, N.D.: Machine learning for digital document processing: from layout analysis to metadata extraction. Stud. Comput. Intell. (SCI) 90 , 105–138 (2008)

Shin, C.K., Doermann, D.: Classification of document page images based on visual similarity of layout structures. In: Proceedings of the SPIE document recognition and retrieval VII, pp. 182–190 (2000)

Buttler, D.: A short survey of document structure similarity algorithms. In: Proceedings of the 5th international conference on Internet computing (2004)

Brin, S., Page, L.: The anatomy of a large-scale hypertextual Web search engine. Comput. Netw. ISDN Syst. 30 (1–7), 107–117 (1998)

McBryan, O.A.: GENVL and WWWW: tools for taming the Web. In: Proceedings of the 1st international World Wide Web conference, vol. 341 (1994)

Shi, S., Xing, F., Zhu, M., Nie, Z., Wen, J.-R.: Anchor text extraction for academic search. In: Proceedings of the 2009 workshop on Text and citation analysis for scholarly digital libraries (ACL-IJCNLP 2009), pp. 10–18 (2009)

Manning, C.D., Raghavan, P., Schütze, H.: An Introduction to Information Retrieval, Online edn. Cambridge University Press, Cambridge (2009)

Councill, I.G., Giles, C.L., Kan, M.Y.: ParsCit: an open-source CRF reference string parsing package. Proc. LREC 2008 , 661–667 (2008)

Marinai, S.: Metadata extraction from PDF papers for digital library ingest. 10th international conference on Document analysis and recognition (2009)

Goldberg, D., Nichols, D., Oki, B.M., Terry, D.: Using collaborative filtering to weave an information Tapestry. Commun. ACM 35 (12), 61–70 (1992)

Brooks, T.A.: Private acts and public objects: an investigation of citer motivations. J. Am. Soc. Inf. Sci. 36 (4), 223–229 (1985)

Liu, M.: Progress in documentation the complexities of citation practice: a review of citation studies. J. Documentation 49 (4), 370–408 (1993)

MacRoberts, M.H., MacRoberts, B.: Problems of citation analysis. Scientometrics 36 , 435–444 (1996)

Sosnovsky, S., Dicheva, D.: Ontological technologies for user modeling. Int. J. Metadata Semant. Ontol. 5 (1), 32–71 (2010)

Sundar, S.S., Oeldorf-Hirsch, A., Xu, Q.: The bandwagon effect of collaborative filtering technology. In: CHI’08 extended abstracts on Human factors in computing systems, pp. 3453–3458 (2008)

Mehta, B., Hofmann, T., Fankhauser, P.: Lies and propaganda: detecting spam users in collaborative filtering. In: Proceedings of the 12th international conference on Intelligent user interfaces, pp. 14–21 (2007)

Mehta, B., Hofmann, T., Nejdl, W.: Robust collaborative filtering. In: Proceedings of the 2007 ACM conference on Recommender systems, pp. 49–56 (2007)

Mehta, B., Nejdl, W.: Attack resistant collaborative filtering. In: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, pp. 75–82 (2008)

Sugiyama, K., Kan, M.Y.: Serendipitous recommendation for scholarly papers considering relations among researchers. In: Proceeding of the 11th annual international ACM/IEEE joint conference on Digital libraries, pp. 307–310 (2011)

Burke, R.: Hybrid web recommender systems. The adaptive web, pp. 377–408 (2007)

Ahlgren, P., Colliander, C.: Document-document similarity approaches and science mapping: experimental comparison of five approaches. J. Informetr. 3 (1), 49–63 (2009)

Hammouda, K.M., Kamel, M.S.: Phrase-based document similarity based on an index graph model. In: Data mining, 2002. ICDM 2003. Proceedings. 2002 IEEE international conference on, pp. 203–210 (2002)

Lee, M.D., Pincombe, B., Welsh, M.: An empirical evaluation of models of text document similarity. In: Proceedings of the 27th annual conference of the Cognitive Science Society, pp. 1254–1259 (2005)

Tsymbal, A.: The Problem of Concept Drift: Definitions and Related Work. Computer Science Department, Trinity College, Dublin (2004)

Victor, P., De Cock, M., Cornelis, C.: Trust and recommendations. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P. B. (eds.) Recommender Systems Handbook, pp. 645–675. Springer (2011)

Verbert, K., Parra, D., Brusilovsky, P., Duval, E.: Visualizing recommendations to support exploration, transparency and controllability. In: Proceedings of the 2013 international conference on Intelligent user interfaces, pp. 351–362 (2013)

Lam, S., Frankowski, D., Riedl, J.: Do you trust your recommendations? An exploration of security and privacy issues in recommender systems. Emerging Trends in Information and Communication Security, pp. 14–29 (2006)

Ziegler, C.N., McNee, S.M., Konstan, J.A., Lausen, G.: Improving recommendation lists through topic diversification. In: Proceedings of the 14th international conference on World Wide Web, pp. 22–32 (2005)

Burke, R., Ramezani, M.: Matching recommendation technologies and domains. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 367–386. Springer (2011)

Zhang, Y., Callan, J., Minka, T.: Novelty and redundancy detection in adaptive filtering. In: Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, pp. 81–88 (2002)

Pizzato, L., Rej, T., Yacef, K., Koprinska, I., Kay, J.: Finding someone you will like and who won’t reject you In: A. Konstan, J.A., Conejo, R., Marzo, J.L., Oliver, N. (eds.) User Modeling, Adaption and Personalization, pp. 269–280. Springer, Berlin (2011)

Cosley, D., Lam, S.K., Albert, I., Konstan, J.A., Riedl, J.: Is seeing believing? How recommender system interfaces affect users’ opinions. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 585–592 (2003)

Herlocker, J.L., Konstan, J.A., Riedl, J.: Explaining collaborative filtering recommendations. In: Proceedings of the 2000 ACM conference on Computer supported cooperative work, pp. 241–250 (2000)

Carmagnola, F., Cena, F., Gena, C.: User model interoperability: a survey. User Model. User-Adapt. Interact. 21 (3), 285–331 (2011)

Download references

Author information

Authors and affiliations.

Docear, Magdeburg, Germany

Joeran Beel

University of Konstanz, Konstanz, Germany

Otto-von-Guericke University, Magdeburg, Germany

Stefan Langer

Linnaeus University, Kalmar, Sweden

Corinna Breitinger

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Joeran Beel .

Rights and permissions

Reprints and permissions

About this article

Beel, J., Gipp, B., Langer, S. et al. Research-paper recommender systems: a literature survey. Int J Digit Libr 17 , 305–338 (2016). https://doi.org/10.1007/s00799-015-0156-0

Download citation

Received : 24 February 2014

Revised : 10 June 2015

Accepted : 22 June 2015

Published : 26 July 2015

Issue Date : November 2016

DOI : https://doi.org/10.1007/s00799-015-0156-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Recommender system
• User modeling
• Research paper recommender systems
• Content based filtering
• Find a journal
• Publish with us
• Track your research

Something went wrong when searching for seed articles. Please try again soon.

No articles were found for that search term.

Author, year The title of the article goes here

LITERATURE REVIEW SOFTWARE FOR BETTER RESEARCH

“This tool really helped me to create good bibtex references for my research papers”

Ali Mohammed-Djafari

Director of Research at LSS-CNRS, France

“Any researcher could use it! The paper recommendations are great for anyone and everyone”

Swansea University, Wales

“As a student just venturing into the world of lit reviews, this is a tool that is outstanding and helping me find deeper results for my work.”

Franklin Jeffers

South Oregon University, USA

“One of the 3 most promising tools that (1) do not solely rely on keywords, (2) does nice visualizations, (3) is easy to use”

Singapore Management University

“Incredibly useful tool to get to know more literature, and to gain insight in existing research”

KU Leuven, Belgium

“Seeing my literature list as a network enhances my thinking process!”

Katholieke Universiteit Leuven, Belgium

“I can’t live without you anymore! I also recommend you to my students.”

Professor at The Chinese University of Hong Kong

“This has helped me so much in researching the literature. Currently, I am beginning to investigate new fields and this has helped me hugely”

Aran Warren

Canterbury University, NZ

“It's nice to get a quick overview of related literature. Really easy to use, and it helps getting on top of the often complicated structures of referencing”

Christoph Ludwig

Technische Universität Dresden, Germany

“Litmaps is extremely helpful with my research. It helps me organize each one of my projects and see how they relate to each other, as well as to keep up to date on publications done in my field”

Daniel Fuller

Clarkson University, USA

“Litmaps is a game changer for finding novel literature... it has been invaluable for my productivity.... I also got my PhD student to use it and they also found it invaluable, finding several gaps they missed”

Varun Venkatesh

Austin Health, Australia

Our Course: Learn and Teach with Litmaps