research in the humanities

• Make a Gift
• Directories

Search form

You are here.

• Programs & Courses
• Undergraduate
• Undergraduate Research

What is Humanities Research?

Research in the humanities is frequently misunderstood. When we think of research, what immediately comes to mind for many of us is a laboratory setting, with white-coated scientists hunched over microscopes. Because research in the humanities is often a rather solitary activity, it can be difficult for newcomers to gain a sense of what research looks like within the scope of English Studies. (For examples, see Student Research Profiles .)

A common misconception about research is reinforced when we view it solely in terms of the discovery of things previously unknown (such as a new species or an archaelogical artifact) rather than as a process that includes the reinterpretation or rediscovery of known artifacts (such as texts and other cultural products) from a critical or creative perspective to generate innovative art or new analyses. Fundamental to the concept of research is precisely this creation of something new. In the humanities, this might consist of literary authorship, which creates new knowledge in the form of art, or scholarly research, which adds new knowledge by examining texts and other cultural artifacts in the pursuit of particular lines of scholarly inquiry.

Research is often narrowly construed as an activity that will eventually result in a tangible product aimed at solving a world or social problem. Instead, research has many aims and outcomes and is a discipline-specific process, based upon the methods, conventions, and critical frameworks inherent in particular academic areas. In the humanities, the products of research are predominantly intellectual and intangible, with the results contributing to an academic discipline and also informing other disciplines, a process which often effects individual or social change over time.

The University of Washington Undergraduate Research Program provides this basic definition of research:

"Very generally speaking, most research is characterized by the evidence-based exploration of a question or hypothesis that is important to those in the discipline in which the work is being done. Students, then, must know something about the research methodology of a discipline (what constitutes "evidence" and how do you obtain it) and how to decide if a question or line of inquiry that is interesting to that student is also important to the discipline, to be able to embark on a research project."

While individual research remains the most prevalent form in the humanities, collaborative and cross-disciplinary research does occur. One example is the "Modern Girl Around the World" project, in which a group of six primary UW researchers from various humanities and social sciences disciplines explored the international emergence of the figure of the Modern Girl in the early 20th century. Examples of other research clusters are "The Race/Knowledge Project: Anti-Racist Praxis in the Global University," "The Asian American Studies Research Cluster," " The Queer + Public + Performance Project ," " The Moving Images Research Group ," to name a few.

English Studies comprises, or contains elements of, many subdisciplines. A few examples of areas in which our faculty and students engage are Textual Studies , Digital Humanities , American Studies , Language and Rhetoric , Cultural Studies , Critical Theory , and Medieval Studies . Each UW English professor engages in research in one or more specialty areas. You can read about English faculty specializations, research, and publications in the English Department Profiles to gain a sense of the breadth of current work being performed by Department researchers.

Undergraduates embarking on an independent research project work under the mentorship of one or more faculty members. Quite often this occurs when an advanced student completes an upper-division class and becomes fascinated by a particular, more specific line of inquiry, leading to additional investigation in an area beyond the classroom. This also occurs when students complete the English Honors Program , which culminates in a guided research-based thesis. In order for faculty members to agree to mentor a student, the project proposal must introduce specific approaches and lines of inquiry, and must be deemed sufficiently well defined and original enough to contribute to the discipline. If a faculty member in English has agreed to support your project proposal and serve as your mentor, credit is available through ENGL 499.

Beyond English Department resources, another source of information is the UW Undergraduate Research Program , which sponsors the annual Undergraduate Research Symposium . They also offer a one-credit course called Research Exposed (GEN ST 391) , in which a variety of faculty speakers discuss their research and provide information about research methods. Another great campus resource is the Simpson Center for the Humanities which supports interdisciplinary study. A number of our students have also been awarded Mary Gates Research Scholarships .

Each year, undergraduate English majors participate in the UW's Annual Undergraduate Research Symposium as well as other symposia around the nation. Here are some research abstracts from the symposia proceedings archive by recent English-major participants.

UW English Majors Recently Presenting at the UW's Annual Undergraduate Research Symposium

For additional examples, see Student Profiles and Past Honors Students' Thesis Projects .

•   Instagram
•   Newsletter

• Privacy Policy

Buy Me a Coffee

Home » Humanities Research – Types, Methods and Examples

Humanities Research – Types, Methods and Examples

Table of Contents

Humanities Research

Definition:

Humanities research is a systematic and critical investigation of human culture, values, beliefs, and practices, including the study of literature, philosophy, history, art, languages, religion, and other aspects of human experience.

Types of Humanities Research

Types of Humanities Research are as follows:

Historical Research

This type of research involves studying the past to understand how societies and cultures have evolved over time. Historical research may involve examining primary sources such as documents, artifacts, and other cultural products, as well as secondary sources such as scholarly articles and books.

Cultural Studies

This type of research involves examining the cultural expressions and practices of a particular society or community. Cultural studies may involve analyzing literature, art, music, film, and other forms of cultural production to understand their social and cultural significance.

Linguistics Research

This type of research involves studying language and its role in shaping cultural and social practices. Linguistics research may involve analyzing the structure and use of language, as well as its historical development and cultural variations.

Anthropological Research

This type of research involves studying human cultures and societies from a comparative and cross-cultural perspective. Anthropological research may involve ethnographic fieldwork, participant observation, interviews, and other qualitative research methods.

Philosophy Research

This type of research involves examining fundamental questions about the nature of reality, knowledge, morality, and other philosophical concepts. Philosophy research may involve analyzing philosophical texts, conducting thought experiments, and engaging in philosophical discourse.

Art History Research

This type of research involves studying the history and significance of art and visual culture. Art history research may involve analyzing the formal and aesthetic qualities of art, as well as its historical context and cultural significance.

Literary Studies Research

This type of research involves analyzing literature and other forms of written expression. Literary studies research may involve examining the formal and structural qualities of literature, as well as its historical and cultural context.

Digital Humanities Research

This type of research involves using digital technologies to study and analyze cultural artifacts and practices. Digital humanities research may involve analyzing large datasets, creating digital archives, and using computational methods to study cultural phenomena.

Data Collection Methods

Data Collection Methods in Humanities Research are as follows:

• Interviews : This method involves conducting face-to-face, phone or virtual interviews with individuals who are knowledgeable about the research topic. Interviews may be structured, semi-structured, or unstructured, depending on the research questions and objectives. Interviews are often used in qualitative research to gain in-depth insights and perspectives.
• Surveys : This method involves distributing questionnaires or surveys to a sample of individuals or groups. Surveys may be conducted in person, through the mail, or online. Surveys are often used in quantitative research to collect data on attitudes, behaviors, and other characteristics of a population.
• Observations : This method involves observing and recording behavior or events in a natural or controlled setting. Observations may be structured or unstructured, and may involve the use of audio or video recording equipment. Observations are often used in qualitative research to collect data on social practices and behaviors.
• Archival Research: This method involves collecting data from historical documents, artifacts, and other cultural products. Archival research may involve accessing physical archives or online databases. Archival research is often used in historical and cultural studies to study the past.
• Case Studies : This method involves examining a single case or a small number of cases in depth. Case studies may involve collecting data through interviews, observations, and archival research. Case studies are often used in cultural studies, anthropology, and sociology to understand specific social or cultural phenomena.
• Focus Groups : This method involves bringing together a small group of individuals to discuss a particular topic or issue. Focus groups may be conducted in person or online, and are often used in qualitative research to gain insights into social and cultural practices and attitudes.
• Participatory Action Research : This method involves engaging with individuals or communities in the research process, with the goal of promoting social change or addressing a specific social problem. Participatory action research may involve conducting focus groups, interviews, or surveys, as well as involving participants in data analysis and interpretation.

Data Analysis Methods

Some common data analysis methods used in humanities research:

• Content Analysis : This method involves analyzing the content of texts or cultural artifacts to identify patterns, themes, and meanings. Content analysis is often used in literary studies, media studies, and cultural studies to analyze the meanings and representations conveyed in cultural products.
• Discourse Analysis: This method involves analyzing the use of language and discourse to understand social and cultural practices and identities. Discourse analysis may involve analyzing the structure, meaning, and power dynamics of language and discourse in different social contexts.
• Narrative Analysis: This method involves analyzing the structure, content, and meaning of narratives in different cultural contexts. Narrative analysis may involve analyzing the themes, symbols, and narrative devices used in literary texts or other cultural products.
• Ethnographic Analysis : This method involves analyzing ethnographic data collected through participant observation, interviews, and other qualitative methods. Ethnographic analysis may involve identifying patterns and themes in the data, as well as interpreting the meaning and significance of social and cultural practices.
• Statistical Analysis: This method involves using statistical methods to analyze quantitative data collected through surveys or other quantitative methods. Statistical analysis may involve using descriptive statistics to describe the characteristics of the data, or inferential statistics to test hypotheses and make inferences about a population.
• Network Analysis: This method involves analyzing the structure and dynamics of social networks to understand social and cultural practices and relationships. Network analysis may involve analyzing patterns of social interaction, communication, and influence.
• Visual Analysis : This method involves analyzing visual data, such as images, photographs, and art, to understand their cultural and social significance. Visual analysis may involve analyzing the formal and aesthetic qualities of visual products, as well as their historical and cultural context.

Examples of Humanities Research

Some Examples of Humanities Research are as follows:

• Literary research on diversity and representation: Scholars of literature are exploring the representation of different groups in literature and how those representations have changed over time. They are also studying how literature can promote empathy and understanding across different cultures and communities.
• Philosophical research on ethics and technology: Philosophers are examining the ethical implications of emerging technologies, such as artificial intelligence and biotechnology. They are asking questions about what it means to be human in a world where technology is becoming increasingly advanced.
• Anthropological research on cultural identity: Anthropologists are studying the ways in which culture shapes individual and collective identities. They are exploring how cultural practices and beliefs can shape social and political systems, as well as how individuals and communities resist or adapt to dominant cultural norms.
• Linguistic research on language and communication: Linguists are studying the ways in which language use and communication can impact social and political power dynamics. They are exploring how language can reinforce or challenge social hierarchies and how language use can reflect cultural values and norms.

How to Conduct Humanities Research

Conducting humanities research involves a number of steps, including:

• Define your research question or topic : Identify a question or topic that you want to explore in-depth. This can be a broad or narrow topic, depending on the scope of your research project.
• Conduct a literature review: Before beginning your research, read extensively on your topic. This will help you understand the existing scholarship and identify gaps in the literature that your research can address.
• Develop a research methodology: Determine the methods you will use to collect and analyze data, such as interviews, surveys, archival research, or textual analysis. Your methodology should be appropriate to your research question and topic.
• Collect data: Collect data using the methods you have chosen. This may involve conducting interviews, surveys, or archival research, or analyzing primary or secondary sources.
• Analyze data: Once you have collected data, analyze it using appropriate methods. This may involve coding, categorizing, or comparing data, or interpreting texts or other sources.
• Draw conclusions: Based on your analysis, draw conclusions about your research question or topic. These conclusions should be supported by your data and should contribute to existing scholarship.
• Communicate your findings : Communicate your findings through writing, presentations, or other forms of dissemination. Your work should be clearly written and accessible to a broad audience.

Applications of Humanities Research

Humanities research has many practical applications in various fields, including:

• Policy-making: Humanities research can inform policy-making by providing insights into social, cultural, and historical contexts. It can help policymakers understand the impact of policies on communities and identify potential unintended consequences.
• Education: Humanities research can inform curriculum development and pedagogy. It can provide insights into how to teach critical thinking, cross-cultural understanding, and communication skills.
• Cultural heritage preservation: Humanities research can help to preserve cultural heritage by documenting and analyzing cultural practices, traditions, and artifacts. It can also help to promote cultural tourism and support local economies.
• Business and industry: Humanities research can provide insights into consumer behavior, cultural preferences, and historical trends that can inform marketing, branding, and product design.
• Healthcare : Humanities research can contribute to the development of patient-centered healthcare by exploring the impact of social and cultural factors on health and illness. It can also help to promote cross-cultural understanding and empathy in healthcare settings.
• Social justice: Humanities research can contribute to social justice by providing insights into the experiences of marginalized communities, documenting historical injustices, and promoting cross-cultural understanding.

Purpose of Humanities Research

The purpose of humanities research is to deepen our understanding of human experience, culture, and history. Humanities research aims to explore the human condition and to provide insights into the diversity of human perspectives, values, and beliefs.

Humanities research can contribute to knowledge in various fields, including history, literature, philosophy, anthropology, and more. It can help us to understand how societies and cultures have evolved over time, how they have been shaped by various factors, and how they continue to change.

Humanities research also aims to promote critical thinking and creativity. It encourages us to question assumptions, to challenge dominant narratives, and to seek out new perspectives. Humanities research can help us to develop empathy and understanding for different cultures and communities, and to appreciate the richness and complexity of human experience.

Overall, the purpose of humanities research is to contribute to a deeper understanding of ourselves, our communities, and our world. It helps us to grapple with fundamental questions about the human experience and to develop the skills and insights needed to address the challenges of the future.

When to use Humanities Research

Humanities research can be used in various contexts where a deeper understanding of human experience, culture, and history is required. Here are some examples of when humanities research may be appropriate:

• Exploring social and cultural phenomena: Humanities research can be used to explore social and cultural phenomena such as art, literature, religion, and politics. It can help to understand how these phenomena have evolved over time and how they relate to broader social, cultural, and historical contexts.
• Understanding historical events: Humanities research can be used to understand historical events such as wars, revolutions, and social movements. It can provide insights into the motivations, experiences, and perspectives of the people involved, and help to contextualize these events within broader historical trends.
• Promoting cultural understanding : Humanities research can be used to promote cross-cultural understanding and to challenge stereotypes and biases. It can provide insights into the diversity of human experiences, values, and beliefs, and help to build empathy and mutual respect across different cultures and communities.
• Informing policy-making: Humanities research can be used to inform policy-making by providing insights into social, cultural, and historical contexts. It can help policymakers understand the impact of policies on communities and identify potential unintended consequences.
• Promoting innovation and creativity : Humanities research can be used to promote innovation and creativity in various fields. It can help to generate new ideas, perspectives, and approaches to complex problems, and to challenge conventional thinking and assumptions.

Characteristics of Humanities Research

Some of the key characteristics of humanities research:

• Focus on human experience: Humanities research focuses on the study of human experience, culture, and history. It aims to understand the human condition, explore human values and beliefs, and analyze the ways in which societies and cultures have evolved over time.
• Interpretive approach: Humanities research takes an interpretive approach to data analysis. It seeks to understand the meaning behind texts, artifacts, and cultural practices, and to explore the multiple perspectives and contexts that shape human experience.
• Contextualization : Humanities research emphasizes the importance of contextualization. It seeks to understand how social, cultural, and historical factors shape human experience, and to place individual phenomena within broader cultural and historical contexts.
• Subjectivity : Humanities research recognizes the subjective nature of human experience. It acknowledges that human values, beliefs, and experiences are shaped by individual perspectives, and that these perspectives can vary across cultures, communities, and time periods.
• Narrative analysis : Humanities research often uses narrative analysis to explore the stories, myths, and cultural narratives that shape human experience. It seeks to understand how these narratives are constructed, how they evolve over time, and how they influence individual and collective identity.
• Multi-disciplinary: Humanities research is often interdisciplinary, drawing on a range of disciplines such as history, literature, philosophy, anthropology, and more. It seeks to bring together different perspectives and approaches to understand complex human phenomena.

Advantages of Humanities Research

Some of the key advantages of humanities research:

• Promotes critical thinking: Humanities research encourages critical thinking by challenging assumptions and exploring different perspectives. It requires researchers to analyze and interpret complex texts, artifacts, and cultural practices, and to make connections between different phenomena.
• Enhances cultural understanding : Humanities research promotes cross-cultural understanding by exploring the diversity of human experiences, values, and beliefs. It helps to challenge stereotypes and biases and to build empathy and mutual respect across different cultures and communities.
• Builds historical awareness: Humanities research helps us to understand the historical context of current events and social issues. It provides insights into how societies and cultures have evolved over time and how they have been shaped by various factors, and helps us to contextualize current social, political, and cultural trends.
• Contributes to public discourse: Humanities research contributes to public discourse by providing insights into complex social, cultural, and historical phenomena. It helps to inform public policy and public debate by providing evidence-based analysis and insights into social issues and problems.
• Promotes creativity and innovation: Humanities research promotes creativity and innovation by challenging conventional thinking and assumptions. It encourages researchers to generate new ideas and perspectives and to explore alternative ways of understanding and addressing complex problems.
• Builds communication skills: Humanities research requires strong communication skills, including the ability to analyze and interpret complex texts, artifacts, and cultural practices, and to communicate findings and insights in a clear and compelling way.

Limitations of Humanities Research

Some of the key limitations of humanities research:

• Subjectivity: Humanities research relies heavily on interpretation and analysis, which are inherently subjective. Researchers bring their own perspectives, biases, and values to the analysis, which can affect the conclusions they draw.
• Lack of generalizability : Humanities research often focuses on specific texts, artifacts, or cultural practices, which can limit the generalizability of findings to other contexts. It is difficult to make broad generalizations based on limited samples, which can be a challenge when trying to draw broader conclusions.
• Limited quantitative data : Humanities research often relies on qualitative data, such as texts, images, and cultural practices, which can be difficult to quantify. This can make it difficult to conduct statistical analyses or to draw quantitative conclusions.
• Limited replicability: Humanities research often involves in-depth analysis of specific texts, artifacts, or cultural practices, which can make it difficult to replicate studies. This can make it challenging to test the validity of findings or to compare results across studies.
• Limited funding: Humanities research may not always receive the same level of funding as other types of research. This can make it challenging for researchers to conduct large-scale studies or to have access to the same resources as other researchers in different fields.
• Limited impact : Humanities research may not always have the same level of impact as research in other fields, particularly in terms of policy and practical applications. This can make it challenging for researchers to demonstrate the relevance and impact of their work.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Documentary Research – Types, Methods and...

Scientific Research – Types, Purpose and Guide

Original Research – Definition, Examples, Guide

Historical Research – Types, Methods and Examples

Artistic Research – Methods, Types and Examples

writingxhumanities

A "how to" guide for UC Berkeley writers

Doing Research

In the humanities, as in the sciences, research begins with formulating a hypothesis about a subject and “testing” it out by engaging with evidence. But the nature of the evidence, the method of testing, and the kind of knowledge produced in humanities research differ from those in the sciences.

Rather than using quantitative or qualitative evidence as researchers do in the sciences, humanities researchers rely for evidence on their own close readings of primary and secondary texts. Whereas research in the sciences is grounded in the empirical method, humanities research uses a diverse array of methodologies, sometimes combining historical, conceptual, and/or critical methodologies in a single study. And instead of attempting to “prove” their hypotheses once and for all, humanities researchers develop arguments using their hypotheses in order to contribute and extend to ongoing critical conversations. These conversations themselves, one might say, constitute knowledge in the humanities.

Doing research into what has already been written about on your topic will allow you not only to situate your hypothesis within these conversations, but also to come up with questions that haven’t yet been asked. Through critical reading, you can identify gaps in previous writing on your topic and orient your own project to address these gaps if they seem like meaningful oversights. Your aim as a humanities researcher, then, is to develop a fuller understanding of your subject matter in dialogue with other scholars, and thus to move this ongoing conversation forward.

To develop a research question, you might start by making a bulleted list of topics or issues that you might want to pursue, based on your interests and the concerns of your course. When making this list, you should be expansive. Don’t limit yourself: your goal at this stage is to generate ideas, not evaluate them.

Next, you could try free-writing on the themes you see emerging from your list of potential topics. Are there any areas of interest that seem to repeat or echo? Can you start to create umbrella terms? Are you noticing friction within or between some of these concerns? These are just a few of the questions that may lead to a more focused line of investigation.  

Then, see if you can reframe your topic or theme as a research question by asking “How?” or “Why?” Keep in mind that your research question should be debatable and defendable : you may find that other scholars have very different points of view on this question or its answer. This is a good sign, and an invitation for you to step in and contribute to the scholarly discussion.

  Here are some helpful suggestions for broadening, narrowing, or otherwise tweaking a preliminary research topic so that you will be poised to write a well-focused essay.

And here is a concise overview that will guide you on the road to finding your research question . 

To engage fully with your secondary sources, you will need to read critically. But reading sources critically doesn’t necessarily mean disagreeing with them or reading them in a negative way. Rather, it means analyzing the details of an argument-driven piece of popular or scholarly writing that you may ultimately use towards fashioning your own argument. It means asking how these details reflect the assumptions, values, and stakes of the writer’s argument. Reading critically means navigating between doubting and believing what you read. 

When reading critically, we treat what we read less like objective, self-evident data and more like evidence being presented to persuade us of something. Sometimes it’s obvious that what we’re reading is trying to persuade us of something—think of an editorial that takes an explicitly controversial stance or a scholarly article that clearly signposts its position using phrases like “here I argue…” Other texts may make their arguments more subtly or indirectly. But either way, reading critically involves considering how the components and structure of the argument contribute to its actual or intended effect upon the reader.

One approach to reading critically is to investigate a text’s “ way of thinking .” Rather than reading simply for comprehension or information, you can also attend to a text’s claims, contexts, kinds of reasoning, and evidence in order to evaluate the effectiveness of its argument and to think about what you might add to the discussion or redirect it.  

Another approach involves asking questions about the relationship between the parts and the whole of an argument. These questions might address the role of pattern, process, sequence, causality, and other elements of argumentation and structure.

Here are some tips for reading secondary sources and some step-by-step exercises for reading critically.

Since you can’t write an interesting research paper without engaging with other scholars’ ideas, you’ll have to find secondary sources that are accurate and pertinent to your argument.

You’ll almost certainly want to start online, but the internet is an enormously complex—and just plain enormous!—compendium of resources. You can use Wikipedia to check out some basic facts and get some rough background information, but it’s not a reliable, scholarly source–see What’s Wrong With Wikipedia . Likewise, Googling, even using Google Scholar , can only get you so far, since there are valuable scholarly resources out there that are discoverable only by using humanities-specific Web directories ( Voice of the Shuttle , started in 1994, is the granddaddy of all online humanities research directories) and databases ( JSTOR , Project Muse , and the MLA International Bibliography , among others), along with the Berkeley library’s online catalogue .

Before you begin searching in a library catalog or journal database, familiarize yourself with using advanced search functions , such as “Boolean operators” (the basic ones are AND, OR, and NOT), truncation and wildcards (*, !, ?, or #), and keywords and subject headings. Here are some suggestions for coming up with searchable keywords .

You can read many of the sources (especially the articles) that you find through databases entirely online; you’ll need to track down other sources (like most books and book chapters) in the library. Once you’ve found a book in the library, take some time to scan nearby shelves for other books on the same topic that might not have come up in your catalog search. Also scan the bibliographies and footnotes of books and articles you’ve already found: these are great places to find more sources on your topic.

“Good” here means two different things: “good” as in relevant to your topic and useful for the kind of argument you’re making, but also “good” as in scholarly, reputable, and current. 

To determine the usefulness and relevance of a secondary source, begin by skimming it .  If the source at hand is an article, read the abstract. If it doesn’t have an abstract, read the opening paragraph, the section headings, and/or first sentences of a few of the body paragraphs, plus the concluding paragraph. If it is a book, read the blurb on the back cover, the table of contents, and the beginning and ending of the introduction and/or conclusion. From this quick but strategic dip into a secondary source, you should be able to glean something about its overall argument and whether it provides useful material for your own project. 

Remember: don’t discard a scholarly source just because it contradicts what you want to say. Do you disagree with its premises, its use of evidence, or its conclusions? Any of these points of disagreement might provide a foothold for advancing your own argument. Or if a secondary source does not address your primary text directly, does it put forth a theory or provide information that will help you to analyze your text?

Your quick dip into a source may also give you a feel for the quality of its scholarship. But there are also many concrete questions you can ask about a source to evaluate its reliability. What are the credentials of the author? Who is the publisher and the intended audience? When was it published? What sources does it cite? For a more detailed breakdown of ways to evaluate print and online sources, consult one or more of the following (reliable!) resources:

Evaluating Resources (UC Berkeley Library) Quality of Sources (Dartmouth College) Critically Analyzing Information Sources (Cornell University Library) Evaluating Web Pages (Cornell University Library) Evaluating Print vs. Internet Sources (Purdue OWL) Interpreting Sources (University of Michigan)

Skimming is a valuable reading technique. It can give you a general sense of what a  text is about, and it can help you to locate key arguments and passages relevant to your own research and writing. Skimming is also useful when you’re short on time. Let’s be honest: skimming is sometimes the only reading you have time to do!

But skimming does not mean speed-reading an entire text; zooming through a whole story or story at breakneck speed is, generally speaking, a waste of your time. Instead, skim strategically. Read the first and last pages of the whole work, and then read the beginning and ending of each chapter. Look for words, especially names, which appear repeatedly, dipping selectively into the text to get a sense of the style and “texture” of the writing.

Scholarly and critical texts, typically used as secondary sources, often have abstracts that give a summary of their main argument, and they also often have section headings, topic sentences, and transitions to guide readers through their discussion. Introductions are good places to look for roadmaps and thesis statements, while conclusions often summarize the whole argument. Focusing on these signposts will help you get the gist of an article or book without reading the whole thing in depth. These techniques are especially useful when your research turns up multiple secondary sources and you need to select the ones most relevant to your own topic. 

You can find some pointers on skimming in this very thorough set of instructions on How to Read a Book . 

Taking notes is an essential part of doing research. Obviously, the notes you take should provide a clear record of what you’ve read. But the very act of note-taking can help you to develop your thinking about your research question and ultimately to use evidence to support your argument.

To help yourself read secondary sources critically, you should take thorough but not overly detailed notes . Make sure to record key terms but don’t write out everything word for word; paraphrase whenever possible to make sure you have a grasp of relevant points; and don’t highlight or underline without also making marginal notes about the significance of the marked words or passages and any questions you might have about them. 

Make sure to include complete bibliographic data with your notes on each source so that you can retrace your steps if you end up needing to go back for more information or to check that you are quoting accurately. There are many different formats for note-taking—in the margins of the text or on Post-Its; on paper or in a Word document—and you’ll need to experiment to figure out what works best for you. You will also need to work out a good system for organizing and reviewing the notes you take. If you are juggling multiple sources, for example, consider using index cards or a citation management software ( Zotero and Mendeley are two popular ones) to organize and annotate your sources. You can even use these programs to generate properly formatted endnotes and Works Cited lists.

You can use your notes as the basis for an annotated bibliography ; here are some additional resources on writing annotated bibliographies.  Even if your instructor does not require you to produce a formal bibliography as a preliminary phase of your research paper, your own notes will still serve as the basis for situating and differentiating your own argument within a field of existing literature. Your notes should help you to take a strong, well-informed, and original stance in your writing. 

Research, like writing, is not a linear process. You will probably begin with a broad topic that will gradually become more focused over the course of your research and writing. You’ll then have to do additional research on this more focused and developed version of your original topic. Remember that the evolution of your topic depends not only on your reading but also on your writing throughout the research process. Such preliminary writing might include producing an annotated bibliography and/or a prospectus, as well as in less formal modes such as freewriting, mind mapping, outlining, and drafting.

The process of researching and refining, and researching and refining again, could go on endlessly, but don’t let it. You have to strike a balance between your responsibility as a researcher—citing and integrating the sources most relevant to your topic (not just the first three sources you find!)—and what is humanly possible. You can’t read every source ever written on your topic, and you shouldn’t. Remember that your goal is to develop and answer your specific research question. When you have a good sense of how your argument fits into the existing conversation, you can stop.

How do you know when enough is enough? Here are some diagnostic questions to help you answer the question: “When Can I Stop Researching?”

It’s not enough to use secondary sources merely for factual or historical information, although this is certainly one thing that sources can do for you. It’s also not enough to simply say “I agree” or “I disagree” with what other scholars have written, although this can be a starting place for developing a tentative research question and even a tentative thesis. Through thoughtful selection of and engagement with secondary sources, you can participate in ongoing critical conversations and even propel them in new directions. (Of course, the initial phases of your attempt to enter one of these conversations might reveal to you that you need to find different or additional sources!)

You might begin by considering how your sources are using their own sources . They probably cite other research to support their own claims (“Yes, and…”), to make a new claim instead (“No, because…”), or acknowledge other arguments in order to show a critical “gap” in the conversation, which their own argument will fill (“Although X and Y, nonetheless Z…”). In turn, you can use the specifics of their critical positioning to situate yourself in the discussion.

There are any number of ways of navigating secondary sources effectively , most of them involving either limited alignment or partial dissent. Some of these modes of engagement include adopting a term, adapting a theory, and changing the question, using moves that might be described as “picking a fight,” “drawing battle lines,” “piggybacking,” “leapfrogging,” and “matchmaking.” Here are two overviews of basic maneuvers and fundamental strategies for using secondary sources to develop an argument.

In order to effectively use your secondary sources to develop an argument, you need to clearly and gracefully integrate material from those sources into your sentences. Whether you paraphrase, quote, or summarize this material, you must fully signpost its relation to your argument, whether via limited alignment or focused dissent. As when you are making an argument about a primary text , you need to ensure that you are analyzing your source rather than leaving it undigested for your reader. Toward that end, there are a few things you should consider when integrating sources:

• Evaluate whether you need to quote, paraphrase, or summarize . Here are some tips on when and how to   summarize effectively. And here is some advice for determining whether   paraphrasing or quoting will best serve your purposes.
• When paraphrasing, be sure to do so accurately and fairly . You can practice this skill by doing this quick paraphrasing exercise . Just as when you quote word-for-word from a text, it is crucial to avoid plagiarizing when paraphrasing . For more information on avoiding plagiarism when quoting and paraphrasing, jump to the next section, “How do I responsibly cite my sources and avoid plagiarism?”
• Always be sure to Introduce, Cite, and Explain (“ICE,”for short) your evidence! Another way that you can ICE your evidence is by “sandwiching ” it between a claim and analysis, much as you would introduce and analyze a primary text.
• Familiarize yourself with guidelines about when quoting is most effective rhetorically and how to avoid quoting extraneous material . Determine which type of quotation —a block, spliced, or signaled quote—best suits your purposes.
• Before you submit your essay, be sure to review the   nuts and bolts of integrating source material into your prose, i ncluding using proper punctuation. Bonus: here’s a list of signal verbs and phrases to help you synthesize the words and ideas of other scholars.

Whenever you use language, ideas, or arguments from others, you need to cite them. The Berkeley Campus Student Code of Conduct frames improper citation as a form of academic misconduct —failure to cite one’s sources properly constitutes the theft of intellectual property. Plagiarism can make you subject to penalties ranging from failing an assignment to failing a course. Under some circumstances, suspension or even dismissal from the university may be imposed as a sanction.

Correct citation is not only a matter of personal and scholarly integrity. There’s another important reason to cite the arguments of other scholars who have written on your topic: citing them accurately and fully allows other scholars to track down those sources themselves . . . and it also allows you to check your own work. This practice might be compared to writing a detailed account of your experimental method in the sciences—citation is what gives research in the humanities its “replicability.” In sum, citation allows the scholarly conversation to continue moving forward. 

What counts as plagiarism? It can take a wide variety of forms, not just the act of submitting the work of another person as your own. Copying language directly from a source, like “patch-working” together sentences or paragraphs or ideas from multiple sources, is also plagiarism. And paraphrasing or rewording of sources without attribution can also be considered plagiarism. Even if you discover that a source makes the same point that you made first on your own, you still have to cite it! 

The easiest way to avoid plagiarism? If you’re in doubt about whether or not you should cite something, cite it! Accidental plagiarism is still considered plagiarism and is subject to the same repercussions. Keeping scrupulous notes throughout the research and writing process will help you to remember which ideas and phrasings are your own and which came directly from your sources. 

There are several different formats for citing sources; your instructor will tell you which style—MLA style, Chicago, style etc.—to use. For more details on MLA style, the most common citation format in the humanities, try the Purdue OWL or The MLA Style Center .

There is a wealth of resources–guides, grants, mentoring, awards, and more–for undergraduate research across the humanities at UC Berkeley. Here are some of the best-known ones, though you may be able to find others under the auspices of specific departments or programs.

You can begin searching for materials through the Library Guide to Research , which is organized by discipline or area of study. You can also find here the contact information for the particular UC Berkeley librarian who is an expert on research offerings in each subject area, and who can guide you to these resources. There are many research databases and subject-specific guides listed here under each heading. One that you won’t find listed there, A Guide to the Archive Resources of the Bay Area , lists some of the primary research offerings at UC Berkeley’s Bancroft Library and other university archives around the Bay. 

The Office of Undergraduate Research and Scholarships (OURS) is a clearing house for information about university-wide research grants and mentoring programs across the disciplines including the Haas Scholars Program , Student Mentoring and Research Teams (SMART), Summer Undergraduate Research Fellowships (SURF), Underrepresented Researchers of Color (UROC), and Undergraduate Research Apprentices Project (URAP). They have databases with information about many other internal and external grants and programs, as well as offering help with finding and applying for these. Don’t forget to check with individual departments for major-specific prizes. UC Berkeley’s English Department, for example, offers travel grants for undergraduates to attend conferences and to do research at archives elsewhere in the U.S. and abroad. 

When your research is done or close to being finished (the submission deadline is in mid-April!), you should consider applying for the Library Prize for Undergraduate Research . 

For additional materials, go to  Teaching Research  in the  For Instructors  section of this website.

• Divisions and Offices
• Grants Search
• Manage Your Award
• NEH's Application Review Process
• Professional Development
• Grantee Communications Toolkit
• NEH Virtual Grant Workshops
• Awards & Honors
• American Tapestry
• Humanities Magazine
• NEH Resources for Native Communities
• Search Our Work
• Office of Communications
• Office of Congressional Affairs
• Office of Data and Evaluation
• Budget / Performance
• Contact NEH
• Equal Employment Opportunity
• Human Resources
• Information Quality
• National Council on the Humanities
• Office of the Inspector General
• Privacy Program
• State and Jurisdictional Humanities Councils
• Office of the Chair
• NEH-DOI Federal Indian Boarding School Initiative Partnership
• NEH Equity Action Plan
• GovDelivery

Division of Research Programs

The Division of Research supports scholarly research that advances knowledge and understanding of the humanities. Through twelve annual funding opportunities, awards are made to scholars—individuals, collaborative teams, or institutions—working on research projects of significance to specific humanities fields and to the humanities as a whole. The projects that the division supports are as diverse as America itself:  editions of the Dead Sea Scrolls , the  history of “The Star Spangled Banner,”  and the  autobiography of Mark Twain .

NEH’s  Fellowships  program was established over 50 years ago and was the first award offered by the Endowment. Since then, approximately seven thousand books have been written by NEH fellows. In the academic world, “getting an NEH” is a shorthand for receiving an NEH Fellowship, which indicates the award’s widely respected reputation and prestige. Recognizing the specific needs of certain scholars,  Awards for Faculty  offer more flexible fellowships to those employed at Historically Black Colleges and Universities, Hispanic Serving Institutions, and Tribal Colleges and Universities.  Summer Stipends  offer two-month awards to allow scholars to take a shorter break to pursue focused research.  Public Scholar  awards encourage writing books for a wide readership. Placing NEH at the forefront of innovative methods in the humanities,  the  Fellowships for Digital Publication  support projects that require digital expression and digital publication. Books resulting from all of these grants regularly earn awards and recognition. NEH-funded work has been honored with Pulitzer Prizes, Bancroft Prizes, as well as awards from academic associations across the country and accolades from reviewers in major newspapers and literature journals.

While Research Programs is the only NEH division to make awards to individuals, institutional grants are also available.  Collaborative Research  supports projects by teams of scholars.  Scholarly Editions and Scholarly Translations  provides funding for time-intensive editing projects such as the  Papers of George Washington , and  Fellowship Programs at Independent Research Institutions  provides American scholars access to unique collections at American centers for humanities research around the world.  Humanities Research Centers on Artificial Intelligence  supports the creation of new collaborative research centers exploring the ethical, legal, or societal implications of artificial intelligence.

Teachers, too, make use of NEH-supported research in their classrooms – often with the aid of the web resources and books resulting from many projects. For example,  the papers of William F. Cody (Buffalo Bill)  and the  Freedmen and Southern Society Project  on the history of Emancipation are used in U.S. history classes, the  papers of Albert Einstein  in physics, and the literary works and letters of writers such  Willa Cather ,  Ernest Hemingway , and  Samuel Beckett  are taught in English classes. Archaeology projects unearth artifacts used by museum curators in mounting exhibitions that teach us about life in ancient civilizations. Translations of materials in other languages bring little-known foreign works ranging from  ancient Roman graffiti  in Pompeii to  letters of the Dakota people , and contemporary  Ukrainian literature  to American readers. Documentarians, artists, and producers of all sorts of fiction and nonfiction media rely on new research findings in many fields—American history, literature, music, the history of science and technology—to inform and inspire their audiences. Projects like these add to the existing store of knowledge and reach every area of the humanities.

Featured Content

Celebrating Día de los Muertos : Humanities Research on Mexican History, Literature, and Culture

Most recent from the division of research programs, humanities researchers celebrate national native american heritage month, the division of research celebrates lgbtqia+ history month, hispanic heritage in the humanities: recent projects funded by the division of research programs, other people’s colleges: an interview with neh fellow ethan w. ris.

• Skip to main content
• Skip to footer

UCI Humanities Core

Research in the Humanities

By Matt Roberts Revised for 2018-19

Introduction

Welcome to the University of California, Irvine! As a student enrolled in the Humanities Core program, you will study a variety of cultural artifacts related to the theme of “Empire and Its Ruins.” Research will be important to your engagement with course material and topics, and the UCI Libraries is here to help you.

The UCI Libraries is home to several research librarians, who can provide you with expert service. To learn more, please visit the UCI Libraries’ Directory of Research Librarians .

To take advantage of the UCI Libraries’ resources for Humanities Core, please visit the Humanities Core Research Guide . You find the guide by visiting the UCI Libraries’ home page. Then click on the “ Research Guides ” Quick Link.

What is Humanities Research?

It is worth asking what it means to do research in the Humanities. As David Pan (Professor of German, UCI School of Humanities) writes, “[i]nterpretation is the primary method of the humanities because the meaning that humanities scholars search for is not a constant one. Rather, standards of meaning change when one moves in time and space from one cultural context to another. Negotiating this movement is the primary task in humanities inquiry” (6) .

As Pan emphasizes, humanities work is an interpretive process. For instance, for your first major writing assignment, you must perform a close reading of a passage from the Aeneid . You will describe how the selection’s formal elements—such as symbolism or diction—support an overarching theme in the epic as a whole. While this assignment does not require that you conduct research related to the  Aeneid , it nevertheless invites you to research the meaning of “symbolism” or “diction.” Let us therefore assess our information need and design a process to find the definition of, for example, diction.

Question: Where do we find the definition of a word or specialized term such as “diction”?

Answer: We find definitions in a dictionary.

Note that as the  Aeneid is an epic poem, it is important to determine what “diction” means within a literary context. Therefore, let’s consult a dictionary of literary terms.

Question : How do we find a dictionary of literary terms?

Answer: Consult the UCI Libraries’ webpage . Then search Library Search , the new finding aid for the UCI Libraries’ catalog. Since we don’t know the exact title of the dictionary that we might use, we can simply search for “dictionary of literary terms.”

We must choose the title that best matches our information need. I chose The Oxford Dictionary of Literary Terms because Oxford University Press publishes exemplary reference materials. The green “Available Online” link takes me directly to this resource. I can then use the resource to search for the definition of “Diction.”

So, what have we learned?

• Research begins with a question (“What is diction?”).
• The research question(s) that we ask helps us to discover information.
• The kind of information that we need to discover will often determine what UCI Libraries’ resource to consult (e.g., a book, reference resource, database, etc.).
• Discovering information helps us to interpret an object of study.
• The interpretation of the object of study enables us to craft clearly argued responses to writing assignments.

In other words, humanities research is a strategic exploration whereby information discovery facilitates your interpretive abilities.

Doing Research: Primary Sources and Secondary Sources

As we have already noticed, the research process involves the ability to discover a source that provides information of some kind. It is therefore important to distinguish between sources so that you can readily access that information.

Scholars typically distinguish between kinds of sources, particularly between primary and secondary sources. As you will be reading a variety of texts this year, it is important to recognize that texts are not objectively primary or secondary. Ultimately, the extent to which sources are primary depends upon the questions you ask about them and the way that you use them (Arndt 93). For instance, a primary source is an object that bears witness to a historical event. Primary sources therefore call us to consider how they make meaning of the event to which they bear witness. On the other hand, secondary sources interpret a primary source. For example, if you were to write a paper that interprets how diction in the  Aeneid characterizes human experience, you would be creating a secondary source.

Determining the difference between a primary source and a secondary source can be difficult. Let us explore each type of source in greater detail so that we can begin to understand what types of question we can ask of primary sources, and how we can engage with them in order to research secondary source material.

Primary Sources

On your syllabus, you will find a variety of texts, or sources, that you will read over the course of the year. In the fall quarter, these texts include, but are not limited to:

• Virgil’s Aeneid  (19 B.C.E.)
• Jean-Jacques Rousseau’s “Discourse on the Origin and Foundations of Inequality among Men” (1754/1755)
• J.M. Coetzee’s  Waiting for the Barbarians  (1980)

These are generally considered to be primary sources, but how can you be sure? To answer this question, consult the Primary Sources Research Guide , curated by the UCI Libraries’ History Librarian, Madelynn Dickerson.

According to this Research Guide, primary sources can include diaries, memoirs, letters, newspapers, speeches, interviews, government documents, photographs, works of art, video recordings, maps, manuscripts, data, and physical artifacts.

Question: Is the  Aeneid a primary source?

Answer: Yes. Most often it qualifies because it is a work of art, specifically an epic poem. However, one might use the Fagles’ version of The Aeneid as a secondary source, demonstrating how the translation is an interpretation of the original Latin text and using that analysis to offer one’s own interpretation of the epic poem. You can learn more about the interpretive politics of translation from Giovanna Fogli and Nuccia Malinverni’s chapter on “Translation” in your  Writer’s Handbook .

Throughout the year, you will be encouraged to find primary sources related to the topic of Empire and Its Ruins. To do this, it is useful to visit the Humanities Core Research Guide . Select the “ Primary Sources and Artifacts” tab to locate primary source discovery resources.

Additionally, you will have the unique opportunity to visit the Special Collections and Archives .

Follow the link to the SCA webpage, and you can discover a variety of primary source collections. Be sure to also visit the UCI Libraries’ Southeast Asian Archive webpage for a wealth of primary sources related to Empire and Its Ruins.

You can also use Library Finder to find primary sources and cultural artifacts in the SCA collection.

When analyzing a primary source, it is important to ask certain questions. Questions such as those listed below will help you to discover information, which you can use to interpret a primary source. In other words, questions such as these will help you to determine how a primary source makes meaning of the event to which they bear witness.

• Who made the primary source? What was that person’s race, gender, class? How, if at all, would that matter within the historical period in which the source is created? (Authorship)
• Where, when, and why was the primary source written/made? Does it describe specific attitudes of a historical period and place? What motivated its production? (Historical Context)
• For whom was the primary source written/made? For public or private use? Was it reproduced for a mass audience? How might that audience have used or responded to this source? (Audience)
• Of what is the primary source made? How might that shape how the source is understood and interpreted? (Materiality)
• What are the limitations of this type of source? What can’t it tell us? For whom is it not made available?

Simple questions such as these can help you to formulate more sophisticated research questions or topics of inquiry. You may therefore want to examine other primary sources or search for how expert scholars interpret the primary source that interests you.

Secondary Sources

When you do research, you are embarking on a journey that requires you to engage with how others interpret the primary source or sources that interest you. Whereas primary sources are original works that document an event as it took place, a secondary source interprets a primary source, often long after it was created. Secondary sources include, but are not limited to: Peer-reviewed academic books; chapters published in peer-reviewed academic books; peer-reviewed journal articles; newspaper and magazine articles published after, and therefore not explicitly covering, a historical event. Peer-reviewed works are considered to be most reliable in academic settings because they have been scrutinized and vetted by scholars to determine if the research presented makes a significant contribution.

Question : How do we find peer-reviewed journal articles?

Answer: Let’s return to the Humanities Core Research Guide, where we will find links to, and descriptions of, the databases related to Humanities Core. Select the “ Find Articles and Secondary Sources” tab to find peer-review journal articles.

I began my search with Academic Search Complete , which covers a wide-range of topics including social sciences, humanities, education, and science. It is therefore a good place to begin to do secondary source research. However, you can consult other databases such as the MLA International Bibliography , Project Muse , or JSTOR .

The above screen shot indicates how one might find scholarly articles related to the topic of the  Aeneid and Empire. I also checked the “Scholarly (Peer Reviewed) Journals” box to limit my results. This search populates the following result list:

The article entitled “ Epic and Empire” appears directly related to our search. Selecting that article will open up the record for the article.

For this article, we can click on the “PDF Full Text” icon and download the article. For articles that do not have a link to the PDF, please click on the “UC E-Links” icon to search for alternative methods of access.

Please remember that secondary sources are interpretations. Consequently, as a researcher, situate yourself in an active role. In this way, you will understand that secondary sources should not speak for you or instead of you. Rather, you should integrate them into your analysis of a primary source to demonstrate your own contribution to a scholarly conversation on a topic that interests you. We do not do research simply to communicate how others have interpreted a primary source. We do research to discover how to interpret something for ourselves.

• Primary sources are original sources created at the time a historical event occurs and are directly associated with their creator. A primary source is the subjective interpretation of a witness to an event.
• Secondary sources are scholarly or popular sources that interpret a primary source and can be created long after the primary source was created.
• Research is a process that requires us to question and interrogate the information that we discover in order to interpret an object of study. It requires the evaluation and integration of both primary and secondary sources.
• The UCI Libraries have several research resources that can help you to discover and choose primary sources and secondary sources that interest you.
• Research Librarians at the UCI Libraries are here to help you!

Please note that you will be able to take advantage of several of the UCI Libraries’ instruction and reference services throughout the year. You can meet with research librarians who specialize in specific disciplines to learn more about conducting searches and vetting sources.

Works Cited

Arndt, Ava. “What to do with Primary Sources.” Humanities Core Writer’s Handbook: War, edited by Larisa Castillo, Pearson, 2014, pp. 90-94.

Baldick, Chris. “Diction.” The Oxford Dictionary of Literary Terms . 3 rd ed., 2008. doi:10.1093/acref/9780199208272.001.0001. Accessed 12 September 2016.

Imamoto, Rebecca. Primary Sources for History: Primary Sources . University of California, Irvine. Sept. 2016. http://guides.lib.uci.edu/primary_sources. Accessed 12 September 2016.

Pan, David. “What are the Humanities?” Humanities Core Writer’s Handbook: War , edited by Larisa Castillo, Pearson, 2014, pp. 5-9.

Quint, David. “Epic and Empire.” Comparative Literature , vol. 26, no. 1, 2001, pp. 620-26.1. Academic Search Complete . Web. 16 Sept. 2016. http://search.ebscohost.com/login.aspx?direct=true&db=a9h&AN=7073526&site=ehost-live&scope=site. Accessed 16 September 2016.

Roberts, Matthew. Humanities Core Course . University of California, Irvine. Sept. 2016. http://guides.lib.uci.edu/primary_sources. Accessed 12 September 2016.

185 Humanities Instructional Building, UC Irvine , Irvine, CA 92697 [email protected]   949-824-1964   Instagram Privacy Policy

Instructor Access

• Search Menu
• Advance articles
• Author Guidelines
• Submission Site
• Open Access
• Why Publish?
• About Research Evaluation
• Editorial Board
• Advertising and Corporate Services
• Journals Career Network
• Self-Archiving Policy
• Dispatch Dates
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

• 1. Introduction
• 4. Discussion
• Acknowledgements

Four types of research in the humanities: Setting the stage for research quality criteria in the humanities

• Article contents
• Figures & tables
• Supplementary Data

Michael Ochsner, Sven E. Hug, Hans-Dieter Daniel, Four types of research in the humanities: Setting the stage for research quality criteria in the humanities, Research Evaluation , Volume 22, Issue 2, June 2013, Pages 79–92, https://doi.org/10.1093/reseval/rvs039

• Permissions Icon Permissions

This study presents humanities scholars’ conceptions of research and subjective notions of quality in the three disciplines German literature studies, English literature studies, and art history, captured using 21 Repertory Grid interviews. We identified three dimensions that structure the scholars’ conceptions of research: quality, time, and success. Further, the results revealed four types of research in the humanities: positively connoted ‘traditional’ research (characterized as individual, discipline-oriented, and ground-breaking research), positively connoted ‘modern’ research (cooperative, interdisciplinary, and socially relevant), negatively connoted ‘traditional’ research (isolated, reproductive, and conservative), and negatively connoted ‘modern’ research (career oriented, epigonal, calculated). In addition, 15 quality criteria for research in the three disciplines German literature studies, English literature studies, and art history were derived from the Repertory Grid interviews.

Email alerts

Citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1471-5449
• Print ISSN 0958-2029
• Copyright © 2024 Oxford University Press
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Digital Humanities

Oxford has been at the leading edge of technology and scholarship in ‘Digital Humanities’ since the early 1970s, when some of the earliest databases in the world were created in Classics, History and Oriental Studies.

Since then Oxford has increased and developed its relationship with Digital Humanities into an exciting, innovative and forward-look area of activity where teaching and research in Humanities disciplines intersect with the latest developments in computing technology.

The University now hosts the largest concentration of digital humanities projects in the UK, and probably the world.

These cutting-edge projects and programmes involve leading academics in the various Humanities disciplines as well as staff in the University’s Computing Services, Oxford e-Research Centre (OeRC), Bodleian Libraries, Museums, and colleges.

For the first time, many of these projects are now represented on one website:

Digital.Humanities@Oxford  (DH@O)

Research in Conversation

Research in conversation is a series of interviews with researchers across Oxford University. Each interviewee raises a question arising from their research, which the next interview follows up on, approaching from a different discipline. Together, these linked interviews form 'chains' that collectively, and from many different perspectives, ask big questions like what it is to be human, how to live a healthy life and our changing relationship with information.

http://www.ox.ac.uk/research/research-in-conversation

How to live a happy life

It's a question that's bothered us from as long ago as we have recorded thought. Now read five Oxford academics address this question from backgrounds of philosophy, practical ethics, psychology, psychiatry and Buddhist studies – and make your own mind up. It could be the most important thing you do all day.

​ http://www.ox.ac.uk/research/research-in-conversation/how-live-happy-life

Being Human

An anthropologist who heads up an interdisciplinary team looking at human ritual contrasts with a philosopher and theologian on the extent to which religion short-circuits our intuitive moral faculty.

http://www.ox.ac.uk/research/research-in-conversation/being-human

Centre for the Study of Ancient Documents

Ancient inscriptions and cutting-edge imaging software – unexpected collaborations between Oxford’s Faculty of Classics and Silicon Valley entrepreneurs.

http://www.ox.ac.uk/research/research-in-conversation/manuscript-megabyte

Randomness and order

This series grew out of  The Oxford Research Centre in the Humanities TORCH)  conference  ‘Randomness and Order’ , at which academics in the fields of quantum physics, music, probability and medieval history discussed what randomness meant in their different disciplines. You can see the recording of the conference  here .

http://www.ox.ac.uk/research/research-in-conversation/randomness-and-order

Research Showcase

The Oxford Arts Blog gives you all the latest news and views in the arts, humanities and culture at Oxford University. The blog is updated by Stuart Gillespie., contact him at [email protected] http://www.ox.ac.uk/news/arts-blog

Search form

• Find Stories
• For Journalists

Students learn about humanities research through hands-on discovery

A new one-week course, called Humanities Research Intensive , teaches first- and second-year undergraduate students what research in the humanities could be and what skills are needed to do it.

One Monday afternoon during spring break, a group of 15 Stanford first- and second-year undergraduate students gathered around dozens of archival books and texts, some of which were around 800 years old, on the second floor of the Green Library.

Go to the web site to view the video.

In the Humanities Research Intensive course, undergraduates learn skills necessary to do research in the humanities.

The students examined the ancient treasures as part of the new one-week course Humanities Research Intensive , created by the Stanford School of Humanities and Sciences and the Stanford Humanities Center . The course will be repeated this coming academic year.

The inaugural group of Humanities Research Intensive fellows learned what it takes to do research in the humanities under the direction of Stanford historian Caroline Winterer and manuscript scholar Elaine Treharne .

“We took the students through the entirety of the humanities research process in miniature,” said Winterer, a professor of history, the Anthony P. Meier Family Professor in the Humanities and director of the Stanford Humanities Center. “Over five days, we taught them how to ask a research question, how to find scholarly literature and how to produce a project proposal.”

A unique research opportunity

Humanities Research Intensive is aimed at first- and second-year undergraduate students who are interested in the humanities, which is the study of how people process and document the human experience, involving fields like philosophy, literature, religion, art, music, history and language.

History Professor Caroline Winterer and Humanities Research Intensive students examine archival treasures in the Special Collections reading room at Green Library. (Image credit: L.A. Cicero)

The course is the brainchild of Treharne and Winterer, who proposed its creation to introduce students to what research in the humanities could be and what skills are needed to do it.

The course is designed to be equivalent to the opportunities that students majoring in the sciences get during their first years at Stanford.

“Undergraduates pursuing STEM majors have a considerable number of opportunities as early as their first freshman quarter for developing their research through lab work or internships. But the humanities students don’t really have access to the same opportunities until later in their academic career,” said Treharne, professor of English, the Roberta Bowman Denning Professor in the School of Humanities and Sciences and director of Stanford Text Technologies .

Treharne said that it is important to introduce students to humanities research, which is often misunderstood.

“People often think that humanities research is comprised of getting some books of poetry and reading them, but that’s only one kind of humanities research,” Treharne said. “There is actually a tremendous amount of excitement to be generated about the relics of the past which human endeavor has produced. Archival research is about being able to contextualize, read and describe objects and put them into a broader social and cultural set of significances, often by using digital tools and approaches, too.”

An archival “petting zoo”

On the first day of the five-day course, instructors took the students to the Special Collections department at Green Library. There, the students were greeted with five tables filled with 43 books and objects – representing some of the best archival treasures that Stanford has.

English Professor Elaine Treharne works with undergraduate Lore Olivera. (Image credit: L.A. Cicero)

Among them were colorful medieval manuscripts, such as a 13th-century French portable Bible written on animal skin, and more modern, famous texts, such as a copy of John Milton’s Paradise Lost signed by both Thomas Jefferson and James Madison.

“What we did was similar to a petting zoo for little kids – we let the students have direct contact with the things that we want them to learn about,” Winterer said.

Students were given a couple of hours to look at the materials and pick an object that interested them. They then had to come up with a research question about the object and write a five-page proposal for a research project by the end of the week.

For example, Arman Kassam, one of the students, examined a 17th-century book about the coronation of King James II of Britain. He became curious when he saw mangoes on the menu of items served at the feast. His question was: How did a mango end up on the table of King James II when England’s colonization of India was only beginning? Kassam traced the trade connections between different parts of the world in the 17th century as part of his final proposal.

Aside from Green Library, the students also visited the Hoover Institution Library and Archives, which contain documents related to war, revolution and peace in the 20th and 21st centuries. There, curators assembled a table filled with powerful historical materials, including Adolf Hitler’s daily briefing and an X-ray of Hitler’s skull as well as the piece of paper that issued the order to drop the nuclear bombs on Hiroshima and Nagasaki.

For Treharne, that moment was one of the most powerful ones during the entire week.

“I said to the students, ‘Can you imagine the human impact of these pieces of paper on this one table?’” Treharne said. “And there was an absolutely resounding silence. They had learned how ephemera like these diaries and papers can have an incredible significance for the human experience.”

An enlightening experience

Freshmen Lena Han and Arman Kassam said they applied to be Humanities Research Intensive fellows because they wanted to experience what humanities research is and if it was something they could pursue in the future.

“The Humanities Research Intensive gives Stanford students the opportunity to work closely with some of our leading faculty so they can learn what it means to conduct research in the humanities and advance our knowledge and understanding of the world.” —R. Lanier Anderson Senior Associate Dean for the Humanities and Arts, School of Humanities and Sciences

Han said she was interested in art, politics and other humanities fields, but she never had an opportunity to do any intense research on her own. Kassam said he has previously done research in the sciences but was confused about what methods people in the humanities used in their projects.

Han examined an immigration pamphlet from 1876 that included a testimony by an attorney about Chinese immigrants. She noticed that rhetoric that seemed biased toward the Chinese, so she decided to analyze the pamphlet and compare it to others made during later time periods.

“The experience has been really informative in terms of making research more accessible,” Han said. “We learned a lot about how to interact with primary sources and how to develop research questions out of our own interests.”

Both Han and Kassam said their favorite part of the course was interacting with students, as well as with Winterer and Treharne.

“Stanford has top-ranked departments in the humanities and arts,” said R. Lanier Anderson , senior associate dean for the humanities and arts in the School of Humanities and Sciences and the J. E. Wallace Sterling Professor in the Humanities. “The Humanities Research Intensive gives Stanford students the opportunity to work closely with some of our leading faculty so they can learn what it means to conduct research in the humanities and advance our knowledge and understanding of the world. We are really indebted to professors Treharne and Winterer for designing this innovative program and giving our students such a terrific beginning research experience.”

Humanities Research Intensive occurred during spring break in March 2019. It is scheduled to repeat during spring break in 2020.

Research in the Humanities

How do you conduct research in the humanities? Where do you begin? Here are a few tips to consider when thinking about research in the humanities.

• Start early. You don’t have to know that you want to research medieval castles and building practices in 12th century England just yet! But you should be thinking about your interests. For example: Do you like American history, British literature, or Chinese art? What time periods interest you? These are great questions to start with. Identifying your exact topic of research takes some time, but once you know where your interests lie, it’s easier to find a specific topic to start researching. You can also review the Carolina Digital Repository , past SURF projects, and archives of past projects from the OUR’s Celebration of Undergraduate Research to get an idea of what kinds of projects other students in your discipline have
• Many books offer advice on this topic, but especially helpful are Chapters 3 and 4 of Wayne Booth’s The Craft of Research (which can help you move from a broad topic to a specific research question) and They Say/I Say: The Moves That Matter in Academic Writing by Cathy Birkenstein and Gerald Graff—a very short book that offers an accessible introductory crash course in academic research. And you may also consider checking out the free research methods courses at the Odum Institute , especially if you are interested in engaging in work in both the humanities and the social sciences.
• Take classes! Many students learn what interests them by taking a variety of classes taught by different professors, and these classes will also introduce you to research methods and recent scholarship in the field. Classes can offer some of the best preparation for starting a research project and starting to understand how to write literature reviews (which help you understand existing research on a topic of interest). Your classes can also introduce you to professors that you might be interested in working with.
• Chat with professors and graduate students in your department about their research. We learn so much about our own interests by talking to other people about theirs. Everyone approaches their projects a little differently, and getting feedback from multiple people may help you figure out what approaches sound like ones you could adopt. These meetings will help you learn more about the instructors in your department, making it easier to connect when you want to start a research project. It may seem a little daunting to set up these meetings, but your professors usually love to talk about their research with an interested student.
• The next step is to find a faculty member as a potential mentor. Think first about interesting professors you’ve taken classes with or worked with in other capacities. Did you take a 20th century U.S. history class and love how the professor teaches? Do they have some of the same research interests as you? Also look through your department’s faculty page and use Google to see who at UNC is doing research that’s interesting to you. It’s okay to reach out to professors you don’t know yet. Finding the right professor can take time, but don’t be afraid to talk to likely candidates about your potential research interests and learn more about theirs. Start talking to professors as early as possible so you’ll have a great network when it’s time to ask a faculty member to be your research mentor.
• You can also talk to graduate teaching assistants about professors in their department. Graduate students know a lot about professors and their approach to mentoring students, and they can help you identify a faculty member who shares your interests and who might help you develop a project. Teaching assistants might feel more approachable if you’ve been in a large lecture class where the professor has interacted with you less than your TA.
• If you have to decide between a mentor whose teaching you love and one whose research is exactly on your topic, you may want to consider focusing more on the former than the latter; other students’ experiences have suggested that a good interpersonal and intellectual fit with your mentor can matter at least as much as a shared research focus, but hopefully you can find a balance of both. Also keep in mind you can meet with faculty regarding subject expertise even if they are not the faculty mentor sponsoring your independent study or honors thesis. You can have more than one mentor on a single project.
• Different faculty have different models for mentoring undergraduates in undergraduate research, but many will be willing to talk about your interests and theirs, and to give you advice and feedback on your research proposal, SURF application, or other forms of initial research It’s important to have a conversation with your faculty mentor about their expectations and yours. Will you schedule recurring meetings to review your progress? Will you check in informally on email? Will you work independently unless you have a problem? Clarifying these expectations at the beginning of the project prevents miscommunication and misunderstanding.
• After you have gotten a sense of where your interests lie and have found one (or two or three) professors with whom you might work, start thinking about how you can proceed with your research. Do you want to do an independent study? Consider applying for a Summer Undergraduate Research Fellowship (SURF) or Summer Award for Research Courses (SARC) to work with a faculty mentor on developing your project. Are you interested in writing an honors thesis? Talk to your departmental advisor or your department’s faculty liaison to see what that entails. Regardless, explore SURFs and other fellowships or funding opportunities to help fund your research, giving you more time to focus on what interests you.
• Once you have developed a pathway to begin your research, work on crafting a specific research question. Use the library tutorials , schedule an appointment with a reference librarian in your research area, and use books like the ones mentioned above to help you navigate key words. Databases could also help you discover and define your topic and develop your research question(s). Keep in mind that it can take a while to develop and revise your exact research question, so give yourself plenty of time to plan and think things through, and use your conversations with faculty to help you in this process. Also be aware of deadlines if you are applying to certain programs like SURF or an Honors Thesis. Once you know what it is you want to research, develop a plan with your faculty
• When you have figured out all of these things, it is time to begin tracking down and reading the actual sources for your research! For some, this can be the most exciting process of the entire project, but for others it can be frustrating. Give yourself plenty of time to consult resources, and seek help and advice from books like Booth and Birkenstein/Graff (above), librarians, and faculty mentors on where to begin this process, how to put your own ideas into dialogue with your sources, and how to structure your project. With all of these steps completed, you can successfully conduct research in the humanities!

(Prepared by Dr. Hilary Lithgow, Dr. Donna Bickford, and OUR Ambassador Sarah Faircloth in November 2014. Revised by OUR Graduate Assistant Aaron Pattillo-Lunt and Dr. Robert Pleasants in February 2024.)

Humanities Research Guide: HOME

• DATABASES & JOURNALS
• BOOKS & eBOOKS
• EXTERNAL RESOURCES FOR HUMANITIES
• ANTHOLOGIES, ARCHIVES, CATALOGUES, COLLECTIONS & DIGITAL PROJECTS
• DICTIONARIES & GLOSSARIES
• PRIMARY SOURCES - SPECIAL COLLECTIONS
• TEACHING WITH RARE BOOKS & SPECIALIZED MATERIALS
• SELECT MANUSCRIPTS AT USC
• BLOGS & PODCASTS - HUMANITIES
• HUMANITIES CONFERENCES & GRANT OPPORTUNITIES
• USC LIBRARIES STREAMING SERVICES
• STRATEGIC RESEARCH TUTORIALS

LEARN HOW TO FIND & USE IMPACT FACTOR

RLT #42: Impact Factor

RESEARCH, LEARNING & TEACHING (RLTs) DIGITAL SERIES

Learn more about USC Libraries Services and Resources!

To get to the full list of RLTs, click this link:

https://bit.ly/RLTSeries1

VISIONS & VOICES - EVENTS & RESOURCE THEME GUIDES FOR FOCUSED RESEARCH

Research resources, usc libraries resource theme guides, visions & voices theme guides, usc center for the premodern world.

USC Dornsife Center for the Premodern World

The Center for the Premodern World at USC creates space and offers resources for the study of cultures and civilizations, beginning with the earliest historical eras up to the end of the Middle Ages and the beginning of the modern world. 

For Info on Upcoming Events: Click Here

Welcome to the Strategic Research Guide for Humanities

This guide features key resources for Humanities.

Need more resources for, research, learning, & teaching - explore usc libraries..

https://linktr.ee/libresources

RELATED USC RESEARCH GUIDES

Primary Sources

Fair Use: Using Images and Non-Textual Materials in Presentations, Papers, Theses, and Dissertations

Organizing Research for Arts and Humanities Papers & Theses

Theater & Drama

English Language & Literature

Nineteenth-Century Literary Studies

Reading Early Printed Books

Medieval and Manuscript Studies and Research

EXTERNAL RESEARCH GUIDES

The Huntington Library, Art Museum, and Botanical Gardens

Incunabula at The Huntington

Huntington Incunabula in the Digital Era

From Parchment to Pixel: Conservation and Digitization of Illuminated Manuscripts

Duke University Libraries

Teaching Materiality Online with the Rubenstein Library

Additional External Research Guides:

Duke University Humanities Research Guides

University of Michigan Humanities Research Guides

University of Chicago Humanities Research Guides

NYU Digital Humanities Research Guide

Northwestern University Digital Humanities Research Guide

DR. MELISSA L. MILLER, HEAD, HOOSE LIBRARY OF PHILOSOPHY, HUMANITIES LIBRARIAN

CHRISTINA SNIDER, MMLIS

Hoose Library of Philosophy, Library & Student Assistant Supervisor

Email: [email protected]   Pronouns: she/her/hers

MONTHLY EVENTS AND PROGRAMMING

Explore all of the 2022-2023 Newsletters highlighting monthly events and programming!

Click this link: https://bit.ly/2022HooseNewsletters

Look back at all of the 2021 Newsletters highlighting monthly events and programming!

https://bit.ly/2021HooseNewsletters

HOOSE LIBRARY OF PHILOSOPHY USAGE STATISTICS

Hoose Library of Philosophy is one of the most popular places on campus,

a USC HOT SPOT!

Check out our Stats - for a full list of 2022-2023 Infographics,

Click this link: https://bit.ly/2022HooseInfographics

• Next: DATABASES & JOURNALS >>
• Last Updated: Feb 2, 2024 1:53 PM
• URL: https://libguides.usc.edu/humanitiesresearch

WashU Libraries

Research approaches in the humanities, social sciences, or sciences.

• Conducting Research in 6 Steps
• Social Sciences

Subject Librarians

The staff at Reference Help Desk can also help you finding appropriate sources for background information.  You are also welcome to contact a Subject Librarian for assistance:

Subject Librarians, alphabetical listing

Subject Librarians, subject listing

Researching in the Humanities

The fields in the Humanities discipline generally include the visual and performing arts, philosophy, literature, religion, history, languages, art history, and classics.  Although research methods differ among the Humanities, the Social Sciences, and the Sciences, any research project in any discipline starts with curiosity and a hypothesis.  Often research topics are interdisciplinary and may include multiple subject areas and methods from more than just one discipline.

When beginning a research project in the humanities, you must develop a deep knowledge base in a subject area, choose original sources to examine, locate and evaluate sources that also explore your areas of interest, and then come to your own original conclusions. Libraries can help you find the material you need to get started.  The research guides listed on the right are created by WUSTL Subject Librarians.  Subject Librarians have expertise in both searching techniques and academic fields, and their online guides suggest resources for the different phases of the research process: 1) Use background sources to establish your knowledge base.  These could be subject encyclopedias, key works in a field, bibliographies, etc. 2) Select original sources, commonly referred to as primary sources, for your analysis.  Primary sources are simply original works, e.g., novels, photographs, diaries, correspondence, advertisements, eyewitness accounts. 3) Find articles, reviews, and books that analyze primary sources.  These are known as secondary sources. Then, synthesize all this information with your own thinking and draw your original conclusions, thus creating new research in the field.

Recommended Research Guides

• A Guide to African and African-American Studies by Rudolph Clay Last Updated Apr 4, 2024 313 views this year
• A Guide to American History by Michael Schaefer Last Updated Apr 22, 2024 1018 views this year
• A Guide to Comparative Literature by Walter Schlect Last Updated Apr 11, 2024 387 views this year
• A Guide to East Asian Studies by Joan Wang Last Updated Apr 4, 2024 207 views this year
• A Guide to English and American Literature by Kris Helbling Last Updated Apr 11, 2024 307 views this year
• A Guide to Germanic Studies by Walter Schlect Last Updated Apr 12, 2024 251 views this year
• A Guide to Islamic and Middle Eastern Studies by AJ Robinson Last Updated Apr 22, 2024 203 views this year
• A Guide to Philosophy Resources by Michael Schaefer Last Updated Mar 22, 2024 310 views this year
• A Guide to Philosophy-Neuroscience-Psychology (PNP) Resources by Sam Lindgren Last Updated Apr 4, 2024 303 views this year
• A Guide to Religious Studies by Michael Schaefer Last Updated Mar 12, 2024 332 views this year
• Romance Languages & Literatures Subject Guide by Lino Mioni Last Updated Oct 5, 2022 42 views this year
• A Guide to Women, Gender & Sexuality Studies by AJ Robinson Last Updated Apr 22, 2024 187 views this year
• A Guide to World History by Michael Schaefer Last Updated Apr 11, 2024 257 views this year
• << Previous: Conducting Research in 6 Steps
• Next: Social Sciences >>
• Last Updated: Apr 11, 2024 3:23 PM
• URL: https://libguides.wustl.edu/researchapproaches

Institute for Research in the Humanities

News and Announcements

Amanda shubert virtual lecture at the centre for global jewish studies, university of warwick.

An Iconography of Diaspora: The Illustrated Ketubah in British India

The Graduate Early Modern Student Society is Hosting its Symposium

Renaissance Hybridity

Lisa Cooper Elected to University Research Council

She will help to advise the overall UW–Madison research enterprise

Review of Steven Nadler’s new biography of Descartes

Descartes: The Renewal of Philosophy (London: Reaktion Books, 2023)

IRH Fellows Enjoy the Eclipse on Library Mall

April 8th at 2:06pm

Simon P. Newman Finalist for ACLS Open Access Book Prize and Arcadia Open Access Publishing Award

Freedom Seekers: Escaping from Slavery in Restoration London (University of London Press, 2022)

Remembering Professor Emeritus David Bordwell

Senior Fellow, 1993-1998 and Jacques Ledoux Professor Emeritus of Film Studies

IRH Fellows Meet at Renaissance Society of America Conference

Exciting Updates from the IRH Community at RSA 2024

• More News and Announcements More

Upcoming Events

Savage Pasts, Baroque Futures: Racial Hierarchy and the Political Theology of Spanish Empire April 25, 2024 @ 4:30 pm - 5:45 pm University Club, Room 212

Burdick-Vary Lecture with Speaker Mauro Caraccioli

“I Was Their Sacrifice”: Gary Tyler and The Criminalization of Black Students during Desegregation April 29, 2024 @ 3:30 pm - 5:00 pm

Monday Seminar: Walter C. Stern

Recent Publications

Jerusalem Through the Ages From Its Beginnings to the Crusades

Jodi Magness

Libels and Theater in Shakespeare’s England: Publics, Politics, Performance

Joseph Mansky

Promiscuous Grace Imagining Beauty and Holiness with Saint Mary of Egypt

Sonia Velázquez

• More Recent Publications More

Research Assessment in the Humanities

Towards Criteria and Procedures

• Open Access
• © 2016

You have full access to this open access Book

• Michael Ochsner 0 ,
• Sven E. Hug 1 ,
• Hans-Dieter Daniel 2

Social Psychology and Research Edu., Univ of Zurich, D-GESS, ETH & Ev. O, Zurich, Switzerland

You can also search for this editor in PubMed   Google Scholar

Social Psychology and Research Edu., University of Zurich, D-GESS, ETH &, Zurich, Switzerland

D-GESS, ETH Zurich, Zurich, Switzerland

• This book is open access, which means that you have free and unlimited access.
• Presents a state-of-the-art overview of research assessment in the humanities
• Paints a comprehensive picture of research evaluation in the humanities
• Contains contributions of humanities scholars, research funders, and researchers on higher education
• Includes supplementary material: sn.pub/extras

203k Accesses

157 Citations

79 Altmetric

Buy print copy

• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Table of contents (18 chapters)

Front matter, research assessment in the humanities: introduction.

• Michael Ochsner, Sven E. Hug, Hans-Dieter Daniel

Setting Sail into Stormy Waters

The ‘mesurer les performances de la recherche’ project of the rectors’ conference of the swiss universities (crus) and its further development.

• Antonio Loprieno, Raymond Werlen, Alexander Hasgall, Jaromir Bregy

Yes We Should; Research Assessment in the Humanities

• Wiljan van den Akker

How Quality Is Recognized by Peer Review Panels: The Case of the Humanities

• Michèle Lamont, Joshua Guetzkow

Humanities Scholars’ Conceptions of Research Quality

The current state of quality-based publication rankings and publication databases, the esf scoping project ‘towards a bibliometric database for the social sciences and humanities’.

• Gerhard Lauer

Publication-Based Funding: The Norwegian Model

• Gunnar Sivertsen

Assessment of Journal & Book Publishers in the Humanities and Social Sciences in Spain

• Elea Giménez Toledo

European Educational Research Quality Indicators (EERQI): An Experiment

• Ingrid Gogolin

Bibliometrics in the Humanities

Beyond coverage: toward a bibliometrics for the humanities.

• Björn Hammarfelt

Quotation Statistics and Culture in Literature and in Other Humanist Disciplines

• Remigius Bunia

Evaluation of Research in the Humanities in Practice

Peer review in the social sciences and humanities at the european level: the experiences of the european research council.

• Thomas König

The Four ‘I’s: Quality Indicators for the Humanities

• Wilhelm Krull, Antje Tepperwien

Bottom Up from the Bottom: A New Outlook on Research Evaluation for the SSH in France

• Geoffrey Williams, Ioana Galleron

The ‘Forschungsrating’ of the German Council of Science and Humanities. Risks and Opportunities for the Humanities: The Case of the Anglistik/Amerikanistik Pilot Study

• ESF Scoping Project
• assessment of humanities research
• bibliometric issues concerning humanities research
• bibliometrics in the humanities
• evaluating research
• evaluation of humanities research
• humanities in practice
• humanities research
• humanities research assessment
• procedures for evaluating research
• publication databases
• publication-based funding
• research assessments
• research performance indicators
• research quality

About this book

Editors and affiliations, social psychology and research edu., univ of zurich, d-gess, eth & ev. o, zurich, switzerland.

Michael Ochsner

Social Psychology and Research Edu., University of Zurich, D-GESS, ETH &, Zurich, Switzerland

Sven E. Hug

Hans-Dieter Daniel

About the editors

Michael Ochsner finished his doctoral studies at the Institute of Sociology of the University of Zurich in 2012 and received his PhD in 2014. Since 2009, he has been a research associate in the CRUS-organized projects ‘Developing and Testing Research Quality Criteria in the Humanities, with an emphasis on Literature Studies and Art History’ and ‘Application of Bottom-up Criteria in the Assessment of Grant Proposals of Junior Researchers in the Social Sciences and Humanities’. Since 2013, he has also worked at the Swiss Centre of Expertise in the Social Sciences (FORS) at the University of Lausanne as a senior researcher in the team ‘international surveys’. He is vice-president of the EvalHum initiative, a European association for research evaluation in the SSH.

Sven E. Hug studied German language and literature as well as psychology at the University of Zurich and worked in various companies as a market research analyst. He is currently working as a project manager at the evaluation office of the University of Zurich and furthermore acts as a research associate in the CRUS-organized project ‘Application of Bottom-up Criteria in the Assessment of Grant Proposals of Junior Researchers in the Social Sciences and Humanities’ at the professorship for social psychology and research on higher education (ETH Zurich).

Hans-Dieter Daniel holds a dual professorship at ETH Zurich and at the University of Zurich. Since 2001, he has been the director of the evaluation office of the University of Zurich and since 2002, professor for social psychology and research on higher education at ETH Zurich. Dr. Daniel is a psychologist by training. Since 2011, he has been a member of the evaluation committee of the German Council of Science and Humanities (Wissenschaftsrat). His scholarly interests include research on peer review and evaluative bibliometrics. He is a highly cited researcher and co- author of several highly cited journal articles in Essential Science Indicators from Thomson Reuters as well as author of the book Guardians of Science—Fairness and Reliability of Peer Review . 

Bibliographic Information

Book Title : Research Assessment in the Humanities

Book Subtitle : Towards Criteria and Procedures

Editors : Michael Ochsner, Sven E. Hug, Hans-Dieter Daniel

DOI : https://doi.org/10.1007/978-3-319-29016-4

Publisher : Springer Cham

eBook Packages : Education , Education (R0)

Copyright Information : The Editor(s) (if applicable) and the Author(s) 2016

Hardcover ISBN : 978-3-319-29014-0 Published: 29 April 2016

Softcover ISBN : 978-3-319-80472-9 Published: 22 April 2018

eBook ISBN : 978-3-319-29016-4 Published: 19 April 2016

Edition Number : 1

Number of Pages : XVIII, 247

Number of Illustrations : 22 illustrations in colour

Topics : Assessment, Testing and Evaluation , Higher Education , Ethics , Public Administration

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Visit the University of Nebraska–Lincoln
• Apply to the University of Nebraska–Lincoln
• Give to the University of Nebraska–Lincoln

Search Form

Center for digital research in the humanities.

Advancing cultural inquiry

For the public good

Collaborative Research — The Center for Digital Research in the Humanities is a community of researchers collaborating to build digital content and systems in order to generate and express knowledge of the humanities.

Joint Initiative — The Center for Digital Research in the Humanities is a joint initiative of the University of Nebraska-Lincoln Libraries and the College of Arts & Sciences .

Transdisciplinary Exploration — The Center for Digital Research in the Humanities promotes collaborative, transdisciplinary digital humanities research.

Discover cutting edge Digital Humanities projects

Visit our Digital Scholarship page

Featured Projects

African Poetry Digital Portal

The African Poetry Digital Portal is a resource for the study of African Poetry providing access to biographical information, news, images and documents.

COVE is a scholar-driven open-access platform that publishes both peer-reviewed material and "flipped classroom" student projects built with our online tools.

Nebraska Stories of Humanity

Open ONI is a community-maintained project to make historic American newspapers browsable and searchable on the web.

Student Involvement

Digital humanities faculty and fellows are looking for University of Nebraska-Lincoln undergraduates to participate in paid research assistantships through UCARE . Learn more about The Digital Humanities Program at Nebraska .

Center for Digital Research in the Humanities News

See more news

Follow the Center for Digital Research in the Humanities

Stanford Humanities Today

Arcade: a digital salon.

Hope: The Future of an Idea | 2024 Spring Salon

Where is hope in humanities research? Perhaps it's a concept with a particular history, perhaps a force whose effects are latent or invisible; or it may be absent altogether for reasons to explain. Does hope motivate one's work? What does hope mean intellectually and personally?

Please join us for brief responses to these questions by current fellows, followed by a general discussion with Q&A moderated by SHC Director Roland Greene . The event will conclude with a reception.

About the Speakers

Samia Errazzouki (Mellon Postdoctoral Fellow) is a historian of early Northwest Africa. She holds a PhD in history from the University of California, Davis and an MA in Arab Studies from Georgetown University. Her research and teaching focuses on trans-regional histories of racial capitalism, slavery, and empire. Errazzouki formerly worked as a Morocco-based journalist with the Associated Press, and later, with Reuters. She is currently a co-editor of Jadaliyya and assistant editor of The Journal of North African Studies .

Jisha Menon (Violet Andrews Whittier Internal Fellow) is Professor of Theater and Performance Studies, and, by courtesy, of Comparative Literature at Stanford University. She is the author of Brutal Beauty: Aesthetics and Aspiration in Urban India (Northwestern UP, 2021) and The Performance of Nationalism: India, Pakistan and the Memory of Partition (Cambridge UP, 2013). She is also co-editor of two volumes: Violence Performed: Local Roots and Global Routes of Conflict (Palgrave-Macmillan Press, 2009) and Performing the Secular: Religion, Representation, and Politics (Palgrave Macmillan, 2017).

Joseph Wager (SHC Dissertation Prize Fellow) is a PhD Candidate in Iberian and Latin American Cultures at Stanford University. He is writing a dissertation focused on the form of the stories about desaparecidos, what is said about desaparecidos, in contemporary Colombia and Mexico. The dissertation places social-scientific inquiry, the work of activists and collectives, and legal instruments in dialogue with art installations, film, novels, performances, and poems. Underpinning this combination is 1. the idea that human-rights changes stem from how individual and collective actions resist institutionalization or translate into institutions and 2. that cultural products (e.g., art) and their form are crucial to the understanding of such processes.

Ya Zuo (External Faculty Fellow) is an associate professor of History at University of California, Santa Barbara. She is a cultural historian of middle and late imperial China. She is the author of Shen Gua’s Empiricism (Harvard University Press, 2018) and a range of articles on subjects such as theory of knowledge, sensory history, medical history, book history, and the history of emotions.

Stanford Humanities Center 424 Santa Teresa Street

Also online via Zoom

Discovering Humanities Research at Stanford

Stanford Humanities Center

Advancing Research in the Humanities

Humanities Research for a Digital Future

The Humanities in the World

• Annual Reports
• Humanitrees
• Humanities Center Fellows
• Advisory Council
• Director's Alumni Cabinet
• Honorary Fellows
• Hume Honors Fellows
• International Visitors
• Workshop Coordinators
• Why Humanities Matter
• Fellowships for External Faculty
• Fellowships for Stanford Faculty
• Mellon Fellowship of Scholars in the Humanities
• Dissertation Fellowships FAQs
• Career Launch Fellowships
• Fellowships for Stanford Undergraduates
• International Visitors Program
• FAQ Mellon Fellowship
• Faculty Fellowships FAQ
• Events Archive
• Public Lectures
• Checklist for Event Coordinators
• Facilities Guidelines
• co-sponsorship from the Stanford Humanities Center
• Call for Proposals
• Manuscript Workshops
• Past Workshops
• Research Workshops

National Humanities Center renews partnership with Lincoln Center for responsible AI research

Image courtesy Pexels

The National Humanities Center has announced  that Arizona State University's  Lincoln Center for Applied Ethics  is one of four organizations to receive funding for the second phase of their Responsible AI Curriculum Design Project , identifying it once again as a leading institution conducting research in the field of humane technology.

The Lincoln Center was originally identified in the first round of awards from NHC in 2021, which was launched with support from Google to generate undergraduate curriculum on responsible AI. Their participation resulted in the development of the first class at ASU about the human impacts of AI .

“As AI tools become more accessible and we move to integrate them into more aspects of our lives, it is crucial to keep human well-being the core guiding principle,” says Sarah Florini , interim director of the Lincoln Center and an associate professor of film and media studies. “The Lincoln Center is focused on imagining a future where technology is centered on the human, which is vital in this moment.” 

The project supports a partnership with Mesa Community College  to develop responsible AI curricula. Florini will be partnering with Dave Yount , professor of philosophy and religious studies at MCC, to develop courses and materials for both institutions. Paulette Stevenson , English faculty and director of writing programs at MCC, will collaborate on the project.

This award comes after the Lincoln Center launched two workgroups this semester focusing on artificial intelligence, convening scholars from across ASU to investigate the impacts of AI as part of the center’s focus on ethical innovation and humane technologies.

The two workgroups address the ethics and the sustainability of AI , and include members from The College of Liberal Arts and Sciences , the Ira A. Fulton Schools of Engineering , the New College of Interdisciplinary Arts and Sciences , the College of Integrative Arts and Sciences , the College of Global Futures , and the Herberger Institute for Design and the Arts .

The formation of these groups is spearheaded by Florini and is organized in collaboration with Erica O’Neil , a research program manager at the Lincoln Center. These research-focused groups highlight strategic priorities of the center, which focus on human flourishing as the most crucial outcome of technological innovations.

Florini co-chairs the AI & Ethics workgroup with Nicholas Proferes , an associate professor in the School of Social and Behavioral Sciences whose research centers on data ethics and policy.

“It’s so cool to be at ASU, because it is a leader in a lot of ways in terms of integrating AI into the classroom,” Proferes says. “But it also gives us a moment to reflect on the adoption process. We are asking the questions that need to be asked.”

The ethics group seeks to engage in critical discussions about technologies often referred to as “AI” and their broader potential impacts, and shape practices that result in the development of more responsible AI.

“I’m particularly interested in how we develop effective policies around AI,” Proferes says. “One of the things we’re really looking forward to with this group is to create a space where we can engage deeply in these conversations — not just how we (integrate AI) at ASU, but how it will impact education as a whole.”

Similarly, the sustainability cohort aims to develop discourse around AI that prioritizes sustainability measures over the exponential, unchecked growth of technological innovations.

“Scaling artificial intelligence to match the ideological aspirations of the private sector will require an exponential increase in computational power, which means more data centers,” O’Neil says. “Arizona is a hot market for these energy intensive centers that require a huge amount of water to operate.

“Do we want to use all the drinkable water in the Southwest to cool data centers so we don’t have to summarize meeting notes or write emails? Our workgroup explores the tensions between resource allocation and value additions of AI.”

With the renewed partnership between the National Humanities Center and the Lincoln Center, ASU students can expect AI engagement in the classroom that has been developed in direct collaboration with leading experts and technologists at Google. The new class will be offered during the 2025–26 academic year.

This summer, the Lincoln Center will also be launching a third workgroup focused on reparative archives and digital humanities, organized by the Digital Humanities Initiative . This group will explore postcolonial archives, cultural heritage work in AI, and more.

“The Lincoln Center aims to create a variety of opportunities to engage with the ethical implications of AI, including through undergraduate courses, faculty research and public-facing conversations," Florini says. "Through these RAI projects and the workgroups, we are laying a foundation for sustained and vigorous dialogue around these issues.”

More Science and technology

Celebrating 34 years of space discovery with NASA

This year, NASA's Hubble Space Telescope (HST) is celebrating its 34th anniversary of the world's first space-based optical telescope, which paved the scientific pathway for NASA's James Webb Space…

Making magic happen: Engineering and designing theme parks

The themed entertainment industry is widespread and diverse, encompassing everything from theme parks to aquariums, zoos, water parks, museums and more. The Theme Park Engineering and Design…

AI-equipped feeders allow ASU Online students to study bird behavior remotely

ASU Online students are participating in a research opportunity that's for the birds — literally. Online Bird Buddies is a project that allows students to observe birds remotely, using bird feeders…

Humanities Research Centre

• Social feeds
• Study with us
• Australian Museums and Galleries Association (ACT Branch)
• Australian National Centre for the Public Awareness of Science
• Australian Studies Institute
• ANU Collections Hub
• Centre for Classical Studies
• Centre for Digital Humanities Research
• Classics Museum
• Consortium of Humanities Centers and Institutes
• Digital Humanities Lab
• Freilich Project for the Study of Bigotry
• Gender Institute
• Interdisciplinary and Cross-Cultural Research
• Sydney Social Sciences and Humanities Advanced Research Centre, University of Sydney
• The Australasian Consortium of Humanities Researchers & Centres
• The Centre for Creative and Cultural Research, University of Canberra
• U3A Canberra

Related Sites

• ANU College of Arts and Social Sciences
• Research School of Humanities and the Arts
• Research School of Social Sciences

You are here

HRC partner in 2025 Global Humanities Institute on Indigenous Mobilities, Tourism, and Racial Capitalism

We are thrilled to be part of a team that has been awarded $200K USD from the Consortium of Humanities Centres and Institutes with the support of a grant from the Andrew W. Mellon Foundation to run a Global Humanities Institute in 2025 on the topic of “Indigenous Mobilities, Tourism, and Racial Capitalism”.

The Institute will explore how Indigenous communities creatively respond to the cascading effects of mega-development projects and mass tourism—such as neglect of local needs, dispossession and displacement, destruction of local ecosystems, and other consequences—at various sites around the world.

Guiding questions include:

• What are the pressing concerns that Indigenous communities have regarding tourism and mega-development projects?
• How has this played out in different local contexts and on the land itself?
• What are strategies for centering Indigenous experiences and demands in the face of megadevelopment?
• How can a humanities-centered approach help address these crises and imagine new futures?
• Is it possible to construct futures in which Indigeneity, non-human relations, tourism, and mega-development goals align?

The main institute meeting will be hosted by Universidad Autónoma de Yucatán and held in Yucatán, Mexico in January 2026. A lead-up event will be held at the Humanities Research Centre ANU in January 2025. There will be an application process to fund up to 20 interdisciplinary scholars, with special focus on early career and emerging scholars, as well as community partners, to join the main institute meeting.

Professor Kylie Message-Jones (HRC) and Dr Yujie Zhu (CHMS) are leading the ANU component of the GHI in partnership with international collaborators from The Institute for Advanced Study at the University of Minnesota (Professor Bianet Castellanos), The Unidad de Proyectos Sociales (Center for Social Projects) at the Universidad Autónoma de Yucatán (UADY) (Professor Samuel François Jouault ), Co’ox Mayab, Alba Sud, The Confluencenter for Creative Inquiry at the University of Arizona (Yadira Cabarello and Professor Javier Duran) and the University of California San Diego Department of Communication (Matilde Córdoba Azcárte).

The full announcement and timeline of the 2025 Global Humanities Institute is here . Queries can be directed to [email protected]

Tweets by ‎@hrc_ANU

There were no tweets found.

Updated:   24 April 2024 / Responsible Officer:   Head, Centre / Page Contact:   CASS Marketing & Communications

• Contact ANU
• Freedom of Information

+61 2 6125 5111 The Australian National University, Canberra TEQSA Provider ID: PRV12002 (Australian University) CRICOS Provider : 00120C ABN : 52 234 063 906

Help | Advanced Search

Computer Science > Artificial Intelligence

Title: the collective use and evaluation of generative ai tools in digital humanities research: survey-based results.

Abstract: The advent of generative artificial intelligence (GenAI) technologies has revolutionized research, with significant implications for Digital Humanities (DH), a field inherently intertwined with technological progress. This article investigates how digital humanities scholars adopt, practice, as well as critically evaluate, GenAI technologies such as ChatGPT in the research process. Drawing on 76 responses collected from an international survey study, we explored digital humanities scholars' rationale for GenAI adoption in research, identified specific use cases and practices of using GenAI to support various DH research tasks, and analyzed scholars' collective perceptions of GenAI's benefits, risks, and impact on DH research. The survey results suggest that DH research communities hold divisive sentiments towards the value of GenAI in DH scholarship, whereas the actual usage diversifies among individuals and across research tasks. Our survey-based analysis has the potential to serve as a basis for further empirical research on the impact of GenAI on the evolution of DH scholarship.

Submission history

Access paper:.

• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Trinity Long Room Hub Arts & Humanities Research Institute

View the contact page for more contact and location information

• Critical ChangeLab project

Young adults from Kildare and Wicklow visit Trinity as part of Critical ChangeLab

On Thursday, March 28, the InSync Junior Board and Area Coordinators visited Trinity as part of the Critical ChangeLab project to discuss youth-led approaches to democracy.

Critical ChangeLab is an EU-funded project which aims to develop innovative, transdisciplinary, youth-led approaches to democracy and global citizenship education.

Trinity is part of a consortium using participatory action research (PAR) with civil society organisations in ten EU countries to create a new model of democratic pedagogy (Critical ChangeLabs). Critical ChangeLabs involve collaboration with local stakeholders to address challenges relevant to young people.

In February, researchers from Trinity’s School of Education and the Trinity Long Room Hub co-designed a series of workshops with InSync , an organisation providing youth and family services in Kildare and West Wicklow. The theme of ‘community and identity’ was chosen in consultation with the InSync Junior Board and their Area Coordinator youth workers. This topic is particularly pertinent in the context of the growing influence of the far right in Ireland, an issue of concern for the young people who participated in the project.

The Junior Board are young people aged 16 – 24 who contribute youth perspectives to decision-making for the entire InSync service. Participants worked with Trinity researchers to explore historical examples of exclusion and social change and articulate their visions for inclusive futures and better youth agency within their local communities. The Junior Board created a series of handmade zines at the end of the workshops to capture their reflections and messages in relation to these themes.

Speaking of the collaboration with Trinity, an area coordinator with the Junior Board said:

“ The Junior Board and the Critical Change Lab started a process to look at democracy and through that process what we discovered, and affirmed, was that some of the most marginalised young people in our communities are living in worlds where they see themselves as having no power to change things and in lots of ways with little voice.”

At the end of March, the Junior Board and Area Coordinators came to Trinity for the final research focus group, a campus tour, a visit to Leinster House, and to share their zines with colleagues from the School of Education and Trinity Long Room Hub.

Commenting on their participation in the workshops, one member of the Junior Board said they felt their “opinion as a young person is valued” and that through the participatory research “you’re provided a way to give your opinion, your thoughts, and your observations on ongoing issues that you have found in your area.”

Another participant highlighted the importance of being part of the Critical ChangeLabs, and seeing how historical societal injustices were changed in the past: "even if it did take 60-70 years, change can be made ... it gave that empowerment that it might take time, but we can make changes to the society we're in and solve the issues we're facing... "

Following the latest workshops, Dr Mairéad Hurley, Assistant Professor in Science Education at Trinity, reflected on the experience, and outlined the next steps of the project:

"It has been a joy to work with InSync as co-researchers, and the young people have constructed powerful messages full of creativity, humour, and vulnerability. We look forward to continuing this collaboration and supporting this incredible group in using the Critical ChangeLab model to further investigate inequality and injustice, envision alternate futures, and collaboratively take action on the issues that matter to them.

With our partners across Europe, the Trinity team is now reflecting on the first iteration of the Critical ChangeLab and planning for the next series of workshops. We're also preparing to work with a new group of young people this summer as part of our ongoing collaboration with the Digital Hub and the Beta Festival of Art & Technology.

If any schools, artists, or non-formal learning institutions are interested in getting involved, please email us at [email protected] —we'd love to talk!"

The Critical ChangeLab project draws upon the expertise of Trinity’s multidisciplinary Science & Society research group , based in the School of Education

The Critical ChangeLab project also builds upon the Trinity Long Room Hub’s work in this area in recent years, including The Democracy Forum and CEPRAH (Community Engagement Praxis for Research in the Arts and Humanities).

Team: Mairéad Hurley, Caitlin White, Karen Fowler (School of Education), Elspeth Payne (Trinity Long Room Hub)

Project website: https://ars.electronica.art/criticalchangelab/en/

Contact: [email protected]

Critical Changelab is funded by the European Union’s Horizon Europe research and innovation programme under grant agreement number 101094217. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Executive Agency (REA). Neither the European Union nor the granting authority can be held responsible for them .

Aoife King, Communications Officer | Trinity Long Room Hub | [email protected] | 01 896 3895

From literature to AI: UC grad shares career path to success

Humanities and storytelling lead to lucrative tech entrepreneurship.

Before Katie Trauth Taylor worked with international organizations like NASA, Boeing and Hershey, and before receiving accolades for her work in the generative AI space, she was in a much different industry – English and literature. 

Taylor earned her master's in English and Comparative Literature in UC’s College of Arts and Sciences. She completed her educational journey at Purdue University with a doctorate degree in rhetoric and composition. After working as a graduate assistant at Purdue and UC, she became a research professor at Miami University. 

It might seem from there that her career was set—perhaps a tenured professorship or a university administrative position. That might have been her path, but Taylor had her eyes set on different goals.

So how did Taylor transition from literature and composition to tech entrepreneurship? She enjoys sharing that part of her story.

A&S masters grad Katie Trauth Taylor, who has launched two businesses. Photo/Provided

“I love that question, and I think it's so important for everyone in the humanities to see that they have just as much entrepreneurial potential as any other person.”

Taylor founded Untold Content in 2015, jumping from academia to entrepreneurship.  Untold Content is a business-to-business content strategy company that recognizes storytelling patterns and frameworks that will help innovative companies make their ideas, clear, compelling and impactful to an audience.  

She then took innovation to the next level in 2022 with the launch of Narratize, a generative-AI storytelling platform that transforms the ideas of scientific, technical and medical innovators into impactful and detailed content and stories quickly.

It's so important for everyone in the humanities to see that they have just as much entrepreneurial potential as any other person.

Katie Trauth Taylor Entrepreneur and A&S master's grad in English

English to entrepreneurship

With 10 years of successful partnerships with lucrative companies like AAA and Boeing, Taylor is just getting started. And yet, it took a while for her to get here.

“I think sometimes we feel stuck—like, ‘I have to follow a path because I made us a choice about what to major in or what to study.’

  “But really, once you get out into the workforce, you see that everything is almost interdisciplinary and you can see and cut across those boundaries more seamlessly as long as you're a strong critical thinker, a great communicator, and hungry to constantly learn.” 

Taylor was just that person during college. As she matriculated through her institutions, she formed bonds and met many mentors along the way who pushed her towards starting her own business.

“I think wherever you want to grow in your professional life, you just have to ask those questions. Ask ‘what do I need to learn to get there and find my mentors?’ And sometimes, mentors are thrust upon you in the most wonderful ways. And it surprises you,” said Taylor. 

Along the way she continued her love for storytelling and sharing that passion. And yet, she felt that she could help scientists, tech moguls, and medical professionals share their stories in more effective and impactful manners. 

“I had a mentor somewhere along the way, who said ‘look me in the eyes. You know, you could just start a business, right?’” 

And Taylor did just that with Untold Content while she was a research professor at Miami University. Untold Content kept growing and growing, so she made her decision and pursued her new passion full-time, as the leader of the then up-and-coming narrative storytelling strategy company.

Making her mark in tech

“I never got a business degree, but here I am running my second company,” Taylor said.

The generative AI leader took herself into a new avenue of business and networking. However, she identified the potential of using generative AI to push the ideas of innovators in science, technology and medicine into impactful storytelling and long-form content.  As such, she found what would be her company’s foothold in the tech industry. 

“We came to generative AI as a way to solve a problem that we had already discovered, so we already knew the problem. The problem was, it is so challenging to communicate effectively within science, tech and medical industries, and there are so many roadblocks related to that,” said Taylor.

“I've always been in love with that problem. So, we were in this really intensive moment in my first company where we were trying to ask ‘how do we solve that problem at scale?’And so, we were designing tech platform for communicating internally and help people craft more clear and impactful messages.”

She received the perfect gem of knowledge pertaining to this issue during her time at UC from associate professor of English Laura Micciche—a lesson at the basis of her women-led companies.

UC for me was completely life-changing. The people I met were the most generous and thoughtful subject-matter experts that I have ever met, and they remain strong forces in my success today.

Katie Trauth Taylor A&S alum, tech entrepreneur

“Laura Micciche taught me about rhetoric and composition and the way that words hold power and how they structure society and public rhetoric the way they shape opinion,” Taylor said.

Taylor and her company never left their storytelling roots. Her team consists of data scientists, engineers, and ‘narrative scientists,’ people who are science, tech and medical communicators with journalistic and marketing backgrounds. 

“Narrative scientists see the way words make meaning. And that is essentially the design of the predictive models,” said Taylor. “That is the most exciting, possible moment to be able to bring those diverse skill sets together.  That's where the magic happens with Gen AI. And that's where we see solutions that will actually be transformative.”

The teams at Narratize and Untold Content continue with these tenets in mind under Taylor’s leadership.

To the future

Narratize and Untold Content work to help a variety of stakeholders understand the ideas and insights of innovators around the world, from consumers to potential investors. However, given the recent surge in open AI, creatives like journalists and marketers are concerned.

AI software continues to evolve, and as technology advances, it is inevitable that jobs may be lost. And yet, Taylor has her own response to this question: ‘What happens to creatives as AI continues to get better?'

“It's the question everybody is asking. How do we keep pace with this rapid rise of technology? We set out really early on to ask how do we empower people? How do we do AI with good intent?”

One of the major pillars of both companies is to keep AI in the hands of creatives and to ensure AI is nothing but human-led. 

“And that means that we design the user experience to pull insights from the person and to support the person in their workflow and embed the AI into those workflows so they can be more effective in their work and do that more efficiently.”

The focus is on teaching and empowering creatives with generative AI, and according to Taylor, the best path forward is through embracing AI and maintaining focus on amplifying storytelling. An emphasis on focus that she strengthened during her time at UC. 

"UC for me was completely life-changing. The people I met were the most generous and thoughtful subject-matter experts that I have ever met," Taylor said, "and they remain strong forces for my success today."

Featured image at top: Graphic representation of artificial intelligence. Credit/Steve Johnson on Unsplash.

By Serigne Thiam

Student Journalist , A&S Department of Marketing and Communication

[email protected]

• Department of English
• College of Arts and Sciences

Related Stories

Uc‘s college of arts and sciences taps innovative new leadership.

December 20, 2023

The College of Arts and Sciences announced Ryan J. White and Rina Williams as the newest divisional deans of Natural Sciences and Social Sciences. White and Kennedy’s inclusion will bring new focuses and structure around student success and the college of Arts and Sciences’ advancement. Both will officially begin their new terms on Jan. 1, 2024.

UC offers new social justice, Latin American studies degrees

October 7, 2020

University of Cincinnati students can now enroll to earn a Bachelor’s degree in two new humanities programs: Social Justice, and Latin American, Caribbean and Latinx Studies, offered through UC’s College of Arts and Sciences.

UC to offer certificate in disability studies

June 30, 2022

Students at the University of Cincinnati will be able to enroll to earn a certificate in disability studies in fall semester, 2022. Offered through the College of Arts and Sciences, the certificate will explore the experiences of disabled people, and the role of the disability community and the community at large in addressing equity, access and justice. The creation of the certificate program was inspired in part by the experiences of students, says Cheli Reutter, associate professor of American and African-American literature, who helped develop the program.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 21 March 2017

The future of research assessment in the humanities: bottom-up assessment procedures

• Michael Ochsner 1 , 2 ,
• Sven Hug 1 , 3 &
• Ioana Galleron 4  

Palgrave Communications volume  3 , Article number:  17020 ( 2017 ) Cite this article

6627 Accesses

18 Citations

10 Altmetric

Metrics details

• Science, technology and society

Research assessment in the social sciences and humanities (SSH) is delicate. Assessment procedures meet strong criticisms from SSH scholars and bibliometric research shows that the methods that are usually applied are ill-adapted to SSH research. While until recently research on assessment in the SSH disciplines focused on the deficiencies of the current assessment methods, we present some European initiatives that take a bottom-up approach. They focus on research practices in SSH and reflect on how to assess SSH research with its own approaches instead of applying and adjusting the methods developed for and in the natural and life sciences. This is an important development because we can learn from previous evaluation exercises that whenever scholars felt that assessment procedures were imposed in a top-down manner without proper adjustments to SSH research, it resulted in boycotts or resistance. Applying adequate evaluation methods not only helps foster a better valorization of SSH research within the research community, among policymakers and colleagues from the natural sciences, but it will also help society to better understand SSH’s contributions to solving major societal challenges. Therefore, taking the time to encourage bottom-up evaluation initiatives should result in being able to better confront the main challenges facing modern society. This article is published as part of a collection on the future of research assessment.

Similar content being viewed by others

Mapping the community: use of research evidence in policy and practice

Elizabeth N. Farley-Ripple, Kathryn Oliver & Annette Boaz

Insights from a cross-sector review on how to conceptualise the quality of use of research evidence

Mark Rickinson, Connie Cirkony, … Annette Boaz

Social sciences and humanities research funded under the European Union Sixth Framework Programme (2002–2006): a long-term assessment of projects, acknowledgements and publications

Jordi Ardanuy, Llorenç Arguimbau & Ángel Borrego

Introduction

While there is more than a 100 years of scientific inquiry on research and dissemination practices in the natural and life sciences, until recently bibliometric and social studies on science and technology research neglected the SSH ( Hemlin, 1996 ). Therefore, there are methods for research assessment in the natural and life sciences that relate to the practices in these fields and are accepted by the community (even though there are more and more critical voices, see for example, Lawrence, 2002 ; Molinié and Bodenhausen, 2010 ) and the measurement properties are tested by bibliometric research. In the meantime, knowledge on research and dissemination practices in the SSH is scarce, while research assessment did not stop at the gate of the SSH disciplines ( Guillory, 2005 ; Burrows, 2012 ). The growing pressure of accountability, prevailing government practices based on New Public Management and the availability of quantitative data led to the implementation of (quantitative) research assessments also in the SSH during the last decades ( Kekäle, 2002 ; Hammarfelt and de Rijcke, 2015 ; Hamann, 2016 ). The creation of the European Research Area (ERA) increased the importance of research evaluation: the initial communication “Towards a European Research Area” listed under the first theme of action the “mapping of European centres of excellence” and “Financing plan for centres of excellence on the basis of competition” ( Commission of the European Communities, 2000 ); 15 years later, the ERA Roadmap listed the following as the first among the Roadmap’s priorities: “Strengthening the evaluation of research and innovation policies and seeking complementarities between, and rationalization of, instruments at EU and national levels”. ( European Research Area and Innovation Committee, 2015 : 5). The vast majority of research assessments, however, were implemented in a top-down manner by either governments or university administrators. In addition, research assessment procedures usually apply bibliometric and scientometric methods developed for the natural and life sciences that do not reflect SSH research and disseminations practices. Bibliometric research shows that these methods cannot readily be used for the SSH ( Hicks, 2004 ; Lariviere et al., 2006 ; Nederhof, 2006 ). Therefore, research assessment procedures (and oftentimes research evaluation in general) meet strong opposition in the scholarly communities of the SSH.

In the last decade, a number of projects were initiated in Europe to explore research assessment procedures that adequately reflect SSH research practices. These projects did not arise from within the discipline in the sense of auto-regulation or the discontent with the quality or the standing of the discipline. Rather, they are the reaction on how research is assessed through procedures not linked to the functioning of the disciplines itself but to top-down decisions on how research is to be evaluated. Also, with the ERA Roadmap in place, the discussion could no longer be whether research should be subject to systematic research assessments but rather how to assess it. With a few exceptions, however, the bottom-up initiatives unfortunately do not get the attention of research evaluators and policymakers they deserve.

In this article, we give an overview of selected European initiatives that are genuinely reflecting the SSH research practices and were initiated or developed by scholars with an SSH background. Due to restrictions of space, we do not report how SSH research is assessed in unitary evaluation procedures, that is exercises that apply the same basic procedure for all disciplines (for sciences, technology, engineering and mathematics (STEM), as well as for SSH disciplines) and allow only for small adaptions to SSH research practices (for example, use of bibliometrics or not, types of eligible outputs). For this reason, we do not report how SSH research is evaluated in the RAE and REF in the United Kingdom Footnote 1 or the RQF and ERA in Australia as they are clearly top-down (see for example, Kwok, 2013 ), follow a unitary approach and the SSH are not having a major impact on the design of the exercise. Furthermore, the RAE/REF and RQF/ERA procedures are well-documented in the literature. For the SSH in the RAE/REF, see for example Arts and Humanities Research Council (2006 , 2009) ; Butler and McAllister (2009) ; Hamann (2016) ; Johnston (2008) ; Norris and Oppenheim (2003) ; Oppenheim and Summers (2008) . For the RAE/REF in general, see for example Barker (2007) and Hicks (2012) . For SSH related matters in the Australian RQF/ERA, see for example Butler (2008) , Butler and Visser (2006) , Council for the Humanities, Arts and Social Sciences (2009) , Genoni and Haddow (2009) , Kwok (2013) , Redden (2008) . Because there is a wealth of such SSH initiatives in Europe, we also restrict our review to European initiatives and do not report other initiatives such as the Australian ERA and the Humanities Indicators project in the United States ( www.humanitiesindicators.org ).

In what follows, we first present the issues of research assessment in the SSH, such as the methodological issues and the SSH scholars’ critique of the assessment procedures. We then move on to present several bottom-up initiatives taken up in (mainly continental) Europe by concerned SSH scholars. These initiatives set out at different levels and with different scope, from simply improving the situation of SSH data availability and accuracy to complex evaluation procedures involving a broad range of quality criteria and indicators. Some initiatives take place at a local level, others at a national level; and there are even European initiatives concerned with bottom-up research evaluation in the SSH. We conclude with some recommendations for future research evaluation in the humanities.

Research assessment in the SSH

To describe the current situation of research assessment in the SSH, we analyse them from two perspectives. First, we take the perspective of bibliometricians and scientometricians and focus on what they say regarding the adequacy of their methods for SSH research. Second, we analyse the critiques of the SSH scholars regarding those methods, which gives us hints at how to design adequate methods for research assessment in the SSH.

Bibliometrics and scientometrics in SSH research assessments

The application of bibliometric methods to the SSH proved to be problematic and yielded unsatisfying results, so that even bibliometricians caution from applying bibliometric methods to SSH disciplines (see for example, Nederhof et al., 1989 ; Glänzel, 1996 ; Lariviere et al., 2006 ). This is because of several reasons, that we summarize in two main issues: coverage issues and methodological issues.

Coverage issues arise for several reasons. First, in the SSH, chapters in books and monographs are more frequently used as publication channels and get cited more often than journal articles ( Hicks, 2004 ; Nederhof, 2006 ). This leads to severe coverage issues in the most important databases for bibliometric analyses, which are mainly or exclusively based on scholarly journals ( van Leeuwen, 2013 ). Furthermore, even internationally oriented European journals are not covered well in the relevant databases compared with American journals ( Nederhof, 2006 ).

Second, some SSH disciplines are characterized by a more pronounced national and regional orientation ( Nederhof, 2006 ). Nederhof states in his review of bibliometric monitoring in the SSH: “Societies differ, and therefore results from humanities or social science studies obtained in one country may not always be very useful to researchers in other countries” ( Nederhof, 2006 : 83). Thus, even though the topics might be internationally relevant, this kind of output is less visible, as often written in the national languages, seldom covered in the bibliometric databases (see for example, Chi, 2012 ), or even published in other publication channels that are not covered at all (example, reports and other publications directed to national or regional readership).

Third, SSH scholars write not only for the scholarly readers but also for the lay public ( Hicks, 2004 ). This type of literature is usually not taken into consideration in evaluations and certainly not included in the databases used for bibliometric analyses. However, non-scholarly publications are an important part of SSH research and its societal impact.

Methodological issues arise amongst others from the fact that citation behaviour is different in the SSH disciplines. The age of references is remarkably high. Glänzel noted for example in his analysis from 1996 that a 3-year citation window is too short. Given the distribution of the citations over time, almost a 10-year citation window would have to be applied, leading to an obsolete publication set for evaluation purposes ( Glänzel, 1996 ). Furthermore, the citation culture is different ( Hellqvist, 2010 ; Hammarfelt, 2012 ; Bunia, 2016 ). Hicks (2004) notes also that SSH journals are usually more transdisciplinary, which leads to methodological problems such as field normalization.

While this is not a comprehensive analysis of methodological issues of quantitative assessments, it shows that there are several problems with the application of bibliometric indicators in research assessments in the humanities. Importantly, it makes evident that today’s bibliometric methods do not reflect SSH scholarship.

SSH scholars’ critique of quantitative research assessments

If research assessment procedures are to be accepted and the tools and methods should help determining the quantity and quality of humanities research without significant delays, refusal or boycott by the scholarly community, the criticisms put forward by humanities scholars become an important issue. We have analysed SSH scholars’ critique of (quantitative) research assessments elsewhere and summarized them into four main reservations ( Hug and Ochsner, 2014 ). We will only briefly summarize our findings, as relevant for the purpose of this article.

The first reservation relates to the section above: the methods were developed for, and reflect the research practices in, the natural and life sciences ( Vec, 2009 ). This means not only that the assessment practices do not account for SSH dissemination practices (monographs, diverse languages, local orientation, individual scholarship) as noted in the section above, but also that the assessment practices follow the natural sciences’ linear understanding of progress while the SSH scholars share the notion of the “coexistence of competing ideas” ( Lack, 2008 : 14), that is, an ever-increasing knowledge base. This conception of knowledge that is diverse and not dying out is not reflected in most evaluation practices.

Second, SSH scholars have strong reservations about quantification. A joint letter by 24 international philosophers to the Australian government as a reaction to the journal ranking in the Excellence in Research for Australia (ERA) exercise points to this issue: “The problem is not that judgments of quality in research cannot currently be made, but rather that in disciplines like Philosophy, those standards cannot be given simple, mechanical, or quantitative expression” ( Academics Australia, 2008 ). Other scholars argue that research does not produce products or goods in a free market, in which value can be defined according to the products’ economic value or efficiency ( Plumpe, 2010 ; Palumbo and Pennisi, 2015 ). Thus, many SSH scholars fear that the intrinsic benefits of the arts and humanities will be neglected or even lost because of the focus on quantitative measures. The report for the Humanities and Social Sciences Federation of Canada says for example that “some efforts soar and others sink, but it is not the measurable success that matters, rather the effort” ( Fisher et al., 2000 , “The Value of a Liberal Education”, para. 18; see also the report for the RAND corporation McCarthy et al., 2004 ).

The third reservation is the fear of negative steering effects of indicators. SSH scholars anticipate many dysfunctional effects such as mainstreaming or conservative effects of indicators, a loss of diversity of research topics or disciplines due to selection effects introduced by the use of indicators, or importance of spectacular research findings leading to unethical reporting of findings ( Fisher et al., 2000 ; Andersen et al., 2009 ; Hose, 2009 ; Burrows, 2012 ). More and more such negative steering effects of indicators are observed also in the natural sciences ( Butler, 2003 , 2007 ; Mojon-Azzi et al., 2003 ; Moonesinghe et al., 2007 ; Unreliable research. Trouble at the lab, 2013 ). Such findings support the fear of negative steering effects in the SSH.

Fourth, the SSH are characterized by a heterogeneity of research topics, methods and paradigms. Finding shared quality criteria or standards for research assessments becomes an intricate task if there is no consensus on research questions, the suitability of the methods applied and even the definition of disciplines and sub-disciplines ( Herbert and Kaube, 2008 ; van Gestel et al., 2012 ; Hornung et al., 2016 ). If criteria can be found, they are usually informal, refer to one (sub-)discipline and cannot easily be transferred to other sub-disciplines or evaluation situations ( Herbert and Kaube, 2008 ).

Bottom-up procedures for research assessment in the humanities

Despite these critiques of both bibliometricians and scientometricians on the one hand and SSH scholars on the other hand, more and more research assessments in the SSH are implemented. Usually, the procedures for research assessments are implemented in a top-down manner, not taking the situation at the coal face of research into account. However, there are several initiatives that reflect the characteristics of SSH research. In the following, we focus on initiatives that come from within the SSH research communities or are at least developed by scholars from SSH disciplines, genuinely taking into account SSH research practices in their approaches 2 . All of them address at least one of the issues mentioned in the previous section. While these bottom-up initiatives are more likely to be accepted by SSH scholars, some of them still face strong opposition or are boycotted.

Improving the databases

Considering that typical SSH publications (for example, books, proceedings, publications in local languages) are badly represented in current databases Footnote 2 , efforts have been made in several countries to improve coverage, especially in the countries with a performance-based funding model, like Spain, Norway, Denmark, Belgium (Flanders) and Finland ( Giménez-Toledo et al., 2016 ). There was also an attempt to create a full-coverage bibliographic/bibliometric database for Europe, but it did not result in an implementation of a European-wide database or standard ( Martin et al., 2010 ). In parallel, the ERIH project intended to create a European journal list for the SSH to overcome the problems of under-representation of (European) SSH journals in the main bibliometric databases; however, the project faced strong opposition ( Andersen et al., 2009 ), had to be remodelled (see Lauer, 2016 ) and was relaunched under the name ERIH Plus 4 .

Attempts to create publication databases suitable for the humanities have sometimes also been organized at the level of disciplines. The EERQI project included such a database for the educational sciences on the European level; it also investigated methods for using the data in research evaluations in a meaningful way ( Gogolin et al., 2014 ; Gogolin, 2016 ). The database allows scholars to search for publications using keywords in one language, while retrieving results in all four languages covered in the database. Therefore, beyond evaluative purposes, centralized and systematic coverage of SSH production appears as an endeavour with multiple potential benefits, such as improving information retrieval for scholars and widening access to publications in multiple languages.

In all cases, consciousness is raising about the need to compile complete and interoperable databases of SSH scholarly and non-scholarly outputs, so as to gain accurate knowledge about productivity and publication behaviour in these very diverse disciplines. At the same time, the creation of such databases should go hand in hand with the development of standards regarding their use, including standards on how not to use them.

An SSH approach towards bibliometrics and scientometrics

Bibliometric analyses face many problems when applied to SSH disciplines ( Nederhof et al., 1989 ; Archambault et al., 2006 ; Nederhof, 2006 ; van Leeuwen, 2013 ). However, Hammarfelt (2016 : 115) observes a shift from investigating coverage issues towards studying the characteristics of SSH publication practices and developing bibliometric approaches sensitive to the organization of SSH research fields. This includes, but is not limited to, extending bibliometric analyses to non-source items ( Butler and Visser, 2006 ; Chi, 2014 ) or the relatively new Book Citation Index ( Gorraiz et al., 2013 ), using other databases like Google Scholar ( Kousha and Thelwall, 2009 ) or data from social media services, the so-called altmetrics ( Holmberg and Thelwall, 2014 ; Mohammadi and Thelwall, 2014 ; Zuccala et al., 2015 ; Zuccala and Cornacchia, 2016 ), analysing the inclusion in library catalogues ( White et al., 2009 ), exploring national databases with full coverage ( Giménez-Toledo et al., 2016 ), extending data to references in research grant proposals ( Hammarfelt, 2013 ) or to book reviews ( Zuccala and van Leeuwen, 2011 ; Zuccala et al., 2015 ), exploring collaboration ( Ossenblok and Engels, 2015 ) and publication patterns ( Chi, 2012 ; Ossenblok et al., 2012 ; Verleysen and Weeren, 2016 ). From a more pragmatic point of view, attempts are made to “weigh” the various outputs, such as journals or books in the SSH, similar to the journal impact factor, commonly used in the sciences ( Giménez-Toledo, 2016 ).

While most of this research is done by bibliometricians and scientometricians, there are more and more SSH scholars still focusing on their SSH career and at the same time investigating research practices in their disciplines, such as citation practices ( Drabek et al., 2015 ; Bunia, 2016 ), the influence of databases ( Lauer, 2016 ), the relation of bibliometric indicators to research practices ( Gogolin, 2016 ) or career building and dissemination ( Williams and Galleron, 2016 ). Also, more methodological analyses are conducted by SSH scholars, such as the investigation of the inter-rater reliability of research assessment procedures ( Riordan et al., 2011 ; Plag, 2016 ) or the correlation of bibliometric and expert-based procedures ( Ferrara and Bonaccorsi, 2016 ). While Hammarfelt requests to build a “bibliometrics for the humanities” ( Hammarfelt, 2016 : 115), Zuccala (2016 : 149) goes further and demands that bibliometricians find ways to teach bibliometrics to humanities students so that a “new breed of humanistic bibliometrician can emerge successfully”.

Bunia (2016) , a German literature scholar, argues that the problem of applicability of citation analyses might, besides coverage and technical issues, as well be intrinsic to the field of literary studies: literature scholars seem not to read the work of their colleagues in the same field or at least they do not use or cite them in their own publications. He advocates using bibliometric analyses to study the citation behaviour of literary scholars since this is also important knowledge for the scholarly community in the field. The use of bibliometric methods in research assessment will not be possible until light is shed on this issue.

Summarising the situation of bibliometrics and scientometrics in the SSH, bibliometric methods cannot be readily used for research assessment in the SSH. But bibliometrics adapted to the SSH can help to study research practices, publication and citation practices as well as other practices important for knowledge production in the SSH. A thorough look at citation habits can also broach some delicate issues in research practice. Applied with some care, some quantitative indicators can also be used to complement peer review if they are defined bottom-up, that is, from within the disciplines.

Funding SSH research grants

Third-party funding becomes more and more important because, first, a higher share of the research budget in most countries is competitively distributed through funding organizations ( van den Akker, 2016 ), second, because the amount of third-party funding is used in most assessment procedures at least as an information criterion ( Ochsner et al., 2012 ). Especially for the careers of young scholars, grant allocation gains importance: on the one hand, job opportunities of young researchers are more and more characterized by short-term contracts based on external funding ( van Arensbergen et al., 2014b ); on the other hand, allocated grants serve as a prove of excellence in talent selection decisions ( van Arensbergen et al., 2014a ).

Third-party funding implies ex-ante research assessment, that is, research is assessed before it has been conducted. While most ex ante assessments are based on peer review, many of them use bibliometric data to inform the peers. Certainly, these processes have been already in place for some time, mainly unnoticed by most SSH scholars because research grants are less important for them as they do not need expensive infrastructure to do their research ( Krull and Tepperwien, 2016 ). The growing importance of grants in science policy at the national and international level, however, has drawn the attention of SSH scholars to the processes of distributing research grants because there are huge differences in the distribution of grants between the STEM and SSH disciplines ( Krull and Tepperwien, 2016 ), not to mention the differences of amounts.

The lower chances and the lower amount of acquired third-party funding have their roots in the epistemic differences of research practices between the STEM and SSH, as well as in a different disciplinary organization and divergent practices of research evaluation. Only a minority of SSH scholars needs expensive instruments to conduct experiments, as opposed to the basic needs SSH scholars usually express, which are a computer, access to archives, travel expenses and research time ( Krull and Tepperwien, 2016 ). Therefore, third-party funding did not play a role for a long time in most SSH disciplines and grants are usually of a comparably low amount.

Second, the way SSH scholars appreciate research output of colleagues is quite different from how STEM researchers do. SSH scholars are much more critical. They criticize even work they value as excellent. A bit-by-bit examination is considered a proof of love. In interdisciplinary panels, STEM researchers do not agree on funding research that is heavily criticized. Because SSH scholars always do criticize the work of their colleagues, irrespective of the quality of the research, SSH scholars are often discriminated in interdisciplinary granting schemes ( Krull and Tepperwien, 2016 ) even though this practice of criticizing works fine within SSH disciplines ( König, 2016 ; Krull and Tepperwien, 2016 ).

Third, in the STEM disciplines, paradigmatic issues are usually disputed internally while at the outside there is coherence. The SSH disciplines, however, do not resolve such issues but allow for diversity within their fields ( van den Akker, 2016 ). Of course, this is rooted in a different understanding of scholarly work—linear progress in the STEM disciplines versus increase of the knowledge base in the SSH disciplines ( Lack, 2008 )—but it is also the result of a lack of organization. This leads to further marginalization as the SSH disciplines do not stand together to criticize univocally the short-sighted focus on the linear progress of science ( van den Akker, 2016 ) and to demand funding schemes adequate for SSH research with a powerful united voice.

At the same time, some funders are frustrated that their schemes do not attract more proposals from SSH disciplines ( König, 2016 ), maybe because SSH scholars do not take the risk of writing a proposal when past experiences seem to make it likely that it will be turned down. Therefore, the Fritz Thyssen Stiftung and the VolkswagenStiftung have created a funding programme adapted to the needs of humanities scholars entitled “Focus on Humanities” that includes the grant Opus Magnum that could bridge the gap between the humanist way of doing research and at the same time adding a competitive component. In addition, the VolkswagenStiftung (2014) has established bottom-up guidelines regarding how to recognize intellectual quality in the humanities collected in a workshop with renowned scholars and young scholars.

SSH research practices and criteria for research quality

To assess research performance, there should be an explicit understanding of what “good” research is, since any assessment points out “high quality” research or tries to judge which research is “better” ( Butler, 2007 ). However, not much is known what actually research quality means (see e.g. Kekäle, 2002 ), especially so in the SSH. The literature on research assessment actively avoids this topic, while existing tools and procedures of research assessment do not include an explicit understanding of research quality ( Glänzel et al., 2016 ). Rather, authors revert to “impact”, which is easier to measure but not congruent with “quality” ( Gumpenberger et al., 2016 ) Footnote 3 . Therefore, if SSH research is to be assessed appropriately, there must be knowledge on what actually research quality means in these disciplines and assessment procedures must relate to the conceptions of research quality of the assessed scholars. To get a grasp on what guides judgement on what is good or bad research, we need empirical knowledge on research practices and the notions of quality that humanities scholars use to interpret, structure and evaluate the events and entities in their research activities.

During the last hundred years, scholars analysed research practices of the STEM disciplines, especially the natural sciences, in detail; however, the newly emerging field of social studies of science neglected its own (SSH) disciplines until recently ( Hemlin, 1996 : 53; Hammarfeldt, 2012 : 164). The literature so far describes the characteristics of SSH research in the following way: a) SSH research is interpretative, that is, humanities research is mainly text- and theory-driven and social sciences are more concept-driven, while the natural sciences set up their studies to answer specific questions and are progress-driven ( MacDonald, 1994 ; Guetzkow et al., 2004 ; Lamont, 2009 ); b) it is reflective and introduces new perspectives in academia, by fostering discursive controversy and competing visions ( Fisher et al., 2000 ; Hellqvist, 2010 ). With regard to the society, they bring a decisive contribution to the training of critical thinking as a prerequisite for democracy ( Nussbaum, 2010 ) or to the critical examination of modern trends, such as technologisation ( Luckmann, 2004) ; c) it is mainly individual ( Finkenstaedt, 1990 ; Weingart et al., 1991 ), few publications are co-authored ( Hemlin, 1996 ; Hellqvist, 2010 ) and research is often connected to the person conducting it ( Hemlin and Gustafsson, 1996 ; Guetzkow et al., 2004 ); d) productivity is not that important for research performance in the SSH ( Hemlin, 1993 ; Fisher et al., 2000 ; Hug et al., 2013 ); e) societal orientation is important, i.e. research is meant to influence society, direct interaction with society is part of SSH research ( Weingart et al., 1991 ; Hellqvist, 2010 ; Hug et al., 2013 ); but f) the influence of society or other stakeholders outside of academia, such as external funding, on SSH research is evaluated negatively ( Hemlin, 1993 ; Hug et al., 2013 ; Ochsner et al., 2013 ).

These characteristics must be considered when assessing SSH research. Therefore, there are several bottom-up projects by SSH scholars that analyse how quality is perceived in the SSH disciplines. The European Educational Research Qualitiy Indicators (EERQI) project ( Gogolin et al., 2014 ) started from the discontent with the current assessment practices applied to educational research ( Gogolin, 2016 : 105–106). The project lasted from 2008 to 2011 and aimed at the development of a set of tools (as opposed to a ranking or rating or a single indicator) to detect research quality (for a summary of the project and its tools, see Gogolin, 2016 ). The project differentiates between extrinsic quality indicators, that is, quality indicators that are not inherent to the text (such as number of citations, webometrics, authorships), and intrinsic quality indicators, that is, indicators that are inherent to the text (such as rigour, stringency). Part of this set of tools was a peer review questionnaire that included five intrinsic quality criteria for educational research: rigour, originality, significance, style and integrity. The criteria were developed in collaboration with experts in the field, mainly organized within national associations ( Gogolin and Stumm, 2014 ). The project included also an exploratory natural language processing system to highlight the most important sentences in an article. The idea behind the tool was to help reviewers judge an article’s quality by guiding their attention to the most important parts of an article ( Sandor and Vorndran, 2014a ). The tests with the tool showed that while texts in STEM disciplines follow a clear structure and reveal a high potential for automated highlighting, articles in SSH disciplines do not follow such a standard structure. Using keywords and different categories of sentences (for example, problem, summary), the authors argue that highlighting might considerably reduce the time needed for reviewing an article. However, highlighting did not cover two criteria appropriately, that is, integrity and rigour, thus, reviewers using highlighted versions of the article did not always rate those criteria. Furthermore, accuracy of the highlighting differs between (sub-)disciplines and the agreement between automated summaries and reviewers’ summary differed between languages ( Sandor and Vorndran, 2014a : 50–52). While the authors argue that automatic highlighting seems to work to a certain degree and that a highlighting tool is a promising help to ease peer review workload, the results suggest also that there are severe limits to its usefulness for the assessment of SSH manuscript, especially with regard to the quality criteria. Two out of five criteria tend to be overseen (i.e. integrity and rigour) and language and (sub-)discipline impact the results: summaries by English experts are closer to the sentences highlighted by the tool than the summaries of the French, while the error rate of the highlighting tool is higher for psychological articles than for sociological or historical. However, the authors used this tool also in the multilingual search engine for the EERQI-database and found that it can enhance the search results ( Sandor and Vorndran, 2014b ).

Also for educational research, Oancea and Furlong (2007) developed criteria for research performance. They define educational research as practice-based and state that such research is not confined to scientificity (that is, discoveries of universal findings or even laws), impact or economic efficiency but also encompasses, amongst others, methodological and theoretical rigour, dialogue, deliberation, participation, ethics and personal growth. They argue that the evaluation of practice-based research has to cope with the entanglement of research and practice, which means that evaluation still has to reflect reasoning and knowledge but it has also to open up for more experimental modes of knowledge coming from within a context of concrete situations and first-person action. While they do not aim at setting standards of good research practice, they conclude that research assessment needs to re-integrate a cultural and philosophical dimension that had been lost in the current discourse of research assessment ( Oancea and Furlong, 2007 ).

A more descriptive approach was chosen by Guetzkow, Lamont and Mallard (2004) . They analysed interviews with peer review panellists from multidisciplinary fellowship competitions and found that originality was the most frequently mentioned criterion for judging applications. They thus focused on analysing originality and found that originality is defined differently across different disciplines: Humanists referred often to originality of data and approach whereas social scientists emphasized originality of methods. Besides originality, however, there were also other important criteria, for example, clarity, social relevance, interdisciplinarity, feasibility, importance. Note that those criteria are not necessarily criteria for judging research quality but proposals for a fellowship. Because the authors focused on originality for a more thorough analysis, we do not learn whether there were also disciplinary differences in the salience of those other criteria and in the meaning that was given to the criteria. Given the results regarding originality, however, it is likely that such differences do exist.

The project “Developing and Testing Quality Criteria for Research in the Humanities” ( Ochsner et al., 2016 ) applied a strict bottom-up approach and developed a framework for the exploration and development for quality criteria for SSH research ( Hug and Ochsner, 2014 ) that consists of four pillars: adopting an inside-out approach (adequate representation of the scholarly community, also of young scholars, in the development process; discipline specific criteria), applying a sound measurement approach (linking indicators to quality criteria derived from the scholars’ notions of quality), making the notions of quality explicit (apply methods that can elicit criteria from the scholars’ tacit knowing of research quality to draw a comprehensive picture of what research quality is in a given discipline; make transparent which quality aspects are measured or included in the assessment and which are not), and striving for consensus (methods and especially criteria to be applied in research assessment have to be accepted by the community). This framework was applied to three humanities disciplines, known to be difficult to assess with scientometric methods: German literature studies, English literature studies and art history. In a first step, the scholars’ implicit knowing about research activities was investigated, made explicit and summarized into different conceptions of research using Repertory Grid interviews ( Ochsner et al., 2013 ). The results showed that two conceptions of research exist, specifically a modern and a traditional one. This differentiation is not connected to quality: both the modern as well as the traditional research can be of excellent or bad quality. Remarkably, the results also reveal that many commonly used indicators for research assessment, such as interdisciplinarity, internationality, cooperation and social impact, are, in fact, indicators for the modern conception of research and are not related to quality ( Ochsner et al., 2013 ). Besides the observations about scholars’ conceptions of research, quality criteria were extracted from the scholars’ notions of quality. In a second step, these quality criteria were completed and rated by all scholars in the three disciplines at the Swiss and LERU universities (League of European Research Universities), thus identifying consensual quality criteria for research using the Delphi method ( Hug et al., 2013 ). According the measurement approach, indicators were identified for the consensual quality criteria ( Ochsner et al., 2012 ) and also rated by the scholars. The results of the project indicate that there are a lot of quality criteria for research in the humanities to consider in research assessments. Many criteria are common to all three disciplines but there are also some discipline specific criteria. Furthermore, there is a mismatch between the humanities scholars’ quality criteria and the criteria applied in evaluation procedures ( Hug et al., 2013 ). Importantly, only about 50% of the relevant quality criteria can be measured with quantitative indicators. Therefore, humanities scholars will be critical of research assessments by means of indicators . Concerning a research assessment by means of quality criteria the studies show that a broad range of quality criteria must be applied and disciplinary differences have to be taken into account. With a certain amount of care, research indicators linked to the relevant criteria can be used to support the experts in research assessments (informed peer review). The project shows that humanities scholars are ready to take part in the development of quality criteria for research assessment if a strict bottom-approach is followed and transparency is assured ( Ochsner et al., 2014 ).

In the context of a broad examination of research assessment in law studies, Lienhard et al. (2016) present quality criteria for research in law studies drawing from the first findings of the project described above ( Hug et al., 2013 ) and complementing them with discipline specific criteria from the law studies. Being a discipline closely connected to a profession, the authors also included professionals (lawyers) into their analysis and find differences in the preferences for quality criteria between professors and lawyers, such as originality, reflexivity and theoretical soundness being emphasized much more by professors than lawyers, while clear language and correctness was more important to lawyers. Besides differentiating evaluations by different stakeholders, for example professors, lawyers or funders, they also differentiate between different assessment situations, for example, research evaluation, assessment of dissertations and habilitations or assessment of scholarly journals ( Lienhard et al., 2016 : 177).

In France, the Maison des Sciences de l’Homme en Bretagne (MSHB) supported two bottom-up projects related to research assessment in the humanities (for an overview see Williams and Galleron, 2016 ). The first project, IMPRESHS, was destined to investigate the dissemination practices and impact paths of research conducted by Breton scholars from various SSH disciplines (see https://www.mshb.fr/ projets_mshb/impreshs/2314/). Through focus group interviews and a thorough analysis of CVs, the project tried to identify publications with potential impact outside academia, as well as non-academic stakeholders of SSH researchers. The goal of the project was to understand what kind of relations SSH scholars build with these stakeholders, and to what extent one finds practices of co-creation of knowledge in France, such as described within the European project SIAMPI ( http://www.siampi.eu ). One of the major outcomes of the project is to have uncovered that many SSH scholars exercise a form of auto-censorship when it comes to declaring forms of research or outputs destined to a broader or non-scholar readership, these not being included in institutional forms of reporting or in CVs. This finding draw the attention of the project team upon the problems French scholars face when they come to declaring their work, since available fields in templates from AERES (the national agency for evaluation of higher education and research), or metadata structure in national repositories (such as HAL—Hyper Articles en Ligne) do not do justice to the large variety of outputs SSH research produces beyond the well-known books traditionally associated with the field. The project ultimately produced a more refined typology of outputs, which supported the creation of a pilot database destined to cope in a more appropriate way with the wealth and variety of SSH research.

The second project, QualiSHS, looked at how evaluative reports produced by AERES reflect disciplinary representations of quality. All evaluative reports produced in 2010–2011 about the activity of all the research units in history and law from two French regions (Bretagne and Rhône-Alpes) have been scrutinized using methods and tools from corpus linguistics, in search of formulations allowing to understand how peer experts conceptualize and perceive quality in the activities and outputs they evaluate. While interviews conducted in parallel confirmed that experts from the two investigated fields diverge regarding their perceptions of quality—a finding which is in line with what other studies pointed out about the diversity of SSH disciplines when it comes to the conceptualization of research quality (see for example, Hug et al., 2013 ; Gogolin and Stumm, 2014 ; Lienhard et al., 2016 )—it appears that reports do not echo these specificities adequately, since the main criteria they put forward are invariably the coherence of the research conducted in the evaluated unit and its productivity. It is not surprising, therefore, that the French SSH community found that the evaluation conducted by AERES was unsatisfactory on the whole and called for a radical modification of the exercise—a vow that was only very partially answered through the evolution of AERES towards HCERES 6 .

National research evaluation practices and the SSH

There are several projects on a national level that approach (national) research assessment in the SSH from a bottom-up perspective or that have designed the model to reflect SSH specifities. The inclusion of the SSH follows different degrees, from implementation of a performance-based funding model under the lead of an SSH scholar and thus accounting for SSH research practices from the beginning (some even say that the system gives the SSH an advantage, see Aagaard et al., 2015 ) in Norway ( Sivertsen, 2016 ) to a purely bottom-up approach based on research on SSH research practices and their impact on evaluation methods in Switzerland ( Loprieno et al., 2016 ).

The so-called “Norwegian model” ( Schneider, 2009 ) has caught considerable attention during the last years, and similar models were implemented in several countries (Belgium: Flanders, Denmark, Finland and Portugal). The Norwegian model is a performance-based funding model that should “represent all areas of research equally and properly” ( Sivertsen, 2016 : 80). The design of the model is a “simple pragmatic compromise” ( Sivertsen, 2016 : 80): one bibliometric indicator to cover all areas of research comprehensively rather than several representations of publication practices for individual disciplines. It consists of three components: a national data base that fully covers peer-reviewed scholarly output from all disciplines including books, a simple publication indicator dividing publications in level 1 and level 2 publications with a system of weights that makes discipline-specific publication traditions comparable at the level of institutions, and a performance-based funding model that reallocates a small fraction of the yearly funding according to the results of the indicator ( Sivertsen, 2016 : 79). Of course, the Norwegian model would also work without the third component (performance-based funding).

The indicator separates non-academic from academic publications by channels (books: publishers, journal articles: journals). The non-academic publications are not eligible for the performance indicator, while the academic publications are further divided into level 1 and level 2 publications. Level 2 publications cannot represent more than 20% of the world’s publications in a field. The government selects renowned scholars (deans, representatives from learned societies), from all major areas of research to be involved in the assignment process of publishers and journals to the levels, resulting in discipline-specific lists of channels.

The system gets more attention from the SSH scholars than from scholars of other areas. While initially the reaction was negative because it turns scholarly output into measures and the system is not designed to cover all scholarly activity but only academic publications, the evaluation of the system showed that there was no major discontent about the system among the scholars ( Aagaard et al., 2015 ). This might be well because of the fact that the indicator showed a high productivity of the SSH disciplines. In addition, while the main effect of the system is an increase of publication activity, the publication patterns did not change: book publishing, international publishing, and language use remained stable. Of course, the evaluation showed also some issues of the funding system: the fractionalizing of authorships favours the SSH, the assignment of experts in the definition of the publication levels is not transparent, and there is unintended use of the system on the individual level ( Aagaard et al., 2015 ).

In the Netherlands, the Royal Academy of the Arts and Sciences criticized the predominance of methods for (and from) national and life sciences in assessment practices in a report called “Judging Research on its Merits” and asked for specific methods for evaluating SSH disciplines in 2005 ( Royal Netherlands Academy of Arts and Sciences, 2005 ). In 2009, the Committee on the National Plan for the Future of the Humanities stated that the existing assessment tools are inadequate to judge the quality of humanities research and advised the Academy to develop a simple, clear and effective system of indicators for the humanities ( Committee on the National Plan for the Future of the Humanities, 2009 ). Thus, the Academy installed a Committee on Quality Indicators in the Humanities, whose report was published in 2011 ( Royal Netherlands Academy of Arts and Sciences, 2011 ). The committee summarizes the situation of research assessment in the humanities in the following way: some policy makers have too high expectations for a simple and purely metric system to compare research performance between research groups and even disciplines. On the other hand, there is too high an aversion against “measuring” research quality and management tools in general in the humanities disciplines. The committee thus suggests a mid-way solution and promotes applying an informed peer review process for SSH research assessments. Peer reviewers assess research along two dimensions, scholarly output and societal quality. Each of the dimensions is assessed using three criteria, that is, scholarly/societal publications or output, scholarly/societal use of output, evidence of scholarly/societal recognition. Each of these criteria can be measured by some quantitative indicators to support the peers in the decision making (for a schematic overview, see Royal Netherlands Academy of Arts and Sciences, 2011 : 47). This should add some inter-subjectivity to the peer review process while at the same time recognizing that also the quantitative indicators usually find their base in peer review in the first place ( Royal Netherlands Academy of Arts and Sciences, 2011 : 11).

The German Council of Science and Humanities ( Wissenschaftsrat ) reacted in 2004 to the growing importance of university rankings criticizing their methodology and validity with recommendations on research rankings ( Wissenschaftsrat, 2004 ). It established a comprehensive pilot study for developing and testing a national research rating in the disciplines chemistry and sociology. While such exercises rarely provoke strong reactions in the natural and life sciences, it is more controversial in SSH disciplines. Nevertheless, the research rating in sociology worked out well but met also criticism, especially the non-transparency of the plenary discussions in the panel annihilating the independency of the judgements of the two peers per research unit was pointed out as a danger to the objectivity and validity of the rating ( Riordan et al., 2011 ). In 2008, the Wissenschaftsrat decided that pilot studies in other disciplines are to be conducted to improve the procedure ( Mair, 2016 ). History was selected for the pilot study in the humanities. However, the rating for history spurred strong resistance and ended with a boycott by the Association of German Historians ( Plumpe, 2009 ). Mair (2016) suggests that the resistance of the historians was mainly due to miscommunication of the Wissenschaftsrat leading to a perception of a top-down-imposed assessment. To make the bottom-up intentions more explicit, a working group was created that worked out modifications to adapt the procedure to the characteristics of humanities research ( Wissenschaftsrat, 2010 : 203–205). In 2012, a pilot study in the humanities was eventually conducted. While still against the notion of quantifying research performance, the associations of English and American Studies decided to take part in the exercise ( Stierstorfer and Schneck, 2016 ). The Wissenschaftsrat qualified the exercise as a success that showed that such a rating is possible in the humanities; the humanities scholars involved in the exercise acknowledged the effort by the Wissenschaftsrat to adapt the procedure to the humanities but also identified some negative aspects and consequences of the exercise, such as a division into different sub-disciplines instead of a focus on commonalities ( Hornung et al., 2016 ).

In Switzerland, the Rectors’ Conference of the Swiss Universities (CRUS, since 1 January 2016 called swissuniversities) published in 2008 a position paper on research assessment entitled “The Swiss Way to University Quality”, which includes ten recommendations for quality monitoring ( CRUS, 2008 ). According to the CRUS, each Swiss university has its own specialization. Therefore, quality assurance has to be accustomed to the mission of each university. A national assessment procedure would therefore not make much sense. Instead, each university should build its own quality assurance system. A potential analysis for bibliometric indicators for research monitoring showed that these procedures are not fitted for use in the SSH. Therefore, a project entitled “Mesurer les performances de la recherche” was initiated that focused on the diversity of SSH research because research “includes a wide array of aspects, from the discovery of new knowledge and promoting young researchers to potential impacts on the scientific community and society” ( Loprieno et al., 2016 : 14). Since the relevance of these aspects differs between disciplines and university missions, the project paid particular attention to such differences and particularities of the disciplines. The project lasted from 2008 to 2012 and was followed by a second project during the time period of 2013 to 2016. In these two projects, several bottom-up initiatives were funded that researched such diverse topics as, amongst others (for a complete overview of the projects, see Loprieno et al., 2016 ), profiling in communication sciences ( Probst et al., 2011 ), cooperation of research teams with university partners as well as external stakeholders ( Perret et al., 2011 ), notions of quality of literature studies and art history scholars ( Ochsner et al., 2016 ), evaluation procedures and quality conceptions in law studies ( Lienhard et al., 2016 ), academic reputation and networks in economics ( Hoffmann et al., 2015 ).

At the same time, the Swiss Academy of Humanities and Social Sciences (SAGW) started a bottom-up initiative on reflections on research assessment in SSH disciplines. Following a conference on the broader topic entitled “For a New Culture in the Humanities” ( SAGW, 2012b ), the SAGW published a position paper on new developments in the humanities, including recommendations on assessment practices ( SAGW, 2012a : 32–36) that emphasizes the importance of bottom-up definitions on quality criteria and methods. The SAGW subsequently funded projects within their member associations to develop their recommendations or standards for research assessments in their disciplines. The resulting report features statements from Asian and Oriental studies, area studies, cultural and social anthropology, peace research, political sciences, art history and environmental humanities accompanied by a synthesis report by the SAGW ( Iseli, 2016 ).

Bottom-up initiatives at the European level

The different assessment procedures applied at the university or national level, the initial exclusion of SSH research in the ERC Grant-schemes as well as the initial concerns of severe cut-backs for the SSH in the Horizon 2020 program ( König, 2016 : 154–155) led to a higher interest of SSH scholars in the topic of research assessment. As the sections above show, there is a rise in SSH research on research assessment and evaluation, leading to sessions or even tracks dedicated to SSH research assessment at international scientometric conferences like the ISSI 2015 ( www.issi2015.org ) or the STI 2016 (sti2016.org) conferences, or to an international conference dedicated exclusively to SSH research evaluation, RESSH 2015 ( www.ressh.eu ). Even more important, SSH scholars team up with scientometricians concerned about the state of SSH research assessment (often SSH scholars themselves) in a European association called EvalHum initiative ( www.evalhum.eu ). EvalHum sets out to motivate and support bottom-up work on research evaluation in the SSH and encourages best practices in research evaluation in SSH that ensure adequate assessment procedures for the respective disciplines. EvalHum is also a forum on this topic and will strive for an accurate recognition of SSH research at the European level.

Currently, there is a COST Action entitled “European Network for Research Evaluation in the Social Sciences and Humanities (ENRESSH)” (CA-15137) that brings together SSH scholars from 30 European countries working together to improve assessment procedures in and for the SSH ( http://www.cost.eu/COST_Actions/ca/CA15137 ). The idea behind the action is “evaluating to valorize” because applying ill-adapted methods lead to under-valuation of SSH research. Participants in the Action share data about SSH research and confront methodologies, resulting in co-authored publications but also in policy briefs, collections of best practices and, ultimately, guidelines for SSH research evaluation. ENRESSH seeks also to involve the different stakeholders having a say in assessment principles and processes, to progress towards adequate frameworks and practices of SSH research. The Action consists of 4 Work Groups. The first Work Group focuses on the conceptual frameworks for SSH research assessment and studies the SSH knowledge production processes and strategies as a basis for developing adequate assessment procedures reflecting the SSH research practices. It investigates SSH scholars’ perceptions of research quality, peer review practices and national assessment practices. The second Work Group is about societal impact and relevance of SSH research. It observes the structural requirements needed for a smooth transfer of SSH research to the society, the national policies towards transfer to socio-economic or NGO partners, proposes procedures to collect data about engagement with the society and measures to better value the SSH. The third Work Group concerns databases and the use of data for understanding SSH research. It builds standards for the interoperability of, and methods for integrating data from, current research information systems and repositories dedicated to the SSH, to allow comparability of SSH publishing practices in various countries. It analyses the characteristics of SSH dissemination channels, develops common rules for building databases, designs a roadmap for a European bibliometric database and develops alternative metrics for the SSH. The fourth Work Group is concerned with the dissemination of the results of the Action. It builds a list of relevant European stakeholders in SSH research assessment and interacts actively with them and organizes conferences.

The future of research assessment in the humanities

While until recently research on assessment in the SSH focused on the deficiencies of the current assessment methods, such as bibliometrics and scientometrics, there is now much research going on that takes a bottom-up approach and focuses on research practices in the SSH and reflects on how to assess SSH research with its own methods instead of applying and adjusting the methods developed for and in the natural and life sciences (see also Hammarfelt, 2016 : 115). This is an important development because we can learn from the examples shown in the sections above that whenever the scholars felt that the assessment procedures were imposed top-down without proper adjustments to SSH research, it resulted in boycott or resistance (see for example, Academics Australia, 2008 ; Andersen et al., 2009 ; Mair, 2016 ).

The projects presented in this article show furthermore that if the assessment procedures adequately reflect the SSH research practices, scholars are ready to collaborate (for example, Giménez-Toledo et al., 2013 ; Ochsner et al., 2014 ) and to accept more easily research assessment, like in the Norwegian or German case ( Aagaard et al., 2015 ; Sivertsen, 2016 ; Stierstorfer and Schneck, 2016 ). Full-coverage databases including all relevant document types are of value for scholarly work ( Gogolin, 2016 ; Sandor and Vorndran, 2014a , b ) and increase the visibility of humanities research production ( Aagaard et al., 2015 ). While there are some degrees of convergence in some countries regarding their databases ( Giménez-Toledo et al., 2016 ), the conditions for full interoperability have yet to be discussed. It also has to be born in mind that universities fulfil different missions and countries face diverse challenges. Criteria and procedures for research evaluation should be adapted to the missions of the universities and to the specific aims of the evaluation ( Loprieno et al., 2016 ).

The future of research assessment in the humanities lies therefore in bottom-up procedures that are based on the research practices in the respective disciplines. However, the projects presented in this article show that more research on the research practices in the humanities is needed. Such research has only started. If bottom-up approaches are to be followed, more knowledge is needed on how research is conducted and disseminated as well as how it is used by different stakeholders including the SSH researchers themselves.

Combining the approaches and the insights on SSH research production presented in this article, we propose the following recommendations for research assessment in the humanities (these recommendations draw on Ochsner et al., 2015 ):

The preferred method of evaluation is informed peer review : peer review is accepted among scholars as an assessment procedure. However, it has several drawbacks such as, for example, poor inter-subjectivity and low reliability through dependency on the panel composition ( Bornmann, 2011 ; Riordan et al., 2011 ; Royal Netherlands Academy of Arts and Sciences, 2011 ). Scientific and political measures can however be taken to reduce these inconveniences, such as applying a fair evaluation process that grants the evaluated scholars the possibility to comment upon the process and its results.

A broad range of quality criteria has to be taken into account. The quality criteria must be developed bottom-up and reflect the notions of quality of the assessed scholars ( Hug et al., 2013 ; Ochsner et al., 2013 ) as they alone can judge what quality in the discipline actually is and they do see research quality predominantly as academic quality ( Kekäle, 2002 ). To assure that all paradigms and research traditions as well as new ways of thinking are included, quality criteria should be developed surveying all scholars to be evaluated.

For the quality criteria that reach consensus among the scholars, indicators can be identified. The scholars should rate the indicators with regard to how these indicators are measuring the criterion adequately.

From the quality criteria and indicators that reach consensus among the scholars, an evaluation sheet is to be created. The evaluation sheet thus includes both criteria that can be measured with indicators and criteria that cannot be measured ( Ochsner et al., 2012 ).

Other stakeholders’ criteria for research performance can be included in the evaluation sheet to take into account other goals of research than academic research quality ( Royal Netherlands Academy of Arts and Sciences, 2011 ). While not developed specifically for the humanities but in a way that allows a bottom-up approach to societal impact, the “Evaluating Research in Context”-project could serve as an example ( Spaapen et al., 2007 ). The criteria and indicators from other stakeholders should be indicated as such to ensure the transparency to the researchers and to make visible what is important from an academic point of view and what is important from other stakeholders’ view.

The peers must rate every criterion on its own, which is in line with the insights of Thorngate et al. (2009) who summarize the findings of their comprehensive research on decision making the following way: judging something overall is usually inconsistent and not adequate for judging merit while judging separately according to specified criteria reveals more reliable results ( Thorngate et al., 2009 : 26). The peers’ reading should be restricted to a reasonable amount of effort.

Rankings or ratings with an overall measure should not be published. Instead, the results of every single criterion should be provided. If overall ratings are produced, the weighting procedure has to be made transparent. However, it should be kept in mind that research units have different missions to fulfil, therefore an overall rating might favour some missions over others leading to a structural discrimination of some research units.

Many important issues of our times are global in nature and society has high hopes in a technical solution. The SSH, and specifically the humanities, are therefore not in the focus of the public discourse. Especially the critical questions SSH disciplines are asking are not high on the political agenda. However, complex global issues such as, for example, global warming, migration crisis, ageing or HIV cannot be sufficiently resolved without the knowledge of SSH disciplines. The critical questions challenging the blind technological faith in overcoming such problems are crucial. Not being on the top of the political agenda, however, does not mean to give in to the mainstream neo-positivist notion of a parametrically steered research policy. Nor does it mean that SSH scholars should frown at all requests for accountability. Instead, SSH disciplines should step forward and self-confidently and openly question truisms or blind technological faith and propose alternatives to simple but misleading practices. This paper presents many bottom-up actions of SSH scholars taking research assessment in their own hands. Certainly, these bottom-up procedures will lead to a more adequate assessment of SSH research but they might also help fostering a better valorization of SSH research among policy makers and colleagues from the natural sciences. And eventually, maybe some scientists will find these approaches also fruitful for their own disciplines? At the same time, an adequate evaluation and valorization of SSH research will also help society to better understand what the SSH contribution to solving major societal challenges can be. Therefore, taking the time to encourage bottom-up evaluation initiatives should result in better solving of modern societies’ issues.

Data availability

Data sharing is not applicable to this article as no datasets were analysed or generated.

Additional information

How to cite this article : Ochsner M et al. (2017) The future of research assessment in the humanities: bottom-up assessment procedures. Palgrave Communications . 3:17020 doi: 10.1057/palcomms.2017.20. Footnote 4 Footnote 5 Footnote 6

Because it comes from SSH scholars and is clearly bottom-up in nature, we include, however, the initiative by Oancea and Furlong (2007) that was motivated by, but did not have a visible impact on, the RAE in the United Kingdom.

Despite the inclusion of (some) books in the commercial databases in recent years as well as the rise of networking sites also promoting bibliographic data, the under-coverage of certain disciplines and languages remains while technical challenges arise and issues of transparency persist ( Gorraiz et al., 2013 ; Murray, 2014 ; Zuccala and Cornacchia, 2016 )

Others argue that there is a difference between performance-based funding and research evaluation. The first is used to distribute scarce funds and needs not to be related to quality while the latter is formative in nature and includes an understanding of quality. However, while this might be true from the evaluator’s perspective, it is misleading regarding the effect on the scholars’ behaviour. If scholars are assessed by indicators, they perceive these not only as incentives but also as indications of what is expected from them (as well as what is valued as ‘good’ research) and they will adjust their behaviour accordingly (see for example, Hammarfelt and de Rijcke, 2015 ; Williams and Galleron, 2016 ).

As mentioned in the introduction, we focus on European initiatives for coherence reasons and because of restrictions of space.

See the official website https://dbh.nsd.uib.no/publiseringskanaler/erihplus/

The Evaluation Agency for Research and Higher Education (Agence d'évaluation de la recherche et de l'enseignement supérieur, AERES) was replaced by the High Council for the Evaluation of Research and Higher Education (Haut Conseil de l'évaluation de la recherche et de l'enseignement supérieur, HCERES) on 17 November 2014.

Aagaard K, Bloch C and Schneider JW (2015) Impacts of performance-based research funding systems: The case of the Norwegian Publication Indicator. Research Evaluation ; 24 (2): 106–117.

Google Scholar  

Academics Australia. (2008) Letter to Senator the Honourable Kim Carr, Minister for Innovation, Industry, Science and Research , https://web.archive.org/web/20091221195149/http://www.academics-australia.org/AA/ERA/era.pdf , accessed 8 February 2017.

van den Akker W (2016) Yes we should; research assessment in the humanities. In: Ochsner M, Hug SE and Daniel H-D (eds). Research Assessment in the Humanities. Towards Criteria and Procedures . Springer International Publishing: Cham, Switzerland, pp 23–29.

Andersen H et al. (2009) Editorial journals under threat: A joint response from history of science. Technology and Medicine Editors. Social Studies of Science ; 39 (1): 6–9.

Archambault É, Vignola-Gagne E, Cote G, Lariviere V and Gingras Y (2006) Benchmarking scientific output in the social sciences and humanities: The limits of existing databases. Scientometrics ; 68 (3): 329–342.

van Arensbergen P, van der Weijden I and van den Besselaar P (2014a) Different views on scholarly talent: What are the talents we are looking for in science? Research Evaluation ; 23 (4): 273–284.

van Arensbergen P, van der Weijden I and van den Besselaar P (2014b) The selection of talent as a group process. A literature review on the social dynamics of decision making in grant panels. Research Evaluation ; 23 (4): 298–311.

Arts and Humanities Research Council. (2006) Use of research metrics in the arts and humanities ; Report of the expert group set up jointly by the Arts and Humanities Research Council and the Higher Education Funding Council for England AHRC: Bristol, UK.

Arts and Humanities Research Council. (2009) Leading the World. The economic Impact of UK Arts and Humanities Research . AHRC: Bristol, UK.

Barker K (2007) The UK research assessment exercise: The evolution of a national research evaluation system. Research Evaluation ; 16 (1): 3–12.

Bornmann L (2011) Scientific peer review. Annual Review of Information Science and Technology ; 45 (1): 197–245.

Bunia R (2016) Quotation statistics and culture in literature and in other humanist disciplines. In: Ochsner M, Hug SE and Daniel H-D (eds). Research Assessment in the Humanities. Towards Criteria and Procedures . Springer International Publishing: Cham, Switzerland, pp 133–148.

Burrows R (2012) Living with the h-index? Metric assemblages in the contemporary academy. The Sociological Review ; 60 (2): 355–372.

Butler L (2003) Explaining Australia’s increased share of ISI publications—The effects of a funding formula based on publication counts. Research Policy ; 32 (1): 143–155.

Butler L (2007) Assessing university research: A plea for a balanced approach. Science and Public Policy ; 34 (8): 565–574.

Butler L (2008) Using a balanced approach to bibliometrics: Quantitative performance measures in the Australian research quality framework. Ethics in Science and Environmental Politics ; 8 (1): 83–92.

Butler L and McAllister I (2009) Metrics or peer review? Evaluating the 2001 UK research assessment exercise in political science. Political Studies Review ; 7 (1): 3–17.

Butler L and Visser MS (2006) Extending citation analysis to non-source items. Scientometrics ; 66 (2): 327–343.

CAS   Google Scholar  

Chi P-S (2012) Bibliometric characteristics of political science research in Germany. Proceedings of the American Society for Information Science and Technology ; 49 (1): 1–6.

Chi P-S (2014) Which role do non-source items play in the social sciences? A case study in political science in Germany. Scientometrics ; 101 (2): 1195–1213.

Committee on the National Plan for the Future of the Humanities. (2009) Sustainable Humanities: Report from the National Committee on the Future of the Humanities in the Netherlands . Amsterdam University Press: Amsterdam, The Netherlands.

Commission of the European Communities. (2000) Communication from the Commission to the Council, the European Parliament, the Economic and Social Committee and the Committee of the Regions: Towards a European research area . Commission of the European Communities: Brussels, UK.

Council for the Humanities, Arts and Social Sciences. (2009) Humanities and Creative Arts: Recognising Esteem Factors and Non- traditional Publication in Excellence in Research for Australia (ERA) Initiative ; CHASS Papers. Council for the Humanities Arts and Social Sciences: Canberra, Australia.

CRUS. (2008) The Swiss Way to University Quality . Rectors’ Conference of the Swiss Universities (CRUS): Bern, Switzerland.

Drabek A, Rozkosz EA, Hołowiecki M and Kulczycki E (2015) Polski Współczynnik Wpływu a kultury cytowań w humanistyce. Nauka i Szkolnictwo Wyższe ; 46 (2): 121–138.

European Research Area and Innovation Committee. (2015) European Research Area (ERA) Roadmap 2015–2020 . European Research Area and Innovation Committee: Brussels, UK.

Ferrara A and Bonaccorsi A (2016) How robust is journal rating in Humanities and Social Sciences? Evidence from a large-scale, multi-method exercise. Research Evaluation ; 25 (3): 279–291.

Finkenstaedt T (1990) Measuring research performance in the humanities. Scientometrics ; 19 (5): 409–417.

Fisher D, Rubenson K, Rockwell K, Grosjean G and Atkinson-Grosjean J (2000) Performance Indicators and the Humanities and Social Sciences . Centre for Policy Studies in Higher Education and Training: Vancouver, BC.

Genoni P and Haddow G (2009) ERA and the ranking of australian humanities journals. Australian Humanities Review ; 46 , 5–24.

van Gestel R, Micklitz H-W and Poiares MM (2012) Methodology in the new legal world. EUI Working Papers LAW 2012/13. doi: 10.2139/ssrn.2069872.

Giménez-Toledo E (2016) Assessment of journal & book publishers in the humanities and social sciences in Spain. In: Ochsner M, Hug SE and Daniel H-D (eds). Research Assessment in the Humanities. Towards Criteria and Procedures . Springer International Publishing: Cham, Switzerland, pp 91–102.

Giménez-Toledo E et al. (2016) Taking scholarly books into account: Current developments in five European countries. Scientometrics ; 107 (2): 1–15.

Giménez-Toledo E, Tejada-Artigas C and Mañana-Rodríguez J (2013) Evaluation of scientific books’ publishers in social sciences and humanities: Results of a survey. Research Evaluation ; 22 (1): 64–77.

Glänzel W (1996) A bibliometric approach to social sciences. National research performances in 6 selected social science areas, 1990–1992. Scientometrics ; 35 (3): 291–307.

Glänzel W, Thijs B and Debackere K (2016) Productivity, performance, efficiency, impact—What do we measure anyway? Journal of Informetrics ; 10 (2): 658–660.

Gogolin I (2016) European educational research quality indicators (EERQI): An experiment. In: Ochsner M, Hug SE and Daniel H-D (eds). Research Assessment in the Humanities. Towards Criteria and Procedures . Springer International Publishing: Cham, Switzerland, pp 103–111.

Gogolin I, Astrom F and Hansen A (eds) (2014) Assessing Quality in European Educational Research . Springer VS: Wiesbaden, Germany.

Gogolin I, Stumm V (2014) The EERQI peer review questionnaire. In: Gogolin I, Astrom F and Hansen A (eds). Assessing Quality in European Educational Research . Springer VS: Wiesbaden, Germany, pp 107–120.

Gorraiz J, Purnell PJ and Glänzel W (2013) Opportunities for and limitations of the Book Citation Index. Journal of The American Society For Information Science and Technology ; 64 (7): 1388–1398.

Guetzkow J, Lamont M and Mallard G (2004) What Is originality in the humanities and the social sciences? American Sociological Review ; 69 (2): 190–212.

Guillory J (2005) Valuing the humanities, evaluating scholarship. Profession (MLA) ; 11 , 28–38.

Gumpenberger C, Glänzel W and Gorraiz J (2016) The ecstasy and the agony of the altmetric score. Scientometrics ; 108 (2): 977–982.

Hamann J (2016) The visible hand of research performance assessment. Higher Education ; 72 (6): 761–779.

Hammarfelt B (2012) Following the footnotes: A bibliometric analysis of citation patterns in literary studies . Doctoral dissertation. Skrifter utgivna vid institutionen för ABM vid Uppsala Universitet (Vol. 5). Uppsala Universitet: Uppsala, http://www.diva-portal.org/smash/get/diva2:511996/FULLTEXT01.pdf .

Hammarfelt B (2013) Harvesting footnotes in a rural field: Citation patterns in Swedish literary studies. Journal of Documentation ; 68 (4): 536–558.

Hammarfelt B (2016) Beyond coverage: Toward a bibliometrics for the humanities. In: Ochsner M, Hug SE and Daniel H-D (eds). Research Assessment in the Humanities. Towards Criteria and Procedures . Springer International Publishing: Cham, Switzerland, pp 115–131.

Hammarfelt B and de Rijcke S (2015) Accountability in context: Effects of research evaluation systems on publication practices, disciplinary norms, and individual working routines in the faculty of Arts at Uppsala University. Research Evaluation ; 24 (1): 63–77.

Hellqvist B (2010) Referencing in the humanities and its implications for citation analysis. Journal of The American Society For Information Science And Technology ; 61 (2): 310–318.

Hemlin S (1993) Scientific quality in the eyes of the scientist. A questionnaire study. Scientometrics ; 27 (1): 3–18.

Hemlin S (1996) Social studies of the humanities. A case study of research conditions and performance in ancient history and classical archaeology and English. Research Evaluation ; 6 (1): 53–61.

Hemlin S and Gustafsson M (1996) Research production in the arts and humanities. A questionnaire study of factors influencing research performance. Scientometrics ; 37 (3): 417–432.

Herbert U and Kaube J (2008) Die Mühen der Ebene: Über Standards, Leistung und Hochschulreform. In: Lack E and Markschies C (eds). What the hell is quality? Qualitätsstandards in den Geisteswissenschaften . Campus-Verlag: Frankfurt, Germany, pp 37–51.

Hicks D (2004) The four literatures of social science. In: Moed H, Glänzel W and Schmoch U (eds). Handbook of Quantitative Science and Technology Research . Kluwer Academic Publishers: New York, pp 473–496.

Hicks D (2012) Performance-based university research funding systems. Research Policy ; 41 (2): 251–261.

Hoffmann CP, Lutz C and Meckel M (2015) A relational altmetric? Network centrality on ResearchGate as an indicator of scientific impact. Journal of the Association for Information Science and Technology ; 67 (4): 765–775.

Holmberg K and Thelwall M (2014) Disciplinary differences in Twitter scholarly communication. Scientometrics ; 101 (2): 1027–1042.

Hornung A, Khlavna V and Korte B (2016) Research Rating Anglistik/Amerikanistik of the German Council of Science and Humanities. In: Ochsner M, Hug SE and Daniel H-D (eds). Research Assessment in the Humanities. Towards Criteria and Procedures . Springer International Publishing: Cham, Switzerland, pp 219–233.

Hose M (2009) Qualitätsmessung: Glanz und Elend der Zahl. In: Prinz C and Hohls R (eds). Qualitätsmessung, Evaluation, Forschungsrating. Risiken und Chancen für die Geschichtswissenschaft? . Historisches Forum. Clio-online: Berlin, Germany, pp 91–98.

Hug SE and Ochsner M (2014) A framework to explore and develop criteria for assessing research quality in the humanities. International Journal of Education Law and Policy ; 10 (1): 55–68.

Hug SE, Ochsner M and Daniel H-D (2013) Criteria for assessing research quality in the humanities: A Delphi study among scholars of English literature, German literature and art history. Research Evaluation ; 22 (5): 369–383.

Iseli M (2016) Qualitäts- und Leistungsbeurteilung in den Geistes- und Sozialwissenschaften: Prinzipien, Ansätze und Verfahren . SAGW: Bern, Switzerland.

Johnston R (2008) On structuring subjective judgements: Originality, significance and rigour in RAE 2008. Higher Education Quarterly ; 62 (1/2): 120–147.

Kekäle J (2002) Conceptions of quality in four different disciplines. Tertiary Education and Management ; 8 (1): 65–80.

König T (2016) Peer review in the social sciences and humanities at the European Level: The experiences of the European research council. In: Ochsner M, Hug SE and Daniel H-D (eds). Research Assessment in the Humanities. Towards Criteria and Procedures . Springer International Publishing: Cham, Switzerland, pp 151–163.

Kousha K and Thelwall M (2009) Google book search: Citation analysis for social science and the humanities. Journal of The American Society For Information Science And Technology ; 60 (8): 1537–1549.

Krull W and Tepperwien A (2016) The four ‘I’s: Quality indicators for the humanities. In: Ochsner M, Hug SE and Daniel H-D (eds). Research Assessment in the Humanities. Towards Criteria and Procedures . Springer International Publishing: Cham, Switzerland, pp 165–179.

Kwok JT (2013) Impact of ERA Research Assessment on University Behaviour and their Staff . National Tertiary Education Union: South Melbourne, Australia.

Lack E (2008) Einleitung—Das Zauberwort “Standards”. In: Lack E and Markschies C (eds). What the hell is quality? Qualitätsstandards in den Geisteswissenschaften . Campus-Verlag: Frankfurt, Germany, pp 9–34.

Lamont M (2009) How Professors Think: Inside the Curious World of Academic Judgment . Harvard University Press: Harvard, UK.

Lariviere V, Gingras Y and Archambault É (2006) Canadian collaboration networks: A comparative analysis of the natural sciences, social sciences and the humanities. Scientometrics ; 68 (3): 519–533.

Lauer G (2016) The ESF scoping project “towards a bibliometric database for the social sciences and humanities”. In: Ochsner M, Hug SE and Daniel H-D (eds). Research Assessment in the Humanities. Towards Criteria and Procedures . Springer International Publishing: Cham, Switzerland, pp 73–77.

Lawrence PA (2002) Rank injustice. Nature ; 415 (6874): 835–836.

van Leeuwen TN (2013) Bibliometric research evaluations, web of science and the social sciences and Humanities: A problematic relationship? Bibliometrie—Praxis und Forschung ; 2 , 8.

Lienhard A, Tanquerel T, Flückiger A, Amschwand F, Byland K and Herrmann E (2016) Forschungsevaluation in der Rechtswissenschaft: Grundlagen und empirische Analyse in der Schweiz . Stämpfli Verlag: Bern, Switzerland.

Loprieno A, Werlen R, Hasgall A and Bregy J (2016) The “Mesurer les Performances de la Recherche” Project of the Rectors’ Conference of the Swiss Universities (CRUS) and Its Further Development. In: Ochsner M, Hug SE and Daniel H-D (eds). Research Assessment in the Humanities. Towards Criteria and Procedures . Springer International Publishing: Cham, Switzerland, pp 13–21.

Luckman S (2004) More than the sum of its parts: The humanities and communicating the ‘hidden work’ of research. In: Kenway J, Bullen E and Robb S (eds). Innovation and Tradition: The Arts, Humanities, and the Knowledge Economy . Peter Lang: New York, pp 82–90.

MacDonald SP (1994) Professional Academic Writing in the Humanities and Social Sciences . Southern Illinois University Press: Carbondale, Edwardsville, IL.

Mair C (2016) Rating research performance in the humanities: An interim report on an initiative of the German Wissenschaftsrat. In: Ochsner M, Hug SE and Daniel H-D (eds). Research Assessment in the Humanities. Towards Criteria and Procedures . Springer International Publishing: Cham, Switzerland, pp 201–209.

Martin BR et al. (2010) Towards a bibliometric database for the social sciences and humanities. A European scoping project (A report produced for DFG, ESRC, AHRC, NWO, ANR and ESF). Science and Technology Policy Research Unit: Sussex.

McCarthy KF, Ondaatje EH, Zakaras L and Brooks A (2004) Gifts of the Muse. Refraiming the Debate About the Benefits of the Arts . RAND Corporation: Santa Monica, CA.

Mohammadi E and Thelwall M (2014) Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows. Journal of the Association for Information Science and Technology ; 65 (8): 1627–1638.

Mojon-Azzi SM, Jiang X, Wagner U and Mojon DS (2003) Journals: Redundant publications are bad news. Nature ; 421 (6920): 209.

Molinié A and Bodenhausen G (2010) Bibliometrics as weapons of mass citation. CHIMIA International Journal for Chemistry ; 64 (1): 78–89.

Moonesinghe R, Khoury MJ and Janssens A C J W (2007) Most published research findings are false—But a little replication goes a long way. PLoS Medicine ; 4 (2): e28.

Murray M (2014) Analysis of a scholarly social networking site: The case of the dormant user. SAIS 2014 Proceedings . Paper 24, http://aisel.aisnet.org/sais2014 , accessed 8 February 2017.

Nederhof AJ (2006) Bibliometric monitoring of research performance in the social sciences and the humanities: A review. Scientometrics ; 66 (1): 81–100.

Nederhof AJ, Zwaan R, de Bruin R and Dekker P (1989) Assessing the usefulness of bibliometric indicators for the humanities and the social sciences—a comparative study. Scientometrics ; 15 (5): 423–435.

Norris M and Oppenheim C (2003) Citation counts and the research assessment exercise V—Archaeology and the 2001 RAE. Journal of Documentation ; 59 (6): 709–730.

Nussbaum MC (2010) Not for Profit: Why Democracy Needs the Humanities . Princeton University Press: Princeton, NJ.

Oancea A and Furlong J (2007) Expressions of excellence and the assessment of applied and practice-based research. Research Papers in Education ; 22 (2): 119–137.

Ochsner M, Hug SE and Daniel H-D (2012) Indicators for research quality in the humanities: Opportunities and limitations. Bibliometrie—Praxis und Forschung ; 1 , 4.

Ochsner M, Hug SE and Daniel H-D (2013) Four types of research in the humanities: Setting the stage for research quality criteria in the humanities. Research Evaluation ; 22 (2): 79–92.

Ochsner M, Hug SE and Daniel H-D (2014) Setting the stage for the assessment of research quality in the humanities. Consolidating the results of four empirical studies. Zeitschrift für Erziehungswissenschaft ; 17 (6): 111–132.

Ochsner M, Hug SE, Daniel H-D (eds) (2016) Humanities scholars’ conceptions of research quality. In: Research Assessment in the Humanities. Towards Criteria and Procedures . Springer International Publishing: Cham, Switzerland, pp 43–69.

Ochsner M, Wolbring T and Hug SE (2015) Quality criteria for sociology? What sociologists can learn from the project “developing and testing research quality criteria in the humanities”. Sociologia E Politiche Sociali ; 18 (2): 90–110.

Oppenheim C and Summers M (2008) Citation counts and the research assessment exercise, part VI: Unit of assessment 67 (music). Information Research ; 13 (2).

Ossenblok T and Engels T (2015) Edited books in the social sciences and humanities: Characteristics and collaboration analysis. Scientometrics ; 104 (1): 219–237.

Ossenblok TLB, Engels TCE and Sivertsen G (2012) The representation of the social sciences and humanities in the web of science: A comparison of publication patterns and incentive structures in Flanders and Norway (2005–9). Research Evaluation ; 21 (4): 280–290.

Palumbo M and Pennisi C (2015) Criteri corretti e condivisi per una valutazione buona e utile della ricerca [Correct and shared criteria for a good and useful evaluation of research]. Sociologia E Politiche Sociali ; 18 (2): 73–89.

Perret JF, Sormani P, Bovet A and Kohler A (2011) Décrire et mesurer la “fécondité” des recherches en sciences humanies et sociales. Bulletin SAGW ; 2011 (2): 40–42.

Plag I (2016) Research assessment in a philological discipline: Criteria and rater reliability. In: Ochsner M, Hug SE and Daniel H-D (eds). Research Assessment in the Humanities. Towards Criteria and Procedures . Springer International Publishing: Cham, Switzerland, pp 235–247.

Plumpe W (2009) Qualitätsmessung: Stellungnahme zum Rating des Wissenschaftsrates aus Sicht des Historikerverbandes. In: Prinz C and Hohls R (eds). Qualitätsmessung, Evaluation, Forschungsrating. Risiken und Chancen für die Geschichtswissenschaft? . Historisches Forum. Clio-online: Berlin, Germany, pp 121–126.

Plumpe W (2010) Der Teufel der Unvergleichbarkeit. Über das quantitative Messen und Bewerten von Forschung. Forschung und Lehre ; 17 (8): 572–574.

Probst C, Lepori B, de Filippo D and Ingenhoff D (2011) Profiles and beyond: Constructing consensus on measuring research output in communication sciences. Research Evaluation ; 20 (1): 73–88.

Redden G (2008) From RAE to ERA: Research evaluation at work in the corporate university. Australian Humanities Review ; 45 , 7–26.

Riordan P, Ganser C and Wolbring T (2011) Measuring the quality of research. Kölner Zeitschrift für Soziologie und Sozialpsychologie ; 63 (1): 147–172.

Royal Netherlands Academy of Arts and Sciences. (2005) Judging research on its Merits . An advisory report by the Council for the Humanities and the Social Sciences Council. Royal Netherlands Academy of Arts and Sciences: Amsterdam.

Royal Netherlands Academy of Arts and Sciences. (2011) Quality Indicators for Research in the Humanities . Royal Netherlands Academy of Arts and Sciences: Amsterdam, The Netherlands.

SAGW. (2012a) Für eine Erneuerung der Geisteswissenschaften. Empfehlungen der SAGW zuhanden der Leitungsorgane der Hochschulen, der Lehrenden, der Förderorganisationen und des Staatssekretariats für Bildung und Forschung . SAGW: Bern, Switzerland.

SAGW. (2012b) Für eine neue Kultur der Geisteswissenschaften? Akten des Kongresses vom 30. November bis 2. Dezember 2011, Bern. SAGW: Bern, Switzerland.

Sandor A and Vorndran A (2014a) Enhancing relevance ranking of the EERQI search engine. In: Gogolin I, Astrom F and Hansen A (eds). Assessing Quality in European Educational Research . Springer VS: Wiesbaden, Germany, pp 56–59.

Sandor A and Vorndran A (2014b) Highlighting salient sentences for reading assistance. In: Gogolin I, Astrom F and Hansen A (eds). Assessing Quality in European Educational Research . Springer VS: Wiesbaden, Germany, pp 43–55.

Schneider JW (2009) An outline of the bibliometric indicator used for performance-based funding of research institutions in Norway. European Political Science ; 8 (3): 364–378.

Sivertsen G (2016) Publication-based funding: The Norwegian model. In: Ochsner M, Hug SE and Daniel H-D (eds). Research Assessment in the Humanities. Towards Criteria and Procedures . Springer International Publishing: Cham, Switzerland, pp 79–90.

Spaapen J, Dijstelbloem H and Wamelink F (2007) Evaluating Research in Context: A Method for Comprehensive Assessment , Second edition, Consultative Committee of Sector Councils for Research and Development: The Hague, The Netherlands.

Stierstorfer K and Schneck P (2016) “21 Grams”: Interdisciplinarity and the assessment of quality in the humanities. In: Ochsner M, Hug SE and Daniel H-D (eds). Research Assessment in the Humanities. Towards Criteria and Procedures . Springer International Publishing: Cham, Switzerland, pp 211–218.

Thorngate W, Dawes RM and Foddy M (2009) Judging merit . Pychology Press Taylor & Francis Group: New York, Hove, UK.

Unreliable research. Trouble at the lab. (2013) The Economist . 19 October, http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble .

Vec M (2009) Qualitätsmessung: Die vergessene Freiheit. Steuerung und Kontrolle der Geisteswissenschaften unter der Prämisse der Prävention. In: Prinz C and Hohls R (eds). Qualitätsmessung, Evaluation, Forschungsrating. Risiken und Chancen für die Geschichtswissenschaft? . Historisches Forum. Clio-online: Berlin, Germany, pp 79–90.

Verleysen FT and Weeren A (2016) Clustering by publication patterns of senior authors in the social sciences and humanities. Journal of Informetrics ; 10 (1): 254–272.

VolkswagenStiftung. (2014) What is Intellectual Quality in the Humanities? Some Guidelines . VolkswagenStiftung: Hannover, Germany.

Weingart P, Prinz W, Kastner M, Maasen S and Walter W (1991) Die sogenannten Geisteswissenschaften: Aussenansichten: Die Entwicklung der Geisteswissenschaften in der BRD, 1954–1987 . Suhrkamp: Frankfurt am Main, Germany.

White HD, Boell SK, Yu H, Davis M, Wilson CS and Cole FTH (2009) Libcitations: A measure for comparative assessment of book publications in the humanities and social sciences. Journal of The American Society For Information Science and Technology ; 60 (6): 1083–1096.

Williams G and Galleron I (2016) Bottom Up from the bottom: A new outlook on research evaluation for the SSH in France. In: Ochsner M, Hug SE and Daniel H-D (eds). Research Assessment in the Humanities. Towards Criteria and Procedures . Springer International Publishing: Cham, Switzerland, pp 181–198.

Wissenschaftsrat. (2004) Recommendations for rankings in the system of higher education and research. Part 1: Research . Wissenschaftsrat: Hamburg.

Wissenschaftsrat. (2010) Empfehlungen zur vergleichenden Forschungsbewertung in den Geisteswissenschaften . Wissenschaftsrat: Köln, Germany.

Zuccala AA (2016) Inciting the metric oriented humanist: Teaching bibliometrics in a faculty of humanities. Education for Information ; 32 (2): 149–164.

Zuccala AA, Verleysen FT, Cornacchia R and Engels TCE (2015) Altmetrics for the humanities. Aslib Journal of Information Management ; 67 (3): 320–336.

Zuccala AA and Cornacchia R (2016) Data matching, integration, and interoperability for a metric assessment of monographs. Scientometrics ; 108 (1): 465–484.

Zuccala AA and van Leeuwen T (2011) Book reviews in humanities research evaluations. Journal of The American Society For Information Science and Technology ; 62 (10): 1979–1991.

Download references

Acknowledgements

This article is based upon work from COST Action CA 15137 ‘European Network for Research Evaluation in the SSH (ENRESSH)’, supported by COST (European Cooperation in Science and Technology). Michael Ochsner and Sven E. Hug would like to thank swissuniversities for their grant for the project “Application of Bottom-up Criteria in the Assessment of Grant Proposals of Junior Researchers” within the “Programme P-3 Performances de la recherche en sciences humaines et socials”. Matching funds were provided by the University of Zurich.

Author information

Authors and affiliations.

ETH Zürich, Zürich, Switzerland

Michael Ochsner & Sven Hug

FORS, Lausanne, Switzerland

Michael Ochsner

University of Zürich, Zürich, Switzerland

University Grenobles Alpes, Grenoble, France

Ioana Galleron

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Michael Ochsner .

Ethics declarations

Competing interests.

The authors declare that they have no competing financial interests.

Rights and permissions

This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Reprints and permissions

About this article

Cite this article.

Ochsner, M., Hug, S. & Galleron, I. The future of research assessment in the humanities: bottom-up assessment procedures. Palgrave Commun 3 , 17020 (2017). https://doi.org/10.1057/palcomms.2017.20

Download citation

Received : 22 August 2016

Accepted : 27 February 2017

Published : 21 March 2017

DOI : https://doi.org/10.1057/palcomms.2017.20

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Philosophers’ appraisals of bibliometric indicators and their use in evaluation: from recognition to knee-jerk rejection.

• Ramón A. Feenstra
• Emilio Delgado López-Cózar

Scientometrics (2022)

Toward a classification of Spanish scholarly journals in social sciences and humanities considering their impact and visibility

• Daniela De Filippo
• Rafael Aleixandre-Benavent
• Elías Sanz-Casado

Scientometrics (2020)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies