presents information gathered through the research

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Data Collection | Definition, Methods & Examples

Data Collection | Definition, Methods & Examples

Published on June 5, 2020 by Pritha Bhandari . Revised on June 21, 2023.

Data collection is a systematic process of gathering observations or measurements. Whether you are performing research for business, governmental or academic purposes, data collection allows you to gain first-hand knowledge and original insights into your research problem .

While methods and aims may differ between fields, the overall process of data collection remains largely the same. Before you begin collecting data, you need to consider:

• The  aim of the research
• The type of data that you will collect
• The methods and procedures you will use to collect, store, and process the data

To collect high-quality data that is relevant to your purposes, follow these four steps.

Table of contents

Step 1: define the aim of your research, step 2: choose your data collection method, step 3: plan your data collection procedures, step 4: collect the data, other interesting articles, frequently asked questions about data collection.

Before you start the process of data collection, you need to identify exactly what you want to achieve. You can start by writing a problem statement : what is the practical or scientific issue that you want to address and why does it matter?

Next, formulate one or more research questions that precisely define what you want to find out. Depending on your research questions, you might need to collect quantitative or qualitative data :

• Quantitative data is expressed in numbers and graphs and is analyzed through statistical methods .
• Qualitative data is expressed in words and analyzed through interpretations and categorizations.

If your aim is to test a hypothesis , measure something precisely, or gain large-scale statistical insights, collect quantitative data. If your aim is to explore ideas, understand experiences, or gain detailed insights into a specific context, collect qualitative data. If you have several aims, you can use a mixed methods approach that collects both types of data.

• Your first aim is to assess whether there are significant differences in perceptions of managers across different departments and office locations.
• Your second aim is to gather meaningful feedback from employees to explore new ideas for how managers can improve.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Based on the data you want to collect, decide which method is best suited for your research.

• Experimental research is primarily a quantitative method.
• Interviews , focus groups , and ethnographies are qualitative methods.
• Surveys , observations, archival research and secondary data collection can be quantitative or qualitative methods.

Carefully consider what method you will use to gather data that helps you directly answer your research questions.

When you know which method(s) you are using, you need to plan exactly how you will implement them. What procedures will you follow to make accurate observations or measurements of the variables you are interested in?

For instance, if you’re conducting surveys or interviews, decide what form the questions will take; if you’re conducting an experiment, make decisions about your experimental design (e.g., determine inclusion and exclusion criteria ).

Operationalization

Sometimes your variables can be measured directly: for example, you can collect data on the average age of employees simply by asking for dates of birth. However, often you’ll be interested in collecting data on more abstract concepts or variables that can’t be directly observed.

Operationalization means turning abstract conceptual ideas into measurable observations. When planning how you will collect data, you need to translate the conceptual definition of what you want to study into the operational definition of what you will actually measure.

• You ask managers to rate their own leadership skills on 5-point scales assessing the ability to delegate, decisiveness and dependability.
• You ask their direct employees to provide anonymous feedback on the managers regarding the same topics.

You may need to develop a sampling plan to obtain data systematically. This involves defining a population , the group you want to draw conclusions about, and a sample, the group you will actually collect data from.

Your sampling method will determine how you recruit participants or obtain measurements for your study. To decide on a sampling method you will need to consider factors like the required sample size, accessibility of the sample, and timeframe of the data collection.

Standardizing procedures

If multiple researchers are involved, write a detailed manual to standardize data collection procedures in your study.

This means laying out specific step-by-step instructions so that everyone in your research team collects data in a consistent way – for example, by conducting experiments under the same conditions and using objective criteria to record and categorize observations. This helps you avoid common research biases like omitted variable bias or information bias .

This helps ensure the reliability of your data, and you can also use it to replicate the study in the future.

Creating a data management plan

Before beginning data collection, you should also decide how you will organize and store your data.

• If you are collecting data from people, you will likely need to anonymize and safeguard the data to prevent leaks of sensitive information (e.g. names or identity numbers).
• If you are collecting data via interviews or pencil-and-paper formats, you will need to perform transcriptions or data entry in systematic ways to minimize distortion.
• You can prevent loss of data by having an organization system that is routinely backed up.

Finally, you can implement your chosen methods to measure or observe the variables you are interested in.

The closed-ended questions ask participants to rate their manager’s leadership skills on scales from 1–5. The data produced is numerical and can be statistically analyzed for averages and patterns.

To ensure that high quality data is recorded in a systematic way, here are some best practices:

• Record all relevant information as and when you obtain data. For example, note down whether or how lab equipment is recalibrated during an experimental study.
• Double-check manual data entry for errors.
• If you collect quantitative data, you can assess the reliability and validity to get an indication of your data quality.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Likert scale

Research bias

• Implicit bias
• Framing effect
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

When conducting research, collecting original data has significant advantages:

• You can tailor data collection to your specific research aims (e.g. understanding the needs of your consumers or user testing your website)
• You can control and standardize the process for high reliability and validity (e.g. choosing appropriate measurements and sampling methods )

However, there are also some drawbacks: data collection can be time-consuming, labor-intensive and expensive. In some cases, it’s more efficient to use secondary data that has already been collected by someone else, but the data might be less reliable.

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

Reliability and validity are both about how well a method measures something:

• Reliability refers to the  consistency of a measure (whether the results can be reproduced under the same conditions).
• Validity   refers to the  accuracy of a measure (whether the results really do represent what they are supposed to measure).

If you are doing experimental research, you also have to consider the internal and external validity of your experiment.

Operationalization means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioral avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalize the variables that you want to measure.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 21). Data Collection | Definition, Methods & Examples. Scribbr. Retrieved April 8, 2024, from https://www.scribbr.com/methodology/data-collection/

Is this article helpful?

Pritha Bhandari

Other students also liked, qualitative vs. quantitative research | differences, examples & methods, sampling methods | types, techniques & examples, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• AI Templates
• Get a demo Sign up for free Log in Log in

Buttoning up research: How to present and visualize qualitative data

15 Minute Read

There is no doubt that data visualization is an important part of the qualitative research process. Whether you're preparing a presentation or writing up a report, effective visualizations can help make your findings clear and understandable for your audience. 

In this blog post, we'll discuss some tips for creating effective visualizations of qualitative data. 

First, let's take a closer look at what exactly qualitative data is.

What is qualitative data?

Qualitative data is information gathered through observation, questionnaires, and interviews. It's often subjective, meaning that the researcher has to interpret it to draw meaningful conclusions from it. 

The difference between qualitative data and quantitative data

When researchers use the terms qualitative and quantitative, they're referring to two different types of data. Qualitative data is subjective and descriptive, while quantitative data is objective and numerical.

Qualitative data is often used in research involving psychology or sociology. This is usually where a researcher may be trying to identify patterns or concepts related to people's behavior or attitudes. It may also be used in research involving economics or finance, where the focus is on numerical values such as price points or profit margins. 

Before we delve into how best to present and visualize qualitative data, it's important that we highlight how to be gathering this data in the first place. ‍

How best to gather qualitative data

In order to create an effective visualization of qualitative data, ensure that the right kind of information has been gathered. 

Here are six ways to gather the most accurate qualitative data:

• Define your research question: What data is being set out to collect? A qualitative research question is a definite or clear statement about a condition to be improved, a project’s area of concern, a troubling question that exists, or a difficulty to be eliminated. It not only defines who the participants will be but guides the data collection methods needed to achieve the most detailed responses.
• ‍ Determine the best data collection method(s): The data collected should be appropriate to answer the research question. Some common qualitative data collection methods include interviews, focus groups, observations, or document analysis. Consider the strengths and weaknesses of each option before deciding which one is best suited to answer the research question.  ‍
• Develop a cohesive interview guide: Creating an interview guide allows researchers to ask more specific questions and encourages thoughtful responses from participants. It’s important to design questions in such a way that they are centered around the topic of discussion and elicit meaningful insight into the issue at hand. Avoid leading or biased questions that could influence participants’ answers, and be aware of cultural nuances that may affect their answers.
• ‍ Stay neutral – let participants share their stories: The goal is to obtain useful information, not to influence the participant’s answer. Allowing participants to express themselves freely will help to gather more honest and detailed responses. It’s important to maintain a neutral tone throughout interviews and avoid judgment or opinions while they are sharing their story. 
• ‍ Work with at least one additional team member when conducting qualitative research: Participants should always feel comfortable while providing feedback on a topic, so it can be helpful to have an extra team member present during the interview process – particularly if this person is familiar with the topic being discussed. This will ensure that the atmosphere of the interview remains respectful and encourages participants to speak openly and honestly.
• ‍ Analyze your findings: Once all of the data has been collected, it’s important to analyze it in order to draw meaningful conclusions. Use tools such as qualitative coding or content analysis to identify patterns or themes in the data, then compare them with prior research or other data sources. This will help to draw more accurate and useful insights from the results. 

By following these steps, you will be well-prepared to collect and analyze qualitative data for your research project. Next, let's focus on how best to present the qualitative data that you have gathered and analyzed.

Create your own AI-powered templates for better, faster research synthesis. Discover new customer insights from data instantly.

The top 10 things Notably shipped in 2023 and themes for 2024.

How to visually present qualitative data.

When it comes to how to present qualitative data visually, the goal is to make research findings clear and easy to understand. To do this, use visuals that are both attractive and informative. 

Presenting qualitative data visually helps to bring the user’s attention to specific items and draw them into a more in-depth analysis. Visuals provide an efficient way to communicate complex information, making it easier for the audience to comprehend. 

Additionally, visuals can help engage an audience by making a presentation more interesting and interactive.

Here are some tips for creating effective visuals from qualitative data:

• ‍ Choose the right type of visualization: Consider which type of visual would best convey the story that is being told through the research. For example, bar charts or line graphs might be appropriate for tracking changes over time, while pie charts or word clouds could help show patterns in categorical data. 
• ‍ Include contextual information: In addition to showing the actual numbers, it's helpful to include any relevant contextual information in order to provide context for the audience. This can include details such as the sample size, any anomalies that occurred during data collection, or other environmental factors.
• ‍ Make it easy to understand: Always keep visuals simple and avoid adding too much detail or complexity. This will help ensure that viewers can quickly grasp the main points without getting overwhelmed by all of the information. 
• ‍ Use color strategically: Color can be used to draw attention to certain elements in your visual and make it easier for viewers to find the most important parts of it. Just be sure not to use too many different colors, as this could create confusion instead of clarity. 
• ‍ Use charts or whiteboards: Using charts or whiteboards can help to explain the data in more detail and get viewers engaged in a discussion. This type of visual tool can also be used to create storyboards that illustrate the data over time, helping to bring your research to life. 

Visualizing qualitative data in Notably

Notably helps researchers visualize their data on a flexible canvas, charts, and evidence based insights. As an all-in-one research platform, Notably enables researchers to collect, analyze and present qualitative data effectively.

Notably provides an intuitive interface for analyzing data from a variety of sources, including interviews, surveys, desk research, and more. Its powerful analytics engine then helps you to quickly identify insights and trends in your data . Finally, the platform makes it easy to create beautiful visuals that will help to communicate research findings with confidence. 

Research Frameworks in Analysis

The canvas in Analysis is a multi-dimensional workspace to play with your data spatially to find likeness and tension. Here, you may use a grounded theory approach to drag and drop notes into themes or patterns that emerge in your research. Utilizing the canvas tools such as shapes, lines, and images, allows researchers to build out frameworks such as journey maps, empathy maps, 2x2's, etc. to help synthesize their data.

Going one step further, you may begin to apply various lenses to this data driven canvas. For example, recoloring by sentiment shows where pain points may distributed across your customer journey. Or, recoloring by participant may reveal if one of your participants may be creating a bias towards a particular theme.

Exploring Qualitative Data through a Quantitative Lens

Once you have begun your analysis, you may visualize your qualitative data in a quantitative way through charts. You may choose between a pie chart and or a stacked bar chart to visualize your data. From here, you can segment your data to break down the ‘bar’ in your bar chart and slices in your pie chart one step further.

To segment your data, you can choose between ‘Tag group’, ‘Tag’, ‘Theme’, and ‘Participant'. Each group shows up as its own bar in the bar chart or slice in the pie chart. For example, try grouping data as ‘Participant’ to see the volume of notes assigned to each person. Or, group by ‘Tag group’ to see which of your tag groups have the most notes.

Depending on how you’ve grouped or segmented your charts will affect the options available to color your chart. Charts use colors that are a mix of sentiment, tag, theme, and default colors. Consider color as a way of assigning another layer of meaning to your data. For example, choose a red color for tags or themes that are areas of friction or pain points. Use blue for tags that represent opportunities.

AI Powered Insights and Cover Images

One of the most powerful features in Analysis is the ability to generate insights with AI. Insights combine information, inspiration, and intuition to help bridge the gap between knowledge and wisdom. Even before you have any tags or themes, you may generate an AI Insight from your entire data set. You'll be able to choose one of our AI Insight templates that are inspired by trusted design thinking frameworks to stimulate generative, and divergent thinking. With just the click of a button, you'll get an insight that captures the essence and story of your research. You may experiment with a combination of tags, themes, and different templates or, create your own custom AI template. These insights are all evidence-based, and are centered on the needs of real people. You may package these insights up to present your research by embedding videos, quotes and using AI to generate unique cover image.

You can sign up to run an end to end research project for free and receive tips on how to make the most out of your data. Want to chat about how Notably can help your team do better, faster research? Book some time here for a 1:1 demo with your whole team.

Introducing Notably + Miro Integration: 3 Tips to Analyze Miro Boards with AI in Notably

5 Steps to turn data into insights with Notably

Give your research synthesis superpowers..

Try Teams for 7 days

Free for 1 project

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

4 Gathering and Analyzing Qualitative Data

Gathering and analyzing qualitative data.

As the role of clinician researchers expands beyond the bedside, it is important to consider the possibilities of inquiry beyond the quantitative approach. In contrast to the quantitative approach, qualitative methodology is highly inductive and relies on the background and interpretation of the researcher to derive meaning from the gathering and analytic processes central to qualitative inquiry.

Chapter 4: Learning Objectives

As you explore the research opportunities central to your interests to consider whether qualitative component would enrich your work, you’ll be able to:

• Define what qualitative research is
• Compare qualitative and quantitative approaches
• Describe the process of creating themes from recurring ideas gleaned from narrative interviews

What Is Qualitative Research?

Quantitative researchers typically start with a focused research question or hypothesis, collect a small amount of numerical data from a large number of individuals, describe the resulting data using statistical techniques, and draw general conclusions about some large population. Although this method is by far the most common approach to conducting empirical research in fields such as respiratory care and other clinical fields, there is an important alternative called qualitative research. Qualitative research originated in the disciplines of anthropology and sociology but is now used to study psychological topics as well. Qualitative researchers generally begin with a less focused research question, collect large amounts of relatively “unfiltered” data from a relatively small number of individuals, and describe their data using nonstatistical techniques, such as grounded theory, thematic analysis, critical discourse analysis, or interpretative phenomenological analysis. They are usually less concerned with drawing general conclusions about human behavior than with understanding in detail the experience of their research participants.

Consider, for example, a study by researcher Per Lindqvist and his colleagues, who wanted to learn how the families of teenage suicide victims cope with their loss (Lindqvist, Johansson, & Karlsson, 2008). They did not have a specific research question or hypothesis, such as, What percentage of family members join suicide support groups? Instead, they wanted to understand the variety of reactions that families had, with a focus on what it is like from their perspectives. To address this question, they interviewed the families of 10 teenage suicide victims in their homes in rural Sweden. The interviews were relatively unstructured, beginning with a general request for the families to talk about the victim and ending with an invitation to talk about anything else that they wanted to tell the interviewer. One of the most important themes that emerged from these interviews was that even as life returned to “normal,” the families continued to struggle with the question of why their loved one committed suicide. This struggle appeared to be especially difficult for families in which the suicide was most unexpected.

The Purpose of Qualitative Research

The strength of quantitative research is its ability to provide precise answers to specific research questions and to draw general conclusions about human behavior. This method is how we know that people have a strong tendency to obey authority figures, for example, and that female undergraduate students are not substantially more talkative than male undergraduate students. But while quantitative research is good at providing precise answers to specific research questions, it is not nearly as good at generating novel and interesting research questions. Likewise, while quantitative research is good at drawing general conclusions about human behavior, it is not nearly as good at providing detailed descriptions of the behavior of particular groups in particular situations. And quantitative research is not very good at communicating what it is actually like to be a member of a particular group in a particular situation.

But the relative weaknesses of quantitative research are the relative strengths of qualitative research. Qualitative research can help researchers to generate new and interesting research questions and hypotheses. The research of Lindqvist and colleagues, for example, suggests that there may be a general relationship between how unexpected a suicide is and how consumed the family is with trying to understand why the teen committed suicide. This relationship can now be explored using quantitative research. But it is unclear whether this question would have arisen at all without the researchers sitting down with the families and listening to what they themselves wanted to say about their experience. Qualitative research can also provide rich and detailed descriptions of human behavior in the real-world contexts in which it occurs. Among qualitative researchers, this depth is often referred to as “thick description” (Geertz, 1973) .

Similarly, qualitative research can convey a sense of what it is actually like to be a member of a particular group or in a particular situation—what qualitative researchers often refer to as the “lived experience” of the research participants. Lindqvist and colleagues, for example, describe how all the families spontaneously offered to show the interviewer the victim’s bedroom or the place where the suicide occurred—revealing the importance of these physical locations to the families. It seems unlikely that a quantitative study would have discovered this detail. The table below lists some contrasts between qualitative and quantitative research

Data Collection and Analysis in Qualitative Research

Data collection approaches in qualitative research are quite varied and can involve naturalistic observation, participant observation, archival data, artwork, and many other things. But one of the most common approaches, especially for psychological research, is to conduct interviews. Interviews in qualitative research can be unstructured—consisting of a small number of general questions or prompts that allow participants to talk about what is of interest to them—or structured, where there is a strict script that the interviewer does not deviate from. Most interviews are in between the two and are called semi-structured interviews, where the researcher has a few consistent questions and can follow up by asking more detailed questions about the topics that come up. Such interviews can be lengthy and detailed, but they are usually conducted with a relatively small sample. The unstructured interview was the approach used by Lindqvist and colleagues in their research on the families of suicide victims because the researchers were aware that how much was disclosed about such a sensitive topic should be led by the families, not by the researchers.

Another approach used in qualitative research involves small groups of people who participate together in interviews focused on a particular topic or issue, known as focus groups. The interaction among participants in a focus group can sometimes bring out more information than can be learned in a one- on-one interview. The use of focus groups has become a standard technique in business and industry among those who want to understand consumer tastes and preferences. The content of all focus group interviews is usually recorded and transcribed to facilitate later analyses. However, we know from social psychology that group dynamics are often at play in any group, including focus groups, and it is useful to be aware of those possibilities. For example, the desire to be liked by others can lead participants to provide inaccurate answers that they believe will be perceived favorably by the other participants. The same may be said for personality characteristics. For example, highly extraverted participants can sometimes dominate discussions within focus groups.

Data Analysis in Qualitative Research

Although quantitative and qualitative research generally differ along several important dimensions (e.g., the specificity of the research question, the type of data collected), it is the method of data analysis that distinguishes them more clearly than anything else. To illustrate this idea, imagine a team of researchers that conducts a series of unstructured interviews with people recovering from alcohol use disorder to learn about the role of their religious faith in their recovery. Although this project sounds like qualitative research, imagine further that once they collect the data, they code the data in terms of how often each participant mentions God (or a “higher power”), and they then use descriptive and inferential statistics to find out whether those who mention God more often are more successful in abstaining from alcohol. Now it sounds like quantitative research. In other words, the quantitative-qualitative distinction depends more on what researchers do with the data they have collected than with why or how they collected the data.

But what does qualitative data analysis look like? Just as there are many ways to collect data in qualitative research, there are many ways to analyze data. Here we focus on one general approach called grounded theory (Glaser & Strauss, 1967) . This approach was developed within the field of sociology in the 1960s and has gradually gained popularity in psychology. Remember that in quantitative research, it is typical for the researcher to start with a theory, derive a hypothesis from that theory, and then collect data to test that specific hypothesis. In qualitative research using grounded theory, researchers start with the data and develop a theory or an interpretation that is “grounded in” those data. They do this analysis in stages. First, they identify ideas that are repeated throughout the data. Then they organize these ideas into a smaller number of broader themes. Finally, they write a theoretical narrative—an interpretation of the data in terms of the themes that they have identified. This theoretical narrative focuses on the subjective experience of the participants and is usually supported by many direct quotations from the participants themselves.

As an example, consider a study by researchers Laura Abrams and Laura Curran, who used the grounded theory approach to study the experience of postpartum depression symptoms among low-income mothers (Abrams & Curran, 2009) . Their data were the result of unstructured interviews with 19 participants. The table below hows the five broad themes the researchers identified and the more specific repeating ideas that made up each of those themes. In their research report, they provide numerous quotations from their participants, such as this one from “Destiny:”

“Well, just recently my apartment was broken into and the fact that his Medicaid for some reason was cancelled so a lot of things was happening within the last two weeks all at one time. So that in itself I don’t want to say almost drove me mad but it put me in a funk….Like I really was depressed. (p. 357)”

Their theoretical narrative focused on the participants’ experience of their symptoms, not as an abstract “affective disorder” but as closely tied to the daily struggle of raising children alone under often difficult circumstances. The table below illustrates the process of creating themes from repeating ideas in the qualitative research gathering and analysis process.

Given their differences, it may come as no surprise that quantitative and qualitative research do not coexist in complete harmony. Some quantitative researchers criticize qualitative methods on the grounds that they lack objectivity, are difficult to evaluate in terms of reliability and validity, and do not allow generalization to people or situations other than those actually studied. At the same time, some qualitative researchers criticize quantitative methods on the grounds that they overlook the richness of human behavior and experience and instead answer simple questions about easily quantifiable variables.

In general, however, qualitative researchers are well aware of the issues of objectivity, reliability, validity, and generalizability. In fact, they have developed a number of frameworks for addressing these issues (which are beyond the scope of our discussion). And in general, quantitative researchers are well aware of the issue of oversimplification. They do not believe that all human behavior and experience can be adequately described in terms of a small number of variables and the statistical relationships among them. Instead, they use simplification as a strategy for uncovering general principles of human behavior.

Many researchers from both the quantitative and qualitative camps now agree that the two approaches can and should be combined into what has come to be called mixed-methods research (Todd, Nerlich, McKeown, & Clarke, 2004). In fact, the studies by Lindqvist and colleagues and by Abrams and Curran both combined quantitative and qualitative approaches. One approach to combining quantitative and qualitative research is to use qualitative research for hypothesis generation and quantitative research for hypothesis testing. Again, while a qualitative study might suggest that families who experience an unexpected suicide have more difficulty resolving the question of why, a well-designed quantitative study could test a hypothesis by measuring these specific variables in a large sample. A second approach to combining quantitative and qualitative research is referred to as triangulation. The idea is to use both quantitative and qualitative methods simultaneously to study the same general questions and to compare the results. If the results of the quantitative and qualitative methods converge on the same general conclusion, they reinforce and enrich each other. If the results diverge, then they suggest an interesting new question: Why do the results diverge and how can they be reconciled?

Using qualitative research can often help clarify quantitative results via triangulation. Trenor, Yu, Waight, Zerda, and Sha (2008) investigated the experience of female engineering students at a university. In the first phase, female engineering students were asked to complete a survey, where they rated a number of their perceptions, including their sense of belonging. Their results were compared across the student ethnicities, and statistically, the various ethnic groups showed no differences in their ratings of their sense of belonging.

One might look at that result and conclude that ethnicity does not have anything to do with one’s sense of belonging. However, in the second phase, the authors also conducted interviews with the students, and in those interviews, many minority students reported how the diversity of cultures at the university enhanced their sense of belonging. Without the qualitative component, we might have drawn the wrong conclusion about the quantitative results.

This example shows how qualitative and quantitative research work together to help us understand human behavior. Some researchers have characterized qualitative research as best for identifying behaviors or the phenomenon whereas quantitative research is best for understanding meaning or identifying the mechanism. However, Bryman (2012) argues for breaking down the divide between these arbitrarily different ways of investigating the same questions.

Key Takeaways

• The qualitative approach is centered on an inductive method of reasoning
• The qualitative approach focuses on understanding phenomenon through the perspective of those experiencing it
• Researchers search for recurring topics and group themes to build upon theory to explain findings
• A mixed methods approach uses both quantitative and qualitative methods to explain different aspects of a phenomenon, processes, or practice
• This chapter can be attributed to Research Methods in Psychology by Rajiv S. Jhangiani, I-Chant A. Chiang, Carrie Cuttler, & Dana C. Leighton is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License, except where otherwise noted. This adaptation constitutes the fourth edition of this textbook, and builds upon the second Canadian edition by Rajiv S. Jhangiani (Kwantlen Polytechnic University) and I-Chant A. Chiang (Quest University Canada), the second American edition by Dana C. Leighton (Texas A&M University-Texarkana), and the third American edition by Carrie Cuttler (Washington State University) and feedback from several peer reviewers coordinated by the Rebus Community. This edition is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. ↵

Gathering and Analyzing Qualitative Data Copyright © by megankoster is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 7. The Results
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

The results section is where you report the findings of your study based upon the methodology [or methodologies] you applied to gather information. The results section should state the findings of the research arranged in a logical sequence without bias or interpretation. A section describing results should be particularly detailed if your paper includes data generated from your own research.

Annesley, Thomas M. "Show Your Cards: The Results Section and the Poker Game." Clinical Chemistry 56 (July 2010): 1066-1070.

Importance of a Good Results Section

When formulating the results section, it's important to remember that the results of a study do not prove anything . Findings can only confirm or reject the hypothesis underpinning your study. However, the act of articulating the results helps you to understand the problem from within, to break it into pieces, and to view the research problem from various perspectives.

The page length of this section is set by the amount and types of data to be reported . Be concise. Use non-textual elements appropriately, such as figures and tables, to present findings more effectively. In deciding what data to describe in your results section, you must clearly distinguish information that would normally be included in a research paper from any raw data or other content that could be included as an appendix. In general, raw data that has not been summarized should not be included in the main text of your paper unless requested to do so by your professor.

Avoid providing data that is not critical to answering the research question . The background information you described in the introduction section should provide the reader with any additional context or explanation needed to understand the results. A good strategy is to always re-read the background section of your paper after you have written up your results to ensure that the reader has enough context to understand the results [and, later, how you interpreted the results in the discussion section of your paper that follows].

Bavdekar, Sandeep B. and Sneha Chandak. "Results: Unraveling the Findings." Journal of the Association of Physicians of India 63 (September 2015): 44-46; Brett, Paul. "A Genre Analysis of the Results Section of Sociology Articles." English for Specific Speakers 13 (1994): 47-59; Go to English for Specific Purposes on ScienceDirect;Burton, Neil et al. Doing Your Education Research Project . Los Angeles, CA: SAGE, 2008; Results. The Structure, Format, Content, and Style of a Journal-Style Scientific Paper. Department of Biology. Bates College; Kretchmer, Paul. Twelve Steps to Writing an Effective Results Section. San Francisco Edit; "Reporting Findings." In Making Sense of Social Research Malcolm Williams, editor. (London;: SAGE Publications, 2003) pp. 188-207.

Structure and Writing Style

I.  Organization and Approach

For most research papers in the social and behavioral sciences, there are two possible ways of organizing the results . Both approaches are appropriate in how you report your findings, but use only one approach.

• Present a synopsis of the results followed by an explanation of key findings . This approach can be used to highlight important findings. For example, you may have noticed an unusual correlation between two variables during the analysis of your findings. It is appropriate to highlight this finding in the results section. However, speculating as to why this correlation exists and offering a hypothesis about what may be happening belongs in the discussion section of your paper.
• Present a result and then explain it, before presenting the next result then explaining it, and so on, then end with an overall synopsis . This is the preferred approach if you have multiple results of equal significance. It is more common in longer papers because it helps the reader to better understand each finding. In this model, it is helpful to provide a brief conclusion that ties each of the findings together and provides a narrative bridge to the discussion section of the your paper.

NOTE :   Just as the literature review should be arranged under conceptual categories rather than systematically describing each source, you should also organize your findings under key themes related to addressing the research problem. This can be done under either format noted above [i.e., a thorough explanation of the key results or a sequential, thematic description and explanation of each finding].

II.  Content

In general, the content of your results section should include the following:

• Introductory context for understanding the results by restating the research problem underpinning your study . This is useful in re-orientating the reader's focus back to the research problem after having read a review of the literature and your explanation of the methods used for gathering and analyzing information.
• Inclusion of non-textual elements, such as, figures, charts, photos, maps, tables, etc. to further illustrate key findings, if appropriate . Rather than relying entirely on descriptive text, consider how your findings can be presented visually. This is a helpful way of condensing a lot of data into one place that can then be referred to in the text. Consider referring to appendices if there is a lot of non-textual elements.
• A systematic description of your results, highlighting for the reader observations that are most relevant to the topic under investigation . Not all results that emerge from the methodology used to gather information may be related to answering the " So What? " question. Do not confuse observations with interpretations; observations in this context refers to highlighting important findings you discovered through a process of reviewing prior literature and gathering data.
• The page length of your results section is guided by the amount and types of data to be reported . However, focus on findings that are important and related to addressing the research problem. It is not uncommon to have unanticipated results that are not relevant to answering the research question. This is not to say that you don't acknowledge tangential findings and, in fact, can be referred to as areas for further research in the conclusion of your paper. However, spending time in the results section describing tangential findings clutters your overall results section and distracts the reader.
• A short paragraph that concludes the results section by synthesizing the key findings of the study . Highlight the most important findings you want readers to remember as they transition into the discussion section. This is particularly important if, for example, there are many results to report, the findings are complicated or unanticipated, or they are impactful or actionable in some way [i.e., able to be pursued in a feasible way applied to practice].

NOTE:   Always use the past tense when referring to your study's findings. Reference to findings should always be described as having already happened because the method used to gather the information has been completed.

III.  Problems to Avoid

When writing the results section, avoid doing the following :

• Discussing or interpreting your results . Save this for the discussion section of your paper, although where appropriate, you should compare or contrast specific results to those found in other studies [e.g., "Similar to the work of Smith [1990], one of the findings of this study is the strong correlation between motivation and academic achievement...."].
• Reporting background information or attempting to explain your findings. This should have been done in your introduction section, but don't panic! Often the results of a study point to the need for additional background information or to explain the topic further, so don't think you did something wrong. Writing up research is rarely a linear process. Always revise your introduction as needed.
• Ignoring negative results . A negative result generally refers to a finding that does not support the underlying assumptions of your study. Do not ignore them. Document these findings and then state in your discussion section why you believe a negative result emerged from your study. Note that negative results, and how you handle them, can give you an opportunity to write a more engaging discussion section, therefore, don't be hesitant to highlight them.
• Including raw data or intermediate calculations . Ask your professor if you need to include any raw data generated by your study, such as transcripts from interviews or data files. If raw data is to be included, place it in an appendix or set of appendices that are referred to in the text.
• Be as factual and concise as possible in reporting your findings . Do not use phrases that are vague or non-specific, such as, "appeared to be greater than other variables..." or "demonstrates promising trends that...." Subjective modifiers should be explained in the discussion section of the paper [i.e., why did one variable appear greater? Or, how does the finding demonstrate a promising trend?].
• Presenting the same data or repeating the same information more than once . If you want to highlight a particular finding, it is appropriate to do so in the results section. However, you should emphasize its significance in relation to addressing the research problem in the discussion section. Do not repeat it in your results section because you can do that in the conclusion of your paper.
• Confusing figures with tables . Be sure to properly label any non-textual elements in your paper. Don't call a chart an illustration or a figure a table. If you are not sure, go here .

Annesley, Thomas M. "Show Your Cards: The Results Section and the Poker Game." Clinical Chemistry 56 (July 2010): 1066-1070; Bavdekar, Sandeep B. and Sneha Chandak. "Results: Unraveling the Findings." Journal of the Association of Physicians of India 63 (September 2015): 44-46; Burton, Neil et al. Doing Your Education Research Project . Los Angeles, CA: SAGE, 2008;  Caprette, David R. Writing Research Papers. Experimental Biosciences Resources. Rice University; Hancock, Dawson R. and Bob Algozzine. Doing Case Study Research: A Practical Guide for Beginning Researchers . 2nd ed. New York: Teachers College Press, 2011; Introduction to Nursing Research: Reporting Research Findings. Nursing Research: Open Access Nursing Research and Review Articles. (January 4, 2012); Kretchmer, Paul. Twelve Steps to Writing an Effective Results Section. San Francisco Edit ; Ng, K. H. and W. C. Peh. "Writing the Results." Singapore Medical Journal 49 (2008): 967-968; Reporting Research Findings. Wilder Research, in partnership with the Minnesota Department of Human Services. (February 2009); Results. The Structure, Format, Content, and Style of a Journal-Style Scientific Paper. Department of Biology. Bates College; Schafer, Mickey S. Writing the Results. Thesis Writing in the Sciences. Course Syllabus. University of Florida.

Writing Tip

Why Don't I Just Combine the Results Section with the Discussion Section?

It's not unusual to find articles in scholarly social science journals where the author(s) have combined a description of the findings with a discussion about their significance and implications. You could do this. However, if you are inexperienced writing research papers, consider creating two distinct sections for each section in your paper as a way to better organize your thoughts and, by extension, your paper. Think of the results section as the place where you report what your study found; think of the discussion section as the place where you interpret the information and answer the "So What?" question. As you become more skilled writing research papers, you can consider melding the results of your study with a discussion of its implications.

Driscoll, Dana Lynn and Aleksandra Kasztalska. Writing the Experimental Report: Methods, Results, and Discussion. The Writing Lab and The OWL. Purdue University.

• << Previous: Insiderness
• Next: Using Non-Textual Elements >>
• Last Updated: Apr 5, 2024 1:38 PM
• URL: https://libguides.usc.edu/writingguide

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

11.4 Strategies for Gathering Reliable Information

Learning objectives.

• Distinguish between primary and secondary sources.
• Identify strategies for locating relevant print and electronic resources efficiently.
• Identify instances when it is appropriate to use human sources, such as interviews or eyewitness testimony.
• Identify criteria for evaluating research resources.
• Understand why many electronic resources are not reliable.

Now that you have planned your research project, you are ready to begin the research. This phase can be both exciting and challenging. As you read this section, you will learn ways to locate sources efficiently, so you have enough time to read the sources, take notes, and think about how to use the information.

Of course, the technological advances of the past few decades—particularly the rise of online media—mean that, as a twenty-first-century student, you have countless sources of information available at your fingertips. But how can you tell whether a source is reliable? This section will discuss strategies for evaluating sources critically so that you can be a media-savvy researcher.

In this section, you will locate and evaluate resources for your paper and begin taking notes. As you read, begin gathering print and electronic resources, identify at least eight to ten sources by the time you finish the chapter, and begin taking notes on your research findings.

Locating Useful Resources

When you chose a paper topic and determined your research questions, you conducted preliminary research to stimulate your thinking. Your research proposal included some general ideas for how to go about your research—for instance, interviewing an expert in the field or analyzing the content of popular magazines. You may even have identified a few potential sources. Now it is time to conduct a more focused, systematic search for informative primary and secondary sources.

Using Primary and Secondary Sources

Writers classify research resources in two categories: primary sources and secondary sources. Primary sources are direct, firsthand sources of information or data. For example, if you were writing a paper about the First Amendment right to freedom of speech, the text of the First Amendment in the Bill of Rights would be a primary source.

Other primary sources include the following:

• Research articles
• Literary texts
• Historical documents such as diaries or letters
• Autobiographies or other personal accounts

Secondary sources discuss, interpret, analyze, consolidate, or otherwise rework information from primary sources. In researching a paper about the First Amendment, you might read articles about legal cases that involved First Amendment rights, or editorials expressing commentary on the First Amendment. These sources would be considered secondary sources because they are one step removed from the primary source of information.

The following are examples of secondary sources:

• Magazine articles
• Biographical books
• Literary and scientific reviews
• Television documentaries

Your topic and purpose determine whether you must cite both primary and secondary sources in your paper. Ask yourself which sources are most likely to provide the information that will answer your research questions. If you are writing a research paper about reality television shows, you will need to use some reality shows as a primary source, but secondary sources, such as a reviewer’s critique, are also important. If you are writing about the health effects of nicotine, you will probably want to read the published results of scientific studies, but secondary sources, such as magazine articles discussing the outcome of a recent study, may also be helpful.

Once you have thought about what kinds of sources are most likely to help you answer your research questions, you may begin your search for print and electronic resources. The challenge here is to conduct your search efficiently. Writers use strategies to help them find the sources that are most relevant and reliable while steering clear of sources that will not be useful.

Finding Print Resources

Print resources include a vast array of documents and publications. Regardless of your topic, you will consult some print resources as part of your research. (You will use electronic sources as well, but it is not wise to limit yourself to electronic sources only, because some potentially useful sources may be available only in print form.) Table 11.1 “Library Print Resources” lists different types of print resources available at public and university libraries.

Table 11.1 Library Print Resources

Some of these resources are also widely available in electronic format. In addition to the resources noted in the table, library holdings may include primary texts such as historical documents, letters, and diaries.

Writing at Work

Businesses, government organizations, and nonprofit organizations produce published materials that range from brief advertisements and brochures to lengthy, detailed reports. In many cases, producing these publications requires research. A corporation’s annual report may include research about economic or industry trends. A charitable organization may use information from research in materials sent to potential donors.

Regardless of the industry you work in, you may be asked to assist in developing materials for publication. Often, incorporating research in these documents can make them more effective in informing or persuading readers.

As you gather information, strive for a balance of accessible, easy-to-read sources and more specialized, challenging sources. Relying solely on lightweight books and articles written for a general audience will drastically limit the range of useful, substantial information. On the other hand, restricting oneself to dense, scholarly works could make the process of researching extremely time-consuming and frustrating.

Make a list of five types of print resources you could use to find information about your research topic. Include at least one primary source. Be as specific as possible—if you have a particular resource or type of resource in mind, describe it.

To find print resources efficiently, first identify the major concepts and terms you will use to conduct your search—that is, your keywords . These, along with the research questions you identified in Chapter 11 “Writing from Research: What Will I Learn?” , Section 11.2 “Steps in Developing a Research Proposal” , will help you find sources using any of the following methods:

• Using the library’s online catalog or card catalog
• Using periodicals indexes and databases
• Consulting a reference librarian

You probably already have some keywords in mind based on your preliminary research and writing. Another way to identify useful keywords is to visit the Library of Congress’s website at http://id.loc.gov/authorities . This site allows you to search for a topic and see the related subject headings used by the Library of Congress, including broader terms, narrower terms, and related terms. Other libraries use these terms to classify materials. Knowing the most-used terms will help you speed up your keyword search.

Jorge used the Library of Congress site to identify general terms he could use to find resources about low-carb dieting. His search helped him identify potentially useful keywords and related topics, such as carbohydrates in human nutrition, glycemic index, and carbohydrates—metabolism. These terms helped Jorge refine his search.

Knowing the right keywords can sometimes make all the difference in conducting a successful search. If you have trouble finding sources on a topic, consult a librarian to see whether you need to modify your search terms.

Visit the Library of Congress’s website at http://id.loc.gov/authorities and conduct searches on a few terms related to your topic.

• Review your search results and identify six to eight additional terms you might use when you conduct your research.
• Print out the search results or save the results to your research folder on your computer or portable storage device.

Using Periodicals, Indexes, and Databases

Library catalogs can help you locate book-length sources, as well as some types of nonprint holdings, such as CDs, DVDs, and audio books. To locate shorter sources, such as magazine and journal articles, you will need to use a periodical index or an online periodical database . These tools index the articles that appear in newspapers, magazines, and journals. Like catalogs, they provide publication information about an article and often allow users to access a summary or even the full text of the article.

Print indexes may be available in the periodicals section of your library. Increasingly, libraries use online databases that users can access through the library website. A single library may provide access to multiple periodical databases. These can range from general news databases to specialized databases. Table 11.2 “Commonly Used Indexes and Databases” describes some commonly used indexes and databases.

Table 11.2 Commonly Used Indexes and Databases

Reading Popular and Scholarly Periodicals

When you search for periodicals, be sure to distinguish among different types. Mass-market publications, such as newspapers and popular magazines, differ from scholarly publications in their accessibility, audience, and purpose.

Newspapers and magazines are written for a broader audience than scholarly journals. Their content is usually quite accessible and easy to read. Trade magazines that target readers within a particular industry may presume the reader has background knowledge, but these publications are still reader-friendly for a broader audience. Their purpose is to inform and, often, to entertain or persuade readers as well.

Scholarly or academic journals are written for a much smaller and more expert audience. The creators of these publications assume that most of their readers are already familiar with the main topic of the journal. The target audience is also highly educated. Informing is the primary purpose of a scholarly journal. While a journal article may advance an agenda or advocate a position, the content will still be presented in an objective style and formal tone. Entertaining readers with breezy comments and splashy graphics is not a priority.

Because of these differences, scholarly journals are more challenging to read. That doesn’t mean you should avoid them. On the contrary, they can provide in-depth information unavailable elsewhere. Because knowledgeable professionals carefully review the content before publication, scholarly journals are far more reliable than much of the information available in popular media. Seek out academic journals along with other resources. Just be prepared to spend a little more time processing the information.

Periodicals databases are not just for students writing research papers. They also provide a valuable service to workers in various fields. The owner of a small business might use a database such as Business Source Premiere to find articles on management, finance, or trends within a particular industry. Health care professionals might consult databases such as MedLine to research a particular disease or medication. Regardless of what career path you plan to pursue, periodicals databases can be a useful tool for researching specific topics and identifying periodicals that will help you keep up with the latest news in your industry.

Consulting a Reference Librarian

Sifting through library stacks and database search results to find the information you need can be like trying to find a needle in a haystack. If you are not sure how you should begin your search, or if it is yielding too many or too few results, you are not alone. Many students find this process challenging, although it does get easier with experience. One way to learn better search strategies is to consult a reference librarian.

Reference librarians are intimately familiar with the systems libraries use to organize and classify information. They can help you locate a particular book in the library stacks, steer you toward useful reference works, and provide tips on how to use databases and other electronic research tools. Take the time to see what resources you can find on your own, but if you encounter difficulties, ask for help. Many university librarians hold virtual office hours and are available for online chatting.

Visit your library’s website or consult with a reference librarian to determine what periodicals indexes or databases would be useful for your research. Depending on your topic, you may rely on a general news index, a specialized index for a particular subject area, or both. Search the catalog for your topic and related keywords. Print out or bookmark your search results.

• Identify at least one to two relevant periodicals, indexes, or databases.
• Conduct a keyword search to find potentially relevant articles on your topic.
• Save your search results. If the index you are using provides article summaries, read these to determine how useful the articles are likely to be.
• Identify at least three to five articles to review more closely. If the full article is available online, set aside time to read it. If not, plan to visit our library within the next few days to locate the articles you need.

One way to refine your keyword search is to use Boolean operators. These operators allow you to combine keywords, find variations on a word, and otherwise expand or limit your results. Here are some of the ways you can use Boolean operators:

• Combine keywords with and or + to limit results to citations that include both keywords—for example, diet + nutrition .
• Combine keywords with not or – to search for the first word without the second. This can help you eliminate irrelevant results based on words that are similar to your search term. For example, searching for obesity not childhood locates materials on obesity but excludes materials on childhood obesity.
• Enclose a phrase in quotation marks to search for an exact phrase, such as “ morbid obesity .”
• Use parentheses to direct the order of operations in a search string. For example, since Type II diabetes is also known as adult-onset diabetes, you could search (Type II or adult-onset) and diabetes to limit your search results to articles on this form of the disease.
• Use a wildcard symbol such as # , ? , or $ after a word to search for variations on a term. For instance, you might type diabet# to search for information on diabetes and diabetics. The specific symbol used varies with different databases.

Finding and Using Electronic Resources

With the expansion of technology and media over the past few decades, a wealth of information is available to you in electronic format. Some types of resources, such as a television documentary, may only be available electronically. Other resources—for instance, many newspapers and magazines—may be available in both print and electronic form. The following are some of the electronic sources you might consult:

• Online databases
• Popular web search engines
• Websites maintained by businesses, universities, nonprofit organizations, or government agencies
• Newspapers, magazines, and journals published on the web
• Audio books
• Industry blogs
• Radio and television programs and other audio and video recordings
• Online discussion groups

The techniques you use to locate print resources can also help you find electronic resources efficiently. Libraries usually include CD-ROMs, audio books, and audio and video recordings among their holdings. You can locate these materials in the catalog using a keyword search. The same Boolean operators used to refine database searches can help you filter your results in popular search engines.

Using Internet Search Engines Efficiently

When faced with the challenge of writing a research paper, some students rely on popular search engines as their first source of information. Typing a keyword or phrase into a search engine instantly pulls up links to dozens, hundreds, or even thousands of related websites—what could be easier? Unfortunately, despite its apparent convenience, this research strategy has the following drawbacks to consider:

• Results do not always appear in order of reliability. The first few hits that appear in search results may include sites whose content is not always reliable, such as online encyclopedias that can be edited by any user. Because websites are created by third parties, the search engine cannot tell you which sites have accurate information.
• Results may be too numerous for you to use. The amount of information available on the web is far greater than the amount of information housed within a particular library or database. Realistically, if your web search pulls up thousands of hits, you will not be able to visit every site—and the most useful sites may be buried deep within your search results.
• Search engines are not connected to the results of the search. Search engines find websites that people visit often and list the results in order of popularity. The search engine, then, is not connected to any of the results. When you cite a source found through a search engine, you do not need to cite the search engine. Only cite the source.

A general web search can provide a helpful overview of a topic and may pull up genuinely useful resources. To get the most out of a search engine, however, use strategies to make your search more efficient. Use multiple keywords and Boolean operators to limit your results. Click on the Advanced Search link on the homepage to find additional options for streamlining your search. Depending on the specific search engine you use, the following options may be available:

• Limit results to websites that have been updated within a particular time frame.
• Limit results by language or country.
• Limit results to scholarly works available online.
• Limit results by file type.
• Limit results to a particular domain type, such as .edu (school and university sites) or .gov (government sites). This is a quick way to filter out commercial sites, which can often lead to more objective results.

Use the Bookmarks or Favorites feature of your web browser to save and organize sites that look promising.

Using Other Information Sources: Interviews

With so many print and electronic media readily available, it is easy to overlook another valuable information resource: other people. Consider whether you could use a person or group as a primary source. For instance, you might interview a professor who has expertise in a particular subject, a worker within a particular industry, or a representative from a political organization. Interviews can be a great way to get firsthand information.

To get the most out of an interview, you will need to plan ahead. Contact your subject early in the research process and explain your purpose for requesting an interview. Prepare detailed questions. Open-ended questions, rather than questions with simple yes-or-no answers, are more likely to lead to an in-depth discussion. Schedule a time to meet, and be sure to obtain your subject’s permission to record the interview. Take careful notes and be ready to ask follow-up questions based on what you learn.

If scheduling an in-person meeting is difficult, consider arranging a telephone interview or asking your subject to respond to your questions via e-mail. Recognize that any of these formats takes time and effort. Be prompt and courteous, avoid going over the allotted interview time, and be flexible if your subject needs to reschedule.

Evaluating Research Resources

As you gather sources, you will need to examine them with a critical eye. Smart researchers continually ask themselves two questions: “Is this source relevant to my purpose?” and “Is this source reliable?” The first question will help you avoid wasting valuable time reading sources that stray too far from your specific topic and research questions. The second question will help you find accurate, trustworthy sources.

Determining Whether a Source Is Relevant

At this point in your research process, you may have identified dozens of potential sources. It is easy for writers to get so caught up in checking out books and printing out articles that they forget to ask themselves how they will use these resources in their research. Now is a good time to get a little ruthless. Reading and taking notes takes time and energy, so you will want to focus on the most relevant sources.

To weed through your stack of books and articles, skim their contents. Read quickly with your research questions and subtopics in mind. Table 11.3 “Tips for Skimming Books and Articles” explains how to skim to get a quick sense of what topics are covered. If a book or article is not especially relevant, put it aside. You can always come back to it later if you need to.

Table 11.3 Tips for Skimming Books and Articles

Determining Whether a Source Is Reliable

All information sources are not created equal. Sources can vary greatly in terms of how carefully they are researched, written, edited, and reviewed for accuracy. Common sense will help you identify obviously questionable sources, such as tabloids that feature tales of alien abductions, or personal websites with glaring typos. Sometimes, however, a source’s reliability—or lack of it—is not so obvious. For more information about source reliability, see Chapter 12 “Writing a Research Paper” .

To evaluate your research sources, you will use critical thinking skills consciously and deliberately. You will consider criteria such as the type of source, its intended purpose and audience, the author’s (or authors’) qualifications, the publication’s reputation, any indications of bias or hidden agendas, how current the source is, and the overall quality of the writing, thinking, and design.

Evaluating Types of Sources

The different types of sources you will consult are written for distinct purposes and with different audiences in mind. This accounts for other differences, such as the following:

• How thoroughly the writers cover a given topic
• How carefully the writers research and document facts
• How editors review the work
• What biases or agendas affect the content

A journal article written for an academic audience for the purpose of expanding scholarship in a given field will take an approach quite different from a magazine feature written to inform a general audience. Textbooks, hard news articles, and websites approach a subject from different angles as well. To some extent, the type of source provides clues about its overall depth and reliability. Table 11.4 “Source Rankings” ranks different source types.

Table 11.4 Source Rankings

Free online encyclopedias and wikis may seem like a great source of information. They usually appear among the first few results of a web search. They cover thousands of topics, and many articles use an informal, straightforward writing style. Unfortunately, these sites have no control system for researching, writing, and reviewing articles. Instead, they rely on a community of users to police themselves. At best, these sites can be a starting point for finding other, more trustworthy sources. Never use them as final sources.

Evaluating Credibility and Reputability

Even when you are using a type of source that is generally reliable, you will still need to evaluate the author’s credibility and the publication itself on an individual basis. To examine the author’s credibility —that is, how much you can believe of what the author has to say—examine his or her credentials. What career experience or academic study shows that the author has the expertise to write about this topic?

Keep in mind that expertise in one field is no guarantee of expertise in another, unrelated area. For instance, an author may have an advanced degree in physiology, but this credential is not a valid qualification for writing about psychology. Check credentials carefully.

Just as important as the author’s credibility is the publication’s overall reputability. Reputability refers to a source’s standing and reputation as a respectable, reliable source of information. An established and well-known newspaper, such as the New York Times or the Wall Street Journal , is more reputable than a college newspaper put out by comparatively inexperienced students. A website that is maintained by a well-known, respected organization and regularly updated is more reputable than one created by an unknown author or group.

If you are using articles from scholarly journals, you can check databases that keep count of how many times each article has been cited in other articles. This can be a rough indication of the article’s quality or, at the very least, of its influence and reputation among other scholars.

Checking for Biases and Hidden Agendas

Whenever you consult a source, always think carefully about the author’s or authors’ purpose in presenting the information. Few sources present facts completely objectively. In some cases, the source’s content and tone are significantly influenced by biases or hidden agendas.

Bias refers to favoritism or prejudice toward a particular person or group. For instance, an author may be biased against a certain political party and present information in a way that subtly—or not so subtly—makes that organization look bad. Bias can lead an author to present facts selectively, edit quotations to misrepresent someone’s words, and distort information.

Hidden agendas are goals that are not immediately obvious but influence how an author presents the facts. For instance, an article about the role of beef in a healthy diet would be questionable if it were written by a representative of the beef industry—or by the president of an animal-rights organization. In both cases, the author would likely have a hidden agenda.

As Jorge conducted his research, he read several research studies in which scientists found significant benefits to following a low-carbohydrate diet. He also noticed that many studies were sponsored by a foundation associated with the author of a popular series of low-carbohydrate diet books. Jorge read these studies with a critical eye, knowing that a hidden agenda might be shaping the researchers’ conclusions.

Using Current Sources

Be sure to seek out sources that are current, or up to date. Depending on the topic, sources may become outdated relatively soon after publication, or they may remain useful for years. For instance, online social networking sites have evolved rapidly over the past few years. An article published in 2002 about this topic will not provide current information. On the other hand, a research paper on elementary education practices might refer to studies published decades ago by influential child psychologists.

When using websites for research, check to see when the site was last updated. Many sites publish this information on the homepage, and some, such as news sites, are updated daily or weekly. Many nonfunctioning links are a sign that a website is not regularly updated. Do not be afraid to ask your professor for suggestions if you find that many of your most relevant sources are not especially reliable—or that the most reliable sources are not relevant.

Evaluating Overall Quality by Asking Questions

When you evaluate a source, you will consider the criteria previously discussed as well as your overall impressions of its quality. Read carefully, and notice how well the author presents and supports his or her statements. Stay actively engaged—do not simply accept an author’s words as truth. Ask questions to determine each source’s value. Checklist 11.1 lists ten questions to ask yourself as a critical reader.

Checklist 11.1

Source Evaluation

• Is the type of source appropriate for my purpose? Is it a high-quality source or one that needs to be looked at more critically?
• Can I establish that the author is credible and the publication is reputable?
• Does the author support ideas with specific facts and details that are carefully documented? Is the source of the author’s information clear? (When you use secondary sources, look for sources that are not too removed from primary research.)
• Does the source include any factual errors or instances of faulty logic?
• Does the author leave out any information that I would expect to see in a discussion of this topic?
• Do the author’s conclusions logically follow from the evidence that is presented? Can I see how the author got from one point to another?
• Is the writing clear and organized, and is it free from errors, clichés, and empty buzzwords? Is the tone objective, balanced, and reasonable? (Be on the lookout for extreme, emotionally charged language.)
• Are there any obvious biases or agendas? Based on what I know about the author, are there likely to be any hidden agendas?
• Are graphics informative, useful, and easy to understand? Are websites organized, easy to navigate, and free of clutter like flashing ads and unnecessary sound effects?
• Is the source contradicted by information found in other sources? (If so, it is possible that your sources are presenting similar information but taking different perspectives, which requires you to think carefully about which sources you find more convincing and why. Be suspicious, however, of any source that presents facts that you cannot confirm elsewhere.)

The critical thinking skills you use to evaluate research sources as a student are equally valuable when you conduct research on the job. If you follow certain periodicals or websites, you have probably identified publications that consistently provide reliable information. Reading blogs and online discussion groups is a great way to identify new trends and hot topics in a particular field, but these sources should not be used for substantial research.

Use a search engine to conduct a web search on your topic. Refer to the tips provided earlier to help you streamline your search. Evaluate your search results critically based on the criteria you have learned. Identify and bookmark one or more websites that are reliable, reputable, and likely to be useful in your research.

Managing Source Information

As you determine which sources you will rely on most, it is important to establish a system for keeping track of your sources and taking notes. There are several ways to go about it, and no one system is necessarily superior. What matters is that you keep materials in order; record bibliographical information you will need later; and take detailed, organized notes.

Keeping Track of Your Sources

Think ahead to a moment a few weeks from now, when you’ve written your research paper and are almost ready to submit it for a grade. There is just one task left—writing your list of sources.

As you begin typing your list, you realize you need to include the publication information for a book you cited frequently. Unfortunately, you already returned it to the library several days ago. You do not remember the URLs for some of the websites you used or the dates you accessed them—information that also must be included in your bibliography. With a sinking feeling, you realize that finding this information and preparing your bibliography will require hours of work.

This stressful scenario can be avoided. Taking time to organize source information now will ensure that you are not scrambling to find it at the last minute. Throughout your research, record bibliographical information for each source as soon as you begin using it. You may use pen-and-paper methods, such as a notebook or note cards, or maintain an electronic list. (If you prefer the latter option, many office software packages include separate programs for recording bibliographic information.)

Table 11.5 “Details for Commonly Used Source Types” shows the specific details you should record for commonly used source types. Use these details to develop a working bibliography —a preliminary list of sources that you will later use to develop the references section of your paper. You may wish to record information using the formatting system of the American Psychological Association (APA) or the Modern Language Association (MLA), which will save a step later on. (For more information on APA and MLA formatting, see Chapter 13 “APA and MLA Documentation and Formatting” .)

Table 11.5 Details for Commonly Used Source Types

Your research may involve less common types of sources not listed in Table 11.5 “Details for Commonly Used Source Types” . For additional information on citing different sources, see Chapter 13 “APA and MLA Documentation and Formatting” .

Create a working bibliography using the format that is most convenient for you. List at least five sources you plan to use. Continue to add sources to your working bibliography throughout the research process.

To make your working bibliography even more complete, you may wish to record additional details, such as a book’s call number or contact information for a person you interviewed. That way, if you need to locate a source again, you have all the information you need right at your fingertips. You may also wish to assign each source a code number to use when taking notes (1, 2, 3, or a similar system).

Taking Notes Efficiently

Good researchers stay focused and organized as they gather information from sources. Before you begin taking notes, take a moment to step back and think about your goal as a researcher—to find information that will help you answer your research question. When you write your paper, you will present your conclusions about the topic supported by research. That goal will determine what information you record and how you organize it.

Writers sometimes get caught up in taking extensive notes, so much so that they lose sight of how their notes relate to the questions and ideas they started out with. Remember that you do not need to write down every detail from your reading. Focus on finding and recording details that will help you answer your research questions. The following strategies will help you take notes efficiently.

Use Headings to Organize Ideas

Whether you use old-fashioned index cards or organize your notes using word-processing software, record just one major point from each source at a time, and use a heading to summarize the information covered. Keep all your notes in one file, digital or otherwise. Doing so will help you identify connections among different pieces of information. It will also help you make connections between your notes and the research questions and subtopics you identified earlier.

Know When to Summarize, Paraphrase, or Directly Quote a Source

Your notes will fall under three categories—summary notes, paraphrased information, and direct quotations from your sources. Effective researchers make choices about which type of notes is most appropriate for their purpose.

• Summary notes sum up the main ideas in a source in a few sentences or a short paragraph. A summary is considerably shorter than the original text and captures only the major ideas. Use summary notes when you do not need to record specific details but you intend to refer to broad concepts the author discusses.
• Paraphrased notes restate a fact or idea from a source using your own words and sentence structure.
• Direct quotations use the exact wording used by the original source and enclose the quoted material in quotation marks. It is a good strategy to copy direct quotations when an author expresses an idea in an especially lively or memorable way. However, do not rely exclusively on direct quotations in your note taking.

Most of your notes should be paraphrased from the original source. Paraphrasing as you take notes is usually a better strategy than copying direct quotations, because it forces you to think through the information in your source and understand it well enough to restate it. In short, it helps you stay engaged with the material instead of simply copying and pasting. Synthesizing will help you later when you begin planning and drafting your paper. (For detailed guidelines on summarizing, paraphrasing, and quoting, see Chapter 11 “Writing from Research: What Will I Learn?” , Section 11.6 “Writing from Research: End-of-Chapter Exercises” .)

Maintain Complete, Accurate Notes

Regardless of the format used, any notes you take should include enough information to help you organize ideas and locate them instantly in the original text if you need to review them. Make sure your notes include the following elements:

• Heading summing up the main topic covered
• Author’s name, a source code, or an abbreviated source title
• Page number
• Full URL of any pages buried deep in a website

Throughout the process of taking notes, be scrupulous about making sure you have correctly attributed each idea to its source. Always include source information so you know exactly which ideas came from which sources. Use quotation marks to set off any words for phrases taken directly from the original text. If you add your own responses and ideas, make sure they are distinct from ideas you quoted or paraphrased.

Finally, make sure your notes accurately reflect the content of the original text. Make sure quoted material is copied verbatim. If you omit words from a quotation, use ellipses to show the omission and make sure the omission does not change the author’s meaning. Paraphrase ideas carefully, and check your paraphrased notes against the original text to make sure that you have restated the author’s ideas accurately in your own words.

Use a System That Works for You

There are several formats you can use to take notes. No technique is necessarily better than the others—it is more important to choose a format you are comfortable using. Choosing the format that works best for you will ensure your notes are organized, complete, and accurate. Consider implementing one of these formats when you begin taking notes:

• Use index cards. This traditional format involves writing each note on a separate index card. It takes more time than copying and pasting into an electronic document, which encourages you to be selective in choosing which ideas to record. Recording notes on separate cards makes it easy to later organize your notes according to major topics. Some writers color-code their cards to make them still more organized.
• Use note-taking software. Word-processing and office software packages often include different types of note-taking software. Although you may need to set aside some time to learn the software, this method combines the speed of typing with the same degree of organization associated with handwritten note cards.
• Maintain a research notebook. Instead of using index cards or electronic note cards, you may wish to keep a notebook or electronic folder, allotting a few pages (or one file) for each of your sources. This method makes it easy to create a separate column or section of the document where you add your responses to the information you encounter in your research.
• Annotate your sources. This method involves making handwritten notes in the margins of sources that you have printed or photocopied. If using electronic sources, you can make comments within the source document. For example, you might add comment boxes to a PDF version of an article. This method works best for experienced researchers who have already thought a great deal about the topic because it can be difficult to organize your notes later when starting your draft.

Choose one of the methods from the list to use for taking notes. Continue gathering sources and taking notes. In the next section, you will learn strategies for organizing and synthesizing the information you have found.

Key Takeaways

• A writer’s use of primary and secondary sources is determined by the topic and purpose of the research. Sources used may include print sources, such as books and journals; electronic sources, such as websites and articles retrieved from databases; and human sources of information, such as interviews.
• Strategies that help writers locate sources efficiently include conducting effective keyword searches, understanding how to use online catalogs and databases, using strategies to narrow web search results, and consulting reference librarians.
• Writers evaluate sources based on how relevant they are to the research question and how reliable their content is.
• Skimming sources can help writers determine their relevance efficiently.
• Writers evaluate a source’s reliability by asking questions about the type of source (including its audience and purpose); the author’s credibility, the publication’s reputability, the source’s currency, and the overall quality of the writing, research, logic, and design in the source.
• In their notes, effective writers record organized, complete, accurate information. This includes bibliographic information about each source as well as summarized, paraphrased, or quoted information from the source.

Writing for Success Copyright © 2015 by University of Minnesota is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

• Privacy Policy

Buy Me a Coffee

Home » Information in Research – Types and Examples

Information in Research – Types and Examples

Table of Contents

Information in Research

Definition:

Information in research refers to the data and knowledge that researchers gather, analyze, and interpret in order to answer research questions or test hypotheses. This information can come from a variety of sources, such as experiments, surveys, interviews, observations, literature reviews, and existing datasets.

Types of Information in Research

Types of Information in Research are as follows:

Quantitative information

This refers to numerical data that can be measured and analyzed using statistical methods. Examples of quantitative information include measurements of height, weight, blood pressure, and IQ scores.

Qualitative information

This refers to non-numerical data that is collected through observations, interviews, and other subjective methods. Examples of qualitative information include descriptions of emotions, attitudes, beliefs, and behaviors.

Primary information

This refers to information that is collected directly from the source, such as through experiments, surveys, and interviews. Primary information is typically more reliable than secondary information, which is obtained from other sources.

Secondary information

This refers to information that is collected from existing sources, such as books, articles, and databases. While secondary information can be useful in research, it is important to verify its accuracy and reliability before using it.

Empirical information

This refers to information that is based on observation or experience rather than theory or speculation. Empirical information is typically considered more reliable than non-empirical information.

Theoretical information

This refers to information that is based on theories or hypotheses. Theoretical information can be useful in developing new ideas and concepts, but it must be tested empirically to verify its accuracy.

Importance of Information in Research

Here are some key reasons why information is important in research:

• Formulating research questions: Before starting any research project, researchers must identify a problem or question that they want to investigate. This requires a thorough understanding of the current state of knowledge about the topic, which can only be obtained through information gathering.
• Designing studies: Once researchers have identified their research questions, they need to design a study that will enable them to answer those questions. This requires a clear understanding of the relevant theories, concepts, and methods, which can only be obtained through information gathering.
• Collecting data: Researchers must collect data that is reliable and valid in order to draw accurate conclusions. They need to know what types of data to collect, how to collect them, and how to ensure the quality of the data. This requires a thorough understanding of the relevant literature and best practices, which can only be obtained through information gathering.
• Analyzing results: Researchers must analyze the data they have collected in order to draw conclusions about their research questions. They need to know what statistical tests to use, how to interpret the results, and how to draw valid conclusions. This requires a thorough understanding of the relevant methods and techniques, which can only be obtained through information gathering.
• Drawing conclusions: Finally, researchers must draw conclusions about their research questions based on the data analysis. They need to know how to interpret their findings in light of the relevant literature, and how to make recommendations for future research. This requires a thorough understanding of the current state of knowledge about the topic, which can only be obtained through information gathering.

Purpose of Information in Research

The purpose of information in research is to provide a foundation for understanding the research question, developing hypotheses, and testing those hypotheses. Information is used to gather evidence, support or refute theories, and provide a basis for making informed conclusions.

Information can come from a variety of sources, such as primary and secondary research, literature reviews, surveys, experiments, and observations. Researchers must critically evaluate the information they gather to ensure it is accurate, relevant, and reliable.

In addition to supporting the research process, information plays an important role in communicating research findings to others. Researchers must be able to clearly and effectively present their findings to others in order to advance knowledge and understanding in their field.

Characteristics of Information in Research

Information used in research should possess the following characteristics:

• Accuracy : Information should be factual, free from errors or biases, and supported by evidence.
• Reliability : The information should come from reliable sources and be verifiable, consistent, and repeatable.
• Relevance : Information should be relevant to the research question, hypothesis, or objective.
• Currency : Information should be up-to-date and reflect the most current understanding of the topic.
• Objectivity : Information should be free from personal opinions, biases, or values that may influence its interpretation.
• Completeness : Information should be comprehensive, covering all relevant aspects of the research question or topic.
• Accessibility : Information should be easily accessible and available to researchers through credible sources such as peer-reviewed articles, scholarly publications, and academic databases.
• Coherence : Information should be logically organized and presented in a manner that is easy to understand and interpret.

Advantages of Information in Research

Information plays a crucial role in research, providing numerous advantages, including:

• Building knowledge : Information provides a foundation for developing hypotheses, designing studies, and advancing knowledge in a given field.
• Supporting decision-making: Information helps researchers make informed decisions about research design, sampling, data collection, and analysis.
• Saving time and resources: Access to existing information can save time and resources by providing a basis for future research and avoiding duplication of efforts.
• Providing evidence: Information can provide evidence to support or refute theories and hypotheses, allowing researchers to draw sound conclusions and make valid interpretations.
• Enhancing credibility: Using high-quality and reliable information in research enhances the credibility of the study, increasing the likelihood of acceptance and dissemination of findings.
• Facilitating collaboration : Access to information enables researchers to collaborate with others in their field, increasing the potential for interdisciplinary research and fostering innovation.
• Identifying gaps: Information can help researchers identify gaps in existing knowledge and inform future research directions.

Limitations of Information in Research

While information is crucial to research, there are several limitations that researchers need to be aware of, including:

• Inaccuracy : Information can be inaccurate or contain errors, leading to flawed conclusions and misleading interpretations.
• Bias : Information can be biased, influenced by personal opinions, values, or interests, affecting the validity and reliability of research findings.
• Accessibility : Some information may not be easily accessible, limiting the scope and depth of research.
• Outdated information: Some information may become outdated or irrelevant over time, reducing its usefulness in research.
• Quality : Information may vary in quality and credibility, with some sources providing higher-quality information than others.
• Lack of context : Information may lack context or fail to account for the complexity of a given phenomenon, leading to oversimplification or incomplete understanding.
• Interpretation : Information requires interpretation, and researchers may differ in their interpretations, leading to conflicting findings and conclusions.
• Overreliance : Overreliance on a single source of information or type of information may limit the scope and depth of research and lead to bias.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Primary Data – Types, Methods and Examples

Qualitative Data – Types, Methods and Examples

Research Data – Types Methods and Examples

Quantitative Data – Types, Methods and Examples

Secondary Data – Types, Methods and Examples

• My Library Account
• Articles, Books & More
• Course Reserves
• Site Search
• Advanced Search
• Sac State Library
• Research Guides

Step by Step: Details of the Research Process

• Gather Information
• Select Topic
• Develop Search Strategy
• Evaluating Information
• Citing Sources

Gathering Information

Once you have a list of resources that you want to use for your research, it is time to get a hold of them so that you can actually get to the real work of reading, understanding and finally writing. This is another chance to determine if there is enough authoritative information on your topic.

It will actually save you time and decrease your stress level if you don't wait until the last minute to locate the books and articles you want to use, because if we don't have them, you can request them for free. Learn more about borrowing from InterLibrary Loan .

As you locate your books and articles, make sure you note all the information that you will need to create a bibliography for your paper. It is very easy to end up with content from a really useful article/chapter/webpage, but not have all the pieces necessary to create a reference. If you are unsure, ask your professor what format you need to follow and then go to the Citing Sources tab above to learn more about how to cite in a particular style.

Find Articles Using the Library Databases

Use one of the Library Databases to search for articles. The slide deck will walk you through conducting a search in one of the Social Work databases. Use the Social Work Research Guide for suggested databases.

Find Books Using OneSearch

• Finding Books in the Library (call numbers)
• Library Floor Maps
• CSUS Thesis Project or Dissertation

Text Version of Research Flowchart

• Do Background Reading
• Narrow/Broaden Search
• Create Thesis Statement
• Create list of keywords
• List questions to be answered
• Write a draft outline
• << Previous: Develop Search Strategy
• Next: Evaluating Information >>
• Last Updated: Jan 24, 2024 11:43 AM
• URL: https://csus.libguides.com/StepbyStep

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Gather Information

After locating sources that are relevant and have a strong credibility score, begin gathering information from the chosen sources. Probably the first step is to review the table of contents for the books, and note any chapters that may be especially useful, especially if the book is a edited compilation of material from many authors.

Review all the chosen sources and organize them. Researchers often review the introduction or the first few paragraphs of each source to determine what information the material will provide. Undergraduate students may want to read the tertiary and secondary sources first to get a solid understanding of the topic before delving into the professional literature. Or, researchers may want to organize the sources for different content sections of the research project. Decide the order that makes the best use of the resources to understand the topic. Remember that reading many of the advanced sources will require reading or reviewing several times.

Regardless of the order the researcher begins gathering information from the sources, it is important to keep the material organized. Begin by noting the source citation, and for each piece of information (whether quoted, paraphrased or summarized), provide location of that specific data. A print source should have a page number, while electronic formats have some type of locating tool to use. After compiling the information from that source, move to the next source of information. The notes can be handwritten, compiled in a word processing program, or other editable format, but make sure to gain material from each source.

Bridging the Gap: A Guide to College-Level Research Copyright © 2021 by Catherine J Gray is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License , except where otherwise noted.

Share This Book

Top tips for presenting your data according to research

What do we really know about how to present complex data in ways that are easy to understand and have impacts that might help address complex issues such as climate change? Dr Lucy Richardson explores some of the useful tips provided by data visualisation and communication research that can help you effectively communicate complex information.

This article is part of the ISC’s Transform21 series, which features resources from our network of scientists and change-makers to help inform the urgent transformations needed to achieve climate and biodiversity goals.

Over the last year or so, many people across the world have become used to seeing charts and graphs with COVID-19 statistics in their news feeds, but all charts are not created equal when it comes to effectively communicating a key message.

Researchers have been examining how different aspects of data presentation influence audiences for many years. They have looked at the issue from diverse angles such as which components are viewed in what order and why, and whether text, graphs or maps are more engaging and easily understood. These diverse research questions have been addressed using a wide variety of methods ranging from tracking audience eye movements to surveys and social media polls. From this collection of research, we have gained valuable insights that can help make data visuals more effective communication tools.

A useful framework to think about when designing data visualisations follows the broad process of audience interaction with the presented information: (a) first the audience perceives the information (b) then they think about the information, and (c) then some sort of change or impact occurs due to those thoughts.

Perceiving the information (Perception)

Assuming that your data visualisation is presented to your target audience in a time and place where they are likely to see it, your audience needs to be able to perceive and differentiate each of the key components of your visualisation in order to discern its meaning.

Perception tends to happen in sequence, following a visual hierarchy of attention based on the following characteristics of any object (including maps and graphs): size, colour, contrast, alignment, repetition, proximity, whitespace, and texture and styles. Within each of these elements are further sub-hierarchies. For example, people tend to notice large elements before smaller ones, and bright colours before muted ones. Similarly, dramatic contrasting components are noticed more than those with less contrast.

The effect of these hierarchical elements can be impacted by perception challenges and should be carefully considered to ensure that they promote your message rather than confusing or distracting your audience. There are a range of different perception challenges that can impact on the effectiveness of data visualisations, but did you know there are actually seven different forms of colour blindness ? You can even run your data visualisation through a colour blindness simulator to see how it might be viewed by someone with these challenges.

Thinking about the information (Cognition)

When your audience thinks about and derives meaning from information they perceive, this is known as cognitive processing. It includes thinking, knowing, remembering, judging, and problem-solving; any number of which may be used when processing information associated with visualised data.

Some things you can do to help encourage the desired interpretation of meaning from your data visualisation include providing chart titles that are the main message rather than just a description of the content. A title such as ‘Higher amounts of green vegetation in cities is associated with lower summer temperatures’ is much more effective at guiding meaning-making than titling the same chart as ‘Green vegetation and temperature in Australian cities’.

Some topic areas that may require data visualisations can also have underlying psycho-social (psychological, social and/or political) factors that should be considered. This is particularly the case for climate change, a heavily politicised issue that is quite polarising in some countries. When presenting data relating to climate change, some valuable tips include:

• Avoid catastrophic messaging that can cause people to shut down as a coping response to their fear.
• Include solutions-based information can help counteract fear by promoting a sense that climate change can be addressed.
• Provide locally relevant information where possible, as this will resonate more strongly. People are naturally most interested in what happens in their local area.
• Where possible, consider if there are other ways to cover the issue without mentioning ‘climate change’ if communicating to audiences who may not accept current scientific evidence of its existence and urgency. This is easier for messages relating to adapting to changes in climate than mitigation, as there are often diverse benefits beyond climate change that can be used to frame adaptation information.

It’s also important to recognize that people are generally more likely to remember meaning than detail. This means that people are more likely to remember a trend—such as it’s getting ‘worse’ or ‘better’, ‘increasing’ or ‘decreasing’—but may not remember the specific amount or rate of that increase or decrease.

Changes effected (Impact)

There are a range of possible impacts that might arise from audiences viewing your data visualisation. These could be changes in thought (for example, awareness, understanding, attitudes or concern), or changes in behaviour (for example, information seeking, discussion with others, or even adoption of climate-friendly behaviours). The likelihood of change being effected due to your data visualisation will be enhanced by ensuring your messages are clear and relevant, where clarity will come from effectively addressing perception and cognition considerations and relevance will come from appropriate message framing and consideration of psycho-social factors. Knowing the kind of change you want to achieve will be critical in determining how best to integrate these various factors into your work.

Alternative formats

While most people wishing to present complex scientific data tend to think of charts, graphs, maps, and infographics, it is also possible to present information for perception by other senses such as through sound. Some researchers have been testing data sonification as an alternative to visual data representation. Sonification takes each data point and applies a mix of sound elements that can allow trends to be distinguished—for example, pitch, volume, and choice of instrument—to provide an audio representation of the information. NASA has done this so that people can ‘listen’ to the Milky Way Galaxy , and researchers at the Monash University Climate Change Communication Research Hub have sonified cyclone Debbie ’s movements around Australia in 2017.

A free best practice guide has been developed based on a review of data visualisation research. Hopefully, it will help you decide how you can best present your data for effective perception, cognition and impact. You can access the Best practice data visualisation: Guidelines and case study on the Monash Climate Change Communication Research Hub website .

Lucy Richardson

Dr Lucy Richardson is based at the Monash Climate Change Communication Research Hub, Monash University, on the lands of the Kulin Nations, Melbourne, Australia, and a member of the  Commonwealth Futures Climate Research Cohort  established by The Association of Commonwealth Universities and the British Council to support 26 rising-star researchers to bring local knowledge to a global stage in the lead-up to COP26.

The header image was created by NASA’s Scientific Visualization Studio to support a series of talks from NASA scientists for COP26. It is a still from a video that shows the atmosphere in three dimensions and highlights the accumulation of CO 2  during a single calendar year. You can watch the visualisation and find out more about the data on which it’s based here .

Related Items

The isc’s centre for science futures secures over one million dollars grant to explore the impacts of ai on science systems in the global south .

New report on future of wood supply highlights importance of innovation

1 in every 10 women in the world lives in extreme poverty

2024 World Data System Scientific Committee (WDS-SC) Nominations are Now Open

Pacific Islands Academy of Sciences and Humanities: A Pivotal Step Towards a Resilient Future

Spacecrafts burning in the upper atmosphere: what consequences on climate?

Establishment of the ISC expert group on plastic pollution

World Climate Research Programme launches a Lighthouse Activity on Climate Intervention Research

Kigali Declaration: Climate science for a sustainable future for all

Climate inequality: The stark realities and the road to equitable solutions

Biodiversity data is distorted by past inequities. Scientists are wrestling to get a clearer picture.

Early-career researchers' insights on climate

Science reading list from the International Science Council

Readers' choice: our most shared science blogs this year

Seizing the momentum: a year of climate science advocacy

Large Ocean States and climate change: a plea for adaptation

COP28 agreement: embracing the urgency of the scientific consensus?

Fostering tomorrow’s science: the ISC's engagements with Early and Mid-Career Researchers in 2023

"What's holding us back?": how economists and social scientists might hold the key to climate action

Podcast with Cory Doctorow: Science Fiction and the Future of Science: Leveraging Digital Advancements for the Future

"Beyond weathering the storm": rethinking climate resilience

For science-based decision-making on the climate emergency: 10 new insights in climate science

COP28: The International Science Council's engagement in climate science

Scientific communication: Join the Ethnografilm festival for a filmmaking experience in Paris

publications

Policy brief: global sea-level rise, the costs of shifting scenarios: why the ipcc should maintain consistent vocabulary in climate assessments.

New ISC policy brief: A call for a formal scientific voice in the global fight against plastic pollution

Converging and interdependent crises are amplifying the impacts of one another with often devastating consequences

Reframing Trust in Science for Multilateral Policy: Insights from the Science Journalists Forum

One Planet Polar Summit, scientists striving to bridge the science-policy gap for urgent action: "every tenth of a degree Celsius matters"

Science Fiction and the Future of Science

Call for nomination of experts to draft an outline of the IPCC Special Report on Climate Change and Cities - deadline 15 November

Kigali declaration pledges to bridge climate injustice

The human dimension of disaster risk reduction: social sciences and climate adaptation 

From monsoon joy to fear: a climate crisis awakening

Global solidarity for climate justice: perspectives from an early-career researcher

One world, one climate: a planetary call to action

Transdisciplinary research for sustainability solutions in urban Africa

Cultivating a proactive approach to crises: first meeting of the UNEP/ISC foresight expert panel

Clearing the air: is ammonia our key to cleaner skies?

Unearthing sustainability: soil science for the SDGs

Helping newsrooms get disaster risk ready, an International Science Council and World Editors Forum collaboration

Our Future Depends on Us: Antarctic climate change and the environment

Unveiling the health benefits of forests and trees

How do we talk about science and uncertainty?

From Authoritarian Threats to Funding Disparities: Key Challenges in Global Science

Frontiers Planet Prize, second edition: celebrating the world’s most innovative sustainability scientists by 1 November

Protecting the ocean: 5 essential reads on invasive species, overfishing and other threats to sea life

The Frontiers Planet Prize unveils its champions

ISC's nomination track ensures transdisciplinary and global representation in UNEP's Scientific Advisory Group for GEO-7 Assessment

UN 2023 Water Conference carries new engagements towards realizing SDG6 and future avenues for a decade of action

IPCC report: the world must cut emissions and urgently adapt to the new climate realities

ISC-BBC StoryWorks partnership ends on a high note, delivering some of the highest engagement for the BBC

Youth perspectives on climate ‘COP’ negotiations and ways to get involved 

Contribute a blog or news item to the ISC

Public participation in scientific programmes: Citizen science for biodiversity  

The ISC and UNEP to cooperate on advancing the use of science in environmental policy and decision-making

Science in Times of Crisis Episode 3 - The Fallout of Conflict: The Arctic and Outer Space

Innovative approaches to storytelling in science celebrated at Women in Science Film Festival in Cape Town

Ten key messages for the Convention on Biological Diversity

COP27 ends with commitment on financial support

There are 8 years left to meet the UN Sustainable Development Goals, but is it enough time?

Special interview series on COP 27- Interview with Nick Perkins about climate change and science communication

People on the front lines of climate change must be included in climate action

Are we in a new era of climate adaptation implementation? The role of regional governments in facilitating local action

In the face of climate threats, we cannot afford not to act

The Role of Integrated Science in Understanding the Earth-Human System

In the face of extreme weather events, coordinated global action to address climate change is needed at COP27

Perspectives from the ISC network on expectations for the COP

Professor Carlos Lopes on why Africa needs to stick to renewables despite the temptation of gas

How can scientists make a difference at COP27?

Call for contributions for the Sustainability Research & Innovation Congress (SRI 2023)

Frontiers Planet Prize: Science for a Sustainable Planet

The scientific community can support the call for decisive action at cop27.

Stories of transformations to sustainability

Happy birthday to the Montreal Protocol – the most successful environmental treaty of all time?

Risk of passing multiple climate tipping points escalates above 1.5°C global warming

World Youth Skills Day 2022: From Resilience to Fearlessness 

WorldFAIR: Global cooperation on FAIR data policy and practice – Kick-Off Meeting introduces major new initiative to advance implementation of the FAIR data principles

Six Takeaways on Science Communication from our Talk Back Better Webinar Series

What's on the horizon for scientific data services? The latest from the World Data System

Implementing FAIR data principles – what’s behind the acronym?

WorldFAIR: Global cooperation on FAIR data policy and practice

A letter to our fellow citizens of Earth

The Biggest Carbon Sink of All

Policy Brief: Harnessing data to accelerate the transition from disaster response to recovery

CODATA and ISC celebrate Metrology in the Digital Era on World Metrology Day

Managed retreat from areas threatened by floods can catalyse positive social transformations

‘Now or never’ to limit warming to 1.5°C, according to latest IPCC report

How Science Affects my Everyday Life as a Fourteen-Year-Old

Joint Statement of Intent on the Digital Transformation in the International Scientific and Quality Infrastructure

The African Open Science Platform begins to take shape

Video tutorials on science ethics and science communication

Women Leading on Equitable and Inclusive Solutions to address the Climate Emergency: Webinar

Principles and Structures of Science Advice: An outline

The window for climate action to avoid dangerous systemic risks is narrowing, warns latest IPCC report

The transformative potential of managed retreat in the face of rising sea levels

The seven warmest years on record were the last seven

The Paris Agreement is working as intended, but we’ve still got a long way to go

An early career perspective on the science-policy interface in the decade of climate action following COP26

Holiday binge-watching: Science edition

Our most popular stories from 2021

Global Risks Perceptions Report 2021 released

Staying below 1.5°C: what are the chances?

Picturing the future of complex, cascading climate risks

If universities want to hit climate targets, they should use their land for carbon offsetting

Emerging Climate Risks and what will it take to limit global warming to 2.0°C?

What Antarctica can teach us about global climate change

How to teach energy transition and climate in business schools

Convening the scientific knowledge required to boost climate action

Major scientific assessment of the Amazon region issues urgent call to end deforestation and avert tipping points

Urban Health and Wellbeing in the Anthropocene

Compelling stories, curious science: #UnlockingScience launched

Building resilience in a climate challenged world.

Climate change projections for Pakistan: the need for sustainable solutions to protect its people and biodiversity

Four considerations for accelerating progress on climate change at the science-policy interface

Ten New Insights in Climate Science 2021 report highlights critical research and policy implications for addressing the climate crisis

Ahead of COP26, Ekanem I. Braide shares her perspective on the priorities for action and the role of science

Climate risk assessment gaps: seamless integration of weather and climate information for community resilience

Increasing the participation of women in the climate change debate, including as leaders, is essential for a carbon-zero future

Deepening interactions between science and policy on the way to COP26: What role for science publishers?

Predicting the climate of the next decade

Global citizens and scientists on how to achieve a thriving net zero future

Big Earth Data Advances Science and Engineering for SDGs

Call for emergency action to limit global temperature increases, restore biodiversity, and protect health

Professor Karen O'Brien: People are the most powerful solution to climate change

Climate justice and the decarbonization of shipping

A call for reconceptualizing carbon pricing policies with both eyes open

Coastal communities in the Arctic rely on structural measures to adapt to climate change, but should they?

Enabling positive tipping points towards global sustainability in uncertain times

Solar Geoengineering at a Standstill?

Scientific advances underpinning latest IPCC report demonstrate need for rapid action

Deep and sustained emissions reductions required to head off rapid climate change affecting all regions of the world

The carbon skyscraper

What we are reading

The mental health burden of climate change is growing – now it’s time to act

Climate finance - a sticking point for the COP26?

Call for papers: Discussion meeting on statistical aspects of climate change

World Youth Skills Day 2021: resilience and creativity

Tell me a story – why climate change communication needs to embrace our childlike curiosity

Climate change solutions in focus ahead of COP26

Four insights on collaborating at scale to advance climate adaptation

We're in the midst of a global wake-up call

A Global Survey of Science Offers Hope and Challenging Lessons

Can zero emissions and economic growth go together? Yes, but conditions apply

Climate explained: why is the Arctic warming faster than other parts of the world?

Why water-driven migration and displacement must be part of the climate agenda

COP26 Climate Action Champion Nigel Topping on creating an 'ambition loop' for bolder pathways to change

Climate scientists: concept of net zero is a dangerous trap

Building a climate future requires a regional approach

Designing responsible policy pathways for a zero-carbon transition

Earth Day 2021: we need bold, creative, innovative solutions

Strengthening the links between science and society for action on climate change in France

Target high-carbon emitters to accelerate green transition, say leading experts on behavioural change

What would a 3°C warmer world mean for Australia?

Citizen scientists: perhaps without a degree but certainly making a difference

The new climate change activism is emotional, and it’s a good thing

ISC Science Communications Network

Top ten insights in climate science from the past year

Mary Robinson - No Time to Spare for the Paris Climate Promise

Global change research and the COVID-19 pandemic

Children’s comic book series introduces solar-terrestrial physics

Working together: Future Earth and WCRP announce partnership to jointly address major societal challenges

Global Carbon Budget 2020 Finds Record Drop in Emissions

What are the climate breakthroughs in 2020?

Taking the temperature of the Paris Agreement: perspectives from our community

The “How” of transformation

Project Syndicate talks with Mary Robinson on climate change and her new podcast

Taking stock of progress on global change: what to expect from the unep global assessments synthesis report.

Ozone hole over Antarctica ‘largest’ and ‘deepest’ it has been in years

Redefining business as usual for scientific publishing

Learning from COVID-19 and building more resilient food systems

Global Science TV: Arctic ice keeps shrinking. Here’s what that means for all of us

The COVID-19 Pandemic Illustrates the Need for Open Science

World’s first high-level science initiative dedicated to the survival of the Amazon

Statistical thinking as an essential skill for reading the news

Global Science TV: Why can't we deal with climate change as urgently as COVID-19?

Why can’t we deal with climate change as urgently as COVID-19?

Earth day 50th anniversary calls for climate action.

A Data Ecosystem to Defeat COVID-19

Making data work for cross-domain grand challenges

Tackling Climate Change with COVID-19 Urgency - by ISC Patron, Mary Robinson and ISC President, Daya Reddy

Four major international data organizations join forces to optimize the research data ecosystem, launching a COVID-19 appeal as their first joint action

Call for Expressions of Interest to Host the World Data System International Programme Office (Partial Submissions Allowed)

Join the first ‘Transformation Talks’ webinar to explore how research communication can be transformative

TROP ICSU - Climate Change Education Across the Globe

A rapidly-evolving new normal: Pep Canadell comments on Australia’s Fires

ISC joins WCRP in celebrating its 40th year of international climate science

COP25: Time for action

Call for nominations of experts to serve on the Editorial Board of the IPCC Emission Factor Database

Why we need a UN charter

World Data System Data Stewardship Award 2019

Achieving Risk Reduction Across Sendai, Paris And The SDGs

Disaster Loss Data In Monitoring The Implementation Of The Sendai Framework

The political challenge of achieving transformations to 1.5ºC – the role of social justice

A vision for the African Open Science Platform

COP24 side event on The CitiesIPCC Research and Action Agenda for effective urban responses to climate change

International data week gets underway in Gaborone, Botswana

Transforming southern African cities in a changing climate – Q and A with Alice McClure from the University of Cape Town

Review of the World Climate Research Programme (WCRP)

Ten essentials for research that responds to the climate challenge

Vacancy: executive director of the icsu world data system (wds) (re-advertised).

Vacancy: Communications Intern for the LIRA 2030 programme

News from LIRA2030: Seedbeds of Transformation conference, South Africa

World data system workshop held in rio de janeiro.

ICSU World Data System International Technology Office to open in Canada

Pavel Kabat appointed WMO Chief Scientist and Research Director

Why the IPCC's upcoming 1.5°C report offers an unexpected glimpse of hope

The IPCC at 30: Is the 1.5°C Special Report a turning point?

The origins of the IPCC: How the world woke up to climate change

The state of biodiversity in the regions: What to expect from the IPBES in 2018

Highlights of 2017

Why 2018 is a big year for global environmental assessments

Heide Hackmann recieves award for science diplomacy

IAMAS urges United States to continue support of Earth Observation systems

Cop23 side event on climate change- when and where will habitability limits be reached.

Largest Ever Science Gathering in the Middle East for World Science Forum 2017

Belmont Forum announces Mustapha Mokrane as new Co-Lead of Open Data Initiative

Future of science: voices from our partners.

Future of science: Voices from our partners

Call for scientific and non-academic reviewers for lira 2030, early career scientists gather for lira trans-disciplinary workshop in uganda.

Focus on Interactions: Second Nexus Conference Announced for 2018

International Council for Science calls on United States to support international efforts to combat dangerous climate change

ICSU President to receive International Meterological Prize from WMO

ICSU at the U.N. Ocean Conference

Science Plan on Global Environmental Change – ICSU Regional Office for Africa

Committee on data (codata).

World Data System (WDS)

Climate Change

Future Earth

"Open Data in a Big Data World" accord passes 120 endorsements

World Climate Research Programme (WCRP)

NZ Government thanks IRDR and CODATA groups for their help following 2016 Kaikoura earthquake

Making a case for science at the United Nations

ICSU Unions receive award to launch multi-year initiatives in science outreach and education

New commentary published: climate research must sharpen its view, african open science platform to boost the impact of open data for science and society, icsu co-organizes side event at cop22 on urgent questions in climate research.

Future Earth-PROVIA-IPCC risks and solutions workshop: livestream available

Call for nominations - early career scientists at habitat iii, open data in a big data world, world data system marks fifth anniversary of international programme office, advisory note: science communication (2010/2016), leading science groups urge global accord on open data in a big data world, 30 years pioneering collaboration on global change research: igbp closes down dec 2015, the international council for science and climate change.

Twelve things we've learned on the Road to Paris

Science International to agree international accord on open data

New scientific committee and chair appointed for icsu world data system, landmark scientifc data conference ends with strong support of data sharing for sustainability, future earth 2025 vision.

Future Earth Strategic Research Agenda 2014

Open access to scientific data and literature and the assessment of research by metrics

Icsu-endorsed initiative sustainable deltas 2015 launches in rotterdam.

International Council for Science endorses open access to scientific record; cautions against misuse of metrics

Celebrating 30 years of global change research, call for proposals: transformations to sustainability.

Review of CODATA, the Committee on Data for Science and Technology

Future earth initial design report: executive summary.

Annual global carbon emissions set to reach record 36 billion tonnes in 2013

World social science report 2013: changing global environments.

Scientists meet at the UN for Expert Group Meeting on Sustainable Development Goals

Black carbon report from igbp project generates significant media coverage, ad-hoc strategic coordinating committee on information and data (sccid report), how to describe nanomaterials – an icsu workshop in paris, the international council for science pledges support for scientists in the l’aquila case, icsu-unesco regional science and technology workshops for rio+20, climate research in the next decade of earth system research, icsu's new world data system opens new international progamme office in tokyo, international programme office of icsu’s new world data system opened, icsu foresight analysis peer-reviewed, workshop on the description of nanomaterials, advisory note on access to shared data to reduce global inequality, rio+20 policy briefs released by the gec programmes, advisory note on sharing scientific data, with a focus on developing countries, icsu releases statement on the controversy around the 4th ipcc assessment, polar year comes to a close, a vision for earth system research: have your say, polar research reveals new evidence of global environmental change, upcoming release of new evidence about change in the polar regions, review of the international geosphere-biosphere programme, review of the world climate research programme (2009), ipy polar day focusing above the polar regions, international science community agrees on first steps to establish a global virtual library for scientific data, icsu launches new programme to understand the human impact on earth’s life-support systems, international council for science (icsu) launches major research programme on natural disasters, ipy polar day focusing on people, ipy polar land and life day, report from the ad hoc strategic committee on information and data, ipy day focusing on changing earth, ipy day focusing on ice sheets, ipy presents sea ice day, global launch of international polar year (ipy) 2007-2008, co2 rise heightens concern over vulnerability of polar regions, icsu hosts conference on hazards and disasters, icsu and climate science (2006), at pivotal event in china, the international council for science releases new strategy to strengthen international science for the benefit of society, international experts call for new approach to ensure challenges to data access and management don’t slow scientific progress, icsu pursues new initiative that challenges science to do more to prevent natural disasters, socioeconomic data in relation to the integrated global observing strategy partnership igos-p (2004), priority area assessment on scientific data and information, cern announces major conference on the information society, icsu launches an agenda for action in advance of the world summit on the information society, science in the information society: policy issues for scientific information (2003), science in the information society: optimizing knowledge (2003), science in the information society: decision making and governance (2003), science in the information society: universal access to scientific knowledge (2003), icsu/codata launch online forum for world summit on the information society.

VIEW ALL RELATED ITEMS

https://council.science/current/blog/tips-for-presenting-your-data-research/">

• Disclaimer: The translations are automatically generated by Google Translate and may contain errors. The ISC is not liable for any damage or issues that may arise from these translations. You can provide your feedback by emailing us at [email protected] Note: Please note that the ‘Science Summit at the UN General Assembly’ in September 2023 is not an event of the International Science Council. The International Science Council has no association with this event nor with the organizer of the event.

Privacy Overview

• school Campus Bookshelves
• menu_book Bookshelves
• perm_media Learning Objects
• login Login
• how_to_reg Request Instructor Account
• hub Instructor Commons
• Download Page (PDF)
• Download Full Book (PDF)
• Periodic Table
• Physics Constants
• Scientific Calculator
• Reference & Cite
• Tools expand_more
• Readability

selected template will load here

This action is not available.

14.3: Strategies for Gathering Reliable Information (Part 2)

• Last updated
• Save as PDF
• Page ID 6313

• Amber Kinonen, Jennifer McCann, Todd McCann, & Erica Mead
• Bay College Library

Using Other Information Sources: Interviews

With so many print and electronic media readily available, it is easy to overlook another valuable information resource: other people. Consider whether you could use a person or group as a primary source. For instance, you might interview a professor who has expertise in a particular subject, a worker within a particular industry, or a representative from a political organization. Interviews can be a great way to get firsthand information.

To get the most out of an interview, you will need to plan ahead. Contact your subject early in the research process and explain your purpose for requesting an interview. Prepare detailed questions. Open-ended questions, rather than questions with simple yes-or-no answers, are more likely to lead to an in-depth discussion. Schedule a time to meet, and be sure to obtain your subject’s permission to record the interview. Take careful notes and be ready to ask follow-up questions based on what you learn.

If scheduling an in-person meeting is difficult, consider arranging a telephone interview or asking your subject to respond to your questions via e-mail. Recognize that any of these formats takes time and effort. Be prompt and courteous, avoid going over the allotted interview time, and be flexible if your subject needs to reschedule.

Evaluating Research Resources

As you gather sources, you will need to examine them with a critical eye. Smart researchers continually ask themselves two questions: “Is this source relevant to my purpose?” and “Is this source reliable?” The first question will help you avoid wasting valuable time reading sources that stray too far from your specific topic and research questions. The second question will help you find accurate, trustworthy sources.

Determining Whether a Source Is Relevant

At this point in your research process, you may have identified dozens of potential sources. It is easy for writers to get so caught up in checking out books and printing out articles that they forget to ask themselves how they will use these resources in their research. Now is a good time to get a little ruthless. Reading and taking notes takes time and energy, so you will want to focus on the most relevant sources.

To weed through your stack of books and articles, skim their contents. Read quickly with your research questions and subtopics in mind. The table below explains how to skim to get a quick sense of what topics are covered. If a book or article is not especially relevant, put it aside. You can always come back to it later if you need to.

Determining Whether a Source Is Reliable

All information sources are not created equal. Sources can vary greatly in terms of how carefully they are researched, written, edited, and reviewed for accuracy. Common sense will help you identify obviously questionable sources, such as tabloids that feature tales of alien abductions, or personal websites with glaring typos. Sometimes, however, a source’s reliability—or lack of it—is not so obvious.

To evaluate your research sources, you will use critical thinking skills consciously and deliberately. You will consider criteria such as the type of source, its intended purpose and audience, the author’s (or authors’) qualifications, the publication’s reputation, any indications of bias or hidden agendas, how current the source is, and the overall quality of the writing, thinking, and design.

Evaluating Types of Sources

The different types of sources you will consult are written for distinct purposes and with different audiences in mind. This accounts for other differences, such as the following:

• How thoroughly the writers cover a given topic
• How carefully the writers research and document facts
• How editors review the work
• What biases or agendas affect the content

A journal article written for an academic audience for the purpose of expanding scholarship in a given field will take an approach quite different from a magazine feature written to inform a general audience. Textbooks, hard news articles, and websites approach a subject from different angles as well. To some extent, the type of source provides clues about its overall depth and reliability. The table below ranks different source types.

Free online encyclopedias and wikis may seem like a great source of information. They usually appear among the first few results of a web search. They cover thousands of topics, and many articles use an informal, straightforward writing style. Unfortunately, these sites have no control system for researching, writing, and reviewing articles. Instead, they rely on a community of users to police themselves. At best, these sites can be a starting point for finding other, more trustworthy sources. Never use them as final sources.

Evaluating Credibility and Reputability

Even when you are using a type of source that is generally reliable, you will still need to evaluate the author’s credibility and the publication itself on an individual basis. To examine the author’s credibility—that is, how much you can believe of what the author has to say—examine his or her credentials. What career experience or academic study shows that the author has the expertise to write about this topic?

Keep in mind that expertise in one field is no guarantee of expertise in another, unrelated area. For instance, an author may have an advanced degree in physiology, but this credential is not a valid qualification for writing about psychology. Check credentials carefully.

Just as important as the author’s credibility is the publication’s overall reputability. Reputability refers to a source’s standing and reputation as a respectable, reliable source of information. An established and well-known newspaper, such as the New York Times or the Wall Street Journal, is more reputable than a college newspaper put out by comparatively inexperienced students. A website that is maintained by a well-known, respected organization and regularly updated is more reputable than one created by an unknown author or group.

If you are using articles from scholarly journals, you can check databases that keep count of how many times each article has been cited in other articles. This can be a rough indication of the article’s quality or, at the very least, of its influence and reputation among other scholars.

Checking for Biases and Hidden Agendas

Whenever you consult a source, always think carefully about the author’s or authors’ purpose in presenting the information. Few sources present facts completely objectively. In some cases, the source’s content and tone are significantly influenced by biases or hidden agendas.

Bias refers to favoritism or prejudice toward a particular person or group. For instance, an author may be biased against a certain political party and present information in a way that subtly—or not so subtly— makes that organization look bad. Bias can lead an author to present facts selectively, edit quotations to misrepresent someone’s words, and distort information.

Hidden agendas are goals that are not immediately obvious but influence how an author presents the facts. For instance, an article about the role of beef in a healthy diet would be questionable if it were written by a representative of the beef industry—or by the president of an animal-rights organization. In both cases, the author would likely have a hidden agenda.

Using Current Sources

Be sure to seek out sources that are current, or up to date. Depending on the topic, sources may become outdated relatively soon after publication, or they may remain useful for years. For instance, online social networking sites have evolved rapidly over the past few years. An article published in 2002 about this topic will not provide current information. On the other hand, a research paper on elementary education practices might refer to studies published decades ago by influential child psychologists.

When using websites for research, check to see when the site was last updated. Many sites publish this information on the homepage, and some, such as news sites, are updated daily or weekly. Many nonfunctioning links are a sign that a website is not regularly updated. Do not be afraid to ask your professor for suggestions if you find that many of your most relevant sources are not especially reliable—or that the most reliable sources are not relevant.

Evaluating Overall Quality by Asking Questions

When you evaluate a source, you will consider the criteria previously discussed as well as your overall impressions of its quality. Read carefully, and notice how well the author presents and supports his or her statements. Stay actively engaged—do not simply accept an author’s words as truth. Ask questions to determine each source’s value. The checklist below lists ten questions to ask yourself as a critical reader.

Source Evaluation Checklist

• Is the type of source appropriate for my purpose? Is it a high-quality source or one that needs to be looked at more critically?
• Can I establish that the author is credible and the publication is reputable?
• Does the author support ideas with specific facts and details that are carefully documented? Is the source of the author’s information clear? (When you use secondary sources, look for sources that are not too removed from primary research.)
• Does the source include any factual errors or instances of faulty logic?
• Does the author leave out any information that I would expect to see in a discussion of this topic?
• Do the author’s conclusions logically follow from the evidence that is presented? Can I see how the author got from one point to another?
• Is the writing clear and organized, and is it free from errors, clichés, and empty buzzwords? Is the tone objective, balanced, and reasonable? (Be on the lookout for extreme, emotionally charged language.)
• Are there any obvious biases or agendas? Based on what I know about the author, are there likely to be any hidden agendas?
• Are graphics informative, useful, and easy to understand? Are websites organized, easy to navigate, and free of clutter like flashing ads and unnecessary sound effects?
• Is the source contradicted by information found in other sources? (If so, it is possible that your sources are presenting similar information but taking different perspectives, which requires you to think carefully about which sources you find more convincing and why. Be suspicious, however, of any source that presents facts that you cannot confirm elsewhere.)

Writing at Work

The critical thinking skills you use to evaluate research sources as a student are equally valuable when you conduct research on the job. If you follow certain periodicals or websites, you have probably identified publications that consistently provide reliable information. Reading blogs and online discussion groups is a great way to identify new trends and hot topics in a particular field, but these sources should not be used for substantial research.

Use a search engine to conduct a web search on your topic. Refer to the tips provided earlier to help you streamline your search. Evaluate your search results critically based on the criteria you have learned. Identify and bookmark one or more websites that are reliable, reputable, and likely to be useful in your research.

Managing Source Information

As you determine which sources you will rely on most, it is important to establish a system for keeping track of your sources and taking notes. There are several ways to go about it, and no one system is necessarily superior. What matters is that you keep materials in order; record bibliographical information you will need later; and take detailed, organized notes.

Keeping Track of Your Sources

Think ahead to a moment a few weeks from now, when you’ve written your research paper and are almost ready to submit it for a grade. There is just one task left—writing your list of sources.

As you begin typing your list, you realize you need to include the publication information for a book you cited frequently. Unfortunately, you already returned it to the library several days ago. You do not remember the URLs for some of the websites you used or the dates you accessed them— information that also must be included in your bibliography. With a sinking feeling, you realize that finding this information and preparing your bibliography will require hours of work.

This stressful scenario can be avoided. Taking time to organize source information now will ensure that you are not scrambling to find it at the last minute. Throughout your research, record bibliographical information for each source as soon as you begin using it. You may use pen-and-paper methods, such as a notebook or note cards, or maintain an electronic list. (If you prefer the latter option, many office software packages include separate programs for recording bibliographic information.)

The table below shows the specific details you should record for commonly used source types. Use these details to develop a working bibliography—a preliminary list of sources that you will later use to develop the references section of your paper. You may wish to record information using the formatting system of the American Psychological Association (APA) or the Modern Language Association (MLA), which will save a step later on.

Taking Notes Efficiently

Good researchers stay focused and organized as they gather information from sources. Before you begin taking notes, take a moment to step back and think about your goal as a researcher—to find information that will help you answer your research question. When you write your paper, you will present your conclusions about the topic supported by research. That goal will determine what information you record and how you organize it.

Writers sometimes get caught up in taking extensive notes, so much so that they lose sight of how their notes relate to the questions and ideas they started out with. Remember that you do not need to write down every detail from your reading. Focus on finding and recording details that will help you answer your research questions. The following strategies will help you take notes efficiently.

Use Headings to Organize Ideas

Whether you use old-fashioned index cards or organize your notes using word-processing software, record just one major point from each source at a time, and use a heading to summarize the information covered. Keep all your notes in one file, digital or otherwise. Doing so will help you identify connections among different pieces of information. It will also help you make connections between your notes and the research questions and subtopics you identified earlier.

Know When to Summarize, Paraphrase, or Directly Quote a Source

Your notes will fall under three categories—summary notes, paraphrased information, and direct quotations from your sources. Effective researchers make choices about which type of notes is most appropriate for their purpose.

• Summary notes sum up the main ideas in a source in a few sentences or a short paragraph. A summary is considerably shorter than the original text and captures only the major ideas. Use summary notes when you do not need to record specific details but you intend to refer to broad concepts the author discusses.
• Paraphrased notes restate a fact or idea from a source using your own words and sentence structure.
• Direct quotations use the exact wording used by the original source and enclose the quoted material in quotation marks. It is a good strategy to copy direct quotations when an author expresses an idea in an especially lively or memorable way. However, do not rely exclusively on direct quotations in your note taking.

Most of your notes should be paraphrased from the original source. Paraphrasing as you take notes is usually a better strategy than copying direct quotations, because it forces you to think through the information in your source and understand it well enough to restate it. In short, it helps you stay engaged with the material instead of simply copying and pasting. Synthesizing will help you later when you begin planning and drafting your paper. Whether you directly quote, paraphrase or synthesize, you must give credit to the original source via citations (discussed in the next section of this chapter) to avoid plagiarism.

Maintain Complete, Accurate Notes

Regardless of the format used, any notes you take should include enough information to help you organize ideas and locate them instantly in the original text if you need to review them. Make sure your notes include the following elements:

• Heading summing up the main topic covered
• Author’s name, a source code, or an abbreviated source title
• Page number
• Full URL of any pages buried deep in a website

Throughout the process of taking notes, be scrupulous about making sure you have correctly attributed each idea to its source. Always include source information so you know exactly which ideas came from which sources. Use quotation marks to set off any words for phrases taken directly from the original text. If you add your own responses and ideas, make sure they are distinct from ideas you quoted or paraphrased.

Finally, make sure your notes accurately reflect the content of the original text. Make sure quoted material is copied verbatim. If you omit words from a quotation, use ellipses to show the omission and make sure the omission does not change the author’s meaning. Paraphrase ideas carefully, and check your paraphrased notes against the original text to make sure that you have restated the author’s ideas accurately in your own words.

Use a System That Works for You

There are several formats you can use to take notes. No technique is necessarily better than the others— it is more important to choose a format you are comfortable using. Choosing the format that works best for you will ensure your notes are organized, complete, and accurate. Consider implementing one of these formats when you begin taking notes:

• Use index cards . This traditional format involves writing each note on a separate index card. It takes more time than copying and pasting into an electronic document, which encourages you to be selective in choosing which ideas to record. Recording notes on separate cards makes it easy to later organize your notes according to major topics. Some writers color-code their cards to make them still more organized.
• Use note-taking software . Word-processing and office software packages often include different types of note-taking software. Although you may need to set aside some time to learn the software, this method combines the speed of typing with the same degree of organization associated with handwritten note cards.
• Maintain a research notebook . Instead of using index cards or electronic note cards, you may wish to keep a notebook or electronic folder, allotting a few pages (or one file) for each of your sources. This method makes it easy to create a separate column or section of the document where you add your responses to the information you encounter in your research.
• Annotate your sources . This method involves making handwritten notes in the margins of sources that you have printed or photocopied. If using electronic sources, you can make comments within the source document. For example, you might add comment boxes to a PDF version of an article. This method works best for experienced researchers who have already thought a great deal about the topic because it can be difficult to organize your notes later when starting your draft.

Choose one of the methods from the list to use for taking notes. Continue gathering sources and taking notes. In the next section, you will learn strategies for organizing and synthesizing the information you have found.

key takeaways

• A writer’s use of primary and secondary sources is determined by the topic and purpose of the research. Sources used may include print sources, such as books and journals; electronic sources, such as websites and articles retrieved from databases; and human sources of information, such as interviews.
• Strategies that help writers locate sources efficiently include conducting effective keyword searches, understanding how to use online catalogs and databases, using strategies to narrow web search results, and consulting reference librarians.
• Writers evaluate sources based on how relevant they are to the research question and how reliable their content is.
• Skimming sources can help writers determine their relevance efficiently.
• Writers evaluate a source’s reliability by asking questions about the type of source (including its audience and purpose); the author’s credibility, the publication’s reputability, the source’s currency, and the overall quality of the writing, research, logic, and design in the source.
• In their notes, effective writers record organized, complete, accurate information. This includes bibliographic information about each source as well as summarized, paraphrased, or quoted information from the source.

• Archives & Special Collections home
• Art Library home
• Ekstrom Library home
• Kornhauser Health Sciences Library home
• Law Library home
• Music Library home
• University of Louisville Hospital home
• Interlibrary Loan
• Off-Campus Login
• Renew Books
• Cardinal Card
• My Print Center
• Business Ops
• Cards Career Connection

Search Site

Search catalog, critical thinking and academic research: information.

• Information
• Point of View
• Assumptions
• Implications

Gather the Information

Research involves gathering and interpreting information. To answer a question or understand the complexity of an issue, you have to seek relevant information, which helps you develop your own point of view.

It's important to remember, though, that information from outside sources should not stand in for your thinking. Sometimes, people think that gathering information and summarizing it in a paper is all there is to the research process. But finding information is just part of the process.

Research involves applying critical thinking to information, whether it comes from an encyclopedia entry, a journal article, a website, or a documentary. A researcher analyzes the material and develops a perspective on it. The goal is to think critically about the information, not simply repeat its ideas.

The purpose of your research and the questions you're trying to answer will determine what information is relevant and useful. If you're trying to understand public opinion on an issue, it might be worthwhile to look at news articles and blog entries. On the other hand, such sources may not be appropriate for a formal philosophical argument or a medical study.

Sources used in academic papers might include scholarly journals, books, research reports, government documents, films, comic books, magazines, newspapers, maps, statistics, letters, diaries, dictionaries, musical recordings, and more. It all depends on your purpose.

The Complexity of the Information Universe

The information universe is very complex, so it's important to understand the differences among information sources. For instance, online information includes commercial websites, personal blogs, subscription databases, professional news sites, government resources, Wikipedia entries, Facebook profiles, Twitter feeds, YouTube videos, and much more. Different research projects require different types of sources. In many cases, you will need to look beyond the free web to find scholarly information in subscription library databases such as ProQuest Direct and EBSCO Academic Search Premier.

Understanding varying levels of complexity in information sources is also important. For example, a reference encyclopedia might provide useful background information on postmodernism, but it will not provide the level of sophistication and depth offered in an original work of postmodern theory or a scholarly article that applies that theory.

While background sources are useful and will help you understand more complex material, most professors expect you to explore in-depth, scholarly sources, most of which are not available on the free web. That's one reason why learning to use library resources is crucial.

Critical Questions

• What information do I need to address this question or understand this topic?
• How much information do I need?
• Where can I find this information?
• How do I know this information is reliable and authoritative?
• Is this information relevant to my purpose?
• Who is the audience for this information?
• What perspective does this information come from? What are its biases?
• Is the information current enough?
• " More on Evaluating Sources
• << Previous: Questions
• Next: Concepts >>
• Last Updated: Jul 10, 2023 11:50 AM
• Librarian Login

4 Beginning Your Research

Scholarly communication as a conversation.

Scholarly communication is defined by the Association of College & Research Libraries (2020) as

the system through which research and other scholarly writings are created, evaluated for quality, disseminated to the scholarly community, and preserved for future use. The system includes both formal means of communication, such as publication in peer-reviewed journals, and informal channels, such as electronic listservs. (What is Scholarly Communication? section)

It is also a conversation between the creators of this information, who have a variety of backgrounds and perspectives, and who use a variety of methods to share their ideas. These individuals write about, present, publish, and post their ideas with the intent that others will use the information to push the conversation to the next level, and even start new conversations. For example, there are a host of articles, websites , and books on climate change and pollution. There is also literature on how the auto industry is responding to changing emission standards, including the technology in electric vehicles. A researcher watching these conversations unfold might begin a new conversation connecting the two: a discussion of the role technology can play in reducing pollution — this may include improved electric cars or genetically modified plants that absorb more CO2.

All of this communication takes place in a variety of resource formats — as we discussed in Chapter 2 — and all have value and contribute to scholarly communication . A single scholarly work may not represent the only perspective, or even the majority perspective on an issue. Consider that scholarship on a topic will grow and evolve over time. New information will be discovered which may support or contradict previous information. This new information will be discussed and debated, along with new interpretations and new theories that develop over time. Good scholars will make an effort to document previous work on the topic that informs their own studies. College students are expected to follow this practice as well, by citing the work they use in research papers and projects, and positioning their ideas within the larger conversation about this topic.

This process of positioning one’s ideas within the larger conversation on a topic is called a review of the literature. Sometimes a literature review stands alone, and sometimes it is part of a larger research project, but in both cases its purpose is to investigate the context of the topic you are researching to better situate your research — your part of the conversation — within the larger conversation, and to support your conclusions. This includes finding out about the research already done on a topic, or existing knowledge, and finding gaps in the knowledge where further research is needed to fill out the bigger picture and add new information to the conversation.

For example, this piece of a picture is clear enough. It’s a street-lamp style light, and some leaves. This could be a piece of a conversation about technology’s contribution to light pollution, for example. Sure, it stands alone, but it starts to make real sense when you see the whole picture all at once — now we know the context for our piece (see below):

In this case, the larger picture shows part of Weber State University’s Ogden campus. The larger conversation here — WSU — includes many smaller pieces that contribute to the whole. In addition to the lamp-post, other pieces of this conversation could include the clock tower and WSU’s history, the landscaping and WSU’s carbon-footprint reduction plan, WSU’s new science building, or WSU’s position in the region.

But where do you go to find out about the scholarly conversation in progress about a topic and get that big picture? When searching for information, where you look will depend partly on when it happened:

A news crew can film something happening live, send it back to the station, edit it, and broadcast it on the news in just a few hours. People can take pictures with cell phones and post them immediately on social media. For significant events, newspapers and magazines can arrange to cover the event in the next issue that will appear, usually within days.

Coverage of the event in scholarly journals will take longer because those sources are published less frequently. Coverage in these sources is typically more in-depth, and the time to study and research takes longer. These sources are often peer-reviewed , which adds even more time to the process.

Books and encyclopedias take even longer to write, edit, and publish. In general, the longer it takes for the source to appear, the more in-depth the coverage will be. Encyclopedias are probably the one exception; these typically provide general overviews and are meant only to introduce you to a topic.

The Web is unique in that it can be edited and/or revised hourly, but may also contain information from any point in time. Even if information was uploaded recently, it may be old information that reflects only what was known at the time of its creation. This is particularly true with some kinds of social media. For example, sometimes news posts shared on Facebook end up re-circulating long after the date they were originally posted because people note the headline and get worked up, then repost it without checking the date, or even the facts. It is important to verify information found on the Web with sources that have a clear creation date.

A SAMPLE SEARCH

If you have been assigned a research assignment about former President George H. W. Bush’s efforts to persuade the American public that the 1991 Gulf War was a worthwhile struggle, you might do some preliminary reading in the Encyclopedia Britannica and discover entries such as Iraq and the War of 1991, George Herbert Walker Bush, or Saddam Hussein. Additional background reading might be found doing a simple Google  search for (“Gulf War” OR “Iraq War 1991”) AND “public relations.”

Once you are familiar with the topic and what went on during that time, you might search one of the library databases, such as Academic OneFile , Academic Search Ultimate , and JSTOR for full-text, peer-reviewed journal articles using a similar search statement . Or, you might search the library’s online catalog to find books on this topic using the search statement “Iraq War 1991” AND “mass media.” These sources will give you more in-depth coverage of the event. An academic video database like AVON might provide some in-depth documentaries with information on the topic from a variety of perspectives. For example, The Case Against Saddam Hussein documents a Dan Rather interview of Secretary of State Colin Powell, after Powell made a historic speech making the case against Saddam Hussein to an audience at the United Nations.

You could also do a YouTube search for “Desert Storm” AND “Gulf War 1991” to see primary (first-hand) videos of events during that time period, or search library newspapers from October 1990 to February 1991 for examples of new stories, ads, and editorials to determine the mood of US society at the time.

This provides a simple example of a few sources one might consult on a specific topic. All of these sources come from different time periods. They contain information from different perspectives and at different points in the timeline of information production. Understanding how they all fit together to form a complete picture will give you a much broader perspective than relying on a single source, even if it is a peer-reviewed scholarly journal article. It is important to remember that there are lots of conversations happening and not everyone agrees. All of the individual sources you find are just a small piece of the larger conversation. No one source will represent the only — or even the majority — perspective on a topic. A single source only represents the research and view of one author. You can’t read everything on a topic, but it is always a good idea to look at multiple sources from a variety of perspectives to get a better idea of the big picture.

SCIENCE = TRUTH?

Many equate scientific studies with truth. One of the hallmarks of high quality scientific research is the ability to replicate experiments. Dr. Brian Nosek at the University of Virginia led a project where researchers attempted to replicate one hundred psychology experiments published in three leading journals. The results of his study showed that nearly two thirds of the results of the experiments could not be replicated (Open Science Collaboration, 2015). According to Nosek [1] , “our best methodologies to try to figure out truth mostly reveal to us that figuring out truth is really hard. And we’re going to get contradictions. One year, we’re going to learn that coffee is good for us. The next year, we’re going to learn that it’s bad for us. The next year, we’re going to learn we don’t know” (Vedantam, 2015, para. 7).

When an experiment cannot be replicated, does that mean it’s wrong? Nosek said it’s possible that both studies are right, and that the inability to reproduce a study may be “a sign of uncertainty, not a sign of untrustworthiness… a signal there’s something going on that we don’t understand” (Vedantam, 2015, para. 8).

Vedantam (2015) believes that many “look to science to provide us with answers and certainty when science really is in the business of producing questions and producing more uncertainty” (para. 15). He talks about the equivalence between science and journalism: journalists

paint a picture of the world every day, whether that’s a war zone or financial markets. But we’re always doing it in the context of imperfect information. And especially when we’re covering things we don’t know much about – you know, a big breaking story, what we discover in the first few days is likely to get revised down the road. Now, you can throw up your hands and say, let’s not waste time reading or listening to the first draft of history. Let me just wait a month or a year for the whole picture to emerge. But I think most people would say the best information is still valuable, even if it’s going to get updated tomorrow. We need to think about scientific studies the same way. (para. 15)

Just like news stories develop over time, science changes too. Truth can change, as new information is learned — such truths are all pieces of larger conversations, and conversations build with each voice that contributes its discoveries. Anyone who has been on a road trip with a kid will tell you how boring an unchanging conversation can be (Are we there yet? Are we there yet? Are we there yet?). It’s up to everyone to add what they can to scholarly conversations to keep them from stagnating, and to keep progress — academic, scientific, technological, and artistic progress — moving, changing, and advancing.

FINDING IDEAS FOR TOPICS

Most of the time, you’ll be given a topic to write about. Other times, you’ll need to come up with your own idea, which can be the most challenging part of the research process . You can find ideas for topics in a number of places. Here’s a list to help you get started:

• Worried that your topic is too broad? Talk to a subject librarian if you need help narrowing it down.
• Textbooks are great for an overview of topics. If you are taking an interesting course, look in your textbook for topic ideas, or ask your professor.
• Reference books are good for identifying specific areas of a subject you might be interested in pursuing. Browse the reference area and look for encyclopedias or other reference sources that will give you background on a variety of topics.
• Magazines and newspapers are good sources for exploring current events issues.
• Journal articles are best for exploring more scholarly topics. Ask a librarian for recommendations.
• General or subject-specific article databases are great if you already have a topic in mind. Academic Search Ultimate is a good general database to start with if you aren’t sure where to look.
• There are several great databases for argumentative papers. CQ Researcher presents both sides of the story for controversial topics, and Opposing Viewpoints in Context allows you to browse specific kinds of information (e.g., academic journals, editorials, statistics, and magazines) on controversial topics.
• Google can help you narrow down a topic if you already have a general idea of what you want to research.
• Wikipedia can also be a good place for some background information on a topic, as well as possible keywords and phrases and search ideas. It should not be used as a source for a paper, but sometimes you see something about a topic that you didn’t know before and can find some good ideas for your paper.
• Don’t forget personal issues you are dealing with or things you are passionate about. This could be a pet, a hobby, or a problem you or a friend or family member is dealing with. Sometimes it is very personal, like single parenting, or a certain disease a loved one deals with. Think about questions you can ask about these things. For example, how did this hobby become a thing? Who started it? Is there some psychological benefit to owning a pet? What makes people mistreat pets? Are single parents treated equally in the workplace? What is the current state of research on this disease? Use these simple questions as a starting point to develop a focused research question or thesis statement that will work for the scope of the project you will be working on.

SCOPE OF THE PROJECT

Once you decide on an idea, it is important to think about the scope of the assignment or project you will be working on. The length of a paper is important to consider because that will give some indication of how many sources will be needed — and how broad or focused the topic should be. For example, are you working on a 4-page paper that discusses your personal views on a particular topic, or are you working on a 30-page scholarly literature review? Are you required to find a minimum of two sources of any kind, or are you required to find a minimum of 15 peer-reviewed articles on the topic? If your topic is too narrow, you won’t be able to find the required number of sources or have enough information to fill out the required length for the paper.

If you are required to use specific types (e.g., scholarly) or formats (e.g., books ) of sources, then you can’t go with just any topic. For example, you may not be able to locate 15 peer-reviewed articles on hip-hop music or football. However, if you modify the topic to focus on the role that social class plays in hip-hop music or the issue of recurrent concussions in college football players, you will be able to locate academic journal articles or books. If you are required to use a minimum of two books for your paper, you won’t be able to find any on a topic that is too current, such as a presidential election that happened a month ago. However, if you revise your focus to a related issue, such as campaign financing or the merits of the electoral college, you’ll be able to locate books. The scope of your assignment or project will impact the types and formats of information you’ll be able to find, as well as how broad or narrow your topic will need to be. These are important to consider before you begin looking for information. But remember, it’s okay to go back and revise your topic if you find that you are having a hard time finding sources — it’s all part of the process. Just be sure to clear it with your professor.

FOCUSING THE TOPIC & FORMULATING A RESEARCH QUESTION OR THESIS STATEMENT

Most college level research papers will have either a research question or a thesis statement . Both identify the issue you will be writing about; one is phrased as a question and the other is phrased as a statement. Both should be focused to work with the scope of your project or assignment. There are many ways to focus a topic idea. You can focus a topic by geography, a specific population, or a particular time period. You don’t have to focus your topic in all these ways; these are just some possibilities:

Why is it important to focus a topic? If it’s too broad, you’ll be overwhelmed. For example, searching for sources on “education” would bring up an enormous amount of information, but focusing your search on “how school vouchers will hurt already economically strapped public schools” will provide a more manageable set of results. On the other hand, it is important that your topic is not too narrow, as you will have a hard time finding enough information (or the required type of information), and not have enough issues to develop effectively. In this situation, your paper may lack context and depth.

Following are some examples of research questions and thesis statements that are problematic, and some suggestions for fixing them:

PROBLEM: This topic is too current.

How have this year’s extreme weather events impacted learning outcomes for children in affected areas?

In this case you will most likely not be able to find either books or scholarly journal articles on this topic because it’s so recent that none of these materials have been published yet. If the assignment is a brief paper asking you to locate some current information from news outlets about recent weather events’ impacts on schools, this question will work just fine as is.

SOLUTION: Try a related topic that has a greater timespan.

How have extreme weather events impacted school systems in affected areas?

Rather than focusing on this year’s extreme weather and something very specific (learning outcomes), you might focus on a related issue, such as extreme weather’s impacts on schools as a whole. This is broad enough that you’ll be able to find information in a wider range of sources, including scholarly journal articles and books.

PROBLEM: Topic is too narrow. You will have a hard time finding sources on this topic.

How will an increase in the number of charter schools in Ogden impact the local public schools in the area?

This is actually a great question, but you will have a hard time finding information on this topic, and may only find information in a few local newspapers or on the Web. If your paper is lengthy or requires you to use particular formats, such as scholarly journals , you’ll need to think a little more broadly. If it’s a short paper that requires just a few sources, and it doesn’t matter what they are, this question might work just fine as is.

SOLUTION: Rewrite it so that it’s broad enough to include scholarly information.

How does an increase in the number of charter schools in small towns impact the local schools in the area?

In this example, you are no longer limiting yourself to Ogden, Utah, or to a particular type of school. You’ll have a much easier time finding information on this broader topic, as there will be more coverage in a wider range of sources. That will be more appropriate for a longer paper.

PROBLEM: Popular topics that require little in-depth research should be avoided.

Examples in this category might include the topics of hip-hop music or football .

How would a zombie apocalypse cause mental breaks in humans? While a zombie apocalypse is a little unorthodox for a research topic , with some adjustments it can still be made into a valid research question.

SOLUTION: Come up with a focus that might be covered in scholarly literature.

For example, you might research the role social class plays in hip-hop music , or the issue of recurrent concussions in college football players . These focus on issues that might be covered in academic journals or books.

How would the psychological concept of abjection play a role in how human beings reacted to a zombie apocalypse? This narrows the focus of the question to abjection and the zombie apocalypse. Someone researching this question would need to look at the psychological literature on abjection as well as the literature on zombies.

PROBLEM: Topic can be answered with an encyclopedia, dictionary, or found quickly with a simple Google search.

What is evolution?

This question involves no research or creative thought. It is a simple definition or encyclopedia-type question.

SOLUTION: Reword topic to require research and creative thought.

Why should evolution be taught in school? OR The concept of evolution should not be taught in school because of its religious connotations.

Both the research question and the thesis statement clearly take a side on this issue. Your research will use the literature to provide reasons why it should (or should not) be taught. There is abundant literature on this issue that will provide support for either position.

PROBLEM: The research question is too broad or vague.

How does television influence viewers?

Readers might ask: “How does television influence what viewers? What do you mean by influence?”

How are people motivated?

This question is too broad. What people are you referring to? Colleagues, teenagers, athletes, the elderly? Motivated in what ways? It could be positive or negative reinforcement, it could be through team leadership, etc.

SOLUTION: Reword so that the question is focused and more defined.

How does viewing of cartoons on TV by children under age two affect their cognitive development?

Here you’ve focused on a more specific group, young children, and you’ve also defined what you mean by influence.

How do teachers motivate students through positive reinforcement and how does this affect their academic performance?

This helps narrow the focus to teachers and positive reinforcement and allows for the results to be examined through their academic performance.

PROBLEM: The thesis statement is too broad or vague.

People need to stay healthy.

Who are we talking about? What kinds of programs would help people to stay healthy? Exercise programs? Dietary changes? Individuals require different exercise programs and eating habits to be healthy. A program for an Olympic athlete would be totally different than an elderly woman or an elementary aged child.

More attention should be paid to the food choices available to school children.

This thesis asserts your position on the issue, but the term “more attention” and “food choices” are somewhat vague and could be more descriptive. This question also doesn’t explain why we should pay attention. Is it because cafeteria workers don’t have time to clean up the mess? Is it because of food waste?

SOLUTION: Reword so that the thesis statement is focused and more defined.

Due to the prevalence of childhood obesity in the United States, elementary school lunch programs should look into sustainable farming to support children’s daily requirements of fresh fruits and vegetables.

Here the focus is on elementary aged children and school lunch programs who use sustainable farming as a way to provide healthy meal choices to children.

Because half of all American school children consume nine times the recommended daily allowance of sugar, schools should be required to replace the beverages in soda machines with healthy alternatives.

This thesis is very specific and addresses what should be done about excessive sugar consumption as well as who should address it. This also addresses beverage machines in particular, rather than the vague phrase “food choices.”

PROBLEM: Thesis is a simple statement of fact or opinion. There is no argument or proposed solution, even if the writer’s position is clear.

The behaviorist style works better than the constructivist approach.

Mark Twain is the best writer that has ever lived.

Trained animals have been shown to have a therapeutic effect on humans.

Pollution is bad for the environment.

SOLUTION: Reword so that there is a clear argument.

After a time of fervent dedication to behaviorist pedagogy by teachers and administrators, elementary school education has begun to emphasize constructivist approaches; however, there is little evidence to attest to the efficacy of this method.

Mark Twain’s success as a writer lies in his use of humor to critique American life.

Trained therapy animals have been shown to have a calming effect for humans under stress; therefore, therapy dogs should be present in all colleges and universities during finals.

America’s anti-pollution efforts should focus on privately owned cars because it would allow most citizens to contribute to national efforts and care about the outcome.

PROBLEM: This question can be answered with a simple yes or no.

Should evolution be taught in schools?

This is a great pro/con topic on a controversial issue, but it is phrased as an opinion type question that can be answered with a simple “yes” or “no.”

Will education in prisons affect recidivism rates?

This is a great question, but as worded, it is a simple yes or no question.

SOLUTION: Reword topic to elicit more than a simple “yes” or “no” answer.

Why should evolution be taught in school?

You are no longer asking a yes or no question; rather, you are taking a side on this issue, and your research will provide reasons why it should be taught. You could also take the opposite point of view.

How might education in correctional facilities play a role in reducing recidivism?

This question implies that education impacts recidivism rates.

SEARCH TECHNIQUES

Search techniques are the methods you use to search for information on your topic, and allow you to turn your research question or thesis statement into a search statement . This section covers different techniques that are used to formulate effective searches: keywords / synonyms , Boolean operators, phrase searching , and truncation . These techniques are combined to formulate search statements, which are used in various search tools (e.g., library catalogs , article databases, Google and Google Scholar ). An additional technique covered later in the textbook involves using the limiters (sometimes called facets) available in the search tools you are using.

KEYWORDS & SYNONYMS

Keywords are the main ideas in your research question or thesis statement, and these are the words that you will use to search for information on your topic. Sometimes an idea will include more than one word to describe the concept, and in this case, you’ll have a phrase . Examples of phrases are concepts like “ global warming ,” “ stem cell ,” or “ gun control .” Different authors may use different terms to describe the same concept — but you don’t want to miss their articles just because they use a different word. That’s why synonyms are important when you think about searching for information. For example, a synonym for “cat” is “feline.”

Let’s take the question: How has the Internet influenced students’ learning styles?

• There are three main concepts in the question: Internet, students, and learning styles .
• Internet and students are both keywords .
• “ Learning styles ” is a phrase , because this concept has more than one word. When you use a phrase in a search statement , you will use quotation marks around the phrase.
• Synonyms for the keyword Internet are Web , online , WWW , or World Wide Web . Depending on how much information you find, you could also use additional terms for things one might spend time doing on the internet, like social media or YouTube.
• Synonyms for the keyword students include learners, pupils , or scholars . In this case, it is important to consider the time period when these words would have been used most frequently. Pupils , for example, may not be used very often in modern research about students, so a search with this might retrieve old results, or even medical research, since pupil is a part of the anatomy of the eye. Instead, consider students from another perspective: what kind of students? The term undergraduate could be used to investigate higher education learning styles, while K-12 or high school could be used for learning styles in primary education.
• Synonyms for the phrase learning styles might be hard to think of, but consider that you might want to focus on learning styles as a whole, but another researcher might have focused on just one or two learning styles. For this reason, it is important to write down not only the actual words in the question or thesis, but alternate terms as well — in this case, terms such as auditory, visual, tactile, or kinetic would be good.

Here’s another example. All of the main concepts are highlighted in yellow:

How might education in correctional facilities play a role in reducing recidivism ?

PHRASE SEARCHING

Phrase searching occurs by placing quotation marks around multiple words or phrases , the database looks only for the phrase and not for the words separately. By using quotation marks, you are telling the computer to only bring back results containing the exact terms you entered, in the exact order you typed them, such as in these examples:

“school vouchers”

“charter schools”

“academic achievement”

“school prayer”

Let’s take a look at our previous example:

The term “ correctional facilities ” is a phrase and should be written in quotation marks when doing a search so the database searches for the terms together instead of separately. For example, by putting this phrase in quotes, you’ll avoid mismatched results about correctional techniques in horse training, or staffing problems in health facilities, and instead retrieve only results that specifically include something about correctional facilities. Another phrase for this example would be the synonym for recidivism : “ repeat offense .”

Note: Phrases should be kept short, only 2-3 words in length. The longer the phrase, the less likely you are to find that exact phrase in your search. For example, you wouldn’t use “ education in correctional facilities ” or “ role in reducing recidivism ” in a search.

To truncate means to “chop off.” For example, think about the trunk of a tree – you could chop off all the branches and still be left with the trunk — the base of the tree. When you truncate a word, you chop off the end, so the search tool can search for multiple endings – or branches that extend from the base. When you truncate, you chop off the end of a word so the computer can search for multiple endings. Here’s an example of how truncation works in a database:

• You search for the word advertisement and retrieve 23 results. All of these have the word advertisement in them.
• advertis ED
• advertis ER
• advertis ES
• advertis ING
• advertis EMENT
• advertis EMENTS

Truncation is a great way to broaden your search and pick up more results, but there are a few things that are important to remember when using truncation:

• Different databases use different truncation symbols. Most use an asterisk (*), but a few use exclamation points, question marks, and dollar signs. Use the database help screens to determine which symbols are appropriate. * ! ? $
• Truncation only picks up word ENDINGS, not synonyms . For example, truncating after the ‘n’ in science (scien*) will find science, sciences, and scientific, but it will NOT find the words biology, chemistry, or astronomy.
• Be careful where you place the truncation symbol. You don’t want to over truncate (or cut too much) and you also don’t want to under truncate (cut too little). For example, when we do a search for the topic computers , we want to truncate to search for terms related to computers:

BOOLEAN OPERATORS

There are a number of Boolean operators . The ones we will focus on in this course are AND and OR . It is a good idea to always capitalize Boolean operators when you use them in a search because some databases require it.

One of the less common Boolean operators is NOT , which is not generally used, even by advanced searchers, because it can inadvertently eliminate useful sources from a search’s results. For that reason, this course will focus on AND and OR . However, we will provide a brief overview of NOT here.

As you might have guessed, the NOT operator eliminates all results that include the term you specify. For example, if you do a search for information about roadrunners, the species of bird, you may end up with articles that also talk about Roadrunner, the cartoon character. You could potentially get around this by searching for roadrunner NOT cartoon, or roadrunner NOT Wile E. Coyote. However, a better way to find the information you need could be a search for the scientific name, Geococcyx californianus; or you could include additional terms such as habitat, diet, predators, etc. The NOT might help, but it is usually possible to achieve similar results using other search techniques . Here are some other examples:

lions AND diet NOT zoo* — if you want to learn about what lions eat in the wild. Note that this would exclude any article that compared the diets of zoo lions to wild lions, which could be useful.

“peanut butter” AND sandwich NOT jelly — if you’d like to find out what types of sandwiches are out there that use peanut butter other than PB&J. Note again that this would exclude a list of sandwich recipes if it included jelly in even one.

“traumatic brain injury” NOT football — if you want to learn about TBI that is caused by other things. Once again, note that any article that included football as a cause in addition to other causes would be eliminated.

AND connects all of the main concepts together, and tells the computer you want ALL of the words in your search results. This narrows your search, giving you FEWER results. Usually we think of AND as getting more, not less. For Thanksgiving dinner, you want pumpkin pie AND the apple pie; in this example, you get more with AND. In a database, however, AND does the opposite, like if you walk into an ice cream shop and ask for ice cream, they give you 32 scoops. That’s too much, so you won’t have room for pie. So you ask to only give you a scoop if it has chocolate: there are ten different flavors with chocolate, so now you are down to ten scoops — still too much. So then you ask to give you a scoop if it has chocolate AND caramel because you have to have both. Now you have two scoops — and room for pie.

Here’s how that would look as a search:

“ice cream” = 32 scoops “ice cream” AND chocolate = 10 scoops “ice cream” AND chocolate AND caramel = 2 scoops

Another Example:

“peanut butter” AND jelly AND sandwich

If you connect these terms with AND, you will only get peanut butter and jelly sandwiches. You will NOT get sandwiches with jelly only, sandwiches with just peanut butter, or peanut butter and jelly on crackers.

All three of those terms MUST be present, and you only get one kind of sandwich.

OR broadens your search, giving you MORE results. Using OR tells the database to find ANY of the terms where each word may appear separately, as well as where any terms may appear together. Note: You MUST use parentheses around OR terms.

(“peanut butter” OR jelly) AND sandwich

If you connect the first two terms with OR , you will get peanut butter and jelly sandwiches, sandwiches with jelly only, or sandwiches with just peanut butter. You still won’t get crackers, but you’ll get different kinds of sandwiches.

Peanut butter can be present, jelly can be present, or both of them can be present. Instead of only one kind of sandwich, you now have three different options. Keep in mind you may also end up with jelly-and-cream cheese sandwiches, or peanut butter and banana. They’ll still include your search term, but they may not be quite what you were thinking of… of course, it could give you useful information you hadn’t thought of before.

(“peanut butter” OR jelly OR sandwich)

If you connect all of the terms with OR , you’ll probably get a lot more than you want. Not only will you get peanut butter and jelly sandwiches, sandwiches with jelly only, or sandwiches with just peanut butter, but you’ll also get ham and cheese sandwiches, sandwiches with roast beef and onions, or pretty much any sandwich you can think of. You may also get how to make jelly, or peanut butter allergies. Here you are telling the database to search for anything with any one of those terms in it.

CREATING SEARCH STATEMENTS

Search statements are what you type into a search box to find information on your topic. You combine all of the search techniques covered above in this chapter ( keywords and synonyms , truncation , phrase searching , and Boolean operators ) to create search statements. Let’s take a look at our previous example and go through the process step-by-step to create search statements:

Research question: How might education in correctional facilities play a role in reducing recidivism?

• Choose keywords (main concepts): education, correctional facilities, recidivism
• education schooling instruction learning tutoring
• correctional facilities jail prison penitentiary incarceration
• recidivism reoffending reoffence repeat offense repeat offenders
• educat* school* instruct* learn* tutor*
• correctional facilit* jail* prison* penitentiar* incarcerat*
• recidivis* reoffen* reoffence repeat offense* repeat offend*
• Put quotation marks around phrases (concepts that are two words or more)
• Combine search techniques to create search statements . Any research question or thesis statement can have many possible search statements, and it’s a good idea to try more than one — you never know who may have answers to your questions but be using different vocabulary combinations. These are just a few examples:

SEARCH STATEMENTS VS NATURAL LANGUAGE QUERIES

If you use Google to search for information, you probably know that it is easy to simply type in your whole question to find information. This is called a natural language query ; it means putting things into the search box in your own words, without any special symbols or format — as if you were talking to someone. While this works pretty well in Google , it does not work as well (or at all) in most of the library’s subscription databases. For example, if you wanted to find out how high the Empire State Building was using a natural language search, you might type in How high is the Empire State Building? If you are using a keyword-based search in a database, you would probably type in the search statement “ Empire State Building ” AND height .

• See article “Health Effects of Coffee: Where Do We Stand?” at https://www.cnn.com/2015/08/14/health/coffee-health by Sandee LaMotte and published on the CNN website and updated on April 12, 2018. ↵

The system through which scholarly information is created, evaluated, shared, and preserved; referred to as a conversation between scholars for its back-and-forth nature, this system uses formal and informal means to communicate information, such as scholarly journal articles, blog posts, or even student papers.

One of the components of the Research Process , which involves the practice of appraising the value of an information source both in its own right and as it relates to your topic, typically by investigating its Authority , Credibility , Currency , Bias , and Documentation .

A process some scholarly articles go through prior to publication, where scholars in that field read and review articles submitted for publication, usually with the option to require edits, approve, or deny publication, and often without knowing the name of the authors.

A collection of information hosted online with a common URL , usually found by searching a Web Search Engine or navigating directly to a known URL , and generally made up of several related Webpages and organized by the inclusion of a menu linking the pages together.

Traditionally a written or printed work consisting of pages glued or sewn together along one side and bound in covers, also available in audio, electronic, and braille formats, making it both a Multi-Format Information source and one of the Long Formats of information.

Giving credit to authors of whose works are used to inform new works, often by Summarizing , Paraphrasing , or Quoting , and providing Attribution , thereby informing readers of where the information came from.

A report evaluating, summarizing, and describing information found in research literature such as articles, books, and reports on a topic one is researching, to provide a basis for and guide future research.

One of the components of the Research Process , which involves discovering information sources to fulfill the information need identified during the Investigation component.

A type of Periodical , containing articles and some photographs and advertisements. Often specific to a city or region; sometimes covering a particular subject or area such as business or finance. Typically a Popular Source .

A type of Periodical , containing articles, illustrations, and advertisements, and sometimes covering a particular subject or area such as hobbies, popular culture, or parenting. Typically a Popular Source .

A type of Periodical containing articles written by experts in specific disciplines, often Peer Reviewed .

A type of Reference Source that provides information about topics in a comprehensive, but summary fashion, like an overview. They are useful for providing facts and giving a broad survey of a topic, and are often written by specialists.

One the most commonly used Web Search Engines , used widely to search for information on millions of topics. Also used as a verb, meaning to use a Web Search Engine to search for information.

A Boolean Operator used to combine Keywords into Search Statements , enclosed in parentheses that will retrieve results including any of the specified search terms, but not necessarily all of the search terms.

A Boolean Operator used to combine Keywords into Search Statements , limiting search results to only items including all of the specified search terms.

A specially formulated query, which combines Keywords , Phrases , Truncation , and Boolean Operators with specific punctuation, used to locate resources within a Library Search Tool .

The concept of scientific inquiry as a nonlinear and iterative process composed of several components, including Investigate , Search , Locate , Evaluate , Document , and Utilize .

An information source typically used as a starting point for research or to look up facts, definitions, overviews, and other information, including Almanacs , Atlases , Bibliographies , Biographies , Concordances , Dictionaries , Directories , Encyclopedias , Gazetteers , Guidebooks , Handbooks , and Manuals .

A Wiki open to and editable by the public, with very broad topic coverage, that can be a useful tool for the beginning stages of the Research Process , such as choosing a topic or finding Keywords , but which is generally not considered to have sufficient Credibility to be used as a source for college-level research

A main idea or important word in a research question or thesis statement; two or more keywords can be combined with Boolean Operators to form the Search Statements used to locate sources in Library Search Tools .

A small group of words standing together to represent a concept or name of a place, person, or thing, such as United States, Rosa Parks, or air conditioner, used in Phrase Searching .

A statement formally articulating an information need in the form of an explicit, detailed question to guide the Research Process ; sometimes framed as a Thesis Statement .

A detailed, explicit statement formally articulating an information need to guide the Research Process ; sometimes framed as a Research Question .

The extent of the area or subject matter that something deals with or to which it is relevant; in Information Literacy , this refers particularly to the breadth and depth of a research project or information need, an important consideration when thinking about the number and types of resources needed to answer the Research Question or fulfill the information need.

Methods used to search for information on a topic in Library Search Tools or Web Search Engines , and to turn a Research Question into a Search Statement .

A word or phrase that means the same as another or can take the place of it, sometimes in a particular context; particularly useful when using Keywords to build Search Statements with the Boolean Operator , OR, or when searching for information on a topic which can be referred to in many ways, such as college students (i.e., undergraduates, university students, graduate students) or climate change (i.e., global warming, sea level rise, green energy, renewable energy, CO2, greenhouse gas, carbon footprint).

The use of a Phrase in quotation marks as part of a Search Statement to locate information in a Library Search Tool , such as “Great Salt Lake” or “information literacy.”

A Search Technique for retrieving results for all possible Keywords stemming from a single common root by placing an asterisk ( ) at the end of the common root; for example, medic will retrieve results for medicine, medical, medics, medication, etc

A Library Search Tool where information about the library’s book collection is kept and made searchable, to allow users to discover and locate needed information

A Web Search Engine designed and operated by the Google company to filter general websites out of results, and retrieve scholarly sources such as articles, books, theses, preprints, and technical reports.

Simple words ( AND , OR , and NOT ) used to combine or exclude Keywords in a Search Statement , resulting in more focused and productive search results.

A Boolean Operator used in Search Statements , limiting search results to only items that do not include the specified search term.

Performing a search for information by using standard language, such as a question in your own words, rather than specially formulated Search Statements . Typically useful in Web Search Engines , but not Library Search Tools .

Information Navigator Copyright © 2022 by Weber State University Stewart Library is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

National Research Council; Division of Behavioral and Social Sciences and Education; Commission on Behavioral and Social Sciences and Education; Committee on Basic Research in the Behavioral and Social Sciences; Gerstein DR, Luce RD, Smelser NJ, et al., editors. The Behavioral and Social Sciences: Achievements and Opportunities. Washington (DC): National Academies Press (US); 1988.

The Behavioral and Social Sciences: Achievements and Opportunities.

• Hardcopy Version at National Academies Press

5 Methods of Data Collection, Representation, and Analysis

This chapter concerns research on collecting, representing, and analyzing the data that underlie behavioral and social sciences knowledge. Such research, methodological in character, includes ethnographic and historical approaches, scaling, axiomatic measurement, and statistics, with its important relatives, econometrics and psychometrics. The field can be described as including the self-conscious study of how scientists draw inferences and reach conclusions from observations. Since statistics is the largest and most prominent of methodological approaches and is used by researchers in virtually every discipline, statistical work draws the lion’s share of this chapter’s attention.

Problems of interpreting data arise whenever inherent variation or measurement fluctuations create challenges to understand data or to judge whether observed relationships are significant, durable, or general. Some examples: Is a sharp monthly (or yearly) increase in the rate of juvenile delinquency (or unemployment) in a particular area a matter for alarm, an ordinary periodic or random fluctuation, or the result of a change or quirk in reporting method? Do the temporal patterns seen in such repeated observations reflect a direct causal mechanism, a complex of indirect ones, or just imperfections in the data? Is a decrease in auto injuries an effect of a new seat-belt law? Are the disagreements among people describing some aspect of a subculture too great to draw valid inferences about that aspect of the culture?

Such issues of inference are often closely connected to substantive theory and specific data, and to some extent it is difficult and perhaps misleading to treat methods of data collection, representation, and analysis separately. This report does so, as do all sciences to some extent, because the methods developed often are far more general than the specific problems that originally gave rise to them. There is much transfer of new ideas from one substantive field to another—and to and from fields outside the behavioral and social sciences. Some of the classical methods of statistics arose in studies of astronomical observations, biological variability, and human diversity. The major growth of the classical methods occurred in the twentieth century, greatly stimulated by problems in agriculture and genetics. Some methods for uncovering geometric structures in data, such as multidimensional scaling and factor analysis, originated in research on psychological problems, but have been applied in many other sciences. Some time-series methods were developed originally to deal with economic data, but they are equally applicable to many other kinds of data.

• In economics: large-scale models of the U.S. economy; effects of taxation, money supply, and other government fiscal and monetary policies; theories of duopoly, oligopoly, and rational expectations; economic effects of slavery.
• In psychology: test calibration; the formation of subjective probabilities, their revision in the light of new information, and their use in decision making; psychiatric epidemiology and mental health program evaluation.
• In sociology and other fields: victimization and crime rates; effects of incarceration and sentencing policies; deployment of police and fire-fighting forces; discrimination, antitrust, and regulatory court cases; social networks; population growth and forecasting; and voting behavior.

Even such an abridged listing makes clear that improvements in methodology are valuable across the spectrum of empirical research in the behavioral and social sciences as well as in application to policy questions. Clearly, methodological research serves many different purposes, and there is a need to develop different approaches to serve those different purposes, including exploratory data analysis, scientific inference about hypotheses and population parameters, individual decision making, forecasting what will happen in the event or absence of intervention, and assessing causality from both randomized experiments and observational data.

This discussion of methodological research is divided into three areas: design, representation, and analysis. The efficient design of investigations must take place before data are collected because it involves how much, what kind of, and how data are to be collected. What type of study is feasible: experimental, sample survey, field observation, or other? What variables should be measured, controlled, and randomized? How extensive a subject pool or observational period is appropriate? How can study resources be allocated most effectively among various sites, instruments, and subsamples?

The construction of useful representations of the data involves deciding what kind of formal structure best expresses the underlying qualitative and quantitative concepts that are being used in a given study. For example, cost of living is a simple concept to quantify if it applies to a single individual with unchanging tastes in stable markets (that is, markets offering the same array of goods from year to year at varying prices), but as a national aggregate for millions of households and constantly changing consumer product markets, the cost of living is not easy to specify clearly or measure reliably. Statisticians, economists, sociologists, and other experts have long struggled to make the cost of living a precise yet practicable concept that is also efficient to measure, and they must continually modify it to reflect changing circumstances.

Data analysis covers the final step of characterizing and interpreting research findings: Can estimates of the relations between variables be made? Can some conclusion be drawn about correlation, cause and effect, or trends over time? How uncertain are the estimates and conclusions and can that uncertainty be reduced by analyzing the data in a different way? Can computers be used to display complex results graphically for quicker or better understanding or to suggest different ways of proceeding?

Advances in analysis, data representation, and research design feed into and reinforce one another in the course of actual scientific work. The intersections between methodological improvements and empirical advances are an important aspect of the multidisciplinary thrust of progress in the behavioral and social sciences.

• Designs for Data Collection

Four broad kinds of research designs are used in the behavioral and social sciences: experimental, survey, comparative, and ethnographic.

Experimental designs, in either the laboratory or field settings, systematically manipulate a few variables while others that may affect the outcome are held constant, randomized, or otherwise controlled. The purpose of randomized experiments is to ensure that only one or a few variables can systematically affect the results, so that causes can be attributed. Survey designs include the collection and analysis of data from censuses, sample surveys, and longitudinal studies and the examination of various relationships among the observed phenomena. Randomization plays a different role here than in experimental designs: it is used to select members of a sample so that the sample is as representative of the whole population as possible. Comparative designs involve the retrieval of evidence that is recorded in the flow of current or past events in different times or places and the interpretation and analysis of this evidence. Ethnographic designs, also known as participant-observation designs, involve a researcher in intensive and direct contact with a group, community, or population being studied, through participation, observation, and extended interviewing.

Experimental Designs

Laboratory experiments.

Laboratory experiments underlie most of the work reported in Chapter 1 , significant parts of Chapter 2 , and some of the newest lines of research in Chapter 3 . Laboratory experiments extend and adapt classical methods of design first developed, for the most part, in the physical and life sciences and agricultural research. Their main feature is the systematic and independent manipulation of a few variables and the strict control or randomization of all other variables that might affect the phenomenon under study. For example, some studies of animal motivation involve the systematic manipulation of amounts of food and feeding schedules while other factors that may also affect motivation, such as body weight, deprivation, and so on, are held constant. New designs are currently coming into play largely because of new analytic and computational methods (discussed below, in “Advances in Statistical Inference and Analysis”).

Two examples of empirically important issues that demonstrate the need for broadening classical experimental approaches are open-ended responses and lack of independence of successive experimental trials. The first concerns the design of research protocols that do not require the strict segregation of the events of an experiment into well-defined trials, but permit a subject to respond at will. These methods are needed when what is of interest is how the respondent chooses to allocate behavior in real time and across continuously available alternatives. Such empirical methods have long been used, but they can generate very subtle and difficult problems in experimental design and subsequent analysis. As theories of allocative behavior of all sorts become more sophisticated and precise, the experimental requirements become more demanding, so the need to better understand and solve this range of design issues is an outstanding challenge to methodological ingenuity.

The second issue arises in repeated-trial designs when the behavior on successive trials, even if it does not exhibit a secular trend (such as a learning curve), is markedly influenced by what has happened in the preceding trial or trials. The more naturalistic the experiment and the more sensitive the meas urements taken, the more likely it is that such effects will occur. But such sequential dependencies in observations cause a number of important conceptual and technical problems in summarizing the data and in testing analytical models, which are not yet completely understood. In the absence of clear solutions, such effects are sometimes ignored by investigators, simplifying the data analysis but leaving residues of skepticism about the reliability and significance of the experimental results. With continuing development of sensitive measures in repeated-trial designs, there is a growing need for more advanced concepts and methods for dealing with experimental results that may be influenced by sequential dependencies.

Randomized Field Experiments

The state of the art in randomized field experiments, in which different policies or procedures are tested in controlled trials under real conditions, has advanced dramatically over the past two decades. Problems that were once considered major methodological obstacles—such as implementing randomized field assignment to treatment and control groups and protecting the randomization procedure from corruption—have been largely overcome. While state-of-the-art standards are not achieved in every field experiment, the commitment to reaching them is rising steadily, not only among researchers but also among customer agencies and sponsors.

The health insurance experiment described in Chapter 2 is an example of a major randomized field experiment that has had and will continue to have important policy reverberations in the design of health care financing. Field experiments with the negative income tax (guaranteed minimum income) conducted in the 1970s were significant in policy debates, even before their completion, and provided the most solid evidence available on how tax-based income support programs and marginal tax rates can affect the work incentives and family structures of the poor. Important field experiments have also been carried out on alternative strategies for the prevention of delinquency and other criminal behavior, reform of court procedures, rehabilitative programs in mental health, family planning, and special educational programs, among other areas.

In planning field experiments, much hinges on the definition and design of the experimental cells, the particular combinations needed of treatment and control conditions for each set of demographic or other client sample characteristics, including specification of the minimum number of cases needed in each cell to test for the presence of effects. Considerations of statistical power, client availability, and the theoretical structure of the inquiry enter into such specifications. Current important methodological thresholds are to find better ways of predicting recruitment and attrition patterns in the sample, of designing experiments that will be statistically robust in the face of problematic sample recruitment or excessive attrition, and of ensuring appropriate acquisition and analysis of data on the attrition component of the sample.

Also of major significance are improvements in integrating detailed process and outcome measurements in field experiments. To conduct research on program effects under field conditions requires continual monitoring to determine exactly what is being done—the process—how it corresponds to what was projected at the outset. Relatively unintrusive, inexpensive, and effective implementation measures are of great interest. There is, in parallel, a growing emphasis on designing experiments to evaluate distinct program components in contrast to summary measures of net program effects.

Finally, there is an important opportunity now for further theoretical work to model organizational processes in social settings and to design and select outcome variables that, in the relatively short time of most field experiments, can predict longer-term effects: For example, in job-training programs, what are the effects on the community (role models, morale, referral networks) or on individual skills, motives, or knowledge levels that are likely to translate into sustained changes in career paths and income levels?

Survey Designs

Many people have opinions about how societal mores, economic conditions, and social programs shape lives and encourage or discourage various kinds of behavior. People generalize from their own cases, and from the groups to which they belong, about such matters as how much it costs to raise a child, the extent to which unemployment contributes to divorce, and so on. In fact, however, effects vary so much from one group to another that homespun generalizations are of little use. Fortunately, behavioral and social scientists have been able to bridge the gaps between personal perspectives and collective realities by means of survey research. In particular, governmental information systems include volumes of extremely valuable survey data, and the facility of modern computers to store, disseminate, and analyze such data has significantly improved empirical tests and led to new understandings of social processes.

Within this category of research designs, two major types are distinguished: repeated cross-sectional surveys and longitudinal panel surveys. In addition, and cross-cutting these types, there is a major effort under way to improve and refine the quality of survey data by investigating features of human memory and of question formation that affect survey response.

Repeated cross-sectional designs can either attempt to measure an entire population—as does the oldest U.S. example, the national decennial census—or they can rest on samples drawn from a population. The general principle is to take independent samples at two or more times, measuring the variables of interest, such as income levels, housing plans, or opinions about public affairs, in the same way. The General Social Survey, collected by the National Opinion Research Center with National Science Foundation support, is a repeated cross sectional data base that was begun in 1972. One methodological question of particular salience in such data is how to adjust for nonresponses and “don’t know” responses. Another is how to deal with self-selection bias. For example, to compare the earnings of women and men in the labor force, it would be mistaken to first assume that the two samples of labor-force participants are randomly selected from the larger populations of men and women; instead, one has to consider and incorporate in the analysis the factors that determine who is in the labor force.

In longitudinal panels, a sample is drawn at one point in time and the relevant variables are measured at this and subsequent times for the same people. In more complex versions, some fraction of each panel may be replaced or added to periodically, such as expanding the sample to include households formed by the children of the original sample. An example of panel data developed in this way is the Panel Study of Income Dynamics (PSID), conducted by the University of Michigan since 1968 (discussed in Chapter 3 ).

Comparing the fertility or income of different people in different circumstances at the same time to find correlations always leaves a large proportion of the variability unexplained, but common sense suggests that much of the unexplained variability is actually explicable. There are systematic reasons for individual outcomes in each person’s past achievements, in parental models, upbringing, and earlier sequences of experiences. Unfortunately, asking people about the past is not particularly helpful: people remake their views of the past to rationalize the present and so retrospective data are often of uncertain validity. In contrast, generation-long longitudinal data allow readings on the sequence of past circumstances uncolored by later outcomes. Such data are uniquely useful for studying the causes and consequences of naturally occurring decisions and transitions. Thus, as longitudinal studies continue, quantitative analysis is becoming feasible about such questions as: How are the decisions of individuals affected by parental experience? Which aspects of early decisions constrain later opportunities? And how does detailed background experience leave its imprint? Studies like the two-decade-long PSID are bringing within grasp a complete generational cycle of detailed data on fertility, work life, household structure, and income.

Advances in Longitudinal Designs

Large-scale longitudinal data collection projects are uniquely valuable as vehicles for testing and improving survey research methodology. In ways that lie beyond the scope of a cross-sectional survey, longitudinal studies can sometimes be designed—without significant detriment to their substantive interests—to facilitate the evaluation and upgrading of data quality; the analysis of relative costs and effectiveness of alternative techniques of inquiry; and the standardization or coordination of solutions to problems of method, concept, and measurement across different research domains.

Some areas of methodological improvement include discoveries about the impact of interview mode on response (mail, telephone, face-to-face); the effects of nonresponse on the representativeness of a sample (due to respondents’ refusal or interviewers’ failure to contact); the effects on behavior of continued participation over time in a sample survey; the value of alternative methods of adjusting for nonresponse and incomplete observations (such as imputation of missing data, variable case weighting); the impact on response of specifying different recall periods, varying the intervals between interviews, or changing the length of interviews; and the comparison and calibration of results obtained by longitudinal surveys, randomized field experiments, laboratory studies, onetime surveys, and administrative records.

It should be especially noted that incorporating improvements in methodology and data quality has been and will no doubt continue to be crucial to the growing success of longitudinal studies. Panel designs are intrinsically more vulnerable than other designs to statistical biases due to cumulative item non-response, sample attrition, time-in-sample effects, and error margins in repeated measures, all of which may produce exaggerated estimates of change. Over time, a panel that was initially representative may become much less representative of a population, not only because of attrition in the sample, but also because of changes in immigration patterns, age structure, and the like. Longitudinal studies are also subject to changes in scientific and societal contexts that may create uncontrolled drifts over time in the meaning of nominally stable questions or concepts as well as in the underlying behavior. Also, a natural tendency to expand over time the range of topics and thus the interview lengths, which increases the burdens on respondents, may lead to deterioration of data quality or relevance. Careful methodological research to understand and overcome these problems has been done, and continued work as a component of new longitudinal studies is certain to advance the overall state of the art.

Longitudinal studies are sometimes pressed for evidence they are not designed to produce: for example, in important public policy questions concerning the impact of government programs in such areas as health promotion, disease prevention, or criminal justice. By using research designs that combine field experiments (with randomized assignment to program and control conditions) and longitudinal surveys, one can capitalize on the strongest merits of each: the experimental component provides stronger evidence for casual statements that are critical for evaluating programs and for illuminating some fundamental theories; the longitudinal component helps in the estimation of long-term program effects and their attenuation. Coupling experiments to ongoing longitudinal studies is not often feasible, given the multiple constraints of not disrupting the survey, developing all the complicated arrangements that go into a large-scale field experiment, and having the populations of interest overlap in useful ways. Yet opportunities to join field experiments to surveys are of great importance. Coupled studies can produce vital knowledge about the empirical conditions under which the results of longitudinal surveys turn out to be similar to—or divergent from—those produced by randomized field experiments. A pattern of divergence and similarity has begun to emerge in coupled studies; additional cases are needed to understand why some naturally occurring social processes and longitudinal design features seem to approximate formal random allocation and others do not. The methodological implications of such new knowledge go well beyond program evaluation and survey research. These findings bear directly on the confidence scientists—and others—can have in conclusions from observational studies of complex behavioral and social processes, particularly ones that cannot be controlled or simulated within the confines of a laboratory environment.

Memory and the Framing of Questions

A very important opportunity to improve survey methods lies in the reduction of nonsampling error due to questionnaire context, phrasing of questions, and, generally, the semantic and social-psychological aspects of surveys. Survey data are particularly affected by the fallibility of human memory and the sensitivity of respondents to the framework in which a question is asked. This sensitivity is especially strong for certain types of attitudinal and opinion questions. Efforts are now being made to bring survey specialists into closer contact with researchers working on memory function, knowledge representation, and language in order to uncover and reduce this kind of error.

Memory for events is often inaccurate, biased toward what respondents believe to be true—or should be true—about the world. In many cases in which data are based on recollection, improvements can be achieved by shifting to techniques of structured interviewing and calibrated forms of memory elicitation, such as specifying recent, brief time periods (for example, in the last seven days) within which respondents recall certain types of events with acceptable accuracy.

• “Taking things altogether, how would you describe your marriage? Would you say that your marriage is very happy, pretty happy, or not too happy?”
• “Taken altogether how would you say things are these days—would you say you are very happy, pretty happy, or not too happy?”

Presenting this sequence in both directions on different forms showed that the order affected answers to the general happiness question but did not change the marital happiness question: responses to the specific issue swayed subsequent responses to the general one, but not vice versa. The explanations for and implications of such order effects on the many kinds of questions and sequences that can be used are not simple matters. Further experimentation on the design of survey instruments promises not only to improve the accuracy and reliability of survey research, but also to advance understanding of how people think about and evaluate their behavior from day to day.

Comparative Designs

Both experiments and surveys involve interventions or questions by the scientist, who then records and analyzes the responses. In contrast, many bodies of social and behavioral data of considerable value are originally derived from records or collections that have accumulated for various nonscientific reasons, quite often administrative in nature, in firms, churches, military organizations, and governments at all levels. Data of this kind can sometimes be subjected to careful scrutiny, summary, and inquiry by historians and social scientists, and statistical methods have increasingly been used to develop and evaluate inferences drawn from such data. Some of the main comparative approaches are cross-national aggregate comparisons, selective comparison of a limited number of cases, and historical case studies.

Among the more striking problems facing the scientist using such data are the vast differences in what has been recorded by different agencies whose behavior is being compared (this is especially true for parallel agencies in different nations), the highly unrepresentative or idiosyncratic sampling that can occur in the collection of such data, and the selective preservation and destruction of records. Means to overcome these problems form a substantial methodological research agenda in comparative research. An example of the method of cross-national aggregative comparisons is found in investigations by political scientists and sociologists of the factors that underlie differences in the vitality of institutions of political democracy in different societies. Some investigators have stressed the existence of a large middle class, others the level of education of a population, and still others the development of systems of mass communication. In cross-national aggregate comparisons, a large number of nations are arrayed according to some measures of political democracy and then attempts are made to ascertain the strength of correlations between these and the other variables. In this line of analysis it is possible to use a variety of statistical cluster and regression techniques to isolate and assess the possible impact of certain variables on the institutions under study. While this kind of research is cross-sectional in character, statements about historical processes are often invoked to explain the correlations.

More limited selective comparisons, applied by many of the classic theorists, involve asking similar kinds of questions but over a smaller range of societies. Why did democracy develop in such different ways in America, France, and England? Why did northeastern Europe develop rational bourgeois capitalism, in contrast to the Mediterranean and Asian nations? Modern scholars have turned their attention to explaining, for example, differences among types of fascism between the two World Wars, and similarities and differences among modern state welfare systems, using these comparisons to unravel the salient causes. The questions asked in these instances are inevitably historical ones.

Historical case studies involve only one nation or region, and so they may not be geographically comparative. However, insofar as they involve tracing the transformation of a society’s major institutions and the role of its main shaping events, they involve a comparison of different periods of a nation’s or a region’s history. The goal of such comparisons is to give a systematic account of the relevant differences. Sometimes, particularly with respect to the ancient societies, the historical record is very sparse, and the methods of history and archaeology mesh in the reconstruction of complex social arrangements and patterns of change on the basis of few fragments.

Like all research designs, comparative ones have distinctive vulnerabilities and advantages: One of the main advantages of using comparative designs is that they greatly expand the range of data, as well as the amount of variation in those data, for study. Consequently, they allow for more encompassing explanations and theories that can relate highly divergent outcomes to one another in the same framework. They also contribute to reducing any cultural biases or tendencies toward parochialism among scientists studying common human phenomena.

One main vulnerability in such designs arises from the problem of achieving comparability. Because comparative study involves studying societies and other units that are dissimilar from one another, the phenomena under study usually occur in very different contexts—so different that in some cases what is called an event in one society cannot really be regarded as the same type of event in another. For example, a vote in a Western democracy is different from a vote in an Eastern bloc country, and a voluntary vote in the United States means something different from a compulsory vote in Australia. These circumstances make for interpretive difficulties in comparing aggregate rates of voter turnout in different countries.

The problem of achieving comparability appears in historical analysis as well. For example, changes in laws and enforcement and recording procedures over time change the definition of what is and what is not a crime, and for that reason it is difficult to compare the crime rates over time. Comparative researchers struggle with this problem continually, working to fashion equivalent measures; some have suggested the use of different measures (voting, letters to the editor, street demonstration) in different societies for common variables (political participation), to try to take contextual factors into account and to achieve truer comparability.

A second vulnerability is controlling variation. Traditional experiments make conscious and elaborate efforts to control the variation of some factors and thereby assess the causal significance of others. In surveys as well as experiments, statistical methods are used to control sources of variation and assess suspected causal significance. In comparative and historical designs, this kind of control is often difficult to attain because the sources of variation are many and the number of cases few. Scientists have made efforts to approximate such control in these cases of “many variables, small N.” One is the method of paired comparisons. If an investigator isolates 15 American cities in which racial violence has been recurrent in the past 30 years, for example, it is helpful to match them with 15 cities of similar population size, geographical region, and size of minorities—such characteristics are controls—and then search for systematic differences between the two sets of cities. Another method is to select, for comparative purposes, a sample of societies that resemble one another in certain critical ways, such as size, common language, and common level of development, thus attempting to hold these factors roughly constant, and then seeking explanations among other factors in which the sampled societies differ from one another.

Ethnographic Designs

Traditionally identified with anthropology, ethnographic research designs are playing increasingly significant roles in most of the behavioral and social sciences. The core of this methodology is participant-observation, in which a researcher spends an extended period of time with the group under study, ideally mastering the local language, dialect, or special vocabulary, and participating in as many activities of the group as possible. This kind of participant-observation is normally coupled with extensive open-ended interviewing, in which people are asked to explain in depth the rules, norms, practices, and beliefs through which (from their point of view) they conduct their lives. A principal aim of ethnographic study is to discover the premises on which those rules, norms, practices, and beliefs are built.

The use of ethnographic designs by anthropologists has contributed significantly to the building of knowledge about social and cultural variation. And while these designs continue to center on certain long-standing features—extensive face-to-face experience in the community, linguistic competence, participation, and open-ended interviewing—there are newer trends in ethnographic work. One major trend concerns its scale. Ethnographic methods were originally developed largely for studying small-scale groupings known variously as village, folk, primitive, preliterate, or simple societies. Over the decades, these methods have increasingly been applied to the study of small groups and networks within modern (urban, industrial, complex) society, including the contemporary United States. The typical subjects of ethnographic study in modern society are small groups or relatively small social networks, such as outpatient clinics, medical schools, religious cults and churches, ethnically distinctive urban neighborhoods, corporate offices and factories, and government bureaus and legislatures.

As anthropologists moved into the study of modern societies, researchers in other disciplines—particularly sociology, psychology, and political science—began using ethnographic methods to enrich and focus their own insights and findings. At the same time, studies of large-scale structures and processes have been aided by the use of ethnographic methods, since most large-scale changes work their way into the fabric of community, neighborhood, and family, affecting the daily lives of people. Ethnographers have studied, for example, the impact of new industry and new forms of labor in “backward” regions; the impact of state-level birth control policies on ethnic groups; and the impact on residents in a region of building a dam or establishing a nuclear waste dump. Ethnographic methods have also been used to study a number of social processes that lend themselves to its particular techniques of observation and interview—processes such as the formation of class and racial identities, bureaucratic behavior, legislative coalitions and outcomes, and the formation and shifting of consumer tastes.

Advances in structured interviewing (see above) have proven especially powerful in the study of culture. Techniques for understanding kinship systems, concepts of disease, color terminologies, ethnobotany, and ethnozoology have been radically transformed and strengthened by coupling new interviewing methods with modem measurement and scaling techniques (see below). These techniques have made possible more precise comparisons among cultures and identification of the most competent and expert persons within a culture. The next step is to extend these methods to study the ways in which networks of propositions (such as boys like sports, girls like babies) are organized to form belief systems. Much evidence suggests that people typically represent the world around them by means of relatively complex cognitive models that involve interlocking propositions. The techniques of scaling have been used to develop models of how people categorize objects, and they have great potential for further development, to analyze data pertaining to cultural propositions.

Ideological Systems

Perhaps the most fruitful area for the application of ethnographic methods in recent years has been the systematic study of ideologies in modern society. Earlier studies of ideology were in small-scale societies that were rather homogeneous. In these studies researchers could report on a single culture, a uniform system of beliefs and values for the society as a whole. Modern societies are much more diverse both in origins and number of subcultures, related to different regions, communities, occupations, or ethnic groups. Yet these subcultures and ideologies share certain underlying assumptions or at least must find some accommodation with the dominant value and belief systems in the society.

The challenge is to incorporate this greater complexity of structure and process into systematic descriptions and interpretations. One line of work carried out by researchers has tried to track the ways in which ideologies are created, transmitted, and shared among large populations that have traditionally lacked the social mobility and communications technologies of the West. This work has concentrated on large-scale civilizations such as China, India, and Central America. Gradually, the focus has generalized into a concern with the relationship between the great traditions—the central lines of cosmopolitan Confucian, Hindu, or Mayan culture, including aesthetic standards, irrigation technologies, medical systems, cosmologies and calendars, legal codes, poetic genres, and religious doctrines and rites—and the little traditions, those identified with rural, peasant communities. How are the ideological doctrines and cultural values of the urban elites, the great traditions, transmitted to local communities? How are the little traditions, the ideas from the more isolated, less literate, and politically weaker groups in society, transmitted to the elites?

India and southern Asia have been fruitful areas for ethnographic research on these questions. The great Hindu tradition was present in virtually all local contexts through the presence of high-caste individuals in every community. It operated as a pervasive standard of value for all members of society, even in the face of strong little traditions. The situation is surprisingly akin to that of modern, industrialized societies. The central research questions are the degree and the nature of penetration of dominant ideology, even in groups that appear marginal and subordinate and have no strong interest in sharing the dominant value system. In this connection the lowest and poorest occupational caste—the untouchables—serves as an ultimate test of the power of ideology and cultural beliefs to unify complex hierarchical social systems.

Historical Reconstruction

Another current trend in ethnographic methods is its convergence with archival methods. One joining point is the application of descriptive and interpretative procedures used by ethnographers to reconstruct the cultures that created historical documents, diaries, and other records, to interview history, so to speak. For example, a revealing study showed how the Inquisition in the Italian countryside between the 1570s and 1640s gradually worked subtle changes in an ancient fertility cult in peasant communities; the peasant beliefs and rituals assimilated many elements of witchcraft after learning them from their persecutors. A good deal of social history—particularly that of the family—has drawn on discoveries made in the ethnographic study of primitive societies. As described in Chapter 4 , this particular line of inquiry rests on a marriage of ethnographic, archival, and demographic approaches.

Other lines of ethnographic work have focused on the historical dimensions of nonliterate societies. A strikingly successful example in this kind of effort is a study of head-hunting. By combining an interpretation of local oral tradition with the fragmentary observations that were made by outside observers (such as missionaries, traders, colonial officials), historical fluctuations in the rate and significance of head-hunting were shown to be partly in response to such international forces as the great depression and World War II. Researchers are also investigating the ways in which various groups in contemporary societies invent versions of traditions that may or may not reflect the actual history of the group. This process has been observed among elites seeking political and cultural legitimation and among hard-pressed minorities (for example, the Basque in Spain, the Welsh in Great Britain) seeking roots and political mobilization in a larger society.

Ethnography is a powerful method to record, describe, and interpret the system of meanings held by groups and to discover how those meanings affect the lives of group members. It is a method well adapted to the study of situations in which people interact with one another and the researcher can interact with them as well, so that information about meanings can be evoked and observed. Ethnography is especially suited to exploration and elucidation of unsuspected connections; ideally, it is used in combination with other methods—experimental, survey, or comparative—to establish with precision the relative strengths and weaknesses of such connections. By the same token, experimental, survey, and comparative methods frequently yield connections, the meaning of which is unknown; ethnographic methods are a valuable way to determine them.

• Models for Representing Phenomena

The objective of any science is to uncover the structure and dynamics of the phenomena that are its subject, as they are exhibited in the data. Scientists continuously try to describe possible structures and ask whether the data can, with allowance for errors of measurement, be described adequately in terms of them. Over a long time, various families of structures have recurred throughout many fields of science; these structures have become objects of study in their own right, principally by statisticians, other methodological specialists, applied mathematicians, and philosophers of logic and science. Methods have evolved to evaluate the adequacy of particular structures to account for particular types of data. In the interest of clarity we discuss these structures in this section and the analytical methods used for estimation and evaluation of them in the next section, although in practice they are closely intertwined.

A good deal of mathematical and statistical modeling attempts to describe the relations, both structural and dynamic, that hold among variables that are presumed to be representable by numbers. Such models are applicable in the behavioral and social sciences only to the extent that appropriate numerical measurement can be devised for the relevant variables. In many studies the phenomena in question and the raw data obtained are not intrinsically numerical, but qualitative, such as ethnic group identifications. The identifying numbers used to code such questionnaire categories for computers are no more than labels, which could just as well be letters or colors. One key question is whether there is some natural way to move from the qualitative aspects of such data to a structural representation that involves one of the well-understood numerical or geometric models or whether such an attempt would be inherently inappropriate for the data in question. The decision as to whether or not particular empirical data can be represented in particular numerical or more complex structures is seldom simple, and strong intuitive biases or a priori assumptions about what can and cannot be done may be misleading.

Recent decades have seen rapid and extensive development and application of analytical methods attuned to the nature and complexity of social science data. Examples of nonnumerical modeling are increasing. Moreover, the widespread availability of powerful computers is probably leading to a qualitative revolution, it is affecting not only the ability to compute numerical solutions to numerical models, but also to work out the consequences of all sorts of structures that do not involve numbers at all. The following discussion gives some indication of the richness of past progress and of future prospects although it is by necessity far from exhaustive.

In describing some of the areas of new and continuing research, we have organized this section on the basis of whether the representations are fundamentally probabilistic or not. A further useful distinction is between representations of data that are highly discrete or categorical in nature (such as whether a person is male or female) and those that are continuous in nature (such as a person’s height). Of course, there are intermediate cases involving both types of variables, such as color stimuli that are characterized by discrete hues (red, green) and a continuous luminance measure. Probabilistic models lead very naturally to questions of estimation and statistical evaluation of the correspondence between data and model. Those that are not probabilistic involve additional problems of dealing with and representing sources of variability that are not explicitly modeled. At the present time, scientists understand some aspects of structure, such as geometries, and some aspects of randomness, as embodied in probability models, but do not yet adequately understand how to put the two together in a single unified model. Table 5-1 outlines the way we have organized this discussion and shows where the examples in this section lie.

A Classification of Structural Models.

Probability Models

Some behavioral and social sciences variables appear to be more or less continuous, for example, utility of goods, loudness of sounds, or risk associated with uncertain alternatives. Many other variables, however, are inherently categorical, often with only two or a few values possible: for example, whether a person is in or out of school, employed or not employed, identifies with a major political party or political ideology. And some variables, such as moral attitudes, are typically measured in research with survey questions that allow only categorical responses. Much of the early probability theory was formulated only for continuous variables; its use with categorical variables was not really justified, and in some cases it may have been misleading. Recently, very significant advances have been made in how to deal explicitly with categorical variables. This section first describes several contemporary approaches to models involving categorical variables, followed by ones involving continuous representations.

Log-Linear Models for Categorical Variables

Many recent models for analyzing categorical data of the kind usually displayed as counts (cell frequencies) in multidimensional contingency tables are subsumed under the general heading of log-linear models, that is, linear models in the natural logarithms of the expected counts in each cell in the table. These recently developed forms of statistical analysis allow one to partition variability due to various sources in the distribution of categorical attributes, and to isolate the effects of particular variables or combinations of them.

Present log-linear models were first developed and used by statisticians and sociologists and then found extensive application in other social and behavioral sciences disciplines. When applied, for instance, to the analysis of social mobility, such models separate factors of occupational supply and demand from other factors that impede or propel movement up and down the social hierarchy. With such models, for example, researchers discovered the surprising fact that occupational mobility patterns are strikingly similar in many nations of the world (even among disparate nations like the United States and most of the Eastern European socialist countries), and from one time period to another, once allowance is made for differences in the distributions of occupations. The log-linear and related kinds of models have also made it possible to identify and analyze systematic differences in mobility among nations and across time. As another example of applications, psychologists and others have used log-linear models to analyze attitudes and their determinants and to link attitudes to behavior. These methods have also diffused to and been used extensively in the medical and biological sciences.

Regression Models for Categorical Variables

Models that permit one variable to be explained or predicted by means of others, called regression models, are the workhorses of much applied statistics; this is especially true when the dependent (explained) variable is continuous. For a two-valued dependent variable, such as alive or dead, models and approximate theory and computational methods for one explanatory variable were developed in biometry about 50 years ago. Computer programs able to handle many explanatory variables, continuous or categorical, are readily available today. Even now, however, the accuracy of the approximate theory on given data is an open question.

Using classical utility theory, economists have developed discrete choice models that turn out to be somewhat related to the log-linear and categorical regression models. Models for limited dependent variables, especially those that cannot take on values above or below a certain level (such as weeks unemployed, number of children, and years of schooling) have been used profitably in economics and in some other areas. For example, censored normal variables (called tobits in economics), in which observed values outside certain limits are simply counted, have been used in studying decisions to go on in school. It will require further research and development to incorporate information about limited ranges of variables fully into the main multivariate methodologies. In addition, with respect to the assumptions about distribution and functional form conventionally made in discrete response models, some new methods are now being developed that show promise of yielding reliable inferences without making unrealistic assumptions; further research in this area promises significant progress.

One problem arises from the fact that many of the categorical variables collected by the major data bases are ordered. For example, attitude surveys frequently use a 3-, 5-, or 7-point scale (from high to low) without specifying numerical intervals between levels. Social class and educational levels are often described by ordered categories. Ignoring order information, which many traditional statistical methods do, may be inefficient or inappropriate, but replacing the categories by successive integers or other arbitrary scores may distort the results. (For additional approaches to this question, see sections below on ordered structures.) Regression-like analysis of ordinal categorical variables is quite well developed, but their multivariate analysis needs further research. New log-bilinear models have been proposed, but to date they deal specifically with only two or three categorical variables. Additional research extending the new models, improving computational algorithms, and integrating the models with work on scaling promise to lead to valuable new knowledge.

Models for Event Histories

Event-history studies yield the sequence of events that respondents to a survey sample experience over a period of time; for example, the timing of marriage, childbearing, or labor force participation. Event-history data can be used to study educational progress, demographic processes (migration, fertility, and mortality), mergers of firms, labor market behavior, and even riots, strikes, and revolutions. As interest in such data has grown, many researchers have turned to models that pertain to changes in probabilities over time to describe when and how individuals move among a set of qualitative states.

Much of the progress in models for event-history data builds on recent developments in statistics and biostatistics for life-time, failure-time, and hazard models. Such models permit the analysis of qualitative transitions in a population whose members are undergoing partially random organic deterioration, mechanical wear, or other risks over time. With the increased complexity of event-history data that are now being collected, and the extension of event-history data bases over very long periods of time, new problems arise that cannot be effectively handled by older types of analysis. Among the problems are repeated transitions, such as between unemployment and employment or marriage and divorce; more than one time variable (such as biological age, calendar time, duration in a stage, and time exposed to some specified condition); latent variables (variables that are explicitly modeled even though not observed); gaps in the data; sample attrition that is not randomly distributed over the categories; and respondent difficulties in recalling the exact timing of events.

Models for Multiple-Item Measurement

For a variety of reasons, researchers typically use multiple measures (or multiple indicators) to represent theoretical concepts. Sociologists, for example, often rely on two or more variables (such as occupation and education) to measure an individual’s socioeconomic position; educational psychologists ordinarily measure a student’s ability with multiple test items. Despite the fact that the basic observations are categorical, in a number of applications this is interpreted as a partitioning of something continuous. For example, in test theory one thinks of the measures of both item difficulty and respondent ability as continuous variables, possibly multidimensional in character.

Classical test theory and newer item-response theories in psychometrics deal with the extraction of information from multiple measures. Testing, which is a major source of data in education and other areas, results in millions of test items stored in archives each year for purposes ranging from college admissions to job-training programs for industry. One goal of research on such test data is to be able to make comparisons among persons or groups even when different test items are used. Although the information collected from each respondent is intentionally incomplete in order to keep the tests short and simple, item-response techniques permit researchers to reconstitute the fragments into an accurate picture of overall group proficiencies. These new methods provide a better theoretical handle on individual differences, and they are expected to be extremely important in developing and using tests. For example, they have been used in attempts to equate different forms of a test given in successive waves during a year, a procedure made necessary in large-scale testing programs by legislation requiring disclosure of test-scoring keys at the time results are given.

An example of the use of item-response theory in a significant research effort is the National Assessment of Educational Progress (NAEP). The goal of this project is to provide accurate, nationally representative information on the average (rather than individual) proficiency of American children in a wide variety of academic subjects as they progress through elementary and secondary school. This approach is an improvement over the use of trend data on university entrance exams, because NAEP estimates of academic achievements (by broad characteristics such as age, grade, region, ethnic background, and so on) are not distorted by the self-selected character of those students who seek admission to college, graduate, and professional programs.

Item-response theory also forms the basis of many new psychometric instruments, known as computerized adaptive testing, currently being implemented by the U.S. military services and under additional development in many testing organizations. In adaptive tests, a computer program selects items for each examinee based upon the examinee’s success with previous items. Generally, each person gets a slightly different set of items and the equivalence of scale scores is established by using item-response theory. Adaptive testing can greatly reduce the number of items needed to achieve a given level of measurement accuracy.

Nonlinear, Nonadditive Models

Virtually all statistical models now in use impose a linearity or additivity assumption of some kind, sometimes after a nonlinear transformation of variables. Imposing these forms on relationships that do not, in fact, possess them may well result in false descriptions and spurious effects. Unwary users, especially of computer software packages, can easily be misled. But more realistic nonlinear and nonadditive multivariate models are becoming available. Extensive use with empirical data is likely to force many changes and enhancements in such models and stimulate quite different approaches to nonlinear multivariate analysis in the next decade.

Geometric and Algebraic Models

Geometric and algebraic models attempt to describe underlying structural relations among variables. In some cases they are part of a probabilistic approach, such as the algebraic models underlying regression or the geometric representations of correlations between items in a technique called factor analysis. In other cases, geometric and algebraic models are developed without explicitly modeling the element of randomness or uncertainty that is always present in the data. Although this latter approach to behavioral and social sciences problems has been less researched than the probabilistic one, there are some advantages in developing the structural aspects independent of the statistical ones. We begin the discussion with some inherently geometric representations and then turn to numerical representations for ordered data.

Although geometry is a huge mathematical topic, little of it seems directly applicable to the kinds of data encountered in the behavioral and social sciences. A major reason is that the primitive concepts normally used in geometry—points, lines, coincidence—do not correspond naturally to the kinds of qualitative observations usually obtained in behavioral and social sciences contexts. Nevertheless, since geometric representations are used to reduce bodies of data, there is a real need to develop a deeper understanding of when such representations of social or psychological data make sense. Moreover, there is a practical need to understand why geometric computer algorithms, such as those of multidimensional scaling, work as well as they apparently do. A better understanding of the algorithms will increase the efficiency and appropriateness of their use, which becomes increasingly important with the widespread availability of scaling programs for microcomputers.

Over the past 50 years several kinds of well-understood scaling techniques have been developed and widely used to assist in the search for appropriate geometric representations of empirical data. The whole field of scaling is now entering a critical juncture in terms of unifying and synthesizing what earlier appeared to be disparate contributions. Within the past few years it has become apparent that several major methods of analysis, including some that are based on probabilistic assumptions, can be unified under the rubric of a single generalized mathematical structure. For example, it has recently been demonstrated that such diverse approaches as nonmetric multidimensional scaling, principal-components analysis, factor analysis, correspondence analysis, and log-linear analysis have more in common in terms of underlying mathematical structure than had earlier been realized.

Nonmetric multidimensional scaling is a method that begins with data about the ordering established by subjective similarity (or nearness) between pairs of stimuli. The idea is to embed the stimuli into a metric space (that is, a geometry with a measure of distance between points) in such a way that distances between points corresponding to stimuli exhibit the same ordering as do the data. This method has been successfully applied to phenomena that, on other grounds, are known to be describable in terms of a specific geometric structure; such applications were used to validate the procedures. Such validation was done, for example, with respect to the perception of colors, which are known to be describable in terms of a particular three-dimensional structure known as the Euclidean color coordinates. Similar applications have been made with Morse code symbols and spoken phonemes. The technique is now used in some biological and engineering applications, as well as in some of the social sciences, as a method of data exploration and simplification.

One question of interest is how to develop an axiomatic basis for various geometries using as a primitive concept an observable such as the subject’s ordering of the relative similarity of one pair of stimuli to another, which is the typical starting point of such scaling. The general task is to discover properties of the qualitative data sufficient to ensure that a mapping into the geometric structure exists and, ideally, to discover an algorithm for finding it. Some work of this general type has been carried out: for example, there is an elegant set of axioms based on laws of color matching that yields the three-dimensional vectorial representation of color space. But the more general problem of understanding the conditions under which the multidimensional scaling algorithms are suitable remains unsolved. In addition, work is needed on understanding more general, non-Euclidean spatial models.

Ordered Factorial Systems

One type of structure common throughout the sciences arises when an ordered dependent variable is affected by two or more ordered independent variables. This is the situation to which regression and analysis-of-variance models are often applied; it is also the structure underlying the familiar physical identities, in which physical units are expressed as products of the powers of other units (for example, energy has the unit of mass times the square of the unit of distance divided by the square of the unit of time).

There are many examples of these types of structures in the behavioral and social sciences. One example is the ordering of preference of commodity bundles—collections of various amounts of commodities—which may be revealed directly by expressions of preference or indirectly by choices among alternative sets of bundles. A related example is preferences among alternative courses of action that involve various outcomes with differing degrees of uncertainty; this is one of the more thoroughly investigated problems because of its potential importance in decision making. A psychological example is the trade-off between delay and amount of reward, yielding those combinations that are equally reinforcing. In a common, applied kind of problem, a subject is given descriptions of people in terms of several factors, for example, intelligence, creativity, diligence, and honesty, and is asked to rate them according to a criterion such as suitability for a particular job.

In all these cases and a myriad of others like them the question is whether the regularities of the data permit a numerical representation. Initially, three types of representations were studied quite fully: the dependent variable as a sum, a product, or a weighted average of the measures associated with the independent variables. The first two representations underlie some psychological and economic investigations, as well as a considerable portion of physical measurement and modeling in classical statistics. The third representation, averaging, has proved most useful in understanding preferences among uncertain outcomes and the amalgamation of verbally described traits, as well as some physical variables.

For each of these three cases—adding, multiplying, and averaging—researchers know what properties or axioms of order the data must satisfy for such a numerical representation to be appropriate. On the assumption that one or another of these representations exists, and using numerical ratings by subjects instead of ordering, a scaling technique called functional measurement (referring to the function that describes how the dependent variable relates to the independent ones) has been developed and applied in a number of domains. What remains problematic is how to encompass at the ordinal level the fact that some random error intrudes into nearly all observations and then to show how that randomness is represented at the numerical level; this continues to be an unresolved and challenging research issue.

During the past few years considerable progress has been made in understanding certain representations inherently different from those just discussed. The work has involved three related thrusts. The first is a scheme of classifying structures according to how uniquely their representation is constrained. The three classical numerical representations are known as ordinal, interval, and ratio scale types. For systems with continuous numerical representations and of scale type at least as rich as the ratio one, it has been shown that only one additional type can exist. A second thrust is to accept structural assumptions, like factorial ones, and to derive for each scale the possible functional relations among the independent variables. And the third thrust is to develop axioms for the properties of an order relation that leads to the possible representations. Much is now known about the possible nonadditive representations of both the multifactor case and the one where stimuli can be combined, such as combining sound intensities.

Closely related to this classification of structures is the question: What statements, formulated in terms of the measures arising in such representations, can be viewed as meaningful in the sense of corresponding to something empirical? Statements here refer to any scientific assertions, including statistical ones, formulated in terms of the measures of the variables and logical and mathematical connectives. These are statements for which asserting truth or falsity makes sense. In particular, statements that remain invariant under certain symmetries of structure have played an important role in classical geometry, dimensional analysis in physics, and in relating measurement and statistical models applied to the same phenomenon. In addition, these ideas have been used to construct models in more formally developed areas of the behavioral and social sciences, such as psychophysics. Current research has emphasized the communality of these historically independent developments and is attempting both to uncover systematic, philosophically sound arguments as to why invariance under symmetries is as important as it appears to be and to understand what to do when structures lack symmetry, as, for example, when variables have an inherent upper bound.

Many subjects do not seem to be correctly represented in terms of distances in continuous geometric space. Rather, in some cases, such as the relations among meanings of words—which is of great interest in the study of memory representations—a description in terms of tree-like, hierarchial structures appears to be more illuminating. This kind of description appears appropriate both because of the categorical nature of the judgments and the hierarchial, rather than trade-off, nature of the structure. Individual items are represented as the terminal nodes of the tree, and groupings by different degrees of similarity are shown as intermediate nodes, with the more general groupings occurring nearer the root of the tree. Clustering techniques, requiring considerable computational power, have been and are being developed. Some successful applications exist, but much more refinement is anticipated.

Network Models

Several other lines of advanced modeling have progressed in recent years, opening new possibilities for empirical specification and testing of a variety of theories. In social network data, relationships among units, rather than the units themselves, are the primary objects of study: friendships among persons, trade ties among nations, cocitation clusters among research scientists, interlocking among corporate boards of directors. Special models for social network data have been developed in the past decade, and they give, among other things, precise new measures of the strengths of relational ties among units. A major challenge in social network data at present is to handle the statistical dependence that arises when the units sampled are related in complex ways.

• Statistical Inference and Analysis

As was noted earlier, questions of design, representation, and analysis are intimately intertwined. Some issues of inference and analysis have been discussed above as related to specific data collection and modeling approaches. This section discusses some more general issues of statistical inference and advances in several current approaches to them.

Causal Inference

Behavioral and social scientists use statistical methods primarily to infer the effects of treatments, interventions, or policy factors. Previous chapters included many instances of causal knowledge gained this way. As noted above, the large experimental study of alternative health care financing discussed in Chapter 2 relied heavily on statistical principles and techniques, including randomization, in the design of the experiment and the analysis of the resulting data. Sophisticated designs were necessary in order to answer a variety of questions in a single large study without confusing the effects of one program difference (such as prepayment or fee for service) with the effects of another (such as different levels of deductible costs), or with effects of unobserved variables (such as genetic differences). Statistical techniques were also used to ascertain which results applied across the whole enrolled population and which were confined to certain subgroups (such as individuals with high blood pressure) and to translate utilization rates across different programs and types of patients into comparable overall dollar costs and health outcomes for alternative financing options.

A classical experiment, with systematic but randomly assigned variation of the variables of interest (or some reasonable approach to this), is usually considered the most rigorous basis from which to draw such inferences. But random samples or randomized experimental manipulations are not always feasible or ethically acceptable. Then, causal inferences must be drawn from observational studies, which, however well designed, are less able to ensure that the observed (or inferred) relationships among variables provide clear evidence on the underlying mechanisms of cause and effect.

Certain recurrent challenges have been identified in studying causal inference. One challenge arises from the selection of background variables to be measured, such as the sex, nativity, or parental religion of individuals in a comparative study of how education affects occupational success. The adequacy of classical methods of matching groups in background variables and adjusting for covariates needs further investigation. Statistical adjustment of biases linked to measured background variables is possible, but it can become complicated. Current work in adjustment for selectivity bias is aimed at weakening implausible assumptions, such as normality, when carrying out these adjustments. Even after adjustment has been made for the measured background variables, other, unmeasured variables are almost always still affecting the results (such as family transfers of wealth or reading habits). Analyses of how the conclusions might change if such unmeasured variables could be taken into account is essential in attempting to make causal inferences from an observational study, and systematic work on useful statistical models for such sensitivity analyses is just beginning.

The third important issue arises from the necessity for distinguishing among competing hypotheses when the explanatory variables are measured with different degrees of precision. Both the estimated size and significance of an effect are diminished when it has large measurement error, and the coefficients of other correlated variables are affected even when the other variables are measured perfectly. Similar results arise from conceptual errors, when one measures only proxies for a theoretical construct (such as years of education to represent amount of learning). In some cases, there are procedures for simultaneously or iteratively estimating both the precision of complex measures and their effect on a particular criterion.

Although complex models are often necessary to infer causes, once their output is available, it should be translated into understandable displays for evaluation. Results that depend on the accuracy of a multivariate model and the associated software need to be subjected to appropriate checks, including the evaluation of graphical displays, group comparisons, and other analyses.

New Statistical Techniques

Internal resampling.

One of the great contributions of twentieth-century statistics was to demonstrate how a properly drawn sample of sufficient size, even if it is only a tiny fraction of the population of interest, can yield very good estimates of most population characteristics. When enough is known at the outset about the characteristic in question—for example, that its distribution is roughly normal—inference from the sample data to the population as a whole is straightforward, and one can easily compute measures of the certainty of inference, a common example being the 95 percent confidence interval around an estimate. But population shapes are sometimes unknown or uncertain, and so inference procedures cannot be so simple. Furthermore, more often than not, it is difficult to assess even the degree of uncertainty associated with complex data and with the statistics needed to unravel complex social and behavioral phenomena.

Internal resampling methods attempt to assess this uncertainty by generating a number of simulated data sets similar to the one actually observed. The definition of similar is crucial, and many methods that exploit different types of similarity have been devised. These methods provide researchers the freedom to choose scientifically appropriate procedures and to replace procedures that are valid under assumed distributional shapes with ones that are not so restricted. Flexible and imaginative computer simulation is the key to these methods. For a simple random sample, the “bootstrap” method repeatedly resamples the obtained data (with replacement) to generate a distribution of possible data sets. The distribution of any estimator can thereby be simulated and measures of the certainty of inference be derived. The “jackknife” method repeatedly omits a fraction of the data and in this way generates a distribution of possible data sets that can also be used to estimate variability. These methods can also be used to remove or reduce bias. For example, the ratio-estimator, a statistic that is commonly used in analyzing sample surveys and censuses, is known to be biased, and the jackknife method can usually remedy this defect. The methods have been extended to other situations and types of analysis, such as multiple regression.

There are indications that under relatively general conditions, these methods, and others related to them, allow more accurate estimates of the uncertainty of inferences than do the traditional ones that are based on assumed (usually, normal) distributions when that distributional assumption is unwarranted. For complex samples, such internal resampling or subsampling facilitates estimating the sampling variances of complex statistics.

An older and simpler, but equally important, idea is to use one independent subsample in searching the data to develop a model and at least one separate subsample for estimating and testing a selected model. Otherwise, it is next to impossible to make allowances for the excessively close fitting of the model that occurs as a result of the creative search for the exact characteristics of the sample data—characteristics that are to some degree random and will not predict well to other samples.

Robust Techniques

Many technical assumptions underlie the analysis of data. Some, like the assumption that each item in a sample is drawn independently of other items, can be weakened when the data are sufficiently structured to admit simple alternative models, such as serial correlation. Usually, these models require that a few parameters be estimated. Assumptions about shapes of distributions, normality being the most common, have proved to be particularly important, and considerable progress has been made in dealing with the consequences of different assumptions.

More recently, robust techniques have been designed that permit sharp, valid discriminations among possible values of parameters of central tendency for a wide variety of alternative distributions by reducing the weight given to occasional extreme deviations. It turns out that by giving up, say, 10 percent of the discrimination that could be provided under the rather unrealistic assumption of normality, one can greatly improve performance in more realistic situations, especially when unusually large deviations are relatively common.

These valuable modifications of classical statistical techniques have been extended to multiple regression, in which procedures of iterative reweighting can now offer relatively good performance for a variety of underlying distributional shapes. They should be extended to more general schemes of analysis.

In some contexts—notably the most classical uses of analysis of variance—the use of adequate robust techniques should help to bring conventional statistical practice closer to the best standards that experts can now achieve.

Many Interrelated Parameters

In trying to give a more accurate representation of the real world than is possible with simple models, researchers sometimes use models with many parameters, all of which must be estimated from the data. Classical principles of estimation, such as straightforward maximum-likelihood, do not yield reliable estimates unless either the number of observations is much larger than the number of parameters to be estimated or special designs are used in conjunction with strong assumptions. Bayesian methods do not draw a distinction between fixed and random parameters, and so may be especially appropriate for such problems.

A variety of statistical methods have recently been developed that can be interpreted as treating many of the parameters as or similar to random quantities, even if they are regarded as representing fixed quantities to be estimated. Theory and practice demonstrate that such methods can improve the simpler fixed-parameter methods from which they evolved, especially when the number of observations is not large relative to the number of parameters. Successful applications include college and graduate school admissions, where quality of previous school is treated as a random parameter when the data are insufficient to separately estimate it well. Efforts to create appropriate models using this general approach for small-area estimation and undercount adjustment in the census are important potential applications.

Missing Data

In data analysis, serious problems can arise when certain kinds of (quantitative or qualitative) information is partially or wholly missing. Various approaches to dealing with these problems have been or are being developed. One of the methods developed recently for dealing with certain aspects of missing data is called multiple imputation: each missing value in a data set is replaced by several values representing a range of possibilities, with statistical dependence among missing values reflected by linkage among their replacements. It is currently being used to handle a major problem of incompatibility between the 1980 and previous Bureau of Census public-use tapes with respect to occupation codes. The extension of these techniques to address such problems as nonresponse to income questions in the Current Population Survey has been examined in exploratory applications with great promise.

Computer Packages and Expert Systems

The development of high-speed computing and data handling has fundamentally changed statistical analysis. Methodologies for all kinds of situations are rapidly being developed and made available for use in computer packages that may be incorporated into interactive expert systems. This computing capability offers the hope that much data analyses will be more carefully and more effectively done than previously and that better strategies for data analysis will move from the practice of expert statisticians, some of whom may not have tried to articulate their own strategies, to both wide discussion and general use.

But powerful tools can be hazardous, as witnessed by occasional dire misuses of existing statistical packages. Until recently the only strategies available were to train more expert methodologists or to train substantive scientists in more methodology, but without the updating of their training it tends to become outmoded. Now there is the opportunity to capture in expert systems the current best methodological advice and practice. If that opportunity is exploited, standard methodological training of social scientists will shift to emphasizing strategies in using good expert systems—including understanding the nature and importance of the comments it provides—rather than in how to patch together something on one’s own. With expert systems, almost all behavioral and social scientists should become able to conduct any of the more common styles of data analysis more effectively and with more confidence than all but the most expert do today. However, the difficulties in developing expert systems that work as hoped for should not be underestimated. Human experts cannot readily explicate all of the complex cognitive network that constitutes an important part of their knowledge. As a result, the first attempts at expert systems were not especially successful (as discussed in Chapter 1 ). Additional work is expected to overcome these limitations, but it is not clear how long it will take.

Exploratory Analysis and Graphic Presentation

The formal focus of much statistics research in the middle half of the twentieth century was on procedures to confirm or reject precise, a priori hypotheses developed in advance of collecting data—that is, procedures to determine statistical significance. There was relatively little systematic work on realistically rich strategies for the applied researcher to use when attacking real-world problems with their multiplicity of objectives and sources of evidence. More recently, a species of quantitative detective work, called exploratory data analysis, has received increasing attention. In this approach, the researcher seeks out possible quantitative relations that may be present in the data. The techniques are flexible and include an important component of graphic representations. While current techniques have evolved for single responses in situations of modest complexity, extensions to multiple responses and to single responses in more complex situations are now possible.

Graphic and tabular presentation is a research domain in active renaissance, stemming in part from suggestions for new kinds of graphics made possible by computer capabilities, for example, hanging histograms and easily assimilated representations of numerical vectors. Research on data presentation has been carried out by statisticians, psychologists, cartographers, and other specialists, and attempts are now being made to incorporate findings and concepts from linguistics, industrial and publishing design, aesthetics, and classification studies in library science. Another influence has been the rapidly increasing availability of powerful computational hardware and software, now available even on desktop computers. These ideas and capabilities are leading to an increasing number of behavioral experiments with substantial statistical input. Nonetheless, criteria of good graphic and tabular practice are still too much matters of tradition and dogma, without adequate empirical evidence or theoretical coherence. To broaden the respective research outlooks and vigorously develop such evidence and coherence, extended collaborations between statistical and mathematical specialists and other scientists are needed, a major objective being to understand better the visual and cognitive processes (see Chapter 1 ) relevant to effective use of graphic or tabular approaches.

Combining Evidence

Combining evidence from separate sources is a recurrent scientific task, and formal statistical methods for doing so go back 30 years or more. These methods include the theory and practice of combining tests of individual hypotheses, sequential design and analysis of experiments, comparisons of laboratories, and Bayesian and likelihood paradigms.

There is now growing interest in more ambitious analytical syntheses, which are often called meta-analyses. One stimulus has been the appearance of syntheses explicitly combining all existing investigations in particular fields, such as prison parole policy, classroom size in primary schools, cooperative studies of therapeutic treatments for coronary heart disease, early childhood education interventions, and weather modification experiments. In such fields, a serious approach to even the simplest question—how to put together separate estimates of effect size from separate investigations—leads quickly to difficult and interesting issues. One issue involves the lack of independence among the available studies, due, for example, to the effect of influential teachers on the research projects of their students. Another issue is selection bias, because only some of the studies carried out, usually those with “significant” findings, are available and because the literature search may not find out all relevant studies that are available. In addition, experts agree, although informally, that the quality of studies from different laboratories and facilities differ appreciably and that such information probably should be taken into account. Inevitably, the studies to be included used different designs and concepts and controlled or measured different variables, making it difficult to know how to combine them.

Rich, informal syntheses, allowing for individual appraisal, may be better than catch-all formal modeling, but the literature on formal meta-analytic models is growing and may be an important area of discovery in the next decade, relevant both to statistical analysis per se and to improved syntheses in the behavioral and social and other sciences.

• Opportunities and Needs

This chapter has cited a number of methodological topics associated with behavioral and social sciences research that appear to be particularly active and promising at the present time. As throughout the report, they constitute illustrative examples of what the committee believes to be important areas of research in the coming decade. In this section we describe recommendations for an additional $16 million annually to facilitate both the development of methodologically oriented research and, equally important, its communication throughout the research community.

Methodological studies, including early computer implementations, have for the most part been carried out by individual investigators with small teams of colleagues or students. Occasionally, such research has been associated with quite large substantive projects, and some of the current developments of computer packages, graphics, and expert systems clearly require large, organized efforts, which often lie at the boundary between grant-supported work and commercial development. As such research is often a key to understanding complex bodies of behavioral and social sciences data, it is vital to the health of these sciences that research support continue on methods relevant to problems of modeling, statistical analysis, representation, and related aspects of behavioral and social sciences data. Researchers and funding agencies should also be especially sympathetic to the inclusion of such basic methodological work in large experimental and longitudinal studies. Additional funding for work in this area, both in terms of individual research grants on methodological issues and in terms of augmentation of large projects to include additional methodological aspects, should be provided largely in the form of investigator-initiated project grants.

Ethnographic and comparative studies also typically rely on project grants to individuals and small groups of investigators. While this type of support should continue, provision should also be made to facilitate the execution of studies using these methods by research teams and to provide appropriate methodological training through the mechanisms outlined below.

Overall, we recommend an increase of $4 million in the level of investigator-initiated grant support for methodological work. An additional $1 million should be devoted to a program of centers for methodological research.

Many of the new methods and models described in the chapter, if and when adopted to any large extent, will demand substantially greater amounts of research devoted to appropriate analysis and computer implementation. New user interfaces and numerical algorithms will need to be designed and new computer programs written. And even when generally available methods (such as maximum-likelihood) are applicable, model application still requires skillful development in particular contexts. Many of the familiar general methods that are applied in the statistical analysis of data are known to provide good approximations when sample sizes are sufficiently large, but their accuracy varies with the specific model and data used. To estimate the accuracy requires extensive numerical exploration. Investigating the sensitivity of results to the assumptions of the models is important and requires still more creative, thoughtful research. It takes substantial efforts of these kinds to bring any new model on line, and the need becomes increasingly important and difficult as statistical models move toward greater realism, usefulness, complexity, and availability in computer form. More complexity in turn will increase the demand for computational power. Although most of this demand can be satisfied by increasingly powerful desktop computers, some access to mainframe and even supercomputers will be needed in selected cases. We recommend an additional $4 million annually to cover the growth in computational demands for model development and testing.

Interaction and cooperation between the developers and the users of statistical and mathematical methods need continual stimulation—both ways. Efforts should be made to teach new methods to a wider variety of potential users than is now the case. Several ways appear effective for methodologists to communicate to empirical scientists: running summer training programs for graduate students, faculty, and other researchers; encouraging graduate students, perhaps through degree requirements, to make greater use of the statistical, mathematical, and methodological resources at their own or affiliated universities; associating statistical and mathematical research specialists with large-scale data collection projects; and developing statistical packages that incorporate expert systems in applying the methods.

Methodologists, in turn, need to become more familiar with the problems actually faced by empirical scientists in the laboratory and especially in the field. Several ways appear useful for communication in this direction: encouraging graduate students in methodological specialties, perhaps through degree requirements, to work directly on empirical research; creating postdoctoral fellowships aimed at integrating such specialists into ongoing data collection projects; and providing for large data collection projects to engage relevant methodological specialists. In addition, research on and development of statistical packages and expert systems should be encouraged to involve the multidisciplinary collaboration of experts with experience in statistical, computer, and cognitive sciences.

A final point has to do with the promise held out by bringing different research methods to bear on the same problems. As our discussions of research methods in this and other chapters have emphasized, different methods have different powers and limitations, and each is designed especially to elucidate one or more particular facets of a subject. An important type of interdisciplinary work is the collaboration of specialists in different research methodologies on a substantive issue, examples of which have been noted throughout this report. If more such research were conducted cooperatively, the power of each method pursued separately would be increased. To encourage such multidisciplinary work, we recommend increased support for fellowships, research workshops, and training institutes.

Funding for fellowships, both pre-and postdoctoral, should be aimed at giving methodologists experience with substantive problems and at upgrading the methodological capabilities of substantive scientists. Such targeted fellowship support should be increased by $4 million annually, of which $3 million should be for predoctoral fellowships emphasizing the enrichment of methodological concentrations. The new support needed for research workshops is estimated to be $1 million annually. And new support needed for various kinds of advanced training institutes aimed at rapidly diffusing new methodological findings among substantive scientists is estimated to be $2 million annually.

• Cite this Page National Research Council; Division of Behavioral and Social Sciences and Education; Commission on Behavioral and Social Sciences and Education; Committee on Basic Research in the Behavioral and Social Sciences; Gerstein DR, Luce RD, Smelser NJ, et al., editors. The Behavioral and Social Sciences: Achievements and Opportunities. Washington (DC): National Academies Press (US); 1988. 5, Methods of Data Collection, Representation, and Analysis.
• PDF version of this title (16M)

In this Page

Other titles in this collection.

• The National Academies Collection: Reports funded by National Institutes of Health

Recent Activity

• Methods of Data Collection, Representation, and Analysis - The Behavioral and So... Methods of Data Collection, Representation, and Analysis - The Behavioral and Social Sciences: Achievements and Opportunities

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

13.5 Research Process: Making Notes, Synthesizing Information, and Keeping a Research Log

Learning outcomes.

By the end of this section, you will be able to:

• Employ the methods and technologies commonly used for research and communication within various fields.
• Practice and apply strategies such as interpretation, synthesis, response, and critique to compose texts that integrate the writer’s ideas with those from appropriate sources.
• Analyze and make informed decisions about intellectual property based on the concepts that motivate them.
• Apply citation conventions systematically.

As you conduct research, you will work with a range of “texts” in various forms, including sources and documents from online databases as well as images, audio, and video files from the Internet. You may also work with archival materials and with transcribed and analyzed primary data. Additionally, you will be taking notes and recording quotations from secondary sources as you find materials that shape your understanding of your topic and, at the same time, provide you with facts and perspectives. You also may download articles as PDFs that you then annotate. Like many other students, you may find it challenging to keep so much material organized, accessible, and easy to work with while you write a major research paper. As it does for many of those students, a research log for your ideas and sources will help you keep track of the scope, purpose, and possibilities of any research project.

A research log is essentially a journal in which you collect information, ask questions, and monitor the results. Even if you are completing the annotated bibliography for Writing Process: Informing and Analyzing , keeping a research log is an effective organizational tool. Like Lily Tran’s research log entry, most entries have three parts: a part for notes on secondary sources, a part for connections to the thesis or main points, and a part for your own notes or questions. Record source notes by date, and allow room to add cross-references to other entries.

Summary of Assignment: Research Log

Your assignment is to create a research log similar to the student model. You will use it for the argumentative research project assigned in Writing Process: Integrating Research to record all secondary source information: your notes, complete publication data, relation to thesis, and other information as indicated in the right-hand column of the sample entry.

Another Lens. A somewhat different approach to maintaining a research log is to customize it to your needs or preferences. You can apply shading or color coding to headers, rows, and/or columns in the three-column format (for colors and shading). Or you can add columns to accommodate more information, analysis, synthesis, or commentary, formatting them as you wish. Consider adding a column for questions only or one for connections to other sources. Finally, consider a different visual format , such as one without columns. Another possibility is to record some of your comments and questions so that you have an aural rather than a written record of these.

Writing Center

At this point, or at any other point during the research and writing process, you may find that your school’s writing center can provide extensive assistance. If you are unfamiliar with the writing center, now is a good time to pay your first visit. Writing centers provide free peer tutoring for all types and phases of writing. Discussing your research with a trained writing center tutor can help you clarify, analyze, and connect ideas as well as provide feedback on works in progress.

Quick Launch: Beginning Questions

You may begin your research log with some open pages in which you freewrite, exploring answers to the following questions. Although you generally would do this at the beginning, it is a process to which you likely will return as you find more information about your topic and as your focus changes, as it may during the course of your research.

• What information have I found so far?
• What do I still need to find?
• Where am I most likely to find it?

These are beginning questions. Like Lily Tran, however, you will come across general questions or issues that a quick note or freewrite may help you resolve. The key to this section is to revisit it regularly. Written answers to these and other self-generated questions in your log clarify your tasks as you go along, helping you articulate ideas and examine supporting evidence critically. As you move further into the process, consider answering the following questions in your freewrite:

• What evidence looks as though it best supports my thesis?
• What evidence challenges my working thesis?
• How is my thesis changing from where it started?

Creating the Research Log

As you gather source material for your argumentative research paper, keep in mind that the research is intended to support original thinking. That is, you are not writing an informational report in which you simply supply facts to readers. Instead, you are writing to support a thesis that shows original thinking, and you are collecting and incorporating research into your paper to support that thinking. Therefore, a research log, whether digital or handwritten, is a great way to keep track of your thinking as well as your notes and bibliographic information.

In the model below, Lily Tran records the correct MLA bibliographic citation for the source. Then, she records a note and includes the in-text citation here to avoid having to retrieve this information later. Perhaps most important, Tran records why she noted this information—how it supports her thesis: The human race must turn to sustainable food systems that provide healthy diets with minimal environmental impact, starting now . Finally, she makes a note to herself about an additional visual to include in the final paper to reinforce the point regarding the current pressure on food systems. And she connects the information to other information she finds, thus cross-referencing and establishing a possible synthesis. Use a format similar to that in Table 13.4 to begin your own research log.

Types of Research Notes

Taking good notes will make the research process easier by enabling you to locate and remember sources and use them effectively. While some research projects requiring only a few sources may seem easily tracked, research projects requiring more than a few sources are more effectively managed when you take good bibliographic and informational notes. As you gather evidence for your argumentative research paper, follow the descriptions and the electronic model to record your notes. You can combine these with your research log, or you can use the research log for secondary sources and your own note-taking system for primary sources if a division of this kind is helpful. Either way, be sure to include all necessary information.

Bibliographic Notes

These identify the source you are using. When you locate a useful source, record the information necessary to find that source again. It is important to do this as you find each source, even before taking notes from it. If you create bibliographic notes as you go along, then you can easily arrange them in alphabetical order later to prepare the reference list required at the end of formal academic papers. If your instructor requires you to use MLA formatting for your essay, be sure to record the following information:

• Title of source
• Title of container (larger work in which source is included)
• Other contributors
• Publication date

When using MLA style with online sources, also record the following information:

• Date of original publication
• Date of access
• DOI (A DOI, or digital object identifier, is a series of digits and letters that leads to the location of an online source. Articles in journals are often assigned DOIs to ensure that the source can be located, even if the URL changes. If your source is listed with a DOI, use that instead of a URL.)

It is important to understand which documentation style your instructor will require you to use. Check the Handbook for MLA Documentation and Format and APA Documentation and Format styles . In addition, you can check the style guide information provided by the Purdue Online Writing Lab .

Informational Notes

These notes record the relevant information found in your sources. When writing your essay, you will work from these notes, so be sure they contain all the information you need from every source you intend to use. Also try to focus your notes on your research question so that their relevance is clear when you read them later. To avoid confusion, work with separate entries for each piece of information recorded. At the top of each entry, identify the source through brief bibliographic identification (author and title), and note the page numbers on which the information appears. Also helpful is to add personal notes, including ideas for possible use of the information or cross-references to other information. As noted in Writing Process: Integrating Research , you will be using a variety of formats when borrowing from sources. Below is a quick review of these formats in terms of note-taking processes. By clarifying whether you are quoting directly, paraphrasing, or summarizing during these stages, you can record information accurately and thus take steps to avoid plagiarism.

Direct Quotations, Paraphrases, and Summaries

A direct quotation is an exact duplication of the author’s words as they appear in the original source. In your notes, put quotation marks around direct quotations so that you remember these words are the author’s, not yours. One advantage of copying exact quotations is that it allows you to decide later whether to include a quotation, paraphrase, or summary. ln general, though, use direct quotations only when the author’s words are particularly lively or persuasive.

A paraphrase is a restatement of the author’s words in your own words. Paraphrase to simplify or clarify the original author’s point. In your notes, use paraphrases when you need to record details but not exact words.

A summary is a brief condensation or distillation of the main point and most important details of the original source. Write a summary in your own words, with facts and ideas accurately represented. A summary is useful when specific details in the source are unimportant or irrelevant to your research question. You may find you can summarize several paragraphs or even an entire article or chapter in just a few sentences without losing useful information. It is a good idea to note when your entry contains a summary to remind you later that it omits detailed information. See Writing Process Integrating Research for more detailed information and examples of quotations, paraphrases, and summaries and when to use them.

Other Systems for Organizing Research Logs and Digital Note-Taking

Students often become frustrated and at times overwhelmed by the quantity of materials to be managed in the research process. If this is your first time working with both primary and secondary sources, finding ways to keep all of the information in one place and well organized is essential.

Because gathering primary evidence may be a relatively new practice, this section is designed to help you navigate the process. As mentioned earlier, information gathered in fieldwork is not cataloged, organized, indexed, or shelved for your convenience. Obtaining it requires diligence, energy, and planning. Online resources can assist you with keeping a research log. Your college library may have subscriptions to tools such as Todoist or EndNote. Consult with a librarian to find out whether you have access to any of these. If not, use something like the template shown in Figure 13.8 , or another like it, as a template for creating your own research notes and organizational tool. You will need to have a record of all field research data as well as the research log for all secondary sources.

As an Amazon Associate we earn from qualifying purchases.

This book may not be used in the training of large language models or otherwise be ingested into large language models or generative AI offerings without OpenStax's permission.

Want to cite, share, or modify this book? This book uses the Creative Commons Attribution License and you must attribute OpenStax.

Access for free at https://openstax.org/books/writing-guide/pages/1-unit-introduction

• Authors: Michelle Bachelor Robinson, Maria Jerskey, featuring Toby Fulwiler
• Publisher/website: OpenStax
• Book title: Writing Guide with Handbook
• Publication date: Dec 21, 2021
• Location: Houston, Texas
• Book URL: https://openstax.org/books/writing-guide/pages/1-unit-introduction
• Section URL: https://openstax.org/books/writing-guide/pages/13-5-research-process-making-notes-synthesizing-information-and-keeping-a-research-log

© Dec 19, 2023 OpenStax. Textbook content produced by OpenStax is licensed under a Creative Commons Attribution License . The OpenStax name, OpenStax logo, OpenStax book covers, OpenStax CNX name, and OpenStax CNX logo are not subject to the Creative Commons license and may not be reproduced without the prior and express written consent of Rice University.

A Note on Reflection

This course is heavy on reflection. Every module, I’ll be asking you to reflect on your learning during the previous week, and you’ll also be reflecting on your search processes in your research log as part of  the research scenario project (more on that later). Additionally, at the start of every textbook reading, you’ll be given a series of statements from the Information Literacy Reflection Tool that asks you to notice and appreciate your current approaches to gathering and using information.

It’s not an accident that you’re going to be reflecting so much! When students describe their learning, how it changed, and how it is relevant to future experiences through reflection opportunities, it’s been shown to help information stick past the immediate lesson and apply what was learned to situations beyond the classroom examples. Reflection has even been shown to improve performance! So let’s get reflecting!

Information Literacy Reflection Tool

This tool was created by a group of Oregon librarians to help students think about their information processes. Read through each statement, noting your 1-6 ranking and then calculate your total in the question below.

Introduction to Finding Information Copyright © by Kirsten Hostetler is licensed under a Creative Commons Attribution 4.0 International License , except where otherwise noted.

Performance Measurement Tool Box and Reporting System for Research Programs and Projects (2008)

Chapter: chapter 2 - information gathering and analysis.

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

7 CHAPTER 2 – INFORMATION GATHERING AND ANALYSIS A significant task of this project was the gathering and analysis of performance measurement information, particularly as it may regard research project and program performance evaluation. Information was gathered from both the literature and a set of three national surveys. The gathered information was critical to the selection of standard performance measures and the tools to be included in the RPM System. LITERATURE SEARCH Of the numerous documents obtained and reviewed during the process of this project, of particular note was NCHRP Synthesis 300, Performance Measures for Research, Development and Technology Programs (2). In this synthesis Sabol captured the state of practice in research performance measurement among state transportation agencies in 2000. One of the noteworthy findings, which may have been a primary motivator for the development of this project, was that there was not yet a commonly accepted set of research performance measures for use by state transportation agencies. Hatry similarly points out the necessity of well-understood and commonly accepted performance measures, and that the first step in being successful in performance measurement is establishing common definitions among the various programs within an agency (3). It is logical that if a first and critical step in performance measurement within an organization is establishing sound and commonly accepted definitions, then the importance of achieving this goal between state transportation agencies in the AASHTO organization will also be critical although undoubtedly more difficult. The definitions and methods provided in this study will hopefully become the basis for more common understanding and coordinated use of research performance measures throughout the member agencies of AASHTO. Other noteworthy findings of NCHRP Synthesis 300 that this study addresses or incorporates in some manner include: • research performance measures should have a focus on agency strategic goals, • a need exists for additional quantitative research performance measures, and • there is a lack of performance measures monitoring program-level benefits. Another interesting perspective was found in the 2004 report by the international scanning team which visited Australia, Canada, Japan, and New Zealand to study transportation performance measurement (4). While this group did not specifically pursue research-related performance measurement information, some of their observations are quite applicable to this project. The scanning team reported finding examples where performance measurement was much more interwoven into decision making than is usually found in the United States. They also noted greater understanding of the critical difference between outcome and output measures among the transportation officials with whom they met. In Japan, they found that a small core set of measures focused on critically important areas of transportation operations had been identified at the national level, with the prefectures (states) given the ability to create additional measures uniquely desired or needed for their circumstances. In the United States, AASHTO Standing Committee on Research (SCOR) and AASHTO RAC appear to be well positioned to similarly select a “critical few” research performance measures, and then to provide leadership and encouragement to all AASHTO member agencies to utilize them. Another possibility is that a lead states team could be formed to champion research performance measurement and, more importantly, to establish

8 common language, definitions, and direction for research performance measurement. Without a group to take the lead, much progress may be made in instituting research performance measurement at the state level without the commonality needed in the gathered information to make it of value at the national level when justification and support are needed for federal research funding requests. A series of information sources acquired and found generally useful in the execution of this project is provided in the bibliography. In addition, the creation of the Resource Collection of statistical information sources, to be discussed in detail later in this report, was a major accomplishment of the literature search efforts during this project. NATIONAL SURVEYS One of the important objectives of this project was to assemble a useful and practical collection of research performance measures for the primary use of state transportation agencies. To accomplish this task, it was necessary to assure that recently developed transportation research performance metrics, perhaps not yet documented in the literature, were also identified. The research team developed and distributed three electronic surveys for gathering current information. The survey audiences were AASHTO RAC members, AASHTO agency administrators who are members of the AASHTO Standing Committee on Highways (SCOH), and a group of federal and private industry research managers and executives. The survey template presented 20 different performance measures which had been identified through a review of literature relevant to transportation research and other associated research areas. These performance measures included outcome, output, efficiency, resource allocation, and stakeholder metrics. Each of the three surveys contained the same list of performance measures, and feedback concerning each of the performance measures was requested. The following is a list of the performance measures presented as part of the surveys: 1. return on investment or benefit-cost ratio; 2. lives saved; 3. construction, maintenance, and operations cost savings; 4. reduction in crashes; 5. reduction in system delays; 6. positive environmental impact; 7. quality of life enhancement; 8. safety enhancement; 9. level of knowledge increased; 10. management tool or policy improvement; 11. public image enhancement; 12. technical practices or standards upgrades; 13. leadership; 14. percent of projects/products implemented; 15. percent of projects completed on time; 16. percent of projects completed within budget; 17. number of contractors; 18. number of contractor partnerships; 19. percent of satisfied customers; and 20. contribution to the overall mission of the department.

9 The surveys requested information about the organization’s experience with each of these research performance measures and the perceived value of each if it were used in their organization. A numerical means for rating the perceived value of individual performance measures was provided in the surveys, thereby allowing a more objective analysis of survey responses. Numerical ratings were on a scale of one to five with one indicating that the respondent believed that the performance measure would offer little value in their environment, while a rating of five indicated that the performance measure would be extremely valuable in their environment. The three survey instruments are provided in Appendix C. In addition to rating and commenting on the performance measures provided in the surveys, the respondents were encouraged to identify and describe any other research-related performance measures they had utilized in their agencies. The overall survey response was considered reasonably good. Forty AASHTO RAC members returned the survey, while twenty-four agency administrator responses were obtained from the AASHTO states. A slightly lower response was obtained from the survey of other federal and private industry research managers and executives. Twenty responses were obtained from this group. The organizations surveyed and responding are shown in Appendix D. After calculating the mean perceived-value rating for each performance measure by each of the surveyed groups, the performance measures were placed in rank order according to these mean scores as shown in Table 1. The performance measures are presented in this table beginning at the top with the measure with the highest average rating under each survey group. The number in the left-hand column, then, represents the ranking for the performance measure listed in that row for each of the three survey groups.

10 Table 1. Mean Perceived-Value Ratings for Performance Measures, by Survey Group RAC Members Transportation Agency Administrators Federal & Private Industry Managers Performance Measure M ea n Pe rc ei ve d- V al ue R at in g Performance Measure M ea n Pe rc ei ve d- V al ue R at in g Performance Measure M ea n Pe rc ei ve d- V al ue R at in g 1 Construction, maintenance, & operations cost savings 4.37 Lives saved 4.46 Return on investment or benefit-cost ratio 4.05 2 Percent of satisfied customers 4.13 Reduction in crashes 4.45 Reduction in system delays 3.86 3 Reduction in crashes 4.03 Return on investment or benefit-cost ratio 4.36 Construction, maintenance, & operations cost savings 3.81 4 Lives saved 4.00 Construction, maintenance, & operations cost savings 4.07 Reduction in crashes 3.81 5 Return on investment or benefit-cost ratio 3.91 Safety enhancement 3.89 Lives saved 3.67 6 Percent of projects/products implemented 3.68 Reduction in system delays 3.79 Percent of satisfied customers 3.67 7 Contribution to the overall mission of the department 3.68 Technical practices or standards upgraded 3.79 Contribution to the overall mission of the department 3.62 8 Safety enhancement 3.67 Percent of satisfied customers 3.61 Percent of projects completed within budget 3.52 9 Technical practices or standards upgraded 3.67 Positive environmental impact 3.36 Percent of projects/products implemented 3.38 10 Reduction in system delays 3.58 Contribution to the overall mission of the department 3.36 Management & policy improvement 3.33 11 Management & policy improvement 3.47 Management & policy improvement 3.21 Technical practices or standards upgraded 3.24 12 Positive environmental impact 3.35 Percent of projects/products implemented 3.18 Safety enhancement 3.14 13 Leadership 2.91 Public image enhancement 3.07 Leadership 3.00 14 Public image enhancement 2.82 Level of knowledge increased 3.04 Percent of projects completed on time 3.00 15 Level of knowledge increased 2.74 Quality of life enhancement 3.00 Positive environmental impact 2.91 16 Percent of projects completed on time 2.53 Leadership 2.96 Quality of life enhancement 2.71 17 Percent of projects completed within budget 2.42 Percent of projects completed within budget 2.96 Level of knowledge increased 2.67 18 Number of contractor partnerships 2.42 Percent of projects completed on time 2.89 Public image enhancement 2.06 19 Quality of life enhancement 2.28 Number of contractor partnerships 2.18 Number of contractor partnerships 1.95 20 Number of contractors 2.00 Number of contractors 2.11 Number of contractors 1.84

11 Table 2 compares the performance measure rankings identified in Table 1 for each performance measure included in the three surveys. Table 1 and Table 2 illustrate how the responses from each survey group were summarized for viewing during the process of selecting standard performance measures for the RPM System. It is interesting to note that three of the four highest perceived-value performance measures are the same for transportation agency administrators and the RAC members who manage their agency’s research program. These performance measures are: lives saved; reduction in crashes; and construction, maintenance, and operations cost savings. It is probably not coincidental that these three closely associate with the core mission of state transportation agencies. The importance of outcome measures which monitor the major results sought by an agency is a point made by Hatry, as he states that the mission statement and the primary objectives of an organization should be the starting place for creating outcome performance measures. Not only did survey respondents provide perceived-value ratings for performance measures, but they often provided optional comments about the performance measures with which they have had experience. This additional information significantly informed the analysis of the results. The comments related to individual performance measures were also summarized, and these are provided in Appendix E.

12 Table 2. Comparison of Performance Measure Rankings, by Survey Group Survey Group and Ranking Performance Measure RAC Members Agency Administrators Federal & Private Mean Ranking Lives saved 3 1 1 1.7 Reduction in crashes 2 3 3 2.7 Return on investment or cost- benefit ratio 5 2 2 3.0 Construction, maintenance, & operations cost savings 1 4 4 3.0 Safety enhancement 8 5 5 6.0 Percent of satisfied customers 4 8 8 6.7 Reduction in systems delays 9 6 6 7.0 Technical practices/standards upgraded 10 7 7 8.0 Contribution to the overall mission of the department 6 10 10 8.7 Positive environmental impact 12 9 9 10.0 Percent of projects/products implemented 7 12 12 10.3 Management tool or policy improvement 11 11 11 11.0 Level of knowledge increased 14 13 13 13.3 Leadership 13 14 14 13.7 Public image enhancement 15 15 15 15.0 Quality of life enhancement 19 16 16 17.0 Percent of projects completed on time 16 18 18 17.3 Percent of projects completed within budget 18 17 17 17.3 Number of contractor partnerships 17 19 19 18.3 Number of contractors 20 20 20 20.0

TRB’s National Cooperative Highway Research Program (NCHRP) Web-Only Document 127: Performance Measurement Tool Box and Reporting System for Research Programs and Projects explores the integration of standard performance measures and tools to assist users in implementing performance measures into the Research Performance Measurement (RPM) System.

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

How to Gather Information for Your Research Smartly

A graduate student who works with me once remarked that he “wasn’t very good at doing literature searches.” I replied that researchers often spend as much time in the library as they do in the lab, and that he had better hone his skills at gathering information if he wanted to make a career as a researcher. When it comes to gathering information for research papers or research programs, it’s important to be as efficient as possible in order to free up more time for actual research and writing . Here are some of the best ways I have found to gather information smartly.

Keyword Searches

It has never been easier to do literature searches if you know how to use the Boolean system for keywords—keyword1 “and” keyword2; plus keyword1 “or” keyword2. The former narrows a search that might otherwise generate too many responses to manage; the latter broadens a search when you are uncertain which keywords might be used in articles of interest. An author’s name might substitute for one or more keywords. There are many combinations using advanced search options and a little experimentation will give you a feel for what works best to pull up articles of interest.

Citation Searches

Guessing at keywords can be a hit or miss affair. Searching for citations is often a simpler way to locate references in an area of interest. If there is a key paper in an area, subsequent researchers will reference it in their papers. Searching for citations will generate publications that may be of interest, not only for research ideas but to include as references an eventual research paper . This can be a real shortcut when it comes to documenting past work. Of course, a really key paper will be cited so often that you may need to narrow the search with “and” keywords or pick a citation in a narrower area.

Abstracting services can be a time saver to keep up to date with the literature. The CA Selects service offered by Chemical Abstracts is one I have used. It offers current abstracts on thirty-three topics ranging from adhesives to zeolites. This is a cost effective way for a whole research group to stay up to date on several topics. In graduate school our group leader subscribed to five topics in areas of silicon, phosphorus, sulfur, and halogen chemistry. Each abstract would circulate by turn to each of ten group members, to be checked off and handed on to the next person. Our group leader touted CA Selects as a “relatively painless way to stay current in research,” and for people who are good at skimming abstracts I agree.

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

• Manuscripts & Grants
• Reporting Research

Finding Relevant Scholarly Research for Literature Review: How can we be systematic?

On an average, it takes 15 clicks for a researcher to find an article (which…

• AI in Academia

What If You Could Explore, Store, and Read Scholarly Research Data on a Single AI-based Platform?

“How am I supposed to organize all this information?” “How should I prioritize which paper…

How to Increase the Visibility of Your Research Paper With the Help of SEO

While there is no single, agreed-upon rubric for ranking research papers, various SEO strategies can…

• Publishing Research
• Selecting Journals

Useful Boolean Operators for Article Searches

Does your article search return thousands of articles, many irrelevant to your research? To increase…

• Career Corner
• Product & Service Reviews

Microsoft Academic 2.0: Is It Any Better?

Microsoft released Microsoft Academic 2.0 in 2016 as its new search engine for academic searches.…

Omnity: A New Multilingual Search Engine for Academics

Yewno for Education: A New Approach to Knowledge Discovery

Reaxys: Chemistry Search Engine by Elsevier

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

What should universities' stance be on AI tools in research and academic writing?

• Event Website Publish a modern and mobile friendly event website.
• Registration & Payments Collect registrations & online payments for your event.
• Abstract Management Collect and manage all your abstract submissions.
• Peer Reviews Easily distribute and manage your peer reviews.
• Conference Program Effortlessly build & publish your event program.
• Virtual Poster Sessions Host engaging virtual poster sessions.
• Customer Success Stories
• Wall of Love ❤️

How to Present Your Research (Guidelines and Tips)

Published on 01 Feb 2023

Presenting at a conference can be stressful, but can lead to many opportunities, which is why coming prepared is super beneficial.

The internet is full to the brim with tips for making a good presentation. From what you wear to how you stand to good slide design, there’s no shortage of advice to make any old presentation come to life. 

But, not all presentations are created equal. Research presentations, in particular, are unique. 

Communicating complex concepts to an audience with a varied range of awareness about your research topic can be tricky. A lack of guidance and preparation can ruin your chance to share important information with a conference community. This could mean lost opportunities in collaboration or funding or lost confidence in yourself and your work.

So, we’ve put together a list of tips with research presentations in mind. Here’s our top to-do’s when preparing to present your research.

Take every research presentation opportunity

The worst thing you could do for your research is to not present it at all. As intimidating as it can be to get up in front of an audience, you shouldn’t let that stop you from seizing a good opportunity to share your work with a wider community.

These contestants from the Vitae Three Minute Thesis Competition have some great advice to share on taking every possible chance to talk about your research. 

Double-check your research presentation guidelines

Before you get started on your presentation, double-check if you’ve been given guidelines for it. 

If you don’t have specific guidelines for the context of your presentation, we’ve put together a general outline to help you get started. It’s made with the assumption of a 10-15 minute presentation time. So, if you have longer to present, you can always extend important sections or talk longer on certain slides:

• Title Slide (1 slide) - This is a placeholder to give some visual interest and display the topic until your presentation begins.
• Short Introduction (2-3 slides) - This is where you pique the interest of your audience and establish the key questions your presentation covers. Give context to your study with a brief review of the literature (focus on key points, not a full review). If your study relates to any particularly relevant issues, mention it here to increase the audience's interest in the topic.
• Hypothesis (1 slide) - Clearly state your hypothesis.
• Description of Methods (2-3 slides) - Clearly, but briefly, summarize your study design including a clear description of the study population, the sample size and any instruments or manipulations to gather the data.
• Results and Data Interpretation (2-4 slides) - Illustrate your results through simple tables, graphs, and images. Remind the audience of your hypothesis and discuss your interpretation of the data/results.
• Conclusion (2-3 slides) - Further interpret your results. If you had any sources of error or difficulties with your methods, discuss them here and address how they could be (or were) improved. Discuss your findings as part of the bigger picture and connect them to potential further outcomes or areas of study.
• Closing (1 slide) - If anyone supported your research with guidance, awards, or funding, be sure to recognize their contribution. If your presentation includes a Q&A session, open the floor to questions.

Plan for about one minute for each slide of information that you have. Be sure that you don’t cram your slides with text (stick to bullet points and images to emphasize key points).

And, if you’re looking for more inspiration to help you in scripting an oral research presentation. University of Virginia has a helpful oral presentation outline script .

A PhD Student working on an upcoming oral presentation.

Put yourself in your listeners shoes

As mentioned in the intro, research presentations are unique because they deal with specialized topics and complicated concepts. There’s a good chance that a large section of your audience won’t have the same understanding of your topic area as you do. So, do your best to understand where your listeners are at and adapt your language/definitions to that.

There’s an increasing awareness around the importance of scientific communication. Comms experts have even started giving TED Talks on how to bridge the gap between science and the public (check out Talk Nerdy to Me ). A general communication tip is to find out what sort of audience will listen to your talk. Then, beware of using jargon and acronyms unless you're 100% certain that your audience knows what they mean. 

On the other end of the spectrum, you don’t want to underestimate your audience. Giving too much background or spending ages summarizing old work to a group of experts in the field would be a waste of valuable presentation time (and would put you at risk of losing your audience's interest). 

Finally, if you can, practice your presentation on someone with a similar level of topic knowledge to the audience you’ll be presenting to.

Use scientific storytelling in your presentation

In scenarios where it’s appropriate, crafting a story allows you to break free from the often rigid tone of scientific communications. It helps your brain hit the refresh button and observe your findings from a new perspective. Plus, it can be a lot of fun to do!

If you have a chance to use scientific storytelling in your presentation, take full advantage of it. The best way to weave a story for your audience into a presentation is by setting the scene during your introduction. As you set the context of your research, set the context of your story/example at the same time. Continue drawing those parallels as you present. Then, deliver the main message of the story (or the “Aha!”) moment during your presentation’s conclusion.

If delivered well, a good story will keep your audience on the edge of their seats and glued to your entire presentation.

Emphasize the “Why” (not the “How”) of your research

Along the same lines as using storytelling, it’s important to think of WHY your audience should care about your work. Find ways to connect your research to valuable outcomes in society. Take your individual points on each slide and bring things back to the bigger picture. Constantly remind your listeners how it’s all connected and why that’s important.

One helpful way to get in this mindset is to look back to the moment before you became an expert on your topic. What got you interested? What was the reason for asking your research question? And, what motivated you to power through all the hard work to come? Then, looking forward, think about what key takeaways were most interesting or surprised you the most. How can these be applied to impact positive change in your research field or the wider community?

Be picky about what you include

It’s tempting to discuss all the small details of your methods or findings. Instead, focus on the most important information and takeaways that you think your audience will connect with. Decide on these takeaways before you script your presentation so that you can set the scene properly and provide only the information that has an added value.

When it comes to choosing data to display in your presentation slides, keep it simple. Wherever possible, use visuals to communicate your findings as opposed to large tables filled with numbers. This article by Richard Chambers has some great tips on using visuals in your slides and graphs.

Hide your complex tables and data in additional slides

With the above tip in mind: Just because you don’t include data and tables in your main presentation slides, doesn’t mean you can’t keep them handy for reference. If there’s a Q&A session after your presentation (or if you’ll be sharing your slides to view on-demand after) one great trick is to include additional slides/materials after your closing slide. You can keep these in your metaphorical “back pocket” to refer to if a specific question is asked about a data set or method. They’re also handy for people viewing your presentation slides later that might want to do a deeper dive into your methods/results.

However, just because you have these extra slides doesn’t mean you shouldn’t make the effort to make that information more accessible. A research conference platform like Fourwaves allows presenters to attach supplementary materials (figures, posters, slides, videos and more) that conference participants can access anytime.

Leave your audience with (a few) questions

Curiosity is a good thing. Whether you have a Q&A session or not, you should want to leave your audience with a few key questions. The most important one:

“Where can I find out more?”

Obviously, it’s important to answer basic questions about your research context, hypothesis, methods, results, and interpretation. If you answer these while focusing on the “Why?” and weaving a good story, you’ll be setting the stage for an engaging Q&A session and/or some great discussions in the halls after your presentation. Just be sure that you have further links or materials ready to provide to those who are curious. 

Conclusion: The true expert in your research presentation

Throughout the entire process of scripting, creating your slides, and presenting, it’s important to remember that no one knows your research better than you do. If you’re nervous, remind yourself that the people who come to listen to your presentation are most likely there due to a genuine interest in your work. The pressure isn’t to connect with an uninterested audience - it’s to make your research more accessible and relevant for an already curious audience.

Finally, to practice what we preached in our last tip: If you’re looking to learn more about preparing for a research presentation, check out our articles on how to dress for a scientific conference and general conference presentation tips .

5 Best Event Registration Platforms for Your Next Conference

By having one software to organize registrations and submissions, a pediatric health center runs aro...

5 Essential Conference Apps for Your Event

In today’s digital age, the success of any conference hinges not just on the content and speakers bu...

Search form

• Table of Contents
• Troubleshooting Guide
• A Model for Getting Started
• Justice Action Toolkit
• Best Change Processes
• Databases of Best Practices
• Online Courses
• Ask an Advisor
• Subscribe to eNewsletter
• Community Stories
• YouTube Channel
• About the Tool Box
• How to Use the Tool Box
• Privacy Statement
• Workstation/Check Box Sign-In
• Online Training Courses
• Capacity Building Training
• Training Curriculum - Order Now
• Community Check Box Evaluation System
• Build Your Toolbox
• Facilitation of Community Processes
• Community Health Assessment and Planning
• Section 2. Information Gathering and Synthesis

Chapter 37 Sections

• Section 1. Choosing Questions and Planning the Evaluation
• Section 3. Data Collection: Designing an Observational System
• Section 4. Selecting an Appropriate Design for the Evaluation
• Section 5. Collecting and Analyzing Data
• Section 6. Gathering and Interpreting Ethnographic Information
• Section 7. Collecting and Using Archival Data
• Main Section

What do we mean by information gathering and synthesis?

Why gather and synthesize information, when should you gather and synthesize information, who should gather and synthesize information, how do you gather and synthesize information.

Suppose you wanted to design a house that used very little energy, took few resources to build and maintain, and was affordable for most families. You might have some original ideas about how this could be done, but you’d want to find out what ideas others had as well. You’d probably read about earth-bermed houses (houses that are built into a hillside or earth mound), solar panels or windmills for producing electricity, efficient insulating windows, waste-water recycling, and non-toxic building materials that reuse waste wood and metal. You’d talk to people who built or owned energy-efficient houses, to hear about the realities of living green. You’d learn about the barriers to some environmentally-friendly strategies, as well as ways to get around those barriers. There’s a huge amount of information out there, and it would make sense to gather as much of it as possible, so that you could put together the information, incorporate appropriate elements into your design, and get new ideas based on what’s already been done.

The same is true if you’re designing an intervention or program to deal with a community health or other issue, or an evaluation of that program. Others have also undoubtedly tried to address that issue, some with success and some without. Knowing what they did, how they did it, and what the results were can help you decide how to design your effort. You might be able to find a method here, and a technique elsewhere that all fit together into exactly the program that will suit the people and conditions in your community. Or you might realize that something you’d intended to do simply hasn’t worked in a number of other instances, and so wouldn’t be likely to work for you, either.

Gathering and using others’ ideas doesn’t mean that you can’t use your own or come up with something new. New ideas tend to come out of what others have attempted. Most artists start out imitating others before they develop their own styles. Einstein didn’t just chance on relativity; he was familiar with it because others had worked on it. You can usually innovate more effectively if you know what’s been tried.

This section looks at gathering all the information you can about your community issue and about attempts to address it, and putting that information together to design an evaluation to address your questions. Although this chapter is about evaluation, much of the material in these sections applies to planning the intervention (or program) and the evaluation: the two really can’t be separated.

An evaluation is a research project: we are trying to discover what works and under what conditions.  The steps for designing and using an evaluation – the subject of this chapter – are essentially the same as those for designing the program you’re evaluating.  The elements that you borrow from others’ successful efforts, and those that you create yourself, will give you an intervention and related evaluation questions. Although this section talks about program design, it also applies to the design of the evaluation.

Information gathering refers to gathering information about the issue you’re facing and the ways other organizations and communities have addressed it. The more information you have about the issue itself and the ways it has been approached, the more likely you are to be able to devise an effective program or intervention of your own.

There are obviously many sources of information, and they vary depending on what you’re looking for. In general, you can consult existing sources or look at “natural examples,” examples of actual programs and interventions that have addressed the issue. We’ll touch on where to find both here, and then go into more detail about them later in the section.

• Existing sources . This term refers to published material of various kinds that might shed light either on the issue or on attempts to deal with it.  These can be conveniently divided into scholarly publications, aimed primarily at researchers and the academic community; mass-market sources, written in a popular style and aimed at the general public; and statistical and demographic information published by various research organizations and government agencies.
• Natural examples . These are programs or interventions developed and tried in communities that have addressed your issue. Studying them can tell you what worked for them and what didn’t, and why. By giving you insight into how issues play out in your or other communities, they can provide nuts-and-bolts ideas about how to (or how not to) conduct a successful program or intervention. For the most part, information sources here are the people who are involved in efforts to address issues similar to yours, or those who can steer you to them. Additionally, there are a number of natural examples (such as single case studies) that have been written about descriptively in the literature of community psychology or public health that may be relevant to your work.

Synthesis is from the Greek; it means putting together. Its English meaning is the same: the putting together of something out of two or more different sources. Synthetic fabrics, for instance, are called that because they’re constructed from a number of different chemical building blocks.

In this section, we’re talking about ideas. Synthesis here refers to analyzing what you’ve learned from your information gathering, and constructing a coherent program or approach by taking ideas from a number of sources and putting them together to create something new that meets the needs of the community and population you’re working with.

Synthesizing in this way requires identifying the functional elements of each idea or program that you’ve looked at that seems to hold lessons for your work. Functional elements are the core components of each program – the methods, framework, activities, techniques, and other aspects – that make up the specific program you’re examining.  Once you’ve separated these parts out, you can put those that meet your needs together with what you’ve learned about the issue and your own ideas to build a program that speaks specifically to your situation.

As we’ve mentioned, the activities of information gathering and synthesis are needed both to create the original program and to develop an evaluation of it that will help you maintain and improve it.  The two really start in the same place, with what you think will address the issue – what shape the program or intervention should take, with whom it should be applied, and what behaviors or conditions it aims to change. This also informs what its short- and long-term goals should be, and by what means you’ll try to achieve those goals.  Once these are determined, they in turn determine your evaluation questions. You can’t construct an evaluation without knowing exactly what you’re trying to evaluate.

If you’re in the process of starting a program to address a community issue, such as violence or early childhood education, you probably know quite a bit about that issue already.  You’ve dealt with it, perhaps, in a variety of ways, and you have some pretty good ideas about what kind of program would work.  Why take the time and trouble, for you and for others engaged in a participatory planning effort, to read a lot of material written by others and to track down people who’ve run programs? If you’re inclined to think this way, there are a lot of good reasons why you should think again. Gathering information beforehand and putting together what you’ve learned could be the most important things you do to make your program effective.  Here’s why:

• It will help you avoid reinventing the wheel . A lot of different organizations have likely approached this issue before you.  Some might have been successful and some might not have, but all of them have probably learned something that would be useful to you in the process. You don’t have to make the same mistakes someone else did if you know about them, and you don’t have to make up something from scratch that may or may not work, when you have a model that has worked.

It’s certainly not a bad thing if you have some of the same good ideas that others have had, but it helps to know that they are good ideas. And there’s a chance that you might have some of the same bad ideas others have had, in which case it helps even more to know that they’re bad ideas.  It will save you a huge amount of trouble, and perhaps be the difference between creating a program that does its job well and one that fails miserably and disappears. Square wheels don’t roll – someone could have told you that.

• It will help you to gain a deep understanding of the issue so that you can address it properly . The first step in figuring out how to deal with an issue is to know what you’re dealing with. The better you understand it – its causes, how it occurs, how people react when they’re affected by it, what its consequences are for individuals and the community, and who can influence it – the more likely it is that you’ll be able to determine how to approach it.
• You need all the tools possible to create the best program you can . Foremost among the tools you need to plan and implement a program or intervention are information, information, and information. Just as with the issue itself, the more you know about what works for whom, how to make things happen, and how to establish or eliminate certain conditions, the more likely that you’ll be able to plan a successful program that addresses all aspects of the issue and leaves nothing to chance. Various kinds of professional and interpersonal skills may help you implement a program, but if what you’re implementing isn’t effective, it doesn’t matter how skillfully you carry it out.
• It’s likely that most solutions aren’t one size fits all. The more information you gather, the greater the variety of approaches, methods, and frameworks you’ll have to choose from. Putting together the right combination will help you to successfully address the particular needs of your community and population.
• It can help you to be culturally sensitive . Not only can you learn more about the culture(s) of the people you’re working with, but you can probably find a number of approaches that have worked with the cultural group you hope will benefit. Perhaps even more important, you can learn to avoid costly mistakes that may take a lot of time and effort – or be impossible – to repair.
• Knowing what’s been done in a variety of other circumstances and understanding the issue from a number of different viewpoints may give you new insights and new ideas for your program . As we discussed at the beginning of this section, new ideas seldom spring from nowhere.  They’re stimulated by your own experience and the ideas and experience – both good and bad, positive and negative – of others. Look to the experience of other fields, communities, and countries. The more different ideas you’re exposed to, and the more ways you can put them together, the greater chance there is that you’ll come up with something new that’s more effective than what’s gone before.

Information gathering and synthesis is crucial to the success of the program and to the relevance and effectiveness of the evaluation. It should start at the beginning of any effort, and contribute to the initial planning. It should also go on throughout the life of the program, so that you can continue to adjust by adding or changing program elements to enhance outcomes, and to generate new ideas.

Major adjustments should generally come at the end of an evaluation cycle, when you have solid information about what worked and what didn’t. That doesn’t mean that you can’t make smaller adjustments in the course of the program to improve results along the way.

There’s a tension here between continually changing a program to make it better and obtaining accurate evaluation results. If you change a method or activity in midstream, your evaluation will not be able to give you a clear assessment of its effectiveness. How much changing you do in the course of a program depends on your intent.  If your first responsibility is to find out what works best, so you can pass it on, then it’s important not to make changes until an evaluation has been completed.  If your primary responsibility is to the current participants in the program, then you should make whatever changes are necessary whenever they’re necessary to ensure the best outcome for them. There can be ethical issues involved here.  In medical experiments with new therapies or drugs, for example, some participants are given the new treatment and others aren’t (all participants consent to this arrangement, and to not knowing which group they’ll be assigned to.)  If the new treatment proves to be harmful, there is an ethical obligation for the researchers to stop administering it.  If, on the other hand, it quickly proves remarkably effective, researchers usually feel ethically bound to extend it to others in the study as soon as they can prove its positive effects.  Not all programs necessarily pose ethical problems that are as clear-cut as those encountered in medical studies, but ethical issues should always be considered.

The assumption throughout this chapter is that the whole process – planning, design, implementation, and evaluation – involves multiple stakeholders.

Typical stakeholders in a community program or intervention might include:

• Program participants or beneficiaries
• Program staff and administrators
• Others affected by the program – police, medical staff, teachers, etc.
• Academics or other researchers
• Local officials
• Community activists

Information gathering

In a participatory process, information gathering can be enhanced by a division of labor determined by the skills and experience of the participants. If there are academics or other professional researchers involved, it would probably make the most sense for them – or others with research experience – to review the evaluation literature. Members of the affected population might be the best ones to collect information about the history of the issue in the community, and about how it currently affects people. Program directors and staff would probably have the best contacts in the field, and thus the best chance to find information about other similar programs. Those with Internet access and computer experience might be the logical on-line searchers, or might act as technical support for others to help them find what they’re looking for. Those with knowledge in the law and legislation might be the ones to examine policies.

There’s also the possibility that training could be provided to the whole group, or to various individuals to allow them to pursue various lines of inquiry. There’s no reason, for instance, that people without research experience couldn’t learn to understand and interpret demographic information or contact programs in other places. (There are some limitations here: levels of related education, materials or computers, and/or inability to connect with other people might all figure in to what kind of research it makes sense to ask others to do.)

It is especially important that all participants in the process be involved in putting together the information. Training new participants to synthesize information will pay dividends in the end, because they may be able to see things in the information that aren’t obvious to experienced researchers. They may know things about the community that shed light on which elements of other programs might be appropriate and which might not.

In any case, information gathering and synthesis, like any other part of the process, should reflect the needs, interests, and abilities of all stakeholders.

There are a number of steps to gathering and putting together the information you need. Most of these can be group activities, part of the participatory process. The actual information gathering can be parceled out to specific individuals or sub-groups.

Decide what you need to know

Not surprisingly, the first step in gathering information is determining what information to gather. There are a number of areas to explore :

• Details about the issue . These might include its immediate and root causes; its general effects on individuals and communities; its consequences; its development through different stages; its history; and the history of attempts to address it.
• How the issue has been dealt with elsewhere . Best practices or approaches for which there is an evidence base; other approaches that have been at least partially effective; and what hasn’t worked, which may give you at least as much important information as what has.
• People who can help . This category encompasses experts in the field and people or organizations that have run or been involved in successful attempts to address the issue.
• Who is affected locally, and how . This really comprises two questions: a) What population groups – geographical, ethnic, cultural, racial, class, etc. – are particularly affected by this issue?  and b) What other groups are affected, but less visibly? These might include those who work with the first group(s) in the community (teachers, for example, or social workers), those who depend on them, and those on whom they depend.
• How important does the community perceive the issue to be? and
• How much and in what ways does the issue actually affect the community as a whole?
• Community needs related to the issue . What has to be added to or removed from the community in order to improve the situation? What kinds of approaches will the community respond to or reject?
• Other context information . Community history, relationships among groups and individuals that might be relevant to your work, community culture, etc.
• Who, if anyone, has some influence or control over changing the situation . Public officials and other policymakers are often in this position. Business leaders, landlords, government enforcement agencies, schools, employers, hospitals and health personnel, and members of the affected group itself might also be in the position to change the situation (by learning new skills or changing practices).

Determine your likely information sources

As mentioned above, these encompass existing (i.e., published) sources and natural (i.e., experiential) examples. Published sources can be divided into scholarly, mass-market, and statistical, each of which can provide different information and a different perspective on the issue and attempts to address it.  Depending on what you decide you’re looking for, you might use all, or any combination, of these sources.

The single largest storehouse of information available is the Internet. Many scholarly articles are published online and accessible – often free, sometimes for a fee – to anyone who’s interested. Virtually all U.S. laws and regulations at every level of government are easily found, most on several websites. General knowledge on just about anything is widely available, as are lists of best practices and successful organizations and the websites of those organizations. Census data and other similar statistical information are also on view.  Add to these the information provided by such all-encompassing sources as Wikipedia (recently, for all its quirks, found to be just about as accurate across its million-plus entries as the Encyclopedia Britannica), and you have a nearly-bottomless well of fact and opinion to draw from. As always, you have to be cautious: most of cyberspace is unedited, and the quality of information varies.  If you stick to reasonably reliable sites, you’re likely to find almost whatever you need, or at least directions to it.

Existing sources

Scholarly sources might include:

• Academic and some professional journals
• Books written for the academic market
• Doctoral dissertations - these are accessible to researchers through university libraries and some Internet sources
• Papers and reports delivered at academic and professional conferences - these are often available online, either on the authors’ websites or in e-published conference proceedings
• Occasional articles in respected mass-market scientific magazines, such as Nature or Scientific American
• Newspaper archives
• Direct contact with academics and other researchers who’ve done work on the issue you’re interested in, or who have conducted studies of attempts to deal with it
• Internet listservs and news groups relating to the issue or the field in question

Mass-market sources of information:

• Widely available books, often marketed as “self-help” or “life-changing,” to the public at large
• Articles in popular magazines, both those devoted to science or behavior and those of general interest
• Newspaper stories, often in Sunday magazine sections

Where to find statistical and demographic information:

• Census data - available on the web and at many libraries
• Community reports, such as community report cards, self-studies, and needs assessments, all of which should be obtainable through the appropriate municipal offices, and sometimes on the web as well
• Organizational and agency data, usually a matter of public record if the agency is public or publicly funded

In addition to these sources, the broadcast media often present stories about critical issues or about successful efforts to address them. In most cases, such stories only skim the surface, since they have to fit into short time slots (public broadcasting, on both radio and TV, breaks this mold more than other media outlets).  They can, however, serve as introductions to further research, raising the importance of one or more aspects of an issue, or providing information about effective programs that you can then contact. Natural examples Some of the more likely sources of natural examples: Program directors Friends or colleagues in the field Funders (particularly public agencies, because their transactions, including whom they fund and why, are a matter of public record) Leaders and members of community coalitions or partnerships Officials who coordinate community-wide efforts Members of the population most directly affected by the issue at hand Current or former participants in or beneficiaries of effective programs People who work in collaboration with programs – police, medical staff, teachers, etc. Key informants in the community Experts – some of them the same academics and other researchers referenced under scholarly sources – who have experience with your issue and efforts to address it Your own experience in the community Don’t be afraid to range far and wide in your search for successful models or new ideas.  Step outside your own field and your own region, and see what’s been done elsewhere.  A model from social work or urban design might work in public health, or vice-versa.  There’s enough overlap among fields that deal with human health and development that you can often find exactly what you need in seemingly odd places.

Devise a plan for collecting information

There are a number of considerations here:

• Who will gather what information?  As we’ve discussed, the ideal group is multi-sectoral and diverse in backgrounds and skills.  Information gathering should be assigned according to participants’ skills, interests, and contacts in the community. We’ve suggested, for instance, that scholarly sources might be mined by academics or other experienced researchers, while members of the affected population might be more successful in approaching key informants in the community. This doesn’t mean, however, that in a given group, these and other apparently logical roles couldn’t or shouldn’t be varied, depending on the individuals involved.
• How will the information be gathered?  Another issue is just how the information will be gathered . Finding and reading written material is relatively straightforward: it’s in the library or on the web, and you can read and take notes on the relevant parts of it. Getting information directly from other people, however, can be more complicated. Will you engage in formal or informal interviews ? In observation? Will you conduct surveys or public meetings ? How will you contact people you don’t know – by letter, by phone, through mutual acquaintances? Your information-gathering methods will be determined by how much time you have, exactly what information you need, the depth of the information you need, and the abilities of the participants.
• What adjustments will be made for particular gaps in experience or skills?   People who don’t read, write, and/or speak the language proficiently may have to devise imaginative ways of recording information. Experienced researchers may have to translate scholarly writing for just about everyone in the group who isn’t an academic. In many cases, most or all of the group may need orientation or training before information gathering can begin. You’ll need to work out what the needs are as a group, and devise ways to meet them.
• What’s the timeline for information gathering?  While information gathering should continue throughout the life of a project, the initial phase should have a time limit, so that action isn’t delayed for too long. The time limit depends on your time constraints, the seriousness and intensity of the issue, the community’s perception of urgency, and whether there are external time restrictions (student interns who are only available until the end of the summer, for instance.) Having a clear deadline will focus the group’s activities, and boost its efficiency.

Collect information

When your plan is completed, it’s time to put it into practice. You’ll have to conduct any trainings that are necessary, and make sure that all the relevant tasks are assigned appropriately. You may also want to set up regular meetings throughout the information-gathering process, in order to give the group the chance to review progress, make suggestions, and report on what they’ve been finding.  In addition to providing support for those new to research, these meetings, by providing a preview of the results of the process, will save everyone having to digest an overwhelming amount of information all at once.

Synthesize: Take it all apart

The process of synthesis involves breaking the information down into its component parts, sifting through those parts to see which fit together best for your situation, and then integrating them into an approach that is likely to work in your community.

There are usually three major areas to be considered:

• What’s known about the issue itself . What personal and environmental factors contribute to the problem? What are its root causes? Do you have the resources to address them, or are they beyond your scope (e.g., global economic forces or climate change)? Does the issue have a number of different effects, and if so, what are they?  What are the likely consequences for the community as a whole if the issue is not resolved?  (An environmental health risk can not only kill or sicken individuals, but might also affect business productivity, insurance availability and rates, hospital costs, the housing market, or even – as in the case of the Love Canal neighborhood in Niagara Falls, NY – the existence of a neighborhood or community itself.)
• The community context of the issue . What are the specific local effects of the issue. Exactly who is affected?  Exactly how are they affected? What are the consequences for those individuals? For their families, friends, neighbors, and others they have dealings with? For the community as a whole?  What has been the community’s experience with this issue in the past?  How, if at all, has it been addressed?  What local conditions would change if the issue were addressed, and how would they change?  Are there underlying conditions that have to change before the issue can be addressed?  Whose attitudes and/or behaviors need to change to have an effect on the issue (for example, among policy makers, those affected, or specific officials)?
• Successful and unsuccessful attempts to address the issue . These may have been gleaned both from the literature on best practices, and directly or at second hand from those involved in them. Here, it’s important to separate out the elements of various approaches. What specific procedures – methods and intervention components – were used? What kinds of training – feedback, role play, modeling, etc. – were provided to participants?  Was information provided to participants about when, why, and how to act? Were there positive or negative consequences that helped to establish or maintain change (or its opposite)?  Were environmental barriers, policies, or regulations put in place or removed?  What was the overall philosophy behind the approach?  What aspects of the issue did it address? What kind(s) of community was it tried in?  What population groups (in terms of culture, age, social class, etc.) were involved?  Who was the approach to benefit?  What were the specific results in the short term?  In the long term?  What makes a particular program, policy, or practice successful or unsuccessful? What events, if any, were critical, to success (or failure)?  What conditions – organizational features, participant characteristics, broader environmental factors – were critical?  Is there a model successful program? Is there a model unsuccessful program?

The existence of a model unsuccessful program doesn’t indicate that if you do the opposite of everything that program did, you’ll be successful.  Even if it failed spectacularly, much of the program may have been potentially effective, but one or two elements – the way participants were approached, recruited, or treated, a particular method – negated what could have worked.  By the same token, most elements of the program may have been fine, but its basic premise might have been mistaken or ineffective – “Just say no” as a way of preventing AIDS among teens, for instance.  It’s important to try to figure out why the program was unsuccessful.  A true model unsuccessful program is one that did everything wrong, but those are few and far between. Lisbeth Schorr ( Common Purpose ) makes a useful distinction between “what works” and conditions under which what works actually works.  Sometimes the presence of a charismatic leader or champion motivates staff and/or participants to succeed. When the leader or original staff members leave, some such programs collapse, while others are able to renew themselves by careful hiring and a faithful implementation of what made it work. In looking for programs to draw from, you need to understand the intervention components and elements that make those programs work.  Also, try to understand the conditions that allow an intervention to be successful.

Synthesize: Put it back together

Analyze the elements you’ve found to determine which of them would be appropriate for the situation and group you’re working with.

• What has been used specifically with your population in your circumstances?  Have the successful programs you’ve looked at been context-specific (i.e., intended for their specific communities and populations)? Can they be adapted to your context if they weren’t intended for it?
• What can be adapted, if it wasn’t originally aimed at your population?  (Techniques used with children or adolescents that could be modified for use with adults, for instance, or vice-versa.)
• What’s missing?  What aspects of the issue in your community are not addressed by what you’ve found?  Are they important enough that they need to be addressed?
• Does what you’ve found confirm or contradict what you thought you already knew?
• Are there factors in your particular situation that make the issue substantially different for you and your participants than for any other programs or approaches you’ve found out about?  How will you deal with that?
• What does your information tell you about the possibility of successfully addressing the issue’s root causes (e.g., income inequality, social exclusion, lack of power)?
• In general, did most or all successful programs direct their change efforts at the same group of people (policy makers, for example), or was there a variety?  If the latter, what do you think is most likely to work in your community?
• Perhaps most important, what’s your definition of success, and which of the programs you learned about came closest to achieving?  What components and elements of those programs addressed what’s needed in your community?

Answering these questions will give you a good sense of which components of other programs may work for you, and should also fit with what you already know to either give you ideas for new elements that you can add, or confirm (or warn you away from) ideas for new elements that you had already.

We don’t want to imply that simply taking a lot of different program components and playing mix-and-match will provide you with an effective way to address a community issue. You have to start with a clear framework informed by your vision and mission, and put together a program that’s coherent and makes sense.  All the elements have to fit; if they fit well enough, you’ll end up with a whole that’s greater than the sum of its parts. If the elements don’t fit together, or aren’t part of a program with a well-defined framework, the chances are you’ll end up with a mess.

Information gathering and knowledge synthesis should continue throughout the course of the program. While you may wait until the results of an initial evaluation to change something, you should always be looking for improvements and better approaches. No program or effort is perfect: everything can be improved. As long as you keep trying to learn more and grow in your understanding of your work, it will continue to get better. If you become complacent (i.e., you feel you know what you’re doing and can relax), your program may start to lose its effectiveness.

Gathering the information that already exists about your issue and attempts to address it is one of the most important aspects of planning a program or evaluation. By putting together what’s known about the issue and the history of the successes and failures of various approaches to it, you can build a program structure that includes your own innovations and elements that have worked for others in similar situations. This synthesis also allows you to avoid ineffective approaches and to incorporate ideas and methods that have been particularly appropriate, culturally or otherwise, to the population and community you’re working with.

Information gathering and synthesis should continue throughout the life of the program. The more information you have, and the more carefully you put it together, the better your chances of implementing a successful program.

Online Resources

Michigan State University’s “ Best Practice Briefs ” gives access to over 30 short but informative articles on best practices in various areas.

The  Colorado Dept. of Health and Environment  has a large listing of  best practices  in health.

Community Problem Solving  provides a list of links to sites that include best practices.

CHNA.org is a free, web-based utility to assist hospitals, non-profit community-based organizations, state and local health departments, financial institutions, and engaged citizens in understanding the needs and assets of their communities. CHNA.org provides Key capabilities available include: a) an intuitive platform to guide you through the process of conducting community health needs assessments, b) the ability to create a community health needs assessment report, c) the ability to select area geography in different ways, d) the ability to identify and profile geographic areas with significant health disparities, e) Single-point access to thousands of public data sources, such as the U.S. Census Bureau and the Behavioral Risk Factor Surveillance System (BRFSS).

This Data Collection Tools for Evaluation resource is a helpful table providing an overview of evaluation methods, including benefits and limitations of each technique.

The  Federal Election Commission  oversees the Campaign Finance Reform Act, and provides campaign finance information.

The CDC offers guidance to help users Gather Credible Evidence in their evaluations.  This manual uses evaluation of antibiotic programs as an example of evidence gathering.

The  Guide to Community Preventive Services  is the website of the Task Force on Community Preventive Services, appointed by the Director of the Centers for Disease Control. The Task Force is an independent body operating under the umbrella of the Dept. of Health and Human Services. The website contains best practice information on a large number of prevention strategies.

The Promising Practices Network provides links to and comprehensive descriptions of proven (i.e., thoroughly researched and found to be effective) and promising programs in a variety of areas.

The World Health Organization  is the directing and coordinating authority for health within the United Nations system. It is responsible for providing leadership on global health matters, shaping the health research agenda, setting norms and standards, articulating evidence-based policy options, providing technical support to countries and monitoring and assessing health trends.

U.S. Government sites can provide a wealth of information:

The U.S. National Archives and Records Administration is the nation's record keeper, storing documents that are important for legal or historical reasons, and making them publicly available.

The U.S. Department of Health and Human Services , the principal agency for protecting the health of U.S. citizens, is comprised of 12 agencies that provide information on their specific domains, such as the Administration on Aging . Others include the Centers for Disease Control , which maintains national health statistics, such as FastStats , which provides quick access to statistics on topics of public health importance and provides links to publications that include the statistics presented, and to sources of more data. The Community Health Status Indicators site provides health assessment information at the local level through a Health Resources and Services Administration-funded collaboration. The " WONDER " system is an access point to a wide variety of CDC reports, guidelines, and public health data to assist in research, decision-making, priority setting, and resource allocation. Also part of the Department of Health and Human Services, the National Institutes of Health is the nation’s medical research agency.

The U.S. National Institute of Mental Health provides statistics and educational information for the public as well as information for researchers.

The U.S. Census Bureau provides demographic information, nationwide, regionally, by state, county, municipality, and census tract.

The U.S. Dept. of Agriculture provides information ranging from assistance for rural communities to food and nutrition resources.

The U.S. Dept. of Education provides information about education policy, research, and grant opportunities.

The U.S. Department of Labor offers statistics about the U.S. workforce, including the Occupational Safety and Health Administration of the Department of Labor .

The U.S. Dept. of the Interior protects America’s natural resources and heritage.

The U.S. Environmental Protection Agency provides information about environmental regulations and research.

The U.S. Dept. of Housing and Urban Development aims to improve lives by creating affordable homes in safe, healthy communities of opportunity, and by protecting the rights and affirming the values of a diverse society.

The U.S. Securities and Exchange Commission oversees stock and bond trading and corporate activities.

The U.S. Supreme Court website stores opinions, dissents, and other information from recent sessions (past three years) available at no charge.

Print Resources

Fawcett, S., et. al. (2008).  Community Tool Box Curriculum Module 12: Evaluating the initiative . Center for Community Health and Development. University of Kansas.

Fawcett, S., Suarez, Y., Balcazar, White, G., Paine, A, Blanchard, K., & Embree, M. (1994).  Conducting intervention research: the design and development process . In Rothman, J., & Thomas, J. (Eds.),  Intervention Research: Design and Development for Human Service . (pp. 25-54). New York, NY: Haworth Press.

• An Ordinary Man, His Extraordinary Journey
• Hours/Admission
• Nearby Dining and Lodging
• Information
• Library Collections
• Online Collections
• Photographs
• Harry S. Truman Papers
• Federal Records
• Personal Papers
• Appointment Calendar
• Audiovisual Materials Collection
• President Harry S. Truman's Cabinet
• President Harry S. Truman's White House Staff
• New Materials
• Research Procedures
• Collection Policy and Donating Materials
• Truman Family Genealogy
• Research Room Regulations
• To Secure These Rights
• Freedom to Serve
• Events and Programs
• Featured programs
• Civics for All of US
• Civil Rights Teacher Workshop
• High School Trivia Contest
• Teacher Lesson Plans
• Truman Library Teacher Conference 2024
• National History Day
• Student Resources
• Truman Library Teachers Conference
• Truman Presidential Inquiries
• Student Research File
• The Truman Footlocker Project
• Truman Trivia
• The White House Decision Center
• Three Branches of Government
• Electing Our Presidents Teacher Workshop
• National History Day Workshops from the National Archives
• Research grants
• Truman Library History
• Contact Staff
• Volunteer Program
• Internships
• Educational Resources

Gathering, Recording, and Presenting Data

Using local newspaper, find and discuss examples of uses of statistics and the ways in which the information is presented

Students will be able to create charts using statistical data

Students will be able to collect statistical data from news sources

Show Me Standard: Social Studies #7 The use of tools of social science inquiry (such as surveys, statistics, maps, documents) Goal#2 Students will acquire knowledge and skills to communicate effectively within and beyond the classroom 2.1 Plan and make written, oral, and visual presentations for a variety of purposes and audiences 2.7 Use technological tools to exchange information andideas

Kansas Standards

Benchmark 4: The student engages in historical thinking skills.

2. (A) examines a variety of primary sources in history and analyzes them in terms of credibility, purpose, and point of view (e.g., census records, diaries, photographs, letters, government documents).

The Independence Examiner

ClarisWorks Word Processing/ Spreadsheet/ Chart making program

Procedure: Lesson One: Instruction

1. Pass out questionnaires to students asking how they got to school. Students are asked to circle the word that describes how they got to school: car, van,truck, motorcycle, bicycle, school bus, walk 2. Gather papers together and as a class go through the process of compiling the information. 3. Discuss options for ways information can be gathered, and pros and cons for each:

• Questionnaires
• Researching documents

4. Discuss options for recording the information gained through the various methods of surveying:

• listing responses

5. Discuss options for presenting the information:

• line graphs
• pictographs

Procedure: Lesson Two: Assignment: Each student will be required to choose a topic on which they would like to do a survey. The surveys will involve our class, a particular grade level at Procter, or all the Procter classrooms. Each student will turn in the following form stating their first and second choices for survey topics.

After being approved by the teacher, each student will conduct their survey, and record the information. The students will be required to create a line graph showing their results using graph paper and colored pencils. The students will also be required to write a paragraph explaining what the subject of their survey was, as well as the results.

Procedure: Lesson Three: Use of Technology The students will be instructed on the use of ClarisWorks to create graphs, and will use the results of their individual surveys to enter information on a spreadsheet. Using the ClarisWorks program, the information will be converted to a graph format. Students will choose the type of format they think is appropriate for their survey. Students will also be required to type their paragraphs on theClarisWorks document along with their graphs. The documents will be printed out and displayed.  

The projects will be assessed according to the following scoring guide

The total solar eclipse 2024 is happening today! Here's what you need to know

Millions are ready to watch  —  are you?

It's the day we've all been waiting for! Total solar eclipse 2024 is finally here. Today (April 8) a total solar eclipse will sweep across North America from Mexico, through 15 U.S. States and up through Canada. For viewers inside the path of totality, the moon will completely cover the sun, darkening the sky and revealing our star's hidden outer atmosphere.

You can keep up to date with the latest eclipse content on our eclipse live blog and watch all the total eclipse action unfold live here on Space.com courtesy of NASA. Our livestream coverage begins at 1 p.m. EDT (1700 GMT). 

If you capture a great photo of the solar eclipse and would like to share it with us please email it to [email protected] . 

Related: 10 things you probably didn't know about the total solar eclipse 2024

Every U.S. State will experience at least a partial solar eclipse today. If you want to check what time you'll be able to see the solar eclipse and find out what it may look like from your location check out this great interactive map from NASA . 

A total solar eclipse occurs when the moon passes directly between the sun and Earth, casting a shadow on our planet. It appears almost exactly the same size as our sun and for those witnessing the event from the path of totality, it will completely cover the sun's visible disk.

Last minute preparations  

If you haven't been able to get hold of a pair of eclipse glasses and are looking for an easy way to still view the solar eclipse safely we have a list of some alternative ways to view the eclipse with items from around the home . 

Get the Space.com Newsletter

Breaking space news, the latest updates on rocket launches, skywatching events and more!

Remember NEVER look directly at the sun. To safely view all of this event, you must use solar glasses or filters. Only those in the path of totality will be able to remove them briefly to see the sun's corona with their naked eyes. Those not in the path of totality must keep them on the entire time. 

Everyone observing the partial phases of this eclipse — and for those outside the path of totality, that's the entire event — will need to wear solar eclipse glasses while cameras, telescopes and binoculars will need solar filters placed in front of their lenses. 

Our how to observe the sun safely guide tells you everything you need to know about safe solar observations. 

Eclipse timings

Submit your photos! If you capture a photo of the April 8 total solar eclipse or any of these strange effects and would like to share it with Space.com's readers, send photos, videos, comments, and your name, location and content usage permission release to [email protected].

Join our Space Forums to keep talking space on the latest missions, night sky and more! And if you have a news tip, correction or comment, let us know at: [email protected].

Daisy Dobrijevic joined Space.com in February 2022 having previously worked for our sister publication All About Space magazine as a staff writer. Before joining us, Daisy completed an editorial internship with the BBC Sky at Night Magazine and worked at the National Space Centre in Leicester, U.K., where she enjoyed communicating space science to the public. In 2021, Daisy completed a PhD in plant physiology and also holds a Master's in Environmental Science, she is currently based in Nottingham, U.K. Daisy is passionate about all things space, with a penchant for solar activity and space weather. She has a strong interest in astrotourism and loves nothing more than a good northern lights chase! 

Total solar eclipse 2024 thrills millions across North America (video, photos)

Total solar eclipse 2024 has begun and here are the first views!

Total solar eclipse 2024: Live updates

• COLGeek Skies good, so far. Telescope setup and looked at sun spots a bit already. I'm kicking myself for not getting a CCD. Hoping for a good viewing today. Reply
• View All 1 Comment

Most Popular

By Mike Wall April 07, 2024

By Jeff Spry April 07, 2024

By Sharmila Kuthunur April 07, 2024

By Robert Lea April 07, 2024

By Tariq Malik April 06, 2024

By Space.com Staff April 06, 2024

By Joe Rao April 06, 2024

By Josh Dinner April 06, 2024

By Mike Wall April 06, 2024

• 2 Total solar eclipse 2024 thrills millions across North America (video, photos)
• 3 Total solar eclipse 2024 has begun and here are the first views!
• 4 Total solar eclipse 2024: Pictures from around the web
• 5 In Indiana, the best spot to see the 2024 solar eclipse is wherever you are

Let-7a-3p overexpression increases chemosensitivity to carmustine and synergistically promotes autophagy and suppresses cell survival in U87MG glioblastoma cancer cells

• Published: 08 April 2024

Cite this article

• Seyedeh Zahra Bahojb Mahdavi 1 , 2 ,
• Nasser Pouladi 2 ,
• Mohammad Amini 1 ,
• Behzad Baradaran 1 ,
• Souzan Najafi 1 ,
• Shiva Vaghef Mehrabani 1 , 2 ,
• Amirhossein Yari 1 ,
• Sania Ghobadi Alamdari 1 , 3 &
• Amir Ali Mokhtarzadeh 1  

In terms of primary brain tumors, glioblastoma is one of the most aggressive and common brain tumors. The high resistance of glioblastoma to chemotherapy has made it vital to find alternative treatments and biological mechanisms to reduce the survival of cancer cells. Given that, the objective of the present research was to explore the potential of let-7a-3p when used in combination with carmustine in human glioblastoma cancer cells. Based on previous studies, the expression of let-7a is downregulated in the U87MG cell line. Let-7a-3p transfected into U87MG glioblastoma cells. Cell viability of the cells was assessed by MTT assay. The apoptotic induction in U87MG cancerous cells was determined through the utilization of DAPI and Annexin V/PI staining techniques. Moreover, the induction of autophagy and cell cycle arrest was evaluated by flow cytometry. Furthermore, cell migration was evaluated by the wound healing assay while colony formation assay was conducted to evaluate colony formation. Also, the expression of the relevant genes was evaluated using qRT-PCR. Transfection of let-7a-3p mimic in U87MG cells increased the expression of the miRNA and also increased the sensitivity of U87MG cells to carmustine. Let-7a-3p and carmustine induced sub-G1 and S phase cell cycle arrest, respectively. Combination treatment of let-7a-3p and carmustine synergistically increased arrested cells and induced apoptosis through regulating involved genes including P53, caspase-3, Bcl-2, and Bax. Combined treatment with let-7a-3p and carmustine also induced autophagy and increased the expression of the ATG5 and Beclin 1 (ATG6). Furthermore, let-7a-3p combined with carmustine inhibited cell migration via decreasing the expression of MMP-2. Moreover, the combination therapy decreased the ability of U87MG to form colonies through downregulating CD-44. In conclusion, our work suggests that combining let-7a-3p replacement therapy with carmustine treatment could be considered a promising strategy in treatment and can increase efficiency of glioblastoma chemotherapy.

Graphical Abstract

Summary of conducted assays in this research. After selection of let-7a-3p, this miRNA transfected into U87MG cells via electroporation. Optimum dose and time of the miRNA were evaluated by the qRT-PCR. In order to find IC50 of the carmustine, U87MG cells were treated with different doses of this chemotherapy drug and MTT assay conducted. Then cells were transfected with optimum doses of let-7a-3p and treated with IC50 of carmustine. In order to assess the combined effect of these two methods, apoptosis induction, cell cycle arrest, autophagy induction, cell migration, colony formation, and expression of the involved genes were evaluated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Data availability

The results will be available upon the reasonable request from the corresponding author.

Agraval H, Yadav UC (2019) MMP-2 and MMP-9 mediate cigarette smoke extract-induced epithelial-mesenchymal transition in airway epithelial cells via EGFR/Akt/GSK3β/β-catenin pathway: amelioration by fisetin. Chem Biol Interact 314:108846

Article   CAS   PubMed   Google Scholar  

Alexandri C, Van Den Steen G, Demeestere I (2022) Let-7a mimic transfection reduces chemotherapy-induced damage in a mouse ovarian transplantation model. Sci Rep 12:10863

Article   CAS   PubMed   PubMed Central   Google Scholar  

Alifieris C, Trafalis DT (2015) Glioblastoma multiforme: pathogenesis and treatment. Pharmacol Ther 152:63–82

Bader AG, Brown D, Winkler M (2010) The promise of microRNA replacement therapy. Can Res 70:7027–7030

Article   CAS   Google Scholar  

Bastiancich C, Danhier P, Préat V, Danhier F (2016) Anticancer drug-loaded hydrogels as drug delivery systems for the local treatment of glioblastoma. J Control Release 243:29–42

Biamonte F, Santamaria G, Sacco A, Perrone FM, Di Cello A, Battaglia AM, Salatino A, Di Vito A, Aversa I, Venturella R (2019) MicroRNA let-7g acts as tumor suppressor and predictive biomarker for chemoresistance in human epithelial ovarian cancer. Sci Rep 9:1–12

Blower PE, Chung J-H, Verducci JS, Lin S, Park J-K, Dai Z, Liu C-G, Schmittgen TD, Reinhold WC, Croce CM (2008) MicroRNAs modulate the chemosensitivity of tumor cells. Mol Cancer Ther 7:1–9

Cai K, Wan Y, Sun G, Shi L, Bao X, Wang Z (2012) Let-7a inhibits proliferation and induces apoptosis by targeting EZH2 in nasopharyngeal carcinoma cells. Oncol Rep 28:2101–2106

Chakraborty C, Sharma AR, Sharma G, Sarkar BK, Lee S-S (2018) The novel strategies for next-generation cancer treatment: miRNA combined with chemotherapeutic agents for the treatment of cancer. Oncotarget 9:10164

Article   PubMed   PubMed Central   Google Scholar  

Che F, Xie X, Wang L, Su Q, Jia F, Ye Y, Zang L, Wang J, Li H, Quan Y (2018) B7–H6 expression is induced by lipopolysaccharide and facilitates cancer invasion and metastasis in human gliomas. Int Immunopharmacol 59:318–327

Chen VCH, Hsieh YH, Chen LJ, Hsu TC, Tzang BS (2018) Escitalopram oxalate induces apoptosis in U-87 MG cells and autophagy in GBM 8401 cells. J Cell Mol Med 22:1167–1178

Chen C, Liang H, Qin R, Li X, Wang L, Du S, Chen Z, Meng X, Lv Z, Wang Q (2022) Doramectin inhibits glioblastoma cell survival via regulation of autophagy in vitro and in vivo. Int J Oncol 60:1–16

Chirshev E, Oberg KC, Ioffe YJ, Unternaehrer JJ (2019) Let-7 as biomarker, prognostic indicator, and therapy for precision medicine in cancer. Clin Transl Med 8:1–14

Article   Google Scholar  

Colella B, Faienza F, Di Bartolomeo S (2019) EMT regulation by autophagy: a new perspective in glioblastoma biology. Cancers 11:312

Davis ME (2016) Glioblastoma: overview of disease and treatment. Clin J Oncol Nurs 20:S2

Di Filippo LD, Duarte JL, Luiz MT, de Araújo JTC, Chorilli M (2021) Drug delivery nanosystems in glioblastoma multiforme treatment: current state of the art. Curr Neuropharmacol 19:787–812

Di Leva G, Garofalo M, Croce CM (2014) MicroRNAs in cancer. Annu Rev Pathol 9:287–314

Article   PubMed   Google Scholar  

Dong Z, Lei Q, Yang R, Zhu S, Ke X-X, Yang L, Cui H, Yi L (2017) Inhibition of neurotensin receptor 1 induces intrinsic apoptosis via let-7a-3p/Bcl-w axis in glioblastoma. Br J Cancer 116:1572–1584

Duan S, Li J, Tian J, Yin H, Zhai Q, Wu Y, Yao S, Zhang L (2019) Crosstalk between let-7a-5p and BCL-xL in the initiation of toxic autophagy in lung cancer. Mol Ther Oncol 15:69–78

Eisenberg-Lerner A, Bialik S, Simon H-U, Kimchi A (2009) Life and death partners: apoptosis, autophagy and the cross-talk between them. Cell Death Differ 16:966–975

Elgendy M, Ciro M, Abdel-Aziz AK, Belmonte G, Dal Zuffo R, Mercurio C, Miracco C, Lanfrancone L, Foiani M, Minucci S (2014) Beclin 1 restrains tumorigenesis through Mcl-1 destabilization in an autophagy-independent reciprocal manner. Nat Commun 5:5637

Engeland K (2018) Cell cycle arrest through indirect transcriptional repression by p53: I have a DREAM. Cell Death Differ 25:114–132

Fan H, Jiang M, Li B, He Y, Huang C, Luo D, Xu H, Yang L, Zhou J (2018) MicroRNA-let-7a regulates cell autophagy by targeting Rictor in gastric cancer cell lines MGC-803 and SGC-7901. Oncol Rep 39:1207–1214

CAS   PubMed   Google Scholar  

Fisher JP, Adamson DC (2021) Current FDA-approved therapies for high-grade malignant gliomas. Biomedicines 9:324

Flum M, Dicks S, Teng Y-H, Schrempp M, Nyström A, Boerries M, Hecht A (2022) Canonical TGFβ signaling induces collective invasion in colorectal carcinogenesis through a Snail1-and Zeb1-independent partial EMT. Oncogene 41:1492–1506

Fulda S, Kögel D (2015) Cell death by autophagy: emerging molecular mechanisms and implications for cancer therapy. Oncogene 34:5105–5113

Giovannetti E, Erozenci A, Smit J, Danesi R, Peters GJ (2012) Molecular mechanisms underlying the role of microRNAs (miRNAs) in anticancer drug resistance and implications for clinical practice. Crit Rev Oncol Hematol 81:103–122

Gómez-Oliva R, Domínguez-García S, Carrascal L, Abalos-Martínez J, Pardillo-Díaz R, Verástegui C, Castro C, Nunez-Abades P, Geribaldi-Doldán N (2021) Evolution of experimental models in the study of glioblastoma: toward finding efficient treatments. Front Oncol 10:614295

Guo X, Xue H, Guo X, Gao X, Xu S, Yan S, Han X, Li T, Shen J, Li G (2015) MiR224-3p inhibits hypoxia-induced autophagy by targeting autophagy-related genes in human glioblastoma cells. Oncotarget 6:41620

Haar CP, Hebbar P, Wallace GC, Das A, Vandergrift WA, Smith JA, Giglio P, Patel SJ, Ray SK, Banik NL (2012) Drug resistance in glioblastoma: a mini review. Neurochem Res 37:1192–1200

Hu X, Yan P, Feng J, Zhang F (2020) Expression of microRNA-210 and the prognosis in glioma patients: a meta-analysis. Biomark Med 14:795–805

Huang X, Qi Q, Hua X, Li X, Zhang W, Sun H, Li S, Wang X, Li B (2014) Beclin 1, an autophagy-related gene, augments apoptosis in U87 glioblastoma cells. Oncol Rep 31:1761–1767

Iwasaki T, Tanaka K, Kawano M, Itonaga I, Tsumura H (2015) Tumor-suppressive microRNA-let-7a inhibits cell proliferation via targeting of E2F2 in osteosarcoma cells. Int J Oncol 46:1543–1550

Kim KH, Lee M-S (2014) Autophagy—a key player in cellular and body metabolism. Nat Rev Endocrinol 10:322–337

Kolliopoulos C, Ali MM, Castillejo-Lopez C, Heldin C-H, Heldin P (2022) CD44 depletion in glioblastoma cells suppresses growth and stemness and induces senescence. Cancers 14:3747

Kong L, Ji H, Gan X, Cao S, Li Z, Jin Y (2022) Knockdown of CD44 inhibits proliferation, migration and invasion of osteosarcoma cells accompanied by downregulation of cathepsin S. J Orthop Surg Res 17:1–10

Lee S-T, Chu K, Oh H-J, Im W-S, Lim J-Y, Kim S-K, Park C-K, Jung K-H, Lee SK, Kim M (2011) Let-7 microRNA inhibits the proliferation of human glioblastoma cells. J Neurooncol 102:19–24

Liu K, Zhang C, Li T, Ding Y, Tu T, Zhou F, Qi W, Chen H, Sun X (2015) Let-7a inhibits growth and migration of breast cancer cells by targeting HMGA1. Int J Oncol 46:2526–2534

Liu T-P, Huang C-C, Yeh K-T, Ke T-W, Wei P-L, Yang J-R, Cheng Y-W (2016) Down-regulation of let-7a-5p predicts lymph node metastasis and prognosis in colorectal cancer: implications for chemotherapy. Surg Oncol 25:429–434

Long X-B, Sun G-B, Hu S, Liang G-T, Wang N, Zhang X-H, Cao P-P, Zhen H-T, Cui Y-H, Liu Z (2009) Let-7a microRNA functions as a potential tumor suppressor in human laryngeal cancer. Oncol Rep 22:1189–1195

Lopez-Bertoni H, Johnson A, Rui Y, Lal B, Sall S, Malloy M, Coulter JB, Lugo-Fagundo M, Shudir S, Khela H (2022) Sox2 induces glioblastoma cell stemness and tumor propagation by repressing TET2 and deregulating 5hmC and 5mC DNA modifications. Signal Transduct Target Ther 7:37

Pan X, Wang G, Wang B (2021) MicroRNA-1182 and let-7a exert synergistic inhibition on invasion, migration and autophagy of cholangiocarcinoma cells through down-regulation of NUAK1. Cancer Cell Int 21:1–16

Reed MR, Maddukuri L, Ketkar A, Byrum SD, Zafar MK, Bostian AC, Tackett AJ, Eoff RL (2021) Inhibition of tryptophan 2, 3-dioxygenase impairs DNA damage tolerance and repair in glioma cells. NAR Cancer 3:zcab014

Sarkaria JN, Kitange GJ, James CD, Plummer R, Calvert H, Weller M, Wick W (2008) Mechanisms of chemoresistance to alkylating agents in malignant glioma. Clin Cancer Res 14:2900–2908

Sukumari-Ramesh S, Prasad N, Alleyne CH, Vender JR, Dhandapani KM (2015) Overexpression of Nrf2 attenuates Carmustine-induced cytotoxicity in U87MG human glioma cells. BMC Cancer 15:1–10

Tsai Y-S, Huang C-I, Tsai P-C, Yeh M-L, Huang C-F, Hsieh M-H, Liu T-W, Lin Y-H, Liang P-C, Lin Z-Y (2022) Circulating let-7 family members as non-invasive biomarkers for predicting hepatocellular carcinoma risk after antiviral treatment among chronic hepatitis C patients. Cancers 14:2023

Tsang WP, Kwok TT (2008) Let-7a microRNA suppresses therapeutics-induced cancer cell death by targeting caspase-3. Apoptosis 13:1215–1222

van Solinge TS, Nieland L, Chiocca EA, Broekman ML (2022) Advances in local therapy for glioblastoma—taking the fight to the tumour. Nat Rev Neurol 18:221–236

Vollmann-Zwerenz A, Leidgens V, Feliciello G, Klein CA, Hau P (2020) Tumor cell invasion in glioblastoma. Int J Mol Sci 21:1932

Wang F, Zhang P, Ma Y, Yang J, Moyer MP, Shi C, Peng J, Qin H (2012) NIRF is frequently upregulated in colorectal cancer and its oncogenicity can be suppressed by let-7a microRNA. Cancer Lett 314:223–231

Wang X-R, Luo H, Li H-L, Cao L, Wang X-F, Yan W, Wang Y-Y, Zhang J-X, Jiang T, Kang C-S (2013a) Overexpressed let-7a inhibits glioma cell malignancy by directly targeting K-ras, independently of PTEN. Neuro Oncol 15:1491–1501

Wang XR, Luo H, Li HL, Cao L, Wang XF, Yan W, Wang YY, Zhang JX, Jiang T, Kang CS, Liu N, You YP (2013b) Overexpressed let-7a inhibits glioma cell malignancy by directly targeting K-ras, independently of PTEN. Neuro Oncol 15:1491–1501

Wang H, Zhou H, Xu J, Lu Y, Ji X, Yao Y, Chao H, Zhang J, Zhang X, Yao S (2021) Different T-cell subsets in glioblastoma multiforme and targeted immunotherapy. Cancer Lett 496:134–143

Wu A, Wu K, Li J, Mo Y, Lin Y, Wang Y, Shen X, Li S, Li L, Yang Z (2015) Let-7a inhibits migration, invasion and epithelial-mesenchymal transition by targeting HMGA2 in nasopharyngeal carcinoma. J Transl Med 13:1–13

Wu T, Chen X, Peng R, Liu H, Yin P, Peng H, Zhou Y, Sun Y, Wen L, Yi H (2016) Let-7a suppresses cell proliferation via the TGF-β/SMAD signaling pathway in cervical cancer. Oncol Rep 36:3275–3282

Xiao Z-Z, Wang Z-F, Lan T, Huang W-H, Zhao Y-H, Ma C, Li Z-Q (2020) Carmustine as a supplementary therapeutic option for glioblastoma: a systematic review and meta-analysis. Front Neurol 11:1036

Xie Q, Yan Y, Huang Z, Zhong X, Huang L (2014a) MicroRNA-221 targeting PI3-K/Akt signaling axis induces cell proliferation and BCNU resistance in human glioblastoma. Neuropathology 34:455–464

Xie Y, Yang Y, Tang C, Sheng H, Jiang Y, Han K, Ding L (2014b) Estrogen combined with progesterone decreases cell proliferation and inhibits the expression of Bcl-2 via microRNA let-7a and miR-34b in ovarian cancer cells. Clin Transl Oncol 16:898–905

Xue F, Liu Y, Zhang H, Wen Y, Yan L, Tang Q, Xiao E, Zhang D (2016) Let-7a enhances the sensitivity of hepatocellular carcinoma cells to cetuximab by regulating STAT3 expression. Onco Targets Ther 9:7253

Yang Q, Jie Z, Cao H, Greenlee AR, Yang C, Zou F, Jiang Y (2011) Low-level expression of let-7a in gastric cancer and its involvement in tumorigenesis by targeting RAB40C. Carcinogenesis 32:713–722

Yang ZY, Wang Y, Liu Q, Wu M (2020) microRNA cluster MC-let-7a-1~ let-7d promotes autophagy and apoptosis of glioma cells by down-regulating STAT3. CNS Neurosci Ther 26:319–331

Yerram P, Reiss SN, Modelevsky L, Gavrilovic IT, Kaley T (2019) Evaluation of toxicity of carmustine with or without bevacizumab in patients with recurrent or progressive high grade gliomas. J Neurooncol 145:57–63

Yin J, Hu W, Pan L, Fu W, Dai L, Jiang Z, Zhang F, Zhao J (2019) Let-7 and miR-17 promote self-renewal and drive gefitinib resistance in non-small cell lung cancer. Oncol Rep 42:495–508

CAS   PubMed   PubMed Central   Google Scholar  

Yoshida T, Matsuda Y, Naito Z, Ishiwata T (2012) CD44 in human glioma correlates with histopathological grade and cell migration. Pathol Int 62:463–470

Zhao W, Hu JX, Hao RM, Zhang Q, Guo JQ, Li YJ, Xie N, Liu LY, Wang PY, Zhang C (2018) Induction of microRNA-let-7a inhibits lung adenocarcinoma cell growth by regulating cyclin D1. Oncol Rep 40:1843–1854

Zhou B, Shan H, Su Y, Xia K, Zou R, Shao Q (2017) Let-7a inhibits migration, invasion and tumor growth by targeting AKT2 in papillary thyroid carcinoma. Oncotarget 8:69746

Zhou W, Yu X, Sun S, Zhang X, Yang W, Zhang J, Zhang X, Jiang Z (2019) Increased expression of MMP-2 and MMP-9 indicates poor prognosis in glioma recurrence. Biomed Pharmacother 118:109369

Download references

The authors are grateful for financial support from the Immunology Research Center, Tabriz University of Medical Science (Grant number:73494).

Author information

Authors and affiliations.

Immunology Research Center, Tabriz University of Medical Sciences, Tabriz, Iran

Seyedeh Zahra Bahojb Mahdavi, Mohammad Amini, Behzad Baradaran, Souzan Najafi, Shiva Vaghef Mehrabani, Amirhossein Yari, Sania Ghobadi Alamdari & Amir Ali Mokhtarzadeh

Department of Biology, Faculty of Basic Sciences, Azarbaijan Shahid Madani University, Tabriz, Iran

Seyedeh Zahra Bahojb Mahdavi, Nasser Pouladi & Shiva Vaghef Mehrabani

Department of Cell and Molecular Biology, Faculty of Basic Science, University of Maragheh, Maragheh, Iran

Sania Ghobadi Alamdari

You can also search for this author in PubMed   Google Scholar

Contributions

S.Z.B.M., the first author of the manuscript, performed the experiment, contributed to the cellular and molecular assays and wrote the manuscript. N.P., revised the main text of the manuscript and analyzed the data. M.A., helped in the cellular and molecular assays and analyzed the data. B.B., revised the manuscript and revised the work for important intellectual content. S.N., helped in the cellular assays. S.V.M. and A.Y., helped with the data categorization and interpreted the results. S.G.A., helped with the data categorization, interpreted the results, and participate in reviewing the manuscript. A.M., the corresponding author of the manuscript, designed and supervised the project, and revised the main text of the manuscript. The authors declare that all data were generated in-house and that no paper mill was used.

Corresponding author

Correspondence to Amir Ali Mokhtarzadeh .

Ethics declarations

Ethical approval.

Not applicable

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Bahojb Mahdavi, S.Z., Pouladi, N., Amini, M. et al. Let-7a-3p overexpression increases chemosensitivity to carmustine and synergistically promotes autophagy and suppresses cell survival in U87MG glioblastoma cancer cells. Naunyn-Schmiedeberg's Arch Pharmacol (2024). https://doi.org/10.1007/s00210-024-03060-4

Download citation

Received : 30 November 2023

Accepted : 18 March 2024

Published : 08 April 2024

DOI : https://doi.org/10.1007/s00210-024-03060-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Glioblastoma
• MicroRNA replacement therapy
• Combination chemotherapy
• Find a journal
• Publish with us
• Track your research

• Manage Account
• The Courier ePaper
• Evening Telegraph ePaper

Walking tours launched to showcase untold tales of Dundee’s medical research

The past, present and future of Dundee’s pioneering scientific research is being celebrated in a series of new walking tours.

Dundee University is inviting locals to explore some of the medical discoveries made in the city by following in the foosteps of the people who made them.

Much of the work on cancer, diabetes and drug development is well known but the team behind the project say there are many stories yet to be told.

The routes are part of a Dundee Discoveries map which was created by the university’s museum services and school of life sciences for this year’s Dundee Science Festival.

It features a series of self-guided walking tours detailing research into medicine, biology, forensics, nursing and dentistry, split into three routes – the City Centre, West End and University of Dundee, and Ninewells Hospital.

A scavenger hunt for young explorers has also been included, to ensure the whole family can get involved.

Matthew Jarron, curator of museum services at the university, said: “Our aim is to highlight locations around the city that have interesting medical or scientific connections.

“The tours give glimpses into the past by looking at historic landmarks, buildings, institutions and figures vital to the city’s progression in medicine and biology.

“For example, where the Malmaison Hotel now stands was once the site of Dundee’s cholera hospital, and close to Dundee Rep Theatre was the location of the GP surgery of Emily Moorhead and Alice Thomson, possibly the first all-female medical practice in Scotland.”

A digital copy of the map can be found on the Dundee Discoveries page of the university’s website , alongside additional information, images and videos.

Paper copies can also be picked up from Dundee Science Centre, which is open at weekends during the festival, which ends on November 29.

Mr Jarron added: “We have worked with many different staff across the university to showcase ground-breaking current research alongside the history, giving glimpses into the future by highlighting the places and people of today that are involved in a variety of crucial medical and scientific research.”

More from Higher Education

Lord George Robertson: From Dundee student sit-ins to meetings with Bush, Chirac and 'thin-skinned'…

Kilgraston open day will showcase unique school spirit and scholarship opportunities

Arbroath mum and son with 'special bond' graduate together from Dundee and Angus College

St Andrews lecture to be given by former EU Commissioner Dr Christos Stylianides

Dundee University graduate triumphs at prestigious film festival in the USA

Ninewells research: Is Far UVC lighting the way to a Covid-19 and flu-free future?