a research data collection

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Data Collection | Definition, Methods & Examples

Data Collection | Definition, Methods & Examples

Published on June 5, 2020 by Pritha Bhandari . Revised on June 21, 2023.

Data collection is a systematic process of gathering observations or measurements. Whether you are performing research for business, governmental or academic purposes, data collection allows you to gain first-hand knowledge and original insights into your research problem .

While methods and aims may differ between fields, the overall process of data collection remains largely the same. Before you begin collecting data, you need to consider:

• The  aim of the research
• The type of data that you will collect
• The methods and procedures you will use to collect, store, and process the data

To collect high-quality data that is relevant to your purposes, follow these four steps.

Table of contents

Step 1: define the aim of your research, step 2: choose your data collection method, step 3: plan your data collection procedures, step 4: collect the data, other interesting articles, frequently asked questions about data collection.

Before you start the process of data collection, you need to identify exactly what you want to achieve. You can start by writing a problem statement : what is the practical or scientific issue that you want to address and why does it matter?

Next, formulate one or more research questions that precisely define what you want to find out. Depending on your research questions, you might need to collect quantitative or qualitative data :

• Quantitative data is expressed in numbers and graphs and is analyzed through statistical methods .
• Qualitative data is expressed in words and analyzed through interpretations and categorizations.

If your aim is to test a hypothesis , measure something precisely, or gain large-scale statistical insights, collect quantitative data. If your aim is to explore ideas, understand experiences, or gain detailed insights into a specific context, collect qualitative data. If you have several aims, you can use a mixed methods approach that collects both types of data.

• Your first aim is to assess whether there are significant differences in perceptions of managers across different departments and office locations.
• Your second aim is to gather meaningful feedback from employees to explore new ideas for how managers can improve.

Prevent plagiarism. Run a free check.

Based on the data you want to collect, decide which method is best suited for your research.

• Experimental research is primarily a quantitative method.
• Interviews , focus groups , and ethnographies are qualitative methods.
• Surveys , observations, archival research and secondary data collection can be quantitative or qualitative methods.

Carefully consider what method you will use to gather data that helps you directly answer your research questions.

When you know which method(s) you are using, you need to plan exactly how you will implement them. What procedures will you follow to make accurate observations or measurements of the variables you are interested in?

For instance, if you’re conducting surveys or interviews, decide what form the questions will take; if you’re conducting an experiment, make decisions about your experimental design (e.g., determine inclusion and exclusion criteria ).

Operationalization

Sometimes your variables can be measured directly: for example, you can collect data on the average age of employees simply by asking for dates of birth. However, often you’ll be interested in collecting data on more abstract concepts or variables that can’t be directly observed.

Operationalization means turning abstract conceptual ideas into measurable observations. When planning how you will collect data, you need to translate the conceptual definition of what you want to study into the operational definition of what you will actually measure.

• You ask managers to rate their own leadership skills on 5-point scales assessing the ability to delegate, decisiveness and dependability.
• You ask their direct employees to provide anonymous feedback on the managers regarding the same topics.

You may need to develop a sampling plan to obtain data systematically. This involves defining a population , the group you want to draw conclusions about, and a sample, the group you will actually collect data from.

Your sampling method will determine how you recruit participants or obtain measurements for your study. To decide on a sampling method you will need to consider factors like the required sample size, accessibility of the sample, and timeframe of the data collection.

Standardizing procedures

If multiple researchers are involved, write a detailed manual to standardize data collection procedures in your study.

This means laying out specific step-by-step instructions so that everyone in your research team collects data in a consistent way – for example, by conducting experiments under the same conditions and using objective criteria to record and categorize observations. This helps you avoid common research biases like omitted variable bias or information bias .

This helps ensure the reliability of your data, and you can also use it to replicate the study in the future.

Creating a data management plan

Before beginning data collection, you should also decide how you will organize and store your data.

• If you are collecting data from people, you will likely need to anonymize and safeguard the data to prevent leaks of sensitive information (e.g. names or identity numbers).
• If you are collecting data via interviews or pencil-and-paper formats, you will need to perform transcriptions or data entry in systematic ways to minimize distortion.
• You can prevent loss of data by having an organization system that is routinely backed up.

Finally, you can implement your chosen methods to measure or observe the variables you are interested in.

The closed-ended questions ask participants to rate their manager’s leadership skills on scales from 1–5. The data produced is numerical and can be statistically analyzed for averages and patterns.

To ensure that high quality data is recorded in a systematic way, here are some best practices:

• Record all relevant information as and when you obtain data. For example, note down whether or how lab equipment is recalibrated during an experimental study.
• Double-check manual data entry for errors.
• If you collect quantitative data, you can assess the reliability and validity to get an indication of your data quality.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Likert scale

Research bias

• Implicit bias
• Framing effect
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

When conducting research, collecting original data has significant advantages:

• You can tailor data collection to your specific research aims (e.g. understanding the needs of your consumers or user testing your website)
• You can control and standardize the process for high reliability and validity (e.g. choosing appropriate measurements and sampling methods )

However, there are also some drawbacks: data collection can be time-consuming, labor-intensive and expensive. In some cases, it’s more efficient to use secondary data that has already been collected by someone else, but the data might be less reliable.

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

Reliability and validity are both about how well a method measures something:

• Reliability refers to the  consistency of a measure (whether the results can be reproduced under the same conditions).
• Validity   refers to the  accuracy of a measure (whether the results really do represent what they are supposed to measure).

If you are doing experimental research, you also have to consider the internal and external validity of your experiment.

Operationalization means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioral avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalize the variables that you want to measure.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 21). Data Collection | Definition, Methods & Examples. Scribbr. Retrieved April 15, 2024, from https://www.scribbr.com/methodology/data-collection/

Is this article helpful?

Pritha Bhandari

Other students also liked, qualitative vs. quantitative research | differences, examples & methods, sampling methods | types, techniques & examples, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• Privacy Policy

Buy Me a Coffee

Home » Data Collection – Methods Types and Examples

Data Collection – Methods Types and Examples

Table of Contents

Data Collection

Definition:

Data collection is the process of gathering and collecting information from various sources to analyze and make informed decisions based on the data collected. This can involve various methods, such as surveys, interviews, experiments, and observation.

In order for data collection to be effective, it is important to have a clear understanding of what data is needed and what the purpose of the data collection is. This can involve identifying the population or sample being studied, determining the variables to be measured, and selecting appropriate methods for collecting and recording data.

Types of Data Collection

Types of Data Collection are as follows:

Primary Data Collection

Primary data collection is the process of gathering original and firsthand information directly from the source or target population. This type of data collection involves collecting data that has not been previously gathered, recorded, or published. Primary data can be collected through various methods such as surveys, interviews, observations, experiments, and focus groups. The data collected is usually specific to the research question or objective and can provide valuable insights that cannot be obtained from secondary data sources. Primary data collection is often used in market research, social research, and scientific research.

Secondary Data Collection

Secondary data collection is the process of gathering information from existing sources that have already been collected and analyzed by someone else, rather than conducting new research to collect primary data. Secondary data can be collected from various sources, such as published reports, books, journals, newspapers, websites, government publications, and other documents.

Qualitative Data Collection

Qualitative data collection is used to gather non-numerical data such as opinions, experiences, perceptions, and feelings, through techniques such as interviews, focus groups, observations, and document analysis. It seeks to understand the deeper meaning and context of a phenomenon or situation and is often used in social sciences, psychology, and humanities. Qualitative data collection methods allow for a more in-depth and holistic exploration of research questions and can provide rich and nuanced insights into human behavior and experiences.

Quantitative Data Collection

Quantitative data collection is a used to gather numerical data that can be analyzed using statistical methods. This data is typically collected through surveys, experiments, and other structured data collection methods. Quantitative data collection seeks to quantify and measure variables, such as behaviors, attitudes, and opinions, in a systematic and objective way. This data is often used to test hypotheses, identify patterns, and establish correlations between variables. Quantitative data collection methods allow for precise measurement and generalization of findings to a larger population. It is commonly used in fields such as economics, psychology, and natural sciences.

Data Collection Methods

Data Collection Methods are as follows:

Surveys involve asking questions to a sample of individuals or organizations to collect data. Surveys can be conducted in person, over the phone, or online.

Interviews involve a one-on-one conversation between the interviewer and the respondent. Interviews can be structured or unstructured and can be conducted in person or over the phone.

Focus Groups

Focus groups are group discussions that are moderated by a facilitator. Focus groups are used to collect qualitative data on a specific topic.

Observation

Observation involves watching and recording the behavior of people, objects, or events in their natural setting. Observation can be done overtly or covertly, depending on the research question.

Experiments

Experiments involve manipulating one or more variables and observing the effect on another variable. Experiments are commonly used in scientific research.

Case Studies

Case studies involve in-depth analysis of a single individual, organization, or event. Case studies are used to gain detailed information about a specific phenomenon.

Secondary Data Analysis

Secondary data analysis involves using existing data that was collected for another purpose. Secondary data can come from various sources, such as government agencies, academic institutions, or private companies.

How to Collect Data

The following are some steps to consider when collecting data:

• Define the objective : Before you start collecting data, you need to define the objective of the study. This will help you determine what data you need to collect and how to collect it.
• Identify the data sources : Identify the sources of data that will help you achieve your objective. These sources can be primary sources, such as surveys, interviews, and observations, or secondary sources, such as books, articles, and databases.
• Determine the data collection method : Once you have identified the data sources, you need to determine the data collection method. This could be through online surveys, phone interviews, or face-to-face meetings.
• Develop a data collection plan : Develop a plan that outlines the steps you will take to collect the data. This plan should include the timeline, the tools and equipment needed, and the personnel involved.
• Test the data collection process: Before you start collecting data, test the data collection process to ensure that it is effective and efficient.
• Collect the data: Collect the data according to the plan you developed in step 4. Make sure you record the data accurately and consistently.
• Analyze the data: Once you have collected the data, analyze it to draw conclusions and make recommendations.
• Report the findings: Report the findings of your data analysis to the relevant stakeholders. This could be in the form of a report, a presentation, or a publication.
• Monitor and evaluate the data collection process: After the data collection process is complete, monitor and evaluate the process to identify areas for improvement in future data collection efforts.
• Ensure data quality: Ensure that the collected data is of high quality and free from errors. This can be achieved by validating the data for accuracy, completeness, and consistency.
• Maintain data security: Ensure that the collected data is secure and protected from unauthorized access or disclosure. This can be achieved by implementing data security protocols and using secure storage and transmission methods.
• Follow ethical considerations: Follow ethical considerations when collecting data, such as obtaining informed consent from participants, protecting their privacy and confidentiality, and ensuring that the research does not cause harm to participants.
• Use appropriate data analysis methods : Use appropriate data analysis methods based on the type of data collected and the research objectives. This could include statistical analysis, qualitative analysis, or a combination of both.
• Record and store data properly: Record and store the collected data properly, in a structured and organized format. This will make it easier to retrieve and use the data in future research or analysis.
• Collaborate with other stakeholders : Collaborate with other stakeholders, such as colleagues, experts, or community members, to ensure that the data collected is relevant and useful for the intended purpose.

Applications of Data Collection

Data collection methods are widely used in different fields, including social sciences, healthcare, business, education, and more. Here are some examples of how data collection methods are used in different fields:

• Social sciences : Social scientists often use surveys, questionnaires, and interviews to collect data from individuals or groups. They may also use observation to collect data on social behaviors and interactions. This data is often used to study topics such as human behavior, attitudes, and beliefs.
• Healthcare : Data collection methods are used in healthcare to monitor patient health and track treatment outcomes. Electronic health records and medical charts are commonly used to collect data on patients’ medical history, diagnoses, and treatments. Researchers may also use clinical trials and surveys to collect data on the effectiveness of different treatments.
• Business : Businesses use data collection methods to gather information on consumer behavior, market trends, and competitor activity. They may collect data through customer surveys, sales reports, and market research studies. This data is used to inform business decisions, develop marketing strategies, and improve products and services.
• Education : In education, data collection methods are used to assess student performance and measure the effectiveness of teaching methods. Standardized tests, quizzes, and exams are commonly used to collect data on student learning outcomes. Teachers may also use classroom observation and student feedback to gather data on teaching effectiveness.
• Agriculture : Farmers use data collection methods to monitor crop growth and health. Sensors and remote sensing technology can be used to collect data on soil moisture, temperature, and nutrient levels. This data is used to optimize crop yields and minimize waste.
• Environmental sciences : Environmental scientists use data collection methods to monitor air and water quality, track climate patterns, and measure the impact of human activity on the environment. They may use sensors, satellite imagery, and laboratory analysis to collect data on environmental factors.
• Transportation : Transportation companies use data collection methods to track vehicle performance, optimize routes, and improve safety. GPS systems, on-board sensors, and other tracking technologies are used to collect data on vehicle speed, fuel consumption, and driver behavior.

Examples of Data Collection

Examples of Data Collection are as follows:

• Traffic Monitoring: Cities collect real-time data on traffic patterns and congestion through sensors on roads and cameras at intersections. This information can be used to optimize traffic flow and improve safety.
• Social Media Monitoring : Companies can collect real-time data on social media platforms such as Twitter and Facebook to monitor their brand reputation, track customer sentiment, and respond to customer inquiries and complaints in real-time.
• Weather Monitoring: Weather agencies collect real-time data on temperature, humidity, air pressure, and precipitation through weather stations and satellites. This information is used to provide accurate weather forecasts and warnings.
• Stock Market Monitoring : Financial institutions collect real-time data on stock prices, trading volumes, and other market indicators to make informed investment decisions and respond to market fluctuations in real-time.
• Health Monitoring : Medical devices such as wearable fitness trackers and smartwatches can collect real-time data on a person’s heart rate, blood pressure, and other vital signs. This information can be used to monitor health conditions and detect early warning signs of health issues.

Purpose of Data Collection

The purpose of data collection can vary depending on the context and goals of the study, but generally, it serves to:

• Provide information: Data collection provides information about a particular phenomenon or behavior that can be used to better understand it.
• Measure progress : Data collection can be used to measure the effectiveness of interventions or programs designed to address a particular issue or problem.
• Support decision-making : Data collection provides decision-makers with evidence-based information that can be used to inform policies, strategies, and actions.
• Identify trends : Data collection can help identify trends and patterns over time that may indicate changes in behaviors or outcomes.
• Monitor and evaluate : Data collection can be used to monitor and evaluate the implementation and impact of policies, programs, and initiatives.

When to use Data Collection

Data collection is used when there is a need to gather information or data on a specific topic or phenomenon. It is typically used in research, evaluation, and monitoring and is important for making informed decisions and improving outcomes.

Data collection is particularly useful in the following scenarios:

• Research : When conducting research, data collection is used to gather information on variables of interest to answer research questions and test hypotheses.
• Evaluation : Data collection is used in program evaluation to assess the effectiveness of programs or interventions, and to identify areas for improvement.
• Monitoring : Data collection is used in monitoring to track progress towards achieving goals or targets, and to identify any areas that require attention.
• Decision-making: Data collection is used to provide decision-makers with information that can be used to inform policies, strategies, and actions.
• Quality improvement : Data collection is used in quality improvement efforts to identify areas where improvements can be made and to measure progress towards achieving goals.

Characteristics of Data Collection

Data collection can be characterized by several important characteristics that help to ensure the quality and accuracy of the data gathered. These characteristics include:

• Validity : Validity refers to the accuracy and relevance of the data collected in relation to the research question or objective.
• Reliability : Reliability refers to the consistency and stability of the data collection process, ensuring that the results obtained are consistent over time and across different contexts.
• Objectivity : Objectivity refers to the impartiality of the data collection process, ensuring that the data collected is not influenced by the biases or personal opinions of the data collector.
• Precision : Precision refers to the degree of accuracy and detail in the data collected, ensuring that the data is specific and accurate enough to answer the research question or objective.
• Timeliness : Timeliness refers to the efficiency and speed with which the data is collected, ensuring that the data is collected in a timely manner to meet the needs of the research or evaluation.
• Ethical considerations : Ethical considerations refer to the ethical principles that must be followed when collecting data, such as ensuring confidentiality and obtaining informed consent from participants.

Advantages of Data Collection

There are several advantages of data collection that make it an important process in research, evaluation, and monitoring. These advantages include:

• Better decision-making : Data collection provides decision-makers with evidence-based information that can be used to inform policies, strategies, and actions, leading to better decision-making.
• Improved understanding: Data collection helps to improve our understanding of a particular phenomenon or behavior by providing empirical evidence that can be analyzed and interpreted.
• Evaluation of interventions: Data collection is essential in evaluating the effectiveness of interventions or programs designed to address a particular issue or problem.
• Identifying trends and patterns: Data collection can help identify trends and patterns over time that may indicate changes in behaviors or outcomes.
• Increased accountability: Data collection increases accountability by providing evidence that can be used to monitor and evaluate the implementation and impact of policies, programs, and initiatives.
• Validation of theories: Data collection can be used to test hypotheses and validate theories, leading to a better understanding of the phenomenon being studied.
• Improved quality: Data collection is used in quality improvement efforts to identify areas where improvements can be made and to measure progress towards achieving goals.

Limitations of Data Collection

While data collection has several advantages, it also has some limitations that must be considered. These limitations include:

• Bias : Data collection can be influenced by the biases and personal opinions of the data collector, which can lead to inaccurate or misleading results.
• Sampling bias : Data collection may not be representative of the entire population, resulting in sampling bias and inaccurate results.
• Cost : Data collection can be expensive and time-consuming, particularly for large-scale studies.
• Limited scope: Data collection is limited to the variables being measured, which may not capture the entire picture or context of the phenomenon being studied.
• Ethical considerations : Data collection must follow ethical principles to protect the rights and confidentiality of the participants, which can limit the type of data that can be collected.
• Data quality issues: Data collection may result in data quality issues such as missing or incomplete data, measurement errors, and inconsistencies.
• Limited generalizability : Data collection may not be generalizable to other contexts or populations, limiting the generalizability of the findings.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Delimitations in Research – Types, Examples and...

Research Process – Steps, Examples and Tips

Research Design – Types, Methods and Examples

Institutional Review Board – Application Sample...

Evaluating Research – Process, Examples and...

Research Questions – Types, Examples and Writing...

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case NPS+ Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research

Data Collection: What It Is, Methods & Tools + Examples

Let’s face it, no one wants to make decisions based on guesswork or gut feelings. The most important objective of data collection is to ensure that the data gathered is reliable and packed to the brim with juicy insights that can be analyzed and turned into data-driven decisions. There’s nothing better than good statistical analysis .

LEARN ABOUT: Level of Analysis

Collecting high-quality data is essential for conducting market research, analyzing user behavior, or just trying to get a handle on business operations. With the right approach and a few handy tools, gathering reliable and informative data.

So, let’s get ready to collect some data because when it comes to data collection, it’s all about the details.

Content Index

What is Data Collection?

Data collection methods, data collection examples, reasons to conduct online research and data collection, conducting customer surveys for data collection to multiply sales, steps to effectively conduct an online survey for data collection, survey design for data collection.

Data collection is the procedure of collecting, measuring, and analyzing accurate insights for research using standard validated techniques.

Put simply, data collection is the process of gathering information for a specific purpose. It can be used to answer research questions, make informed business decisions, or improve products and services.

To collect data, we must first identify what information we need and how we will collect it. We can also evaluate a hypothesis based on collected data. In most cases, data collection is the primary and most important step for research. The approach to data collection is different for different fields of study, depending on the required information.

LEARN ABOUT: Action Research

There are many ways to collect information when doing research. The data collection methods that the researcher chooses will depend on the research question posed. Some data collection methods include surveys, interviews, tests, physiological evaluations, observations, reviews of existing records, and biological samples. Let’s explore them.

LEARN ABOUT: Best Data Collection Tools

Phone vs. Online vs. In-Person Interviews

Essentially there are four choices for data collection – in-person interviews, mail, phone, and online. There are pros and cons to each of these modes.

• Pros: In-depth and a high degree of confidence in the data
• Cons: Time-consuming, expensive, and can be dismissed as anecdotal
• Pros: Can reach anyone and everyone – no barrier
• Cons: Expensive, data collection errors, lag time
• Pros: High degree of confidence in the data collected, reach almost anyone
• Cons: Expensive, cannot self-administer, need to hire an agency
• Pros: Cheap, can self-administer, very low probability of data errors
• Cons: Not all your customers might have an email address/be on the internet, customers may be wary of divulging information online.

In-person interviews always are better, but the big drawback is the trap you might fall into if you don’t do them regularly. It is expensive to regularly conduct interviews and not conducting enough interviews might give you false positives. Validating your research is almost as important as designing and conducting it.

We’ve seen many instances where after the research is conducted – if the results do not match up with the “gut-feel” of upper management, it has been dismissed off as anecdotal and a “one-time” phenomenon. To avoid such traps, we strongly recommend that data-collection be done on an “ongoing and regular” basis.

LEARN ABOUT: Research Process Steps

This will help you compare and analyze the change in perceptions according to marketing for your products/services. The other issue here is sample size. To be confident with your research, you must interview enough people to weed out the fringe elements.

A couple of years ago there was a lot of discussion about online surveys and their statistical analysis plan . The fact that not every customer had internet connectivity was one of the main concerns.

LEARN ABOUT:   Statistical Analysis Methods

Although some of the discussions are still valid, the reach of the internet as a means of communication has become vital in the majority of customer interactions. According to the US Census Bureau, the number of households with computers has doubled between 1997 and 2001.

Learn more: Quantitative Market Research

In 2001 nearly 50% of households had a computer. Nearly 55% of all households with an income of more than 35,000 have internet access, which jumps to 70% for households with an annual income of 50,000. This data is from the US Census Bureau for 2001.

There are primarily three modes of data collection that can be employed to gather feedback – Mail, Phone, and Online. The method actually used for data collection is really a cost-benefit analysis. There is no slam-dunk solution but you can use the table below to understand the risks and advantages associated with each of the mediums:

Keep in mind, the reach here is defined as “All U.S. Households.” In most cases, you need to look at how many of your customers are online and determine. If all your customers have email addresses, you have a 100% reach of your customers.

Another important thing to keep in mind is the ever-increasing dominance of cellular phones over landline phones. United States FCC rules prevent automated dialing and calling cellular phone numbers and there is a noticeable trend towards people having cellular phones as the only voice communication device.

This introduces the inability to reach cellular phone customers who are dropping home phone lines in favor of going entirely wireless. Even if automated dialing is not used, another FCC rule prohibits from phoning anyone who would have to pay for the call.

Learn more: Qualitative Market Research

Multi-Mode Surveys

Surveys, where the data is collected via different modes (online, paper, phone etc.), is also another way of going. It is fairly straightforward and easy to have an online survey and have data-entry operators to enter in data (from the phone as well as paper surveys) into the system. The same system can also be used to collect data directly from the respondents.

Learn more: Survey Research

Data collection is an important aspect of research. Let’s consider an example of a mobile manufacturer, company X, which is launching a new product variant. To conduct research about features, price range, target market, competitor analysis, etc. data has to be collected from appropriate sources.

The marketing team can conduct various data collection activities such as online surveys or focus groups .

The survey should have all the right questions about features and pricing, such as “What are the top 3 features expected from an upcoming product?” or “How much are your likely to spend on this product?” or “Which competitors provide similar products?” etc.

For conducting a focus group, the marketing team should decide the participants and the mediator. The topic of discussion and objective behind conducting a focus group should be clarified beforehand to conduct a conclusive discussion.

Data collection methods are chosen depending on the available resources. For example, conducting questionnaires and surveys would require the least resources, while focus groups require moderately high resources.

Feedback is a vital part of any organization’s growth. Whether you conduct regular focus groups to elicit information from key players or, your account manager calls up all your marquee  accounts to find out how things are going – essentially they are all processes to find out from your customers’ eyes – How are we doing? What can we do better?

Online surveys are just another medium to collect feedback from your customers , employees and anyone your business interacts with. With the advent of Do-It-Yourself tools for online surveys, data collection on the internet has become really easy, cheap and effective.

Learn more:  Online Research

It is a well-established marketing fact that acquiring a new customer is 10 times more difficult and expensive than retaining an existing one. This is one of the fundamental driving forces behind the extensive adoption and interest in CRM and related customer retention tactics.

In a research study conducted by Rice University Professor Dr. Paul Dholakia and Dr. Vicki Morwitz, published in Harvard Business Review, the experiment inferred that the simple fact of asking customers how an organization was performing by itself to deliver results proved to be an effective customer retention strategy.

In the research study, conducted over the course of a year, one set of customers were sent out a satisfaction and opinion survey and the other set was not surveyed. In the next one year, the group that took the survey saw twice the number of people continuing and renewing their loyalty towards the organization data .

Learn more: Research Design

The research study provided a couple of interesting reasons on the basis of consumer psychology, behind this phenomenon:

• Satisfaction surveys boost the customers’ desire to be coddled and induce positive feelings. This crops from a section of the human psychology that intends to “appreciate” a product or service they already like or prefer. The survey feedback collection method is solely a medium to convey this. The survey is a vehicle to “interact” with the company and reinforces the customer’s commitment to the company.
• Surveys may increase awareness of auxiliary products and services. Surveys can be considered modes of both inbound as well as outbound communication. Surveys are generally considered to be a data collection and analysis source. Most people are unaware of the fact that consumer surveys can also serve as a medium for distributing data. It is important to note a few caveats here.
• In most countries, including the US, “selling under the guise of research” is illegal. b. However, we all know that information is distributed while collecting information. c. Other disclaimers may be included in the survey to ensure users are aware of this fact. For example: “We will collect your opinion and inform you about products and services that have come online in the last year…”
• Induced Judgments:  The entire procedure of asking people for their feedback can prompt them to build an opinion on something they otherwise would not have thought about. This is a very underlying yet powerful argument that can be compared to the “Product Placement” strategy currently used for marketing products in mass media like movies and television shows. One example is the extensive and exclusive use of the “mini-Cooper” in the blockbuster movie “Italian Job.” This strategy is questionable and should be used with great caution.

Surveys should be considered as a critical tool in the customer journey dialog. The best thing about surveys is its ability to carry “bi-directional” information. The research conducted by Paul Dholakia and Vicki Morwitz shows that surveys not only get you the information that is critical for your business, but also enhances and builds upon the established relationship you have with your customers.

Recent technological advances have made it incredibly easy to conduct real-time surveys and  opinion polls . Online tools make it easy to frame questions and answers and create surveys on the Web. Distributing surveys via email, website links or even integration with online CRM tools like Salesforce.com have made online surveying a quick-win solution.

So, you’ve decided to conduct an online survey. There are a few questions in your mind that you would like answered, and you are looking for a fast and inexpensive way to find out more about your customers, clients, etc.

First and foremost thing you need to decide what the smart objectives of the study are. Ensure that you can phrase these objectives as questions or measurements. If you can’t, you are better off looking at other data sources like focus groups and other qualitative methods . The data collected via online surveys is dominantly quantitative in nature.

Review the basic objectives of the study. What are you trying to discover? What actions do you  want to take as a result of the survey? –  Answers to these questions help in validating collected data. Online surveys are just one way of collecting and quantifying data .

Learn more: Qualitative Data & Qualitative Data Collection Methods

• Visualize all of the relevant information items you would like to have. What will the output survey research report look like? What charts and graphs will be prepared? What information do you need to be assured that action is warranted?
• Assign ranks to each topic (1 and 2) according to their priority, including the most important topics first. Revisit these items again to ensure that the objectives, topics, and information you need are appropriate. Remember, you can’t solve the research problem if you ask the wrong questions.
• How easy or difficult is it for the respondent to provide information on each topic? If it is difficult, is there an alternative medium to gain insights by asking a different question? This is probably the most important step. Online surveys have to be Precise, Clear and Concise. Due to the nature of the internet and the fluctuations involved, if your questions are too difficult to understand, the survey dropout rate will be high.
• Create a sequence for the topics that are unbiased. Make sure that the questions asked first do not bias the results of the next questions. Sometimes providing too much information, or disclosing purpose of the study can create bias. Once you have a series of decided topics, you can have a basic structure of a survey. It is always advisable to add an “Introductory” paragraph before the survey to explain the project objective and what is expected of the respondent. It is also sensible to have a “Thank You” text as well as information about where to find the results of the survey when they are published.
• Page Breaks – The attention span of respondents can be very low when it comes to a long scrolling survey. Add page breaks as wherever possible. Having said that, a single question per page can also hamper response rates as it increases the time to complete the survey as well as increases the chances for dropouts.
• Branching – Create smart and effective surveys with the implementation of branching wherever required. Eliminate the use of text such as, “If you answered No to Q1 then Answer Q4” – this leads to annoyance amongst respondents which result in increase survey dropout rates. Design online surveys using the branching logic so that appropriate questions are automatically routed based on previous responses.
• Write the questions . Initially, write a significant number of survey questions out of which you can use the one which is best suited for the survey. Divide the survey into sections so that respondents do not get confused seeing a long list of questions.
• Sequence the questions so that they are unbiased.
• Repeat all of the steps above to find any major holes. Are the questions really answered? Have someone review it for you.
• Time the length of the survey. A survey should take less than five minutes. At three to four research questions per minute, you are limited to about 15 questions. One open end text question counts for three multiple choice questions. Most online software tools will record the time taken for the respondents to answer questions.
• Include a few open-ended survey questions that support your survey object. This will be a type of feedback survey.
• Send an email to the project survey to your test group and then email the feedback survey afterward.
• This way, you can have your test group provide their opinion about the functionality as well as usability of your project survey by using the feedback survey.
• Make changes to your questionnaire based on the received feedback.
• Send the survey out to all your respondents!

Online surveys have, over the course of time, evolved into an effective alternative to expensive mail or telephone surveys. However, you must be aware of a few conditions that need to be met for online surveys. If you are trying to survey a sample representing the target population, please remember that not everyone is online.

Moreover, not everyone is receptive to an online survey also. Generally, the demographic segmentation of younger individuals is inclined toward responding to an online survey.

Learn More: Examples of Qualitarive Data in Education

Good survey design is crucial for accurate data collection. From question-wording to response options, let’s explore how to create effective surveys that yield valuable insights with our tips to survey design.

• Writing Great Questions for data collection

Writing great questions can be considered an art. Art always requires a significant amount of hard work, practice, and help from others.

The questions in a survey need to be clear, concise, and unbiased. A poorly worded question or a question with leading language can result in inaccurate or irrelevant responses, ultimately impacting the data’s validity.

Moreover, the questions should be relevant and specific to the research objectives. Questions that are irrelevant or do not capture the necessary information can lead to incomplete or inconsistent responses too.

• Avoid loaded or leading words or questions

A small change in content can produce effective results. Words such as could , should and might are all used for almost the same purpose, but may produce a 20% difference in agreement to a question. For example, “The management could.. should.. might.. have shut the factory”.

Intense words such as – prohibit or action, representing control or action, produce similar results. For example,  “Do you believe Donald Trump should prohibit insurance companies from raising rates?”.

Sometimes the content is just biased. For instance, “You wouldn’t want to go to Rudolpho’s Restaurant for the organization’s annual party, would you?”

• Misplaced questions

Questions should always reference the intended context, and questions placed out of order or without its requirement should be avoided. Generally, a funnel approach should be implemented – generic questions should be included in the initial section of the questionnaire as a warm-up and specific ones should follow. Toward the end, demographic or geographic questions should be included.

• Mutually non-overlapping response categories

Multiple-choice answers should be mutually unique to provide distinct choices. Overlapping answer options frustrate the respondent and make interpretation difficult at best. Also, the questions should always be precise.

For example: “Do you like water juice?”

This question is vague. In which terms is the liking for orange juice is to be rated? – Sweetness, texture, price, nutrition etc.

• Avoid the use of confusing/unfamiliar words

Asking about industry-related terms such as caloric content, bits, bytes, MBS , as well as other terms and acronyms can confuse respondents . Ensure that the audience understands your language level, terminology, and, above all, the question you ask.

• Non-directed questions give respondents excessive leeway

In survey design for data collection, non-directed questions can give respondents excessive leeway, which can lead to vague and unreliable data. These types of questions are also known as open-ended questions, and they do not provide any structure for the respondent to follow.

For instance, a non-directed question like “ What suggestions do you have for improving our shoes?” can elicit a wide range of answers, some of which may not be relevant to the research objectives. Some respondents may give short answers, while others may provide lengthy and detailed responses, making comparing and analyzing the data challenging.

To avoid these issues, it’s essential to ask direct questions that are specific and have a clear structure. Closed-ended questions, for example, offer structured response options and can be easier to analyze as they provide a quantitative measure of respondents’ opinions.

• Never force questions

There will always be certain questions that cross certain privacy rules. Since privacy is an important issue for most people, these questions should either be eliminated from the survey or not be kept as mandatory. Survey questions about income, family income, status, religious and political beliefs, etc., should always be avoided as they are considered to be intruding, and respondents can choose not to answer them.

• Unbalanced answer options in scales

Unbalanced answer options in scales such as Likert Scale and Semantic Scale may be appropriate for some situations and biased in others. When analyzing a pattern in eating habits, a study used a quantity scale that made obese people appear in the middle of the scale with the polar ends reflecting a state where people starve and an irrational amount to consume. There are cases where we usually do not expect poor service, such as hospitals.

• Questions that cover two points

In survey design for data collection, questions that cover two points can be problematic for several reasons. These types of questions are often called “double-barreled” questions and can cause confusion for respondents, leading to inaccurate or irrelevant data.

For instance, a question like “Do you like the food and the service at the restaurant?” covers two points, the food and the service, and it assumes that the respondent has the same opinion about both. If the respondent only liked the food, their opinion of the service could affect their answer.

It’s important to ask one question at a time to avoid confusion and ensure that the respondent’s answer is focused and accurate. This also applies to questions with multiple concepts or ideas. In these cases, it’s best to break down the question into multiple questions that address each concept or idea separately.

• Dichotomous questions

Dichotomous questions are used in case you want a distinct answer, such as: Yes/No or Male/Female . For example, the question “Do you think this candidate will win the election?” can be Yes or No.

• Avoid the use of long questions

The use of long questions will definitely increase the time taken for completion, which will generally lead to an increase in the survey dropout rate. Multiple-choice questions are the longest and most complex, and open-ended questions are the shortest and easiest to answer.

Data collection is an essential part of the research process, whether you’re conducting scientific experiments, market research, or surveys. The methods and tools used for data collection will vary depending on the research type, the sample size required, and the resources available.

Several data collection methods include surveys, observations, interviews, and focus groups. We learn each method has advantages and disadvantages, and choosing the one that best suits the research goals is important.

With the rise of technology, many tools are now available to facilitate data collection, including online survey software and data visualization tools. These tools can help researchers collect, store, and analyze data more efficiently, providing greater results and accuracy.

By understanding the various methods and tools available for data collection, we can develop a solid foundation for conducting research. With these research skills , we can make informed decisions, solve problems, and contribute to advancing our understanding of the world around us.

Analyze your survey data to gauge in-depth market drivers, including competitive intelligence, purchasing behavior, and price sensitivity, with QuestionPro.

You will obtain accurate insights with various techniques, including conjoint analysis, MaxDiff analysis, sentiment analysis, TURF analysis, heatmap analysis, etc. Export quality data to external in-depth analysis tools such as SPSS and R Software, and integrate your research with external business applications. Everything you need for your data collection. Start today for free!

LEARN MORE         FREE TRIAL

MORE LIKE THIS

Top 13 A/B Testing Software for Optimizing Your Website

Apr 12, 2024

21 Best Contact Center Experience Software in 2024

Government Customer Experience: Impact on Government Service

Apr 11, 2024

Employee Engagement App: Top 11 For Workforce Improvement 

Apr 10, 2024

Other categories

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Uncategorized
• Video Learning Series
• What’s Coming Up
• Workforce Intelligence

Data collection in research: Your complete guide

Last updated

31 January 2023

Reviewed by

Cathy Heath

In the late 16th century, Francis Bacon coined the phrase "knowledge is power," which implies that knowledge is a powerful force, like physical strength. In the 21st century, knowledge in the form of data is unquestionably powerful.

But data isn't something you just have - you need to collect it. This means utilizing a data collection process and turning the collected data into knowledge that you can leverage into a successful strategy for your business or organization.

Believe it or not, there's more to data collection than just conducting a Google search. In this complete guide, we shine a spotlight on data collection, outlining what it is, types of data collection methods, common challenges in data collection, data collection techniques, and the steps involved in data collection.

Analyze all your data in one place

Uncover hidden nuggets in all types of qualitative data when you analyze it in Dovetail

• What is data collection?

There are two specific data collection techniques: primary and secondary data collection. Primary data collection is the process of gathering data directly from sources. It's often considered the most reliable data collection method, as researchers can collect information directly from respondents.

Secondary data collection is data that has already been collected by someone else and is readily available. This data is usually less expensive and quicker to obtain than primary data.

• What are the different methods of data collection?

There are several data collection methods, which can be either manual or automated. Manual data collection involves collecting data manually, typically with pen and paper, while computerized data collection involves using software to collect data from online sources, such as social media, website data, transaction data, etc. 

Here are the five most popular methods of data collection:

Surveys are a very popular method of data collection that organizations can use to gather information from many people. Researchers can conduct multi-mode surveys that reach respondents in different ways, including in person, by mail, over the phone, or online.

As a method of data collection, surveys have several advantages. For instance, they are relatively quick and easy to administer, you can be flexible in what you ask, and they can be tailored to collect data on various topics or from certain demographics.

However, surveys also have several disadvantages. For instance, they can be expensive to administer, and the results may not represent the population as a whole. Additionally, survey data can be challenging to interpret. It may also be subject to bias if the questions are not well-designed or if the sample of people surveyed is not representative of the population of interest.

Interviews are a common method of collecting data in social science research. You can conduct interviews in person, over the phone, or even via email or online chat.

Interviews are a great way to collect qualitative and quantitative data . Qualitative interviews are likely your best option if you need to collect detailed information about your subjects' experiences or opinions. If you need to collect more generalized data about your subjects' demographics or attitudes, then quantitative interviews may be a better option.

Interviews are relatively quick and very flexible, allowing you to ask follow-up questions and explore topics in more depth. The downside is that interviews can be time-consuming and expensive due to the amount of information to be analyzed. They are also prone to bias, as both the interviewer and the respondent may have certain expectations or preconceptions that may influence the data.

Direct observation

Observation is a direct way of collecting data. It can be structured (with a specific protocol to follow) or unstructured (simply observing without a particular plan).

Organizations and businesses use observation as a data collection method to gather information about their target market, customers, or competition. Businesses can learn about consumer behavior, preferences, and trends by observing people using their products or service.

There are two types of observation: participatory and non-participatory. In participatory observation, the researcher is actively involved in the observed activities. This type of observation is used in ethnographic research , where the researcher wants to understand a group's culture and social norms. Non-participatory observation is when researchers observe from a distance and do not interact with the people or environment they are studying.

There are several advantages to using observation as a data collection method. It can provide insights that may not be apparent through other methods, such as surveys or interviews. Researchers can also observe behavior in a natural setting, which can provide a more accurate picture of what people do and how and why they behave in a certain context.

There are some disadvantages to using observation as a method of data collection. It can be time-consuming, intrusive, and expensive to observe people for extended periods. Observations can also be tainted if the researcher is not careful to avoid personal biases or preconceptions.

Automated data collection

Business applications and websites are increasingly collecting data electronically to improve the user experience or for marketing purposes.

There are a few different ways that organizations can collect data automatically. One way is through cookies, which are small pieces of data stored on a user's computer. They track a user's browsing history and activity on a site, measuring levels of engagement with a business’s products or services, for example.

Another way organizations can collect data automatically is through web beacons. Web beacons are small images embedded on a web page to track a user's activity.

Finally, organizations can also collect data through mobile apps, which can track user location, device information, and app usage. This data can be used to improve the user experience and for marketing purposes.

Automated data collection is a valuable tool for businesses, helping improve the user experience or target marketing efforts. Businesses should aim to be transparent about how they collect and use this data.

Sourcing data through information service providers

Organizations need to be able to collect data from a variety of sources, including social media, weblogs, and sensors. The process to do this and then use the data for action needs to be efficient, targeted, and meaningful.

In the era of big data, organizations are increasingly turning to information service providers (ISPs) and other external data sources to help them collect data to make crucial decisions. 

Information service providers help organizations collect data by offering personalized services that suit the specific needs of the organizations. These services can include data collection, analysis, management, and reporting. By partnering with an ISP, organizations can gain access to the newest technology and tools to help them to gather and manage data more effectively.

There are also several tools and techniques that organizations can use to collect data from external sources, such as web scraping, which collects data from websites, and data mining, which involves using algorithms to extract data from large data sets. 

Organizations can also use APIs (application programming interface) to collect data from external sources. APIs allow organizations to access data stored in another system and share and integrate it into their own systems.

Finally, organizations can also use manual methods to collect data from external sources. This can involve contacting companies or individuals directly to request data, by using the right tools and methods to get the insights they need.

• What are common challenges in data collection?

There are many challenges that researchers face when collecting data. Here are five common examples:

Big data environments

Data collection can be a challenge in big data environments for several reasons. It can be located in different places, such as archives, libraries, or online. The sheer volume of data can also make it difficult to identify the most relevant data sets.

Second, the complexity of data sets can make it challenging to extract the desired information. Third, the distributed nature of big data environments can make it difficult to collect data promptly and efficiently.

Therefore it is important to have a well-designed data collection strategy to consider the specific needs of the organization and what data sets are the most relevant. Alongside this, consideration should be made regarding the tools and resources available to support data collection and protect it from unintended use.

Data bias is a common challenge in data collection. It occurs when data is collected from a sample that is not representative of the population of interest. 

There are different types of data bias, but some common ones include selection bias, self-selection bias, and response bias. Selection bias can occur when the collected data does not represent the population being studied. For example, if a study only includes data from people who volunteer to participate, that data may not represent the general population.

Self-selection bias can also occur when people self-select into a study, such as by taking part only if they think they will benefit from it. Response bias happens when people respond in a way that is not honest or accurate, such as by only answering questions that make them look good. 

These types of data bias present a challenge because they can lead to inaccurate results and conclusions about behaviors, perceptions, and trends. Data bias can be avoided by identifying potential sources or themes of bias and setting guidelines for eliminating them.

Lack of quality assurance processes

One of the biggest challenges in data collection is the lack of quality assurance processes. This can lead to several problems, including incorrect data, missing data, and inconsistencies between data sets.

Quality assurance is important because there are many data sources, and each source may have different levels of quality or corruption. There are also different ways of collecting data, and data quality may vary depending on the method used. 

There are several ways to improve quality assurance in data collection. These include developing clear and consistent goals and guidelines for data collection, implementing quality control measures, using standardized procedures, and employing data validation techniques. By taking these steps, you can ensure that your data is of adequate quality to inform decision-making.

Limited access to data

Another challenge in data collection is limited access to data. This can be due to several reasons, including privacy concerns, the sensitive nature of the data, security concerns, or simply the fact that data is not readily available.

Legal and compliance regulations

Most countries have regulations governing how data can be collected, used, and stored. In some cases, data collected in one country may not be used in another. This means gaining a global perspective can be a challenge. 

For example, if a company is required to comply with the EU General Data Protection Regulation (GDPR), it may not be able to collect data from individuals in the EU without their explicit consent. This can make it difficult to collect data from a target audience.

Legal and compliance regulations can be complex, and it's important to ensure that all data collected is done so in a way that complies with the relevant regulations.

• What are the key steps in the data collection process?

There are five steps involved in the data collection process. They are:

1. Decide what data you want to gather

Have a clear understanding of the questions you are asking, and then consider where the answers might lie and how you might obtain them. This saves time and resources by avoiding the collection of irrelevant data, and helps maintain the quality of your datasets. 

2. Establish a deadline for data collection

Establishing a deadline for data collection helps you avoid collecting too much data, which can be costly and time-consuming to analyze. It also allows you to plan for data analysis and prompt interpretation. Finally, it helps you meet your research goals and objectives and allows you to move forward.

3. Select a data collection approach

The data collection approach you choose will depend on different factors, including the type of data you need, available resources, and the project timeline. For instance, if you need qualitative data, you might choose a focus group or interview methodology. If you need quantitative data , then a survey or observational study may be the most appropriate form of collection.

4. Gather information

When collecting data for your business, identify your business goals first. Once you know what you want to achieve, you can start collecting data to reach those goals. The most important thing is to ensure that the data you collect is reliable and valid. Otherwise, any decisions you make using the data could result in a negative outcome for your business.

5. Examine the information and apply your findings

As a researcher, it's important to examine the data you're collecting and analyzing before you apply your findings. This is because data can be misleading, leading to inaccurate conclusions. Ask yourself whether it is what you are expecting? Is it similar to other datasets you have looked at? 

There are many scientific ways to examine data, but some common methods include:

looking at the distribution of data points

examining the relationships between variables

looking for outliers

By taking the time to examine your data and noticing any patterns, strange or otherwise, you can avoid making mistakes that could invalidate your research.

• How qualitative analysis software streamlines the data collection process

Knowledge derived from data does indeed carry power. However, if you don't convert the knowledge into action, it will remain a resource of unexploited energy and wasted potential.

Luckily, data collection tools enable organizations to streamline their data collection and analysis processes and leverage the derived knowledge to grow their businesses. For instance, qualitative analysis software can be highly advantageous in data collection by streamlining the process, making it more efficient and less time-consuming.

Secondly, qualitative analysis software provides a structure for data collection and analysis, ensuring that data is of high quality. It can also help to uncover patterns and relationships that would otherwise be difficult to discern. Moreover, you can use it to replace more expensive data collection methods, such as focus groups or surveys.

Overall, qualitative analysis software can be valuable for any researcher looking to collect and analyze data. By increasing efficiency, improving data quality, and providing greater insights, qualitative software can help to make the research process much more efficient and effective.

Learn more about qualitative research data analysis software

Get started today.

Go from raw data to valuable insights with a flexible research platform

Editor’s picks

Last updated: 21 December 2023

Last updated: 16 December 2023

Last updated: 6 October 2023

Last updated: 5 March 2024

Last updated: 25 November 2023

Last updated: 15 February 2024

Last updated: 11 March 2024

Last updated: 12 December 2023

Last updated: 6 March 2024

Last updated: 10 April 2023

Last updated: 20 December 2023

Latest articles

Related topics, log in or sign up.

Get started for free

Data Collection Methods

Data collection is a process of collecting information from all the relevant sources to find answers to the research problem, test the hypothesis (if you are following deductive approach ) and evaluate the outcomes. Data collection methods can be divided into two categories: secondary methods of data collection and primary methods of data collection.

Secondary Data Collection Methods

Secondary data is a type of data that has already been published in books, newspapers, magazines, journals, online portals etc.  There is an abundance of data available in these sources about your research area in business studies, almost regardless of the nature of the research area. Therefore, application of appropriate set of criteria to select secondary data to be used in the study plays an important role in terms of increasing the levels of research validity and reliability.

These criteria include, but not limited to date of publication, credential of the author, reliability of the source, quality of discussions, depth of analyses, the extent of contribution of the text to the development of the research area etc. Secondary data collection is discussed in greater depth in Literature Review chapter.

Secondary data collection methods offer a range of advantages such as saving time, effort and expenses. However they have a major disadvantage. Specifically, secondary research does not make contribution to the expansion of the literature by producing fresh (new) data.

Primary Data Collection Methods

Primary data is the type of data that has not been around before. Primary data is unique findings of your research. Primary data collection and analysis typically requires more time and effort to conduct compared to the secondary data research. Primary data collection methods can be divided into two groups: quantitative and qualitative.

Quantitative data collection methods are based on mathematical calculations in various formats. Methods of quantitative data collection and analysis include questionnaires with closed-ended questions, methods of correlation and regression, mean, mode and median and others.

Quantitative methods are cheaper to apply and they can be applied within shorter duration of time compared to qualitative methods. Moreover, due to a high level of standardisation of quantitative methods, it is easy to make comparisons of findings.

Qualitative research methods , on the contrary, do not involve numbers or mathematical calculations. Qualitative research is closely associated with words, sounds, feeling, emotions, colours and other elements that are non-quantifiable.

Qualitative studies aim to ensure greater level of depth of understanding and qualitative data collection methods include interviews, questionnaires with open-ended questions, focus groups, observation, game or role-playing, case studies etc.

Your choice between quantitative or qualitative methods of data collection depends on the area of your research and the nature of research aims and objectives.

My e-book, The Ultimate Guide to Writing a Dissertation in Business Studies: a step by step assistance offers practical assistance to complete a dissertation with minimum or no stress. The e-book covers all stages of writing a dissertation starting from the selection to the research area to submitting the completed version of the work within the deadline.

John Dudovskiy

What is Data Collection? Methods, Types, Tools, Examples

Appinio Research · 09.11.2023 · 33min read

Are you ready to unlock the power of data? In today's data-driven world, understanding the art and science of data collection is the key to informed decision-making and achieving your objectives.

This guide will walk you through the intricate data collection process, from its fundamental principles to advanced strategies and ethical considerations. Whether you're a business professional, researcher, or simply curious about the world of data, this guide will equip you with the knowledge and tools needed to harness the potential of data collection effectively.

What is Data Collection?

Data collection is the systematic process of gathering and recording information or data from various sources for analysis, interpretation, and decision-making. It is a fundamental step in research, business operations, and virtually every field where information is used to understand, improve, or make informed choices.

Key Elements of Data Collection

• Sources: Data can be collected from a wide range of sources, including surveys , interviews, observations, sensors, databases, social media, and more.
• Methods: Various methods are employed to collect data, such as questionnaires, data entry, web scraping, and sensor networks. The choice of method depends on the type of data, research objectives, and available resources.
• Data Types: Data can be qualitative (descriptive) or quantitative (numerical), structured (organized into a predefined format) or unstructured (free-form text or media), and primary (collected directly) or secondary (obtained from existing sources).
• Data Collection Tools: Technology plays a significant role in modern data collection, with software applications, mobile apps, sensors, and data collection platforms facilitating efficient and accurate data capture.
• Ethical Considerations: Ethical guidelines, including informed consent and privacy protection, must be followed to ensure that data collection respects the rights and well-being of individuals.
• Data Quality: The accuracy, completeness, and reliability of collected data are critical to its usefulness. Data quality assurance measures are implemented to minimize errors and biases.
• Data Storage: Collected data needs to be securely stored and managed to prevent loss, unauthorized access, and breaches. Data storage solutions range from on-premises servers to cloud-based platforms.

Importance of Data Collection in Modern Businesses

Data collection is of paramount importance in modern businesses for several compelling reasons:

• Informed Decision-Making: Collected data serves as the foundation for informed decision-making at all levels of an organization. It provides valuable insights into customer behavior, market trends, operational efficiency, and more.
• Competitive Advantage: Businesses that effectively collect and analyze data gain a competitive edge. Data-driven insights help identify opportunities, optimize processes, and stay ahead of competitors .
• Customer Understanding: Data collection allows businesses to better understand their customers, their preferences, and their pain points. This insight is invaluable for tailoring products, services, and marketing strategies.
• Performance Measurement: Data collection enables organizations to assess the performance of various aspects of their operations, from marketing campaigns to production processes. This helps identify areas for improvement.
• Risk Management: Businesses can use data to identify potential risks and develop strategies to mitigate them. This includes financial risks, supply chain disruptions, and cybersecurity threats.
• Innovation: Data collection supports innovation by providing insights into emerging trends and customer demands. Businesses can use this information to develop new products or services.
• Resource Allocation: Data-driven decision-making helps allocate resources efficiently. For example, marketing budgets can be optimized based on the performance of different channels.

Goals and Objectives of Data Collection

The goals and objectives of data collection depend on the specific context and the needs of the organization or research project. However, there are some common overarching objectives:

• Information Gathering: The primary goal is to gather accurate, relevant, and reliable information that addresses specific questions or objectives.
• Analysis and Insight: Collected data is meant to be analyzed to uncover patterns, trends, relationships, and insights that can inform decision-making and strategy development.
• Measurement and Evaluation: Data collection allows for the measurement and evaluation of various factors, such as performance, customer satisfaction , or market potential.
• Problem Solving: Data collection can be directed toward solving specific problems or challenges faced by an organization, such as identifying the root causes of quality issues.
• Monitoring and Surveillance: In some cases, data collection serves as a continuous monitoring or surveillance function, allowing organizations to track ongoing processes or conditions.
• Benchmarking: Data collection can be used for benchmarking against industry standards or competitors, helping organizations assess their performance relative to others.
• Planning and Strategy: Data collected over time can support long-term planning and strategy development, ensuring that organizations adapt to changing circumstances.

In summary, data collection is a foundational activity with diverse applications across industries and sectors. Its objectives range from understanding customers and making informed decisions to improving processes, managing risks, and driving innovation. The quality and relevance of collected data are pivotal in achieving these goals.

How to Plan Your Data Collection Strategy?

Before kicking things off, we'll review the crucial steps of planning your data collection strategy. Your success in data collection largely depends on how well you define your objectives, select suitable sources, set clear goals, and choose appropriate collection methods.

Defining Your Research Questions

Defining your research questions is the foundation of any effective data collection effort. The more precise and relevant your questions, the more valuable the data you collect.

• Specificity is Key: Make sure your research questions are specific and focused. Instead of asking, "How can we improve customer satisfaction?" ask, "What specific aspects of our service do customers find most satisfying or dissatisfying?"
• Prioritize Questions: Determine the most critical questions that will have the most significant impact on your goals. Not all questions are equally important, so allocate your resources accordingly.
• Alignment with Objectives: Ensure that your research questions directly align with your overall objectives. If your goal is to increase sales, your research questions should be geared toward understanding customer buying behaviors and preferences.

Identifying Key Data Sources

Identifying the proper data sources is essential for gathering accurate and relevant information. Here are some examples of key data sources for different industries and purposes.

• Customer Data: This can include customer demographics, purchase history, website behavior, and feedback from customer service interactions.
• Market Research Reports: Utilize industry reports, competitor analyses, and market trend studies to gather external data and insights.
• Internal Records: Your organization's databases, financial records, and operational data can provide valuable insights into your business's performance.
• Social Media Platforms: Monitor social media channels to gather customer feedback, track brand mentions , and identify emerging trends in your industry.
• Web Analytics: Collect data on website traffic, user behavior, and conversion rates to optimize your online presence.

Setting Clear Data Collection Goals

Setting clear and measurable goals is essential to ensure your data collection efforts remain on track and deliver valuable results. Goals should be:

• Specific: Clearly define what you aim to achieve with your data collection. For instance, increasing website traffic by 20% in six months is a specific goal.
• Measurable: Establish criteria to measure your progress and success. Use metrics such as revenue growth, customer satisfaction scores, or conversion rates.
• Achievable: Set realistic goals that your team can realistically work towards. Overly ambitious goals can lead to frustration and burnout.
• Relevant : Ensure your goals align with your organization's broader objectives and strategic initiatives.
• Time-Bound: Set a timeframe within which you plan to achieve your goals. This adds a sense of urgency and helps you track progress effectively.

Choosing Data Collection Methods

Selecting the correct data collection methods is crucial for obtaining accurate and reliable data. Your choice should align with your research questions and goals. Here's a closer look at various data collection methods and their practical applications.

Types of Data Collection Methods

Now, let's explore different data collection methods in greater detail, including examples of when and how to use them effectively:

Surveys and Questionnaires

Surveys and questionnaires are versatile tools for gathering data from a large number of respondents. They are commonly used for:

• Customer Feedback: Collecting opinions and feedback on products, services, and overall satisfaction.
• Market Research: Assessing market preferences, identifying trends, and evaluating consumer behavior .
• Employee Surveys : Measuring employee engagement, job satisfaction, and feedback on workplace conditions.

Example: If you're running an e-commerce business and want to understand customer preferences, you can create an online survey asking customers about their favorite product categories, preferred payment methods, and shopping frequency.

To enhance your data collection endeavors, check out Appinio , a modern research platform that simplifies the process and maximizes the quality of insights. Appinio offers user-friendly survey and questionnaire tools that enable you to effortlessly design surveys tailored to your needs. It also provides seamless integration with interview and observation data, allowing you to consolidate your findings in one place.

Discover how Appinio can elevate your data collection efforts. Book a demo today to unlock a world of possibilities in gathering valuable insights!

Book a Demo

Interviews involve one-on-one or group conversations with participants to gather detailed insights. They are particularly useful for:

• Qualitative Research: Exploring complex topics, motivations, and personal experiences.
• In-Depth Analysis: Gaining a deep understanding of specific issues or situations.
• Expert Opinions: Interviewing industry experts or thought leaders to gather valuable insights.

Example: If you're a healthcare provider aiming to improve patient experiences, conducting interviews with patients can help you uncover specific pain points and suggestions for improvement.

Observations

Observations entail watching and recording behaviors or events in their natural context. This method is ideal for:

• Behavioral Studies: Analyzing how people interact with products or environments.
• Field Research: Collecting data in real-world settings, such as retail stores, public spaces, or classrooms.
• Ethnographic Research: Immersing yourself in a specific culture or community to understand their practices and customs.

Example: If you manage a retail store, observing customer traffic flow and purchasing behaviors can help optimize store layout and product placement.

Document Analysis

Document analysis involves reviewing and extracting information from written or digital documents. It is valuable for:

• Historical Research: Studying historical records, manuscripts, and archives.
• Content Analysis: Analyzing textual or visual content from websites, reports, or publications.
• Legal and Compliance: Reviewing contracts, policies, and legal documents for compliance purposes.

Example: If you're a content marketer, you can analyze competitor blog posts to identify common topics and keywords used in your industry.

Web Scraping

Web scraping is the automated process of extracting data from websites. It's suitable for:

• Competitor Analysis: Gathering data on competitor product prices, descriptions, and customer reviews.
• Market Research: Collecting data on product listings, reviews, and trends from e-commerce websites.
• News and Social Media Monitoring: Tracking news articles, social media posts, and comments related to your brand or industry.

Example: If you're in the travel industry, web scraping can help you collect pricing data for flights and accommodations from various travel booking websites to stay competitive.

Social Media Monitoring

Social media monitoring involves tracking and analyzing conversations and activities on social media platforms. It's valuable for:

• Brand Reputation Management: Monitoring brand mentions and sentiment to address customer concerns or capitalize on positive feedback.
• Competitor Analysis: Keeping tabs on competitors' social media strategies and customer engagement.
• Trend Identification: Identifying emerging trends and viral content within your industry.

Example: If you run a restaurant, social media monitoring can help you track customer reviews, comments, and hashtags related to your establishment, allowing you to respond promptly to customer feedback and trends.

By understanding the nuances and applications of these data collection methods, you can choose the most appropriate approach to gather valuable insights for your specific objectives. Remember that a well-thought-out data collection strategy is the cornerstone of informed decision-making and business success.

How to Design Your Data Collection Instruments?

Now that you've defined your research questions, identified data sources, set clear goals, and chosen appropriate data collection methods, it's time to design the instruments you'll use to collect data effectively.

Design Effective Survey Questions

Designing survey questions is a crucial step in gathering accurate and meaningful data. Here are some key considerations:

• Clarity: Ensure that your questions are clear and concise. Avoid jargon or ambiguous language that may confuse respondents.
• Relevance: Ask questions that directly relate to your research objectives. Avoid unnecessary or irrelevant questions that can lead to survey fatigue.
• Avoid Leading Questions: Formulate questions that do not guide respondents toward a particular answer. Maintain neutrality to get unbiased responses.
• Response Options: Provide appropriate response options, including multiple-choice, Likert scales, or open-ended formats, depending on the type of data you need.
• Pilot Testing: Before deploying your survey, conduct pilot tests with a small group to identify any issues with question wording or response options.

Craft Interview Questions for Insightful Conversations

Developing interview questions requires thoughtful consideration to elicit valuable insights from participants:

• Open-Ended Questions: Use open-ended questions to encourage participants to share their thoughts, experiences, and perspectives without being constrained by predefined answers.
• Probing Questions: Prepare follow-up questions to delve deeper into specific topics or clarify responses.
• Structured vs. Semi-Structured Interviews: Decide whether your interviews will follow a structured format with predefined questions or a semi-structured approach that allows flexibility.
• Avoid Biased Questions: Ensure your questions do not steer participants toward desired responses. Maintain objectivity throughout the interview.

Build an Observation Checklist for Data Collection

When conducting observations, having a well-structured checklist is essential:

• Clearly Defined Variables: Identify the specific variables or behaviors you are observing and ensure they are well-defined.
• Checklist Format: Create a checklist format that is easy to use and follow during observations. This may include checkboxes, scales, or space for notes.
• Training Observers: If you have a team of observers, provide thorough training to ensure consistency and accuracy in data collection.
• Pilot Observations: Before starting formal data collection, conduct pilot observations to refine your checklist and ensure it captures the necessary information.

Streamline Data Collection with Forms and Templates

Creating user-friendly data collection forms and templates helps streamline the process:

• Consistency: Ensure that all data collection forms follow a consistent format and structure, making it easier to compare and analyze data.
• Data Validation: Incorporate data validation checks to reduce errors during data entry. This can include dropdown menus, date pickers, or required fields.
• Digital vs. Paper Forms: Decide whether digital forms or traditional paper forms are more suitable for your data collection needs. Digital forms often offer real-time data validation and remote access.
• Accessibility: Make sure your forms and templates are accessible to all team members involved in data collection. Provide training if necessary.

The Data Collection Process

Now that your data collection instruments are ready, it's time to embark on the data collection process itself. This section covers the practical steps involved in collecting high-quality data.

1. Preparing for Data Collection

Adequate preparation is essential to ensure a smooth data collection process:

• Resource Allocation: Allocate the necessary resources, including personnel, technology, and materials, to support data collection activities.
• Training: Train data collection teams or individuals on the use of data collection instruments and adherence to protocols.
• Pilot Testing: Conduct pilot data collection runs to identify and resolve any issues or challenges that may arise.
• Ethical Considerations: Ensure that data collection adheres to ethical standards and legal requirements. Obtain necessary permissions or consent as applicable.

2. Conducting Data Collection

During data collection, it's crucial to maintain consistency and accuracy:

• Follow Protocols: Ensure that data collection teams adhere to established protocols and procedures to maintain data integrity.
• Supervision: Supervise data collection teams to address questions, provide guidance, and resolve any issues that may arise.
• Documentation: Maintain detailed records of the data collection process, including dates, locations, and any deviations from the plan.
• Data Security: Implement data security measures to protect collected information from unauthorized access or breaches.

3. Ensuring Data Quality and Reliability

After collecting data, it's essential to validate and ensure its quality:

• Data Cleaning: Review collected data for errors, inconsistencies, and missing values. Clean and preprocess the data to ensure accuracy.
• Quality Checks: Perform quality checks to identify outliers or anomalies that may require further investigation or correction.
• Data Validation: Cross-check data with source documents or original records to verify its accuracy and reliability.
• Data Auditing: Conduct periodic audits to assess the overall quality of the collected data and make necessary adjustments.

4. Managing Data Collection Teams

If you have multiple team members involved in data collection, effective management is crucial:

• Communication: Maintain open and transparent communication channels with team members to address questions, provide guidance, and ensure consistency.
• Performance Monitoring: Regularly monitor the performance of data collection teams, identifying areas for improvement or additional training.
• Problem Resolution: Be prepared to promptly address any challenges or issues that arise during data collection.
• Feedback Loop: Establish a feedback loop for data collection teams to share insights and best practices, promoting continuous improvement.

By following these steps and best practices in the data collection process, you can ensure that the data you collect is reliable, accurate, and aligned with your research objectives. This lays the foundation for meaningful analysis and informed decision-making.

How to Store and Manage Data?

It's time to explore the critical aspects of data storage and management, which are pivotal in ensuring the security, accessibility, and usability of your collected data.

Choosing Data Storage Solutions

Selecting the proper data storage solutions is a strategic decision that impacts data accessibility, scalability, and security. Consider the following factors:

• Cloud vs. On-Premises: Decide whether to store your data in the cloud or on-premises. Cloud solutions offer scalability, accessibility, and automatic backups, while on-premises solutions provide more control but require significant infrastructure investments.
• Data Types: Assess the types of data you're collecting, such as structured, semi-structured, or unstructured data. Choose storage solutions that accommodate your data formats efficiently.
• Scalability: Ensure that your chosen solution can scale as your data volume grows. This is crucial for preventing storage bottlenecks.
• Data Accessibility: Opt for storage solutions that provide easy and secure access to authorized users, whether they are on-site or remote.
• Data Recovery and Backup: Implement robust data backup and recovery mechanisms to safeguard against data loss due to hardware failures or disasters.

Data Security and Privacy

Data security and privacy are paramount, especially when handling sensitive or personal information.

• Encryption: Implement encryption for data at rest and in transit. Use encryption protocols like SSL/TLS for communication and robust encryption algorithms for storage.
• Access Control: Set up role-based access control (RBAC) to restrict access to data based on job roles and responsibilities. Limit access to only those who need it.
• Compliance: Ensure that your data storage and management practices comply with relevant data protection regulations, such as GDPR, HIPAA, or CCPA.
• Data Masking: Use data masking techniques to conceal sensitive information in non-production environments.
• Monitoring and Auditing: Continuously monitor access logs and perform regular audits to detect unauthorized activities and maintain compliance.

Data Organization and Cataloging

Organizing and cataloging your data is essential for efficient retrieval, analysis, and decision-making.

• Metadata Management: Maintain detailed metadata for each dataset, including data source, date of collection, data owner, and description. This makes it easier to locate and understand your data.
• Taxonomies and Categories: Develop taxonomies or data categorization schemes to classify data into logical groups, making it easier to find and manage.
• Data Versioning: Implement data versioning to track changes and updates over time. This ensures data lineage and transparency.
• Data Catalogs: Use data cataloging tools and platforms to create a searchable inventory of your data assets, facilitating discovery and reuse.
• Data Retention Policies: Establish clear data retention policies that specify how long data should be retained and when it should be securely deleted or archived.

How to Analyze and Interpret Data?

Once you've collected your data, let's take a look at the process of extracting valuable insights from your collected data through analysis and interpretation.

Data Cleaning and Preprocessing

Data cleaning and preprocessing are essential steps to ensure that your data is accurate and ready for analysis.

• Handling Missing Data: Develop strategies for dealing with missing data, such as imputation or removal, based on the nature of your data and research objectives.
• Outlier Detection: Identify and address outliers that can skew analysis results. Consider whether outliers should be corrected, removed, or retained based on their significance.
• Normalization and Scaling: Normalize or scale data to bring it within a common range, making it suitable for certain algorithms and models.
• Data Transformation: Apply data transformations, such as logarithmic scaling or categorical encoding, to prepare data for specific types of analysis.
• Data Imbalance: Address class imbalance issues in datasets, particularly machine learning applications, to avoid biased model training.

Exploratory Data Analysis (EDA)

EDA is the process of visually and statistically exploring your data to uncover patterns, trends, and potential insights.

• Descriptive Statistics: Calculate basic statistics like mean, median, and standard deviation to summarize data distributions.
• Data Visualization: Create visualizations such as histograms, scatter plots, and heatmaps to reveal relationships and patterns within the data.
• Correlation Analysis: Examine correlations between variables to understand how they influence each other.
• Hypothesis Testing: Conduct hypothesis tests to assess the significance of observed differences or relationships in your data.

Statistical Analysis Techniques

Choose appropriate statistical analysis techniques based on your research questions and data types.

• Descriptive Statistics: Use descriptive statistics to summarize and describe your data, providing an initial overview of key features.
• Inferential Statistics: Apply inferential statistics, including t-tests, ANOVA, or regression analysis, to test hypotheses and draw conclusions about population parameters.
• Non-parametric Tests: Employ non-parametric tests when assumptions of normality are not met or when dealing with ordinal or nominal data .
• Time Series Analysis: Analyze time-series data to uncover trends, seasonality, and temporal patterns.

Data Visualization

Data visualization is a powerful tool for conveying complex information in a digestible format.

• Charts and Graphs: Utilize various charts and graphs, such as bar charts, line charts, pie charts, and heatmaps, to represent data visually.
• Interactive Dashboards: Create interactive dashboards using tools like Tableau, Power BI, or custom web applications to allow stakeholders to explore data dynamically.
• Storytelling: Use data visualization to tell a compelling data-driven story, highlighting key findings and insights.
• Accessibility: Ensure that data visualizations are accessible to all audiences, including those with disabilities, by following accessibility guidelines.

Drawing Conclusions and Insights

Finally, drawing conclusions and insights from your data analysis is the ultimate goal.

• Contextual Interpretation: Interpret your findings in the context of your research objectives and the broader business or research landscape.
• Actionable Insights: Identify actionable insights that can inform decision-making, strategy development, or future research directions.
• Report Generation: Create comprehensive reports or presentations that communicate your findings clearly and concisely to stakeholders.
• Validation: Cross-check your conclusions with domain experts or subject matter specialists to ensure accuracy and relevance.

By following these steps in data analysis and interpretation, you can transform raw data into valuable insights that drive informed decisions, optimize processes, and create new opportunities for your organization.

How to Report and Present Data?

Now, let's explore the crucial steps of reporting and presenting data effectively, ensuring that your findings are communicated clearly and meaningfully to stakeholders.

1. Create Data Reports

Data reports are the culmination of your data analysis efforts, presenting your findings in a structured and comprehensible manner.

• Report Structure: Organize your report with a clear structure, including an introduction, methodology, results, discussion, and conclusions.
• Visualization Integration: Incorporate data visualizations, charts, and graphs to illustrate key points and trends.
• Clarity and Conciseness: Use clear and concise language, avoiding technical jargon, to make your report accessible to a diverse audience.
• Actionable Insights: Highlight actionable insights and recommendations that stakeholders can use to make informed decisions.
• Appendices: Include appendices with detailed methodology, data sources, and any additional information that supports your findings.

2. Leverage Data Visualization Tools

Data visualization tools can significantly enhance your ability to convey complex information effectively. Top data visualization tools include:

• Tableau: Tableau offers a wide range of visualization options and interactive dashboards, making it a popular choice for data professionals.
• Power BI: Microsoft's Power BI provides powerful data visualization and business intelligence capabilities, suitable for creating dynamic reports and dashboards.
• Python Libraries: Utilize Python libraries such as Matplotlib, Seaborn, and Plotly for custom data visualizations and analysis.
• Excel: Microsoft Excel remains a versatile tool for creating basic charts and graphs, particularly for smaller datasets.
• Custom Development: Consider custom development for specialized visualization needs or when existing tools don't meet your requirements.

3. Communicate Findings to Stakeholders

Effectively communicating your findings to stakeholders is essential for driving action and decision-making.

• Audience Understanding : Tailor your communication to the specific needs and background knowledge of your audience. Avoid technical jargon when speaking to non-technical stakeholders.
• Visual Storytelling: Craft a narrative that guides stakeholders through the data, highlighting key insights and their implications.
• Engagement: Use engaging and interactive presentations or reports to maintain the audience's interest and encourage participation.
• Question Handling: Be prepared to answer questions and provide clarifications during presentations or discussions. Anticipate potential concerns or objections.
• Feedback Loop: Encourage feedback and open dialogue with stakeholders to ensure your findings align with their objectives and expectations.

Data Collection Examples

To better understand the practical application of data collection in various domains, let's explore some real-world examples, including those in the business context. These examples illustrate how data collection can drive informed decision-making and lead to meaningful insights.

Business Customer Feedback Surveys

Scenario: A retail company wants to enhance its customer experience and improve product offerings. To achieve this, they initiate customer feedback surveys.

Data Collection Approach:

• Survey Creation: The company designs a survey with specific questions about customer preferences , shopping experiences , and product satisfaction.
• Distribution: Surveys are distributed through various channels, including email, in-store kiosks, and the company's website.
• Data Gathering: Responses from thousands of customers are collected and stored in a centralized database.

Data Analysis and Insights:

• Customer Sentiment Analysis: Using natural language processing (NLP) techniques, the company analyzes open-ended responses to gauge customer sentiment.
• Product Performance: Analyzing survey data, the company identifies which products receive the highest and lowest ratings, leading to decisions on which products to improve or discontinue.
• Store Layout Optimization: By examining feedback related to in-store experiences, the company can adjust store layouts and signage to enhance customer flow and convenience.

Healthcare Patient Record Digitization

Scenario: A healthcare facility aims to transition from paper-based patient records to digital records for improved efficiency and patient care.

• Scanning and Data Entry: Existing paper records are scanned, and data entry personnel convert them into digital format.
• Electronic Health Record (EHR) Implementation: The facility adopts an EHR system to store and manage patient data securely.
• Continuous Data Entry: As new patient information is collected, it is directly entered into the EHR system.
• Patient History Access: Physicians and nurses gain instant access to patient records, improving diagnostic accuracy and treatment.
• Data Analytics: Aggregated patient data can be analyzed to identify trends in diseases, treatment outcomes, and healthcare resource utilization.
• Resource Optimization: Analysis of patient data allows the facility to allocate resources more efficiently, such as staff scheduling based on patient admission patterns.

Social Media Engagement Monitoring

Scenario: A digital marketing agency manages social media campaigns for various clients and wants to track campaign performance and audience engagement.

• Social Media Monitoring Tools: The agency employs social media monitoring tools to collect data on post engagement, reach, likes, shares, and comments.
• Custom Tracking Links: Unique tracking links are created for each campaign to monitor traffic and conversions.
• Audience Demographics: Data on the demographics of engaged users is gathered from platform analytics.
• Campaign Effectiveness: The agency assesses which campaigns are most effective in terms of engagement and conversion rates.
• Audience Segmentation: Insights into audience demographics help tailor future campaigns to specific target demographics.
• Content Strategy: Analyzing which types of content (e.g., videos, infographics) generate the most engagement informs content strategy decisions.

These examples showcase how data collection serves as the foundation for informed decision-making and strategy development across diverse sectors. Whether improving customer experiences, enhancing healthcare services, or optimizing marketing efforts, data collection empowers organizations to harness valuable insights for growth and improvement.

Ethical Considerations in Data Collection

Ethical considerations are paramount in data collection to ensure privacy, fairness, and transparency. Addressing these issues is not only responsible but also crucial for building trust with stakeholders.

Informed Consent

Obtaining informed consent from participants is an ethical imperative. Transparency is critical, and participants should fully understand the purpose of data collection, how their data will be used, and any potential risks or benefits involved. Consent should be voluntary, and participants should have the option to withdraw their consent at any time without consequences.

Consent forms should be clear and comprehensible, avoiding overly complex language or legal jargon. Special care should be taken when collecting sensitive or personal data to ensure privacy rights are respected.

Privacy Protection

Protecting individuals' privacy is essential to maintain trust and comply with data protection regulations. Data anonymization or pseudonymization should be used to prevent the identification of individuals, especially when sharing or publishing data. Data encryption methods should be implemented to protect data both in transit and at rest, safeguarding it from unauthorized access.

Strict access controls should be in place to restrict data access to authorized personnel only, and clear data retention policies should be established and adhered to, preventing unnecessary data storage. Regular privacy audits should be conducted to identify and address potential vulnerabilities or compliance issues.

Bias and Fairness in Data Collection

Addressing bias and ensuring fairness in data collection is critical to avoid perpetuating inequalities. Data collection methods should be designed to minimize potential biases , such as selection bias or response bias. Efforts should be made to achieve diverse and representative samples , ensuring that data accurately reflects the population of interest. Fair treatment of all participants and data sources is essential, with discrimination based on characteristics such as race, gender, or socioeconomic status strictly avoided.

If algorithms are used in data collection or analysis, biases that may arise from automated processes should be assessed and mitigated. Ethical reviews or expert consultations may be considered when dealing with sensitive or potentially biased data. By adhering to ethical principles throughout the data collection process, individuals' rights are protected, and a foundation for responsible and trustworthy data-driven decision-making is established.

Data collection is the cornerstone of informed decision-making and insight generation in today's data-driven world. Whether you're a business seeking to understand your customers better, a researcher uncovering valuable trends, or anyone eager to harness the power of data, this guide has equipped you with the essential knowledge and tools. Remember, ethical considerations are paramount, and the quality of data matters.

Furthermore, as you embark on your data collection journey, always keep in mind the impact and potential of the information you gather. Each data point is a piece of the puzzle that can help you shape strategies, optimize operations, and make a positive difference. Data collection is not just a task; it's a powerful tool that empowers you to unlock opportunities, solve challenges, and stay ahead in a dynamic and ever-changing landscape. So, continue to explore, analyze, and draw valuable insights from your data, and let it be your compass on the path to success.

How to Collect Data in Minutes?

Imagine having the power to conduct your own market research in minutes, without the need for a PhD in research. Appinio is the real-time market research platform that empowers you to get instant consumer insights, fueling your data-driven decisions. We've transformed market research from boring and intimidating to exciting and intuitive.

Here's why Appinio is your go-to platform:

• Lightning-Fast Insights: From questions to insights in minutes. When you need answers, Appinio delivers swiftly.
• User-Friendly: Our platform is so intuitive that anyone can use it; no research degree required.
• Global Reach: Define your target group from over 1200 characteristics and survey them in 90+ countries.
• Guided Expertise: Our dedicated research consultants will support you every step of the way, ensuring your research journey is seamless and effective.

Join the loop 💌

Be the first to hear about new updates, product news, and data insights. We'll send it all straight to your inbox.

Get the latest market research news straight to your inbox! 💌

Wait, there's more

15.04.2024 | 32min read

What is Market Share? Definition, Formula, Examples

11.04.2024 | 34min read

What is Data Analysis? Definition, Tools, Examples

09.04.2024 | 29min read

What is a Confidence Interval and How to Calculate It?

Data Collection, Analysis, and Interpretation

• First Online: 03 January 2022

Cite this chapter

• Mark F. McEntee 5  

460 Accesses

Often it has been said that proper prior preparation prevents performance. Many of the mistakes made in research have their origins back at the point of data collection. Perhaps it is natural human instinct not to plan; we learn from our experiences. However, it is crucial when it comes to the endeavours of science that we do plan our data collection with analysis and interpretation in mind. In this section on data collection, we will review some fundamental concepts of experimental design, sample size estimation, the assumptions that underlie most statistical processes, and ethical principles.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Al-Murshedi, S., Hogg, P., & England, A. (2018). An investigation into the validity of utilising the CDRAD 2.0 phantom for optimisation studies in digital radiography. The British Journal of Radiology . British Institute of Radiology , 91 (1089), 4. https://doi.org/10.1259/bjr.20180317

Article   Google Scholar  

Alhailiy, A. B., et al. (2019). The associated factors for radiation dose variation in cardiac CT angiography. The British Journal of Radiology . British Institute of Radiology , 92 (1096), 20180793. https://doi.org/10.1259/bjr.20180793

Article   PubMed   PubMed Central   Google Scholar  

Armato, S. G., et al. (2011). The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A completed reference database of lung nodules on CT scans. Medical Physics . John Wiley and Sons Ltd , 38 (2), 915–931. https://doi.org/10.1118/1.3528204

Avison, D. E., et al. (1999). Action research. Communications of the ACM . Association for Computing Machinery (ACM) , 42 (1), 94–97. https://doi.org/10.1145/291469.291479

Båth, M., & Månsson, L. G. (2007). Visual grading characteristics (VGC) analysis: A non-parametric rank-invariant statistical method for image quality evaluation. British Journal of Radiology, 80 (951), 169–176. https://doi.org/10.1259/bjr/35012658

Chakraborty, D. P. (2017). Observer performance methods for diagnostic imaging . CRC Press. https://doi.org/10.1201/9781351228190

Book   Google Scholar  

Couper, M. P., Traugott, M. W., & Lamias, M. J. (2001). Web survey design and administration. Public Opinion Quarterly . Oxford Academic , 65 (2), 230–253. https://doi.org/10.1086/322199

Article   CAS   PubMed   Google Scholar  

European Commission European Guidelines on Quality Criteria for Diagnostic Radiographic Images EUR 16260 EN. (1995).

Google Scholar  

Fähling, M., et al. (2017). Understanding and preventing contrast-induced acute kidney injury. Nature Reviews Nephrology . Nature Publishing Group, 169–180. https://doi.org/10.1038/nrneph.2016.196

Faucon, A. L., Bobrie, G., & Clément, O. (2019). Nephrotoxicity of iodinated contrast media: From pathophysiology to prevention strategies. European Journal of Radiology . Elsevier Ireland Ltd, 231–241. https://doi.org/10.1016/j.ejrad.2019.03.008

Fisher, M. J., & Marshall, A. P. (2009). Understanding descriptive statistics. Australian Critical Care . Elsevier , 22 (2), 93–97. https://doi.org/10.1016/j.aucc.2008.11.003

Article   PubMed   Google Scholar  

Fryback, D. G., & Thornbury, J. R. (1991). The efficacy of diagnostic imaging. Medical Decision Making . Sage PublicationsSage CA: Thousand Oaks, CA , 11 (2), 88–94. https://doi.org/10.1177/0272989X9101100203

Ganesan, A., et al. (2018). A review of factors influencing radiologists’ visual search behaviour. Journal of Medical Imaging and Radiation Oncology . Blackwell Publishing , 62 (6), 747–757. https://doi.org/10.1111/1754-9485.12798

Gilligan, L. A., et al. (2020). Risk of acute kidney injury following contrast-enhanced CT in hospitalized pediatric patients: A propensity score analysis. Radiology . Radiological Society of North America Inc. , 294 (3), 548–556. https://doi.org/10.1148/radiol.2020191931

Good, P. I., & Hardin, J. W. (2012). Common errors in statistics (and how to avoid them): Fourth edition . Wiley. https://doi.org/10.1002/9781118360125

Gusterson, H. (2008). Ethnographic research. In Qualitative methods in international relations (pp. 93–113). Palgrave Macmillan UK. https://doi.org/10.1057/9780230584129_7

Chapter   Google Scholar  

Hansson, J., Månsson, L. G., & Båth, M. (2016). The validity of using ROC software for analysing visual grading characteristics data: An investigation based on the novel software VGC analyzer. Radiation Protection Dosimetry . Oxford University Press , 169 (1–4), 54–59. https://doi.org/10.1093/rpd/ncw035

Home - LUNA16 - Grand Challenge. (n.d.). Available at: https://luna16.grand-challenge.org/ . Accessed 25 Mar 2021.

Huda, W., et al. (1997). Comparison of a photostimulable phosphor system with film for dental radiology. Oral Surgery, Oral Medicine, Oral Pathology, Oral Radiology, and Endodontics . Mosby Inc. , 83 (6), 725–731. https://doi.org/10.1016/S1079-2104(97)90327-9

Iarossi, G. (2006). The power of survey design: A user’s guide for managing surveys, interpreting results, and influencing respondents . Available at: https://books.google.com/books?hl=en&lr=&id=E-8XHVsqoeUC&oi=fnd&pg=PR5&dq=survey+design&ots=fADK9Aznuk&sig=G5DiPgYM18VcoZ-PF05kT7G0OGI . Accessed 21 Mar 2021.

Jang, J. S., et al. (2018). Image quality assessment with dose reduction using high kVp and additional filtration for abdominal digital radiography. Physica Medica . Associazione Italiana di Fisica Medica , 50 , 46–51. https://doi.org/10.1016/j.ejmp.2018.05.007

Jessen, K. A. (2004). Balancing image quality and dose in diagnostic radiology. European Radiology, Supplement . Springer , 14 (1), 9–18. https://doi.org/10.1007/s10406-004-0003-7

King, N., Horrocks, C., & Brooks, J. (2018). Interviews in qualitative research . Available at: https://books.google.com/books?hl=en&lr=&id=ZdB2DwAAQBAJ&oi=fnd&pg=PP1&dq=interviews+in+research&ots=hwRx2cwH3W&sig=_gt8y-4GlHSCnTQAhLfynA3C17E . Accessed: 21 Mar 2021.

Krul, A. J., Daanen, H. A. M., & Choi, H. (2011). Self-reported and measured weight, height and body mass index (BMI) in Italy, the Netherlands and North America. The European Journal of Public Health . Oxford Academic , 21 (4), 414–419. https://doi.org/10.1093/eurpub/ckp228

Kundel, H. L. (1979). Images, image quality and observer performance. New horizons in radiology lecture. Radiology, 132 (2), 265–271. https://doi.org/10.1148/132.2.265

Makary, M. A., & Daniel, M. (2016). Medical error-the third leading cause of death in the US. BMJ (Online) . BMJ Publishing Group , 353 . https://doi.org/10.1136/bmj.i2139

Martin, C. J., Sharp, P. F., & Sutton, D. G. (1999). Measurement of image quality in diagnostic radiology. Applied Radiation and Isotopes . Elsevier Sci Ltd , 50 (1), 21–38. https://doi.org/10.1016/S0969-8043(98)00022-0

Mathematical methods of statistics / by Harald Cramer | National Library of Australia (n.d.). Available at: https://catalogue.nla.gov.au/Record/81100 . Accessed: 22 Mar 2021.

McCollough, C. H., & Schueler, B. A. (2000). Calculation of effective dose. Medical Physics . John Wiley and Sons Ltd , 27 (5), 828–837. https://doi.org/10.1118/1.598948

Meissner, H., et al. (n.d.). Best Practices for Mixed Methods Research in the Health Sciences_the_nature_and_design_of_mixed_methods_research .

Morgan, D. L. (1996). Focus groups. Annual Review of Sociology . Annual Reviews Inc. , 22 , 129–152. https://doi.org/10.1146/annurev.soc.22.1.129

Moses, L. E., Shapiro, D., & Littenberg, B. (1993). Combining independent studies of a diagnostic test into a summary roc curve: Data-analytic approaches and some additional considerations. Statistics in Medicine . John Wiley & Sons, Ltd , 12 (14), 1293–1316. https://doi.org/10.1002/sim.4780121403

Neill Howell 2008 Inferential Statistical Decision Tree – StuDocu . (n.d.). Available at: https://www.studocu.com/en-gb/document/university-of-hertfordshire/using-data-to-address-research-questions/summaries/neill-howell-2008-inferential-statistical-decision-tree/1193346/view . Accessed: 23 Mar 2021.

Neuendorf, K. A., & Kumar, A. (2016). Content analysis. In The international encyclopedia of political communication (pp. 1–10). Wiley. https://doi.org/10.1002/9781118541555.wbiepc065

Nguyen, P. K., et al. (2015). Assessment of the radiation effects of cardiac CT angiography using protein and genetic biomarkers. JACC: Cardiovascular Imaging . Elsevier Inc. , 8 (8), 873–884. https://doi.org/10.1016/j.jcmg.2015.04.016

Noordzij, M., et al. (2010). Sample size calculations: Basic principles and common pitfalls. Nephrology Dialysis Transplantation . Oxford University Press, , 25 (5), 1388–1393. https://doi.org/10.1093/ndt/gfp732

Pisano, E. D., et al. (2005). Diagnostic performance of digital versus film mammography for breast-cancer screening. New England Journal of Medicine . Massachusetts Medical Society , 353 (17), 1773–1783. https://doi.org/10.1056/NEJMoa052911

Article   CAS   Google Scholar  

ROC curve analysis with MedCalc. (n.d.). Available at: https://www.medcalc.org/manual/roc-curves.php . Accessed 30 Mar 2021.

Rudolfer, S. M. (2003). ZHOU, X.-H., OBUCHOWSKI, N. A. and MCCLISH, D. K. statistical methods in diagnostic medicine. Wiley, New York, 2002. xv + 437 pp. $94.95/£70.50. ISBN 0-471-34772-8. Biometrics . Wiley-Blackwell , 59 (1), 203–204. https://doi.org/10.1111/1541-0420.00266

Sudheesh, K., Duggappa, D. R., & Nethra, S. S. (2016). How to write a research proposal? Indian Journal of Anaesthesia . https://doi.org/10.4103/0019-5049.190617

Download references

Author information

Authors and affiliations.

University College Cork, Cork, Ireland

Mark F. McEntee

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Mark F. McEntee .

Editor information

Editors and affiliations.

Medical Imaging, Faculty of Health, University of Canberra, Burnaby, BC, Canada

Euclid Seeram

Faculty of Health, University of Canberra, Canberra, ACT, Australia

Robert Davidson

Brookfield Health Sciences, University College Cork, Cork, Ireland

Andrew England

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

McEntee, M.F. (2021). Data Collection, Analysis, and Interpretation. In: Seeram, E., Davidson, R., England, A., McEntee, M.F. (eds) Research for Medical Imaging and Radiation Sciences. Springer, Cham. https://doi.org/10.1007/978-3-030-79956-4_6

Download citation

DOI : https://doi.org/10.1007/978-3-030-79956-4_6

Published : 03 January 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-79955-7

Online ISBN : 978-3-030-79956-4

eBook Packages : Biomedical and Life Sciences Biomedical and Life Sciences (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• For Small Business

Research Data Collection in 2024: The Comprehensive Guide

• March 17, 2024
• by Terry Tolentino

High-quality research relies on rigorous data collection practices. As technology expands possibilities for tapping insights, a strategic approach is key to gather meaningful information. This comprehensive 3000+ word guide explores expert techniques for collecting robust data to drive breakthrough research.

As an experienced data extraction specialist, I‘ve seen firsthand how methodical data collection practices lead to impactful discoveries and innovations. By approaching projects with care and leveraging the right methods, researchers can unlock deep understandings to benefit humanity.

In this in-depth guide, we’ll cover:

• Defining research data collection
• Qualitative vs. quantitative data
• Primary vs. secondary data sources
• Top 7 data collection methods
• Tips to optimize your research plan
• The future of research data

Let’s get started.

What is Research Data Collection?

Research data collection involves systematically gathering, cleaning, and preparing data to address specific questions or hypotheses within a research project. It provides the raw materials for analysis.

For example, a medical research team may collect data on the efficacy of a new drug by administering surveys, conducting interviews, and reviewing lab results from a sample of clinical trial participants. Or a marketing researcher might gather customer feedback through focus groups, social media monitoring, and online surveys to understand perceptions of a product.

Research data collection is a foundational step in:

• Scientific research
• Medical and health research
• Social science research
• Market research
• Policy research
• Many other contexts

The data collected fuels the research, analysis, and insights that can lead to new discoveries, products, policies, and innovations that benefit society. That‘s why it‘s so important to get it right!

Qualitative vs. Quantitative Data

There are two major categories of research data:

Qualitative data consists of non-numerical information like text, images, video, or audio. It aims to gather in-depth insights into behaviors, attitudes, and motivations. Some examples of qualitative data are interview transcripts, social media posts, photographs, and videos.

Quantitative data involves numerical information that can be counted, measured, and expressed using statistics. It seeks to quantify behaviors, opinions, and other variables. Examples of quantitative data include numbers, metrics, scores, statistics, and rankings.

The type of data needed depends on the research questions and aims of the project. Many studies utilize both qualitative and quantitative data to get a complete picture.

*Qualitative vs. quantitative data examples. Image credit: Aimultiple*

Primary vs. Secondary Data

In addition to qualitative/quantitative, research data can be categorized as:

Primary data : Information collected first-hand specifically for the research project at hand. It directly answers the problem the study seeks to address. Primary data collection methods include surveys, interviews, focus groups, observations, and experiments.

Secondary data: Information that already exists and was originally gathered for another purpose. Secondary data can be sourced from places like government datasets, prior academic research, news archives, websites, social media, and commercial data providers.

Here‘s a comparison of primary vs. secondary data:

*Primary vs. Secondary Data. Image credit: Aimultiple*

While primary research allows tailoring data to the exact study, secondary analysis saves significant time and resources. Many projects incorporate both.

Now let‘s explore popular methods for collecting primary qualitative and quantitative data.

Top 7 Data Collection Methods for Research

Surveys are one of the most common ways to gather data from a sample of people by asking them questions and recording their responses. They can be conducted online, offline, face-to-face, over the phone, or via mobile app.

Well-designed surveys yield important insights for:

• Market research e.g. brand awareness, pricing studies, product concept tests
• Academic research e.g. public opinion polling, sociological studies
• Government and NGO research e.g. census data, program evaluations
• Medical research e.g. patient reported outcomes, experience with treatments

With online survey tools, it‘s possible to survey thousands of people worldwide quickly and cost-effectively. For accurate results, careful survey design and sampling is crucial.

*Global survey response rates. Image credit: SurveyAnyplace*

As this data shows, online surveys tend to have lower response rates compared to other methods. Strategies like keeping surveys concise, sending email reminders, and offering incentives can help increase completion rates.

2. Interviews

In-depth interviews involve an interactive, one-on-one conversation between the researcher and a participant. The interviewer asks open-ended questions and follows up with probes to gather details.

Interviews are ideal for:

• Exploring individuals‘ perspectives, meanings, priorities and decision-making processes
• Gathering insights on personal topics that people may not discuss in groups
• Understanding influences like emotions, body language, and tone on their experiences

Sample sizes tend to be smaller than surveys or focus groups. But interviews yield rich qualitative data and insights that quantitative methods cannot provide. They work best when participants are thoughtfully recruited and skilled interviewers ask good questions.

3. Focus Groups

In focus groups, a moderator leads a discussion with 6-12 participants who share similar characteristics or experiences relevant to the research. Focus groups can uncover:

• Beliefs, attitudes, perceptions, and feelings about a topic
• Reactions to ideas, messages, products, services, brands
• Consensus views or diverse perspectives
• Unexpected insights through group dynamics

The flexible format allows exploring unplanned topics as they arise naturally. Focus groups require an experienced moderator to facilitate productive conversations within a limited timeframe.

4. Observation

Observing behavior and events as they unfold is a defining method of qualitative research. Detailed observation notes provide descriptive data for analysis.

Key strengths of observation include:

• Gathers data on behaviors and interactions in real-world contexts
• Avoids reliance on self-reported data
• Can detect nonverbal patterns and unconscious behaviors
• Flexible to adapt to evolving situations in the field

Challenges include intensive time requirements, potential observer bias, and difficulties categorizing unstructured observations. Advanced ethnographic techniques like photo elicitation can enhance findings.

5. Digital Data Collection

Online technologies open new possibilities for large-scale data collection. Web analytics, social media monitoring, mobile devices, and crowdsourcing platforms allow accessing data digitally.

• Web analytics track user engagement with websites or apps to understand behavior.
• Social media monitoring analyzes public social conversations to identify trends, sentiments, and demographics.
• Mobile data collection apps enable gathering geo-located, multimedia, and sensor-based data from smartphones and wearables.
• Crowdsourcing platforms like Amazon Mechanical Turk provide on-demand access to global respondents for surveys, tasks, and testing.

Digital data collection enables real-time gathering of rich information that was previously difficult to access. AI and automation can help process high volumes of digital data.

6. Public Records & Archival Data

Historical records, documents, artifacts, and archives are rich sources of secondary data for research. Examples include:

• Government records – census data, health records, crime statistics
• News archives – newspapers, magazines, radio, video
• Websites and digital records
• Physical artifacts – buildings, tools, art
• Personal documents – letters, photos, diaries

Digitalization has expanded access to archives for analysis with qualitative data analysis software. Historical records yield insights on change over time.

7. Commercial & Syndicated Data

Companies sell various research data to clients including:

• Market research – satisfaction studies, product tests
• Panel data – opt-in consumer panels for surveys
• Data aggregates – bundled demographic, purchasing, social media data
• Financial data – credit reports, real estate data

Brokers like Nielsen syndicate media consumption and consumer behavior data from samples of volunteers who share their experiences through diaries, meters, surveys, or passive monitoring.

Purchased third-party data can supplement a research project, but quality varies so thorough vetting is required.

Expert Tips to Optimize Your Data Collection

Through my extensive experience gathering all forms of data for research, I‘ve developed best practices to help ensure high-quality results:

• Match methods to research questions: Select techniques that will yield relevant data. Avoid unfocused data collection.
• Validate data sources: Check accuracy, reliability, credibility, and fit.
• Clean data thoroughly: Fix errors, inconsistencies, duplication, outliers.
• Anonymize personal data: Remove identifiers like names when possible to protect privacy.
• Store data securely: Encrypt data and restrict access to prevent breaches.
• Plan analysis approach: Structure data for easy parsing and mining for insights.

I also recommend leveraging automation and AI to streamline processes:

• Text analysis software quickly processes surveys, interviews, documents
• Web scrapers extract online data from APIs or site content
• Image recognition identifies, catalogs, and extracts visual data
• Smart sensors monitor behavior passively with minimal respondent burden

But always validate computer-generated data manually to catch errors.

By implementing best practices in the data collection stage, researchers gain quality results that lead to impactful discoveries.

The Future of Research Data Collection

Advanced technologies continue opening new possibilities for research:

• Mobile & wearables allow ubiquitous 24/7 global data gathering via devices people already own and use.
• Internet of Things sensors embedded in infrastructure like appliances, vehicles, and buildings passively generate behavioral data.
• Artificial intelligence assists in data processing, analysis, and insight generation at scale.
• Augmented & virtual reality creates immersive experiences for behavioral observation research.
• Blockchain increases transparency and trust in research through verifiable, tamper-proof distributed ledgers.

But risks like privacy violations, flawed algorithms, and unethical manipulation must also be addressed through thoughtful oversight.

With sound ethics and methodology, research data collection yields exciting opportunities to propel human understanding, innovation, and discovery. We‘ve only begun tapping the potential of data for good.

The future looks bright for turning information into knowledge that enriches lives worldwide. I look forward to being part of this journey by empowering researchers with robust data collection strategies, tools, and insights.

Table of Contents

What is data collection, why do we need data collection, what are the different data collection methods, data collection tools, the importance of ensuring accurate and appropriate data collection, issues related to maintaining the integrity of data collection, what are common challenges in data collection, what are the key steps in the data collection process, data collection considerations and best practices, choose the right data science program, are you interested in a career in data science, what is data collection: methods, types, tools.

The process of gathering and analyzing accurate data from various sources to find answers to research problems, trends and probabilities, etc., to evaluate possible outcomes is Known as Data Collection. Knowledge is power, information is knowledge, and data is information in digitized form, at least as defined in IT. Hence, data is power. But before you can leverage that data into a successful strategy for your organization or business, you need to gather it. That’s your first step.

So, to help you get the process started, we shine a spotlight on data collection. What exactly is it? Believe it or not, it’s more than just doing a Google search! Furthermore, what are the different types of data collection? And what kinds of data collection tools and data collection techniques exist?

If you want to get up to speed about what is data collection process, you’ve come to the right place. 

Transform raw data into captivating visuals with Simplilearn's hands-on Data Visualization Courses and captivate your audience. Also, master the art of data management with Simplilearn's comprehensive data management courses  - unlock new career opportunities today!

Data collection is the process of collecting and evaluating information or data from multiple sources to find answers to research problems, answer questions, evaluate outcomes, and forecast trends and probabilities. It is an essential phase in all types of research, analysis, and decision-making, including that done in the social sciences, business, and healthcare.

Accurate data collection is necessary to make informed business decisions, ensure quality assurance, and keep research integrity.

During data collection, the researchers must identify the data types, the sources of data, and what methods are being used. We will soon see that there are many different data collection methods . There is heavy reliance on data collection in research, commercial, and government fields.

Before an analyst begins collecting data, they must answer three questions first:

• What’s the goal or purpose of this research?
• What kinds of data are they planning on gathering?
• What methods and procedures will be used to collect, store, and process the information?

Additionally, we can break up data into qualitative and quantitative types. Qualitative data covers descriptions such as color, size, quality, and appearance. Quantitative data, unsurprisingly, deals with numbers, such as statistics, poll numbers, percentages, etc.

Before a judge makes a ruling in a court case or a general creates a plan of attack, they must have as many relevant facts as possible. The best courses of action come from informed decisions, and information and data are synonymous.

The concept of data collection isn’t a new one, as we’ll see later, but the world has changed. There is far more data available today, and it exists in forms that were unheard of a century ago. The data collection process has had to change and grow with the times, keeping pace with technology.

Whether you’re in the world of academia, trying to conduct research, or part of the commercial sector, thinking of how to promote a new product, you need data collection to help you make better choices.

Now that you know what is data collection and why we need it, let's take a look at the different methods of data collection. While the phrase “data collection” may sound all high-tech and digital, it doesn’t necessarily entail things like computers, big data , and the internet. Data collection could mean a telephone survey, a mail-in comment card, or even some guy with a clipboard asking passersby some questions. But let’s see if we can sort the different data collection methods into a semblance of organized categories.

Primary and secondary methods of data collection are two approaches used to gather information for research or analysis purposes. Let's explore each data collection method in detail:

1. Primary Data Collection:

Primary data collection involves the collection of original data directly from the source or through direct interaction with the respondents. This method allows researchers to obtain firsthand information specifically tailored to their research objectives. There are various techniques for primary data collection, including:

a. Surveys and Questionnaires: Researchers design structured questionnaires or surveys to collect data from individuals or groups. These can be conducted through face-to-face interviews, telephone calls, mail, or online platforms.

b. Interviews: Interviews involve direct interaction between the researcher and the respondent. They can be conducted in person, over the phone, or through video conferencing. Interviews can be structured (with predefined questions), semi-structured (allowing flexibility), or unstructured (more conversational).

c. Observations: Researchers observe and record behaviors, actions, or events in their natural setting. This method is useful for gathering data on human behavior, interactions, or phenomena without direct intervention.

d. Experiments: Experimental studies involve the manipulation of variables to observe their impact on the outcome. Researchers control the conditions and collect data to draw conclusions about cause-and-effect relationships.

e. Focus Groups: Focus groups bring together a small group of individuals who discuss specific topics in a moderated setting. This method helps in understanding opinions, perceptions, and experiences shared by the participants.

2. Secondary Data Collection:

Secondary data collection involves using existing data collected by someone else for a purpose different from the original intent. Researchers analyze and interpret this data to extract relevant information. Secondary data can be obtained from various sources, including:

a. Published Sources: Researchers refer to books, academic journals, magazines, newspapers, government reports, and other published materials that contain relevant data.

b. Online Databases: Numerous online databases provide access to a wide range of secondary data, such as research articles, statistical information, economic data, and social surveys.

c. Government and Institutional Records: Government agencies, research institutions, and organizations often maintain databases or records that can be used for research purposes.

d. Publicly Available Data: Data shared by individuals, organizations, or communities on public platforms, websites, or social media can be accessed and utilized for research.

e. Past Research Studies: Previous research studies and their findings can serve as valuable secondary data sources. Researchers can review and analyze the data to gain insights or build upon existing knowledge.

Now that we’ve explained the various techniques, let’s narrow our focus even further by looking at some specific tools. For example, we mentioned interviews as a technique, but we can further break that down into different interview types (or “tools”).

Word Association

The researcher gives the respondent a set of words and asks them what comes to mind when they hear each word.

Sentence Completion

Researchers use sentence completion to understand what kind of ideas the respondent has. This tool involves giving an incomplete sentence and seeing how the interviewee finishes it.

Role-Playing

Respondents are presented with an imaginary situation and asked how they would act or react if it was real.

In-Person Surveys

The researcher asks questions in person.

Online/Web Surveys

These surveys are easy to accomplish, but some users may be unwilling to answer truthfully, if at all.

Mobile Surveys

These surveys take advantage of the increasing proliferation of mobile technology. Mobile collection surveys rely on mobile devices like tablets or smartphones to conduct surveys via SMS or mobile apps.

Phone Surveys

No researcher can call thousands of people at once, so they need a third party to handle the chore. However, many people have call screening and won’t answer.

Observation

Sometimes, the simplest method is the best. Researchers who make direct observations collect data quickly and easily, with little intrusion or third-party bias. Naturally, it’s only effective in small-scale situations.

Accurate data collecting is crucial to preserving the integrity of research, regardless of the subject of study or preferred method for defining data (quantitative, qualitative). Errors are less likely to occur when the right data gathering tools are used (whether they are brand-new ones, updated versions of them, or already available).

Among the effects of data collection done incorrectly, include the following -

• Erroneous conclusions that squander resources
• Decisions that compromise public policy
• Incapacity to correctly respond to research inquiries
• Bringing harm to participants who are humans or animals
• Deceiving other researchers into pursuing futile research avenues
• The study's inability to be replicated and validated

When these study findings are used to support recommendations for public policy, there is the potential to result in disproportionate harm, even if the degree of influence from flawed data collecting may vary by discipline and the type of investigation.

Let us now look at the various issues that we might face while maintaining the integrity of data collection.

In order to assist the errors detection process in the data gathering process, whether they were done purposefully (deliberate falsifications) or not, maintaining data integrity is the main justification (systematic or random errors).

Quality assurance and quality control are two strategies that help protect data integrity and guarantee the scientific validity of study results.

Each strategy is used at various stages of the research timeline:

• Quality control - tasks that are performed both after and during data collecting
• Quality assurance - events that happen before data gathering starts

Let us explore each of them in more detail now.

Quality Assurance

As data collecting comes before quality assurance, its primary goal is "prevention" (i.e., forestalling problems with data collection). The best way to protect the accuracy of data collection is through prevention. The uniformity of protocol created in the thorough and exhaustive procedures manual for data collecting serves as the best example of this proactive step. 

The likelihood of failing to spot issues and mistakes early in the research attempt increases when guides are written poorly. There are several ways to show these shortcomings:

• Failure to determine the precise subjects and methods for retraining or training staff employees in data collecting
• List of goods to be collected, in part
• There isn't a system in place to track modifications to processes that may occur as the investigation continues.
• Instead of detailed, step-by-step instructions on how to deliver tests, there is a vague description of the data gathering tools that will be employed.
• Uncertainty regarding the date, procedure, and identity of the person or people in charge of examining the data
• Incomprehensible guidelines for using, adjusting, and calibrating the data collection equipment.

Now, let us look at how to ensure Quality Control.

The Ultimate Ticket to Top Data Science Job Roles

Quality Control

Despite the fact that quality control actions (detection/monitoring and intervention) take place both after and during data collection, the specifics should be meticulously detailed in the procedures manual. Establishing monitoring systems requires a specific communication structure, which is a prerequisite. Following the discovery of data collection problems, there should be no ambiguity regarding the information flow between the primary investigators and staff personnel. A poorly designed communication system promotes slack oversight and reduces opportunities for error detection.

Direct staff observation conference calls, during site visits, or frequent or routine assessments of data reports to spot discrepancies, excessive numbers, or invalid codes can all be used as forms of detection or monitoring. Site visits might not be appropriate for all disciplines. Still, without routine auditing of records, whether qualitative or quantitative, it will be challenging for investigators to confirm that data gathering is taking place in accordance with the manual's defined methods. Additionally, quality control determines the appropriate solutions, or "actions," to fix flawed data gathering procedures and reduce recurrences.

Problems with data collection, for instance, that call for immediate action include:

• Fraud or misbehavior
• Systematic mistakes, procedure violations 
• Individual data items with errors
• Issues with certain staff members or a site's performance 

Researchers are trained to include one or more secondary measures that can be used to verify the quality of information being obtained from the human subject in the social and behavioral sciences where primary data collection entails using human subjects. 

For instance, a researcher conducting a survey would be interested in learning more about the prevalence of risky behaviors among young adults as well as the social factors that influence these risky behaviors' propensity for and frequency. Let us now explore the common challenges with regard to data collection.

There are some prevalent challenges faced while collecting data, let us explore a few of them to understand them better and avoid them.

Data Quality Issues

The main threat to the broad and successful application of machine learning is poor data quality. Data quality must be your top priority if you want to make technologies like machine learning work for you. Let's talk about some of the most prevalent data quality problems in this blog article and how to fix them.

Inconsistent Data

When working with various data sources, it's conceivable that the same information will have discrepancies between sources. The differences could be in formats, units, or occasionally spellings. The introduction of inconsistent data might also occur during firm mergers or relocations. Inconsistencies in data have a tendency to accumulate and reduce the value of data if they are not continually resolved. Organizations that have heavily focused on data consistency do so because they only want reliable data to support their analytics.

Data Downtime

Data is the driving force behind the decisions and operations of data-driven businesses. However, there may be brief periods when their data is unreliable or not prepared. Customer complaints and subpar analytical outcomes are only two ways that this data unavailability can have a significant impact on businesses. A data engineer spends about 80% of their time updating, maintaining, and guaranteeing the integrity of the data pipeline. In order to ask the next business question, there is a high marginal cost due to the lengthy operational lead time from data capture to insight.

Schema modifications and migration problems are just two examples of the causes of data downtime. Data pipelines can be difficult due to their size and complexity. Data downtime must be continuously monitored, and it must be reduced through automation.

Ambiguous Data

Even with thorough oversight, some errors can still occur in massive databases or data lakes. For data streaming at a fast speed, the issue becomes more overwhelming. Spelling mistakes can go unnoticed, formatting difficulties can occur, and column heads might be deceptive. This unclear data might cause a number of problems for reporting and analytics.

Become a Data Scientist With Real-World Experience

Duplicate Data

Streaming data, local databases, and cloud data lakes are just a few of the sources of data that modern enterprises must contend with. They might also have application and system silos. These sources are likely to duplicate and overlap each other quite a bit. For instance, duplicate contact information has a substantial impact on customer experience. If certain prospects are ignored while others are engaged repeatedly, marketing campaigns suffer. The likelihood of biased analytical outcomes increases when duplicate data are present. It can also result in ML models with biased training data.

Too Much Data

While we emphasize data-driven analytics and its advantages, a data quality problem with excessive data exists. There is a risk of getting lost in an abundance of data when searching for information pertinent to your analytical efforts. Data scientists, data analysts, and business users devote 80% of their work to finding and organizing the appropriate data. With an increase in data volume, other problems with data quality become more serious, particularly when dealing with streaming data and big files or databases.

Inaccurate Data

For highly regulated businesses like healthcare, data accuracy is crucial. Given the current experience, it is more important than ever to increase the data quality for COVID-19 and later pandemics. Inaccurate information does not provide you with a true picture of the situation and cannot be used to plan the best course of action. Personalized customer experiences and marketing strategies underperform if your customer data is inaccurate.

Data inaccuracies can be attributed to a number of things, including data degradation, human mistake, and data drift. Worldwide data decay occurs at a rate of about 3% per month, which is quite concerning. Data integrity can be compromised while being transferred between different systems, and data quality might deteriorate with time.

Hidden Data

The majority of businesses only utilize a portion of their data, with the remainder sometimes being lost in data silos or discarded in data graveyards. For instance, the customer service team might not receive client data from sales, missing an opportunity to build more precise and comprehensive customer profiles. Missing out on possibilities to develop novel products, enhance services, and streamline procedures is caused by hidden data.

Finding Relevant Data

Finding relevant data is not so easy. There are several factors that we need to consider while trying to find relevant data, which include -

• Relevant Domain
• Relevant demographics
• Relevant Time period and so many more factors that we need to consider while trying to find relevant data.

Data that is not relevant to our study in any of the factors render it obsolete and we cannot effectively proceed with its analysis. This could lead to incomplete research or analysis, re-collecting data again and again, or shutting down the study.

Deciding the Data to Collect

Determining what data to collect is one of the most important factors while collecting data and should be one of the first factors while collecting data. We must choose the subjects the data will cover, the sources we will be used to gather it, and the quantity of information we will require. Our responses to these queries will depend on our aims, or what we expect to achieve utilizing your data. As an illustration, we may choose to gather information on the categories of articles that website visitors between the ages of 20 and 50 most frequently access. We can also decide to compile data on the typical age of all the clients who made a purchase from your business over the previous month.

Not addressing this could lead to double work and collection of irrelevant data or ruining your study as a whole.

Dealing With Big Data

Big data refers to exceedingly massive data sets with more intricate and diversified structures. These traits typically result in increased challenges while storing, analyzing, and using additional methods of extracting results. Big data refers especially to data sets that are quite enormous or intricate that conventional data processing tools are insufficient. The overwhelming amount of data, both unstructured and structured, that a business faces on a daily basis. 

The amount of data produced by healthcare applications, the internet, social networking sites social, sensor networks, and many other businesses are rapidly growing as a result of recent technological advancements. Big data refers to the vast volume of data created from numerous sources in a variety of formats at extremely fast rates. Dealing with this kind of data is one of the many challenges of Data Collection and is a crucial step toward collecting effective data. 

Low Response and Other Research Issues

Poor design and low response rates were shown to be two issues with data collecting, particularly in health surveys that used questionnaires. This might lead to an insufficient or inadequate supply of data for the study. Creating an incentivized data collection program might be beneficial in this case to get more responses.

Now, let us look at the key steps in the data collection process.

In the Data Collection Process, there are 5 key steps. They are explained briefly below -

1. Decide What Data You Want to Gather

The first thing that we need to do is decide what information we want to gather. We must choose the subjects the data will cover, the sources we will use to gather it, and the quantity of information that we would require. For instance, we may choose to gather information on the categories of products that an average e-commerce website visitor between the ages of 30 and 45 most frequently searches for. 

2. Establish a Deadline for Data Collection

The process of creating a strategy for data collection can now begin. We should set a deadline for our data collection at the outset of our planning phase. Some forms of data we might want to continuously collect. We might want to build up a technique for tracking transactional data and website visitor statistics over the long term, for instance. However, we will track the data throughout a certain time frame if we are tracking it for a particular campaign. In these situations, we will have a schedule for when we will begin and finish gathering data. 

3. Select a Data Collection Approach

We will select the data collection technique that will serve as the foundation of our data gathering plan at this stage. We must take into account the type of information that we wish to gather, the time period during which we will receive it, and the other factors we decide on to choose the best gathering strategy.

4. Gather Information

Once our plan is complete, we can put our data collection plan into action and begin gathering data. In our DMP, we can store and arrange our data. We need to be careful to follow our plan and keep an eye on how it's doing. Especially if we are collecting data regularly, setting up a timetable for when we will be checking in on how our data gathering is going may be helpful. As circumstances alter and we learn new details, we might need to amend our plan.

5. Examine the Information and Apply Your Findings

It's time to examine our data and arrange our findings after we have gathered all of our information. The analysis stage is essential because it transforms unprocessed data into insightful knowledge that can be applied to better our marketing plans, goods, and business judgments. The analytics tools included in our DMP can be used to assist with this phase. We can put the discoveries to use to enhance our business once we have discovered the patterns and insights in our data.

Let us now look at some data collection considerations and best practices that one might follow.

We must carefully plan before spending time and money traveling to the field to gather data. While saving time and resources, effective data collection strategies can help us collect richer, more accurate, and richer data.

Below, we will be discussing some of the best practices that we can follow for the best results -

1. Take Into Account the Price of Each Extra Data Point

Once we have decided on the data we want to gather, we need to make sure to take the expense of doing so into account. Our surveyors and respondents will incur additional costs for each additional data point or survey question.

2. Plan How to Gather Each Data Piece

There is a dearth of freely accessible data. Sometimes the data is there, but we may not have access to it. For instance, unless we have a compelling cause, we cannot openly view another person's medical information. It could be challenging to measure several types of information.

Consider how time-consuming and difficult it will be to gather each piece of information while deciding what data to acquire.

3. Think About Your Choices for Data Collecting Using Mobile Devices

Mobile-based data collecting can be divided into three categories -

• IVRS (interactive voice response technology) -  Will call the respondents and ask them questions that have already been recorded. 
• SMS data collection - Will send a text message to the respondent, who can then respond to questions by text on their phone. 
• Field surveyors - Can directly enter data into an interactive questionnaire while speaking to each respondent, thanks to smartphone apps.

We need to make sure to select the appropriate tool for our survey and responders because each one has its own disadvantages and advantages.

4. Carefully Consider the Data You Need to Gather

It's all too easy to get information about anything and everything, but it's crucial to only gather the information that we require. 

It is helpful to consider these 3 questions:

• What details will be helpful?
• What details are available?
• What specific details do you require?

5. Remember to Consider Identifiers

Identifiers, or details describing the context and source of a survey response, are just as crucial as the information about the subject or program that we are actually researching.

In general, adding more identifiers will enable us to pinpoint our program's successes and failures with greater accuracy, but moderation is the key.

6. Data Collecting Through Mobile Devices is the Way to Go

Although collecting data on paper is still common, modern technology relies heavily on mobile devices. They enable us to gather many various types of data at relatively lower prices and are accurate as well as quick. There aren't many reasons not to pick mobile-based data collecting with the boom of low-cost Android devices that are available nowadays.

Start your Dream Career with the Best Resources!

1. What is data collection with example?

Data collection is the process of collecting and analyzing information on relevant variables in a predetermined, methodical way so that one can respond to specific research questions, test hypotheses, and assess results. Data collection can be either qualitative or quantitative. Example: A company collects customer feedback through online surveys and social media monitoring to improve their products and services.

2. What are the primary data collection methods?

As is well known, gathering primary data is costly and time intensive. The main techniques for gathering data are observation, interviews, questionnaires, schedules, and surveys.

3. What are data collection tools?

The term "data collecting tools" refers to the tools/devices used to gather data, such as a paper questionnaire or a system for computer-assisted interviews. Tools used to gather data include case studies, checklists, interviews, occasionally observation, surveys, and questionnaires.

4. What’s the difference between quantitative and qualitative methods?

While qualitative research focuses on words and meanings, quantitative research deals with figures and statistics. You can systematically measure variables and test hypotheses using quantitative methods. You can delve deeper into ideas and experiences using qualitative methodologies.

5. What are quantitative data collection methods?

While there are numerous other ways to get quantitative information, the methods indicated above—probability sampling, interviews, questionnaire observation, and document review—are the most typical and frequently employed, whether collecting information offline or online.

6. What is mixed methods research?

User research that includes both qualitative and quantitative techniques is known as mixed methods research. For deeper user insights, mixed methods research combines insightful user data with useful statistics.

7. What are the benefits of collecting data?

Collecting data offers several benefits, including:

• Knowledge and Insight
• Evidence-Based Decision Making
• Problem Identification and Solution
• Validation and Evaluation
• Identifying Trends and Predictions
• Support for Research and Development
• Policy Development
• Quality Improvement
• Personalization and Targeting
• Knowledge Sharing and Collaboration

8. What’s the difference between reliability and validity?

Reliability is about consistency and stability, while validity is about accuracy and appropriateness. Reliability focuses on the consistency of results, while validity focuses on whether the results are actually measuring what they are intended to measure. Both reliability and validity are crucial considerations in research to ensure the trustworthiness and meaningfulness of the collected data and measurements.

Are you thinking about pursuing a career in the field of data science? Simplilearn's Data Science courses are designed to provide you with the necessary skills and expertise to excel in this rapidly changing field. Here's a detailed comparison for your reference:

Program Name Data Scientist Master's Program Post Graduate Program In Data Science Post Graduate Program In Data Science Geo All Geos All Geos Not Applicable in US University Simplilearn Purdue Caltech Course Duration 11 Months 11 Months 11 Months Coding Experience Required Basic Basic No Skills You Will Learn 10+ skills including data structure, data manipulation, NumPy, Scikit-Learn, Tableau and more 8+ skills including Exploratory Data Analysis, Descriptive Statistics, Inferential Statistics, and more 8+ skills including Supervised & Unsupervised Learning Deep Learning Data Visualization, and more Additional Benefits Applied Learning via Capstone and 25+ Data Science Projects Purdue Alumni Association Membership Free IIMJobs Pro-Membership of 6 months Resume Building Assistance Upto 14 CEU Credits Caltech CTME Circle Membership Cost $$ $$$$ $$$$ Explore Program Explore Program Explore Program

We live in the Data Age, and if you want a career that fully takes advantage of this, you should consider a career in data science. Simplilearn offers a Caltech Post Graduate Program in Data Science  that will train you in everything you need to know to secure the perfect position. This Data Science PG program is ideal for all working professionals, covering job-critical topics like R, Python programming , machine learning algorithms , NLP concepts , and data visualization with Tableau in great detail. This is all provided via our interactive learning model with live sessions by global practitioners, practical labs, and industry projects.

Data Science & Business Analytics Courses Duration and Fees

Data Science & Business Analytics programs typically range from a few weeks to several months, with fees varying based on program and institution.

Recommended Reads

Data Science Career Guide: A Comprehensive Playbook To Becoming A Data Scientist

Difference Between Collection and Collections in Java

An Ultimate One-Stop Solution Guide to Collections in C# Programming With Examples

Managing Data

Capped Collection in MongoDB

What Are Java Collections and How to Implement Them?

Get Affiliated Certifications with Live Class programs

Data scientist.

• Add the IBM Advantage to your Learning
• 25 Industry-relevant Projects and Integrated labs

Caltech Data Sciences-Bootcamp

• Exclusive visit to Caltech’s Robotics Lab

Caltech Post Graduate Program in Data Science

• Earn a program completion certificate from Caltech CTME
• Curriculum delivered in live online sessions by industry experts
• PMP, PMI, PMBOK, CAPM, PgMP, PfMP, ACP, PBA, RMP, SP, and OPM3 are registered marks of the Project Management Institute, Inc.

• 7 Data Collection Methods & Tools For Research

• Data Collection

The underlying need for Data collection is to capture quality evidence that seeks to answer all the questions that have been posed. Through data collection businesses or management can deduce quality information that is a prerequisite for making informed decisions.

To improve the quality of information, it is expedient that data is collected so that you can draw inferences and make informed decisions on what is considered factual.

At the end of this article, you would understand why picking the best data collection method is necessary for achieving your set objective. 

Sign up on Formplus Builder to create your preferred online surveys or questionnaire for data collection. You don’t need to be tech-savvy! Start creating quality questionnaires with Formplus.

What is Data Collection?

Data collection is a methodical process of gathering and analyzing specific information to proffer solutions to relevant questions and evaluate the results. It focuses on finding out all there is to a particular subject matter. Data is collected to be further subjected to hypothesis testing which seeks to explain a phenomenon.

Hypothesis testing eliminates assumptions while making a proposition from the basis of reason.

For collectors of data, there is a range of outcomes for which the data is collected. But the key purpose for which data is collected is to put a researcher in a vantage position to make predictions about future probabilities and trends.

The core forms in which data can be collected are primary and secondary data. While the former is collected by a researcher through first-hand sources, the latter is collected by an individual other than the user. 

Types of Data Collection 

Before broaching the subject of the various types of data collection. It is pertinent to note that data collection in itself falls under two broad categories; Primary data collection and secondary data collection.

Primary Data Collection

Primary data collection by definition is the gathering of raw data collected at the source. It is a process of collecting the original data collected by a researcher for a specific research purpose. It could be further analyzed into two segments; qualitative research and quantitative data collection methods. 

• Qualitative Research Method 

The qualitative research methods of data collection do not involve the collection of data that involves numbers or a need to be deduced through a mathematical calculation, rather it is based on the non-quantifiable elements like the feeling or emotion of the researcher. An example of such a method is an open-ended questionnaire.

• Quantitative Method

Quantitative methods are presented in numbers and require a mathematical calculation to deduce. An example would be the use of a questionnaire with close-ended questions to arrive at figures to be calculated Mathematically. Also, methods of correlation and regression, mean, mode and median.

Read Also: 15 Reasons to Choose Quantitative over Qualitative Research

Secondary Data Collection

Secondary data collection, on the other hand, is referred to as the gathering of second-hand data collected by an individual who is not the original user. It is the process of collecting data that is already existing, be it already published books, journals, and/or online portals. In terms of ease, it is much less expensive and easier to collect.

Your choice between Primary data collection and secondary data collection depends on the nature, scope, and area of your research as well as its aims and objectives. 

Importance of Data Collection

There are a bunch of underlying reasons for collecting data, especially for a researcher. Walking you through them, here are a few reasons; 

• Integrity of the Research

A key reason for collecting data, be it through quantitative or qualitative methods is to ensure that the integrity of the research question is indeed maintained.

• Reduce the likelihood of errors

The correct use of appropriate data collection of methods reduces the likelihood of errors consistent with the results. 

• Decision Making

To minimize the risk of errors in decision-making, it is important that accurate data is collected so that the researcher doesn’t make uninformed decisions. 

• Save Cost and Time

Data collection saves the researcher time and funds that would otherwise be misspent without a deeper understanding of the topic or subject matter.

• To support a need for a new idea, change, and/or innovation

To prove the need for a change in the norm or the introduction of new information that will be widely accepted, it is important to collect data as evidence to support these claims.

What is a Data Collection Tool?

Data collection tools refer to the devices/instruments used to collect data, such as a paper questionnaire or computer-assisted interviewing system. Case Studies, Checklists, Interviews, Observation sometimes, and Surveys or Questionnaires are all tools used to collect data.

It is important to decide on the tools for data collection because research is carried out in different ways and for different purposes. The objective behind data collection is to capture quality evidence that allows analysis to lead to the formulation of convincing and credible answers to the posed questions.

The objective behind data collection is to capture quality evidence that allows analysis to lead to the formulation of convincing and credible answers to the questions that have been posed – Click to Tweet

The Formplus online data collection tool is perfect for gathering primary data, i.e. raw data collected from the source. You can easily get data with at least three data collection methods with our online and offline data-gathering tool. I.e Online Questionnaires , Focus Groups, and Reporting. 

In our previous articles, we’ve explained why quantitative research methods are more effective than qualitative methods . However, with the Formplus data collection tool, you can gather all types of primary data for academic, opinion or product research.

Top Data Collection Methods and Tools for Academic, Opinion, or Product Research

The following are the top 7 data collection methods for Academic, Opinion-based, or product research. Also discussed in detail are the nature, pros, and cons of each one. At the end of this segment, you will be best informed about which method best suits your research. 

An interview is a face-to-face conversation between two individuals with the sole purpose of collecting relevant information to satisfy a research purpose. Interviews are of different types namely; Structured, Semi-structured , and unstructured with each having a slight variation from the other.

Use this interview consent form template to let an interviewee give you consent to use data gotten from your interviews for investigative research purposes.

• Structured Interviews – Simply put, it is a verbally administered questionnaire. In terms of depth, it is surface level and is usually completed within a short period. For speed and efficiency, it is highly recommendable, but it lacks depth.
• Semi-structured Interviews – In this method, there subsist several key questions which cover the scope of the areas to be explored. It allows a little more leeway for the researcher to explore the subject matter.
• Unstructured Interviews – It is an in-depth interview that allows the researcher to collect a wide range of information with a purpose. An advantage of this method is the freedom it gives a researcher to combine structure with flexibility even though it is more time-consuming.
• In-depth information
• Freedom of flexibility
• Accurate data.
• Time-consuming
• Expensive to collect.

What are The Best Data Collection Tools for Interviews? 

For collecting data through interviews, here are a few tools you can use to easily collect data.

• Audio Recorder

An audio recorder is used for recording sound on disc, tape, or film. Audio information can meet the needs of a wide range of people, as well as provide alternatives to print data collection tools.

• Digital Camera

An advantage of a digital camera is that it can be used for transmitting those images to a monitor screen when the need arises.

A camcorder is used for collecting data through interviews. It provides a combination of both an audio recorder and a video camera. The data provided is qualitative in nature and allows the respondents to answer questions asked exhaustively. If you need to collect sensitive information during an interview, a camcorder might not work for you as you would need to maintain your subject’s privacy.

Want to conduct an interview for qualitative data research or a special report? Use this online interview consent form template to allow the interviewee to give their consent before you use the interview data for research or report. With premium features like e-signature, upload fields, form security, etc., Formplus Builder is the perfect tool to create your preferred online consent forms without coding experience. 

• QUESTIONNAIRES

This is the process of collecting data through an instrument consisting of a series of questions and prompts to receive a response from the individuals it is administered to. Questionnaires are designed to collect data from a group. 

For clarity, it is important to note that a questionnaire isn’t a survey, rather it forms a part of it. A survey is a process of data gathering involving a variety of data collection methods, including a questionnaire.

On a questionnaire, there are three kinds of questions used. They are; fixed-alternative, scale, and open-ended. With each of the questions tailored to the nature and scope of the research.

• Can be administered in large numbers and is cost-effective.
• It can be used to compare and contrast previous research to measure change.
• Easy to visualize and analyze.
• Questionnaires offer actionable data.
• Respondent identity is protected.
• Questionnaires can cover all areas of a topic.
• Relatively inexpensive.
• Answers may be dishonest or the respondents lose interest midway.
• Questionnaires can’t produce qualitative data.
• Questions might be left unanswered.
• Respondents may have a hidden agenda.
• Not all questions can be analyzed easily.

What are the Best Data Collection Tools for Questionnaires? 

• Formplus Online Questionnaire

Formplus lets you create powerful forms to help you collect the information you need. Formplus helps you create the online forms that you like. The Formplus online questionnaire form template to get actionable trends and measurable responses. Conduct research, optimize knowledge of your brand or just get to know an audience with this form template. The form template is fast, free and fully customizable.

• Paper Questionnaire

A paper questionnaire is a data collection tool consisting of a series of questions and/or prompts for the purpose of gathering information from respondents. Mostly designed for statistical analysis of the responses, they can also be used as a form of data collection.

By definition, data reporting is the process of gathering and submitting data to be further subjected to analysis. The key aspect of data reporting is reporting accurate data because inaccurate data reporting leads to uninformed decision-making.

• Informed decision-making.
• Easily accessible.
• Self-reported answers may be exaggerated.
• The results may be affected by bias.
• Respondents may be too shy to give out all the details.
• Inaccurate reports will lead to uninformed decisions.

What are the Best Data Collection Tools for Reporting?

Reporting tools enable you to extract and present data in charts, tables, and other visualizations so users can find useful information. You could source data for reporting from Non-Governmental Organizations (NGO) reports, newspapers, website articles, and hospital records.

• NGO Reports

Contained in NGO report is an in-depth and comprehensive report on the activities carried out by the NGO, covering areas such as business and human rights. The information contained in these reports is research-specific and forms an acceptable academic base for collecting data. NGOs often focus on development projects which are organized to promote particular causes.

Newspaper data are relatively easy to collect and are sometimes the only continuously available source of event data. Even though there is a problem of bias in newspaper data, it is still a valid tool in collecting data for Reporting.

• Website Articles

Gathering and using data contained in website articles is also another tool for data collection. Collecting data from web articles is a quicker and less expensive data collection Two major disadvantages of using this data reporting method are biases inherent in the data collection process and possible security/confidentiality concerns.

• Hospital Care records

Health care involves a diverse set of public and private data collection systems, including health surveys, administrative enrollment and billing records, and medical records, used by various entities, including hospitals, CHCs, physicians, and health plans. The data provided is clear, unbiased and accurate, but must be obtained under legal means as medical data is kept with the strictest regulations.

• EXISTING DATA

This is the introduction of new investigative questions in addition to/other than the ones originally used when the data was initially gathered. It involves adding measurement to a study or research. An example would be sourcing data from an archive.

• Accuracy is very high.
• Easily accessible information.
• Problems with evaluation.
• Difficulty in understanding.

What are the Best Data Collection Tools for Existing Data?

The concept of Existing data means that data is collected from existing sources to investigate research questions other than those for which the data were originally gathered. Tools to collect existing data include: 

• Research Journals – Unlike newspapers and magazines, research journals are intended for an academic or technical audience, not general readers. A journal is a scholarly publication containing articles written by researchers, professors, and other experts.
• Surveys – A survey is a data collection tool for gathering information from a sample population, with the intention of generalizing the results to a larger population. Surveys have a variety of purposes and can be carried out in many ways depending on the objectives to be achieved.
• OBSERVATION

This is a data collection method by which information on a phenomenon is gathered through observation. The nature of the observation could be accomplished either as a complete observer, an observer as a participant, a participant as an observer, or as a complete participant. This method is a key base for formulating a hypothesis.

• Easy to administer.
• There subsists a greater accuracy with results.
• It is a universally accepted practice.
• It diffuses the situation of the unwillingness of respondents to administer a report.
• It is appropriate for certain situations.
• Some phenomena aren’t open to observation.
• It cannot be relied upon.
• Bias may arise.
• It is expensive to administer.
• Its validity cannot be predicted accurately.

What are the Best Data Collection Tools for Observation?

Observation involves the active acquisition of information from a primary source. Observation can also involve the perception and recording of data via the use of scientific instruments. The best tools for Observation are:

• Checklists – state-specific criteria, that allow users to gather information and make judgments about what they should know in relation to the outcomes. They offer systematic ways of collecting data about specific behaviors, knowledge, and skills.
• Direct observation – This is an observational study method of collecting evaluative information. The evaluator watches the subject in his or her usual environment without altering that environment.

FOCUS GROUPS

The opposite of quantitative research which involves numerical-based data, this data collection method focuses more on qualitative research. It falls under the primary category of data based on the feelings and opinions of the respondents. This research involves asking open-ended questions to a group of individuals usually ranging from 6-10 people, to provide feedback.

• Information obtained is usually very detailed.
• Cost-effective when compared to one-on-one interviews.
• It reflects speed and efficiency in the supply of results.
• Lacking depth in covering the nitty-gritty of a subject matter.
• Bias might still be evident.
• Requires interviewer training
• The researcher has very little control over the outcome.
• A few vocal voices can drown out the rest.
• Difficulty in assembling an all-inclusive group.

What are the Best Data Collection Tools for Focus Groups?

A focus group is a data collection method that is tightly facilitated and structured around a set of questions. The purpose of the meeting is to extract from the participants’ detailed responses to these questions. The best tools for tackling Focus groups are: 

• Two-Way – One group watches another group answer the questions posed by the moderator. After listening to what the other group has to offer, the group that listens is able to facilitate more discussion and could potentially draw different conclusions .
• Dueling-Moderator – There are two moderators who play the devil’s advocate. The main positive of the dueling-moderator focus group is to facilitate new ideas by introducing new ways of thinking and varying viewpoints.
• COMBINATION RESEARCH

This method of data collection encompasses the use of innovative methods to enhance participation in both individuals and groups. Also under the primary category, it is a combination of Interviews and Focus Groups while collecting qualitative data . This method is key when addressing sensitive subjects. 

• Encourage participants to give responses.
• It stimulates a deeper connection between participants.
• The relative anonymity of respondents increases participation.
• It improves the richness of the data collected.
• It costs the most out of all the top 7.
• It’s the most time-consuming.

What are the Best Data Collection Tools for Combination Research? 

The Combination Research method involves two or more data collection methods, for instance, interviews as well as questionnaires or a combination of semi-structured telephone interviews and focus groups. The best tools for combination research are: 

• Online Survey –  The two tools combined here are online interviews and the use of questionnaires. This is a questionnaire that the target audience can complete over the Internet. It is timely, effective, and efficient. Especially since the data to be collected is quantitative in nature.
• Dual-Moderator – The two tools combined here are focus groups and structured questionnaires. The structured questionnaires give a direction as to where the research is headed while two moderators take charge of the proceedings. Whilst one ensures the focus group session progresses smoothly, the other makes sure that the topics in question are all covered. Dual-moderator focus groups typically result in a more productive session and essentially lead to an optimum collection of data.

Why Formplus is the Best Data Collection Tool

• Vast Options for Form Customization 

With Formplus, you can create your unique survey form. With options to change themes, font color, font, font type, layout, width, and more, you can create an attractive survey form. The builder also gives you as many features as possible to choose from and you do not need to be a graphic designer to create a form.

• Extensive Analytics

Form Analytics, a feature in formplus helps you view the number of respondents, unique visits, total visits, abandonment rate, and average time spent before submission. This tool eliminates the need for a manual calculation of the received data and/or responses as well as the conversion rate for your poll.

• Embed Survey Form on Your Website

Copy the link to your form and embed it as an iframe which will automatically load as your website loads, or as a popup that opens once the respondent clicks on the link. Embed the link on your Twitter page to give instant access to your followers.

• Geolocation Support

The geolocation feature on Formplus lets you ascertain where individual responses are coming. It utilises Google Maps to pinpoint the longitude and latitude of the respondent, to the nearest accuracy, along with the responses.

• Multi-Select feature

This feature helps to conserve horizontal space as it allows you to put multiple options in one field. This translates to including more information on the survey form. 

Read Also: 10 Reasons to Use Formplus for Online Data Collection

How to Use Formplus to collect online data in 7 simple steps. 

• Register or sign up on Formplus builder : Start creating your preferred questionnaire or survey by signing up with either your Google, Facebook, or Email account.

Formplus gives you a free plan with basic features you can use to collect online data. Pricing plans with vast features starts at $20 monthly, with reasonable discounts for Education and Non-Profit Organizations. 

2. Input your survey title and use the form builder choice options to start creating your surveys. 

Use the choice option fields like single select, multiple select, checkbox, radio, and image choices to create your preferred multi-choice surveys online.

3. Do you want customers to rate any of your products or services delivery? 

Use the rating to allow survey respondents rate your products or services. This is an ideal quantitative research method of collecting data. 

4. Beautify your online questionnaire with Formplus Customisation features.

• Change the theme color
• Add your brand’s logo and image to the forms
• Change the form width and layout
• Edit the submission button if you want
• Change text font color and sizes
• Do you have already made custom CSS to beautify your questionnaire? If yes, just copy and paste it to the CSS option.

5. Edit your survey questionnaire settings for your specific needs

Choose where you choose to store your files and responses. Select a submission deadline, choose a timezone, limit respondents’ responses, enable Captcha to prevent spam, and collect location data of customers.

Set an introductory message to respondents before they begin the survey, toggle the “start button” post final submission message or redirect respondents to another page when they submit their questionnaires. 

Change the Email Notifications inventory and initiate an autoresponder message to all your survey questionnaire respondents. You can also transfer your forms to other users who can become form administrators.

6. Share links to your survey questionnaire page with customers.

There’s an option to copy and share the link as “Popup” or “Embed code” The data collection tool automatically creates a QR Code for Survey Questionnaire which you can download and share as appropriate. 

Congratulations if you’ve made it to this stage. You can start sharing the link to your survey questionnaire with your customers.

7. View your Responses to the Survey Questionnaire

Toggle with the presentation of your summary from the options. Whether as a single, table or cards.

8. Allow Formplus Analytics to interpret your Survey Questionnaire Data

  With online form builder analytics, a business can determine;

• The number of times the survey questionnaire was filled
• The number of customers reached
• Abandonment Rate: The rate at which customers exit the form without submitting it.
• Conversion Rate: The percentage of customers who completed the online form
• Average time spent per visit
• Location of customers/respondents.
• The type of device used by the customer to complete the survey questionnaire.

7 Tips to Create The Best Surveys For Data Collections

•  Define the goal of your survey – Once the goal of your survey is outlined, it will aid in deciding which questions are the top priority. A clear attainable goal would, for example, mirror a clear reason as to why something is happening. e.g. “The goal of this survey is to understand why Employees are leaving an establishment.”
• Use close-ended clearly defined questions – Avoid open-ended questions and ensure you’re not suggesting your preferred answer to the respondent. If possible offer a range of answers with choice options and ratings.
• Survey outlook should be attractive and Inviting – An attractive-looking survey encourages a higher number of recipients to respond to the survey. Check out Formplus Builder for colorful options to integrate into your survey design. You could use images and videos to keep participants glued to their screens.
•   Assure Respondents about the safety of their data – You want your respondents to be assured whilst disclosing details of their personal information to you. It’s your duty to inform the respondents that the data they provide is confidential and only collected for the purpose of research.
• Ensure your survey can be completed in record time – Ideally, in a typical survey, users should be able to respond in 100 seconds. It is pertinent to note that they, the respondents, are doing you a favor. Don’t stress them. Be brief and get straight to the point.
• Do a trial survey – Preview your survey before sending out your surveys to the intended respondents. Make a trial version which you’ll send to a few individuals. Based on their responses, you can draw inferences and decide whether or not your survey is ready for the big time.
• Attach a reward upon completion for users – Give your respondents something to look forward to at the end of the survey. Think of it as a penny for their troubles. It could well be the encouragement they need to not abandon the survey midway.

Try out Formplus today . You can start making your own surveys with the Formplus online survey builder. By applying these tips, you will definitely get the most out of your online surveys.

Top Survey Templates For Data Collection 

• Customer Satisfaction Survey Template 

On the template, you can collect data to measure customer satisfaction over key areas like the commodity purchase and the level of service they received. It also gives insight as to which products the customer enjoyed, how often they buy such a product, and whether or not the customer is likely to recommend the product to a friend or acquaintance. 

• Demographic Survey Template

With this template, you would be able to measure, with accuracy, the ratio of male to female, age range, and the number of unemployed persons in a particular country as well as obtain their personal details such as names and addresses.

Respondents are also able to state their religious and political views about the country under review.

• Feedback Form Template

Contained in the template for the online feedback form is the details of a product and/or service used. Identifying this product or service and documenting how long the customer has used them.

The overall satisfaction is measured as well as the delivery of the services. The likelihood that the customer also recommends said product is also measured.

• Online Questionnaire Template

The online questionnaire template houses the respondent’s data as well as educational qualifications to collect information to be used for academic research.

Respondents can also provide their gender, race, and field of study as well as present living conditions as prerequisite data for the research study.

• Student Data Sheet Form Template 

The template is a data sheet containing all the relevant information of a student. The student’s name, home address, guardian’s name, record of attendance as well as performance in school is well represented on this template. This is a perfect data collection method to deploy for a school or an education organization.

Also included is a record for interaction with others as well as a space for a short comment on the overall performance and attitude of the student. 

• Interview Consent Form Template

This online interview consent form template allows the interviewee to sign off their consent to use the interview data for research or report to journalists. With premium features like short text fields, upload, e-signature, etc., Formplus Builder is the perfect tool to create your preferred online consent forms without coding experience.

What is the Best Data Collection Method for Qualitative Data?

Answer: Combination Research

The best data collection method for a researcher for gathering qualitative data which generally is data relying on the feelings, opinions, and beliefs of the respondents would be Combination Research.

The reason why combination research is the best fit is that it encompasses the attributes of Interviews and Focus Groups. It is also useful when gathering data that is sensitive in nature. It can be described as an all-purpose quantitative data collection method.

Above all, combination research improves the richness of data collected when compared with other data collection methods for qualitative data.

What is the Best Data Collection Method for Quantitative Research Data?

Ans: Questionnaire

The best data collection method a researcher can employ in gathering quantitative data which takes into consideration data that can be represented in numbers and figures that can be deduced mathematically is the Questionnaire.

These can be administered to a large number of respondents while saving costs. For quantitative data that may be bulky or voluminous in nature, the use of a Questionnaire makes such data easy to visualize and analyze.

Another key advantage of the Questionnaire is that it can be used to compare and contrast previous research work done to measure changes.

Technology-Enabled Data Collection Methods

There are so many diverse methods available now in the world because technology has revolutionized the way data is being collected. It has provided efficient and innovative methods that anyone, especially researchers and organizations. Below are some technology-enabled data collection methods:

• Online Surveys: Online surveys have gained popularity due to their ease of use and wide reach. You can distribute them through email, social media, or embed them on websites. Online surveys allow you to quickly complete data collection, automated data capture, and real-time analysis. Online surveys also offer features like skip logic, validation checks, and multimedia integration.
• Mobile Surveys: With the widespread use of smartphones, mobile surveys’ popularity is also on the rise. Mobile surveys leverage the capabilities of mobile devices, and this allows respondents to participate at their convenience. This includes multimedia elements, location-based information, and real-time feedback. Mobile surveys are the best for capturing in-the-moment experiences or opinions.
• Social Media Listening: Social media platforms are a good source of unstructured data that you can analyze to gain insights into customer sentiment and trends. Social media listening involves monitoring and analyzing social media conversations, mentions, and hashtags to understand public opinion, identify emerging topics, and assess brand reputation.
• Wearable Devices and Sensors: You can embed wearable devices, such as fitness trackers or smartwatches, and sensors in everyday objects to capture continuous data on various physiological and environmental variables. This data can provide you with insights into health behaviors, activity patterns, sleep quality, and environmental conditions, among others.
• Big Data Analytics: Big data analytics leverages large volumes of structured and unstructured data from various sources, such as transaction records, social media, and internet browsing. Advanced analytics techniques, like machine learning and natural language processing, can extract meaningful insights and patterns from this data, enabling organizations to make data-driven decisions.

Read Also: How Technology is Revolutionizing Data Collection

Faulty Data Collection Practices – Common Mistakes & Sources of Error

While technology-enabled data collection methods offer numerous advantages, there are some pitfalls and sources of error that you should be aware of. Here are some common mistakes and sources of error in data collection:

• Population Specification Error: Population specification error occurs when the target population is not clearly defined or misidentified. This error leads to a mismatch between the research objectives and the actual population being studied, resulting in biased or inaccurate findings.
• Sample Frame Error: Sample frame error occurs when the sampling frame, the list or source from which the sample is drawn, does not adequately represent the target population. This error can introduce selection bias and affect the generalizability of the findings.
• Selection Error: Selection error occurs when the process of selecting participants or units for the study introduces bias. It can happen due to nonrandom sampling methods, inadequate sampling techniques, or self-selection bias. Selection error compromises the representativeness of the sample and affects the validity of the results.
• Nonresponse Error: Nonresponse error occurs when selected participants choose not to participate or fail to respond to the data collection effort. Nonresponse bias can result in an unrepresentative sample if those who choose not to respond differ systematically from those who do respond. Efforts should be made to mitigate nonresponse and encourage participation to minimize this error.
• Measurement Error: Measurement error arises from inaccuracies or inconsistencies in the measurement process. It can happen due to poorly designed survey instruments, ambiguous questions, respondent bias, or errors in data entry or coding. Measurement errors can lead to distorted or unreliable data, affecting the validity and reliability of the findings.

In order to mitigate these errors and ensure high-quality data collection, you should carefully plan your data collection procedures, and validate measurement tools. You should also use appropriate sampling techniques, employ randomization where possible, and minimize nonresponse through effective communication and incentives. Ensure you conduct regular checks and implement validation processes, and data cleaning procedures to identify and rectify errors during data analysis.

Best Practices for Data Collection

• Clearly Define Objectives: Clearly define the research objectives and questions to guide the data collection process. This helps ensure that the collected data aligns with the research goals and provides relevant insights.
• Plan Ahead: Develop a detailed data collection plan that includes the timeline, resources needed, and specific procedures to follow. This helps maintain consistency and efficiency throughout the data collection process.
• Choose the Right Method: Select data collection methods that are appropriate for the research objectives and target population. Consider factors such as feasibility, cost-effectiveness, and the ability to capture the required data accurately.
• Pilot Test : Before full-scale data collection, conduct a pilot test to identify any issues with the data collection instruments or procedures. This allows for refinement and improvement before data collection with the actual sample.
• Train Data Collectors: If data collection involves human interaction, ensure that data collectors are properly trained on the data collection protocols, instruments, and ethical considerations. Consistent training helps minimize errors and maintain data quality.
• Maintain Consistency: Follow standardized procedures throughout the data collection process to ensure consistency across data collectors and time. This includes using consistent measurement scales, instructions, and data recording methods.
• Minimize Bias: Be aware of potential sources of bias in data collection and take steps to minimize their impact. Use randomization techniques, employ diverse data collectors, and implement strategies to mitigate response biases.
• Ensure Data Quality: Implement quality control measures to ensure the accuracy, completeness, and reliability of the collected data. Conduct regular checks for data entry errors, inconsistencies, and missing values.
• Maintain Data Confidentiality: Protect the privacy and confidentiality of participants’ data by implementing appropriate security measures. Ensure compliance with data protection regulations and obtain informed consent from participants.
• Document the Process: Keep detailed documentation of the data collection process, including any deviations from the original plan, challenges encountered, and decisions made. This documentation facilitates transparency, replicability, and future analysis.

FAQs about Data Collection

• What are secondary sources of data collection? Secondary sources of data collection are defined as the data that has been previously gathered and is available for your use as a researcher. These sources can include published research papers, government reports, statistical databases, and other existing datasets.
• What are the primary sources of data collection? Primary sources of data collection involve collecting data directly from the original source also known as the firsthand sources. You can do this through surveys, interviews, observations, experiments, or other direct interactions with individuals or subjects of study.
• How many types of data are there? There are two main types of data: qualitative and quantitative. Qualitative data is non-numeric and it includes information in the form of words, images, or descriptions. Quantitative data, on the other hand, is numeric and you can measure and analyze it statistically.

Sign up on Formplus Builder to create your preferred online surveys or questionnaire for data collection. You don’t need to be tech-savvy!

Connect to Formplus, Get Started Now - It's Free!

• academic research
• Data collection method
• data collection techniques
• data collection tool
• data collection tools
• field data collection
• online data collection tool
• product research
• qualitative research data
• quantitative research data
• scientific research
• busayo.longe

You may also like:

Data Collection Sheet: Types + [Template Examples]

Simple guide on data collection sheet. Types, tools, and template examples.

User Research: Definition, Methods, Tools and Guide

In this article, you’ll learn to provide value to your target market with user research. As a bonus, we’ve added user research tools and...

How Technology is Revolutionizing Data Collection

As global industrialization continues to transform, it is becoming evident that there is a ubiquity of large datasets driven by the need...

Data Collection Plan: Definition + Steps to Do It

Introduction A data collection plan is a way to get specific information on your audience. You can use it to better understand what they...

Formplus - For Seamless Data Collection

Collect data the right way with a versatile data collection tool. try formplus and transform your work productivity today..

• Business Essentials
• Leadership & Management
• Credential of Leadership, Impact, and Management in Business (CLIMB)
• Entrepreneurship & Innovation
• *New* Digital Transformation
• Finance & Accounting
• Business in Society
• For Organizations
• Support Portal
• Media Coverage
• Founding Donors
• Leadership Team

• Harvard Business School →
• HBS Online →
• Business Insights →

Business Insights

Harvard Business School Online's Business Insights Blog provides the career insights you need to achieve your goals and gain confidence in your business skills.

• Career Development
• Communication
• Decision-Making
• Earning Your MBA
• Negotiation
• News & Events
• Productivity
• Staff Spotlight
• Student Profiles
• Work-Life Balance
• Alternative Investments
• Business Analytics
• Business Strategy
• Business and Climate Change
• Design Thinking and Innovation
• Digital Marketing Strategy
• Disruptive Strategy
• Economics for Managers
• Entrepreneurship Essentials
• Financial Accounting
• Global Business
• Launching Tech Ventures
• Leadership Principles
• Leadership, Ethics, and Corporate Accountability
• Leading with Finance
• Management Essentials
• Negotiation Mastery
• Organizational Leadership
• Power and Influence for Positive Impact
• Strategy Execution
• Sustainable Business Strategy
• Sustainable Investing
• Winning with Digital Platforms

7 Data Collection Methods in Business Analytics

• 02 Dec 2021

Data is being generated at an ever-increasing pace. According to Statista , the total volume of data was 64.2 zettabytes in 2020; it’s predicted to reach 181 zettabytes by 2025. This abundance of data can be overwhelming if you aren’t sure where to start.

So, how do you ensure the data you use is relevant and important to the business problems you aim to solve? After all, a data-driven decision is only as strong as the data it’s based on. One way is to collect data yourself.

Here’s a breakdown of data types, why data collection is important, what to know before you begin collecting, and seven data collection methods to leverage.

Access your free e-book today.

What Is Data Collection?

Data collection is the methodological process of gathering information about a specific subject. It’s crucial to ensure your data is complete during the collection phase and that it’s collected legally and ethically . If not, your analysis won’t be accurate and could have far-reaching consequences.

In general, there are three types of consumer data:

• First-party data , which is collected directly from users by your organization
• Second-party data , which is data shared by another organization about its customers (or its first-party data)
• Third-party data , which is data that’s been aggregated and rented or sold by organizations that don’t have a connection to your company or users

Although there are use cases for second- and third-party data, first-party data (data you’ve collected yourself) is more valuable because you receive information about how your audience behaves, thinks, and feels—all from a trusted source.

Data can be qualitative (meaning contextual in nature) or quantitative (meaning numeric in nature). Many data collection methods apply to either type, but some are better suited to one over the other.

In the data life cycle , data collection is the second step. After data is generated, it must be collected to be of use to your team. After that, it can be processed, stored, managed, analyzed, and visualized to aid in your organization’s decision-making.

Before collecting data, there are several factors you need to define:

• The question you aim to answer
• The data subject(s) you need to collect data from
• The collection timeframe
• The data collection method(s) best suited to your needs

The data collection method you select should be based on the question you want to answer, the type of data you need, your timeframe, and your company’s budget.

The Importance of Data Collection

Collecting data is an integral part of a business’s success; it can enable you to ensure the data’s accuracy, completeness, and relevance to your organization and the issue at hand. The information gathered allows organizations to analyze past strategies and stay informed on what needs to change.

The insights gleaned from data can make you hyperaware of your organization’s efforts and give you actionable steps to improve various strategies—from altering marketing strategies to assessing customer complaints.

Basing decisions on inaccurate data can have far-reaching negative consequences, so it’s important to be able to trust your own data collection procedures and abilities. By ensuring accurate data collection, business professionals can feel secure in their business decisions.

Explore the options in the next section to see which data collection method is the best fit for your company.

7 Data Collection Methods Used in Business Analytics

Surveys are physical or digital questionnaires that gather both qualitative and quantitative data from subjects. One situation in which you might conduct a survey is gathering attendee feedback after an event. This can provide a sense of what attendees enjoyed, what they wish was different, and areas in which you can improve or save money during your next event for a similar audience.

While physical copies of surveys can be sent out to participants, online surveys present the opportunity for distribution at scale. They can also be inexpensive; running a survey can cost nothing if you use a free tool. If you wish to target a specific group of people, partnering with a market research firm to get the survey in front of that demographic may be worth the money.

Something to watch out for when crafting and running surveys is the effect of bias, including:

• Collection bias : It can be easy to accidentally write survey questions with a biased lean. Watch out for this when creating questions to ensure your subjects answer honestly and aren’t swayed by your wording.
• Subject bias : Because your subjects know their responses will be read by you, their answers may be biased toward what seems socially acceptable. For this reason, consider pairing survey data with behavioral data from other collection methods to get the full picture.

Related: 3 Examples of Bad Survey Questions & How to Fix Them

2. Transactional Tracking

Each time your customers make a purchase, tracking that data can allow you to make decisions about targeted marketing efforts and understand your customer base better.

Often, e-commerce and point-of-sale platforms allow you to store data as soon as it’s generated, making this a seamless data collection method that can pay off in the form of customer insights.

3. Interviews and Focus Groups

Interviews and focus groups consist of talking to subjects face-to-face about a specific topic or issue. Interviews tend to be one-on-one, and focus groups are typically made up of several people. You can use both to gather qualitative and quantitative data.

Through interviews and focus groups, you can gather feedback from people in your target audience about new product features. Seeing them interact with your product in real-time and recording their reactions and responses to questions can provide valuable data about which product features to pursue.

As is the case with surveys, these collection methods allow you to ask subjects anything you want about their opinions, motivations, and feelings regarding your product or brand. It also introduces the potential for bias. Aim to craft questions that don’t lead them in one particular direction.

One downside of interviewing and conducting focus groups is they can be time-consuming and expensive. If you plan to conduct them yourself, it can be a lengthy process. To avoid this, you can hire a market research facilitator to organize and conduct interviews on your behalf.

4. Observation

Observing people interacting with your website or product can be useful for data collection because of the candor it offers. If your user experience is confusing or difficult, you can witness it in real-time.

Yet, setting up observation sessions can be difficult. You can use a third-party tool to record users’ journeys through your site or observe a user’s interaction with a beta version of your site or product.

While less accessible than other data collection methods, observations enable you to see firsthand how users interact with your product or site. You can leverage the qualitative and quantitative data gleaned from this to make improvements and double down on points of success.

5. Online Tracking

To gather behavioral data, you can implement pixels and cookies. These are both tools that track users’ online behavior across websites and provide insight into what content they’re interested in and typically engage with.

You can also track users’ behavior on your company’s website, including which parts are of the highest interest, whether users are confused when using it, and how long they spend on product pages. This can enable you to improve the website’s design and help users navigate to their destination.

Inserting a pixel is often free and relatively easy to set up. Implementing cookies may come with a fee but could be worth it for the quality of data you’ll receive. Once pixels and cookies are set, they gather data on their own and don’t need much maintenance, if any.

It’s important to note: Tracking online behavior can have legal and ethical privacy implications. Before tracking users’ online behavior, ensure you’re in compliance with local and industry data privacy standards .

Online forms are beneficial for gathering qualitative data about users, specifically demographic data or contact information. They’re relatively inexpensive and simple to set up, and you can use them to gate content or registrations, such as webinars and email newsletters.

You can then use this data to contact people who may be interested in your product, build out demographic profiles of existing customers, and in remarketing efforts, such as email workflows and content recommendations.

Related: What Is Marketing Analytics?

7. Social Media Monitoring

Monitoring your company’s social media channels for follower engagement is an accessible way to track data about your audience’s interests and motivations. Many social media platforms have analytics built in, but there are also third-party social platforms that give more detailed, organized insights pulled from multiple channels.

You can use data collected from social media to determine which issues are most important to your followers. For instance, you may notice that the number of engagements dramatically increases when your company posts about its sustainability efforts.

Building Your Data Capabilities

Understanding the variety of data collection methods available can help you decide which is best for your timeline, budget, and the question you’re aiming to answer. When stored together and combined, multiple data types collected through different methods can give an informed picture of your subjects and help you make better business decisions.

Do you want to become a data-driven professional? Explore our eight-week Business Analytics course and our three-course Credential of Readiness (CORe) program to deepen your analytical skills and apply them to real-world business problems. Not sure which course is right for you? Download our free flowchart .

This post was updated on October 17, 2022. It was originally published on December 2, 2021.

About the Author

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Data Collection Methods | Step-by-Step Guide & Examples

Data Collection Methods | Step-by-Step Guide & Examples

Published on 4 May 2022 by Pritha Bhandari .

Data collection is a systematic process of gathering observations or measurements. Whether you are performing research for business, governmental, or academic purposes, data collection allows you to gain first-hand knowledge and original insights into your research problem .

While methods and aims may differ between fields, the overall process of data collection remains largely the same. Before you begin collecting data, you need to consider:

• The  aim of the research
• The type of data that you will collect
• The methods and procedures you will use to collect, store, and process the data

To collect high-quality data that is relevant to your purposes, follow these four steps.

Table of contents

Step 1: define the aim of your research, step 2: choose your data collection method, step 3: plan your data collection procedures, step 4: collect the data, frequently asked questions about data collection.

Before you start the process of data collection, you need to identify exactly what you want to achieve. You can start by writing a problem statement : what is the practical or scientific issue that you want to address, and why does it matter?

Next, formulate one or more research questions that precisely define what you want to find out. Depending on your research questions, you might need to collect quantitative or qualitative data :

• Quantitative data is expressed in numbers and graphs and is analysed through statistical methods .
• Qualitative data is expressed in words and analysed through interpretations and categorisations.

If your aim is to test a hypothesis , measure something precisely, or gain large-scale statistical insights, collect quantitative data. If your aim is to explore ideas, understand experiences, or gain detailed insights into a specific context, collect qualitative data.

If you have several aims, you can use a mixed methods approach that collects both types of data.

• Your first aim is to assess whether there are significant differences in perceptions of managers across different departments and office locations.
• Your second aim is to gather meaningful feedback from employees to explore new ideas for how managers can improve.

Prevent plagiarism, run a free check.

Based on the data you want to collect, decide which method is best suited for your research.

• Experimental research is primarily a quantitative method.
• Interviews , focus groups , and ethnographies are qualitative methods.
• Surveys , observations, archival research, and secondary data collection can be quantitative or qualitative methods.

Carefully consider what method you will use to gather data that helps you directly answer your research questions.

When you know which method(s) you are using, you need to plan exactly how you will implement them. What procedures will you follow to make accurate observations or measurements of the variables you are interested in?

For instance, if you’re conducting surveys or interviews, decide what form the questions will take; if you’re conducting an experiment, make decisions about your experimental design .

Operationalisation

Sometimes your variables can be measured directly: for example, you can collect data on the average age of employees simply by asking for dates of birth. However, often you’ll be interested in collecting data on more abstract concepts or variables that can’t be directly observed.

Operationalisation means turning abstract conceptual ideas into measurable observations. When planning how you will collect data, you need to translate the conceptual definition of what you want to study into the operational definition of what you will actually measure.

• You ask managers to rate their own leadership skills on 5-point scales assessing the ability to delegate, decisiveness, and dependability.
• You ask their direct employees to provide anonymous feedback on the managers regarding the same topics.

You may need to develop a sampling plan to obtain data systematically. This involves defining a population , the group you want to draw conclusions about, and a sample, the group you will actually collect data from.

Your sampling method will determine how you recruit participants or obtain measurements for your study. To decide on a sampling method you will need to consider factors like the required sample size, accessibility of the sample, and time frame of the data collection.

Standardising procedures

If multiple researchers are involved, write a detailed manual to standardise data collection procedures in your study.

This means laying out specific step-by-step instructions so that everyone in your research team collects data in a consistent way – for example, by conducting experiments under the same conditions and using objective criteria to record and categorise observations.

This helps ensure the reliability of your data, and you can also use it to replicate the study in the future.

Creating a data management plan

Before beginning data collection, you should also decide how you will organise and store your data.

• If you are collecting data from people, you will likely need to anonymise and safeguard the data to prevent leaks of sensitive information (e.g. names or identity numbers).
• If you are collecting data via interviews or pencil-and-paper formats, you will need to perform transcriptions or data entry in systematic ways to minimise distortion.
• You can prevent loss of data by having an organisation system that is routinely backed up.

Finally, you can implement your chosen methods to measure or observe the variables you are interested in.

The closed-ended questions ask participants to rate their manager’s leadership skills on scales from 1 to 5. The data produced is numerical and can be statistically analysed for averages and patterns.

To ensure that high-quality data is recorded in a systematic way, here are some best practices:

• Record all relevant information as and when you obtain data. For example, note down whether or how lab equipment is recalibrated during an experimental study.
• Double-check manual data entry for errors.
• If you collect quantitative data, you can assess the reliability and validity to get an indication of your data quality.

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organisations.

When conducting research, collecting original data has significant advantages:

• You can tailor data collection to your specific research aims (e.g., understanding the needs of your consumers or user testing your website).
• You can control and standardise the process for high reliability and validity (e.g., choosing appropriate measurements and sampling methods ).

However, there are also some drawbacks: data collection can be time-consuming, labour-intensive, and expensive. In some cases, it’s more efficient to use secondary data that has already been collected by someone else, but the data might be less reliable.

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to test a hypothesis by systematically collecting and analysing data, while qualitative methods allow you to explore ideas and experiences in depth.

Reliability and validity are both about how well a method measures something:

• Reliability refers to the  consistency of a measure (whether the results can be reproduced under the same conditions).
• Validity   refers to the  accuracy of a measure (whether the results really do represent what they are supposed to measure).

If you are doing experimental research , you also have to consider the internal and external validity of your experiment.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

Operationalisation means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioural avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalise the variables that you want to measure.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Bhandari, P. (2022, May 04). Data Collection Methods | Step-by-Step Guide & Examples. Scribbr. Retrieved 15 April 2024, from https://www.scribbr.co.uk/research-methods/data-collection-guide/

Is this article helpful?

Pritha Bhandari

Other students also liked, qualitative vs quantitative research | examples & methods, triangulation in research | guide, types, examples, what is a conceptual framework | tips & examples.

• Sponsored Article

Advancing Clinical Research Through Effective Data Delivery

Novel data collection and delivery strategies help usher the clinical research industry into its next era..

The clinical research landscape is rapidly transforming. Instead of viewing patients as subjects, sponsors now use the patients’ input to help reduce the burden they face during trials. This patient-centric approach is necessary to ensure that the clinical trial staff recruit and retain enough participants and it has led the industry to modify all stages of the clinical trial life cycle, from design to analysis. “What we are seeing is a lot more openness to innovations, digitization, remote visits for the patient, and telemedicine, for example,” said Rose Kidd, the president of Global Operations Delivery at ICON, who oversees a variety of areas including site and patient solutions, study start up, clinical data science, biostatistics, medical writing, and pharmacovigilance. “It is becoming a lot more decentralized in terms of how we collect clinical data, which is really constructive for the industry, and also hugely positive for patients.” 

The Increasing Complexity of Clinical Trials

Accurate data is central to the success of a clinical trial. “Research results are only as reliable as the data on which they are based,” Kidd remarked. “If your data is of high quality, the conclusions of that data are trustworthy.” Sponsors are now collecting more data than ever through their trials. 1 This allows them to observe trends and make well-informed decisions about a drug’s or device’s development. 

However, these changes in data volume complicate how clinicians design and run their clinical trials. They must capture enough data to fully assess the drug or device without severely disrupting a patient’s lifestyle. Additionally, the investigational sites must ensure that they have enough staff to collect the data in the clinic or through home visits and keep up with their country’s clinical trial regulations. They also must develop efficient data collection and delivery strategies to ensure a trial’s success. While poorly collected data can introduce noise, properly collected data allows clinical trial leads to quickly consolidate and analyze this information. 2 And they often require support with this process. 

Innovative Solutions to Improve Data Collection and Delivery 

Fortunately, sponsors can find that support with ICON, the healthcare intelligence and clinical research organization. “We essentially advance clinical research [by] providing outsourced services to the pharmaceutical industry, to the medical device industry, and also to government and public health organizations,” Kidd explained. With expertise in numerous therapeutic areas, such as oncology, cell and gene therapies, cardiovascular, biosimilars, vaccines, and rare diseases to mention just a few, ICON helps the pharmaceutical industry efficiently bring devices and drugs to the patients that need them, while ensuring patient safety and meeting local regulations. 

One of the areas that Kidd’s team is specifically focused on is providing solutions to advance the collection, delivery, and analysis of clinical data.

The platform that ICON provides to support sponsors in this regard not only stores data directly entered into the system by clinicians during their site or home visits, but also serves as an electronic diary for patients to remotely record their symptoms as they happen. This makes it easier for patients to participate in clinical trials while maintaining their jobs and familial responsibilities. Moreover, this solution provides clinical trial staff with insights into their data as they emerge, such as adverse event profiles and the geographical spread of these events. However, this requires that the data is input into the system in the same manner at every participating site. 

To address this problem, ICON’s solutions also include a site-facing web portal that helps to reduce the training burden by standardizing data capture and allowing site teams to learn key information about a drug or device. The portal also offers a visit-by-visit guide to ensure that clinicians are asking the necessary questions for a particular visit and helps them remember how to record the data correctly. “It is training at their fingertips when they need it most,” Kidd said. Solutions like these help sponsors obtain the high-quality clinical data that they need to progress from the trial to the market.

Clinical research is evolving and data strategies that support sites and patients alike must similarly evolve. With the right expertise, experience, and technology solutions, ICON is supporting better decision-making by sponsors.

• Crowley E, et al. Using systematic data categorisation to quantify the types of data collected in clinical trials: The DataCat project . Trials . 2020;21(1):535.
• McGuckin T, et al. Understanding challenges of using routinely collected health data to address clinical care gaps: A case study in Alberta, Canada . BMJ Open Qual . 2022;11(1):e001491.

Related drug development Research Resources

Building Advanced Cell Models for Toxicity Testing

How Cloud Labs and Remote Research Shape Science 

High Throughput Screening Models Ramp up Lead Discovery

Importance of Data Collection in Public Health

The daily decisions and long-term career choices of healthcare professionals all rely on data. Additionally, accurate and organized data is critical to helping patients, understanding public health crises, and developing solutions based on population-wide epidemiological data. 

Then there is the industry of big data analytics. This market is worth approximately $103 billion as of 2023, and 97 percent of businesses invest in big data, including many in the public health sector. Correctly using information for public health means understanding the importance of data collection and how it benefits students of public health , professionals in the industry, and patients through multidisciplinary approaches.

The Importance of Data Collection in Public Health

Many industries use data collection strategies to understand their clients, customers, and employees. In public health, data collection can contribute to more efficient communication and improved disease and injury prevention strategies. Data in this particular area of health care can also include information about community-wide beliefs and attitudes, local resources, and other contextual issues.

Unstructured data may be collected, but the lack of organization makes it difficult to analyze and implement. A key component of data collection in public health is having a clear organization system to allow patient data to be easily shared with relevant healthcare professionals.

Privacy and Data Collection

The nature of data collection in public health requires professionals to safely store patient information and only share it with relevant parties. Healthcare professionals and anyone working in public health must carefully review the Health Insurance Portability and Accountability Act of 1996 (HIPAA). This privacy rule is meant to ensure data is collected, stored, analyzed, and shared responsibly. Anyone handling patient data and other protected forms of medical data must review HIPAA rules and any other relevant regulations before collecting or accessing data.

Advance Your Public Health Career with a DrPH in Leadership, Advocacy & Equity

Pursue your degree online from tulane university, the benefits of public health data collection.

Making data-driven decisions is virtually impossible without relevant, up-to-date information. Whether tracking the spread of disease, informing the public on the latest prevention strategies, or promoting public health throughout the population, professionals must understand the importance of data collection as they collaborate with other public health practitioners from various disciplines. Below are some of the key benefits that are realized when public health data is gathered in a centralized, responsible, and safe way.

Streamlines Communication

When public health professionals have to request access to separate databases for medical data, it can result in an inefficient, disjointed data analysis process. Storing data in a single location, while still following HIPPA requirements, encourages efficient communication. Professionals can review the full health history of a patient in one place and explore community-wide health data trends.

Encourages Data-Driven Decisions

Some individual illnesses and community-wide outbreaks can be prevented with prompt responses from healthcare professionals. Quality data can dramatically improve the abilities of healthcare leaders in analyzing trends that show early warning signs of an illness or outbreak. Data-driven decisions are crucial to protecting patients and exploring community-wide health strategies.

Reduces Costs

The importance of data collection in leveraging data-driven insights is crucial. It can reduce both the time and the cost of public health initiatives. By first learning more about current public health dynamics, health leaders can focus their time and resources on providing essential services that target community needs confirmed by data.

Well organized data is particularly effective at reducing costs because it can decrease the time it takes to communicate between facilities and professionals and streamline the process of allocating appropriate health resources. High-quality data stored in an efficient database can also reduce the likelihood of errors in data entry, which can lead to costly mistakes in a healthcare setting.

Offers Opportunities for Artificial Intelligence

Artificial Intelligence (AI) is a rapidly growing topic in the healthcare industry and many other industries. Ascension, one of the largest private healthcare systems in the U.S., and the Mayo Clinic are two examples of healthcare organizations exploring ways AI can support clinical decision making.

Multidisciplinary Areas That Inform Public Health Research

How do public health professionals gather data? The importance of data collection hinges on not only the safety and extent of data collected but also on the relevance of the information. Below are a few of the medical disciplines from which public health leaders collect relevant data.

• Biostatistics
• Epidemiology
• Human and social sciences

Data Collection Approaches

There are many strategies for collecting data in various medical disciplines. Public health leaders and researchers should carefully consider the methods used to gather data to better understand how to organize and implement that information. The following are the most common sources for collecting healthcare data.

• Population surveys
• Environmental data
• Notifiable disease registries
• Case-control studies
• Morbidity registries
• Cohort studies
• Medical-administrative databases
• Cross-cutting studies

Distinct data collection methods and sources can help broaden the scope of the data. Access to a broad range of qualitative and quantitative data is particularly important for public health leaders who explore community-wide strategies related to health care and disease prevention.

Discover More Ways to Improve Public Health Outcomes

If you wish to learn more about the importance of data collection as a public health professional, consider enrolling in the Tulane University Online Doctor of Public Health (DrPH) in Leadership, Advocacy, and Equity program. The curriculum includes exploring and advancing strategies for collecting, analyzing, and implementing data to protect communities. 

Take the next step in your career journey — with a globally recognized DrPH program focused on collaborating with experts, serving communities, and preparing equity-minded advocates for careers in public health.

Recommended Readings

The Role of Leadership in Public Health Advocacy

DrPH vs. PhD: What’s the Difference?

Effective Leadership in Public Health: Essential Skills

Centers for Disease Control and Prevention, How Data Authority Improves Public Health

Foresee Medical, Importance of Data Collection in Healthcare

Inserm, Public Health Research

TechReport, “30 Impressive Big Data Statistics for 2023”

Rural Health Information Hub, Data Collection Strategies

U.S. Department of Health and Human Services, Summary of the HIPAA Privacy Rule

World Health Organization, Understanding Global Health Through Data Collection

Request Information

Get important details about Tulane's Online MHA, MPH, MSPH, and DrPH programs, such as admission requirements, your financial aid options, and how to apply.

• Introduction
• Conclusions
• Article Information

The data take into consideration adoption in registrational clinical trials only. eCOA indicates electronic clinical outcome assessment; eDiary, electronic diary.

eFigure 1. Reasons for Adopting Remote Monitoring and Data Collection Technologies

eFigure 2. Potential Short-Term (Within 3 Years) and Long-Term (Next 3+ Years) Impact of Remote Engagement and Data Collection Technologies

eMethods. Bloomberg New Economy International Cancer Coalition–Patient Centricity Survey

Data Sharing Statement

See More About

Sign up for emails based on your interests, select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing

Get the latest research based on your areas of interest.

Others also liked.

• Download PDF
• X Facebook More LinkedIn

Daly B , Brawley OW , Gospodarowicz MK, et al. Remote Monitoring and Data Collection for Decentralized Clinical Trials. JAMA Netw Open. 2024;7(4):e246228. doi:10.1001/jamanetworkopen.2024.6228

Manage citations:

© 2024

• Permissions

Remote Monitoring and Data Collection for Decentralized Clinical Trials

• 1 Department of Medicine, Memorial Sloan Kettering Cancer Center and Weill Cornell Medicine, New York, New York
• 2 School of Medicine, Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland
• 3 Princess Margaret Cancer Center, University of Toronto, Toronto, Ontario, Canada
• 4 Medicine and Human Genetics, Center for Clinical Cancer Genetics and Global Health, University of Chicago Medical Center, Chicago, Illinois
• 5 Oncology Center of Excellence, US Food and Drug Administration, Silver Spring, Maryland
• 6 Susan G. Komen Foundation, Dallas, Texas
• 7 Amgen Inc, Thousand Oaks, California
• 8 Janssen, Johnson & Johnson, New Brunswick, New Jersey
• 9 BeiGene, Cambridge, Massachusetts
• 10 Genentech, South San Francisco, California
• 11 Yale Cancer Center, Yale School of Medicine, New Haven, Connecticut
• 12 Science 37, Durham, North Carolina
• 13 Internal Medicine, Gastrointestinal Oncology, Simmons Comprehensive Cancer Center, University of Texas Southwestern Medical Center, Dallas, Texas
• 14 Jiangsu Hengrui Pharmaceuticals, Shanghai, China
• 15 Novartis, Basel, Switzerland
• 16 AstraZeneca, Cambridge, United Kingdom
• 17 Bayer, Leverkusen, Germany
• 18 Asia Society, New York, New York
• 19 Bloomberg New Economy, Bloomberg LP, New York, New York
• 20 McKinsey Cancer Center, McKinsey & Company, New York, New York
• 21 Guangdong Lung Cancer Institute, Chinese Thoracic Oncology Group, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China

Question   What are the current state and future aspirations for the use of remote technologies in oncology clinical trials?

Findings   In this survey study of 8 biopharmaceutical companies representing 33% of the oncology marketplace, the difference between current and aspired adoption of remote technologies in 5 years is large, with respondents expecting a 40% or greater adoption increase in 8 of 11 enabling technologies.

Meaning   These findings set benchmarks that may galvanize momentum toward greater adoption of enabling technologies, supporting a new paradigm of trials that are more accessible.

Importance   Less than 5% of patients with cancer enroll in a clinical trial, partly due to financial and logistic burdens, especially among underserved populations. The COVID-19 pandemic marked a substantial shift in the adoption of decentralized trial operations by pharmaceutical companies.

Objective   To assess the current global state of adoption of decentralized trial technologies, understand factors that may be driving or preventing adoption, and highlight aspirations and direction for industry to enable more patient-centric trials.

Design, Setting, and Participants   The Bloomberg New Economy International Cancer Coalition, composed of patient advocacy, industry, government regulator, and academic medical center representatives, developed a survey directed to global biopharmaceutical companies of the coalition from October 1 through December 31, 2022, with a focus on registrational clinical trials. The data for this survey study were analyzed between January 1 and 31, 2023.

Exposure   Adoption of decentralized clinical trial technologies.

Main Outcomes and Measures   The survey measured (1) outcomes of different remote monitoring and data collection technologies on patient centricity, (2) adoption of these technologies in oncology and all therapeutic areas, and (3) barriers and facilitators to adoption using descriptive statistics.

Results   All 8 invited coalition companies completed the survey, representing 33% of the oncology market by revenues in 2021. Across nearly all technologies, adoption in oncology trials lags that of all trials. In the current state, electronic diaries and electronic clinical outcome assessments are the most used technology, with a mean (SD) of 56% (19%) and 51% (29%) adoption for all trials and oncology trials, respectively, whereas visits within local physician networks is the least adopted at a mean (SD) of 12% (18%) and 7% (9%), respectively. Looking forward, the difference between the current and aspired adoption rate in 5 years for oncology is large, with respondents expecting a 40% or greater absolute adoption increase in 8 of the 11 technologies surveyed. Furthermore, digitally enabled recruitment, local imaging capabilities, and local physician networks were identified as technologies that could be most effective for improving patient centricity in the long term.

Conclusions and Relevance   These findings may help to galvanize momentum toward greater adoption of enabling technologies to support a new paradigm of trials that are more accessible, less burdensome, and more inclusive.

International oncology societies have stated that clinical trials offer the best care for patients with cancer but that less than 5% of patients enroll in trials worldwide. 1 One cause may be the high financial and logistic burdens of clinical trials, which disproportionately affect underrepresented populations. The disparities may be particularly challenging as most oncology trials are conducted in academic medical centers, but the majority of patients prefer to receive care in the local community. 1 - 4 In the US, nearly one-half of patients with metastatic breast, prostate, colorectal, or non–small cell lung cancer must drive more than 60 minutes each way to access a clinical trial site. 5 A recent survey indicated that 85% of patients with cancer would be more open to joining a trial where they can participate at local facilities, while 82% indicated that they would participate in trials that used wearable technology. 6

These trends underscore the opportunity for sponsors (eg, biopharmaceutical companies, academic institutions) and regulators to adopt remote monitoring and data collection in cancer trials to create a more patient-centric experience. This shift would be timely, as regulators in the US, the European Union, and China are all developing formal guidance on decentralized trials. 7 , 8 The US Food and Drug Omnibus Reform Act, signed into law in December 2022, includes provisions for modernizing clinical trials and requires the US Food and Drug Administration to issue guidance on decentralized trials, including the use of digital health technologies. 9 In April 2022, the Food and Drug Administration issued a draft guidance recommending that sponsors submit diversity plans for clinical trials to ensure inclusion of underrepresented populations. 10 The China National Medical Products Administration’s 2021 draft guideline aims to reduce patient burden to the greatest extent possible during trials without compromising safety or data quality, specifically calling for consideration of telemedicine, wearable medical equipment, and remote research. 11 - 15

A survey on the current and future adoption of remote monitoring and data collection was developed by the Bloomberg New Economy International Cancer Coalition (the coalition) with the following goals: (1) assess the current state of and future aspirations for industry adoption of remote monitoring and data collection in oncology vs other therapeutic areas and (2) understand the environmental factors and objectives driving or preventing the adoption of remote monitoring and data collection in oncology trials. This survey study was not submitted for institutional review board approval because it did not involve human participant research or health care records. Informed consent to participate in the survey was received verbally from each participant at the time of survey initiation. The survey was drafted with input from select coalition members, excluding survey participants, and there was consideration of the American Association for Public Opinion Research ( AAPOR ) guidance in planning, designing, and reporting the survey results. 16

The coalition, established in 2021, represents an international, multistakeholder, private-public collaboration among academia, industry, government, patient advocacy groups, and policy think tanks. 1 It is dedicated to leveraging technology and fostering synergistic collaborations, with a core aim of enhancing patient access to clinical trials worldwide.

The survey, administered from October 1 through December 31, 2022, focused on registrational clinical trials with questions devoted to all therapeutic areas, including oncology, aligned to the broad goals and mission of the coalition. Global biopharmaceutical companies were invited to participate in the survey based on their membership in the coalition at the time of survey administration. Survey participants were not involved in the data analysis, which was performed between January 1 and 31, 2023. The survey consisted of 7 questions (eMethods in Supplement 1 ) organized into 3 sections: (1) outcomes of remote monitoring and data collection technologies associated with patient centricity overall and (2) across all therapeutic areas and oncology and (3) context for adoption and tracking.

The raw data were collected and analyzed using Microsoft Excel, version X (Microsoft Corporation). For adoption rates, the mean and SD were calculated. Effect scores were calculated through assigning answers a number. An answer that ranked the approach as 1 was assigned 3 points, an answer that ranked the approach as 2 was assigned 2 points, and an answer that ranked the approach as 3 was assigned 1 point. Across each answer, the points across the survey respondents were summed.

All 8 invited companies completed the survey for a 100% response rate. These companies comprise approximately 33% of the global oncology pharmaceutical market by revenue.

All organizations reported recent increases in adoption of 11 remote monitoring and data collection technologies surveyed ( Table ). The top 4 reasons for adoption were increased competition for patients in the indication under investigation (6 respondents [75%]), changing expectations on the heels of the COVID-19 pandemic and its associated disruptions (5 respondents [63%]), innovation in novel technical solutions that substantially improved quality and accessibility (5 respondents [63%]), and competition from other stakeholders having encouraged or mandated adoption (5 respondents [63%]). Other reasons included stakeholders (eg, regulatory bodies) encouraging adoption (3 respondents [38%]) (eFigure 1 in Supplement 1 ).

According to the survey, electronic diaries and electronic clinical outcome assessments were the most used technologies, with mean (SD) adoption rates of 56% (19%) and 51% (29%) for all clinical trials and oncology trials, respectively ( Figure 1 ). The next most used technologies were patient engagement dashboards, digitally enabled recruitment, and digitally enabled enrollment, with reported mean (SD) adoption rates of 32% (20%), 27% (10%), and 20% (21%), respectively. The least commonly adopted technologies included permitting patients to use local imaging facilities (mean [SD], 11% [18%] for all trials vs 9% [3%] for oncology trials) or local physician networks (mean [SD], 12% [18%] for all trials vs 7% [9%] for oncology trials), and telemedicine visits (mean [SD], 12% [10%] for all trials vs 11% [13%] for oncology trials) ( Figure 1 ). Adoption rates for remote monitoring and data collection technologies were lower in oncology trials vs all clinical trials for all technologies except digitally enabled enrollment (20% [21%] for all trials vs 26% [28%] for oncology trials).

The difference between current adoption and aspired adoption rates in 5 years of various technologies was large. In 8 of the 11 technologies included in the survey, respondents expected a 40% or greater absolute adoption increase relative to current levels ( Figure 2 ). The greatest mean (SD) differences between current and aspired adoption rates were observed in use of patient engagement dashboards (from 25% [25%] to 79% [20%]), digitally enabled recruitment (from 20% [11%] to 70% [31%]), telemedicine visits (from 11% [13%] to 58% [24%]), visits in local physician networks (from 7% [9%] to 51% [40%]), and digitally enabled enrollment (from 26% [28%] to 70% [26%]).

Respondents identified telemedicine, digitally enabled recruitment, electronic diaries, and electronic clinical outcome assessments as the most effective technologies for advancing patient centricity within trials (defined in the survey as prioritizing the needs of patients) in the short term (within the next 3 years). Over the long term, respondents identified visits in local physician networks, digitally enabled recruitment, and the use of imaging sites near patients and mobile imaging services as the 3 most effective technologies for improving the oncology patient experience (eFigure 2 in Supplement 1 ).

In this survey study of global pharmaceutical companies, the results showed that many of the identified remote monitoring and data collection technologies, such as digital enrollment, are already being adopted across clinical trials. Yet adoption of these innovations is lagging in oncology, despite the great unmet need given the relative lack of clinical trial availability in local community settings. While technology advancements have enabled initial adoption of remote monitoring, 17 , 18 infrastructure has yet to be built to support clinical trial conduct in local physician networks and imaging facilities in diverse global communities. Such an infrastructure is essential to achieve the greatest long-term effect of decentralized trials.

This survey is the first attempt by a broad coalition of stakeholders with vested interest in advancing patient-centric international cancer trials to set an aspiration for remote monitoring and data collection. The difference between current adoption and aspired adoption rates represents an opportunity to rapidly improve the patient experience in cancer trials, broaden access to clinical trials, and correct historical inequities due to structural barriers, such as lack of access to health care. The Food and Drug Omnibus Reform Act 9 and recent guidance from regulatory agencies in the European Union 8 and China 11 - 15 represent an opportunity to accelerate implementation of technologies with the greatest potential to expand patient access to clinical trials and shorten the timeline for development of innovative cancer therapies and preventive interventions.

This study has some limitations. While survey participants represented 33% of the global oncology pharmaceutical market by revenue, the findings are limited by their membership in the coalition at the time of survey administration and may not be entirely representative of the greater global biopharmaceutical industry.

The findings suggest that investment may be required across the drug development ecosystem in both technology access and collaborative research infrastructure to equip stakeholders with the capabilities to adopt decentralized technologies in some of the most complex and historically demanding trials. Moreover, for these advances to be successful, solutions are needed to reduce bureaucratic workload for site staff and investigators and not introduce new administrative hardships. The onus is on the oncology industry and research community to ensure that we create opportunities to fund continued advancements; measure our progress; look for and track balancing measures that could compromise progress; and step back regularly to reflect on the net impact on key clinical development objectives, such as cost, 19 , 20 quality, speed, experience, and equity. 21 , 22

Accepted for Publication: February 13, 2024.

Published: April 12, 2024. doi:10.1001/jamanetworkopen.2024.6228

Open Access: This is an open access article distributed under the terms of the CC-BY License . © 2024 Daly B et al. JAMA Network Open .

Corresponding Authors: Bobby Daly, MD, MBA, Thoracic Oncology Service ( [email protected] ), and Bob T. Li, MD, PhD, MPH ( [email protected] ), Memorial Sloan Kettering Cancer Center, 530 E 74th St, New York, NY 10021.

Author Contributions: Drs Daly and Silverstein had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Daly, Brawley, Olopade, Fashoyin-Aje, Smart, Chang, Tendler, Kim, Fuchs, Zhang, Legos, Duran, Kalidas, Qian, Finnegan, Pilarski, Keane, Shen, Silverstein, Pazdur, Li.

Acquisition, analysis, or interpretation of data: Daly, Gospodarowicz, Fashoyin-Aje, Fuchs, Beg, Legos, Duran, Kalidas, Qian, Pilarski, Silverstein, Wu, Pazdur, Li.

Drafting of the manuscript: Daly, Brawley, Olopade, Fashoyin-Aje, Beg, Kalidas, Qian, Shen, Silverstein, Wu, Pazdur, Li.

Critical review of the manuscript for important intellectual content: Daly, Gospodarowicz, Fashoyin-Aje, Smart, Chang, Tendler, Kim, Fuchs, Zhang, Legos, Duran, Qian, Finnegan, Pilarski, Keane, Shen, Silverstein, Pazdur, Li.

Statistical analysis: Daly, Tendler, Finnegan, Shen.

Administrative, technical, or material support: Daly, Brawley, Olopade, Chang, Tendler, Fuchs, Zhang, Legos, Pilarski, Pazdur, Li.

Supervision: Daly, Fuchs, Beg, Duran, Qian, Finnegan, Pilarski, Pazdur, Li.

Conflict of Interest Disclosures: Dr Daly reported receiving personal fees from Varian Medical Systems during the conduct of the study and being a founding member of the Bloomberg New Economy International Cancer Coalition (unpaid). Dr Olopade reported being cofounder of CancerIQ and receiving other support from Tempus SAB and grants from Color Genomics Research Support and Roche/Genentech outside the submitted work. Dr Kim reported holding patent US11393566B1 for an interoperable platform for reducing redundancy in medical database management. Dr Fuchs reported receiving personal fees from CytomX Therapeutics and being founder of EvolveImmune Therapeutics outside the submitted work. Dr Beg reported receiving personal fees from Ipsen, Seagen, and Foundation Medicine outside the submitted work. Mr Qian reported receiving grants from AstraZeneca during the conduct of the study. Drs Keane, Pilarski, and Silverstein and Ms Shen are consultants with McKinsey & Company, Inc, a global consulting firm that provides services broadly across private, public, and not-for-profit clients, including in the life sciences and health care industries. Dr Wu reported receiving grants from AstraZeneca, Boehringer Ingelheim, Bristol-Myers Squibb, Jiangsu Hengrui Pharmaceuticals, and Roche; personal fees from AstraZeneca, BeiGene, Boehringer Ingelheim, Bristol-Myers Squibb, Eli Lilly, Merck Sharp & Dohme, Pfizer, and Roche; and lecture fees from Sanofi outside the submitted work. Dr Li reported receiving a research project grant and clinical trials funding to his institution from the National Institutes of Health, Amgen, AstraZeneca, Bolt Biotherapeutics, Daiichi Sankyo, Genentech, Jiangsu Hengrui Pharmaceuticals, and Eli Lilly and travel support from Amgen outside the submitted work; holding patents for Memorial Sloan Kettering Cancer Center; being a senior fellow on global health for the Asia Society Policy Institute (unpaid); and being a founding member of the Bloomberg New Economy International Cancer Coalition (unpaid). No other disclosures were reported.

Funding/Support: This work was partially supported by grant P30 CA008748 from the National Institutes of Health (Drs Daly and Li) and a grant from the Emerson Collective (Dr Daly).

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Data Sharing Statement: See Supplement 2 .

• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Neurol Res Pract

How to use and assess qualitative research methods

Loraine busetto.

1 Department of Neurology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120 Heidelberg, Germany

Wolfgang Wick

2 Clinical Cooperation Unit Neuro-Oncology, German Cancer Research Center, Heidelberg, Germany

Christoph Gumbinger

Associated data.

Not applicable.

This paper aims to provide an overview of the use and assessment of qualitative research methods in the health sciences. Qualitative research can be defined as the study of the nature of phenomena and is especially appropriate for answering questions of why something is (not) observed, assessing complex multi-component interventions, and focussing on intervention improvement. The most common methods of data collection are document study, (non-) participant observations, semi-structured interviews and focus groups. For data analysis, field-notes and audio-recordings are transcribed into protocols and transcripts, and coded using qualitative data management software. Criteria such as checklists, reflexivity, sampling strategies, piloting, co-coding, member-checking and stakeholder involvement can be used to enhance and assess the quality of the research conducted. Using qualitative in addition to quantitative designs will equip us with better tools to address a greater range of research problems, and to fill in blind spots in current neurological research and practice.

The aim of this paper is to provide an overview of qualitative research methods, including hands-on information on how they can be used, reported and assessed. This article is intended for beginning qualitative researchers in the health sciences as well as experienced quantitative researchers who wish to broaden their understanding of qualitative research.

What is qualitative research?

Qualitative research is defined as “the study of the nature of phenomena”, including “their quality, different manifestations, the context in which they appear or the perspectives from which they can be perceived” , but excluding “their range, frequency and place in an objectively determined chain of cause and effect” [ 1 ]. This formal definition can be complemented with a more pragmatic rule of thumb: qualitative research generally includes data in form of words rather than numbers [ 2 ].

Why conduct qualitative research?

Because some research questions cannot be answered using (only) quantitative methods. For example, one Australian study addressed the issue of why patients from Aboriginal communities often present late or not at all to specialist services offered by tertiary care hospitals. Using qualitative interviews with patients and staff, it found one of the most significant access barriers to be transportation problems, including some towns and communities simply not having a bus service to the hospital [ 3 ]. A quantitative study could have measured the number of patients over time or even looked at possible explanatory factors – but only those previously known or suspected to be of relevance. To discover reasons for observed patterns, especially the invisible or surprising ones, qualitative designs are needed.

While qualitative research is common in other fields, it is still relatively underrepresented in health services research. The latter field is more traditionally rooted in the evidence-based-medicine paradigm, as seen in " research that involves testing the effectiveness of various strategies to achieve changes in clinical practice, preferably applying randomised controlled trial study designs (...) " [ 4 ]. This focus on quantitative research and specifically randomised controlled trials (RCT) is visible in the idea of a hierarchy of research evidence which assumes that some research designs are objectively better than others, and that choosing a "lesser" design is only acceptable when the better ones are not practically or ethically feasible [ 5 , 6 ]. Others, however, argue that an objective hierarchy does not exist, and that, instead, the research design and methods should be chosen to fit the specific research question at hand – "questions before methods" [ 2 , 7 – 9 ]. This means that even when an RCT is possible, some research problems require a different design that is better suited to addressing them. Arguing in JAMA, Berwick uses the example of rapid response teams in hospitals, which he describes as " a complex, multicomponent intervention – essentially a process of social change" susceptible to a range of different context factors including leadership or organisation history. According to him, "[in] such complex terrain, the RCT is an impoverished way to learn. Critics who use it as a truth standard in this context are incorrect" [ 8 ] . Instead of limiting oneself to RCTs, Berwick recommends embracing a wider range of methods , including qualitative ones, which for "these specific applications, (...) are not compromises in learning how to improve; they are superior" [ 8 ].

Research problems that can be approached particularly well using qualitative methods include assessing complex multi-component interventions or systems (of change), addressing questions beyond “what works”, towards “what works for whom when, how and why”, and focussing on intervention improvement rather than accreditation [ 7 , 9 – 12 ]. Using qualitative methods can also help shed light on the “softer” side of medical treatment. For example, while quantitative trials can measure the costs and benefits of neuro-oncological treatment in terms of survival rates or adverse effects, qualitative research can help provide a better understanding of patient or caregiver stress, visibility of illness or out-of-pocket expenses.

How to conduct qualitative research?

Given that qualitative research is characterised by flexibility, openness and responsivity to context, the steps of data collection and analysis are not as separate and consecutive as they tend to be in quantitative research [ 13 , 14 ]. As Fossey puts it : “sampling, data collection, analysis and interpretation are related to each other in a cyclical (iterative) manner, rather than following one after another in a stepwise approach” [ 15 ]. The researcher can make educated decisions with regard to the choice of method, how they are implemented, and to which and how many units they are applied [ 13 ]. As shown in Fig.  1 , this can involve several back-and-forth steps between data collection and analysis where new insights and experiences can lead to adaption and expansion of the original plan. Some insights may also necessitate a revision of the research question and/or the research design as a whole. The process ends when saturation is achieved, i.e. when no relevant new information can be found (see also below: sampling and saturation). For reasons of transparency, it is essential for all decisions as well as the underlying reasoning to be well-documented.

Iterative research process

While it is not always explicitly addressed, qualitative methods reflect a different underlying research paradigm than quantitative research (e.g. constructivism or interpretivism as opposed to positivism). The choice of methods can be based on the respective underlying substantive theory or theoretical framework used by the researcher [ 2 ].

Data collection

The methods of qualitative data collection most commonly used in health research are document study, observations, semi-structured interviews and focus groups [ 1 , 14 , 16 , 17 ].

Document study

Document study (also called document analysis) refers to the review by the researcher of written materials [ 14 ]. These can include personal and non-personal documents such as archives, annual reports, guidelines, policy documents, diaries or letters.

Observations

Observations are particularly useful to gain insights into a certain setting and actual behaviour – as opposed to reported behaviour or opinions [ 13 ]. Qualitative observations can be either participant or non-participant in nature. In participant observations, the observer is part of the observed setting, for example a nurse working in an intensive care unit [ 18 ]. In non-participant observations, the observer is “on the outside looking in”, i.e. present in but not part of the situation, trying not to influence the setting by their presence. Observations can be planned (e.g. for 3 h during the day or night shift) or ad hoc (e.g. as soon as a stroke patient arrives at the emergency room). During the observation, the observer takes notes on everything or certain pre-determined parts of what is happening around them, for example focusing on physician-patient interactions or communication between different professional groups. Written notes can be taken during or after the observations, depending on feasibility (which is usually lower during participant observations) and acceptability (e.g. when the observer is perceived to be judging the observed). Afterwards, these field notes are transcribed into observation protocols. If more than one observer was involved, field notes are taken independently, but notes can be consolidated into one protocol after discussions. Advantages of conducting observations include minimising the distance between the researcher and the researched, the potential discovery of topics that the researcher did not realise were relevant and gaining deeper insights into the real-world dimensions of the research problem at hand [ 18 ].

Semi-structured interviews

Hijmans & Kuyper describe qualitative interviews as “an exchange with an informal character, a conversation with a goal” [ 19 ]. Interviews are used to gain insights into a person’s subjective experiences, opinions and motivations – as opposed to facts or behaviours [ 13 ]. Interviews can be distinguished by the degree to which they are structured (i.e. a questionnaire), open (e.g. free conversation or autobiographical interviews) or semi-structured [ 2 , 13 ]. Semi-structured interviews are characterized by open-ended questions and the use of an interview guide (or topic guide/list) in which the broad areas of interest, sometimes including sub-questions, are defined [ 19 ]. The pre-defined topics in the interview guide can be derived from the literature, previous research or a preliminary method of data collection, e.g. document study or observations. The topic list is usually adapted and improved at the start of the data collection process as the interviewer learns more about the field [ 20 ]. Across interviews the focus on the different (blocks of) questions may differ and some questions may be skipped altogether (e.g. if the interviewee is not able or willing to answer the questions or for concerns about the total length of the interview) [ 20 ]. Qualitative interviews are usually not conducted in written format as it impedes on the interactive component of the method [ 20 ]. In comparison to written surveys, qualitative interviews have the advantage of being interactive and allowing for unexpected topics to emerge and to be taken up by the researcher. This can also help overcome a provider or researcher-centred bias often found in written surveys, which by nature, can only measure what is already known or expected to be of relevance to the researcher. Interviews can be audio- or video-taped; but sometimes it is only feasible or acceptable for the interviewer to take written notes [ 14 , 16 , 20 ].

Focus groups

Focus groups are group interviews to explore participants’ expertise and experiences, including explorations of how and why people behave in certain ways [ 1 ]. Focus groups usually consist of 6–8 people and are led by an experienced moderator following a topic guide or “script” [ 21 ]. They can involve an observer who takes note of the non-verbal aspects of the situation, possibly using an observation guide [ 21 ]. Depending on researchers’ and participants’ preferences, the discussions can be audio- or video-taped and transcribed afterwards [ 21 ]. Focus groups are useful for bringing together homogeneous (to a lesser extent heterogeneous) groups of participants with relevant expertise and experience on a given topic on which they can share detailed information [ 21 ]. Focus groups are a relatively easy, fast and inexpensive method to gain access to information on interactions in a given group, i.e. “the sharing and comparing” among participants [ 21 ]. Disadvantages include less control over the process and a lesser extent to which each individual may participate. Moreover, focus group moderators need experience, as do those tasked with the analysis of the resulting data. Focus groups can be less appropriate for discussing sensitive topics that participants might be reluctant to disclose in a group setting [ 13 ]. Moreover, attention must be paid to the emergence of “groupthink” as well as possible power dynamics within the group, e.g. when patients are awed or intimidated by health professionals.

Choosing the “right” method

As explained above, the school of thought underlying qualitative research assumes no objective hierarchy of evidence and methods. This means that each choice of single or combined methods has to be based on the research question that needs to be answered and a critical assessment with regard to whether or to what extent the chosen method can accomplish this – i.e. the “fit” between question and method [ 14 ]. It is necessary for these decisions to be documented when they are being made, and to be critically discussed when reporting methods and results.

Let us assume that our research aim is to examine the (clinical) processes around acute endovascular treatment (EVT), from the patient’s arrival at the emergency room to recanalization, with the aim to identify possible causes for delay and/or other causes for sub-optimal treatment outcome. As a first step, we could conduct a document study of the relevant standard operating procedures (SOPs) for this phase of care – are they up-to-date and in line with current guidelines? Do they contain any mistakes, irregularities or uncertainties that could cause delays or other problems? Regardless of the answers to these questions, the results have to be interpreted based on what they are: a written outline of what care processes in this hospital should look like. If we want to know what they actually look like in practice, we can conduct observations of the processes described in the SOPs. These results can (and should) be analysed in themselves, but also in comparison to the results of the document analysis, especially as regards relevant discrepancies. Do the SOPs outline specific tests for which no equipment can be observed or tasks to be performed by specialized nurses who are not present during the observation? It might also be possible that the written SOP is outdated, but the actual care provided is in line with current best practice. In order to find out why these discrepancies exist, it can be useful to conduct interviews. Are the physicians simply not aware of the SOPs (because their existence is limited to the hospital’s intranet) or do they actively disagree with them or does the infrastructure make it impossible to provide the care as described? Another rationale for adding interviews is that some situations (or all of their possible variations for different patient groups or the day, night or weekend shift) cannot practically or ethically be observed. In this case, it is possible to ask those involved to report on their actions – being aware that this is not the same as the actual observation. A senior physician’s or hospital manager’s description of certain situations might differ from a nurse’s or junior physician’s one, maybe because they intentionally misrepresent facts or maybe because different aspects of the process are visible or important to them. In some cases, it can also be relevant to consider to whom the interviewee is disclosing this information – someone they trust, someone they are otherwise not connected to, or someone they suspect or are aware of being in a potentially “dangerous” power relationship to them. Lastly, a focus group could be conducted with representatives of the relevant professional groups to explore how and why exactly they provide care around EVT. The discussion might reveal discrepancies (between SOPs and actual care or between different physicians) and motivations to the researchers as well as to the focus group members that they might not have been aware of themselves. For the focus group to deliver relevant information, attention has to be paid to its composition and conduct, for example, to make sure that all participants feel safe to disclose sensitive or potentially problematic information or that the discussion is not dominated by (senior) physicians only. The resulting combination of data collection methods is shown in Fig.  2 .

Possible combination of data collection methods

Attributions for icons: “Book” by Serhii Smirnov, “Interview” by Adrien Coquet, FR, “Magnifying Glass” by anggun, ID, “Business communication” by Vectors Market; all from the Noun Project

The combination of multiple data source as described for this example can be referred to as “triangulation”, in which multiple measurements are carried out from different angles to achieve a more comprehensive understanding of the phenomenon under study [ 22 , 23 ].

Data analysis

To analyse the data collected through observations, interviews and focus groups these need to be transcribed into protocols and transcripts (see Fig.  3 ). Interviews and focus groups can be transcribed verbatim , with or without annotations for behaviour (e.g. laughing, crying, pausing) and with or without phonetic transcription of dialects and filler words, depending on what is expected or known to be relevant for the analysis. In the next step, the protocols and transcripts are coded , that is, marked (or tagged, labelled) with one or more short descriptors of the content of a sentence or paragraph [ 2 , 15 , 23 ]. Jansen describes coding as “connecting the raw data with “theoretical” terms” [ 20 ]. In a more practical sense, coding makes raw data sortable. This makes it possible to extract and examine all segments describing, say, a tele-neurology consultation from multiple data sources (e.g. SOPs, emergency room observations, staff and patient interview). In a process of synthesis and abstraction, the codes are then grouped, summarised and/or categorised [ 15 , 20 ]. The end product of the coding or analysis process is a descriptive theory of the behavioural pattern under investigation [ 20 ]. The coding process is performed using qualitative data management software, the most common ones being InVivo, MaxQDA and Atlas.ti. It should be noted that these are data management tools which support the analysis performed by the researcher(s) [ 14 ].

From data collection to data analysis

Attributions for icons: see Fig. ​ Fig.2, 2 , also “Speech to text” by Trevor Dsouza, “Field Notes” by Mike O’Brien, US, “Voice Record” by ProSymbols, US, “Inspection” by Made, AU, and “Cloud” by Graphic Tigers; all from the Noun Project

How to report qualitative research?

Protocols of qualitative research can be published separately and in advance of the study results. However, the aim is not the same as in RCT protocols, i.e. to pre-define and set in stone the research questions and primary or secondary endpoints. Rather, it is a way to describe the research methods in detail, which might not be possible in the results paper given journals’ word limits. Qualitative research papers are usually longer than their quantitative counterparts to allow for deep understanding and so-called “thick description”. In the methods section, the focus is on transparency of the methods used, including why, how and by whom they were implemented in the specific study setting, so as to enable a discussion of whether and how this may have influenced data collection, analysis and interpretation. The results section usually starts with a paragraph outlining the main findings, followed by more detailed descriptions of, for example, the commonalities, discrepancies or exceptions per category [ 20 ]. Here it is important to support main findings by relevant quotations, which may add information, context, emphasis or real-life examples [ 20 , 23 ]. It is subject to debate in the field whether it is relevant to state the exact number or percentage of respondents supporting a certain statement (e.g. “Five interviewees expressed negative feelings towards XYZ”) [ 21 ].

How to combine qualitative with quantitative research?

Qualitative methods can be combined with other methods in multi- or mixed methods designs, which “[employ] two or more different methods [ …] within the same study or research program rather than confining the research to one single method” [ 24 ]. Reasons for combining methods can be diverse, including triangulation for corroboration of findings, complementarity for illustration and clarification of results, expansion to extend the breadth and range of the study, explanation of (unexpected) results generated with one method with the help of another, or offsetting the weakness of one method with the strength of another [ 1 , 17 , 24 – 26 ]. The resulting designs can be classified according to when, why and how the different quantitative and/or qualitative data strands are combined. The three most common types of mixed method designs are the convergent parallel design , the explanatory sequential design and the exploratory sequential design. The designs with examples are shown in Fig.  4 .

Three common mixed methods designs

In the convergent parallel design, a qualitative study is conducted in parallel to and independently of a quantitative study, and the results of both studies are compared and combined at the stage of interpretation of results. Using the above example of EVT provision, this could entail setting up a quantitative EVT registry to measure process times and patient outcomes in parallel to conducting the qualitative research outlined above, and then comparing results. Amongst other things, this would make it possible to assess whether interview respondents’ subjective impressions of patients receiving good care match modified Rankin Scores at follow-up, or whether observed delays in care provision are exceptions or the rule when compared to door-to-needle times as documented in the registry. In the explanatory sequential design, a quantitative study is carried out first, followed by a qualitative study to help explain the results from the quantitative study. This would be an appropriate design if the registry alone had revealed relevant delays in door-to-needle times and the qualitative study would be used to understand where and why these occurred, and how they could be improved. In the exploratory design, the qualitative study is carried out first and its results help informing and building the quantitative study in the next step [ 26 ]. If the qualitative study around EVT provision had shown a high level of dissatisfaction among the staff members involved, a quantitative questionnaire investigating staff satisfaction could be set up in the next step, informed by the qualitative study on which topics dissatisfaction had been expressed. Amongst other things, the questionnaire design would make it possible to widen the reach of the research to more respondents from different (types of) hospitals, regions, countries or settings, and to conduct sub-group analyses for different professional groups.

How to assess qualitative research?

A variety of assessment criteria and lists have been developed for qualitative research, ranging in their focus and comprehensiveness [ 14 , 17 , 27 ]. However, none of these has been elevated to the “gold standard” in the field. In the following, we therefore focus on a set of commonly used assessment criteria that, from a practical standpoint, a researcher can look for when assessing a qualitative research report or paper.

Assessors should check the authors’ use of and adherence to the relevant reporting checklists (e.g. Standards for Reporting Qualitative Research (SRQR)) to make sure all items that are relevant for this type of research are addressed [ 23 , 28 ]. Discussions of quantitative measures in addition to or instead of these qualitative measures can be a sign of lower quality of the research (paper). Providing and adhering to a checklist for qualitative research contributes to an important quality criterion for qualitative research, namely transparency [ 15 , 17 , 23 ].

Reflexivity

While methodological transparency and complete reporting is relevant for all types of research, some additional criteria must be taken into account for qualitative research. This includes what is called reflexivity, i.e. sensitivity to the relationship between the researcher and the researched, including how contact was established and maintained, or the background and experience of the researcher(s) involved in data collection and analysis. Depending on the research question and population to be researched this can be limited to professional experience, but it may also include gender, age or ethnicity [ 17 , 27 ]. These details are relevant because in qualitative research, as opposed to quantitative research, the researcher as a person cannot be isolated from the research process [ 23 ]. It may influence the conversation when an interviewed patient speaks to an interviewer who is a physician, or when an interviewee is asked to discuss a gynaecological procedure with a male interviewer, and therefore the reader must be made aware of these details [ 19 ].

Sampling and saturation

The aim of qualitative sampling is for all variants of the objects of observation that are deemed relevant for the study to be present in the sample “ to see the issue and its meanings from as many angles as possible” [ 1 , 16 , 19 , 20 , 27 ] , and to ensure “information-richness [ 15 ]. An iterative sampling approach is advised, in which data collection (e.g. five interviews) is followed by data analysis, followed by more data collection to find variants that are lacking in the current sample. This process continues until no new (relevant) information can be found and further sampling becomes redundant – which is called saturation [ 1 , 15 ] . In other words: qualitative data collection finds its end point not a priori , but when the research team determines that saturation has been reached [ 29 , 30 ].

This is also the reason why most qualitative studies use deliberate instead of random sampling strategies. This is generally referred to as “ purposive sampling” , in which researchers pre-define which types of participants or cases they need to include so as to cover all variations that are expected to be of relevance, based on the literature, previous experience or theory (i.e. theoretical sampling) [ 14 , 20 ]. Other types of purposive sampling include (but are not limited to) maximum variation sampling, critical case sampling or extreme or deviant case sampling [ 2 ]. In the above EVT example, a purposive sample could include all relevant professional groups and/or all relevant stakeholders (patients, relatives) and/or all relevant times of observation (day, night and weekend shift).

Assessors of qualitative research should check whether the considerations underlying the sampling strategy were sound and whether or how researchers tried to adapt and improve their strategies in stepwise or cyclical approaches between data collection and analysis to achieve saturation [ 14 ].

Good qualitative research is iterative in nature, i.e. it goes back and forth between data collection and analysis, revising and improving the approach where necessary. One example of this are pilot interviews, where different aspects of the interview (especially the interview guide, but also, for example, the site of the interview or whether the interview can be audio-recorded) are tested with a small number of respondents, evaluated and revised [ 19 ]. In doing so, the interviewer learns which wording or types of questions work best, or which is the best length of an interview with patients who have trouble concentrating for an extended time. Of course, the same reasoning applies to observations or focus groups which can also be piloted.

Ideally, coding should be performed by at least two researchers, especially at the beginning of the coding process when a common approach must be defined, including the establishment of a useful coding list (or tree), and when a common meaning of individual codes must be established [ 23 ]. An initial sub-set or all transcripts can be coded independently by the coders and then compared and consolidated after regular discussions in the research team. This is to make sure that codes are applied consistently to the research data.

Member checking

Member checking, also called respondent validation , refers to the practice of checking back with study respondents to see if the research is in line with their views [ 14 , 27 ]. This can happen after data collection or analysis or when first results are available [ 23 ]. For example, interviewees can be provided with (summaries of) their transcripts and asked whether they believe this to be a complete representation of their views or whether they would like to clarify or elaborate on their responses [ 17 ]. Respondents’ feedback on these issues then becomes part of the data collection and analysis [ 27 ].

Stakeholder involvement

In those niches where qualitative approaches have been able to evolve and grow, a new trend has seen the inclusion of patients and their representatives not only as study participants (i.e. “members”, see above) but as consultants to and active participants in the broader research process [ 31 – 33 ]. The underlying assumption is that patients and other stakeholders hold unique perspectives and experiences that add value beyond their own single story, making the research more relevant and beneficial to researchers, study participants and (future) patients alike [ 34 , 35 ]. Using the example of patients on or nearing dialysis, a recent scoping review found that 80% of clinical research did not address the top 10 research priorities identified by patients and caregivers [ 32 , 36 ]. In this sense, the involvement of the relevant stakeholders, especially patients and relatives, is increasingly being seen as a quality indicator in and of itself.

How not to assess qualitative research

The above overview does not include certain items that are routine in assessments of quantitative research. What follows is a non-exhaustive, non-representative, experience-based list of the quantitative criteria often applied to the assessment of qualitative research, as well as an explanation of the limited usefulness of these endeavours.

Protocol adherence

Given the openness and flexibility of qualitative research, it should not be assessed by how well it adheres to pre-determined and fixed strategies – in other words: its rigidity. Instead, the assessor should look for signs of adaptation and refinement based on lessons learned from earlier steps in the research process.

Sample size

For the reasons explained above, qualitative research does not require specific sample sizes, nor does it require that the sample size be determined a priori [ 1 , 14 , 27 , 37 – 39 ]. Sample size can only be a useful quality indicator when related to the research purpose, the chosen methodology and the composition of the sample, i.e. who was included and why.

Randomisation

While some authors argue that randomisation can be used in qualitative research, this is not commonly the case, as neither its feasibility nor its necessity or usefulness has been convincingly established for qualitative research [ 13 , 27 ]. Relevant disadvantages include the negative impact of a too large sample size as well as the possibility (or probability) of selecting “ quiet, uncooperative or inarticulate individuals ” [ 17 ]. Qualitative studies do not use control groups, either.

Interrater reliability, variability and other “objectivity checks”

The concept of “interrater reliability” is sometimes used in qualitative research to assess to which extent the coding approach overlaps between the two co-coders. However, it is not clear what this measure tells us about the quality of the analysis [ 23 ]. This means that these scores can be included in qualitative research reports, preferably with some additional information on what the score means for the analysis, but it is not a requirement. Relatedly, it is not relevant for the quality or “objectivity” of qualitative research to separate those who recruited the study participants and collected and analysed the data. Experiences even show that it might be better to have the same person or team perform all of these tasks [ 20 ]. First, when researchers introduce themselves during recruitment this can enhance trust when the interview takes place days or weeks later with the same researcher. Second, when the audio-recording is transcribed for analysis, the researcher conducting the interviews will usually remember the interviewee and the specific interview situation during data analysis. This might be helpful in providing additional context information for interpretation of data, e.g. on whether something might have been meant as a joke [ 18 ].

Not being quantitative research

Being qualitative research instead of quantitative research should not be used as an assessment criterion if it is used irrespectively of the research problem at hand. Similarly, qualitative research should not be required to be combined with quantitative research per se – unless mixed methods research is judged as inherently better than single-method research. In this case, the same criterion should be applied for quantitative studies without a qualitative component.

The main take-away points of this paper are summarised in Table ​ Table1. 1 . We aimed to show that, if conducted well, qualitative research can answer specific research questions that cannot to be adequately answered using (only) quantitative designs. Seeing qualitative and quantitative methods as equal will help us become more aware and critical of the “fit” between the research problem and our chosen methods: I can conduct an RCT to determine the reasons for transportation delays of acute stroke patients – but should I? It also provides us with a greater range of tools to tackle a greater range of research problems more appropriately and successfully, filling in the blind spots on one half of the methodological spectrum to better address the whole complexity of neurological research and practice.

Take-away-points

Acknowledgements

Abbreviations, authors’ contributions.

LB drafted the manuscript; WW and CG revised the manuscript; all authors approved the final versions.

no external funding.

Availability of data and materials

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

An official website of the United States government

Here’s how you know

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock A locked padlock ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Researchers
• Survey Respondents
• Español / Spanish
• Census.gov /
• News Releases /
• Undercounts and Overcounts of Young Children in the 2020 Census

Census Bureau Releases Experimental Estimates of State and County Undercounts and Overcounts of Young Children in the 2020 Census

For Immediate Release: Thursday, April 11, 2024

Census bureau initiatives to address persistent undercount of children.

APRIL 11, 2024 – The U.S. Census Bureau today released new experimental estimates showing children ages 0 to 4 were undercounted in the 2020 Census in every state. Additionally, there were undercounts of children ages 0 to 4 in more than 4 out of 5 counties included in this release. The experimental estimates are available only for counties with a population of 1,000 or more children ages 0 to 4, and are based on the Census Bureau’s Demographic Analysis (DA) estimates .

The number of U.S. children ages 0 to 4 counted in the 2020 Census was previously found to be about 1 million lower than the benchmark population estimate — an undercount of 5.46% . This was a larger undercount than any other age group .

Today’s release of net coverage error rates for young children comes from the Census Bureau’s DA estimates. Instead of conducting a count based on responses collected or on behalf of each household like the 2020 Census, DA uses administrative records to estimate the size of the population and compares those estimates to census counts to assess the accuracy of the 2020 Census.

“The Census Bureau recognizes historical undercounts of young children in our decennial census as well as an under-representation in our demographic surveys. We know these undercounts are often correlated with undercounts of certain race and ethnicity groups along with other factors that we were not able to measure directly.   We are diligently working to address this issue,” Census Bureau Director Robert Santos said. “Our goal is to accurately count every child in the census and to ensure they are fully represented in our surveys. Quality statistics help communities better understand their needs and seek resources.”

DA is one of two methods the Census Bureau is using to assess coverage of the 2020 Census. The other one is the Post-Enumeration Survey (PES). DA uses current and historical birth and death records from the National Center for Health Statistics , data on international migration, and Medicare records to independently produce estimates of the U.S. population on April 1, 2020. Traditionally, DA estimates have been produced at the national level for the total population. Today’s release expands on the official approach by incorporating birth records from 2015 to 2020 and adjusting for migration between counties.

“These experimental estimates build upon previously released quality measurements to use new data and methods for young children in the 2020 Census,” said Eric Jensen, senior advisor for Population Estimates and Coverage Measurement in the Census Bureau’s Population Division. “They offer a unique glimpse into the geographic distribution of the coverage of young children in the 2020 Census and are critical in identifying areas where improvements are needed to more accurately count children in the 2030 Census.”

The net coverage error for young children varied by geography, race and Hispanic origin, household structure, and other demographic, social and economic characteristics. 

How the Census Bureau Is Improving Our Methods to Accurately Count Children

Today’s state and county estimates further highlight the undercount of young children in the 2020 Census despite major efforts by the Census Bureau to mitigate this problem. Understanding the geographic distribution of coverage error for young children will help the Census Bureau identify the underlying causes of the undercounts and develop strategies for improving data collection on young children in the 2030 Census and future surveys.

In 2022, the Census Bureau formed the Undercount of Young Children Working Group , focused on identifying underlying causes for the undercount of young children and improving data on this population. The working group is currently conducting research to improve the count for young children in the 2030 Census.

“We’re taking a broader approach to tackling the undercount this decade by focusing on research, data collection, data products and community outreach ,” said Jensen, who co-chairs the Census Bureau’s Undercount of Young Children Working Group. “Our focus is to improve overall data on young children.”

The working group is seeking to better understand why young children are historically undercounted , exploring how to improve data collection methods to accurately capture responses about children, integrating DA estimates with other data sources to improve Census Bureau data on young children, and actively engaging with stakeholders to create more effective strategies for outreach and inclusion. This includes engagement with government agencies, advocacy groups, and nonprofit organizations who advocate for young children in communities across the nation to ensure trust, awareness and accessibility. 

State Coverage

The 2020 DA state net coverage error estimates for young children ranged from -15.85% for the District of Columbia to -0.02% for Vermont.

“While the estimates show that the biggest undercount by state is D.C., it’s possible this value is inflated. For instance, many mothers who live in Maryland or Virginia may actually give birth in D.C. hospitals. Therefore, because of how birth records are used in the method, these children would be reflected in the DA estimate but counted in their actual home state,” Jensen explained. “With today’s release being experimental, there are various sources of uncertainty that we can’t fully address.”

Table 1 shows the 14 states and one state equivalent whose net coverage error of young children is greater than the national estimate — meaning they had a larger undercount than the nation as a whole. More than two-thirds of them were in the South: the District of Columbia, Florida, Texas, Mississippi, Delaware, Louisiana, North Carolina, Virginia, Georgia, South Carolina and Maryland — and the rest were in the West (Hawaii, California and Arizona) and Northeast (New York).  

Table 1. States With an Undercount of Children Ages 0-4 in the 2020 Census Greater Than the National Value

Note: The Demographic Analysis estimate for Maryland is rounded to the nearest thousand to adhere to disclosure avoidance guidelines.

Source: U.S. Census Bureau, 2020 Demographic Analysis (April 2024 release), 2020 Census Demographic and Housing Characteristics File (DHC), and 2020 Census special tabulation (for the District of Columbia only). 

View the news graphic for the net coverage error of all states.

County Coverage

DA estimates released today are limited to counties with a DA population of 1,000 or more children ages 0 to 4 due to the varying methods of uncertainty, especially for smaller counties.

Of these counties, one third had a net coverage error lower than the national value of -5.46% — meaning they had a larger undercount than the nation. Refer to the news graphic  for a county-level breakdown of net coverage error.

These counties are located throughout the United States with clusters along the West Coast, in the South, and in the Southwest with an emphasis on the border regions of California, Arizona and Texas.

“What stands out here is the large number of counties across the country with an undercount for young children in the 2020 Census,” Jensen said. “But we also see no coverage error or overcounts, specifically in about 16% of counties with a DA population of 1,000 or more, as compared to 84% with an undercount. Additionally, we observed that the counties with overcounts tend to be less spread out, and even see some clustering among these counties in states such as Nevada, New Mexico, New York and Utah.”  

Correlations Between Socioeconomic Characteristics and Undercounts of Children

The Census Bureau today also released an interactive data visualization showing the DA estimates alongside socioeconomic characteristics of counties.

“Children often live in situations where they are vulnerable to being undercounted," Jensen said. "There are many factors that come into play. For example, young children are more likely than other age groups to live in multigenerational households or households with nonrelatives. These complex living situations can be difficult to enumerate correctly.”

These data can help the public understand factors  that may lead to young children being missed in surveys and population counts.

“This information not only underscores our dedication to openness but also empowers stakeholders to engage in constructive dialogues aimed at refining the accuracy of future census endeavors,” Santos stressed. “As we navigate these complexities, we remain steadfast in our pursuit to count everyone.”

For more information about today’s release, visit the Demographic Analysis webpage and Demographic Analysis press kit . Also refer to the 2020 Census Data Quality webpage for more information on how the Census Bureau measures quality. 

The Census Bureau also today released the Vintage 2023 U.S. population estimates by age and sex . 

Public Information Office 301-763-3030 or 877-861-2010 (U.S. and Canada only) [email protected]

Related Information

2024 IFD&TC Spotlights Westat Field Data Collection

April 15, 2024

Westat researchers and field directors will present on techniques that seek to improve data quality, increase inclusivity in response rates, enhance interviewer job satisfactions, and more at the 57th Annual International Field Directors and Technologies Conference (IFD&TC) .

The IFD&TC brings together field directors, field technicians, and survey managers from government agencies, academic institutions, and other corporate entities to present ideas and strategies regarding field data collection. This year’s conference will be held April 21-23 in Cleveland, Ohio.

The following Westat staff (names in bold) will provide presentations on submitted papers and featured on panels with experts from other agencies. The asterisk (*) indicates presenting authors.

Presentations

Victor Barajas *, Victoria Vignare , Tania Johnson , and Amanda Hall . Is It Time to Change Our Outreach Approach? Examining Hispanic Response Rates and Strategies to Gaining Their Cooperation. ¿Será Tiempo de Cambiar Nuestra Estrategia de Difusión? Examinando la Taza de Respuesta de los Hispanos y Estrategias Para Obtener Mejor Participación.

Victoria Vignare*, Tammy Cook, Erika Sofelkanik, Susan Genoversa . Biggest Bang for Your Buck: Local Field Data Collectors vs. Travelers?

Grant Benson, Kurt Johnson, Mary Catherine Potter, Tammy Cook* , and Kyle Fennell. Interviewer Attrition (Part 1 & 2).

Capabilities

Deep Dive with Our Experts

Washington State Department of Corrections

Agency for Healthcare Research and Quality (AHRQ)

Patient-Centered Outcomes Research Institute (PCORI)

How can we help?

We welcome messages from job seekers, collaborators, and potential clients and partners.

Want to work with us?

You’ll be in great company.

• Privacy Overview
• Strictly Necessary Cookies
• 3rd Party Cookies
• Additional Cookies

This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognizing you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.

Strictly Necessary Cookie should be enabled at all times so that we can save your preferences for cookie settings.

This website uses Google Analytics to collect anonymous information such as the number of visitors to the site, and the most popular pages.

Keeping this cookie enabled helps us to improve our website.

Please enable Strictly Necessary Cookies first so that we can save your preferences!

This website uses the following additional cookies:

• Google Analytics
• Google Tag Manager
• Google Search Console
• Google Sitekit

Search code, repositories, users, issues, pull requests...

Provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications

A collection of guides and examples for the Gemini API.

google-gemini/cookbook

Folders and files, repository files navigation, welcome to the gemini api cookbook.

This is a collection of guides and examples for the Gemini API, including quickstart tutorials for writing prompts and using different features of the API, and examples of things you can build.

Get started with the Gemini API

The Gemini API gives you access to Gemini models created by Google DeepMind . Gemini models are built from the ground up to be multimodal, so you can reason seamlessly across text, images, code, and audio. You can use these to develop a range of applications .

Start developing

• Go to Google AI Studio .
• Login with your Google account.
• Create an API key.
• Use a quickstart for Python, or call the REST API using curl .

Capabilities

Learn about the capabilities of the Gemini API by checking out the quickstarts for safety , embeddings , function calling , audio , and more.

Official SDKs

The Gemini API is a REST API. You can call the API using a command line tool like curl , or by using one of our official SDKs:

• Dart (Flutter)

Open an issue on GitHub.

Contributing

Contributions are welcome. See contributing to learn more.

Thank you for developing with the Gemini API! We’re excited to see what you create.

Contributors 10

• Jupyter Notebook 99.9%