measurement in research methodology

Popular searches

• How to Get Participants For Your Study
• How to Do Segmentation?
• Conjoint Preference Share Simulator
• MaxDiff Analysis
• Likert Scales
• Reliability & Validity

Request consultation

Do you need support in running a pricing or product study? We can help you with agile consumer research and conjoint analysis.

Looking for an online survey platform?

Conjointly offers a great survey tool with multiple question types, randomisation blocks, and multilingual support. The Basic tier is always free.

Research Methods Knowledge Base

• Navigating the Knowledge Base
• Foundations
• Construct Validity
• Reliability
• Levels of Measurement
• Survey Research
• Scaling in Measurement
• Qualitative Measures
• Unobtrusive Measures
• Research Design
• Table of Contents

Fully-functional online survey tool with various question types, logic, randomisation, and reporting for unlimited number of surveys.

Completely free for academics and students .

Measurement

Measurement is the process of observing and recording the observations that are collected as part of a research effort. There are two major issues that will be considered here.

First, you have to understand the fundamental ideas involved in measuring. Here we consider two of major measurement concepts. In Levels of Measurement , I explain the meaning of the four major levels of measurement: nominal, ordinal, interval and ratio. Then we move on to the reliability of measurement, including consideration of true score theory and a variety of reliability estimators.

Second, you have to understand the different types of measures that you might use in social research. We consider four broad categories of measurements. Survey research includes the design and implementation of interviews and questionnaires. Scaling involves consideration of the major methods of developing and implementing a scale. Qualitative research provides an overview of the broad range of non-numerical measurement approaches. And unobtrusive measures presents a variety of measurement methods that don’t intrude on or interfere with the context of the research.

Cookie Consent

Conjointly uses essential cookies to make our site work. We also use additional cookies in order to understand the usage of the site, gather audience analytics, and for remarketing purposes.

For more information on Conjointly's use of cookies, please read our Cookie Policy .

Which one are you?

I am new to conjointly, i am already using conjointly.

• school Campus Bookshelves
• menu_book Bookshelves
• perm_media Learning Objects
• login Login
• how_to_reg Request Instructor Account
• hub Instructor Commons
• Download Page (PDF)
• Download Full Book (PDF)
• Periodic Table
• Physics Constants
• Scientific Calculator
• Reference & Cite
• Tools expand_more
• Readability

selected template will load here

This action is not available.

4.1: What is Measurement?

• Last updated
• Save as PDF
• Page ID 124458

Learning Objective

• Define measurement.

Measurement

Measurement is important. Recognizing that fact, and respecting it, will be of great benefit to you—both in research methods and in other areas of life as well. If, for example, you have ever baked a cake, you know well the importance of measurement. As someone who much prefers rebelling against precise rules over following them, I once learned the hard way that measurement matters. A couple of years ago I attempted to bake my husband a birthday cake without the help of any measuring utensils. I’d baked before, I reasoned, and I had a pretty good sense of the difference between a cup and a tablespoon. How hard could it be? As it turns out, it’s not easy guesstimating precise measures. That cake was the lumpiest, most lopsided cake I’ve ever seen. And it tasted kind of like Play-Doh. Figure 4.1 depicts the monstrosity I created, all because I did not respect the value of measurement.

Measurement is important in baking and in research.

Just as measurement is critical to successful baking, it is as important to successfully pulling off a social scientific research project. In sociology, when we use the term measurement we mean the process by which we describe and ascribe meaning to the key facts, concepts, or other phenomena that we are investigating. At its core, measurement is about defining one’s terms in as clear and precise a way as possible. Of course, measurement in social science isn’t quite as simple as using some predetermined or universally agreed-on tool, such as a measuring cup or spoon, but there are some basic tenants on which most social scientists agree when it comes to measurement. We’ll explore those as well as some of the ways that measurement might vary depending on your unique approach to the study of your topic.

What Do Social Scientists Measure?

The question of what social scientists measure can be answered by asking oneself what social scientists study. Think about the topics you’ve learned about in other sociology classes you’ve taken or the topics you’ve considered investigating yourself. Or think about the many examples of research you’ve read about in this text. Classroom learning environments and the mental health of first grade children. Journal of Health and Social Behavior, 52 , 4–22. of first graders’ mental health. In order to conduct that study, Milkie and Warner needed to have some idea about how they were going to measure mental health. What does mental health mean, exactly? And how do we know when we’re observing someone whose mental health is good and when we see someone whose mental health is compromised? Understanding how measurement works in research methods helps us answer these sorts of questions.

As you might have guessed, social scientists will measure just about anything that they have an interest in investigating. For example, those who are interested in learning something about the correlation between social class and levels of happiness must develop some way to measure both social class and happiness. Those who wish to understand how well immigrants cope in their new locations must measure immigrant status and coping. Those who wish to understand how a person’s gender shapes their workplace experiences must measure gender and workplace experiences. You get the idea. Social scientists can and do measure just about anything you can imagine observing or wanting to study.

How Do Social Scientists Measure?

Measurement in social science is a process. It occurs at multiple stages of a research project: in the planning stages, in the data collection stage, and sometimes even in the analysis stage. Recall that previously we defined measurement as the process by which we describe and ascribe meaning to the key facts, concepts, or other phenomena that we are investigating. Once we’ve identified a research question, we begin to think about what some of the key ideas are that we hope to learn from our project. In describing those key ideas, we begin the measurement process.

Let’s say that our research question is the following: How do new college students cope with the adjustment to college? In order to answer this question, we’ll need to some idea about what coping means. We may come up with an idea about what coping means early in the research process, as we begin to think about what to look for (or observe) in our data-collection phase. Once we’ve collected data on coping, we also have to decide how to report on the topic. Perhaps, for example, there are different types or dimensions of coping, some of which lead to more successful adjustment than others. However we decide to proceed, and whatever we decide to report, the point is that measurement is important at each of these phases.

As the preceding paragraph demonstrates, measurement is a process in part because it occurs at multiple stages of conducting research. We could also think of measurement as a process because of the fact that measurement in itself involves multiple stages. From identifying one’s key terms to defining them to figuring out how to observe them and how to know if our observations are any good, there are multiple steps involved in the measurement process. An additional step in the measurement process involves deciding what elements one’s measures contain. A measure’s elements might be very straightforward and clear, particularly if they are directly observable. Other measures are more complex and might require the researcher to account for different themes or types. These sorts of complexities require paying careful attention to a concept’s level of measurement and its dimensions.

KEY TAKEAWAYS

• Measurement is the process by which we describe and ascribe meaning to the key facts, concepts, or other phenomena that we are investigating.
• Measurement occurs at all stages of research.

Handbook of Behavior Therapy in Education pp 37–65 Cite as

Research Methodology and Measurement

• Frank M. Gresham 3 &
• Michael P. Carey 3  

392 Accesses

3 Citations

Scientific research refers to controlled, systematic, empirical, and critical investigation of natural phenomena that is guided by hypotheses and theory about supposed relations between such phenomena (Kerlinger, 1986). The method of science represents a method of knowing that is unique in that it possesses a self-correcting feature that verifies of disconfirms formally stated predictions (i.e., hypotheses) about natural phenomena. Cohen and Nagel (1934) identified three additional methods of knowing that are diametrically opposed to science: (a) the method of tenacity, (b) the method of authority, and (c) the method of intuition.

• Latent Trait
• Behavioral Assessment
• Error Score
• Apply Behavior Analysis

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, log in via an institution .

Buying options

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Unable to display preview.  Download preview PDF.

Achenbach, T., & Edelbrock, C. (1983). Manual for the Child Behavior Profile . Burlington, VT: University of Vermont.

Google Scholar  

Achinstein, P. (1968). Concepts of science: A philosophical analysis . Baltimore, MD: The John Hopkins Press.

Albee, G. (1970). The uncertain future of clinical psychology. American Psychologist, 25 , 1071–1080.

Article   PubMed   Google Scholar  

Arter, J. A., Jenkins, J. R. (1977). Examining the benefits and prevalence of modality considerations in special education. Journal of Special Education, 11 , 281–298.

Article   Google Scholar  

Atkinson, R. C., Shiffrin, R. M. (1968). Human memory: A proposed system and its control processes. In K. W. Spence, J. T. Spence (Eds.), The psychology of learning and motivation (2nd ed.). New York: Academic Press.

Barlow, D. H. (1980). Behavior therapy: The next decade. Behavior Therapy, 11 , 315–328.

Barlow, D., Hersen, M. (1984). Single case experimental designs (2nd ed.). New York: Pergamon Press.

Barlow, D., Hayes, S., Nelson, R. (1984). The scientist practitioner . New York: Pergamon Press.

Becker, W. C., & Carnine, D. W. (1980). Direct instruction: An effective approach to educational intervention with the disadvantaged and low performers. In B. B. Lahey, A. E. & Kazdin (Eds.), Advances in clinical child psychology (pp. 429–473). New York: Academic Press.

Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37 , 29–51.

Brown, F. G. (1983). Principles of educational and psychological measurement (3rd ed.). New York: Holt, Rinehart, Winston.

Burns, L. (1980). Indirect measurement and behavioral assessment: A case for social behaviorism psycho-metrics. Behavioral Assessment, 2 , 197–206.

Campbell, D. T., Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56 , 81–105.

Cohen, J., Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.

Cohen, M., Nagel, E. (1934). An introduction to logic and scientific method . New York: Harcourt.

Cone, J. D. (1977). The relevance of reliability and validity for behavioral assessment. Behavior Therapy, 8 ,411–426.

Cone, J. D. (1978). The behavioral assessment grid (BAG): A conceptual framework and taxonomy. Behavior Therapy, 9 , 882–888.

Cone, J. (1979). Confounded comparisons in triple response mode assessment. Behavioral Assessment, 1 , 85–95.

Cone, J. (1981). Psychometric considerations. In M. Hersen, A. Bellack (Eds.), Behavioral assessment: A practical handbook (38–70). New York: Pergamon Press.

Crocker, L., Algina, J. (1986). Introduction to classical and modern test theory . New York: Holt, Rinehart, Winston.

Cronbach, L. J., Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52 , 281–302.

Cronbach, L. J., Snow, R. (1977). Aptitude and instructional methods: A handbook for research on interactions . New York: Irvington.

Cronbach, L. J., Gleser, G. C., Nanda, H., Rajaratnam, N. (1972). The dependability of behavioral measurements . New York: Wiley.

Garfield, L. L., Kurtz, R. M. (1976). Clinical psychologists in the 1970s. American Psychologist, 31 , 1–9.

Ghiselli, E., Campbell, J., Zedeck, S. (1981). Measurement theory for the behavioral sciences . San Francisco: W. H. Freeman.

Gresham, F. M. (1982). A model for the behavioral assessment of behavior disorders in children: Measurement considerations and practical application. Journal of School Psychology, 20 , 131–143.

Guilford, J. P. (1959). The three faces of intellect. American Psychologist, 14 , 469.

Guion, R. M., Ironson, G. H. (1983). Latent trait theory for organizational research. Organizational Behavior and Human Performance, 31 , 54–87.

Harris, A. (1980). Response class: A Guttman scale analysis. Journal of Abnormal Child Psychology, 8 , 213–220.

Hambleton, R. K., Cook, L. L. (1977). Latent trait models and their use in the analysis of educational test data. Journal of Educational Measurement, 14 , 75–96.

Hayes, S. C. (1981). Single case experimental design and empirical clinical practice. Journal of Consulting and Clinical Psychology, 49 , 193–211.

Hays, W. (1985). Statistics (3rd ed). New York: Holt.

Hersen, M., Bellack, A. (Eds.). (1981). Behavioral assessment: A practical handbook (2nd ed.). New York: Pergamon Press.

Jackson, D. N. (1969). Multimethod factor analysis in the evaluation of convergent and discriminant validity. Psychological Bulletin, 29 , 259–271.

Jones, R. R., Reid, J. B., Patterson, G. R. (1975). Naturalistic observation in clinical assessment. In P. McReynolds (Ed.), Advances in psychological assessment (Vol. 3). San Francisco: Jossey-Bass.

Johnston, J. M., Pennypacker, H. S. (1980). Strategies and tactics of human behavioral research . Hillsdale, NJ: Erlbaum.

Jorkeskog, K. G., Sorbom, D. (1983). LISREL VI: Analysis of linear structural relationships by maximum likelihood and least squares methods (2nd ed.). Chicago: Natural Education Resources.

Kaufman, A., Kaufman, N. (1983). K-ABC: Kaufman Assessment Battery for Children: Interpretive manual . Circle Pines, MN: American Guidance Service.

Kazdin, A. E. (1977). Assessing the clinical or applied importance of behavior change through social validation. Behavior Modification, 1 , 427–452.

Kazdin, A. E. (1979). Situational-specificity: The two-edged sword of behavioral assessment. Behavioral Assessment, 1 , 57–75.

Kelly, E. L., Goldberg, L. R., Fiske, D. W., Kilkowski, J. (1978). Twenty-five years later: A follow-up study of the graduate students in clinical psychology assessed in the VA Selection Research Project. American Psychologist, 33 , 746–755.

Kerlinger, F. (1986). Foundations of behavioral research (3rd ed.). New York: Holt, Rinehart, Winston.

Kratochwill, T., Mace, F. C., Mott, S. (1985). Research methods from applied behavior analysis. In C. R. Reynolds, V. L. Willson (Eds.), Methodological and statistical advances in the study of individual differences (pp. 335–392). New York: Plenum Press.

Lichstein, K. L., Wahler, R. G. (1976). The ecological assessment of an autistic child. Journal of Abnormal Child Psychology, 4 , 31–54.

Linehan, M. M. (1980). Content validity: Its relevance to behavioral assessment. Behavioral Assessment, 2 , 147–159.

Lord, F. M. (1952). The relationship of the reliability of multiple choice items to the distribution of item difficulties. Psychometrika, 18 , 181–194.

Lord, F. M., Novick, M. R. (1968). Statistical theories of mental test scores . Reading, MA: Addison-Wesley.

Martens, B. K., Keller, H. R. (1987). Training school psychologists in the scientific tradition. School Psychology Review, 16 , 329–337.

Mash, E. J., Terdal, L. G. (Eds.) (1981). Behavioral assessment of childhood disorders . New York: Guilford Press.

Meitner, L., Frisch, O. R. (1939). Disintegration of uranium by neutrons: A new type of nuclear reaction. Nature, 143 , 239.

Mercer, J. (1979). In defense of racially and culturally non-discriminatory assessment. School Psychology Digest, 8 , 89–115.

Mischel, W. (1968). Personality and assessment . New York: Wiley.

Nelson, R. O. (1983). Behavioral assessment: Past, present, and future. Behavioral Assessment, 5 , 195–206.

Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.

Patterson, G. R. (1986). Performance models for antisocial boys. American Psychologist, 41 , 432–444.

Patterson, G. R., Bank, L. (1986). Bootstrapping your way in the nomological thicket. Behavioral Assessment, 8 , 49–73.

Peterson, D. R. (1976). Need for the Doctor of Psychology degree in professional psychology. American Psychologist, 31 , 792–798.

Reynolds, C. R. (1982). The problem of bias in psychological assessment. In C. Reynolds, T. Gutkin (Eds.), Handbook of school psychology (pp. 178–208). New York: Wiley.

Samejima, F. (1973). Homogenous case of the continuous response model. Psychometrika, 38 , 203–219.

Schmitt, N., Coyle, B. W., Saari, B. B. (1977). A review and critique of analyses of multitrait-multimethod matrices. Multivariate Behavioral Research, 12 , 447–478.

Skinner, B. F. (1974). About behaviorism . New York: Alfred A. Knopf.

Spearman, C. (1904). The proof and measurement of association between two things. American Journal of Psychology, 15 , 72–101.

Staats, A. (1981). Paradigmatic behaviorism, unified theory construction methods, and the Zeitgeist of separtism. American Psychologist, 36 , 239–256.

Stokes, T., Baer, D. (1977). An implicit technology of generalization. Journal of Applied Behavior Analysis, 10 , 349–367.

Strain, P. S., Ezzell, D. (1978). The sequence and distribution of behavioral disordered adolescents’ disruptive/inappropriate behaviors: An observational study in a residential setting. Behavior Modification, 2 ,403–425.

Tabachnick, B. G., Fidell, L. S. (1983). Using multivariate statistics . New York:Harper & Row.

Tatusoka, M. M. (1971). Multivariate analysis: Techniques for educational and psychological research . New York: Wiley.

Tryon, R. C. (1957). Reliability and behavior domain validity: Reformulation and historical critique. Psychological Bulletin, 54 , 229–249.

Voeltz, L. M., Evans, I. M. (1982). The assessment of behavioral interrelationships in child behavior therapy. Behavioral Assessment, 4 , 131–165.

Wahler, R. G. (1975). Some structural aspects of deviant child behavior. Journal of Applied Behavior Analysis, 8 , 27–42.

Walker, H. M., Hops, H. (1976). Use of normative peer data as a standard for evaluating classroom treatment effects. Journal of Applied Behavior Analysis, 9 , 159–168.

Wang, M., Stanley, J. (1970). Differential weighting: A review of methods and empirical studies. Review of Educational Research, 40 , 663–705.

Wolf, M. M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11 ,203–214.

Wolfe, V. V., Cone, J. D., Wolfe, D. A. (1986). Social and solipsistic observer training: Effects on agreement with a criterion. Journal of Psychopathology and Behavioral Assessment, 8 , 211–226.

Yeaton, W. H., Sechrest, L. (1981). Critical dimensions in the choice and maintenance of successful treatments: Strength, integrity, and effectiveness. Journal of Consulting and Clinical Psychology, 49 , 156–167.

Download references

Author information

Authors and affiliations.

Department of Psychology, Louisiana State University, Baton Rouge, Louisiana, 70803, USA

Frank M. Gresham & Michael P. Carey

You can also search for this author in PubMed   Google Scholar

Editor information

Editors and affiliations.

Department of Psychology, Louisiana State University, Baton Rouge, Louisiana, USA

Joseph C. Witt  & Frank M. Gresham  & 

Department of Educational Psychology, University of Wisconsin-Madison, Madison, Wisconsin, USA

Stephen N. Elliot

Rights and permissions

Reprints and permissions

Copyright information

© 1988 Plenum Press, New York

About this chapter

Cite this chapter.

Gresham, F.M., Carey, M.P. (1988). Research Methodology and Measurement. In: Witt, J.C., Elliot, S.N., Gresham, F.M. (eds) Handbook of Behavior Therapy in Education. Springer, Boston, MA. https://doi.org/10.1007/978-1-4613-0905-5_2

Download citation

DOI : https://doi.org/10.1007/978-1-4613-0905-5_2

Publisher Name : Springer, Boston, MA

Print ISBN : 978-1-4612-8238-9

Online ISBN : 978-1-4613-0905-5

eBook Packages : Springer Book Archive

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 5: Psychological Measurement

Reliability and Validity of Measurement

Learning Objectives

• Define reliability, including the different types and how they are assessed.
• Define validity, including the different types and how they are assessed.
• Describe the kinds of evidence that would be relevant to assessing the reliability and validity of a particular measure.

Again, measurement involves assigning scores to individuals so that they represent some characteristic of the individuals. But how do researchers know that the scores actually represent the characteristic, especially when it is a construct like intelligence, self-esteem, depression, or working memory capacity? The answer is that they conduct research using the measure to confirm that the scores make sense based on their understanding of the construct being measured. This is an extremely important point. Psychologists do not simply  assume  that their measures work. Instead, they collect data to demonstrate  that they work. If their research does not demonstrate that a measure works, they stop using it.

As an informal example, imagine that you have been dieting for a month. Your clothes seem to be fitting more loosely, and several friends have asked if you have lost weight. If at this point your bathroom scale indicated that you had lost 10 pounds, this would make sense and you would continue to use the scale. But if it indicated that you had gained 10 pounds, you would rightly conclude that it was broken and either fix it or get rid of it. In evaluating a measurement method, psychologists consider two general dimensions: reliability and validity.

Reliability

Reliability  refers to the consistency of a measure. Psychologists consider three types of consistency: over time (test-retest reliability), across items (internal consistency), and across different researchers (inter-rater reliability).

Test-Retest Reliability

When researchers measure a construct that they assume to be consistent across time, then the scores they obtain should also be consistent across time.  Test-retest reliability  is the extent to which this is actually the case. For example, intelligence is generally thought to be consistent across time. A person who is highly intelligent today will be highly intelligent next week. This means that any good measure of intelligence should produce roughly the same scores for this individual next week as it does today. Clearly, a measure that produces highly inconsistent scores over time cannot be a very good measure of a construct that is supposed to be consistent.

Assessing test-retest reliability requires using the measure on a group of people at one time, using it again on the  same  group of people at a later time, and then looking at  test-retest correlation  between the two sets of scores. This is typically done by graphing the data in a scatterplot and computing Pearson’s  r . Figure 5.2 shows the correlation between two sets of scores of several university students on the Rosenberg Self-Esteem Scale, administered two times, a week apart. Pearson’s r for these data is +.95. In general, a test-retest correlation of +.80 or greater is considered to indicate good reliability.

Again, high test-retest correlations make sense when the construct being measured is assumed to be consistent over time, which is the case for intelligence, self-esteem, and the Big Five personality dimensions. But other constructs are not assumed to be stable over time. The very nature of mood, for example, is that it changes. So a measure of mood that produced a low test-retest correlation over a period of a month would not be a cause for concern.

Internal Consistency

A second kind of reliability is  internal consistency , which is the consistency of people’s responses across the items on a multiple-item measure. In general, all the items on such measures are supposed to reflect the same underlying construct, so people’s scores on those items should be correlated with each other. On the Rosenberg Self-Esteem Scale, people who agree that they are a person of worth should tend to agree that that they have a number of good qualities. If people’s responses to the different items are not correlated with each other, then it would no longer make sense to claim that they are all measuring the same underlying construct. This is as true for behavioural and physiological measures as for self-report measures. For example, people might make a series of bets in a simulated game of roulette as a measure of their level of risk seeking. This measure would be internally consistent to the extent that individual participants’ bets were consistently high or low across trials.

Like test-retest reliability, internal consistency can only be assessed by collecting and analyzing data. One approach is to look at a  split-half correlation . This involves splitting the items into two sets, such as the first and second halves of the items or the even- and odd-numbered items. Then a score is computed for each set of items, and the relationship between the two sets of scores is examined. For example, Figure 5.3 shows the split-half correlation between several university students’ scores on the even-numbered items and their scores on the odd-numbered items of the Rosenberg Self-Esteem Scale. Pearson’s  r  for these data is +.88. A split-half correlation of +.80 or greater is generally considered good internal consistency.

Perhaps the most common measure of internal consistency used by researchers in psychology is a statistic called  Cronbach’s α  (the Greek letter alpha). Conceptually, α is the mean of all possible split-half correlations for a set of items. For example, there are 252 ways to split a set of 10 items into two sets of five. Cronbach’s α would be the mean of the 252 split-half correlations. Note that this is not how α is actually computed, but it is a correct way of interpreting the meaning of this statistic. Again, a value of +.80 or greater is generally taken to indicate good internal consistency.

Interrater Reliability

Many behavioural measures involve significant judgment on the part of an observer or a rater.  Inter-rater reliability  is the extent to which different observers are consistent in their judgments. For example, if you were interested in measuring university students’ social skills, you could make video recordings of them as they interacted with another student whom they are meeting for the first time. Then you could have two or more observers watch the videos and rate each student’s level of social skills. To the extent that each participant does in fact have some level of social skills that can be detected by an attentive observer, different observers’ ratings should be highly correlated with each other. Inter-rater reliability would also have been measured in Bandura’s Bobo doll study. In this case, the observers’ ratings of how many acts of aggression a particular child committed while playing with the Bobo doll should have been highly positively correlated. Interrater reliability is often assessed using Cronbach’s α when the judgments are quantitative or an analogous statistic called Cohen’s κ (the Greek letter kappa) when they are categorical.

Validity  is the extent to which the scores from a measure represent the variable they are intended to. But how do researchers make this judgment? We have already considered one factor that they take into account—reliability. When a measure has good test-retest reliability and internal consistency, researchers should be more confident that the scores represent what they are supposed to. There has to be more to it, however, because a measure can be extremely reliable but have no validity whatsoever. As an absurd example, imagine someone who believes that people’s index finger length reflects their self-esteem and therefore tries to measure self-esteem by holding a ruler up to people’s index fingers. Although this measure would have extremely good test-retest reliability, it would have absolutely no validity. The fact that one person’s index finger is a centimetre longer than another’s would indicate nothing about which one had higher self-esteem.

Discussions of validity usually divide it into several distinct “types.” But a good way to interpret these types is that they are other kinds of evidence—in addition to reliability—that should be taken into account when judging the validity of a measure. Here we consider three basic kinds: face validity, content validity, and criterion validity.

Face Validity

Face validity  is the extent to which a measurement method appears “on its face” to measure the construct of interest. Most people would expect a self-esteem questionnaire to include items about whether they see themselves as a person of worth and whether they think they have good qualities. So a questionnaire that included these kinds of items would have good face validity. The finger-length method of measuring self-esteem, on the other hand, seems to have nothing to do with self-esteem and therefore has poor face validity. Although face validity can be assessed quantitatively—for example, by having a large sample of people rate a measure in terms of whether it appears to measure what it is intended to—it is usually assessed informally.

Face validity is at best a very weak kind of evidence that a measurement method is measuring what it is supposed to. One reason is that it is based on people’s intuitions about human behaviour, which are frequently wrong. It is also the case that many established measures in psychology work quite well despite lacking face validity. The Minnesota Multiphasic Personality Inventory-2 (MMPI-2) measures many personality characteristics and disorders by having people decide whether each of over 567 different statements applies to them—where many of the statements do not have any obvious relationship to the construct that they measure. For example, the items “I enjoy detective or mystery stories” and “The sight of blood doesn’t frighten me or make me sick” both measure the suppression of aggression. In this case, it is not the participants’ literal answers to these questions that are of interest, but rather whether the pattern of the participants’ responses to a series of questions matches those of individuals who tend to suppress their aggression.

Content Validity

Content validity  is the extent to which a measure “covers” the construct of interest. For example, if a researcher conceptually defines test anxiety as involving both sympathetic nervous system activation (leading to nervous feelings) and negative thoughts, then his measure of test anxiety should include items about both nervous feelings and negative thoughts. Or consider that attitudes are usually defined as involving thoughts, feelings, and actions toward something. By this conceptual definition, a person has a positive attitude toward exercise to the extent that he or she thinks positive thoughts about exercising, feels good about exercising, and actually exercises. So to have good content validity, a measure of people’s attitudes toward exercise would have to reflect all three of these aspects. Like face validity, content validity is not usually assessed quantitatively. Instead, it is assessed by carefully checking the measurement method against the conceptual definition of the construct.

Criterion Validity

Criterion validity  is the extent to which people’s scores on a measure are correlated with other variables (known as  criteria ) that one would expect them to be correlated with. For example, people’s scores on a new measure of test anxiety should be negatively correlated with their performance on an important school exam. If it were found that people’s scores were in fact negatively correlated with their exam performance, then this would be a piece of evidence that these scores really represent people’s test anxiety. But if it were found that people scored equally well on the exam regardless of their test anxiety scores, then this would cast doubt on the validity of the measure.

A criterion can be any variable that one has reason to think should be correlated with the construct being measured, and there will usually be many of them. For example, one would expect test anxiety scores to be negatively correlated with exam performance and course grades and positively correlated with general anxiety and with blood pressure during an exam. Or imagine that a researcher develops a new measure of physical risk taking. People’s scores on this measure should be correlated with their participation in “extreme” activities such as snowboarding and rock climbing, the number of speeding tickets they have received, and even the number of broken bones they have had over the years. When the criterion is measured at the same time as the construct, criterion validity is referred to as concurrent validity ; however, when the criterion is measured at some point in the future (after the construct has been measured), it is referred to as predictive validity (because scores on the measure have “predicted” a future outcome).

Criteria can also include other measures of the same construct. For example, one would expect new measures of test anxiety or physical risk taking to be positively correlated with existing measures of the same constructs. This is known as convergent validity .

Assessing convergent validity requires collecting data using the measure. Researchers John Cacioppo and Richard Petty did this when they created their self-report Need for Cognition Scale to measure how much people value and engage in thinking (Cacioppo & Petty, 1982) [1] . In a series of studies, they showed that people’s scores were positively correlated with their scores on a standardized academic achievement test, and that their scores were negatively correlated with their scores on a measure of dogmatism (which represents a tendency toward obedience). In the years since it was created, the Need for Cognition Scale has been used in literally hundreds of studies and has been shown to be correlated with a wide variety of other variables, including the effectiveness of an advertisement, interest in politics, and juror decisions (Petty, Briñol, Loersch, & McCaslin, 2009) [2] .

Discriminant Validity

Discriminant validity , on the other hand, is the extent to which scores on a measure are not correlated with measures of variables that are conceptually distinct. For example, self-esteem is a general attitude toward the self that is fairly stable over time. It is not the same as mood, which is how good or bad one happens to be feeling right now. So people’s scores on a new measure of self-esteem should not be very highly correlated with their moods. If the new measure of self-esteem were highly correlated with a measure of mood, it could be argued that the new measure is not really measuring self-esteem; it is measuring mood instead.

When they created the Need for Cognition Scale, Cacioppo and Petty also provided evidence of discriminant validity by showing that people’s scores were not correlated with certain other variables. For example, they found only a weak correlation between people’s need for cognition and a measure of their cognitive style—the extent to which they tend to think analytically by breaking ideas into smaller parts or holistically in terms of “the big picture.” They also found no correlation between people’s need for cognition and measures of their test anxiety and their tendency to respond in socially desirable ways. All these low correlations provide evidence that the measure is reflecting a conceptually distinct construct.

Key Takeaways

• Psychological researchers do not simply assume that their measures work. Instead, they conduct research to show that they work. If they cannot show that they work, they stop using them.
• There are two distinct criteria by which researchers evaluate their measures: reliability and validity. Reliability is consistency across time (test-retest reliability), across items (internal consistency), and across researchers (interrater reliability). Validity is the extent to which the scores actually represent the variable they are intended to.
• Validity is a judgment based on various types of evidence. The relevant evidence includes the measure’s reliability, whether it covers the construct of interest, and whether the scores it produces are correlated with other variables they are expected to be correlated with and not correlated with variables that are conceptually distinct.
• The reliability and validity of a measure is not established by any single study but by the pattern of results across multiple studies. The assessment of reliability and validity is an ongoing process.
• Practice: Ask several friends to complete the Rosenberg Self-Esteem Scale. Then assess its internal consistency by making a scatterplot to show the split-half correlation (even- vs. odd-numbered items). Compute Pearson’s  r too if you know how.
• Discussion: Think back to the last college exam you took and think of the exam as a psychological measure. What construct do you think it was intended to measure? Comment on its face and content validity. What data could you collect to assess its reliability and criterion validity?
• Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42 , 116–131. ↵
• Petty, R. E, Briñol, P., Loersch, C., & McCaslin, M. J. (2009). The need for cognition. In M. R. Leary & R. H. Hoyle (Eds.), Handbook of individual differences in social behaviour (pp. 318–329). New York, NY: Guilford Press. ↵

The consistency of a measure.

The consistency of a measure over time.

The consistency of a measure on the same group of people at different times.

Consistency of people’s responses across the items on a multiple-item measure.

Method of assessing internal consistency through splitting the items into two sets and examining the relationship between them.

A statistic in which α is the mean of all possible split-half correlations for a set of items.

The extent to which different observers are consistent in their judgments.

The extent to which the scores from a measure represent the variable they are intended to.

The extent to which a measurement method appears to measure the construct of interest.

The extent to which a measure “covers” the construct of interest.

The extent to which people’s scores on a measure are correlated with other variables that one would expect them to be correlated with.

In reference to criterion validity, variables that one would expect to be correlated with the measure.

When the criterion is measured at the same time as the construct.

when the criterion is measured at some point in the future (after the construct has been measured).

When new measures positively correlate with existing measures of the same constructs.

The extent to which scores on a measure are not correlated with measures of variables that are conceptually distinct.

Research Methods in Psychology - 2nd Canadian Edition Copyright © 2015 by Paul C. Price, Rajiv Jhangiani, & I-Chant A. Chiang is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Measurements in quantitative research: how to select and report on research instruments

Affiliation.

• 1 Department of Acute and Tertiary Care in the School of Nursing, University of Pittsburgh in Pennsylvania.
• PMID: 24969252
• DOI: 10.1188/14.ONF.431-433

Measures exist to numerically represent degrees of attributes. Quantitative research is based on measurement and is conducted in a systematic, controlled manner. These measures enable researchers to perform statistical tests, analyze differences between groups, and determine the effectiveness of treatments. If something is not measurable, it cannot be tested.

Keywords: measurements; quantitative research; reliability; validity.

• Clinical Nursing Research / methods*
• Clinical Nursing Research / standards
• Fatigue / nursing*
• Neoplasms / nursing*
• Oncology Nursing*
• Quality of Life*
• Reproducibility of Results

What Is Research Methodology? A Plain-Language Explanation & Definition (With Examples)

By Derek Jansen (MBA)  and Kerryn Warren (PhD) | June 2020 (Last updated April 2023)

If you’re new to formal academic research, it’s quite likely that you’re feeling a little overwhelmed by all the technical lingo that gets thrown around. And who could blame you – “research methodology”, “research methods”, “sampling strategies”… it all seems never-ending!

In this post, we’ll demystify the landscape with plain-language explanations and loads of examples (including easy-to-follow videos), so that you can approach your dissertation, thesis or research project with confidence. Let’s get started.

Research Methodology 101

• What exactly research methodology means
• What qualitative , quantitative and mixed methods are
• What sampling strategy is
• What data collection methods are
• What data analysis methods are
• How to choose your research methodology
• Example of a research methodology

What is research methodology?

Research methodology simply refers to the practical “how” of a research study. More specifically, it’s about how  a researcher  systematically designs a study  to ensure valid and reliable results that address the research aims, objectives and research questions . Specifically, how the researcher went about deciding:

• What type of data to collect (e.g., qualitative or quantitative data )
• Who  to collect it from (i.e., the sampling strategy )
• How to  collect  it (i.e., the data collection method )
• How to  analyse  it (i.e., the data analysis methods )

Within any formal piece of academic research (be it a dissertation, thesis or journal article), you’ll find a research methodology chapter or section which covers the aspects mentioned above. Importantly, a good methodology chapter explains not just   what methodological choices were made, but also explains  why they were made. In other words, the methodology chapter should justify  the design choices, by showing that the chosen methods and techniques are the best fit for the research aims, objectives and research questions. 

So, it’s the same as research design?

Not quite. As we mentioned, research methodology refers to the collection of practical decisions regarding what data you’ll collect, from who, how you’ll collect it and how you’ll analyse it. Research design, on the other hand, is more about the overall strategy you’ll adopt in your study. For example, whether you’ll use an experimental design in which you manipulate one variable while controlling others. You can learn more about research design and the various design types here .

Need a helping hand?

What are qualitative, quantitative and mixed-methods?

Qualitative, quantitative and mixed-methods are different types of methodological approaches, distinguished by their focus on words , numbers or both . This is a bit of an oversimplification, but its a good starting point for understanding.

Let’s take a closer look.

Qualitative research refers to research which focuses on collecting and analysing words (written or spoken) and textual or visual data, whereas quantitative research focuses on measurement and testing using numerical data . Qualitative analysis can also focus on other “softer” data points, such as body language or visual elements.

It’s quite common for a qualitative methodology to be used when the research aims and research questions are exploratory  in nature. For example, a qualitative methodology might be used to understand peoples’ perceptions about an event that took place, or a political candidate running for president. 

Contrasted to this, a quantitative methodology is typically used when the research aims and research questions are confirmatory  in nature. For example, a quantitative methodology might be used to measure the relationship between two variables (e.g. personality type and likelihood to commit a crime) or to test a set of hypotheses .

As you’ve probably guessed, the mixed-method methodology attempts to combine the best of both qualitative and quantitative methodologies to integrate perspectives and create a rich picture. If you’d like to learn more about these three methodological approaches, be sure to watch our explainer video below.

What is sampling strategy?

Simply put, sampling is about deciding who (or where) you’re going to collect your data from . Why does this matter? Well, generally it’s not possible to collect data from every single person in your group of interest (this is called the “population”), so you’ll need to engage a smaller portion of that group that’s accessible and manageable (this is called the “sample”).

How you go about selecting the sample (i.e., your sampling strategy) will have a major impact on your study.  There are many different sampling methods  you can choose from, but the two overarching categories are probability   sampling and  non-probability   sampling .

Probability sampling  involves using a completely random sample from the group of people you’re interested in. This is comparable to throwing the names all potential participants into a hat, shaking it up, and picking out the “winners”. By using a completely random sample, you’ll minimise the risk of selection bias and the results of your study will be more generalisable  to the entire population. 

Non-probability sampling , on the other hand,  doesn’t use a random sample . For example, it might involve using a convenience sample, which means you’d only interview or survey people that you have access to (perhaps your friends, family or work colleagues), rather than a truly random sample. With non-probability sampling, the results are typically not generalisable .

To learn more about sampling methods, be sure to check out the video below.

What are data collection methods?

As the name suggests, data collection methods simply refers to the way in which you go about collecting the data for your study. Some of the most common data collection methods include:

• Interviews (which can be unstructured, semi-structured or structured)
• Focus groups and group interviews
• Surveys (online or physical surveys)
• Observations (watching and recording activities)
• Biophysical measurements (e.g., blood pressure, heart rate, etc.)
• Documents and records (e.g., financial reports, court records, etc.)

The choice of which data collection method to use depends on your overall research aims and research questions , as well as practicalities and resource constraints. For example, if your research is exploratory in nature, qualitative methods such as interviews and focus groups would likely be a good fit. Conversely, if your research aims to measure specific variables or test hypotheses, large-scale surveys that produce large volumes of numerical data would likely be a better fit.

What are data analysis methods?

Data analysis methods refer to the methods and techniques that you’ll use to make sense of your data. These can be grouped according to whether the research is qualitative  (words-based) or quantitative (numbers-based).

Popular data analysis methods in qualitative research include:

• Qualitative content analysis
• Thematic analysis
• Discourse analysis
• Narrative analysis
• Interpretative phenomenological analysis (IPA)
• Visual analysis (of photographs, videos, art, etc.)

Qualitative data analysis all begins with data coding , after which an analysis method is applied. In some cases, more than one analysis method is used, depending on the research aims and research questions . In the video below, we explore some  common qualitative analysis methods, along with practical examples.  

Moving on to the quantitative side of things, popular data analysis methods in this type of research include:

• Descriptive statistics (e.g. means, medians, modes )
• Inferential statistics (e.g. correlation, regression, structural equation modelling)

Again, the choice of which data collection method to use depends on your overall research aims and objectives , as well as practicalities and resource constraints. In the video below, we explain some core concepts central to quantitative analysis.

How do I choose a research methodology?

As you’ve probably picked up by now, your research aims and objectives have a major influence on the research methodology . So, the starting point for developing your research methodology is to take a step back and look at the big picture of your research, before you make methodology decisions. The first question you need to ask yourself is whether your research is exploratory or confirmatory in nature.

If your research aims and objectives are primarily exploratory in nature, your research will likely be qualitative and therefore you might consider qualitative data collection methods (e.g. interviews) and analysis methods (e.g. qualitative content analysis). 

Conversely, if your research aims and objective are looking to measure or test something (i.e. they’re confirmatory), then your research will quite likely be quantitative in nature, and you might consider quantitative data collection methods (e.g. surveys) and analyses (e.g. statistical analysis).

Designing your research and working out your methodology is a large topic, which we cover extensively on the blog . For now, however, the key takeaway is that you should always start with your research aims, objectives and research questions (the golden thread). Every methodological choice you make needs align with those three components. 

Example of a research methodology chapter

In the video below, we provide a detailed walkthrough of a research methodology from an actual dissertation, as well as an overview of our free methodology template .

Psst… there’s more (for free)

This post is part of our dissertation mini-course, which covers everything you need to get started with your dissertation, thesis or research project. 

You Might Also Like:

199 Comments

Thank you for this simple yet comprehensive and easy to digest presentation. God Bless!

You’re most welcome, Leo. Best of luck with your research!

I found it very useful. many thanks

This is really directional. A make-easy research knowledge.

Thank you for this, I think will help my research proposal

Thanks for good interpretation,well understood.

Good morning sorry I want to the search topic

Thank u more

Thank you, your explanation is simple and very helpful.

Very educative a.nd exciting platform. A bigger thank you and I’ll like to always be with you

That’s the best analysis

So simple yet so insightful. Thank you.

This really easy to read as it is self-explanatory. Very much appreciated…

Thanks for this. It’s so helpful and explicit. For those elements highlighted in orange, they were good sources of referrals for concepts I didn’t understand. A million thanks for this.

Good morning, I have been reading your research lessons through out a period of times. They are important, impressive and clear. Want to subscribe and be and be active with you.

Thankyou So much Sir Derek…

Good morning thanks so much for the on line lectures am a student of university of Makeni.select a research topic and deliberate on it so that we’ll continue to understand more.sorry that’s a suggestion.

Beautiful presentation. I love it.

please provide a research mehodology example for zoology

It’s very educative and well explained

Thanks for the concise and informative data.

This is really good for students to be safe and well understand that research is all about

Thank you so much Derek sir🖤🙏🤗

Very simple and reliable

This is really helpful. Thanks alot. God bless you.

very useful, Thank you very much..

thanks a lot its really useful

in a nutshell..thank you!

Thanks for updating my understanding on this aspect of my Thesis writing.

thank you so much my through this video am competently going to do a good job my thesis

Thanks a lot. Very simple to understand. I appreciate 🙏

Very simple but yet insightful Thank you

This has been an eye opening experience. Thank you grad coach team.

Very useful message for research scholars

Really very helpful thank you

yes you are right and i’m left

Research methodology with a simplest way i have never seen before this article.

wow thank u so much

Good morning thanks so much for the on line lectures am a student of university of Makeni.select a research topic and deliberate on is so that we will continue to understand more.sorry that’s a suggestion.

Very precise and informative.

Thanks for simplifying these terms for us, really appreciate it.

Thanks this has really helped me. It is very easy to understand.

I found the notes and the presentation assisting and opening my understanding on research methodology

Good presentation

Im so glad you clarified my misconceptions. Im now ready to fry my onions. Thank you so much. God bless

Thank you a lot.

thanks for the easy way of learning and desirable presentation.

Thanks a lot. I am inspired

Well written

I am writing a APA Format paper . I using questionnaire with 120 STDs teacher for my participant. Can you write me mthology for this research. Send it through email sent. Just need a sample as an example please. My topic is ” impacts of overcrowding on students learning

Thanks for your comment.

We can’t write your methodology for you. If you’re looking for samples, you should be able to find some sample methodologies on Google. Alternatively, you can download some previous dissertations from a dissertation directory and have a look at the methodology chapters therein.

All the best with your research.

Thank you so much for this!! God Bless

Thank you. Explicit explanation

Thank you, Derek and Kerryn, for making this simple to understand. I’m currently at the inception stage of my research.

Thnks a lot , this was very usefull on my assignment

excellent explanation

I’m currently working on my master’s thesis, thanks for this! I’m certain that I will use Qualitative methodology.

Thanks a lot for this concise piece, it was quite relieving and helpful. God bless you BIG…

I am currently doing my dissertation proposal and I am sure that I will do quantitative research. Thank you very much it was extremely helpful.

Very interesting and informative yet I would like to know about examples of Research Questions as well, if possible.

I’m about to submit a research presentation, I have come to understand from your simplification on understanding research methodology. My research will be mixed methodology, qualitative as well as quantitative. So aim and objective of mixed method would be both exploratory and confirmatory. Thanks you very much for your guidance.

OMG thanks for that, you’re a life saver. You covered all the points I needed. Thank you so much ❤️ ❤️ ❤️

Thank you immensely for this simple, easy to comprehend explanation of data collection methods. I have been stuck here for months 😩. Glad I found your piece. Super insightful.

I’m going to write synopsis which will be quantitative research method and I don’t know how to frame my topic, can I kindly get some ideas..

Thanks for this, I was really struggling.

This was really informative I was struggling but this helped me.

Thanks a lot for this information, simple and straightforward. I’m a last year student from the University of South Africa UNISA South Africa.

its very much informative and understandable. I have enlightened.

An interesting nice exploration of a topic.

Thank you. Accurate and simple🥰

This article was really helpful, it helped me understanding the basic concepts of the topic Research Methodology. The examples were very clear, and easy to understand. I would like to visit this website again. Thank you so much for such a great explanation of the subject.

Thanks dude

Thank you Doctor Derek for this wonderful piece, please help to provide your details for reference purpose. God bless.

Many compliments to you

Great work , thank you very much for the simple explanation

Thank you. I had to give a presentation on this topic. I have looked everywhere on the internet but this is the best and simple explanation.

thank you, its very informative.

Well explained. Now I know my research methodology will be qualitative and exploratory. Thank you so much, keep up the good work

Well explained, thank you very much.

This is good explanation, I have understood the different methods of research. Thanks a lot.

Great work…very well explanation

Thanks Derek. Kerryn was just fantastic!

Great to hear that, Hyacinth. Best of luck with your research!

Its a good templates very attractive and important to PhD students and lectuter

Thanks for the feedback, Matobela. Good luck with your research methodology.

Thank you. This is really helpful.

You’re very welcome, Elie. Good luck with your research methodology.

Well explained thanks

This is a very helpful site especially for young researchers at college. It provides sufficient information to guide students and equip them with the necessary foundation to ask any other questions aimed at deepening their understanding.

Thanks for the kind words, Edward. Good luck with your research!

Thank you. I have learned a lot.

Great to hear that, Ngwisa. Good luck with your research methodology!

Thank you for keeping your presentation simples and short and covering key information for research methodology. My key takeaway: Start with defining your research objective the other will depend on the aims of your research question.

My name is Zanele I would like to be assisted with my research , and the topic is shortage of nursing staff globally want are the causes , effects on health, patients and community and also globally

Thanks for making it simple and clear. It greatly helped in understanding research methodology. Regards.

This is well simplified and straight to the point

Thank you Dr

I was given an assignment to research 2 publications and describe their research methodology? I don’t know how to start this task can someone help me?

Sure. You’re welcome to book an initial consultation with one of our Research Coaches to discuss how we can assist – https://gradcoach.com/book/new/ .

Thanks a lot I am relieved of a heavy burden.keep up with the good work

I’m very much grateful Dr Derek. I’m planning to pursue one of the careers that really needs one to be very much eager to know. There’s a lot of research to do and everything, but since I’ve gotten this information I will use it to the best of my potential.

Thank you so much, words are not enough to explain how helpful this session has been for me!

Thanks this has thought me alot.

Very concise and helpful. Thanks a lot

Thank Derek. This is very helpful. Your step by step explanation has made it easier for me to understand different concepts. Now i can get on with my research.

I wish i had come across this sooner. So simple but yet insightful

really nice explanation thank you so much

I’m so grateful finding this site, it’s really helpful…….every term well explained and provide accurate understanding especially to student going into an in-depth research for the very first time, even though my lecturer already explained this topic to the class, I think I got the clear and efficient explanation here, much thanks to the author.

It is very helpful material

I would like to be assisted with my research topic : Literature Review and research methodologies. My topic is : what is the relationship between unemployment and economic growth?

Its really nice and good for us.

THANKS SO MUCH FOR EXPLANATION, ITS VERY CLEAR TO ME WHAT I WILL BE DOING FROM NOW .GREAT READS.

Short but sweet.Thank you

Informative article. Thanks for your detailed information.

I’m currently working on my Ph.D. thesis. Thanks a lot, Derek and Kerryn, Well-organized sequences, facilitate the readers’ following.

great article for someone who does not have any background can even understand

I am a bit confused about research design and methodology. Are they the same? If not, what are the differences and how are they related?

Thanks in advance.

concise and informative.

Thank you very much

How can we site this article is Harvard style?

Very well written piece that afforded better understanding of the concept. Thank you!

Am a new researcher trying to learn how best to write a research proposal. I find your article spot on and want to download the free template but finding difficulties. Can u kindly send it to my email, the free download entitled, “Free Download: Research Proposal Template (with Examples)”.

Thank too much

Thank you very much for your comprehensive explanation about research methodology so I like to thank you again for giving us such great things.

Good very well explained.Thanks for sharing it.

Thank u sir, it is really a good guideline.

so helpful thank you very much.

Thanks for the video it was very explanatory and detailed, easy to comprehend and follow up. please, keep it up the good work

It was very helpful, a well-written document with precise information.

how do i reference this?

MLA Jansen, Derek, and Kerryn Warren. “What (Exactly) Is Research Methodology?” Grad Coach, June 2021, gradcoach.com/what-is-research-methodology/.

APA Jansen, D., & Warren, K. (2021, June). What (Exactly) Is Research Methodology? Grad Coach. https://gradcoach.com/what-is-research-methodology/

Your explanation is easily understood. Thank you

Very help article. Now I can go my methodology chapter in my thesis with ease

I feel guided ,Thank you

This simplification is very helpful. It is simple but very educative, thanks ever so much

The write up is informative and educative. It is an academic intellectual representation that every good researcher can find useful. Thanks

Wow, this is wonderful long live.

Nice initiative

thank you the video was helpful to me.

Thank you very much for your simple and clear explanations I’m really satisfied by the way you did it By now, I think I can realize a very good article by following your fastidious indications May God bless you

Thanks very much, it was very concise and informational for a beginner like me to gain an insight into what i am about to undertake. I really appreciate.

very informative sir, it is amazing to understand the meaning of question hidden behind that, and simple language is used other than legislature to understand easily. stay happy.

This one is really amazing. All content in your youtube channel is a very helpful guide for doing research. Thanks, GradCoach.

research methodologies

Please send me more information concerning dissertation research.

Nice piece of knowledge shared….. #Thump_UP

This is amazing, it has said it all. Thanks to Gradcoach

This is wonderful,very elaborate and clear.I hope to reach out for your assistance in my research very soon.

This is the answer I am searching about…

realy thanks a lot

Thank you very much for this awesome, to the point and inclusive article.

Thank you very much I need validity and reliability explanation I have exams

Thank you for a well explained piece. This will help me going forward.

Very simple and well detailed Many thanks

This is so very simple yet so very effective and comprehensive. An Excellent piece of work.

I wish I saw this earlier on! Great insights for a beginner(researcher) like me. Thanks a mil!

Thank you very much, for such a simplified, clear and practical step by step both for academic students and general research work. Holistic, effective to use and easy to read step by step. One can easily apply the steps in practical terms and produce a quality document/up-to standard

Thanks for simplifying these terms for us, really appreciated.

Thanks for a great work. well understood .

This was very helpful. It was simple but profound and very easy to understand. Thank you so much!

Great and amazing research guidelines. Best site for learning research

hello sir/ma’am, i didn’t find yet that what type of research methodology i am using. because i am writing my report on CSR and collect all my data from websites and articles so which type of methodology i should write in dissertation report. please help me. i am from India.

how does this really work?

perfect content, thanks a lot

As a researcher, I commend you for the detailed and simplified information on the topic in question. I would like to remain in touch for the sharing of research ideas on other topics. Thank you

Impressive. Thank you, Grad Coach 😍

Thank you Grad Coach for this piece of information. I have at least learned about the different types of research methodologies.

Very useful content with easy way

Thank you very much for the presentation. I am an MPH student with the Adventist University of Africa. I have successfully completed my theory and starting on my research this July. My topic is “Factors associated with Dental Caries in (one District) in Botswana. I need help on how to go about this quantitative research

I am so grateful to run across something that was sooo helpful. I have been on my doctorate journey for quite some time. Your breakdown on methodology helped me to refresh my intent. Thank you.

thanks so much for this good lecture. student from university of science and technology, Wudil. Kano Nigeria.

It’s profound easy to understand I appreciate

Thanks a lot for sharing superb information in a detailed but concise manner. It was really helpful and helped a lot in getting into my own research methodology.

Comment * thanks very much

This was sooo helpful for me thank you so much i didn’t even know what i had to write thank you!

You’re most welcome 🙂

Simple and good. Very much helpful. Thank you so much.

This is very good work. I have benefited.

Thank you so much for sharing

This is powerful thank you so much guys

I am nkasa lizwi doing my research proposal on honors with the university of Walter Sisulu Komani I m on part 3 now can you assist me.my topic is: transitional challenges faced by educators in intermediate phase in the Alfred Nzo District.

Appreciate the presentation. Very useful step-by-step guidelines to follow.

I appreciate sir

wow! This is super insightful for me. Thank you!

Indeed this material is very helpful! Kudos writers/authors.

I want to say thank you very much, I got a lot of info and knowledge. Be blessed.

I want present a seminar paper on Optimisation of Deep learning-based models on vulnerability detection in digital transactions.

Need assistance

Dear Sir, I want to be assisted on my research on Sanitation and Water management in emergencies areas.

I am deeply grateful for the knowledge gained. I will be getting in touch shortly as I want to be assisted in my ongoing research.

The information shared is informative, crisp and clear. Kudos Team! And thanks a lot!

hello i want to study

Hello!! Grad coach teams. I am extremely happy in your tutorial or consultation. i am really benefited all material and briefing. Thank you very much for your generous helps. Please keep it up. If you add in your briefing, references for further reading, it will be very nice.

All I have to say is, thank u gyz.

Good, l thanks

thank you, it is very useful

Trackbacks/Pingbacks

• What Is A Literature Review (In A Dissertation Or Thesis) - Grad Coach - […] the literature review is to inform the choice of methodology for your own research. As we’ve discussed on the Grad Coach blog,…
• Free Download: Research Proposal Template (With Examples) - Grad Coach - […] Research design (methodology) […]
• Dissertation vs Thesis: What's the difference? - Grad Coach - […] and thesis writing on a daily basis – everything from how to find a good research topic to which…

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

info This is a space for the teal alert bar.

notifications This is a space for the yellow alert bar.

Research Process

• Brainstorming
• Explore Google This link opens in a new window
• Explore Web Resources
• Explore Background Information
• Explore Books
• Explore Scholarly Articles
• Narrowing a Topic
• Primary and Secondary Resources
• Academic, Popular & Trade Publications
• Scholarly and Peer-Reviewed Journals
• Grey Literature
• Clinical Trials
• Evidence Based Treatment
• Scholarly Research
• Database Research Log
• Search Limits
• Keyword Searching
• Boolean Operators
• Phrase Searching
• Truncation & Wildcard Symbols
• Proximity Searching
• Field Codes
• Subject Terms and Database Thesauri
• Reading a Scientific Article
• Website Evaluation
• Article Keywords and Subject Terms
• Cited References
• Citing Articles
• Related Results
• Search Within Publication
• Database Alerts & RSS Feeds
• Personal Database Accounts
• Persistent URLs
• Literature Gap and Future Research
• Web of Knowledge
• Annual Reviews
• Systematic Reviews & Meta-Analyses
• Finding Seminal Works
• Exhausting the Literature
• Finding Dissertations
• Researching Theoretical Frameworks
• Research Methodology & Design

Tests and Measurements

• Organizing Research & Citations This link opens in a new window
• Scholarly Publication
• Learn the Library This link opens in a new window

Tests & Measurements FAQs

• Ask a Librarian Search all of the FAQs here.

NU Dissertation Center

If you are looking for a document in the Dissertation Center or Applied Doctoral Center and can't find it please contact your Chair or The Center for Teaching and Learning at [email protected]

• NCU Dissertation Center Find valuable resources and support materials to help you through your doctoral journey.
• Applied Doctoral Center Collection of resources to support students in completing their project/dissertation-in-practice as part of the Applied Doctoral Experience (ADE).

If you are doing dissertation level research, you will also be collecting your own data using a test or measure designed to address the variables present in your research. Finding the right test or measure can sometimes be difficult. In some cases, tests are copyrighted and must be purchased from commercial publishers. In other cases instruments can be obtained for free directly from the authors or can be found within published articles (in the methods section or as an appendix). The Library can help you with obtaining publisher or author information along with test reviews, if they are available.

One important decision you will eventually face in the dissertation process is whether to use an existing instrument, to modify an instrument, or to create your own instrument from scratch. The latter two will require extensive testing and are not generally recommended. Whichever decision you make should be thought over carefully and discussed with your mentor or dissertation chair committee.

You will need to either purchase the test from a publisher or contact author(s) to obtain the test along with copyright permissions to use it in your research. When contacting an author for copyright permissions you will often send a permission letter. Examples of permission letters are included in the Permission Letters section below. 

Want a  video introduction? See the Introduction to Tests and Measurements Workshop below.

Want to learn via a week's worth of daily lessons ?   Sign up for our Tests & Measures - Learn in a Week series.  

Introduction to Tests & Measurements Workshop

This workshop provides an introduction to library resources which can be used to locate tests and measurements for dissertation research.

• Introduction to Tests and Measurements Workshop Outline

Searching for Tests and Measurements

When conducting a search, remember that different keywords yield different results. Consider these terms and add them to your search string when trying to locate tests or measurements: 

• Survey 
• Instrument 
• Questionnaire 
• Measure 
• Measurement 
• Assessment 

NavigatorSearch Search

Searching in NavigatorSearch

The simplest way to discover instruments relevant to your dissertation research is to carefully read the "Methods" section in peer-reviewed journal articles. A dissertation will build on a field of study and you will be well served by understanding how the constructs you are interested in have been measured. For example, while exploring the topic of depression, read articles and take note of which depression inventories are used and why.

• Start by conducting a keyword search on your topic using NavigatorSearch , the central search box found on the Library's homepage. NavigatorSearch searches most of our Library's database content, so it is a great starting point for any research topic.
• Use advanced search techniques covered in Searching 101 like subject searching, truncation, and Boolean operators to make your search more precise. You may also read about these search techniques by referring to the Preparing to Search section of our Research Process guide.

Library Databases

• APA PsycArticles & APA PsycInfo
• APA PsycTests
• ETS Test Link
• Health and Psychosocial Instruments (HAPI)
• MMY with Tests in Print
• ScienceDirect
• ProQuest Dissertations & Theses

Content: APA database that offers full-text for journals published by APA, the Canadian Psychological Association, Hogrefe Publishing Group and APA's Educational Publishing Foundation. View the  APA PsycArticles Journal History  for a complete coverage list.

Purpose: Important database for psychology, counseling, and education students.

Special Features: The database is updated bi-weekly all content is available in PDF and HTML formats.

Help using this database.

Content: Journal article database from the American Psychological Association that indexes over 2,500 journals along with book chapters and dissertations.

Purpose: Provides a single source of vetted, authoritative research for users across the behavioral and social sciences.

Special Features: citations in APA Style®, updated bi-weekly, spans 600 years of content

Searching in APA PsycArticles and APA PsycInfo

To locate tests and measurements in APA PsycArticles or APA PsycInfo, follow the below steps:

Content: Psychological tests and measures designed for use with social and behavioral science research

Purpose: Allows students and researchers to find and download instruments for research and/or teaching. Focused primarily on unpublished tests, this database was designed to save researchers time from having to reproduce tests when conducting research on previously measured constructs.

Special Features: Records include summary of the construct, and users can find information on reliability, validity, and factor analysis when that data is reported in the source document.

Searching in APA PsycTests

To locate tests and measurements in APA PsycTests, follow the below steps:

Content: EBSCO’s nursing database covering biomedicine, alternative/complementary medicine, consumer health, and allied health disciplines.

Purpose: Database for research in nursing, medicine, and consumer health.

Special Features: Strong qualitative studies. Filter studies by nurse as author, evidence-based practice, and type of study. Includes MESH indexing, PICO search functionality, text-to-speech feature for some articles, and a tool for discovering citing articles.

Searching in CINAHL

To search for tests or measurements in CINAHL, follow the below steps:

Additional Search Strategies for Locating Tests and Measurements in CINAHL 

Content: Government (Department of Education) database focusing on education research and information.

Purpose: Excellent database to use for all topics in education. 

Special Features: After an initial search, filter by audience, grade level, survey used, and main topic. Includes a thesaurus to aid in the discovery process.

This federally subsidized database indexes both journals and other resources important to educators. ERIC Journals (EJ): journal articles ERIC Documents (ED): all non-journal materials (some books, unpublished reports, and presentations) Some faculty limit use of EDs

Searching in ERIC 

To search for tests or measurements in ERIC, follow the below steps:

• On the search results page, use the filters on the left-hand side to limit your results. Select Tests/Questionnaires under Publication Type .

In addition, the  ERIC thesaurus entries list descriptors of tests and scales which may be used to construct a search. Select a broad category and continue narrowing down to your desired term. Click on Search collection using this descriptor to begin your search, as shown below.

For additional information, see the following quick tutorial video:

• ERIC Quick Tutorial Video

Content: A tests and measurements database containing standardized tests and research instruments.

Purpose: Allows users to search for tests and measurements, generally in the education field.

Special Features: A simple keyword strategy reveals many useful tests and measurements.

Searching in ETS TestLink

To locate tests and measurements for education in ETS TestLink, follow the below steps:

• ETS TestLink Quick Tutorial Video

Content: EBSCO database of test instruments found within articles.

Purpose: Provides users with instruments and measurements used in health and behavioral sciences and education

Special Features: Can be used along with APA PsycTests, ETS, and Mental Measurements to learn about instruments in the education and behavioral and health sciences.

A comprehensive bibliographic database providing information about behavioral measurement instruments. Information in the database is abstracted from hundreds of leading journals covering health sciences and psychosocial sciences.

HaPI provides free assistance to students in optimizing searches and locating hard copies and scoring instructions of specific assessment tools.

You can reach HaPI measurement staff either by phone (412-687-6850) or by email ( [email protected] ).

Searching in Health and Psychosocial Instruments (HAPI)

To locate tests and measurements in HAPI, follow the below steps:

Content: Contains reviews of test instruments and measures

Purpose: Users may learn about the strengths and weaknesses of particular test instruments. 

Special Features: Includes automatic translation software

Searching in Mental Measurements Yearbook with Tests in Print

Mental Measurements Yearbook with Tests in Print (MMY with TiP) offers test reviews that are written by experts and contain descriptions of tests and commentary on their psychometric adequacy (Cone & Foster, 2006, pg. 170). You can use MMY with TiP to (1) obtain contact information for an author or publisher, and (2) read descriptive information on the measure of interest. Note that you will need to either purchase the test from the publisher directly, or contact author(s) to obtain the test along with copyright permissions to use it in your research.

To locate tests and measurements in Mental Measurements Yearbook with Tests in Print, follow the below steps:

• Click on a search result to obtain relevant information about the test or measurement, including Publisher Information, Purpose, Population, Time for completion, and Price Data among other details. A detailed review and summary of the test or measurement will also be provided.

Content: Includes citations to millions of biomedical journal articles, as well as some books, book chapters, and reports. 

Purpose: An essential database for biomedical and health topics 

Special Features: Includes MeSH search functionality

Searching in PubMed

To locate tests and measurements in PubMed, use the following strategies:

• Add any of the following MESH subject headings to your topic search string to locate relevant tests or measurements: 
• "Research Design"[Mesh]
• "Surveys and Questionnaires"[Mesh]
• "Personality Inventory"[Mesh]
• "Test Anxiety Scale"[Mesh]
• "Health Care Surveys"[Mesh]
• "Nutrition Surveys"[Mesh]
• "Health Surveys"[Mesh]
• "Dental Health Surveys"[Mesh]
• "Diet Surveys"[Mesh]
• "Behavior Rating Scale"[Mesh]
• "Patient Health Questionnaire"[Mesh]

Below are example search strings incorporating the use of these MESH subject headings:

• "eating disorder" AND "Surveys and Questionnaires"[Mesh]
• depression AND "Patient Health Questionnaire"[Mesh] 
• anxiety AND "Personality Inventory"[Mesh] 

For additional information, see the following training videos:

• PubMed Online Training

Content: Elsevier’s science database covering computer science, health science, and social sciences. Contains peer-reviewed and open-access journal articles and book chapters.

Purpose: A great resource that covers foundational science to new and novel research.

Special Features: Covers theoretical and practical aspects of physical, life, health, and social sciences.

Searching in ScienceDirect 

Use ScienceDirect  to locate tests and measurements used in studies and published articles relevant to your topic. Add any of the following keywords to your search string: 

For additional information, see the following video:

• ScienceDirect Quick Tutorial Video

Content: Citations and articles in multi-disciplines not found through a NavigatorSearch.

Purpose: Used to conduct topic searches as well as find additional resources that have cited a specific resource (citation network).

Searching in Web of Knowledge

Use Web of Knowledge  to locate tests and measurements used in studies and published articles relevant to your topic. Add any of the following keywords to your search string: 

For additional information, visit the following website:

• Web of Science Training Portal

Content: Global student dissertations and literature reviews.

Purpose: Use for foundational research, to locate test instruments and data, and more. 

Special Features: Search by advisor (chair), degree, degree level, or department. Includes a read-aloud feature

The ProQuest Dissertations & Theses database (PQDT) is the world's most comprehensive collection of dissertations and theses. It is the database of record for graduate research, with over 2.3 million dissertations and theses included from around the world.

Content: National University & NCU student dissertations and literature reviews.

Special Features: Search by advisor (chair), degree, degree level, or department. Includes a read-aloud feature.

Searching in ProQuest Dissertations & Theses

Locate tests and measurements in ProQuest Dissertations & Theses by using the following strategies:

Search for related graduate and doctoral-level research that has already been conducted on your topic.  Similar studies may have employed a relevant test or measurement.

• ProQuest Dissertations & Theses Quick Tutorial Video

Internet Search

Lastly, you might try searching for a test or measurement or information about them on the Internet. Google is an excellent search engine for finding information on test instruments. To find information about a particular test or measurement on Google, type the name of the test or measurement into the empty search field and place it in quotes:

Permissions

Unless your test instrument is commercially available (i.e., available for purchase), you will likely need to seek permission to use a test instrument in your dissertation. An exception may be instruments retrieved from the APA PsycTests database. The majority of tests within this database can be used without seeking additional permission. However, the instrument must explicitly state May use for Research/Teaching in the permissions field. 

Also note that obtaining permission to use an instrument is not the same as obtaining permission to reproduce the instrument in its entirety in your dissertation appendix. It is important that you ask for separate permissions to do that.

First, you will need to identify who owns the copyright. The copyright holder is usually the author/creator of the work. Often, the author’s email address appears within the published journal article from which the instrument originated. If you need help tracking down the original article, please contact the Library.

If an email address is not readily available or seems to be outdated, you will need to search for the author’s contact information online. Try using quotation marks around the name or adding an associated institution to narrow your results. Again, if you need assistance with the step, the Library can recommend search techniques. However, the Library will not contact authors on your behalf.

Once you have located the contact information, prepare to introduce yourself and explain why are seeking permission. State clearly who you are, your institutional affiliation (e.g., Northcentral University), and the general nature of your thesis/dissertation research. Also discuss whether you are modifying the instrument, or if you are reproducing the instrument in your appendix. Typically, an email exchange is best, but some authors may prefer mail correspondence or a phone call. There are many sample permissions letters available online, including some examples linked below.

In some cases, authors transfer copyright to another entity, such as a journal publisher or an organization. Publishers often have website forms or letter templates that you can use to submit your request. See an example from Wiley here .

Remember, you will need to document permissions in your dissertation appendix. Make sure to save a copy of the correspondence and the agreement. Documentation allows you to demonstrate to your Chair and others that you have the legal right to use the owner's work.

In some cases, authors or publishers may either not respond to requests or refuse to grant permission to use their work. Therefore, it is important to select a few potential tests or measurements. The Library can certainly assist with searching for alternate test instruments.

For additional information about copyright and permission guidelines, see sections 12.14 - 12.18 in the APA Manual, 7th edition.

• Columbia University: Reprinting into a New Work Model Letter
• Copyright and Your Dissertation or Theses
• St Mary's University: Sample Permission Letter for a Thesis or Dissertation
• University of Pittsburgh: Sample Permission Letter for Dissertations
• University of Michigan: Obtaining Copyright Permissions

Selected Resources

• Open Access
• Tutorials & Guides
• Additional Resources
• Applied Measurement in Education
• Applied Psychological Measurement
• Assessment and Evaluation in Higher Education
• Assessment for Effective Intervention
• Assessment in Education: Principles, Policy& Practice
• Assessment Update
• Educational Assessment
• Educational Assessment, Evaluation andAccountability
• Educationaland Psychological Measurement
• Educational Evaluation and Policy Analysis
• Educational Measurement Issues and Practice
• European Journal of Psychological Assessment
• FairTest Examiner
• International Journal of Educational and Psychological Assessment
• International Journal of Selection and Assessment
• Journal of Educational Measurement
• Journal of Methods and Measurement in the Social Sciences
• Journalof Personality Assessment
• Journal of Psychoeducational Assessment
• Journal of Psychopathology and Behavioral Assessment
• Large-Scale Assessments in Education
• Measurement and Evaluation in Counseling and Development
• Practical Assessment, Research & Evaluation
• Psychological Assessment
• Psychological Test and Assessment Modeling
• Research & Practice in Assessment
• Social Science Research
• Sociological Methods & Research

Content: Books, reference works, journal articles, and instructional videos on research methods and design. 

Purpose: Use to learn more about qualitative, quantitative, and mixed methods research. 

Special Features: Includes a methods map, project planner, and "which stats" test

• Measuring Intimate Partner Violence Victimization and Perpetration: A Compendium of Assessment Tools Includes more than 20 scales for measuring the self-reported incidence and prevalence of Intimate Partner Violence victimization and perpetration.
• Measuring Violence-Related Attitudes, Beliefs, and Behaviors Among Youths: A Compendium of Assessment Tools Contains more than 100 measures designed to assess violence-related beliefs, behaviors, and influences, as well as to evaluate programs to prevent youth violence.
• Practitioner's Guide to Empirically Based Measures of School Behavior Contains descriptions of instruments in Chapter 6.
• Taking the Measure of Work: A Guide to Validated Scales for Organizational Research and Diagnosis Measures included in the book are those that can be completed as part of a questionnaire or survey, or administered as part of an interview.
• Alcohol & Drug Abuse Institute (ADAI) Screening and Assessment Instruments Database Provides instruments on alcohol and other drug use from all relevant disciplines. Some instruments are in the public domain and can be freely downloaded from the web.
• Association of Test Publishers Represents providers of tests and assessment tools and/or services related to assessment, selection, screening, certification, licensing, educational or clinical uses. Includes links to test publishers.
• Atlas of Integrated Behavioral Health Care Quality Measures Provides a list of existing measures relevant to integrated behavioral health care.
• British Psychological Society's Psychological Testing Center Provides test reviews in the areas of Counselling, Education, General Health, Life and Well-being, Occupational and Work, Psychology and more.
• Center for Equity and Excellence in Education Test Database Collection of abstracts and descriptions of almost 200 tests commonly used with Limited English Proficient students.
• Center for HIV Identification, Prevention and Treatment Services Directory of instruments including Abuse Assessment Screen, Depression (Thai Version), Emotional Social Support, and Quality of Life, HIV.
• Center for Outcome Measurement in Brain Injury Provides scales relating to rehabilitation, disability, cognitive functioning, life satisfaction, and more.
• Child Care and Early Education: Datasets, Instruments and Tools for Analysis Search for instruments by keyword or author, or browse by topic.
• Childhood Anxiety Screening Tool Instruments/Rating Scales Includes screening tool instruments and rating scales for supporting children and adolescents experiencing general anxiety disorder and/or post traumatic stress disorder.
• Compendium of Assessment and Research Tools Database that provides information on instruments that measure attributes associated with youth development programs.
• Directory of Tests with Links to Publishers Links will re-direct you to the publishers' sites for price and ordering information. Test topics include ability, achievement, neuropsychology, personality, psychopathology, and more.
• DMOZ Open Directory Project listing of tests and testing links.
• Ericae.net Online gateway to ERIC's resources on assessment and evaluation.
• Health Services and Sciences Research Resources Provides information about research datasets and instruments/indices that are used in Health Services Research, Behavioral and Social Sciences, and Public Health.
• Instrument Wizard Site will help you identify and learn more about screening, diagnostic, research, evaluation, and needs assessment instruments designed to measure substance use and related topics. Requires membership fee.
• International Personality Item Pool Provides access to measures of individual differences, all in the public domain.
• Mental Health Instruments in Non-English Languages Provides links to a range of scales, resources, research and other related information.
• National Information Center on Health Services Research and Health Care Technology Information about research datasets and instruments/indices employed in Health Services Research, Behavioral and Social Sciences and Public Health with links to PubMed.
• National Quality Measures Clearinghouse Provides information on specific evidence-based health care quality measures and measure sets.
• Online Evaluation Resource Directory Collection of sound education plans, reports, and instruments from past and current project evaluations in several content areas.
• Patient-Reported Outcome and Quality of Life Instruments Database (PROQOLID) Offers free, but limited public access to their database. For each instrument in the database, you will find 14 categories of basic information (e.g., author, objective, mode of administration, original language, existing translations, pathology, number of items, etc.). Requires registration.
• Pearson Assessments Commercial site that offers assessments for clinical and psychological use. Tests are clearly explained for functionality and implementation. Tests can be purchased from this site.
• Positive Psychology Questionnaires Information about the positive psychology questionnaires, some of which can be downloaded from the site.
• Psychological Tests for Student Use List of copyrighted tests that students have permission to use (and so don't have to go through the permission inquiry process), from York University.
• Psychosocial Measures for Review (PhenX) List of psychosocial measures with background information, full text of the measure, and scoring instructions.
• RAND Health - Surveys & Tools Surveys include topics such as Aging and Health, Mental Health, & Quality of Life. All of the surveys from RAND Health are public documents, available without charge.
• Registry of Scales and Measures Psychological tests, scales, questionnaires, and checklists can be searched by several parameters, including author, title, year of publication, and topic, as well as by scale and item characteristics.
• Research Instruments Developed, Adapted or Used by the Stanford Patient Education Research Center Includes scales for research subjects with chronic diseases in English and Spanish. You may use any of these scales at no cost without permission.
• SDSU Test Finder San Diego State University librarians have developed a searchable database of tests, instruments, rating scales, and measures available in books.
• SDSU Test Finder for Journal Articles Another tool from San Diego State University allows you to search for complete psychosocial tests, instruments, rating scales, and measures found in the journal literature.
• Self-Report Measures From the University of Miami Department of Psychology, a listing of scales made available for use in research and teaching applications. All are available without charge and without any need for permission.
• Social-Personality Psychology Questionnaire Instrument Compendium (QIC) Directory of public domain tests.
• Statistics Solutions Directory of Survey Instruments Each survey instrument's page includes a description, references, and a link to purchase the instrument directly from the author. Individual authorization from the author is required in order to administer any of the surveys.
• Substance Use Screening & Assessment Instruments Database Instruments used for screening and assessment of substance use and substance use disorders. Some instruments are in the public domain and can be freely downloaded from the web; others can only be obtained from the copyright holder.
• Test Reviews Online Buros Center for Testing site provides reviews for over 4,000 commercially available tests. Over 2,000 of the tests have been reviewed in Mental Measurements. Reviews require purchase.
• Tests and Measures in the Social Sciences An index to 139 print compilations, web sites and other resources for test instruments with nearly 14,000 tests and measurements compiled from 1967 - 2014.
• USF Test & Measures Collection Indexes instruments, questionnaires, surveys, or tests that are contained in various books held in libraries. Click "full record" for the test, you will find annotated details about the instrument.
• Index to Tests in Journal Articles San Diego State University
• Searching for Test Instruments Lister Hill Library of the Health Sciences
• Tests and Measures in the Social Sciences University of Texas at Arlington
• APA: FAQ/Finding Information About Psychological Tests
• ERIC: Questions To Ask When Evaluating Tests
• APA PsycTests on EBSCOhost

Was this resource helpful?

• << Previous: Research Methodology & Design
• Next: Organizing Research & Citations >>
• Last Updated: Mar 31, 2024 4:57 PM
• URL: https://resources.nu.edu/researchprocess

© Copyright 2024 National University. All Rights Reserved.

Privacy Policy | Consumer Information

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

Research Methods | Definitions, Types, Examples

Research methods are specific procedures for collecting and analyzing data. Developing your research methods is an integral part of your research design . When planning your methods, there are two key decisions you will make.

First, decide how you will collect data . Your methods depend on what type of data you need to answer your research question :

• Qualitative vs. quantitative : Will your data take the form of words or numbers?
• Primary vs. secondary : Will you collect original data yourself, or will you use data that has already been collected by someone else?
• Descriptive vs. experimental : Will you take measurements of something as it is, or will you perform an experiment?

Second, decide how you will analyze the data .

• For quantitative data, you can use statistical analysis methods to test relationships between variables.
• For qualitative data, you can use methods such as thematic analysis to interpret patterns and meanings in the data.

Table of contents

Methods for collecting data, examples of data collection methods, methods for analyzing data, examples of data analysis methods, other interesting articles, frequently asked questions about research methods.

Data is the information that you collect for the purposes of answering your research question . The type of data you need depends on the aims of your research.

Qualitative vs. quantitative data

Your choice of qualitative or quantitative data collection depends on the type of knowledge you want to develop.

For questions about ideas, experiences and meanings, or to study something that can’t be described numerically, collect qualitative data .

If you want to develop a more mechanistic understanding of a topic, or your research involves hypothesis testing , collect quantitative data .

You can also take a mixed methods approach , where you use both qualitative and quantitative research methods.

Primary vs. secondary research

Primary research is any original data that you collect yourself for the purposes of answering your research question (e.g. through surveys , observations and experiments ). Secondary research is data that has already been collected by other researchers (e.g. in a government census or previous scientific studies).

If you are exploring a novel research question, you’ll probably need to collect primary data . But if you want to synthesize existing knowledge, analyze historical trends, or identify patterns on a large scale, secondary data might be a better choice.

Descriptive vs. experimental data

In descriptive research , you collect data about your study subject without intervening. The validity of your research will depend on your sampling method .

In experimental research , you systematically intervene in a process and measure the outcome. The validity of your research will depend on your experimental design .

To conduct an experiment, you need to be able to vary your independent variable , precisely measure your dependent variable, and control for confounding variables . If it’s practically and ethically possible, this method is the best choice for answering questions about cause and effect.

Prevent plagiarism. Run a free check.

Your data analysis methods will depend on the type of data you collect and how you prepare it for analysis.

Data can often be analyzed both quantitatively and qualitatively. For example, survey responses could be analyzed qualitatively by studying the meanings of responses or quantitatively by studying the frequencies of responses.

Qualitative analysis methods

Qualitative analysis is used to understand words, ideas, and experiences. You can use it to interpret data that was collected:

• From open-ended surveys and interviews , literature reviews , case studies , ethnographies , and other sources that use text rather than numbers.
• Using non-probability sampling methods .

Qualitative analysis tends to be quite flexible and relies on the researcher’s judgement, so you have to reflect carefully on your choices and assumptions and be careful to avoid research bias .

Quantitative analysis methods

Quantitative analysis uses numbers and statistics to understand frequencies, averages and correlations (in descriptive studies) or cause-and-effect relationships (in experiments).

You can use quantitative analysis to interpret data that was collected either:

• During an experiment .
• Using probability sampling methods .

Because the data is collected and analyzed in a statistically valid way, the results of quantitative analysis can be easily standardized and shared among researchers.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Chi square test of independence
• Statistical power
• Descriptive statistics
• Degrees of freedom
• Pearson correlation
• Null hypothesis
• Double-blind study
• Case-control study
• Research ethics
• Data collection
• Hypothesis testing
• Structured interviews

Research bias

• Hawthorne effect
• Unconscious bias
• Recall bias
• Halo effect
• Self-serving bias
• Information bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

A sample is a subset of individuals from a larger population . Sampling means selecting the group that you will actually collect data from in your research. For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

In statistics, sampling allows you to test a hypothesis about the characteristics of a population.

The research methods you use depend on the type of data you need to answer your research question .

• If you want to measure something or test a hypothesis , use quantitative methods . If you want to explore ideas, thoughts and meanings, use qualitative methods .
• If you want to analyze a large amount of readily-available data, use secondary data. If you want data specific to your purposes with control over how it is generated, collect primary data.
• If you want to establish cause-and-effect relationships between variables , use experimental methods. If you want to understand the characteristics of a research subject, use descriptive methods.

Methodology refers to the overarching strategy and rationale of your research project . It involves studying the methods used in your field and the theories or principles behind them, in order to develop an approach that matches your objectives.

Methods are the specific tools and procedures you use to collect and analyze data (for example, experiments, surveys , and statistical tests ).

In shorter scientific papers, where the aim is to report the findings of a specific study, you might simply describe what you did in a methods section .

In a longer or more complex research project, such as a thesis or dissertation , you will probably include a methodology section , where you explain your approach to answering the research questions and cite relevant sources to support your choice of methods.

Is this article helpful?

Other students also liked, writing strong research questions | criteria & examples.

• What Is a Research Design | Types, Guide & Examples
• Data Collection | Definition, Methods & Examples

More interesting articles

• Between-Subjects Design | Examples, Pros, & Cons
• Cluster Sampling | A Simple Step-by-Step Guide with Examples
• Confounding Variables | Definition, Examples & Controls
• Construct Validity | Definition, Types, & Examples
• Content Analysis | Guide, Methods & Examples
• Control Groups and Treatment Groups | Uses & Examples
• Control Variables | What Are They & Why Do They Matter?
• Correlation vs. Causation | Difference, Designs & Examples
• Correlational Research | When & How to Use
• Critical Discourse Analysis | Definition, Guide & Examples
• Cross-Sectional Study | Definition, Uses & Examples
• Descriptive Research | Definition, Types, Methods & Examples
• Ethical Considerations in Research | Types & Examples
• Explanatory and Response Variables | Definitions & Examples
• Explanatory Research | Definition, Guide, & Examples
• Exploratory Research | Definition, Guide, & Examples
• External Validity | Definition, Types, Threats & Examples
• Extraneous Variables | Examples, Types & Controls
• Guide to Experimental Design | Overview, Steps, & Examples
• How Do You Incorporate an Interview into a Dissertation? | Tips
• How to Do Thematic Analysis | Step-by-Step Guide & Examples
• How to Write a Literature Review | Guide, Examples, & Templates
• How to Write a Strong Hypothesis | Steps & Examples
• Inclusion and Exclusion Criteria | Examples & Definition
• Independent vs. Dependent Variables | Definition & Examples
• Inductive Reasoning | Types, Examples, Explanation
• Inductive vs. Deductive Research Approach | Steps & Examples
• Internal Validity in Research | Definition, Threats, & Examples
• Internal vs. External Validity | Understanding Differences & Threats
• Longitudinal Study | Definition, Approaches & Examples
• Mediator vs. Moderator Variables | Differences & Examples
• Mixed Methods Research | Definition, Guide & Examples
• Multistage Sampling | Introductory Guide & Examples
• Naturalistic Observation | Definition, Guide & Examples
• Operationalization | A Guide with Examples, Pros & Cons
• Population vs. Sample | Definitions, Differences & Examples
• Primary Research | Definition, Types, & Examples
• Qualitative vs. Quantitative Research | Differences, Examples & Methods
• Quasi-Experimental Design | Definition, Types & Examples
• Questionnaire Design | Methods, Question Types & Examples
• Random Assignment in Experiments | Introduction & Examples
• Random vs. Systematic Error | Definition & Examples
• Reliability vs. Validity in Research | Difference, Types and Examples
• Reproducibility vs Replicability | Difference & Examples
• Reproducibility vs. Replicability | Difference & Examples
• Sampling Methods | Types, Techniques & Examples
• Semi-Structured Interview | Definition, Guide & Examples
• Simple Random Sampling | Definition, Steps & Examples
• Single, Double, & Triple Blind Study | Definition & Examples
• Stratified Sampling | Definition, Guide & Examples
• Structured Interview | Definition, Guide & Examples
• Survey Research | Definition, Examples & Methods
• Systematic Review | Definition, Example, & Guide
• Systematic Sampling | A Step-by-Step Guide with Examples
• Textual Analysis | Guide, 3 Approaches & Examples
• The 4 Types of Reliability in Research | Definitions & Examples
• The 4 Types of Validity in Research | Definitions & Examples
• Transcribing an Interview | 5 Steps & Transcription Software
• Triangulation in Research | Guide, Types, Examples
• Types of Interviews in Research | Guide & Examples
• Types of Research Designs Compared | Guide & Examples
• Types of Variables in Research & Statistics | Examples
• Unstructured Interview | Definition, Guide & Examples
• What Is a Case Study? | Definition, Examples & Methods
• What Is a Case-Control Study? | Definition & Examples
• What Is a Cohort Study? | Definition & Examples
• What Is a Conceptual Framework? | Tips & Examples
• What Is a Controlled Experiment? | Definitions & Examples
• What Is a Double-Barreled Question?
• What Is a Focus Group? | Step-by-Step Guide & Examples
• What Is a Likert Scale? | Guide & Examples
• What Is a Prospective Cohort Study? | Definition & Examples
• What Is a Retrospective Cohort Study? | Definition & Examples
• What Is Action Research? | Definition & Examples
• What Is an Observational Study? | Guide & Examples
• What Is Concurrent Validity? | Definition & Examples
• What Is Content Validity? | Definition & Examples
• What Is Convenience Sampling? | Definition & Examples
• What Is Convergent Validity? | Definition & Examples
• What Is Criterion Validity? | Definition & Examples
• What Is Data Cleansing? | Definition, Guide & Examples
• What Is Deductive Reasoning? | Explanation & Examples
• What Is Discriminant Validity? | Definition & Example
• What Is Ecological Validity? | Definition & Examples
• What Is Ethnography? | Definition, Guide & Examples
• What Is Face Validity? | Guide, Definition & Examples
• What Is Non-Probability Sampling? | Types & Examples
• What Is Participant Observation? | Definition & Examples
• What Is Peer Review? | Types & Examples
• What Is Predictive Validity? | Examples & Definition
• What Is Probability Sampling? | Types & Examples
• What Is Purposive Sampling? | Definition & Examples
• What Is Qualitative Observation? | Definition & Examples
• What Is Qualitative Research? | Methods & Examples
• What Is Quantitative Observation? | Definition & Examples
• What Is Quantitative Research? | Definition, Uses & Methods

"I thought AI Proofreading was useless but.."

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Rank the following mobile brand in order of your preference, the most preferred mobile brand should be ranked one, the second most preferred should be ranked two and so on.

To know the descriptive analysis of the ranking scale, watch the video.

Interval Scale

It is the next higher level of measurement. It overcomes the limitation of ordinal scale measurement. In the ordinal scale, the magnitude of the difference is unimportant, but here on an interval scale, the magnitude of the difference is important. In the interval scale, the difference between the two variables has a meaningful interpretation. In the interval scale, the difference between variables is equal distance. The distance between any two adjacent attributes is called an  interval , and intervals are always equal.

Examples of Interval Scale data connection using questionnaire.

How likely do you recommend our product to your friends or relatives?

Likert scale is a tool to collect interval data, which is developed by Rensis Likert

To know the descriptive analysis of the interval scale , watch the video.

Ratio Scale

Ratio scale is purely quantitative.  Among the four levels of measurement, ratio scale is the most precise.  The score of zero in ratio scale is not arbitrary compared to the other three scales.

This is the unique quality of ratio scale data.  It represents all the characteristics of nominal, ordinal, and interval scales.  Examples of ratio scales are age, wight, height, income, distance etc.

Examples of Interval Scale (Ranking Scale) data connection using questionnaire.

Specify you monthly income :

How many students are there in your institution? :

Number of departments in your organisation :

Thank You !!! Successfully Registered !!!

Recent posts.

• UGC NET : Research Methodology 003
• UGC NET – Part I : Research Aptitude 002
• UGC NET – Part I : Research Aptitude 001
• Demographic Data: Role and importance in market research.

Sign Up For Our Newsletter!

No Spam! Just information about our new publication.

You have Successfully Subscribed!

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Psychological Measurement

Learning Objectives

• Define reliability, including the different types and how they are assessed.
• Define validity, including the different types and how they are assessed.
• Describe the kinds of evidence that would be relevant to assessing the reliability and validity of a particular measure.

Again, measurement involves assigning scores to individuals so that they represent some characteristic of the individuals. But how do researchers know that the scores actually represent the characteristic, especially when it is a construct like intelligence, self-esteem, depression, or working memory capacity? The answer is that they conduct research using the measure to confirm that the scores make sense based on their understanding of the construct being measured. This is an extremely important point. Psychologists do not simply  assume  that their measures work. Instead, they collect data to demonstrate  that they work. If their research does not demonstrate that a measure works, they stop using it.

As an informal example, imagine that you have been dieting for a month. Your clothes seem to be fitting more loosely, and several friends have asked if you have lost weight. If at this point your bathroom scale indicated that you had lost 10 pounds, this would make sense and you would continue to use the scale. But if it indicated that you had gained 10 pounds, you would rightly conclude that it was broken and either fix it or get rid of it. In evaluating a measurement method, psychologists consider two general dimensions: reliability and validity.

Reliability

Reliability  refers to the consistency of a measure. Psychologists consider three types of consistency: over time (test-retest reliability), across items (internal consistency), and across different researchers (inter-rater reliability).

Test-Retest Reliability

When researchers measure a construct that they assume to be consistent across time, then the scores they obtain should also be consistent across time.  Test-retest reliability  is the extent to which this is actually the case. For example, intelligence is generally thought to be consistent across time. A person who is highly intelligent today will be highly intelligent next week. This means that any good measure of intelligence should produce roughly the same scores for this individual next week as it does today. Clearly, a measure that produces highly inconsistent scores over time cannot be a very good measure of a construct that is supposed to be consistent.

Assessing test-retest reliability requires using the measure on a group of people at one time, using it again on the  same group of people at a later time, and then looking at the test-retest correlation between the two sets of scores. This is typically done by graphing the data in a scatterplot and computing the correlation coefficient. Figure 4.2 shows the correlation between two sets of scores of several university students on the Rosenberg Self-Esteem Scale, administered two times, a week apart. The correlation coefficient for these data is +.95. In general, a test-retest correlation of +.80 or greater is considered to indicate good reliability.

Again, high test-retest correlations make sense when the construct being measured is assumed to be consistent over time, which is the case for intelligence, self-esteem, and the Big Five personality dimensions. But other constructs are not assumed to be stable over time. The very nature of mood, for example, is that it changes. So a measure of mood that produced a low test-retest correlation over a period of a month would not be a cause for concern.

Internal Consistency

Another kind of reliability is  internal consistency , which is the consistency of people’s responses across the items on a multiple-item measure. In general, all the items on such measures are supposed to reflect the same underlying construct, so people’s scores on those items should be correlated with each other. On the Rosenberg Self-Esteem Scale, people who agree that they are a person of worth should tend to agree that they have a number of good qualities. If people’s responses to the different items are not correlated with each other, then it would no longer make sense to claim that they are all measuring the same underlying construct. This is as true for behavioral and physiological measures as for self-report measures. For example, people might make a series of bets in a simulated game of roulette as a measure of their level of risk seeking. This measure would be internally consistent to the extent that individual participants’ bets were consistently high or low across trials.

Like test-retest reliability, internal consistency can only be assessed by collecting and analyzing data. One approach is to look at a  split-half correlation . This involves splitting the items into two sets, such as the first and second halves of the items or the even- and odd-numbered items. Then a score is computed for each set of items, and the relationship between the two sets of scores is examined. For example, Figure 4.3 shows the split-half correlation between several university students’ scores on the even-numbered items and their scores on the odd-numbered items of the Rosenberg Self-Esteem Scale. The correlation coefficient for these data is +.88. A split-half correlation of +.80 or greater is generally considered good internal consistency.

Perhaps the most common measure of internal consistency used by researchers in psychology is a statistic called  Cronbach’s α  (the Greek letter alpha). Conceptually, α is the mean of all possible split-half correlations for a set of items. For example, there are 252 ways to split a set of 10 items into two sets of five. Cronbach’s α would be the mean of the 252 split-half correlations. Note that this is not how α is actually computed, but it is a correct way of interpreting the meaning of this statistic. Again, a value of +.80 or greater is generally taken to indicate good internal consistency.

Interrater Reliability

Many behavioral measures involve significant judgment on the part of an observer or a rater.  Inter-rater reliability  is the extent to which different observers are consistent in their judgments. For example, if you were interested in measuring university students’ social skills, you could make video recordings of them as they interacted with another student whom they are meeting for the first time. Then you could have two or more observers watch the videos and rate each student’s level of social skills. To the extent that each participant does, in fact, have some level of social skills that can be detected by an attentive observer, different observers’ ratings should be highly correlated with each other. Inter-rater reliability would also have been measured in Bandura’s Bobo doll study. In this case, the observers’ ratings of how many acts of aggression a particular child committed while playing with the Bobo doll should have been highly positively correlated. Interrater reliability is often assessed using Cronbach’s α when the judgments are quantitative or an analogous statistic called Cohen’s κ (the Greek letter kappa) when they are categorical.

Validity  is the extent to which the scores from a measure represent the variable they are intended to. But how do researchers make this judgment? We have already considered one factor that they take into account—reliability. When a measure has good test-retest reliability and internal consistency, researchers should be more confident that the scores represent what they are supposed to. There has to be more to it, however, because a measure can be extremely reliable but have no validity whatsoever. As an absurd example, imagine someone who believes that people’s index finger length reflects their self-esteem and therefore tries to measure self-esteem by holding a ruler up to people’s index fingers. Although this measure would have extremely good test-retest reliability, it would have absolutely no validity. The fact that one person’s index finger is a centimeter longer than another’s would indicate nothing about which one had higher self-esteem.

Discussions of validity usually divide it into several distinct “types.” But a good way to interpret these types is that they are other kinds of evidence—in addition to reliability—that should be taken into account when judging the validity of a measure. Here we consider three basic kinds: face validity, content validity, and criterion validity.

Face Validity

Face validity  is the extent to which a measurement method appears “on its face” to measure the construct of interest. Most people would expect a self-esteem questionnaire to include items about whether they see themselves as a person of worth and whether they think they have good qualities. So a questionnaire that included these kinds of items would have good face validity. The finger-length method of measuring self-esteem, on the other hand, seems to have nothing to do with self-esteem and therefore has poor face validity. Although face validity can be assessed quantitatively—for example, by having a large sample of people rate a measure in terms of whether it appears to measure what it is intended to—it is usually assessed informally.

Face validity is at best a very weak kind of evidence that a measurement method is measuring what it is supposed to. One reason is that it is based on people’s intuitions about human behavior, which are frequently wrong. It is also the case that many established measures in psychology work quite well despite lacking face validity. The Minnesota Multiphasic Personality Inventory-2 (MMPI-2) measures many personality characteristics and disorders by having people decide whether each of over 567 different statements applies to them—where many of the statements do not have any obvious relationship to the construct that they measure. For example, the items “I enjoy detective or mystery stories” and “The sight of blood doesn’t frighten me or make me sick” both measure the suppression of aggression. In this case, it is not the participants’ literal answers to these questions that are of interest, but rather whether the pattern of the participants’ responses to a series of questions matches those of individuals who tend to suppress their aggression.

Content Validity

Content validity  is the extent to which a measure “covers” the construct of interest. For example, if a researcher conceptually defines test anxiety as involving both sympathetic nervous system activation (leading to nervous feelings) and negative thoughts, then his measure of test anxiety should include items about both nervous feelings and negative thoughts. Or consider that attitudes are usually defined as involving thoughts, feelings, and actions toward something. By this conceptual definition, a person has a positive attitude toward exercise to the extent that they think positive thoughts about exercising, feels good about exercising, and actually exercises. So to have good content validity, a measure of people’s attitudes toward exercise would have to reflect all three of these aspects. Like face validity, content validity is not usually assessed quantitatively. Instead, it is assessed by carefully checking the measurement method against the conceptual definition of the construct.

Criterion Validity

Criterion validity   is the extent to which people’s scores on a measure are correlated with other variables (known as criteria) that one would expect them to be correlated with. For example, people’s scores on a new measure of test anxiety should be negatively correlated with their performance on an important school exam. If it were found that people’s scores were in fact negatively correlated with their exam performance, then this would be a piece of evidence that these scores really represent people’s test anxiety. But if it were found that people scored equally well on the exam regardless of their test anxiety scores, then this would cast doubt on the validity of the measure.

A criterion can be any variable that one has reason to think should be correlated with the construct being measured, and there will usually be many of them. For example, one would expect test anxiety scores to be negatively correlated with exam performance and course grades and positively correlated with general anxiety and with blood pressure during an exam. Or imagine that a researcher develops a new measure of physical risk taking. People’s scores on this measure should be correlated with their participation in “extreme” activities such as snowboarding and rock climbing, the number of speeding tickets they have received, and even the number of broken bones they have had over the years. When the criterion is measured at the same time as the construct, criterion validity is referred to as concurrent validity ; however, when the criterion is measured at some point in the future (after the construct has been measured), it is referred to as predictive validity (because scores on the measure have “predicted” a future outcome).

Criteria can also include other measures of the same construct. For example, one would expect new measures of test anxiety or physical risk taking to be positively correlated with existing established measures of the same constructs. This is known as convergent validity .

Assessing convergent validity requires collecting data using the measure. Researchers John Cacioppo and Richard Petty did this when they created their self-report Need for Cognition Scale to measure how much people value and engage in thinking (Cacioppo & Petty, 1982) [1] . In a series of studies, they showed that people’s scores were positively correlated with their scores on a standardized academic achievement test, and that their scores were negatively correlated with their scores on a measure of dogmatism (which represents a tendency toward obedience). In the years since it was created, the Need for Cognition Scale has been used in literally hundreds of studies and has been shown to be correlated with a wide variety of other variables, including the effectiveness of an advertisement, interest in politics, and juror decisions (Petty, Briñol, Loersch, & McCaslin, 2009) [2] .

Discriminant Validity

Discriminant validity , on the other hand, is the extent to which scores on a measure are not correlated with measures of variables that are conceptually distinct. For example, self-esteem is a general attitude toward the self that is fairly stable over time. It is not the same as mood, which is how good or bad one happens to be feeling right now. So people’s scores on a new measure of self-esteem should not be very highly correlated with their moods. If the new measure of self-esteem were highly correlated with a measure of mood, it could be argued that the new measure is not really measuring self-esteem; it is measuring mood instead.

When they created the Need for Cognition Scale, Cacioppo and Petty also provided evidence of discriminant validity by showing that people’s scores were not correlated with certain other variables. For example, they found only a weak correlation between people’s need for cognition and a measure of their cognitive style—the extent to which they tend to think analytically by breaking ideas into smaller parts or holistically in terms of “the big picture.” They also found no correlation between people’s need for cognition and measures of their test anxiety and their tendency to respond in socially desirable ways. All these low correlations provide evidence that the measure is reflecting a conceptually distinct construct.

• Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42 , 116–131. ↵
• Petty, R. E, Briñol, P., Loersch, C., & McCaslin, M. J. (2009). The need for cognition. In M. R. Leary & R. H. Hoyle (Eds.), Handbook of individual differences in social behavior (pp. 318–329). New York, NY: Guilford Press. ↵

Refers to the consistency of a measure.

When researchers measure a construct that they assume to be consistent across time, then the scores they obtain should also be consistent across time.

The consistency of people’s responses across the items on a multiple-item measure.

A score that is derived by splitting the items into two sets and examining the relationship between the two sets of scores in order to assess the internal consistency of a measure.

A statistic that measures internal consistency among items in a measure.

The extent to which different observers are consistent in their judgments.

The extent to which the scores from a measure represent the variable they are intended to.

The extent to which a measurement method appears, on superficial examination, to measure the construct of interest.

The extent to which a measure reflects all aspects of the construct of interest.

The extent to which people’s scores on a measure are correlated with other variables (known as criteria) that one would expect them to be correlated with.

A variable that theoretically should be correlated with the construct being measured (plural: criteria).

A form of criterion validity, where the criterion is measured at the same time (concurrently) as the construct.

A form of validity whereby the criterion is measured at some point in the future (after the construct has been measured), to determine that the construct "predicts" the criterion.

A form of criterion validity whereby new measures are correlated with existing established measures of the same construct.

The extent to which scores on a measure of a construct are not correlated with measures of other, conceptually distinct, constructs and thus discriminate between them.

Research Methods in Psychology Copyright © 2019 by Rajiv S. Jhangiani, I-Chant A. Chiang, Carrie Cuttler, & Dana C. Leighton is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

MBA Knowledge Base

Business • Management • Technology

Home » Research Methodology » Measurement Scales in Research Methodology

Measurement Scales in Research Methodology

Certain research data are qualitative in nature. Data on attitude, opinion or behavior of employees, customers, and sales persons etc., are qualitative. The terms “attitude” and “opinion” have frequently been differentiated in psychological and sociological investigations. A commonly drawn distinction has been to view an attitude as a predisposition to act in a certain way and an opinion as a verbalization of the attitude. Thus, a statement by a respondent that he prefers viewing color to black-and-white television programs would be an opinion expressing one aspect of the respondent’s attitude toward color television. Motivation , commitment, satisfaction, leadership effectiveness, etc involve attitude measurement based on revealed opinions. These qualitative data require measurement scales for being measured.

Types of Measurement Scales used in Research

There are four different scales of measurement used in research ; nominal, ordinal, interval and ratio. The rules used to assign numerals objects define the kind of scale and level of measurement. A brief account of each scaling type is given below;

• Nominal Scales : Nominal scale is the simplest form of measurement. A variable measured on a nominal is one which is divided into two or more categories, for example, gender is categorized as male or female, a question as to whether a family owns a iPhone can be answered ‘Yes’ or ‘No’. It is simply a sorting operation in which all individuals or units or answers can be placed in one category or another (i.e. the categories are exhaustive). The essential characteristic of a nominal scale is that in terms of a given variable, one individual is different from another and the categories are discriminate (i.e. the categories are mutually exclusive). This characteristic of classification if fundamental to all scales of measurement. Nominal scales that consist only two categories such as female-male, agree-disagree,aware-unaware, yes-no, are unique and are called dichotomous scales. Such dichotomous nominal scales are important to researchers because the numerical labels for the two scale categories can be treated as though they are of interval scale value.
• Ordinal Scales : Ordinal scales have all the properties of a nominal scale, but, in addition, categories can be ordered along a continuum, in terms of a given criterion. Given three categories A, B and C, on an ordinal scale, one might be able to say, for e.g., that A is greater than B and B is greater than C. If numerals are assigned to ordinal scale categories, the numerals serve only as ranks for ordering observations from least to most in terms of the characteristic measured and they do not indicate the distance between scale that organizes observations in terms of categories such as high, medium and low or strongly agree, agree, not sure, disagree, and strong disagree.
• Interval Scales : Interval scales incorporate all the properties of nominal and ordinal scales and in addition, indicate the distance or interval between the categories. In formal terms one can say not only that A is greater than B and B is greater than C but also that (A-B)=(B-C) or (A-C)=(A-B)+(B-C). Examples of interval scale include age, income and investments. However, an interval scale is one where there is no absolute zero point. It can be placed anywhere along a continuum e.g., the age can be between 20 to 60 years and need not necessarily start from 0 years. This makes ratio comparison, that A is twice that of B or so wrong.
• Ratio Scales : A special form of interval scale is the ratio scale which differs in that it has a true zero point or a point at which the characteristic that is measured is presumed to be absent. Examples of ratio scales include, weight, length, income, expenditure and others. In each there is a concept of zero income, zero weight, etc. Since ratio scales represent a refinement of interval scales, generally these scales are not distinguished and both the terms are used inter-changeably.

Each of the above four types of scales have a unique method of measurement. Both nominal and ordinal scales consist of discrete number of categories to which numbers are assigned. Thus, a variable such as number of families owning a BMW or iPhone can only take values of 0, 1, 2 3 4 etc. It cannot have values such as 1.5 or 2.5 as the units are integers and indivisible. But interval and ratio scales take any value between two integers, as the variables are continuous. For example, given any ages however close, it is possible to find a third which lies in between. Interval and ratio scales are superior to normal and ordinal scales and a wealth of statistical tools can be employed in their analysis. The different statistical tools are related to these different measurement  scales in research , in that there is usually a correspondence between mathematical assumptions of the statistical tool and the assumptions of the scale of measurement . Care must be always taken to match the tools used with the scale of measurement of variables and to use a method which implies a higher scale measurement than   the variable allows.

Related Posts:

• Guide to the Development of Research Questionnaires
• Social Research - Definition, Steps and Objectives
• The Purpose of Research
• Interview Method of Data Collection in Research
• Schedule as a Data Collection Technique in Research
• The Literature Review in Research
• The Basic Types of Research
• In-Depth Interviewing Techniques
• Classification and Tabulation of Data in Research
• Tips for Effective Research Interviews

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Int J Epidemiol

Reflection on modern methods: five myths about measurement error in epidemiological research

Maarten van smeden.

1 Department of Clinical Epidemiology, Leiden University Medical Center, Leiden, The Netherlands

Timothy L Lash

2 Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA, USA

Rolf H H Groenwold

3 Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands

Epidemiologists are often confronted with datasets to analyse which contain measurement error due to, for instance, mistaken data entries, inaccurate recordings and measurement instrument or procedural errors. If the effect of measurement error is misjudged, the data analyses are hampered and the validity of the study’s inferences may be affected. In this paper, we describe five myths that contribute to misjudgments about measurement error, regarding expected structure, impact and solutions to mitigate the problems resulting from mismeasurements. The aim is to clarify these measurement error misconceptions. We show that the influence of measurement error in an epidemiological data analysis can play out in ways that go beyond simple heuristics, such as heuristics about whether or not to expect attenuation of the effect estimates. Whereas we encourage epidemiologists to deliberate about the structure and potential impact of measurement error in their analyses, we also recommend exercising restraint when making claims about the magnitude or even direction of effect of measurement error if not accompanied by statistical measurement error corrections or quantitative bias analysis. Suggestions for alleviating the problems or investigating the structure and magnitude of measurement error are given.

Introduction

Epidemiologists are often confronted with datasets to analyse which contain data errors. Settings where such errors occur include, but are not limited to, measurements of dietary intake, 1–3 blood pressure, 4–6 physical activity, 7–9 exposure to air pollutants, 10–12 medical treatments received 13–15 and diagnostic coding. 16–18 Mismeasurements and misclassifications, hereinafter collectively referred to as measurement error, are mentioned as a potential study limitation in approximately half of all the original research articles published in highly ranked epidemiology journals. 19 , 20 The actual burden of measurement error in all of epidemiological research is likely to be even higher. 19

Despite the attention given to measurement error in the discussion and limitation sections of many published articles, empirical investigations of measurement error in epidemiological research remain rare. 19–21 Notably, statistical methods that aim to investigate the impact of measurement error or alleviate their consequences for the epidemiological data analyses at hand continue to be rarely used. Authors instead appear to rely on simple heuristics about measurement error structure, e.g. whether or not measurement error is expected to be nondifferential, and impact on epidemiological data analyses, e.g. whether or not the measurement error creates bias towards null effects, despite ample warnings that such heuristics are oversimplified and often wrong. 22–27 As we will illustrate, the impact of measurement error often plays out in a way that counters common conceptions.

In this paper we describe and reply to five myths about measurement error which we perceive to exist in epidemiology. It is our intention to clarify misconceptions about mechanisms and bias introduced by measurement error in epidemiological data analyses, and to encourage researchers to use analytical approaches to investigate measurement error. We first briefly characterize measurement error variants before discussing the five measurement error myths.

Measurement error: settings and terminology

Throughout this article (except for myth 5) we assume that measurement error is to occur in a non-experimental epidemiological study designed to estimate an exposure effect, that is the relationship between a single exposure (denoted by A , e.g. adherence versus non-adherence to a 30-day physical exercise programme) and a single outcome (denoted by Y , e.g. post-programme body weight in kg), statistically controlled for one or more confounding variables (e.g. age, sex and pre-programme body weight). Some simplifying assumptions are made for brevity of this presentation.

It is assumed that the confounders are adequately controlled for by conventional multivariable linear, risk or rate regression (e.g. ordinary least squares, logistic regression, Cox or Poisson regression), or by an exposure model (e.g. propensity score analysis). 28 Besides measurement error, other sources that could affect inferences about the exposure effect are assumed not to play an important role, e.g. no selection bias. 29 Unless otherwise specified, we assume that the measurement error has the classical additive form: Observation = Truth + Error, shortened as O = T + E, where the mean of E is assumed to be zero, meaning that the Observations do not systematically differ from the Truth. Alternative and more complex models for measurement error relevant to epidemiological research, such as systematic and Berkson error models, 30 are not considered here. We also assume that there is an agreed underlying reality (T) of the phenomenon that one aims to measure and an imperfectly measured representation of that reality (O) subject to measurement error (E). This identifiable measurement error assumption is often reasonable in epidemiological research but may be less so in some circumstances, for instance with the measurement of complex diseases. For in-depth discussion on the theories of measurement we refer to the work by Hand. 31

In the simplest setting, we may assume (or in rare cases, know) that the measurement error is univariate, that is to say that measurement error occurs only in a single variable. Measurement error in an exposure variable ( E A ) is further commonly classified as nondifferential if error in the measurement error is independent of the true value of the outcome ( T Y ,   i.e. E A ⫫ T Y ) and differential otherwise. Likewise, error in the measurement of the outcome ( E Y ) is said to be nondifferential only if the error is independent of the true value of the exposure ( T A ,   i.e. E Y ⫫ T A ), 32–35 or in an alternative notation if for each possible outcome status y of T Y , Pr( O Y = y | T Y = y, T A = a ) = c ,   where   c is a constant for all possible values a of T A . (Note that nondifferential error sometimes refers to a broader definition that includes covariates; for a single covariate L with true values, the assumption can then be specified by Pr( O Y = y | T Y = y, T A = a ,   T l = l ) = c .)

Reconsider the hypothetical example of the relation between the exposure physical exercise programme adherence and post-programme body weight. Differential exposure measurement error would mean that mismeasurement of programme adherence occurs more frequently or more infrequently in individuals with a higher (or lower) post-programme body weight. For the binary exposure programme adherence, nondifferential error simplifies to assuming that the sensitivity and specificity of measured programme adherence are the same for all possible true values of post-programme body weight.

If two or more variables in the analysis are subject to measurement error, we may speak of multivariate (or joint) measurement error. When two variables are measured with error, measurement error (which may be differential or nondifferential for either variable) is said to be independent if the errors in the one error-prone variable are statistically unrelated to the errors in the other error-prone variable and dependent otherwise, i.e. multivariate measurement error in A and Y is said to be independent if E Y ⫫   E A . 32–34 Dependent measurement error may for instance occur in an exposure variable when error on exposure becomes more (or less) likely for units that are misclassified on the outcome variable. In the hypothetical example, if both adherence to a physical activity programme and post-programme body weight were self-reported, we may expect error in both exposure and outcome measurements. Further, we may also anticipate that respondents who misreport adherence to the exercise programme also misreport their post-programme body weight, which would result in multivariate dependent measurement error.

Five myths about measurement error

In this section we discuss five myths about measurement in epidemiological research, in particular as regards the impact of measurement error on study results (myths 2 and 5), solutions to mitigate the impact (myths 1 and 4) and the mechanisms of measurement error (myth 3). Each myth is accompanied by a short reply that is substantiated in a more detailed explanation.

Myth 1: measurement error can be compensated for by large numbers of observations

Reply: no, a large number of observations does not resolve the most serious consequences of measurement error in epidemiological data analyses. These remain regardless of the sample size.

Explanation: one intuition is that measurement error distorts the true existing statistical relationships between variables, analogous to noise (the measurement error) lowering the ability to detect a signal (the true statistical relationships) that can be picked up from the data. Continuing on this thought, increasing the sample size would amplify the signal to become better distinguishable from the noise, thereby compensating for the measurement error. Unfortunately, this signal to noise analogy rarely applies to epidemiological studies.

Measurement error can have impact on epidemiological data analyses in at least three ways, as summarized by the Triple Whammy of Measurement Error. 30 First, measurement error can create a bias in the measures of the exposure effect estimate. Second, measurement error affects the precision of the exposure effect estimate, often by reducing it, reflected in larger standard errors and widening of confidence intervals for the exposure effect estimates, and a lower statistical power of the significance test for the null exposure effect. Biased exposure effect estimates may, however, be accompanied by smaller rather than larger expected standard errors and conserved statistical power. 36 Third, measurement error can mask the features of data, such as non-linear functional relationships between the exposure and outcome variables. Figure 1 illustrates feature masking by univariate nondifferential measurement error.

Illustration of Whammy 3: Measurement error may mask a functional relation. True model: outcome = 3/2*exposure^2 + e, e∼N(0, 1). Measurement error model: observed exposure value = true exposure value + exposure error. Line is a LOESS curve.

Key Messages

• The strength and direction of effect of measurement error on any given epidemiological data analysis is generally difficult to conceive without appropriate quantitative investigations.
• Frequently used heuristics about measurement error structure (e.g. nondifferential error) and impact (e.g. bias towards null) are often wrong and encourage a tolerant attitude towards neglecting measurement error in epidemiological research.
• Statistical approaches to mitigate the effects of unavoidable measurement error should be more widely adopted.

With sample size increasing and assuming all else remains equal, exposure effect estimates will on average be closer to their limiting expected values, while not necessarily closer in distance to their respective true values. A large sample size thus improves the assurance that the exposure effect estimate comports to the expected value under the measurement error mechanism, affecting the second Whammy of Measurement Error (precision) but not directly the first (bias) ( Figure 2 ). A larger sample size may thus compensate for the loss in precision and power which is due to the presence of measurement error. The compensation needed for studies with data that contain measurement error can be a 50-fold or more increase of the sample size when the reliability of measurement is low. 37 , 38 In consequence, even a dataset of a spectacularly large size containing measurement errors may or may not yield more precise estimates and more powerful testing than a much smaller dataset without measurement error.

Illustration of Whammy 1 (bias) and Whammy 2 (precision, see width confidence intervals) of measurement error. Dashed is regression line without measurement error (Truth), solid line is regression line with measurement error (With ME). N = sample size. Lines, point estimates and confidence intervals based on 5000 replicate Monte Carlo simulations (Truth: outcome = exposure + e, e∼N(0, 0.6), exposure∼N(0, 1), With ME: Truth + er, er∼N(0, 0.5)). Plotted points the first single simulation replicate.

Myth 2: the exposure effect is underestimated when variables are measured with error

Reply: no, an exposure effect can be over- or underestimated in the presence of measurement error depending on which variables are affected, how measurement error is structured and the expression of other biasing and data sampling factors. In contrast to common understanding, even univariate nondifferential exposure measurement error, which is often expected to bias towards the null, may yield a bias away from null.

Explanation: more than a century ago, Spearman 39 derived his measurement error attenuation formula for a pairwise correlation coefficient between two variables wherein at least one of the variables was measured with error. Spearman identified that this correlation coefficient would on average be underestimated by a predictable amount if the reliability of the measurements was known. This systematic bias towards the null value, also known as regression dilution bias, attenuation to the null and Hausman’s iron law, is now known to apply beyond simple correlations to other types of data and analyses. 25 , 40–42

It is, however, an overstatement to say that—by iron law—the exposure estimates are underestimated in any given epidemiological study analysing data with measurement error. For instance, selective filtering of statistically significant exposure effects in measurement error-contaminated data estimated with low precision is likely to lead to substantial overestimation of the exposure effect estimates for the variables that withstand the significance test. 26 Even if one is willing to assume that measurement error is the only biasing factor, a simplifying assumption that we make in this article for illustration purposes only, statistical estimation is subject to sampling variability. The distance of the exposure effect estimate relative to its true value varies from dataset to dataset. In a particular dataset with measurement error, exposure effects may be overestimated only due to sampling variability, illustrations of which are found in Hutcheon et al. and Jurek et al. 22 , 43 Hence, a defining characteristic of the iron law is that it applies to averages of exposure effect estimates, e.g. after many hypothetical replications of a study with the same measures.

This is not to argue that measurement error in itself cannot, in principle, produce a bias in a predictable direction. The iron law does come with many exceptions. For instance, the law does not apply uniformly to univariate differential measurement error in any variable (which may produce bias in the exposure effect estimate away or towards null 27 , 44–46 ), nor to univariate nondifferential error in the outcome variable (which may not affect bias in the exposure effect in case the outcome is continuous 25 ) nor to univariate nondifferential measurement error in one of the confounding variables (which may create a bias in the exposure effect estimate away or towards null due to residual confounding 27 , 44 , 47 ). For multivariate measurement error in any combination of exposure, outcome and confounders, bias in the exposure can be in either direction, with the exception of (strictly) independent and nondifferential measurement error in dichotomous exposure and outcome. 34

There are also exceptions to the iron law in cases of univariate nondifferential exposure measurement error. Particularly, nondifferential misclassification of a polytomous exposure variable (i.e. with more than two categories) may create bias that is away from null for some of the exposure categories and towards null for others. 29 , 48 , 49 Measurement error of any kind, including nondifferential exposure measurement error, also hampers the evaluation of exposure effect modification, 44 , 50 interaction 51 , 52 and mediation, 53 , 54 creating bias away or towards null.

Myth 3: exposure measurement error is nondifferential if measurements are taken without knowledge of the outcome

Reply: no, exposure measurement error can be differential even if the measurement is taken without knowledge of the outcome.

Explanation: differential exposure measurement error is of particular concern because of its potentially strong biasing effects on exposure effect estimates. 29 , 35 , 55 Differential error is a common suspect in retrospective studies where knowledge of the outcome status can influence the accuracy of measurement of the exposure. For instance, in a case-control study with self-reported exposure data, cases may recall or report their exposure status differently from controls, creating an expectation of differential exposure measurement error. Differential exposure measurement error may also arise due to bias by interviewers or care providers who are not blinded to the outcome status, or by use of different methods of exposure measurement for cases and controls.

Differential exposure measurement error is often not suspected in prospective data collection settings where the measurement of exposure precedes measurement of the outcome. Measurement of exposure before the outcome is nonetheless insufficient to guarantee that exposure measurement error is nondifferential. For example, as White 56 noted: in a prospective design, differential measurement error in the exposure ‘family history of disease’ may be due to a more accurate recollection of family history among individuals with strong family history who are at a higher risk of the disease outcome. The nondifferential exposure measurement assumption is violated, as the measurement error in exposure is not independent of the true value of the outcome (i.e. E A ⫫ T Y is violated) despite that the exposure was measured before the outcome could have been observed.

Measurement error structures are also not invariant to discretization and collapsing of categories. For instance, discretization of a continuous exposure variable measured with nondifferential error into a categorical exposure variable can create differential exposure measurement error within the discrete categories. 48 , 57 , 58 A clear numerical example of this is given in Wacholder et al. , their second table. 59 Although discretization in broader categories may be perceived as more robust to measurement error-induced fluctuations, the possible change in the mechanism of the error may do more harm than good for the estimation of the exposure effect.

Myth 4: measurement error can be prevented but not mitigated in epidemiological data analyses

Reply: no, statistical methods for measurement error bias-corrections can be used in the presence of measurement error provided that data are available on the structure and magnitude of measurement error from an internal or external source. This often requires planning of a measurement error correction approach or quantitative bias analysis, which may require additional data to be collected.

Explanation: a number of approaches have been proposed which examine and correct for the bias due to measurement error in analysis of an epidemiological study, typically focusing on adjusting for the bias in the exposure effect estimator and corresponding confidence intervals. These approaches include: likelihood based approaches 60 ; score function methods 61 ; method-of-moments corrections 62 ; simulation extrapolation 63 ; regression calibration 64 ; latent class modelling 65 ; structural equation models with latent variables 66 ; multiple imputation for measurement error correction 67 ; inverse probability weighing approaches 68 and Bayesian analyses. 69 Comprehensive text books 30 , 62 , 69–71 and a measurement error corrections tutorial 72 are available. Some applied examples are given in Table 1 .

Examples of measurement error corrections, models and bias analyses

Measurement error correction methods 79 require information to be gathered about the structure and magnitude of measurement error. These data may come from within the study data that are currently analysed (i.e. internal validation data) or from other data (i.e. external validation data). If an accurate measurement procedure exists (i.e. a gold standard), either external or on an internal subset, a measurement error validation study can be conducted to extract information about the structure and magnitude of measurement error. If no such gold standard measurement procedure exists, a reliability study can replicate measurements of the same imperfect measure (e.g. multiple blood pressure measurements within the same unit), or alternatively, observations on multiple measures that aim to measure the same phenomenon (e.g. different diagnostic tests for the same disease within the same unit). Different measurement error correction methods, and the ability of the bias correction to return an estimate nearer to the truth, may be more or less applicable depending on the data sources available for the measurement error correction.

The impact of measurement error on study results can also be investigated by a quantitative bias analysis, 79 , 80 even in the absence of reliable information about structure and magnitude. In brief, a quantitative bias analysis is a sensitivity analysis that simulates the effect of measurement error assuming a certain structure and magnitude of that error. Since some degree of uncertainty about measurement error generally remains, in particular about error structure, sensitivity analyses can also be useful following the application of measurement error correction methods mentioned above.

Myth 5: certain types of epidemiological research are unaffected by measurement error

Reply: no, measurement error can affect all types of epidemiological research.

Explanation: measurement error affects epidemiology undoubtedly beyond the settings we have discussed thus far, i.e. studies of a single exposure and outcome variable statistically controlled for confounding. For instance, measurement error has also been linked to issues with data analyses in the context of record linkage, 81 , 82 time series analyses, 10 , 11 Mendelian randomization studies, 83 genome-wide association studies, 84 environmental epidemiology, 85 negative control studies, 86 diagnostic accuracy studies, 87 , 88 disease prevalence studies, 89 prediction modelling studies 90 , 91 and randomized trials. 92

It is worth noting that types of data and analyses can be differently affected by measurement error. This means that even when measurement error is similar in structure and magnitude, the error can have a different impact depending on the analyses conducted. For example, consider a two-arm randomized trial with univariate differential measurement error in the outcome variable. If it is not possible to blind patients and providers to the treatment assignment, then the accuracy of assessment of some outcomes may depend on the assigned treatment arm. In this setting, bias in the exposure (treatment) effect estimate can be in either direction and inflate or deflate both Type I and Type II error for the null hypothesis significance test of no effect of treatment. 92 The impact the measurement error has on the inferences made from the trial’s results depends on whether the trial is a superiority, equivalence or non-inferiority trial. 93

Concluding remarks

Our discussion of five measurement error-related myths adds to an already extensive literature that has warned against the detrimental effects of neglected measurement error, a problem that is widely acknowledged to be ubiquitous in epidemiology. We suspect that these persistent myths have contributed to the tolerant attitude towards neglecting measurement error found in most of the applied epidemiological literature, as evidenced by the slow uptake of quantitative approaches that mitigate or investigate measurement error.

We have shown in this paper that the effect that measurement error can have on a data analysis is often counter-intuitive. Whereas we encourage epidemiologists to deliberate about the structure and potential impact of measurement error in their analyses, for instance via graphical approaches such as causal diagrams, 33 , 45 , 94 we also recommend exercising restraint when making claims about the magnitude or even direction of bias of measurement error if not accompanied by analytical investigations. With the increase of collection and use of epidemiological data that are primarily collected without a specific research question in mind, such as routine care data, 95 we anticipate that attention to measurement error and approaches to mitigate it will only become more important.

R.H.H.G. was funded by the Netherlands Organization for Scientific Research (NWO-Vidi project 917.16.430). T.L.L. was supported, in part, by R01LM013049 from the US National Library of Medicine.

Acknowledgements

We are most grateful for the comments received on our initial draft offered by Dr Katherine Ahrens, Dr Kathryn Snow, the Berlin Epidemiological Methods Colloquium journal club and the anonymous reviewers commissioned by the journal.

Author Contributions

M.S. conceptualized the manuscript. T.L. and R.G. provided inputs and revised the manuscript. All authors read and approved the final manuscript.

Conflict of interest : None declared.