research methodology an introduction to scientific thinking and practice

• Search Menu
• Browse content in Arts and Humanities
• Browse content in Archaeology
• Anglo-Saxon and Medieval Archaeology
• Archaeological Methodology and Techniques
• Archaeology by Region
• Archaeology of Religion
• Archaeology of Trade and Exchange
• Biblical Archaeology
• Contemporary and Public Archaeology
• Environmental Archaeology
• Historical Archaeology
• History and Theory of Archaeology
• Industrial Archaeology
• Landscape Archaeology
• Mortuary Archaeology
• Prehistoric Archaeology
• Underwater Archaeology
• Urban Archaeology
• Zooarchaeology
• Browse content in Architecture
• Architectural Structure and Design
• History of Architecture
• Residential and Domestic Buildings
• Theory of Architecture
• Browse content in Art
• Art Subjects and Themes
• History of Art
• Industrial and Commercial Art
• Theory of Art
• Biographical Studies
• Byzantine Studies
• Browse content in Classical Studies
• Classical History
• Classical Philosophy
• Classical Mythology
• Classical Literature
• Classical Reception
• Classical Art and Architecture
• Classical Oratory and Rhetoric
• Greek and Roman Epigraphy
• Greek and Roman Law
• Greek and Roman Papyrology
• Greek and Roman Archaeology
• Late Antiquity
• Religion in the Ancient World
• Digital Humanities
• Browse content in History
• Colonialism and Imperialism
• Diplomatic History
• Environmental History
• Genealogy, Heraldry, Names, and Honours
• Genocide and Ethnic Cleansing
• Historical Geography
• History by Period
• History of Emotions
• History of Agriculture
• History of Education
• History of Gender and Sexuality
• Industrial History
• Intellectual History
• International History
• Labour History
• Legal and Constitutional History
• Local and Family History
• Maritime History
• Military History
• National Liberation and Post-Colonialism
• Oral History
• Political History
• Public History
• Regional and National History
• Revolutions and Rebellions
• Slavery and Abolition of Slavery
• Social and Cultural History
• Theory, Methods, and Historiography
• Urban History
• World History
• Browse content in Language Teaching and Learning
• Language Learning (Specific Skills)
• Language Teaching Theory and Methods
• Browse content in Linguistics
• Applied Linguistics
• Cognitive Linguistics
• Computational Linguistics
• Forensic Linguistics
• Grammar, Syntax and Morphology
• Historical and Diachronic Linguistics
• History of English
• Language Acquisition
• Language Evolution
• Language Reference
• Language Variation
• Language Families
• Lexicography
• Linguistic Anthropology
• Linguistic Theories
• Linguistic Typology
• Phonetics and Phonology
• Psycholinguistics
• Sociolinguistics
• Translation and Interpretation
• Writing Systems
• Browse content in Literature
• Bibliography
• Children's Literature Studies
• Literary Studies (Asian)
• Literary Studies (European)
• Literary Studies (Eco-criticism)
• Literary Studies (Romanticism)
• Literary Studies (American)
• Literary Studies (Modernism)
• Literary Studies - World
• Literary Studies (1500 to 1800)
• Literary Studies (19th Century)
• Literary Studies (20th Century onwards)
• Literary Studies (African American Literature)
• Literary Studies (British and Irish)
• Literary Studies (Early and Medieval)
• Literary Studies (Fiction, Novelists, and Prose Writers)
• Literary Studies (Gender Studies)
• Literary Studies (Graphic Novels)
• Literary Studies (History of the Book)
• Literary Studies (Plays and Playwrights)
• Literary Studies (Poetry and Poets)
• Literary Studies (Postcolonial Literature)
• Literary Studies (Queer Studies)
• Literary Studies (Science Fiction)
• Literary Studies (Travel Literature)
• Literary Studies (War Literature)
• Literary Studies (Women's Writing)
• Literary Theory and Cultural Studies
• Mythology and Folklore
• Shakespeare Studies and Criticism
• Browse content in Media Studies
• Browse content in Music
• Applied Music
• Dance and Music
• Ethics in Music
• Ethnomusicology
• Gender and Sexuality in Music
• Medicine and Music
• Music Cultures
• Music and Religion
• Music and Media
• Music and Culture
• Music Education and Pedagogy
• Music Theory and Analysis
• Musical Scores, Lyrics, and Libretti
• Musical Structures, Styles, and Techniques
• Musicology and Music History
• Performance Practice and Studies
• Race and Ethnicity in Music
• Sound Studies
• Browse content in Performing Arts
• Browse content in Philosophy
• Aesthetics and Philosophy of Art
• Epistemology
• Feminist Philosophy
• History of Western Philosophy
• Metaphysics
• Moral Philosophy
• Non-Western Philosophy
• Philosophy of Science
• Philosophy of Language
• Philosophy of Mind
• Philosophy of Perception
• Philosophy of Action
• Philosophy of Law
• Philosophy of Religion
• Philosophy of Mathematics and Logic
• Practical Ethics
• Social and Political Philosophy
• Browse content in Religion
• Biblical Studies
• Christianity
• East Asian Religions
• History of Religion
• Judaism and Jewish Studies
• Qumran Studies
• Religion and Education
• Religion and Health
• Religion and Politics
• Religion and Science
• Religion and Law
• Religion and Art, Literature, and Music
• Religious Studies
• Browse content in Society and Culture
• Cookery, Food, and Drink
• Cultural Studies
• Customs and Traditions
• Ethical Issues and Debates
• Hobbies, Games, Arts and Crafts
• Lifestyle, Home, and Garden
• Natural world, Country Life, and Pets
• Popular Beliefs and Controversial Knowledge
• Sports and Outdoor Recreation
• Technology and Society
• Travel and Holiday
• Visual Culture
• Browse content in Law
• Arbitration
• Browse content in Company and Commercial Law
• Commercial Law
• Company Law
• Browse content in Comparative Law
• Systems of Law
• Competition Law
• Browse content in Constitutional and Administrative Law
• Government Powers
• Judicial Review
• Local Government Law
• Military and Defence Law
• Parliamentary and Legislative Practice
• Construction Law
• Contract Law
• Browse content in Criminal Law
• Criminal Procedure
• Criminal Evidence Law
• Sentencing and Punishment
• Employment and Labour Law
• Environment and Energy Law
• Browse content in Financial Law
• Banking Law
• Insolvency Law
• History of Law
• Human Rights and Immigration
• Intellectual Property Law
• Browse content in International Law
• Private International Law and Conflict of Laws
• Public International Law
• IT and Communications Law
• Jurisprudence and Philosophy of Law
• Law and Politics
• Law and Society
• Browse content in Legal System and Practice
• Courts and Procedure
• Legal Skills and Practice
• Primary Sources of Law
• Regulation of Legal Profession
• Medical and Healthcare Law
• Browse content in Policing
• Criminal Investigation and Detection
• Police and Security Services
• Police Procedure and Law
• Police Regional Planning
• Browse content in Property Law
• Personal Property Law
• Study and Revision
• Terrorism and National Security Law
• Browse content in Trusts Law
• Wills and Probate or Succession
• Browse content in Medicine and Health
• Browse content in Allied Health Professions
• Arts Therapies
• Clinical Science
• Dietetics and Nutrition
• Occupational Therapy
• Operating Department Practice
• Physiotherapy
• Radiography
• Speech and Language Therapy
• Browse content in Anaesthetics
• General Anaesthesia
• Neuroanaesthesia
• Browse content in Clinical Medicine
• Acute Medicine
• Cardiovascular Medicine
• Clinical Genetics
• Clinical Pharmacology and Therapeutics
• Dermatology
• Endocrinology and Diabetes
• Gastroenterology
• Genito-urinary Medicine
• Geriatric Medicine
• Infectious Diseases
• Medical Toxicology
• Medical Oncology
• Pain Medicine
• Palliative Medicine
• Rehabilitation Medicine
• Respiratory Medicine and Pulmonology
• Rheumatology
• Sleep Medicine
• Sports and Exercise Medicine
• Clinical Neuroscience
• Community Medical Services
• Critical Care
• Emergency Medicine
• Forensic Medicine
• Haematology
• History of Medicine
• Browse content in Medical Dentistry
• Oral and Maxillofacial Surgery
• Paediatric Dentistry
• Restorative Dentistry and Orthodontics
• Surgical Dentistry
• Browse content in Medical Skills
• Clinical Skills
• Communication Skills
• Nursing Skills
• Surgical Skills
• Medical Ethics
• Medical Statistics and Methodology
• Browse content in Neurology
• Clinical Neurophysiology
• Neuropathology
• Nursing Studies
• Browse content in Obstetrics and Gynaecology
• Gynaecology
• Occupational Medicine
• Ophthalmology
• Otolaryngology (ENT)
• Browse content in Paediatrics
• Neonatology
• Browse content in Pathology
• Chemical Pathology
• Clinical Cytogenetics and Molecular Genetics
• Histopathology
• Medical Microbiology and Virology
• Patient Education and Information
• Browse content in Pharmacology
• Psychopharmacology
• Browse content in Popular Health
• Caring for Others
• Complementary and Alternative Medicine
• Self-help and Personal Development
• Browse content in Preclinical Medicine
• Cell Biology
• Molecular Biology and Genetics
• Reproduction, Growth and Development
• Primary Care
• Professional Development in Medicine
• Browse content in Psychiatry
• Addiction Medicine
• Child and Adolescent Psychiatry
• Forensic Psychiatry
• Learning Disabilities
• Old Age Psychiatry
• Psychotherapy
• Browse content in Public Health and Epidemiology
• Epidemiology
• Public Health
• Browse content in Radiology
• Clinical Radiology
• Interventional Radiology
• Nuclear Medicine
• Radiation Oncology
• Reproductive Medicine
• Browse content in Surgery
• Cardiothoracic Surgery
• Gastro-intestinal and Colorectal Surgery
• General Surgery
• Neurosurgery
• Paediatric Surgery
• Peri-operative Care
• Plastic and Reconstructive Surgery
• Surgical Oncology
• Transplant Surgery
• Trauma and Orthopaedic Surgery
• Vascular Surgery
• Browse content in Science and Mathematics
• Browse content in Biological Sciences
• Aquatic Biology
• Biochemistry
• Bioinformatics and Computational Biology
• Developmental Biology
• Ecology and Conservation
• Evolutionary Biology
• Genetics and Genomics
• Microbiology
• Molecular and Cell Biology
• Natural History
• Plant Sciences and Forestry
• Research Methods in Life Sciences
• Structural Biology
• Systems Biology
• Zoology and Animal Sciences
• Browse content in Chemistry
• Analytical Chemistry
• Computational Chemistry
• Crystallography
• Environmental Chemistry
• Industrial Chemistry
• Inorganic Chemistry
• Materials Chemistry
• Medicinal Chemistry
• Mineralogy and Gems
• Organic Chemistry
• Physical Chemistry
• Polymer Chemistry
• Study and Communication Skills in Chemistry
• Theoretical Chemistry
• Browse content in Computer Science
• Artificial Intelligence
• Computer Architecture and Logic Design
• Game Studies
• Human-Computer Interaction
• Mathematical Theory of Computation
• Programming Languages
• Software Engineering
• Systems Analysis and Design
• Virtual Reality
• Browse content in Computing
• Business Applications
• Computer Security
• Computer Games
• Computer Networking and Communications
• Digital Lifestyle
• Graphical and Digital Media Applications
• Operating Systems
• Browse content in Earth Sciences and Geography
• Atmospheric Sciences
• Environmental Geography
• Geology and the Lithosphere
• Maps and Map-making
• Meteorology and Climatology
• Oceanography and Hydrology
• Palaeontology
• Physical Geography and Topography
• Regional Geography
• Soil Science
• Urban Geography
• Browse content in Engineering and Technology
• Agriculture and Farming
• Biological Engineering
• Civil Engineering, Surveying, and Building
• Electronics and Communications Engineering
• Energy Technology
• Engineering (General)
• Environmental Science, Engineering, and Technology
• History of Engineering and Technology
• Mechanical Engineering and Materials
• Technology of Industrial Chemistry
• Transport Technology and Trades
• Browse content in Environmental Science
• Applied Ecology (Environmental Science)
• Conservation of the Environment (Environmental Science)
• Environmental Sustainability
• Environmentalist Thought and Ideology (Environmental Science)
• Management of Land and Natural Resources (Environmental Science)
• Natural Disasters (Environmental Science)
• Nuclear Issues (Environmental Science)
• Pollution and Threats to the Environment (Environmental Science)
• Social Impact of Environmental Issues (Environmental Science)
• History of Science and Technology
• Browse content in Materials Science
• Ceramics and Glasses
• Composite Materials
• Metals, Alloying, and Corrosion
• Nanotechnology
• Browse content in Mathematics
• Applied Mathematics
• Biomathematics and Statistics
• History of Mathematics
• Mathematical Education
• Mathematical Finance
• Mathematical Analysis
• Numerical and Computational Mathematics
• Probability and Statistics
• Pure Mathematics
• Browse content in Neuroscience
• Cognition and Behavioural Neuroscience
• Development of the Nervous System
• Disorders of the Nervous System
• History of Neuroscience
• Invertebrate Neurobiology
• Molecular and Cellular Systems
• Neuroendocrinology and Autonomic Nervous System
• Neuroscientific Techniques
• Sensory and Motor Systems
• Browse content in Physics
• Astronomy and Astrophysics
• Atomic, Molecular, and Optical Physics
• Biological and Medical Physics
• Classical Mechanics
• Computational Physics
• Condensed Matter Physics
• Electromagnetism, Optics, and Acoustics
• History of Physics
• Mathematical and Statistical Physics
• Measurement Science
• Nuclear Physics
• Particles and Fields
• Plasma Physics
• Quantum Physics
• Relativity and Gravitation
• Semiconductor and Mesoscopic Physics
• Browse content in Psychology
• Affective Sciences
• Clinical Psychology
• Cognitive Psychology
• Cognitive Neuroscience
• Criminal and Forensic Psychology
• Developmental Psychology
• Educational Psychology
• Evolutionary Psychology
• Health Psychology
• History and Systems in Psychology
• Music Psychology
• Neuropsychology
• Organizational Psychology
• Psychological Assessment and Testing
• Psychology of Human-Technology Interaction
• Psychology Professional Development and Training
• Research Methods in Psychology
• Social Psychology
• Browse content in Social Sciences
• Browse content in Anthropology
• Anthropology of Religion
• Human Evolution
• Medical Anthropology
• Physical Anthropology
• Regional Anthropology
• Social and Cultural Anthropology
• Theory and Practice of Anthropology
• Browse content in Business and Management
• Business Strategy
• Business Ethics
• Business History
• Business and Government
• Business and Technology
• Business and the Environment
• Comparative Management
• Corporate Governance
• Corporate Social Responsibility
• Entrepreneurship
• Health Management
• Human Resource Management
• Industrial and Employment Relations
• Industry Studies
• Information and Communication Technologies
• International Business
• Knowledge Management
• Management and Management Techniques
• Operations Management
• Organizational Theory and Behaviour
• Pensions and Pension Management
• Public and Nonprofit Management
• Strategic Management
• Supply Chain Management
• Browse content in Criminology and Criminal Justice
• Criminal Justice
• Criminology
• Forms of Crime
• International and Comparative Criminology
• Youth Violence and Juvenile Justice
• Development Studies
• Browse content in Economics
• Agricultural, Environmental, and Natural Resource Economics
• Asian Economics
• Behavioural Finance
• Behavioural Economics and Neuroeconomics
• Econometrics and Mathematical Economics
• Economic Systems
• Economic History
• Economic Methodology
• Economic Development and Growth
• Financial Markets
• Financial Institutions and Services
• General Economics and Teaching
• Health, Education, and Welfare
• History of Economic Thought
• International Economics
• Labour and Demographic Economics
• Law and Economics
• Macroeconomics and Monetary Economics
• Microeconomics
• Public Economics
• Urban, Rural, and Regional Economics
• Welfare Economics
• Browse content in Education
• Adult Education and Continuous Learning
• Care and Counselling of Students
• Early Childhood and Elementary Education
• Educational Equipment and Technology
• Educational Strategies and Policy
• Higher and Further Education
• Organization and Management of Education
• Philosophy and Theory of Education
• Schools Studies
• Secondary Education
• Teaching of a Specific Subject
• Teaching of Specific Groups and Special Educational Needs
• Teaching Skills and Techniques
• Browse content in Environment
• Applied Ecology (Social Science)
• Climate Change
• Conservation of the Environment (Social Science)
• Environmentalist Thought and Ideology (Social Science)
• Natural Disasters (Environment)
• Social Impact of Environmental Issues (Social Science)
• Browse content in Human Geography
• Cultural Geography
• Economic Geography
• Political Geography
• Browse content in Interdisciplinary Studies
• Communication Studies
• Museums, Libraries, and Information Sciences
• Browse content in Politics
• African Politics
• Asian Politics
• Chinese Politics
• Comparative Politics
• Conflict Politics
• Elections and Electoral Studies
• Environmental Politics
• European Union
• Foreign Policy
• Gender and Politics
• Human Rights and Politics
• Indian Politics
• International Relations
• International Organization (Politics)
• International Political Economy
• Irish Politics
• Latin American Politics
• Middle Eastern Politics
• Political Methodology
• Political Communication
• Political Philosophy
• Political Sociology
• Political Behaviour
• Political Economy
• Political Institutions
• Political Theory
• Politics and Law
• Public Administration
• Public Policy
• Quantitative Political Methodology
• Regional Political Studies
• Russian Politics
• Security Studies
• State and Local Government
• UK Politics
• US Politics
• Browse content in Regional and Area Studies
• African Studies
• Asian Studies
• East Asian Studies
• Japanese Studies
• Latin American Studies
• Middle Eastern Studies
• Native American Studies
• Scottish Studies
• Browse content in Research and Information
• Research Methods
• Browse content in Social Work
• Addictions and Substance Misuse
• Adoption and Fostering
• Care of the Elderly
• Child and Adolescent Social Work
• Couple and Family Social Work
• Developmental and Physical Disabilities Social Work
• Direct Practice and Clinical Social Work
• Emergency Services
• Human Behaviour and the Social Environment
• International and Global Issues in Social Work
• Mental and Behavioural Health
• Social Justice and Human Rights
• Social Policy and Advocacy
• Social Work and Crime and Justice
• Social Work Macro Practice
• Social Work Practice Settings
• Social Work Research and Evidence-based Practice
• Welfare and Benefit Systems
• Browse content in Sociology
• Childhood Studies
• Community Development
• Comparative and Historical Sociology
• Economic Sociology
• Gender and Sexuality
• Gerontology and Ageing
• Health, Illness, and Medicine
• Marriage and the Family
• Migration Studies
• Occupations, Professions, and Work
• Organizations
• Population and Demography
• Race and Ethnicity
• Social Theory
• Social Movements and Social Change
• Social Research and Statistics
• Social Stratification, Inequality, and Mobility
• Sociology of Religion
• Sociology of Education
• Sport and Leisure
• Urban and Rural Studies
• Browse content in Warfare and Defence
• Defence Strategy, Planning, and Research
• Land Forces and Warfare
• Military Administration
• Military Life and Institutions
• Naval Forces and Warfare
• Other Warfare and Defence Issues
• Peace Studies and Conflict Resolution
• Weapons and Equipment

• < Previous chapter
• Next chapter >

35 Scientific Thinking and Reasoning

Kevin N. Dunbar, Department of Human Development and Quantitative Methodology, University of Maryland, College Park, MD

David Klahr, Department of Psychology, Carnegie Mellon University, Pittsburgh, PA

• Published: 21 November 2012
• Cite Icon Cite
• Permissions Icon Permissions

Scientific thinking refers to both thinking about the content of science and the set of reasoning processes that permeate the field of science: induction, deduction, experimental design, causal reasoning, concept formation, hypothesis testing, and so on. Here we cover both the history of research on scientific thinking and the different approaches that have been used, highlighting common themes that have emerged over the past 50 years of research. Future research will focus on the collaborative aspects of scientific thinking, on effective methods for teaching science, and on the neural underpinnings of the scientific mind.

There is no unitary activity called “scientific discovery”; there are activities of designing experiments, gathering data, inventing and developing observational instruments, formulating and modifying theories, deducing consequences from theories, making predictions from theories, testing theories, inducing regularities and invariants from data, discovering theoretical constructs, and others. — Simon, Langley, & Bradshaw, 1981 , p. 2

What Is Scientific Thinking and Reasoning?

There are two kinds of thinking we call “scientific.” The first, and most obvious, is thinking about the content of science. People are engaged in scientific thinking when they are reasoning about such entities and processes as force, mass, energy, equilibrium, magnetism, atoms, photosynthesis, radiation, geology, or astrophysics (and, of course, cognitive psychology!). The second kind of scientific thinking includes the set of reasoning processes that permeate the field of science: induction, deduction, experimental design, causal reasoning, concept formation, hypothesis testing, and so on. However, these reasoning processes are not unique to scientific thinking: They are the very same processes involved in everyday thinking. As Einstein put it:

The scientific way of forming concepts differs from that which we use in our daily life, not basically, but merely in the more precise definition of concepts and conclusions; more painstaking and systematic choice of experimental material, and greater logical economy. (The Common Language of Science, 1941, reprinted in Einstein, 1950 , p. 98)

Nearly 40 years after Einstein's remarkably insightful statement, Francis Crick offered a similar perspective: that great discoveries in science result not from extraordinary mental processes, but rather from rather common ones. The greatness of the discovery lies in the thing discovered.

I think what needs to be emphasized about the discovery of the double helix is that the path to it was, scientifically speaking, fairly commonplace. What was important was not the way it was discovered , but the object discovered—the structure of DNA itself. (Crick, 1988 , p. 67; emphasis added)

Under this view, scientific thinking involves the same general-purpose cognitive processes—such as induction, deduction, analogy, problem solving, and causal reasoning—that humans apply in nonscientific domains. These processes are covered in several different chapters of this handbook: Rips, Smith, & Medin, Chapter 11 on induction; Evans, Chapter 8 on deduction; Holyoak, Chapter 13 on analogy; Bassok & Novick, Chapter 21 on problem solving; and Cheng & Buehner, Chapter 12 on causality. One might question the claim that the highly specialized procedures associated with doing science in the “real world” can be understood by investigating the thinking processes used in laboratory studies of the sort described in this volume. However, when the focus is on major scientific breakthroughs, rather than on the more routine, incremental progress in a field, the psychology of problem solving provides a rich source of ideas about how such discoveries might occur. As Simon and his colleagues put it:

It is understandable, if ironic, that ‘normal’ science fits … the description of expert problem solving, while ‘revolutionary’ science fits the description of problem solving by novices. It is understandable because scientific activity, particularly at the revolutionary end of the continuum, is concerned with the discovery of new truths, not with the application of truths that are already well-known … it is basically a journey into unmapped terrain. Consequently, it is mainly characterized, as is novice problem solving, by trial-and-error search. The search may be highly selective—but it reaches its goal only after many halts, turnings, and back-trackings. (Simon, Langley, & Bradshaw, 1981 , p. 5)

The research literature on scientific thinking can be roughly categorized according to the two types of scientific thinking listed in the opening paragraph of this chapter: (1) One category focuses on thinking that directly involves scientific content . Such research ranges from studies of young children reasoning about the sun-moon-earth system (Vosniadou & Brewer, 1992 ) to college students reasoning about chemical equilibrium (Davenport, Yaron, Klahr, & Koedinger, 2008 ), to research that investigates collaborative problem solving by world-class researchers in real-world molecular biology labs (Dunbar, 1995 ). (2) The other category focuses on “general” cognitive processes, but it tends to do so by analyzing people's problem-solving behavior when they are presented with relatively complex situations that involve the integration and coordination of several different types of processes, and that are designed to capture some essential features of “real-world” science in the psychology laboratory (Bruner, Goodnow, & Austin, 1956 ; Klahr & Dunbar, 1988 ; Mynatt, Doherty, & Tweney, 1977 ).

There are a number of overlapping research traditions that have been used to investigate scientific thinking. We will cover both the history of research on scientific thinking and the different approaches that have been used, highlighting common themes that have emerged over the past 50 years of research.

A Brief History of Research on Scientific Thinking

Science is often considered one of the hallmarks of the human species, along with art and literature. Illuminating the thought processes used in science thus reveal key aspects of the human mind. The thought processes underlying scientific thinking have fascinated both scientists and nonscientists because the products of science have transformed our world and because the process of discovery is shrouded in mystery. Scientists talk of the chance discovery, the flash of insight, the years of perspiration, and the voyage of discovery. These images of science have helped make the mental processes underlying the discovery process intriguing to cognitive scientists as they attempt to uncover what really goes on inside the scientific mind and how scientists really think. Furthermore, the possibilities that scientists can be taught to think better by avoiding mistakes that have been clearly identified in research on scientific thinking, and that their scientific process could be partially automated, makes scientific thinking a topic of enduring interest.

The cognitive processes underlying scientific discovery and day-to-day scientific thinking have been a topic of intense scrutiny and speculation for almost 400 years (e.g., Bacon, 1620 ; Galilei 1638 ; Klahr 2000 ; Tweney, Doherty, & Mynatt, 1981 ). Understanding the nature of scientific thinking has been a central issue not only for our understanding of science but also for our understating of what it is to be human. Bacon's Novumm Organum in 1620 sketched out some of the key features of the ways that experiments are designed and data interpreted. Over the ensuing 400 years philosophers and scientists vigorously debated about the appropriate methods that scientists should use (see Giere, 1993 ). These debates over the appropriate methods for science typically resulted in the espousal of a particular type of reasoning method, such as induction or deduction. It was not until the Gestalt psychologists began working on the nature of human problem solving, during the 1940s, that experimental psychologists began to investigate the cognitive processes underlying scientific thinking and reasoning.

The Gestalt psychologist Max Wertheimer pioneered the investigation of scientific thinking (of the first type described earlier: thinking about scientific content ) in his landmark book Productive Thinking (Wertheimer, 1945 ). Wertheimer spent a considerable amount of time corresponding with Albert Einstein, attempting to discover how Einstein generated the concept of relativity. Wertheimer argued that Einstein had to overcome the structure of Newtonian physics at each step in his theorizing, and the ways that Einstein actually achieved this restructuring were articulated in terms of Gestalt theories. (For a recent and different account of how Einstein made his discovery, see Galison, 2003 .) We will see later how this process of overcoming alternative theories is an obstacle that both scientists and nonscientists need to deal with when evaluating and theorizing about the world.

One of the first investigations of scientific thinking of the second type (i.e., collections of general-purpose processes operating on complex, abstract, components of scientific thought) was carried out by Jerome Bruner and his colleagues at Harvard (Bruner et al., 1956 ). They argued that a key activity engaged in by scientists is to determine whether a particular instance is a member of a category. For example, a scientist might want to discover which substances undergo fission when bombarded by neutrons and which substances do not. Here, scientists have to discover the attributes that make a substance undergo fission. Bruner et al. saw scientific thinking as the testing of hypotheses and the collecting of data with the end goal of determining whether something is a member of a category. They invented a paradigm where people were required to formulate hypotheses and collect data that test their hypotheses. In one type of experiment, the participants were shown a card such as one with two borders and three green triangles. The participants were asked to determine the concept that this card represented by choosing other cards and getting feedback from the experimenter as to whether the chosen card was an example of the concept. In this case the participant may have thought that the concept was green and chosen a card with two green squares and one border. If the underlying concept was green, then the experimenter would say that the card was an example of the concept. In terms of scientific thinking, choosing a new card is akin to conducting an experiment, and the feedback from the experimenter is similar to knowing whether a hypothesis is confirmed or disconfirmed. Using this approach, Bruner et al. identified a number of strategies that people use to formulate and test hypotheses. They found that a key factor determining which hypothesis-testing strategy that people use is the amount of memory capacity that the strategy takes up (see also Morrison & Knowlton, Chapter 6 ; Medin et al., Chapter 11 ). Another key factor that they discovered was that it was much more difficult for people to discover negative concepts (e.g., not blue) than positive concepts (e.g., blue). Although Bruner et al.'s research is most commonly viewed as work on concepts, they saw their work as uncovering a key component of scientific thinking.

A second early line of research on scientific thinking was developed by Peter Wason and his colleagues (Wason, 1968 ). Like Bruner et al., Wason saw a key component of scientific thinking as being the testing of hypotheses. Whereas Bruner et al. focused on the different types of strategies that people use to formulate hypotheses, Wason focused on whether people adopt a strategy of trying to confirm or disconfirm their hypotheses. Using Popper's ( 1959 ) theory that scientists should try and falsify rather than confirm their hypotheses, Wason devised a deceptively simple task in which participants were given three numbers, such as 2-4-6, and were asked to discover the rule underlying the three numbers. Participants were asked to generate other triads of numbers and the experimenter would tell the participant whether the triad was consistent or inconsistent with the rule. They were told that when they were sure they knew what the rule was they should state it. Most participants began the experiment by thinking that the rule was even numbers increasing by 2. They then attempted to confirm their hypothesis by generating a triad like 8-10-12, then 14-16-18. These triads are consistent with the rule and the participants were told yes, that the triads were indeed consistent with the rule. However, when they proposed the rule—even numbers increasing by 2—they were told that the rule was incorrect. The correct rule was numbers of increasing magnitude! From this research, Wason concluded that people try to confirm their hypotheses, whereas normatively speaking, they should try to disconfirm their hypotheses. One implication of this research is that confirmation bias is not just restricted to scientists but is a general human tendency.

It was not until the 1970s that a general account of scientific reasoning was proposed. Herbert Simon, often in collaboration with Allan Newell, proposed that scientific thinking is a form of problem solving. He proposed that problem solving is a search in a problem space. Newell and Simon's theory of problem solving is discussed in many places in this handbook, usually in the context of specific problems (see especially Bassok & Novick, Chapter 21 ). Herbert Simon, however, devoted considerable time to understanding many different scientific discoveries and scientific reasoning processes. The common thread in his research was that scientific thinking and discovery is not a mysterious magical process but a process of problem solving in which clear heuristics are used. Simon's goal was to articulate the heuristics that scientists use in their research at a fine-grained level. By constructing computer programs that simulated the process of several major scientific discoveries, Simon and colleagues were able to articulate the specific computations that scientists could have used in making those discoveries (Langley, Simon, Bradshaw, & Zytkow, 1987 ; see section on “Computational Approaches to Scientific Thinking”). Particularly influential was Simon and Lea's ( 1974 ) work demonstrating that concept formation and induction consist of a search in two problem spaces: a space of instances and a space of rules. This idea has influenced problem-solving accounts of scientific thinking that will be discussed in the next section.

Overall, the work of Bruner, Wason, and Simon laid the foundations for contemporary research on scientific thinking. Early research on scientific thinking is summarized in Tweney, Doherty and Mynatt's 1981 book On Scientific Thinking , where they sketched out many of the themes that have dominated research on scientific thinking over the past few decades. Other more recent books such as Cognitive Models of Science (Giere, 1993 ), Exploring Science (Klahr, 2000 ), Cognitive Basis of Science (Carruthers, Stich, & Siegal, 2002 ), and New Directions in Scientific and Technical Thinking (Gorman, Kincannon, Gooding, & Tweney, 2004 ) provide detailed analyses of different aspects of scientific discovery. Another important collection is Vosnadiau's handbook on conceptual change research (Vosniadou, 2008 ). In this chapter, we discuss the main approaches that have been used to investigate scientific thinking.

How does one go about investigating the many different aspects of scientific thinking? One common approach to the study of the scientific mind has been to investigate several key aspects of scientific thinking using abstract tasks designed to mimic some essential characteristics of “real-world” science. There have been numerous methodologies that have been used to analyze the genesis of scientific concepts, theories, hypotheses, and experiments. Researchers have used experiments, verbal protocols, computer programs, and analyzed particular scientific discoveries. A more recent development has been to increase the ecological validity of such research by investigating scientists as they reason “live” (in vivo studies of scientific thinking) in their own laboratories (Dunbar, 1995 , 2002 ). From a “Thinking and Reasoning” standpoint the major aspects of scientific thinking that have been most actively investigated are problem solving, analogical reasoning, hypothesis testing, conceptual change, collaborative reasoning, inductive reasoning, and deductive reasoning.

Scientific Thinking as Problem Solving

One of the primary goals of accounts of scientific thinking has been to provide an overarching framework to understand the scientific mind. One framework that has had a great influence in cognitive science is that scientific thinking and scientific discovery can be conceived as a form of problem solving. As noted in the opening section of this chapter, Simon ( 1977 ; Simon, Langley, & Bradshaw, 1981 ) argued that both scientific thinking in general and problem solving in particular could be thought of as a search in a problem space. A problem space consists of all the possible states of a problem and all the operations that a problem solver can use to get from one state to the next. According to this view, by characterizing the types of representations and procedures that people use to get from one state to another it is possible to understand scientific thinking. Thus, scientific thinking can be characterized as a search in various problem spaces (Simon, 1977 ). Simon investigated a number of scientific discoveries by bringing participants into the laboratory, providing the participants with the data that a scientist had access to, and getting the participants to reason about the data and rediscover a scientific concept. He then analyzed the verbal protocols that participants generated and mapped out the types of problem spaces that the participants search in (e.g., Qin & Simon, 1990 ). Kulkarni and Simon ( 1988 ) used a more historical approach to uncover the problem-solving heuristics that Krebs used in his discovery of the urea cycle. Kulkarni and Simon analyzed Krebs's diaries and proposed a set of problem-solving heuristics that he used in his research. They then built a computer program incorporating the heuristics and biological knowledge that Krebs had before he made his discoveries. Of particular importance are the search heuristics that the program uses, which include experimental proposal heuristics and data interpretation heuristics. A key heuristic was an unusualness heuristic that focused on unusual findings, which guided search through a space of theories and a space of experiments.

Klahr and Dunbar ( 1988 ) extended the search in a problem space approach and proposed that scientific thinking can be thought of as a search through two related spaces: an hypothesis space and an experiment space. Each problem space that a scientist uses will have its own types of representations and operators used to change the representations. Search in the hypothesis space constrains search in the experiment space. Klahr and Dunbar found that some participants move from the hypothesis space to the experiment space, whereas others move from the experiment space to the hypothesis space. These different types of searches lead to the proposal of different types of hypotheses and experiments. More recent work has extended the dual-space approach to include alternative problem-solving spaces, including those for data, instrumentation, and domain-specific knowledge (Klahr & Simon, 1999 ; Schunn & Klahr, 1995 , 1996 ).

Scientific Thinking as Hypothesis Testing

Many researchers have regarded testing specific hypotheses predicted by theories as one of the key attributes of scientific thinking. Hypothesis testing is the process of evaluating a proposition by collecting evidence regarding its truth. Experimental cognitive research on scientific thinking that specifically examines this issue has tended to fall into two broad classes of investigations. The first class is concerned with the types of reasoning that lead scientists astray, thus blocking scientific ingenuity. A large amount of research has been conducted on the potentially faulty reasoning strategies that both participants in experiments and scientists use, such as considering only one favored hypothesis at a time and how this prevents the scientists from making discoveries. The second class is concerned with uncovering the mental processes underlying the generation of new scientific hypotheses and concepts. This research has tended to focus on the use of analogy and imagery in science, as well as the use of specific types of problem-solving heuristics.

Turning first to investigations of what diminishes scientific creativity, philosophers, historians, and experimental psychologists have devoted a considerable amount of research to “confirmation bias.” This occurs when scientists only consider one hypothesis (typically the favored hypothesis) and ignore other alternative hypotheses or potentially relevant hypotheses. This important phenomenon can distort the design of experiments, formulation of theories, and interpretation of data. Beginning with the work of Wason ( 1968 ) and as discussed earlier, researchers have repeatedly shown that when participants are asked to design an experiment to test a hypothesis they will predominantly design experiments that they think will yield results consistent with the hypothesis. Using the 2-4-6 task mentioned earlier, Klayman and Ha ( 1987 ) showed that in situations where one's hypothesis is likely to be confirmed, seeking confirmation is a normatively incorrect strategy, whereas when the probability of confirming one's hypothesis is low, then attempting to confirm one's hypothesis can be an appropriate strategy. Historical analyses by Tweney ( 1989 ), concerning the way that Faraday made his discoveries, and experiments investigating people testing hypotheses, have revealed that people use a confirm early, disconfirm late strategy: When people initially generate or are given hypotheses, they try and gather evidence that is consistent with the hypothesis. Once enough evidence has been gathered, then people attempt to find the boundaries of their hypothesis and often try to disconfirm their hypotheses.

In an interesting variant on the confirmation bias paradigm, Gorman ( 1989 ) showed that when participants are told that there is the possibility of error in the data that they receive, participants assume that any data that are inconsistent with their favored hypothesis are due to error. Thus, the possibility of error “insulates” hypotheses against disconfirmation. This intriguing hypothesis has not been confirmed by other researchers (Penner & Klahr, 1996 ), but it is an intriguing hypothesis that warrants further investigation.

Confirmation bias is very difficult to overcome. Even when participants are asked to consider alternate hypotheses, they will often fail to conduct experiments that could potentially disconfirm their hypothesis. Tweney and his colleagues provide an excellent overview of this phenomenon in their classic monograph On Scientific Thinking (1981). The precise reasons for this type of block are still widely debated. Researchers such as Michael Doherty have argued that working memory limitations make it difficult for people to consider more than one hypothesis. Consistent with this view, Dunbar and Sussman ( 1995 ) have shown that when participants are asked to hold irrelevant items in working memory while testing hypotheses, the participants will be unable to switch hypotheses in the face of inconsistent evidence. While working memory limitations are involved in the phenomenon of confirmation bias, even groups of scientists can also display confirmation bias. For example, the controversy over cold fusion is an example of confirmation bias. Here, large groups of scientists had other hypotheses available to explain their data yet maintained their hypotheses in the face of other more standard alternative hypotheses. Mitroff ( 1974 ) provides some interesting examples of NASA scientists demonstrating confirmation bias, which highlight the roles of commitment and motivation in this process. See also MacPherson and Stanovich ( 2007 ) for specific strategies that can be used to overcome confirmation bias.

Causal Thinking in Science

Much of scientific thinking and scientific theory building pertains to the development of causal models between variables of interest. For example, do vaccines cause illnesses? Do carbon dioxide emissions cause global warming? Does water on a planet indicate that there is life on the planet? Scientists and nonscientists alike are constantly bombarded with statements regarding the causal relationship between such variables. How does one evaluate the status of such claims? What kinds of data are informative? How do scientists and nonscientists deal with data that are inconsistent with their theory?

A central issue in the causal reasoning literature, one that is directly relevant to scientific thinking, is the extent to which scientists and nonscientists alike are governed by the search for causal mechanisms (i.e., how a variable works) versus the search for statistical data (i.e., how often variables co-occur). This dichotomy can be boiled down to the search for qualitative versus quantitative information about the paradigm the scientist is investigating. Researchers from a number of cognitive psychology laboratories have found that people prefer to gather more information about an underlying mechanism than covariation between a cause and an effect (e.g., Ahn, Kalish, Medin, & Gelman, 1995 ). That is, the predominant strategy that students in simulations of scientific thinking use is to gather as much information as possible about how the objects under investigation work, rather than collecting large amounts of quantitative data to determine whether the observations hold across multiple samples. These findings suggest that a central component of scientific thinking may be to formulate explicit mechanistic causal models of scientific events.

One type of situation in which causal reasoning has been observed extensively is when scientists obtain unexpected findings. Both historical and naturalistic research has revealed that reasoning causally about unexpected findings plays a central role in science. Indeed, scientists themselves frequently state that a finding was due to chance or was unexpected. Given that claims of unexpected findings are such a frequent component of scientists' autobiographies and interviews in the media, Dunbar ( 1995 , 1997 , 1999 ; Dunbar & Fugelsang, 2005 ; Fugelsang, Stein, Green, & Dunbar, 2004 ) decided to investigate the ways that scientists deal with unexpected findings. In 1991–1992 Dunbar spent 1 year in three molecular biology laboratories and one immunology laboratory at a prestigious U.S. university. He used the weekly laboratory meeting as a source of data on scientific discovery and scientific reasoning. (He termed this type of study “in vivo” cognition.) When he looked at the types of findings that the scientists made, he found that over 50% of the findings were unexpected and that these scientists had evolved a number of effective strategies for dealing with such findings. One clear strategy was to reason causally about the findings: Scientists attempted to build causal models of their unexpected findings. This causal model building results in the extensive use of collaborative reasoning, analogical reasoning, and problem-solving heuristics (Dunbar, 1997 , 2001 ).

Many of the key unexpected findings that scientists reasoned about in the in vivo studies of scientific thinking were inconsistent with the scientists' preexisting causal models. A laboratory equivalent of the biology labs involved creating a situation in which students obtained unexpected findings that were inconsistent with their preexisting theories. Dunbar and Fugelsang ( 2005 ) examined this issue by creating a scientific causal thinking simulation where experimental outcomes were either expected or unexpected. Dunbar ( 1995 ) has called the study of people reasoning in a cognitive laboratory “in vitro” cognition. These investigators found that students spent considerably more time reasoning about unexpected findings than expected findings. In addition, when assessing the overall degree to which their hypothesis was supported or refuted, participants spent the majority of their time considering unexpected findings. An analysis of participants' verbal protocols indicates that much of this extra time was spent formulating causal models for the unexpected findings. Similarly, scientists spend more time considering unexpected than expected findings, and this time is devoted to building causal models (Dunbar & Fugelsang, 2004 ).

Scientists know that unexpected findings occur often, and they have developed many strategies to take advantage of their unexpected findings. One of the most important places that they anticipate the unexpected is in designing experiments (Baker & Dunbar, 2000 ). They build different causal models of their experiments incorporating many conditions and controls. These multiple conditions and controls allow unknown mechanisms to manifest themselves. Thus, rather than being the victims of the unexpected, they create opportunities for unexpected events to occur, and once these events do occur, they have causal models that allow them to determine exactly where in the causal chain their unexpected finding arose. The results of these in vivo and in vitro studies all point to a more complex and nuanced account of how scientists and nonscientists alike test and evaluate hypotheses about theories.

The Roles of Inductive, Abductive, and Deductive Thinking in Science

One of the most basic characteristics of science is that scientists assume that the universe that we live in follows predictable rules. Scientists reason using a variety of different strategies to make new scientific discoveries. Three frequently used types of reasoning strategies that scientists use are inductive, abductive, and deductive reasoning. In the case of inductive reasoning, a scientist may observe a series of events and try to discover a rule that governs the event. Once a rule is discovered, scientists can extrapolate from the rule to formulate theories of observed and yet-to-be-observed phenomena. One example is the discovery using inductive reasoning that a certain type of bacterium is a cause of many ulcers (Thagard, 1999 ). In a fascinating series of articles, Thagard documented the reasoning processes that Marshall and Warren went through in proposing this novel hypothesis. One key reasoning process was the use of induction by generalization. Marshall and Warren noted that almost all patients with gastric entritis had a spiral bacterium in their stomachs, and he formed the generalization that this bacterium is the cause of stomach ulcers. There are numerous other examples of induction by generalization in science, such as Tycho De Brea's induction about the motion of planets from his observations, Dalton's use of induction in chemistry, and the discovery of prions as the source of mad cow disease. Many theories of induction have used scientific discovery and reasoning as examples of this important reasoning process.

Another common type of inductive reasoning is to map a feature of one member of a category to another member of a category. This is called categorical induction. This type of induction is a way of projecting a known property of one item onto another item that is from the same category. Thus, knowing that the Rous Sarcoma virus is a retrovirus that uses RNA rather than DNA, a biologist might assume that another virus that is thought to be a retrovirus also uses RNA rather than DNA. While research on this type of induction typically has not been discussed in accounts of scientific thinking, this type of induction is common in science. For an influential contribution to this literature, see Smith, Shafir, and Osherson ( 1993 ), and for reviews of this literature see Heit ( 2000 ) and Medin et al. (Chapter 11 ).

While less commonly mentioned than inductive reasoning, abductive reasoning is an important form of reasoning that scientists use when they are seeking to propose explanations for events such as unexpected findings (see Lombrozo, Chapter 14 ; Magnani, et al., 2010 ). In Figure 35.1 , taken from King ( 2011 ), the differences between inductive, abductive, and deductive thinking are highlighted. In the case of abduction, the reasoner attempts to generate explanations of the form “if situation X had occurred, could it have produced the current evidence I am attempting to interpret?” (For an interesting of analysis of abductive reasoning see the brief paper by Klahr & Masnick, 2001 ). Of course, as in classical induction, such reasoning may produce a plausible account that is still not the correct one. However, abduction does involve the generation of new knowledge, and is thus also related to research on creativity.

The different processes underlying inductive, abductive, and deductive reasoning in science. (Figure reproduced from King 2011 ).)

Turning now to deductive thinking, many thinking processes that scientists adhere to follow traditional rules of deductive logic. These processes correspond to those conditions in which a hypothesis may lead to, or is deducible to, a conclusion. Though they are not always phrased in syllogistic form, deductive arguments can be phrased as “syllogisms,” or as brief, mathematical statements in which the premises lead to the conclusion. Deductive reasoning is an extremely important aspect of scientific thinking because it underlies a large component of how scientists conduct their research. By looking at many scientific discoveries, we can often see that deductive reasoning is at work. Deductive reasoning statements all contain information or rules that state an assumption about how the world works, as well as a conclusion that would necessarily follow from the rule. Numerous discoveries in physics such as the discovery of dark matter by Vera Rubin are based on deductions. In the dark matter case, Rubin measured galactic rotation curves and based on the differences between the predicted and observed angular motions of galaxies she deduced that the structure of the universe was uneven. This led her to propose that dark matter existed. In contemporary physics the CERN Large Hadron Collider is being used to search for the Higgs Boson. The Higgs Boson is a deductive prediction from contemporary physics. If the Higgs Boson is not found, it may lead to a radical revision of the nature of physics and a new understanding of mass (Hecht, 2011 ).

The Roles of Analogy in Scientific Thinking

One of the most widely mentioned reasoning processes used in science is analogy. Scientists use analogies to form a bridge between what they already know and what they are trying to explain, understand, or discover. In fact, many scientists have claimed that the making of certain analogies was instrumental in their making a scientific discovery, and almost all scientific autobiographies and biographies feature one particular analogy that is discussed in depth. Coupled with the fact that there has been an enormous research program on analogical thinking and reasoning (see Holyoak, Chapter 13 ), we now have a number of models and theories of analogical reasoning that suggest how analogy can play a role in scientific discovery (see Gentner, Holyoak, & Kokinov, 2001 ). By analyzing several major discoveries in the history of science, Thagard and Croft ( 1999 ), Nersessian ( 1999 , 2008 ), and Gentner and Jeziorski ( 1993 ) have all shown that analogical reasoning is a key aspect of scientific discovery.

Traditional accounts of analogy distinguish between two components of analogical reasoning: the target and the source (Holyoak, Chapter 13 ; Gentner 2010 ). The target is the concept or problem that a scientist is attempting to explain or solve. The source is another piece of knowledge that the scientist uses to understand the target or to explain the target to others. What the scientist does when he or she makes an analogy is to map features of the source onto features of the target. By mapping the features of the source onto the target, new features of the target may be discovered, or the features of the target may be rearranged so that a new concept is invented and a scientific discovery is made. For example, a common analogy that is used with computers is to describe a harmful piece of software as a computer virus. Once a piece of software is called a virus, people can map features of biological viruses, such as that it is small, spreads easily, self-replicates using a host, and causes damage. People not only map individual features of the source onto the target but also the systems of relations. For example, if a computer virus is similar to a biological virus, then an immune system can be created on computers that can protect computers from future variants of a virus. One of the reasons that scientific analogy is so powerful is that it can generate new knowledge, such as the creation of a computational immune system having many of the features of a real biological immune system. This analogy also leads to predictions that there will be newer computer viruses that are the computational equivalent of retroviruses, lacking DNA, or standard instructions, that will elude the computational immune system.

The process of making an analogy involves a number of key steps: retrieval of a source from memory, aligning the features of the source with those of the target, mapping features of the source onto those of the target, and possibly making new inferences about the target. Scientific discoveries are made when the source highlights a hitherto unknown feature of the target or restructures the target into a new set of relations. Interestingly, research on analogy has shown that participants do not easily use remote analogies (see Gentner et al., 1997 ; Holyoak & Thagard 1995 ). Participants in experiments tend to focus on the sharing of a superficial feature between the source and the target, rather than the relations among features. In his in vivo studies of science, Dunbar ( 1995 , 2001 , 2002 ) investigated the ways that scientists use analogies while they are conducting their research and found that scientists use both relational and superficial features when they make analogies. Whether they use superficial or relational features depends on their goals. If their goal is to fix a problem in an experiment, their analogies are based upon superficial features. However, if their goal is to formulate hypotheses, they focus on analogies based upon sets of relations. One important difference between scientists and participants in experiments is that the scientists have deep relational knowledge of the processes that they are investigating and can hence use this relational knowledge to make analogies (see Holyoak, Chapter 13 for a thorough review of analogical reasoning).

Are scientific analogies always useful? Sometimes analogies can lead scientists and students astray. For example, Evelyn Fox-Keller ( 1985 ) shows how an analogy between the pulsing of a lighthouse and the activity of the slime mold dictyostelium led researchers astray for a number of years. Likewise, the analogy between the solar system (the source) and the structure of the atom (the target) has been shown to be potentially misleading to students taking more advanced courses in physics or chemistry. The solar system analogy has a number of misalignments to the structure of the atom, such as electrons being repelled from each other rather than attracted; moreover, electrons do not have individual orbits like planets but have orbit clouds of electron density. Furthermore, students have serious misconceptions about the nature of the solar system, which can compound their misunderstanding of the nature of the atom (Fischler & Lichtfeld, 1992 ). While analogy is a powerful tool in science, like all forms of induction, incorrect conclusions can be reached.

Conceptual Change in Science

Scientific knowledge continually accumulates as scientists gather evidence about the natural world. Over extended time, this knowledge accumulation leads to major revisions, extensions, and new organizational forms for expressing what is known about nature. Indeed, these changes are so substantial that philosophers of science speak of “revolutions” in a variety of scientific domains (Kuhn, 1962 ). The psychological literature that explores the idea of revolutionary conceptual change can be roughly divided into (a) investigations of how scientists actually make discoveries and integrate those discoveries into existing scientific contexts, and (b) investigations of nonscientists ranging from infants, to children, to students in science classes. In this section we summarize the adult studies of conceptual change, and in the next section we look at its developmental aspects.

Scientific concepts, like all concepts, can be characterized as containing a variety of “knowledge elements”: representations of words, thoughts, actions, objects, and processes. At certain points in the history of science, the accumulated evidence has demanded major shifts in the way these collections of knowledge elements are organized. This “radical conceptual change” process (see Keil, 1999 ; Nersessian 1998 , 2002 ; Thagard, 1992 ; Vosniadou 1998, for reviews) requires the formation of a new conceptual system that organizes knowledge in new ways, adds new knowledge, and results in a very different conceptual structure. For more recent research on conceptual change, The International Handbook of Research on Conceptual Change (Vosniadou, 2008 ) provides a detailed compendium of theories and controversies within the field.

While conceptual change in science is usually characterized by large-scale changes in concepts that occur over extensive periods of time, it has been possible to observe conceptual change using in vivo methodologies. Dunbar ( 1995 ) reported a major conceptual shift that occurred in immunologists, where they obtained a series of unexpected findings that forced the scientists to propose a new concept in immunology that in turn forced the change in other concepts. The drive behind this conceptual change was the discovery of a series of different unexpected findings or anomalies that required the scientists to both revise and reorganize their conceptual knowledge. Interestingly, this conceptual change was achieved by a group of scientists reasoning collaboratively, rather than by a scientist working alone. Different scientists tend to work on different aspects of concepts, and also different concepts, that when put together lead to a rapid change in entire conceptual structures.

Overall, accounts of conceptual change in individuals indicate that it is indeed similar to that of conceptual change in entire scientific fields. Individuals need to be confronted with anomalies that their preexisting theories cannot explain before entire conceptual structures are overthrown. However, replacement conceptual structures have to be generated before the old conceptual structure can be discarded. Sometimes, people do not overthrow their original conceptual theories and through their lives maintain their original views of many fundamental scientific concepts. Whether people actively possess naive theories, or whether they appear to have a naive theory because of the demand characteristics of the testing context, is a lively source of debate within the science education community (see Gupta, Hammer, & Redish, 2010 ).

Scientific Thinking in Children

Well before their first birthday, children appear to know several fundamental facts about the physical world. For example, studies with infants show that they behave as if they understand that solid objects endure over time (e.g., they don't just disappear and reappear, they cannot move through each other, and they move as a result of collisions with other solid objects or the force of gravity (Baillargeon, 2004 ; Carey 1985 ; Cohen & Cashon, 2006 ; Duschl, Schweingruber, & Shouse, 2007 ; Gelman & Baillargeon, 1983 ; Gelman & Kalish, 2006 ; Mandler, 2004 ; Metz 1995 ; Munakata, Casey, & Diamond, 2004 ). And even 6-month-olds are able to predict the future location of a moving object that they are attempting to grasp (Von Hofsten, 1980 ; Von Hofsten, Feng, & Spelke, 2000 ). In addition, they appear to be able to make nontrivial inferences about causes and their effects (Gopnik et al., 2004 ).

The similarities between children's thinking and scientists' thinking have an inherent allure and an internal contradiction. The allure resides in the enthusiastic wonder and openness with which both children and scientists approach the world around them. The paradox comes from the fact that different investigators of children's thinking have reached diametrically opposing conclusions about just how “scientific” children's thinking really is. Some claim support for the “child as a scientist” position (Brewer & Samarapungavan, 1991 ; Gelman & Wellman, 1991 ; Gopnik, Meltzoff, & Kuhl, 1999 ; Karmiloff-Smith 1988 ; Sodian, Zaitchik, & Carey, 1991 ; Samarapungavan 1992 ), while others offer serious challenges to the view (Fay & Klahr, 1996 ; Kern, Mirels, & Hinshaw, 1983 ; Kuhn, Amsel, & O'Laughlin, 1988 ; Schauble & Glaser, 1990 ; Siegler & Liebert, 1975 .) Such fundamentally incommensurate conclusions suggest that this very field—children's scientific thinking—is ripe for a conceptual revolution!

A recent comprehensive review (Duschl, Schweingruber, & Shouse, 2007 ) of what children bring to their science classes offers the following concise summary of the extensive developmental and educational research literature on children's scientific thinking:

Children entering school already have substantial knowledge of the natural world, much of which is implicit.

What children are capable of at a particular age is the result of a complex interplay among maturation, experience, and instruction. What is developmentally appropriate is not a simple function of age or grade, but rather is largely contingent on children's prior opportunities to learn.

Students' knowledge and experience play a critical role in their science learning, influencing four aspects of science understanding, including (a) knowing, using, and interpreting scientific explanations of the natural world; (b) generating and evaluating scientific evidence and explanations, (c) understanding how scientific knowledge is developed in the scientific community, and (d) participating in scientific practices and discourse.

Students learn science by actively engaging in the practices of science.

In the previous section of this article we discussed conceptual change with respect to scientific fields and undergraduate science students. However, the idea that children undergo radical conceptual change in which old “theories” need to be overthrown and reorganized has been a central topic in understanding changes in scientific thinking in both children and across the life span. This radical conceptual change is thought to be necessary for acquiring many new concepts in physics and is regarded as the major source of difficulty for students. The factors that are at the root of this conceptual shift view have been difficult to determine, although there have been a number of studies in cognitive development (Carey, 1985 ; Chi 1992 ; Chi & Roscoe, 2002 ), in the history of science (Thagard, 1992 ), and in physics education (Clement, 1982 ; Mestre 1991 ) that give detailed accounts of the changes in knowledge representation that occur while people switch from one way of representing scientific knowledge to another.

One area where students show great difficulty in understanding scientific concepts is physics. Analyses of students' changing conceptions, using interviews, verbal protocols, and behavioral outcome measures, indicate that large-scale changes in students' concepts occur in physics education (see McDermott & Redish, 1999 , for a review of this literature). Following Kuhn ( 1962 ), many researchers, but not all, have noted that students' changing conceptions resemble the sequences of conceptual changes in physics that have occurred in the history of science. These notions of radical paradigm shifts and ensuing incompatibility with past knowledge-states have called attention to interesting parallels between the development of particular scientific concepts in children and in the history of physics. Investigations of nonphysicists' understanding of motion indicate that students have extensive misunderstandings of motion. Some researchers have interpreted these findings as an indication that many people hold erroneous beliefs about motion similar to a medieval “impetus” theory (McCloskey, Caramazza, & Green, 1980 ). Furthermore, students appear to maintain “impetus” notions even after one or two courses in physics. In fact, some authors have noted that students who have taken one or two courses in physics can perform worse on physics problems than naive students (Mestre, 1991 ). Thus, it is only after extensive learning that we see a conceptual shift from impetus theories of motion to Newtonian scientific theories.

How one's conceptual representation shifts from “naive” to Newtonian is a matter of contention, as some have argued that the shift involves a radical conceptual change, whereas others have argued that the conceptual change is not really complete. For example, Kozhevnikov and Hegarty ( 2001 ) argue that much of the naive impetus notions of motion are maintained at the expense of Newtonian principles even with extensive training in physics. However, they argue that such impetus principles are maintained at an implicit level. Thus, although students can give the correct Newtonian answer to problems, their reaction times to respond indicate that they are also using impetus theories when they respond. An alternative view of conceptual change focuses on whether there are real conceptual changes at all. Gupta, Hammer and Redish ( 2010 ) and Disessa ( 2004 ) have conducted detailed investigations of changes in physics students' accounts of phenomena covered in elementary physics courses. They have found that rather than students possessing a naive theory that is replaced by the standard theory, many introductory physics students have no stable physical theory but rather construct their explanations from elementary pieces of knowledge of the physical world.

Computational Approaches to Scientific Thinking

Computational approaches have provided a more complete account of the scientific mind. Computational models provide specific detailed accounts of the cognitive processes underlying scientific thinking. Early computational work consisted of taking a scientific discovery and building computational models of the reasoning processes involved in the discovery. Langley, Simon, Bradshaw, and Zytkow ( 1987 ) built a series of programs that simulated discoveries such as those of Copernicus, Bacon, and Stahl. These programs had various inductive reasoning algorithms built into them, and when given the data that the scientists used, they were able to propose the same rules. Computational models make it possible to propose detailed models of the cognitive subcomponents of scientific thinking that specify exactly how scientific theories are generated, tested, and amended (see Darden, 1997 , and Shrager & Langley, 1990 , for accounts of this branch of research). More recently, the incorporation of scientific knowledge into computer programs has resulted in a shift in emphasis from using programs to simulate discoveries to building programs that are used to help scientists make discoveries. A number of these computer programs have made novel discoveries. For example, Valdes-Perez ( 1994 ) has built systems for discoveries in chemistry, and Fajtlowicz has done this in mathematics (Erdos, Fajtlowicz, & Staton, 1991 ).

These advances in the fields of computer discovery have led to new fields, conferences, journals, and even departments that specialize in the development of programs devised to search large databases in the hope of making new scientific discoveries (Langley, 2000 , 2002 ). This process is commonly known as “data mining.” This approach has only proved viable relatively recently, due to advances in computer technology. Biswal et al. ( 2010 ), Mitchell ( 2009 ), and Yang ( 2009 ) provide recent reviews of data mining in different scientific fields. Data mining is at the core of drug discovery, our understanding of the human genome, and our understanding of the universe for a number of reasons. First, vast databases concerning drug actions, biological processes, the genome, the proteome, and the universe itself now exist. Second, the development of high throughput data-mining algorithms makes it possible to search for new drug targets, novel biological mechanisms, and new astronomical phenomena in relatively short periods of time. Research programs that took decades, such as the development of penicillin, can now be done in days (Yang, 2009 ).

Another recent shift in the use of computers in scientific discovery has been to have both computers and people make discoveries together, rather than expecting that computers make an entire scientific discovery. Now instead of using computers to mimic the entire scientific discovery process as used by humans, computers can use powerful algorithms that search for patterns on large databases and provide the patterns to humans who can then use the output of these computers to make discoveries, ranging from the human genome to the structure of the universe. However, there are some robots such as ADAM, developed by King ( 2011 ), that can actually perform the entire scientific process, from the generation of hypotheses, to the conduct of experiments and the interpretation of results, with little human intervention. The ongoing development of scientific robots by some scientists (King et al., 2009 ) thus continues the tradition started by Herbert Simon in the 1960s. However, many of the controversies as to whether the robot is a “real scientist” or not continue to the present (Evans & Rzhetsky, 2010 , Gianfelici, 2010 ; Haufe, Elliott, Burian, & O' Malley, 2010 ; O'Malley 2011 ).

Scientific Thinking and Science Education

Accounts of the nature of science and research on scientific thinking have had profound effects on science education along many levels, particularly in recent years. Science education from the 1900s until the 1970s was primarily concerned with teaching students both the content of science (such as Newton's laws of motion) or the methods that scientists need to use in their research (such as using experimental and control groups). Beginning in the 1980s, a number of reports (e.g., American Association for the Advancement of Science, 1993; National Commission on Excellence in Education, 1983; Rutherford & Ahlgren, 1991 ) stressed the need for teaching scientific thinking skills rather than just methods and content. The addition of scientific thinking skills to the science curriculum from kindergarten through adulthood was a major shift in focus. Many of the particular scientific thinking skills that have been emphasized are skills covered in previous sections of this chapter, such as teaching deductive and inductive thinking strategies. However, rather than focusing on one particular skill, such as induction, researchers in education have focused on how the different components of scientific thinking are put together in science. Furthermore, science educators have focused upon situations where science is conducted collaboratively, rather than being the product of one person thinking alone. These changes in science education parallel changes in methodologies used to investigate science, such as analyzing the ways that scientists think and reason in their laboratories.

By looking at science as a complex multilayered and group activity, many researchers in science education have adopted a constructivist approach. This approach sees learning as an active rather than a passive process, and it suggests that students learn through constructing their scientific knowledge. We will first describe a few examples of the constructivist approach to science education. Following that, we will address several lines of work that challenge some of the assumptions of the constructivist approach to science education.

Often the goal of constructivist science education is to produce conceptual change through guided instruction where the teacher or professor acts as a guide to discovery, rather than the keeper of all the facts. One recent and influential approach to science education is the inquiry-based learning approach. Inquiry-based learning focuses on posing a problem or a puzzling event to students and asking them to propose a hypothesis that could explain the event. Next, the student is asked to collect data that test the hypothesis, make conclusions, and then reflect upon both the original problem and the thought processes that they used to solve the problem. Often students use computers that aid in their construction of new knowledge. The computers allow students to learn many of the different components of scientific thinking. For example, Reiser and his colleagues have developed a learning environment for biology, where students are encouraged to develop hypotheses in groups, codify the hypotheses, and search databases to test these hypotheses (Reiser et al., 2001 ).

One of the myths of science is the lone scientist suddenly shouting “Eureka, I have made a discovery!” Instead, in vivo studies of scientists (e.g., Dunbar, 1995 , 2002 ), historical analyses of scientific discoveries (Nersessian, 1999 ), and studies of children learning science at museums have all pointed to collaborative scientific discovery mechanisms as being one of the driving forces of science (Atkins et al., 2009 ; Azmitia & Crowley, 2001 ). What happens during collaborative scientific thinking is that there is usually a triggering event, such as an unexpected result or situation that a student does not understand. This results in other members of the group adding new information to the person's representation of knowledge, often adding new inductions and deductions that both challenge and transform the reasoner's old representations of knowledge (Chi & Roscoe, 2002 ; Dunbar 1998 ). Social mechanisms play a key component in fostering changes in concepts that have been ignored in traditional cognitive research but are crucial for both science and science education. In science education there has been a shift to collaborative learning, particularly at the elementary level; however, in university education, the emphasis is still on the individual scientist. As many domains of science now involve collaborations across scientific disciplines, we expect the explicit teaching of heuristics for collaborative science to increase.

What is the best way to teach and learn science? Surprisingly, the answer to this question has been difficult to uncover. For example, toward the end of the last century, influenced by several thinkers who advocated a constructivist approach to learning, ranging from Piaget (Beilin, 1994 ) to Papert ( 1980 ), many schools answered this question by adopting a philosophy dubbed “discovery learning.” Although a clear operational definition of this approach has yet to be articulated, the general idea is that children are expected to learn science by reconstructing the processes of scientific discovery—in a range of areas from computer programming to chemistry to mathematics. The premise is that letting students discover principles on their own, set their own goals, and collaboratively explore the natural world produces deeper knowledge that transfers widely.

The research literature on science education is far from consistent in its use of terminology. However, our reading suggests that “discovery learning” differs from “inquiry-based learning” in that few, if any, guidelines are given to students in discovery learning contexts, whereas in inquiry learning, students are given hypotheses and specific goals to achieve (see the second paragraph of this section for a definition of inquiry-based learning). Even though thousands of schools have adopted discovery learning as an alternative to more didactic approaches to teaching and learning, the evidence showing that it is more effective than traditional, direct, teacher-controlled instructional approaches is mixed, at best (Lorch et al., 2010 ; Minner, Levy, & Century, 2010 ). In several cases where the distinctions between direct instruction and more open-ended constructivist instruction have been clearly articulated, implemented, and assessed, direct instruction has proven to be superior to the alternatives (Chen & Klahr, 1999 ; Toth, Klahr, & Chen, 2000 ). For example, in a study of third- and fourth-grade children learning about experimental design, Klahr and Nigam ( 2004 ) found that many more children learned from direct instruction than from discovery learning. Furthermore, they found that among the few children who did manage to learn from a discovery method, there was no better performance on a far transfer test of scientific reasoning than that observed for the many children who learned from direct instruction.

The idea of children learning most of their science through a process of self-directed discovery has some romantic appeal, and it may accurately describe the personal experience of a handful of world-class scientists. However, the claim has generated some contentious disagreements (Kirschner, Sweller, & Clark, 2006 ; Klahr, 2010 ; Taber 2009 ; Tobias & Duffy, 2009 ), and the jury remains out on the extent to which most children can learn science that way.

Conclusions and Future Directions

The field of scientific thinking is now a thriving area of research with strong underpinnings in cognitive psychology and cognitive science. In recent years, a new professional society has been formed that aims to facilitate this integrative and interdisciplinary approach to the psychology of science, with its own journal and regular professional meetings. 1 Clearly the relations between these different aspects of scientific thinking need to be combined in order to produce a truly comprehensive picture of the scientific mind.

While much is known about certain aspects of scientific thinking, much more remains to be discovered. In particular, there has been little contact between cognitive, neuroscience, social, personality, and motivational accounts of scientific thinking. Research in thinking and reasoning has been expanded to use the methods and theories of cognitive neuroscience (see Morrison & Knowlton, Chapter 6 ). A similar approach can be taken in exploring scientific thinking (see Dunbar et al., 2007 ). There are two main reasons for taking a neuroscience approach to scientific thinking. First, functional neuroimaging allows the researcher to look at the entire human brain, making it possible to see the many different sites that are involved in scientific thinking and gain a more complete understanding of the entire range of mechanisms involved in this type of thought. Second, these brain-imaging approaches allow researchers to address fundamental questions in research on scientific thinking, such as the extent to which ordinary thinking in nonscientific contexts and scientific thinking recruit similar versus disparate neural structures of the brain.

Dunbar ( 2009 ) has used some novel methods to explore Simon's assertion, cited at the beginning of this chapter, that scientific thinking uses the same cognitive mechanisms that all human beings possess (rather than being an entirely different type of thinking) but combines them in ways that are specific to a particular aspect of science or a specific discipline of science. For example, Fugelsang and Dunbar ( 2009 ) compared causal reasoning when two colliding circular objects were labeled balls or labeled subatomic particles. They obtained different brain activation patterns depending on whether the stimuli were labeled balls or subatomic particles. In another series of experiments, Dunbar and colleagues used functional magnetic resonance imaging (fMRI) to study patterns of activation in the brains of students who have and who have not undergone conceptual change in physics. For example, Fugelsang and Dunbar ( 2005 ) and Dunbar et al. ( 2007 ) have found differences in the activation of specific brain sites (such as the anterior cingulate) for students when they encounter evidence that is inconsistent with their current conceptual understandings. These initial cognitive neuroscience investigations have the potential to reveal the ways that knowledge is organized in the scientific brain and provide detailed accounts of the nature of the representation of scientific knowledge. Petitto and Dunbar ( 2004 ) proposed the term “educational neuroscience” for the integration of research on education, including science education, with research on neuroscience. However, see Fitzpatrick (in press) for a very different perspective on whether neuroscience approaches are relevant to education. Clearly, research on the scientific brain is just beginning. We as scientists are beginning to get a reasonable grasp of the inner workings of the subcomponents of the scientific mind (i.e., problem solving, analogy, induction). However, great advances remain to be made concerning how these processes interact so that scientific discoveries can be made. Future research will focus on both the collaborative aspects of scientific thinking and the neural underpinnings of the scientific mind.

The International Society for the Psychology of Science and Technology (ISPST). Available at http://www.ispstonline.org/

Ahn, W., Kalish, C. W., Medin, D. L., & Gelman, S. A. ( 1995 ). The role of covariation versus mechanism information in causal attribution.   Cognition , 54 , 299–352.

American Association for the Advancement of Science. ( 1993 ). Benchmarks for scientific literacy . New York: Oxford University Press.

Google Scholar

Google Preview

Atkins, L. J., Velez, L., Goudy, D., & Dunbar, K. N. ( 2009 ). The unintended effects of interactive objects and labels in the science museum.   Science Education , 54 , 161–184.

Azmitia, M. A., & Crowley, K. ( 2001 ). The rhythms of scientific thinking: A study of collaboration in an earthquake microworld. In K. Crowley, C. Schunn, & T. Okada (Eds.), Designing for science: Implications from everyday, classroom, and professional settings (pp. 45–72). Mahwah, NJ: Erlbaum.

Bacon, F. ( 1620 /1854). Novum organum (B. Monatgue, Trans.). Philadelphia, P A: Parry & McMillan.

Baillargeon, R. ( 2004 ). Infants' reasoning about hidden objects: Evidence for event-general and event-specific expectations (article with peer commentaries and response, listed below).   Developmental Science , 54 , 391–424.

Baker, L. M., & Dunbar, K. ( 2000 ). Experimental design heuristics for scientific discovery: The use of baseline and known controls.   International Journal of Human Computer Studies , 54 , 335–349.

Beilin, H. ( 1994 ). Jean Piaget's enduring contribution to developmental psychology. In R. D. Parke, P. A. Ornstein, J. J. Rieser, & C. Zahn-Waxler (Eds.), A century of developmental psychology (pp. 257–290). Washington, DC US: American Psychological Association.

Biswal, B. B., Mennes, M., Zuo, X.-N., Gohel, S., Kelly, C., Smith, S.M., et al. ( 2010 ). Toward discovery science of human brain function.   Proceedings of the National Academy of Sciences of the United States of America , 107, 4734–4739.

Brewer, W. F., & Samarapungavan, A. ( 1991 ). Children's theories vs. scientific theories: Differences in reasoning or differences in knowledge? In R. R. Hoffman & D. S. Palermo (Eds.), Cognition and the symbolic processes: Applied and ecological perspectives (pp. 209–232). Hillsdale, NJ: Erlbaum.

Bruner, J. S., Goodnow, J. J., & Austin, G. A. ( 1956 ). A study of thinking . New York: NY Science Editions.

Carey, S. ( 1985 ). Conceptual change in childhood . Cambridge, MA: MIT Press.

Carruthers, P., Stich, S., & Siegal, M. ( 2002 ). The cognitive basis of science . New York: Cambridge University Press.

Chi, M. ( 1992 ). Conceptual change within and across ontological categories: Examples from learning and discovery in science. In R. Giere (Ed.), Cognitive models of science (pp. 129–186). Minneapolis: University of Minnesota Press.

Chi, M. T. H., & Roscoe, R. D. ( 2002 ). The processes and challenges of conceptual change. In M. Limon & L. Mason (Eds.), Reconsidering conceptual change: Issues in theory and practice (pp 3–27). Amsterdam, Netherlands: Kluwer Academic Publishers.

Chen, Z., & Klahr, D. ( 1999 ). All other things being equal: Children's acquisition of the control of variables strategy.   Child Development , 54 (5), 1098–1120.

Clement, J. ( 1982 ). Students' preconceptions in introductory mechanics.   American Journal of Physics , 54 , 66–71.

Cohen, L. B., & Cashon, C. H. ( 2006 ). Infant cognition. In W. Damon & R. M. Lerner (Series Eds.) & D. Kuhn & R. S. Siegler (Vol. Eds.), Handbook of child psychology. Vol. 2: Cognition, perception, and language (6th ed., pp. 214–251). New York: Wiley.

National Commission on Excellence in Education. ( 1983 ). A nation at risk: The imperative for educational reform . Washington, DC: US Department of Education.

Crick, F. H. C. ( 1988 ). What mad pursuit: A personal view of science . New York: Basic Books.

Darden, L. ( 2002 ). Strategies for discovering mechanisms: Schema instantiation, modular subassembly, forward chaining/backtracking.   Philosophy of Science , 69, S354–S365.

Davenport, J. L., Yaron, D., Klahr, D., & Koedinger, K. ( 2008 ). Development of conceptual understanding and problem solving expertise in chemistry. In B. C. Love, K. McRae, & V. M. Sloutsky (Eds.), Proceedings of the 30th Annual Conference of the Cognitive Science Society (pp. 751–756). Austin, TX: Cognitive Science Society.

diSessa, A. A. ( 2004 ). Contextuality and coordination in conceptual change. In E. Redish & M. Vicentini (Eds.), Proceedings of the International School of Physics “Enrico Fermi:” Research on physics education (pp. 137–156). Amsterdam, Netherlands: ISO Press/Italian Physics Society

Dunbar, K. ( 1995 ). How scientists really reason: Scientific reasoning in real-world laboratories. In R. J. Sternberg, & J. Davidson (Eds.), Mechanisms of insight (pp. 365–395). Cambridge, MA: MIT press.

Dunbar, K. ( 1997 ). How scientists think: Online creativity and conceptual change in science. In T. B. Ward, S. M. Smith, & S. Vaid (Eds.), Conceptual structures and processes: Emergence, discovery and change (pp. 461–494). Washington, DC: American Psychological Association.

Dunbar, K. ( 1998 ). Problem solving. In W. Bechtel & G. Graham (Eds.), A companion to cognitive science (pp. 289–298). London: Blackwell

Dunbar, K. ( 1999 ). The scientist InVivo : How scientists think and reason in the laboratory. In L. Magnani, N. Nersessian, & P. Thagard (Eds.), Model-based reasoning in scientific discovery (pp. 85–100). New York: Plenum.

Dunbar, K. ( 2001 ). The analogical paradox: Why analogy is so easy in naturalistic settings, yet so difficult in the psychology laboratory. In D. Gentner, K. J. Holyoak, & B. Kokinov Analogy: Perspectives from cognitive science (pp. 313–334). Cambridge, MA: MIT press.

Dunbar, K. ( 2002 ). Science as category: Implications of InVivo science for theories of cognitive development, scientific discovery, and the nature of science. In P. Caruthers, S. Stich, & M. Siegel (Eds.) Cognitive models of science (pp. 154–170). New York: Cambridge University Press.

Dunbar, K. ( 2009 ). The biology of physics: What the brain reveals about our physical understanding of the world. In M. Sabella, C. Henderson, & C. Singh. (Eds.), Proceedings of the Physics Education Research Conference (pp. 15–18). Melville, NY: American Institute of Physics.

Dunbar, K., & Fugelsang, J. ( 2004 ). Causal thinking in science: How scientists and students interpret the unexpected. In M. E. Gorman, A. Kincannon, D. Gooding, & R. D. Tweney (Eds.), New directions in scientific and technical thinking (pp. 57–59). Mahway, NJ: Erlbaum.

Dunbar, K., Fugelsang, J., & Stein, C. ( 2007 ). Do naïve theories ever go away? In M. Lovett & P. Shah (Eds.), Thinking with Data: 33 rd Carnegie Symposium on Cognition (pp. 193–206). Mahwah, NJ: Erlbaum.

Dunbar, K., & Sussman, D. ( 1995 ). Toward a cognitive account of frontal lobe function: Simulating frontal lobe deficits in normal subjects.   Annals of the New York Academy of Sciences , 54 , 289–304.

Duschl, R. A., Schweingruber, H. A., & Shouse, A. W. (Eds.). ( 2007 ). Taking science to school: Learning and teaching science in grades K-8. Washington, DC: National Academies Press.

Einstein, A. ( 1950 ). Out of my later years . New York: Philosophical Library

Erdos, P., Fajtlowicz, S., & Staton, W. ( 1991 ). Degree sequences in the triangle-free graphs,   Discrete Mathematics , 54 (91), 85–88.

Evans, J., & Rzhetsky, A. ( 2010 ). Machine science.   Science , 54 , 399–400.

Fay, A., & Klahr, D. ( 1996 ). Knowing about guessing and guessing about knowing: Preschoolers' understanding of indeterminacy.   Child Development , 54 , 689–716.

Fischler, H., & Lichtfeldt, M. ( 1992 ). Modern physics and students conceptions.   International Journal of Science Education , 54 , 181–190.

Fitzpatrick, S. M. (in press). Functional brain imaging: Neuro-turn or wrong turn? In M. M., Littlefield & J.M., Johnson (Eds.), The neuroscientific turn: Transdisciplinarity in the age of the brain. Ann Arbor: University of Michigan Press.

Fox-Keller, E. ( 1985 ). Reflections on gender and science . New Haven, CT: Yale University Press.

Fugelsang, J., & Dunbar, K. ( 2005 ). Brain-based mechanisms underlying complex causal thinking.   Neuropsychologia , 54 , 1204–1213.

Fugelsang, J., & Dunbar, K. ( 2009 ). Brain-based mechanisms underlying causal reasoning. In E. Kraft (Ed.), Neural correlates of thinking (pp. 269–279). Berlin, Germany: Springer

Fugelsang, J., Stein, C., Green, A., & Dunbar, K. ( 2004 ). Theory and data interactions of the scientific mind: Evidence from the molecular and the cognitive laboratory.   Canadian Journal of Experimental Psychology , 54 , 132–141

Galilei, G. ( 1638 /1991). Dialogues concerning two new sciences (A. de Salvio & H. Crew, Trans.). Amherst, NY: Prometheus Books.

Galison, P. ( 2003 ). Einstein's clocks, Poincaré's maps: Empires of time . New York: W. W. Norton.

Gelman, R., & Baillargeon, R. ( 1983 ). A review of Piagetian concepts. In P. H. Mussen (Series Ed.) & J. H. Flavell & E. M. Markman (Vol. Eds.), Handbook of child psychology (4th ed., Vol. 3, pp. 167–230). New York: Wiley.

Gelman, S. A., & Kalish, C. W. ( 2006 ). Conceptual development. In D. Kuhn & R. Siegler (Eds.), Handbook of child psychology. Vol. 2: Cognition, perception and language (pp. 687–733). New York: Wiley.

Gelman, S., & Wellman, H. ( 1991 ). Insides and essences.   Cognition , 54 , 214–244.

Gentner, D. ( 2010 ). Bootstrapping the mind: Analogical processes and symbol systems.   Cognitive Science , 54 , 752–775.

Gentner, D., Brem, S., Ferguson, R. W., Markman, A. B., Levidow, B. B., Wolff, P., & Forbus, K. D. ( 1997 ). Analogical reasoning and conceptual change: A case study of Johannes Kepler.   The Journal of the Learning Sciences , 54 (1), 3–40.

Gentner, D., Holyoak, K. J., & Kokinov, B. ( 2001 ). The analogical mind: Perspectives from cognitive science . Cambridge, MA: MIT Press.

Gentner, D., & Jeziorski, M. ( 1993 ). The shift from metaphor to analogy in western science. In A. Ortony (Ed.), Metaphor and thought (2nd ed., pp. 447–480). Cambridge, England: Cambridge University Press.

Gianfelici, F. ( 2010 ). Machine science: Truly machine-aided science.   Science , 54 , 317–319.

Giere, R. ( 1993 ). Cognitive models of science . Minneapolis: University of Minnesota Press.

Gopnik, A. N., Meltzoff, A. N., & Kuhl, P. K. ( 1999 ). The scientist in the crib: Minds, brains and how children learn . New York: Harper Collins

Gorman, M. E. ( 1989 ). Error, falsification and scientific inference: An experimental investigation.   Quarterly Journal of Experimental Psychology: Human Experimental Psychology , 41A , 385–412

Gorman, M. E., Kincannon, A., Gooding, D., & Tweney, R. D. ( 2004 ). New directions in scientific and technical thinking . Mahwah, NJ: Erlbaum.

Gupta, A., Hammer, D., & Redish, E. F. ( 2010 ). The case for dynamic models of learners' ontologies in physics.   Journal of the Learning Sciences , 54 (3), 285–321.

Haufe, C., Elliott, K. C., Burian, R., & O'Malley, M. A. ( 2010 ). Machine science: What's missing.   Science , 54 , 318–320.

Hecht, E. ( 2011 ). On defining mass.   The Physics Teacher , 54 , 40–43.

Heit, E. ( 2000 ). Properties of inductive reasoning.   Psychonomic Bulletin and Review , 54 , 569–592.

Holyoak, K. J., & Thagard, P. ( 1995 ). Mental leaps . Cambridge, MA: MIT Press.

Karmiloff-Smith, A. ( 1988 ) The child is a theoretician, not an inductivist.   Mind and Language , 54 , 183–195.

Keil, F. C. ( 1999 ). Conceptual change. In R. Wilson & F. Keil (Eds.), The MIT encyclopedia of cognitive science . (pp. 179–182) Cambridge, MA: MIT press.

Kern, L. H., Mirels, H. L., & Hinshaw, V. G. ( 1983 ). Scientists' understanding of propositional logic: An experimental investigation.   Social Studies of Science , 54 , 131–146.

King, R. D. ( 2011 ). Rise of the robo scientists.   Scientific American , 54 (1), 73–77.

King, R. D., Rowland, J., Oliver, S. G., Young, M., Aubrey, W., Byrne, E., et al. ( 2009 ). The automation of science.   Science , 54 , 85–89.

Kirschner, P. A., Sweller, J., & Clark, R. ( 2006 ) Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching.   Educational Psychologist , 54 , 75–86

Klahr, D. ( 2000 ). Exploring science: The cognition and development of discovery processes . Cambridge, MA: MIT Press.

Klahr, D. ( 2010 ). Coming up for air: But is it oxygen or phlogiston? A response to Taber's review of constructivist instruction: Success or failure?   Education Review , 54 (13), 1–6.

Klahr, D., & Dunbar, K. ( 1988 ). Dual space search during scientific reasoning.   Cognitive Science , 54 , 1–48.

Klahr, D., & Nigam, M. ( 2004 ). The equivalence of learning paths in early science instruction: effects of direct instruction and discovery learning.   Psychological Science , 54 (10), 661–667.

Klahr, D. & Masnick, A. M. ( 2002 ). Explaining, but not discovering, abduction. Review of L. Magnani (2001) abduction, reason, and science: Processes of discovery and explanation.   Contemporary Psychology , 47, 740–741.

Klahr, D., & Simon, H. ( 1999 ). Studies of scientific discovery: Complementary approaches and convergent findings.   Psychological Bulletin , 54 , 524–543.

Klayman, J., & Ha, Y. ( 1987 ). Confirmation, disconfirmation, and information in hypothesis testing.   Psychological Review , 54 , 211–228.

Kozhevnikov, M., & Hegarty, M. ( 2001 ). Impetus beliefs as default heuristic: Dissociation between explicit and implicit knowledge about motion.   Psychonomic Bulletin and Review , 54 , 439–453.

Kuhn, T. ( 1962 ). The structure of scientific revolutions . Chicago, IL: University of Chicago Press.

Kuhn, D., Amsel, E., & O'Laughlin, M. ( 1988 ). The development of scientific thinking skills . Orlando, FL: Academic Press.

Kulkarni, D., & Simon, H. A. ( 1988 ). The processes of scientific discovery: The strategy of experimentation.   Cognitive Science , 54 , 139–176.

Langley, P. ( 2000 ). Computational support of scientific discovery.   International Journal of Human-Computer Studies , 54 , 393–410.

Langley, P. ( 2002 ). Lessons for the computational discovery of scientific knowledge. In Proceedings of the First International Workshop on Data Mining Lessons Learned (pp. 9–12).

Langley, P., Simon, H. A., Bradshaw, G. L., & Zytkow, J. M. ( 1987 ). Scientific discovery: Computational explorations of the creative processes . Cambridge, MA: MIT Press.

Lorch, R. F., Jr., Lorch, E. P., Calderhead, W. J., Dunlap, E. E., Hodell, E. C., & Freer, B. D. ( 2010 ). Learning the control of variables strategy in higher and lower achieving classrooms: Contributions of explicit instruction and experimentation.   Journal of Educational Psychology , 54 (1), 90–101.

Magnani, L., Carnielli, W., & Pizzi, C., (Eds.) ( 2010 ). Model-based reasoning in science and technology: Abduction, logic,and computational discovery. Series Studies in Computational Intelligence (Vol. 314). Heidelberg/Berlin: Springer.

Mandler, J.M. ( 2004 ). The foundations of mind: Origins of conceptual thought . Oxford, England: Oxford University Press.

Macpherson, R., & Stanovich, K. E. ( 2007 ). Cognitive ability, thinking dispositions, and instructional set as predictors of critical thinking.   Learning and Individual Differences , 54 , 115–127.

McCloskey, M., Caramazza, A., & Green, B. ( 1980 ). Curvilinear motion in the absence of external forces: Naive beliefs about the motion of objects.   Science , 54 , 1139–1141.

McDermott, L. C., & Redish, L. ( 1999 ). Research letter on physics education research.   American Journal of Psychics , 54 , 755.

Mestre, J. P. ( 1991 ). Learning and instruction in pre-college physical science.   Physics Today , 54 , 56–62.

Metz, K. E. ( 1995 ). Reassessment of developmental constraints on children's science instruction.   Review of Educational Research , 54 (2), 93–127.

Minner, D. D., Levy, A. J., & Century, J. ( 2010 ). Inquiry-based science instruction—what is it and does it matter? Results from a research synthesis years 1984 to 2002.   Journal of Research in Science Teaching , 54 (4), 474–496.

Mitchell, T. M. ( 2009 ). Mining our reality.   Science , 54 , 1644–1645.

Mitroff, I. ( 1974 ). The subjective side of science . Amsterdam, Netherlands: Elsevier.

Munakata, Y., Casey, B. J., & Diamond, A. ( 2004 ). Developmental cognitive neuroscience: Progress and potential.   Trends in Cognitive Sciences , 54 , 122–128.

Mynatt, C. R., Doherty, M. E., & Tweney, R. D. ( 1977 ) Confirmation bias in a simulated research environment: An experimental study of scientific inference.   Quarterly Journal of Experimental Psychology , 54 , 89–95.

Nersessian, N. ( 1998 ). Conceptual change. In W. Bechtel, & G. Graham (Eds.), A companion to cognitive science (pp. 157–166). London, England: Blackwell.

Nersessian, N. ( 1999 ). Models, mental models, and representations: Model-based reasoning in conceptual change. In L. Magnani, N. Nersessian, & P. Thagard (Eds.), Model-based reasoning in scientific discovery (pp. 5–22). New York: Plenum.

Nersessian, N. J. ( 2002 ). The cognitive basis of model-based reasoning in science In. P. Carruthers, S. Stich, & M. Siegal (Eds.), The cognitive basis of science (pp. 133–152). New York: Cambridge University Press.

Nersessian, N. J. ( 2008 ) Creating scientific concepts . Cambridge, MA: MIT Press.

O' Malley, M. A. ( 2011 ). Exploration, iterativity and kludging in synthetic biology.   Comptes Rendus Chimie , 54 (4), 406–412 .

Papert, S. ( 1980 ) Mindstorms: Children computers and powerful ideas. New York: Basic Books.

Penner, D. E., & Klahr, D. ( 1996 ). When to trust the data: Further investigations of system error in a scientific reasoning task.   Memory and Cognition , 54 (5), 655–668.

Petitto, L. A., & Dunbar, K. ( 2004 ). New findings from educational neuroscience on bilingual brains, scientific brains, and the educated mind. In K. Fischer & T. Katzir (Eds.), Building usable knowledge in mind, brain, and education Cambridge, England: Cambridge University Press.

Popper, K. R. ( 1959 ). The logic of scientific discovery . London, England: Hutchinson.

Qin, Y., & Simon, H.A. ( 1990 ). Laboratory replication of scientific discovery processes.   Cognitive Science , 54 , 281–312.

Reiser, B. J., Tabak, I., Sandoval, W. A., Smith, B., Steinmuller, F., & Leone, T. J., ( 2001 ). BGuILE: Stategic and conceptual scaffolds for scientific inquiry in biology classrooms. In S. M. Carver & D. Klahr (Eds.), Cognition and instruction: Twenty-five years of progress (pp. 263–306). Mahwah, NJ: Erlbaum

Riordan, M., Rowson, P. C., & Wu, S. L. ( 2001 ). The search for the higgs boson.   Science , 54 , 259–260.

Rutherford, F. J., & Ahlgren, A. ( 1991 ). Science for all Americans. New York: Oxford University Press.

Samarapungavan, A. ( 1992 ). Children's judgments in theory choice tasks: Scientifc rationality in childhood.   Cognition , 54 , 1–32.

Schauble, L., & Glaser, R. ( 1990 ). Scientific thinking in children and adults. In D. Kuhn (Ed.), Developmental perspectives on teaching and learning thinking skills. Contributions to Human Development , (Vol. 21, pp. 9–26). Basel, Switzerland: Karger.

Schunn, C. D., & Klahr, D. ( 1995 ). A 4-space model of scientific discovery. In Proceedings of the 17th Annual Conference of the Cognitive Science Society (pp. 106–111). Mahwah, NJ: Erlbaum.

Schunn, C. D., & Klahr, D. ( 1996 ). The problem of problem spaces: When and how to go beyond a 2-space model of scientific discovery. Part of symposium on Building a theory of problem solving and scientific discovery: How big is N in N-space search? In Proceedings of the 18th Annual Conference of the Cognitive Science Society (pp. 25–26). Mahwah, NJ: Erlbaum.

Shrager, J., & Langley, P. ( 1990 ). Computational models of scientific discovery and theory formation . San Mateo, CA: Morgan Kaufmann.

Siegler, R. S., & Liebert, R. M. ( 1975 ). Acquisition of formal scientific reasoning by 10- and 13-year-olds: Designing a factorial experiment.   Developmental Psychology , 54 , 401–412.

Simon, H. A. ( 1977 ). Models of discovery . Dordrecht, Netherlands: D. Reidel Publishing.

Simon, H. A., Langley, P., & Bradshaw, G. L. ( 1981 ). Scientific discovery as problem solving.   Synthese , 54 , 1–27.

Simon, H. A., & Lea, G. ( 1974 ). Problem solving and rule induction. In H. Simon (Ed.), Models of thought (pp. 329–346). New Haven, CT: Yale University Press.

Smith, E. E., Shafir, E., & Osherson, D. ( 1993 ). Similarity, plausibility, and judgments of probability.   Cognition. Special Issue: Reasoning and decision making , 54 , 67–96.

Sodian, B., Zaitchik, D., & Carey, S. ( 1991 ). Young children's differentiation of hypothetical beliefs from evidence.   Child Development , 54 , 753–766.

Taber, K. S. ( 2009 ). Constructivism and the crisis in U.S. science education: An essay review.   Education Review , 54 (12), 1–26.

Thagard, P. ( 1992 ). Conceptual revolutions . Cambridge, MA: MIT Press.

Thagard, P. ( 1999 ). How scientists explain disease . Princeton, NJ: Princeton University Press.

Thagard, P., & Croft, D. ( 1999 ). Scientific discovery and technological innovation: Ulcers, dinosaur extinction, and the programming language Java. In L. Magnani, N. Nersessian, & P. Thagard (Eds.), Model-based reasoning in scientific discovery (pp. 125–138). New York: Plenum.

Tobias, S., & Duffy, T. M. (Eds.). ( 2009 ). Constructivist instruction: Success or failure? New York: Routledge.

Toth, E. E., Klahr, D., & Chen, Z. ( 2000 ) Bridging research and practice: A cognitively-based classroom intervention for teaching experimentation skills to elementary school children.   Cognition and Instruction , 54 (4), 423–459.

Tweney, R. D. ( 1989 ). A framework for the cognitive psychology of science. In B. Gholson, A. Houts, R. A. Neimeyer, & W. Shadish (Eds.), Psychology of science: Contributions to metascience (pp. 342–366). Cambridge, England: Cambridge University Press.

Tweney, R. D., Doherty, M. E., & Mynatt, C. R. ( 1981 ). On scientific thinking . New York: Columbia University Press.

Valdes-Perez, R. E. ( 1994 ). Conjecturing hidden entities via simplicity and conservation laws: Machine discovery in chemistry.   Artificial Intelligence , 54 (2), 247–280.

Von Hofsten, C. ( 1980 ). Predictive reaching for moving objects by human infants.   Journal of Experimental Child Psychology , 54 , 369–382.

Von Hofsten, C., Feng, Q., & Spelke, E. S. ( 2000 ). Object representation and predictive action in infancy.   Developmental Science , 54 , 193–205.

Vosnaidou, S. (Ed.). ( 2008 ). International handbook of research on conceptual change . New York: Taylor & Francis.

Vosniadou, S., & Brewer, W. F. ( 1992 ). Mental models of the earth: A study of conceptual change in childhood.   Cognitive Psychology , 54 , 535–585.

Wason, P. C. ( 1968 ). Reasoning about a rule.   Quarterly Journal of Experimental Psychology , 54 , 273–281.

Wertheimer, M. ( 1945 ). Productive thinking . New York: Harper.

Yang, Y. ( 2009 ). Target discovery from data mining approaches.   Drug Discovery Today , 54 (3–4), 147–154.

• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Download the free Kindle app and start reading Kindle books instantly on your smartphone, tablet, or computer - no Kindle device required .

Read instantly on your browser with Kindle for Web.

Using your mobile phone camera - scan the code below and download the Kindle app.

Image Unavailable

• To view this video download Flash Player

Research methodology: An introduction to scientific thinking and practice Paperback

• Language English
• Dimensions 8.27 x 0.71 x 11.69 inches
• ISBN-10 9043037885
• ISBN-13 978-9043037884
• See all details

Product details

• Language ‏ : ‎ English
• ISBN-10 ‏ : ‎ 9043037885
• ISBN-13 ‏ : ‎ 978-9043037884
• Item Weight ‏ : ‎ 1.8 pounds
• Dimensions ‏ : ‎ 8.27 x 0.71 x 11.69 inches

Customer reviews

Customer Reviews, including Product Star Ratings help customers to learn more about the product and decide whether it is the right product for them.

To calculate the overall star rating and percentage breakdown by star, we don’t use a simple average. Instead, our system considers things like how recent a review is and if the reviewer bought the item on Amazon. It also analyzed reviews to verify trustworthiness.

A tutorial on methodological studies: the what, when, how and why

Lawrence mbuagbaw.

1 Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON Canada

2 Biostatistics Unit/FSORC, 50 Charlton Avenue East, St Joseph’s Healthcare—Hamilton, 3rd Floor Martha Wing, Room H321, Hamilton, Ontario L8N 4A6 Canada

3 Centre for the Development of Best Practices in Health, Yaoundé, Cameroon

Daeria O. Lawson

Livia puljak.

4 Center for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000 Zagreb, Croatia

David B. Allison

5 Department of Epidemiology and Biostatistics, School of Public Health – Bloomington, Indiana University, Bloomington, IN 47405 USA

Lehana Thabane

6 Departments of Paediatrics and Anaesthesia, McMaster University, Hamilton, ON Canada

7 Centre for Evaluation of Medicine, St. Joseph’s Healthcare-Hamilton, Hamilton, ON Canada

8 Population Health Research Institute, Hamilton Health Sciences, Hamilton, ON Canada

Associated Data

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Methodological studies – studies that evaluate the design, analysis or reporting of other research-related reports – play an important role in health research. They help to highlight issues in the conduct of research with the aim of improving health research methodology, and ultimately reducing research waste.

We provide an overview of some of the key aspects of methodological studies such as what they are, and when, how and why they are done. We adopt a “frequently asked questions” format to facilitate reading this paper and provide multiple examples to help guide researchers interested in conducting methodological studies. Some of the topics addressed include: is it necessary to publish a study protocol? How to select relevant research reports and databases for a methodological study? What approaches to data extraction and statistical analysis should be considered when conducting a methodological study? What are potential threats to validity and is there a way to appraise the quality of methodological studies?

Appropriate reflection and application of basic principles of epidemiology and biostatistics are required in the design and analysis of methodological studies. This paper provides an introduction for further discussion about the conduct of methodological studies.

The field of meta-research (or research-on-research) has proliferated in recent years in response to issues with research quality and conduct [ 1 – 3 ]. As the name suggests, this field targets issues with research design, conduct, analysis and reporting. Various types of research reports are often examined as the unit of analysis in these studies (e.g. abstracts, full manuscripts, trial registry entries). Like many other novel fields of research, meta-research has seen a proliferation of use before the development of reporting guidance. For example, this was the case with randomized trials for which risk of bias tools and reporting guidelines were only developed much later – after many trials had been published and noted to have limitations [ 4 , 5 ]; and for systematic reviews as well [ 6 – 8 ]. However, in the absence of formal guidance, studies that report on research differ substantially in how they are named, conducted and reported [ 9 , 10 ]. This creates challenges in identifying, summarizing and comparing them. In this tutorial paper, we will use the term methodological study to refer to any study that reports on the design, conduct, analysis or reporting of primary or secondary research-related reports (such as trial registry entries and conference abstracts).

In the past 10 years, there has been an increase in the use of terms related to methodological studies (based on records retrieved with a keyword search [in the title and abstract] for “methodological review” and “meta-epidemiological study” in PubMed up to December 2019), suggesting that these studies may be appearing more frequently in the literature. See Fig.  1 .

Trends in the number studies that mention “methodological review” or “meta-

epidemiological study” in PubMed.

The methods used in many methodological studies have been borrowed from systematic and scoping reviews. This practice has influenced the direction of the field, with many methodological studies including searches of electronic databases, screening of records, duplicate data extraction and assessments of risk of bias in the included studies. However, the research questions posed in methodological studies do not always require the approaches listed above, and guidance is needed on when and how to apply these methods to a methodological study. Even though methodological studies can be conducted on qualitative or mixed methods research, this paper focuses on and draws examples exclusively from quantitative research.

The objectives of this paper are to provide some insights on how to conduct methodological studies so that there is greater consistency between the research questions posed, and the design, analysis and reporting of findings. We provide multiple examples to illustrate concepts and a proposed framework for categorizing methodological studies in quantitative research.

What is a methodological study?

Any study that describes or analyzes methods (design, conduct, analysis or reporting) in published (or unpublished) literature is a methodological study. Consequently, the scope of methodological studies is quite extensive and includes, but is not limited to, topics as diverse as: research question formulation [ 11 ]; adherence to reporting guidelines [ 12 – 14 ] and consistency in reporting [ 15 ]; approaches to study analysis [ 16 ]; investigating the credibility of analyses [ 17 ]; and studies that synthesize these methodological studies [ 18 ]. While the nomenclature of methodological studies is not uniform, the intents and purposes of these studies remain fairly consistent – to describe or analyze methods in primary or secondary studies. As such, methodological studies may also be classified as a subtype of observational studies.

Parallel to this are experimental studies that compare different methods. Even though they play an important role in informing optimal research methods, experimental methodological studies are beyond the scope of this paper. Examples of such studies include the randomized trials by Buscemi et al., comparing single data extraction to double data extraction [ 19 ], and Carrasco-Labra et al., comparing approaches to presenting findings in Grading of Recommendations, Assessment, Development and Evaluations (GRADE) summary of findings tables [ 20 ]. In these studies, the unit of analysis is the person or groups of individuals applying the methods. We also direct readers to the Studies Within a Trial (SWAT) and Studies Within a Review (SWAR) programme operated through the Hub for Trials Methodology Research, for further reading as a potential useful resource for these types of experimental studies [ 21 ]. Lastly, this paper is not meant to inform the conduct of research using computational simulation and mathematical modeling for which some guidance already exists [ 22 ], or studies on the development of methods using consensus-based approaches.

When should we conduct a methodological study?

Methodological studies occupy a unique niche in health research that allows them to inform methodological advances. Methodological studies should also be conducted as pre-cursors to reporting guideline development, as they provide an opportunity to understand current practices, and help to identify the need for guidance and gaps in methodological or reporting quality. For example, the development of the popular Preferred Reporting Items of Systematic reviews and Meta-Analyses (PRISMA) guidelines were preceded by methodological studies identifying poor reporting practices [ 23 , 24 ]. In these instances, after the reporting guidelines are published, methodological studies can also be used to monitor uptake of the guidelines.

These studies can also be conducted to inform the state of the art for design, analysis and reporting practices across different types of health research fields, with the aim of improving research practices, and preventing or reducing research waste. For example, Samaan et al. conducted a scoping review of adherence to different reporting guidelines in health care literature [ 18 ]. Methodological studies can also be used to determine the factors associated with reporting practices. For example, Abbade et al. investigated journal characteristics associated with the use of the Participants, Intervention, Comparison, Outcome, Timeframe (PICOT) format in framing research questions in trials of venous ulcer disease [ 11 ].

How often are methodological studies conducted?

There is no clear answer to this question. Based on a search of PubMed, the use of related terms (“methodological review” and “meta-epidemiological study”) – and therefore, the number of methodological studies – is on the rise. However, many other terms are used to describe methodological studies. There are also many studies that explore design, conduct, analysis or reporting of research reports, but that do not use any specific terms to describe or label their study design in terms of “methodology”. This diversity in nomenclature makes a census of methodological studies elusive. Appropriate terminology and key words for methodological studies are needed to facilitate improved accessibility for end-users.

Why do we conduct methodological studies?

Methodological studies provide information on the design, conduct, analysis or reporting of primary and secondary research and can be used to appraise quality, quantity, completeness, accuracy and consistency of health research. These issues can be explored in specific fields, journals, databases, geographical regions and time periods. For example, Areia et al. explored the quality of reporting of endoscopic diagnostic studies in gastroenterology [ 25 ]; Knol et al. investigated the reporting of p -values in baseline tables in randomized trial published in high impact journals [ 26 ]; Chen et al. describe adherence to the Consolidated Standards of Reporting Trials (CONSORT) statement in Chinese Journals [ 27 ]; and Hopewell et al. describe the effect of editors’ implementation of CONSORT guidelines on reporting of abstracts over time [ 28 ]. Methodological studies provide useful information to researchers, clinicians, editors, publishers and users of health literature. As a result, these studies have been at the cornerstone of important methodological developments in the past two decades and have informed the development of many health research guidelines including the highly cited CONSORT statement [ 5 ].

Where can we find methodological studies?

Methodological studies can be found in most common biomedical bibliographic databases (e.g. Embase, MEDLINE, PubMed, Web of Science). However, the biggest caveat is that methodological studies are hard to identify in the literature due to the wide variety of names used and the lack of comprehensive databases dedicated to them. A handful can be found in the Cochrane Library as “Cochrane Methodology Reviews”, but these studies only cover methodological issues related to systematic reviews. Previous attempts to catalogue all empirical studies of methods used in reviews were abandoned 10 years ago [ 29 ]. In other databases, a variety of search terms may be applied with different levels of sensitivity and specificity.

Some frequently asked questions about methodological studies

In this section, we have outlined responses to questions that might help inform the conduct of methodological studies.

Q: How should I select research reports for my methodological study?

A: Selection of research reports for a methodological study depends on the research question and eligibility criteria. Once a clear research question is set and the nature of literature one desires to review is known, one can then begin the selection process. Selection may begin with a broad search, especially if the eligibility criteria are not apparent. For example, a methodological study of Cochrane Reviews of HIV would not require a complex search as all eligible studies can easily be retrieved from the Cochrane Library after checking a few boxes [ 30 ]. On the other hand, a methodological study of subgroup analyses in trials of gastrointestinal oncology would require a search to find such trials, and further screening to identify trials that conducted a subgroup analysis [ 31 ].

The strategies used for identifying participants in observational studies can apply here. One may use a systematic search to identify all eligible studies. If the number of eligible studies is unmanageable, a random sample of articles can be expected to provide comparable results if it is sufficiently large [ 32 ]. For example, Wilson et al. used a random sample of trials from the Cochrane Stroke Group’s Trial Register to investigate completeness of reporting [ 33 ]. It is possible that a simple random sample would lead to underrepresentation of units (i.e. research reports) that are smaller in number. This is relevant if the investigators wish to compare multiple groups but have too few units in one group. In this case a stratified sample would help to create equal groups. For example, in a methodological study comparing Cochrane and non-Cochrane reviews, Kahale et al. drew random samples from both groups [ 34 ]. Alternatively, systematic or purposeful sampling strategies can be used and we encourage researchers to justify their selected approaches based on the study objective.

Q: How many databases should I search?

A: The number of databases one should search would depend on the approach to sampling, which can include targeting the entire “population” of interest or a sample of that population. If you are interested in including the entire target population for your research question, or drawing a random or systematic sample from it, then a comprehensive and exhaustive search for relevant articles is required. In this case, we recommend using systematic approaches for searching electronic databases (i.e. at least 2 databases with a replicable and time stamped search strategy). The results of your search will constitute a sampling frame from which eligible studies can be drawn.

Alternatively, if your approach to sampling is purposeful, then we recommend targeting the database(s) or data sources (e.g. journals, registries) that include the information you need. For example, if you are conducting a methodological study of high impact journals in plastic surgery and they are all indexed in PubMed, you likely do not need to search any other databases. You may also have a comprehensive list of all journals of interest and can approach your search using the journal names in your database search (or by accessing the journal archives directly from the journal’s website). Even though one could also search journals’ web pages directly, using a database such as PubMed has multiple advantages, such as the use of filters, so the search can be narrowed down to a certain period, or study types of interest. Furthermore, individual journals’ web sites may have different search functionalities, which do not necessarily yield a consistent output.

Q: Should I publish a protocol for my methodological study?

A: A protocol is a description of intended research methods. Currently, only protocols for clinical trials require registration [ 35 ]. Protocols for systematic reviews are encouraged but no formal recommendation exists. The scientific community welcomes the publication of protocols because they help protect against selective outcome reporting, the use of post hoc methodologies to embellish results, and to help avoid duplication of efforts [ 36 ]. While the latter two risks exist in methodological research, the negative consequences may be substantially less than for clinical outcomes. In a sample of 31 methodological studies, 7 (22.6%) referenced a published protocol [ 9 ]. In the Cochrane Library, there are 15 protocols for methodological reviews (21 July 2020). This suggests that publishing protocols for methodological studies is not uncommon.

Authors can consider publishing their study protocol in a scholarly journal as a manuscript. Advantages of such publication include obtaining peer-review feedback about the planned study, and easy retrieval by searching databases such as PubMed. The disadvantages in trying to publish protocols includes delays associated with manuscript handling and peer review, as well as costs, as few journals publish study protocols, and those journals mostly charge article-processing fees [ 37 ]. Authors who would like to make their protocol publicly available without publishing it in scholarly journals, could deposit their study protocols in publicly available repositories, such as the Open Science Framework ( https://osf.io/ ).

Q: How to appraise the quality of a methodological study?

A: To date, there is no published tool for appraising the risk of bias in a methodological study, but in principle, a methodological study could be considered as a type of observational study. Therefore, during conduct or appraisal, care should be taken to avoid the biases common in observational studies [ 38 ]. These biases include selection bias, comparability of groups, and ascertainment of exposure or outcome. In other words, to generate a representative sample, a comprehensive reproducible search may be necessary to build a sampling frame. Additionally, random sampling may be necessary to ensure that all the included research reports have the same probability of being selected, and the screening and selection processes should be transparent and reproducible. To ensure that the groups compared are similar in all characteristics, matching, random sampling or stratified sampling can be used. Statistical adjustments for between-group differences can also be applied at the analysis stage. Finally, duplicate data extraction can reduce errors in assessment of exposures or outcomes.

Q: Should I justify a sample size?

A: In all instances where one is not using the target population (i.e. the group to which inferences from the research report are directed) [ 39 ], a sample size justification is good practice. The sample size justification may take the form of a description of what is expected to be achieved with the number of articles selected, or a formal sample size estimation that outlines the number of articles required to answer the research question with a certain precision and power. Sample size justifications in methodological studies are reasonable in the following instances:

• Comparing two groups
• Determining a proportion, mean or another quantifier
• Determining factors associated with an outcome using regression-based analyses

For example, El Dib et al. computed a sample size requirement for a methodological study of diagnostic strategies in randomized trials, based on a confidence interval approach [ 40 ].

Q: What should I call my study?

A: Other terms which have been used to describe/label methodological studies include “ methodological review ”, “methodological survey” , “meta-epidemiological study” , “systematic review” , “systematic survey”, “meta-research”, “research-on-research” and many others. We recommend that the study nomenclature be clear, unambiguous, informative and allow for appropriate indexing. Methodological study nomenclature that should be avoided includes “ systematic review” – as this will likely be confused with a systematic review of a clinical question. “ Systematic survey” may also lead to confusion about whether the survey was systematic (i.e. using a preplanned methodology) or a survey using “ systematic” sampling (i.e. a sampling approach using specific intervals to determine who is selected) [ 32 ]. Any of the above meanings of the words “ systematic” may be true for methodological studies and could be potentially misleading. “ Meta-epidemiological study” is ideal for indexing, but not very informative as it describes an entire field. The term “ review ” may point towards an appraisal or “review” of the design, conduct, analysis or reporting (or methodological components) of the targeted research reports, yet it has also been used to describe narrative reviews [ 41 , 42 ]. The term “ survey ” is also in line with the approaches used in many methodological studies [ 9 ], and would be indicative of the sampling procedures of this study design. However, in the absence of guidelines on nomenclature, the term “ methodological study ” is broad enough to capture most of the scenarios of such studies.

Q: Should I account for clustering in my methodological study?

A: Data from methodological studies are often clustered. For example, articles coming from a specific source may have different reporting standards (e.g. the Cochrane Library). Articles within the same journal may be similar due to editorial practices and policies, reporting requirements and endorsement of guidelines. There is emerging evidence that these are real concerns that should be accounted for in analyses [ 43 ]. Some cluster variables are described in the section: “ What variables are relevant to methodological studies?”

A variety of modelling approaches can be used to account for correlated data, including the use of marginal, fixed or mixed effects regression models with appropriate computation of standard errors [ 44 ]. For example, Kosa et al. used generalized estimation equations to account for correlation of articles within journals [ 15 ]. Not accounting for clustering could lead to incorrect p -values, unduly narrow confidence intervals, and biased estimates [ 45 ].

Q: Should I extract data in duplicate?

A: Yes. Duplicate data extraction takes more time but results in less errors [ 19 ]. Data extraction errors in turn affect the effect estimate [ 46 ], and therefore should be mitigated. Duplicate data extraction should be considered in the absence of other approaches to minimize extraction errors. However, much like systematic reviews, this area will likely see rapid new advances with machine learning and natural language processing technologies to support researchers with screening and data extraction [ 47 , 48 ]. However, experience plays an important role in the quality of extracted data and inexperienced extractors should be paired with experienced extractors [ 46 , 49 ].

Q: Should I assess the risk of bias of research reports included in my methodological study?

A : Risk of bias is most useful in determining the certainty that can be placed in the effect measure from a study. In methodological studies, risk of bias may not serve the purpose of determining the trustworthiness of results, as effect measures are often not the primary goal of methodological studies. Determining risk of bias in methodological studies is likely a practice borrowed from systematic review methodology, but whose intrinsic value is not obvious in methodological studies. When it is part of the research question, investigators often focus on one aspect of risk of bias. For example, Speich investigated how blinding was reported in surgical trials [ 50 ], and Abraha et al., investigated the application of intention-to-treat analyses in systematic reviews and trials [ 51 ].

Q: What variables are relevant to methodological studies?

A: There is empirical evidence that certain variables may inform the findings in a methodological study. We outline some of these and provide a brief overview below:

• Country: Countries and regions differ in their research cultures, and the resources available to conduct research. Therefore, it is reasonable to believe that there may be differences in methodological features across countries. Methodological studies have reported loco-regional differences in reporting quality [ 52 , 53 ]. This may also be related to challenges non-English speakers face in publishing papers in English.
• Authors’ expertise: The inclusion of authors with expertise in research methodology, biostatistics, and scientific writing is likely to influence the end-product. Oltean et al. found that among randomized trials in orthopaedic surgery, the use of analyses that accounted for clustering was more likely when specialists (e.g. statistician, epidemiologist or clinical trials methodologist) were included on the study team [ 54 ]. Fleming et al. found that including methodologists in the review team was associated with appropriate use of reporting guidelines [ 55 ].
• Source of funding and conflicts of interest: Some studies have found that funded studies report better [ 56 , 57 ], while others do not [ 53 , 58 ]. The presence of funding would indicate the availability of resources deployed to ensure optimal design, conduct, analysis and reporting. However, the source of funding may introduce conflicts of interest and warrant assessment. For example, Kaiser et al. investigated the effect of industry funding on obesity or nutrition randomized trials and found that reporting quality was similar [ 59 ]. Thomas et al. looked at reporting quality of long-term weight loss trials and found that industry funded studies were better [ 60 ]. Kan et al. examined the association between industry funding and “positive trials” (trials reporting a significant intervention effect) and found that industry funding was highly predictive of a positive trial [ 61 ]. This finding is similar to that of a recent Cochrane Methodology Review by Hansen et al. [ 62 ]
• Journal characteristics: Certain journals’ characteristics may influence the study design, analysis or reporting. Characteristics such as journal endorsement of guidelines [ 63 , 64 ], and Journal Impact Factor (JIF) have been shown to be associated with reporting [ 63 , 65 – 67 ].
• Study size (sample size/number of sites): Some studies have shown that reporting is better in larger studies [ 53 , 56 , 58 ].
• Year of publication: It is reasonable to assume that design, conduct, analysis and reporting of research will change over time. Many studies have demonstrated improvements in reporting over time or after the publication of reporting guidelines [ 68 , 69 ].
• Type of intervention: In a methodological study of reporting quality of weight loss intervention studies, Thabane et al. found that trials of pharmacologic interventions were reported better than trials of non-pharmacologic interventions [ 70 ].
• Interactions between variables: Complex interactions between the previously listed variables are possible. High income countries with more resources may be more likely to conduct larger studies and incorporate a variety of experts. Authors in certain countries may prefer certain journals, and journal endorsement of guidelines and editorial policies may change over time.

Q: Should I focus only on high impact journals?

A: Investigators may choose to investigate only high impact journals because they are more likely to influence practice and policy, or because they assume that methodological standards would be higher. However, the JIF may severely limit the scope of articles included and may skew the sample towards articles with positive findings. The generalizability and applicability of findings from a handful of journals must be examined carefully, especially since the JIF varies over time. Even among journals that are all “high impact”, variations exist in methodological standards.

Q: Can I conduct a methodological study of qualitative research?

A: Yes. Even though a lot of methodological research has been conducted in the quantitative research field, methodological studies of qualitative studies are feasible. Certain databases that catalogue qualitative research including the Cumulative Index to Nursing & Allied Health Literature (CINAHL) have defined subject headings that are specific to methodological research (e.g. “research methodology”). Alternatively, one could also conduct a qualitative methodological review; that is, use qualitative approaches to synthesize methodological issues in qualitative studies.

Q: What reporting guidelines should I use for my methodological study?

A: There is no guideline that covers the entire scope of methodological studies. One adaptation of the PRISMA guidelines has been published, which works well for studies that aim to use the entire target population of research reports [ 71 ]. However, it is not widely used (40 citations in 2 years as of 09 December 2019), and methodological studies that are designed as cross-sectional or before-after studies require a more fit-for purpose guideline. A more encompassing reporting guideline for a broad range of methodological studies is currently under development [ 72 ]. However, in the absence of formal guidance, the requirements for scientific reporting should be respected, and authors of methodological studies should focus on transparency and reproducibility.

Q: What are the potential threats to validity and how can I avoid them?

A: Methodological studies may be compromised by a lack of internal or external validity. The main threats to internal validity in methodological studies are selection and confounding bias. Investigators must ensure that the methods used to select articles does not make them differ systematically from the set of articles to which they would like to make inferences. For example, attempting to make extrapolations to all journals after analyzing high-impact journals would be misleading.

Many factors (confounders) may distort the association between the exposure and outcome if the included research reports differ with respect to these factors [ 73 ]. For example, when examining the association between source of funding and completeness of reporting, it may be necessary to account for journals that endorse the guidelines. Confounding bias can be addressed by restriction, matching and statistical adjustment [ 73 ]. Restriction appears to be the method of choice for many investigators who choose to include only high impact journals or articles in a specific field. For example, Knol et al. examined the reporting of p -values in baseline tables of high impact journals [ 26 ]. Matching is also sometimes used. In the methodological study of non-randomized interventional studies of elective ventral hernia repair, Parker et al. matched prospective studies with retrospective studies and compared reporting standards [ 74 ]. Some other methodological studies use statistical adjustments. For example, Zhang et al. used regression techniques to determine the factors associated with missing participant data in trials [ 16 ].

With regard to external validity, researchers interested in conducting methodological studies must consider how generalizable or applicable their findings are. This should tie in closely with the research question and should be explicit. For example. Findings from methodological studies on trials published in high impact cardiology journals cannot be assumed to be applicable to trials in other fields. However, investigators must ensure that their sample truly represents the target sample either by a) conducting a comprehensive and exhaustive search, or b) using an appropriate and justified, randomly selected sample of research reports.

Even applicability to high impact journals may vary based on the investigators’ definition, and over time. For example, for high impact journals in the field of general medicine, Bouwmeester et al. included the Annals of Internal Medicine (AIM), BMJ, the Journal of the American Medical Association (JAMA), Lancet, the New England Journal of Medicine (NEJM), and PLoS Medicine ( n  = 6) [ 75 ]. In contrast, the high impact journals selected in the methodological study by Schiller et al. were BMJ, JAMA, Lancet, and NEJM ( n  = 4) [ 76 ]. Another methodological study by Kosa et al. included AIM, BMJ, JAMA, Lancet and NEJM ( n  = 5). In the methodological study by Thabut et al., journals with a JIF greater than 5 were considered to be high impact. Riado Minguez et al. used first quartile journals in the Journal Citation Reports (JCR) for a specific year to determine “high impact” [ 77 ]. Ultimately, the definition of high impact will be based on the number of journals the investigators are willing to include, the year of impact and the JIF cut-off [ 78 ]. We acknowledge that the term “generalizability” may apply differently for methodological studies, especially when in many instances it is possible to include the entire target population in the sample studied.

Finally, methodological studies are not exempt from information bias which may stem from discrepancies in the included research reports [ 79 ], errors in data extraction, or inappropriate interpretation of the information extracted. Likewise, publication bias may also be a concern in methodological studies, but such concepts have not yet been explored.

A proposed framework

In order to inform discussions about methodological studies, the development of guidance for what should be reported, we have outlined some key features of methodological studies that can be used to classify them. For each of the categories outlined below, we provide an example. In our experience, the choice of approach to completing a methodological study can be informed by asking the following four questions:

• What is the aim?

A methodological study may be focused on exploring sources of bias in primary or secondary studies (meta-bias), or how bias is analyzed. We have taken care to distinguish bias (i.e. systematic deviations from the truth irrespective of the source) from reporting quality or completeness (i.e. not adhering to a specific reporting guideline or norm). An example of where this distinction would be important is in the case of a randomized trial with no blinding. This study (depending on the nature of the intervention) would be at risk of performance bias. However, if the authors report that their study was not blinded, they would have reported adequately. In fact, some methodological studies attempt to capture both “quality of conduct” and “quality of reporting”, such as Richie et al., who reported on the risk of bias in randomized trials of pharmacy practice interventions [ 80 ]. Babic et al. investigated how risk of bias was used to inform sensitivity analyses in Cochrane reviews [ 81 ]. Further, biases related to choice of outcomes can also be explored. For example, Tan et al investigated differences in treatment effect size based on the outcome reported [ 82 ].

Methodological studies may report quality of reporting against a reporting checklist (i.e. adherence to guidelines) or against expected norms. For example, Croituro et al. report on the quality of reporting in systematic reviews published in dermatology journals based on their adherence to the PRISMA statement [ 83 ], and Khan et al. described the quality of reporting of harms in randomized controlled trials published in high impact cardiovascular journals based on the CONSORT extension for harms [ 84 ]. Other methodological studies investigate reporting of certain features of interest that may not be part of formally published checklists or guidelines. For example, Mbuagbaw et al. described how often the implications for research are elaborated using the Evidence, Participants, Intervention, Comparison, Outcome, Timeframe (EPICOT) format [ 30 ].

Sometimes investigators may be interested in how consistent reports of the same research are, as it is expected that there should be consistency between: conference abstracts and published manuscripts; manuscript abstracts and manuscript main text; and trial registration and published manuscript. For example, Rosmarakis et al. investigated consistency between conference abstracts and full text manuscripts [ 85 ].

In addition to identifying issues with reporting in primary and secondary studies, authors of methodological studies may be interested in determining the factors that are associated with certain reporting practices. Many methodological studies incorporate this, albeit as a secondary outcome. For example, Farrokhyar et al. investigated the factors associated with reporting quality in randomized trials of coronary artery bypass grafting surgery [ 53 ].

Methodological studies may also be used to describe methods or compare methods, and the factors associated with methods. Muller et al. described the methods used for systematic reviews and meta-analyses of observational studies [ 86 ].

Some methodological studies synthesize results from other methodological studies. For example, Li et al. conducted a scoping review of methodological reviews that investigated consistency between full text and abstracts in primary biomedical research [ 87 ].

Some methodological studies may investigate the use of names and terms in health research. For example, Martinic et al. investigated the definitions of systematic reviews used in overviews of systematic reviews (OSRs), meta-epidemiological studies and epidemiology textbooks [ 88 ].

In addition to the previously mentioned experimental methodological studies, there may exist other types of methodological studies not captured here.

• 2. What is the design?

Most methodological studies are purely descriptive and report their findings as counts (percent) and means (standard deviation) or medians (interquartile range). For example, Mbuagbaw et al. described the reporting of research recommendations in Cochrane HIV systematic reviews [ 30 ]. Gohari et al. described the quality of reporting of randomized trials in diabetes in Iran [ 12 ].

Some methodological studies are analytical wherein “analytical studies identify and quantify associations, test hypotheses, identify causes and determine whether an association exists between variables, such as between an exposure and a disease.” [ 89 ] In the case of methodological studies all these investigations are possible. For example, Kosa et al. investigated the association between agreement in primary outcome from trial registry to published manuscript and study covariates. They found that larger and more recent studies were more likely to have agreement [ 15 ]. Tricco et al. compared the conclusion statements from Cochrane and non-Cochrane systematic reviews with a meta-analysis of the primary outcome and found that non-Cochrane reviews were more likely to report positive findings. These results are a test of the null hypothesis that the proportions of Cochrane and non-Cochrane reviews that report positive results are equal [ 90 ].

• 3. What is the sampling strategy?

Methodological reviews with narrow research questions may be able to include the entire target population. For example, in the methodological study of Cochrane HIV systematic reviews, Mbuagbaw et al. included all of the available studies ( n  = 103) [ 30 ].

Many methodological studies use random samples of the target population [ 33 , 91 , 92 ]. Alternatively, purposeful sampling may be used, limiting the sample to a subset of research-related reports published within a certain time period, or in journals with a certain ranking or on a topic. Systematic sampling can also be used when random sampling may be challenging to implement.

• 4. What is the unit of analysis?

Many methodological studies use a research report (e.g. full manuscript of study, abstract portion of the study) as the unit of analysis, and inferences can be made at the study-level. However, both published and unpublished research-related reports can be studied. These may include articles, conference abstracts, registry entries etc.

Some methodological studies report on items which may occur more than once per article. For example, Paquette et al. report on subgroup analyses in Cochrane reviews of atrial fibrillation in which 17 systematic reviews planned 56 subgroup analyses [ 93 ].

This framework is outlined in Fig.  2 .

A proposed framework for methodological studies

Conclusions

Methodological studies have examined different aspects of reporting such as quality, completeness, consistency and adherence to reporting guidelines. As such, many of the methodological study examples cited in this tutorial are related to reporting. However, as an evolving field, the scope of research questions that can be addressed by methodological studies is expected to increase.

In this paper we have outlined the scope and purpose of methodological studies, along with examples of instances in which various approaches have been used. In the absence of formal guidance on the design, conduct, analysis and reporting of methodological studies, we have provided some advice to help make methodological studies consistent. This advice is grounded in good contemporary scientific practice. Generally, the research question should tie in with the sampling approach and planned analysis. We have also highlighted the variables that may inform findings from methodological studies. Lastly, we have provided suggestions for ways in which authors can categorize their methodological studies to inform their design and analysis.

Acknowledgements

Abbreviations, authors’ contributions.

LM conceived the idea and drafted the outline and paper. DOL and LT commented on the idea and draft outline. LM, LP and DOL performed literature searches and data extraction. All authors (LM, DOL, LT, LP, DBA) reviewed several draft versions of the manuscript and approved the final manuscript.

This work did not receive any dedicated funding.

Availability of data and materials

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

DOL, DBA, LM, LP and LT are involved in the development of a reporting guideline for methodological studies.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Download the free Kindle app and start reading Kindle books instantly on your smartphone, tablet or computer – no Kindle device required .

Read instantly on your browser with Kindle for Web.

Using your mobile phone camera, scan the code below and download the Kindle app.

Image Unavailable

• To view this video, download Flash Player

Research methodology: An introduction to scientific thinking and practice Paperback

• Language English
• Dimensions 21 x 1.8 x 29.7 cm
• ISBN-10 9043037885
• ISBN-13 978-9043037884
• See all details

Product details

• Language ‏ : ‎ English
• ISBN-10 ‏ : ‎ 9043037885
• ISBN-13 ‏ : ‎ 978-9043037884
• Item weight ‏ : ‎ 817 g
• Dimensions ‏ : ‎ 21 x 1.8 x 29.7 cm

Customer reviews

No customer reviews.

• Amazon and Our Planet
• Investor Relations
• Press Releases
• Amazon Science
• Sell on Amazon
• Supply to Amazon
• Become an Affiliate
• Protect & Build Your Brand
• Sell on Amazon Handmade
• Advertise Your Products
• Independently Publish with Us
• Host an Amazon Hub
• Amazon.ca Rewards Mastercard
• Shop with Points
• Reload Your Balance
• Amazon Currency Converter
• Amazon Cash
• Shipping Rates & Policies
• Amazon Prime
• Returns Are Easy
• Manage your Content and Devices
• Recalls and Product Safety Alerts
• Customer Service
• Conditions of Use
• Privacy Notice
• Interest-Based Ads
• Amazon.com.ca ULC | 40 King Street W 47th Floor, Toronto, Ontario, Canada, M5H 3Y2 |1-877-586-3230

Scientific Method

1. the practice of science: an introduction to research methods.

When some people think of science, they think of formulas and facts to memorize. Many of us probably studied for a test in a science class by memorizing the names of the four nucleotides in DNA (adenine, cytosine, guanine, and thymine) or by practicing with one of Newton ‘s laws of motion, like f = ma (force equals mass times acceleration). While this knowledge is an important part of science, it is not all of science. In addition to a body of knowledge that includes formulas and facts, science is a practice by which we pursue answers to questions that can be approached scientifically. This practice is referred to collectively as scientific research , and while the techniques that scientists use to conduct research may differ between disciplines, the underlying principles and objectives are similar. Whether you are talking about biology, chemistry, geology, physics, or any other scientific field, the body of knowledge that is built through these disciplines is based on the collection of data that are then analyzed and interpreted in light of other research findings. How do we know about adenine, cytosine, guanine, and thymine? These were not revealed by chance, but through the work of many scientists collecting data, evaluating the results, and putting together a comprehensive theory that explained their observations .

A brief history of scientific practice

The recorded roots of formal scientific research lie in the collective work of a number of individuals in ancient Greek, Persian, Arab, Indian, Chinese, and European cultures, rather than from a single person or event. The Greek mathematician Pythagoras is regarded as the first person to promote a scientific hypothesis when, based on his descriptive study of the movement of stars in the sky in the 5 th century BCE , he proposed that the Earth was round. The Indian mathematician and astronomer Aryabhata used descriptive records regarding the movement of objects in the night sky to propose in the 6 th century CE that the sun was the center of the solar system. In the 9 th century, Chinese alchemists invented gunpowder while performing experiments attempting to make gold from other substances. And the Middle Eastern scientist Alhazen is credited with devising the concept of the scientific experiment while researching properties related to vision and light around 1000 CE.

These and other events demonstrate that a scientific approach to addressing questions about the natural world has long been present in many cultures. The roots of modern scientific research methods , however, are considered by many historians to lie in the Scientific Revolution that occurred in Europe in the 16 th and 17 th centuries. Most historians cite the beginning of the Scientific Revolution as the publication of De Revolutionibus Orbium Coelestium (On the Revolutions of the Heavenly Spheres) in 1543 by the Polish astronomer Nicolaus Copernicus . Copernicus’s careful observation and description of the movement of planets in relation to the Earth led him to hypothesize that the sun was the center of the solar system and the planets revolved around the sun in progressively larger orbits in the following order: Mercury, Venus, Earth, Mars, Jupiter, and Saturn (Figure 1). Though Copernicus was not the first person to propose a heliocentric view of the solar system, his systematic gathering of data provided a rigorous argument that challenged the commonly held belief that Earth was the center of the universe .

The Scientific Revolution was subsequently fueled by the work of Galileo Galilei , Johannes Kepler , Isaac Newton (Figure 2), and others, who not only challenged the traditional geocentric view of the universe , but explicitly rejected the older philosophical approaches to natural science popularized by Aristotle . A key event marking the rejection of the philosophical method was the publication of Novum Organum: New Directions Concerning the Interpretation of Nature by Francis Bacon in 1620. Bacon was not a scientist, but rather an English philosopher and essayist, and Novum is a work on logic. In it, Bacon presented an inductive method of reasoning that he argued was superior to the philosophical approach of Aristotle. The Baconian method involved a repeating cycle of observation , hypothesis , experimentation, and the need for independent verification. Bacon’s work championed a method that was objective, logical, and empirical and provided a basis for the development of scientific research methodology .

Bacon’s method of scientific reasoning was further refined by the publication of Philosophiæ Naturalis Principia Mathematica (Mathematical Principles of Natural Philosophy) by the English physicist and mathematician Isaac Newton in 1686. Principia established four rules (described in more detail here ) that have become the basis of modern approaches to science. In brief, Newton ‘s rules proposed that the simplest explanation of natural phenomena is often the best, countering the practice that was common in his day of assigning complicated explanations derived from belief systems , the occult, and observations of natural events. And Principia maintained that special explanations of new data should not be used when a reasonable explanation already exists, specifically criticizing the tendency of many of Newton ‘s contemporaries to embellish the significance of their findings with exotic new explanations.

Bacon and Newton laid the foundation that has been built upon by modern scientists and researchers in developing a rigorous methodology for investigating natural phenomena. In particular, the English statisticians Karl Pearson and Ronald Fisher significantly refined scientific research in the 20 th century by developing statistical techniques for data analysis and research design (see our Statistics in Science module). And the practice of science continues to evolve today, as new tools and technologies become available and our knowledge about the natural world grows. The practice of science is commonly misrepresented as a simple, four- or five-step path to answering a scientific question, called “The Scientific Method.” In reality, scientists rarely follow such a straightforward path through their research. Instead, scientific research includes many possible paths, not all of which lead to unequivocal answers. The real scientific method , or practice of science, is much more dynamic and interesting.

Comprehension Checkpoint

Scientific research, if done correctly, follows a straightforward five-step path and leads to definite answers.

More than one Scientific Method

The typical presentation of the Scientific Method (Figure 3) suggests that scientific research follows a linear path, proceeding from a question through observation , hypothesis formation, experimentation, and finally producing results and a conclusion. However, scientific research does not always proceed linearly. For example, prior to the mid 1800s, a popular scientific hypothesis held that maggots and microorganisms could be spontaneously generated from the inherent life-force that existed in some foods. Louis Pasteur doubted this hypothesis, and this led him to conduct a series of experiments that would eventually disprove the theory of spontaneous generation (see our Experimentation in Scientific Research module). Pasteur’s work would be difficult to characterize using Figure 3 – while it did involve experimentation, he did not develop a hypothesis prior to his experiments. Instead he was motivated to disprove an existing hypothesis. Or consider the work of Grove Karl Gilbert , who conducted research on the Henry Mountains in Utah in the late 1800s (see our Description in Scientific Research module). Gilbert was not drawn to the area by a pressing scientific question, but rather he was sent there by the US government to explore the region. Further, Gilbert did not perform a single experiment in the Henry Mountains; his work was based solely on observation and description, yet no one would dispute that Gilbert was practicing science. The traditional and simplistic Scientific Method presented in Figure 3 does not begin to reflect the richness or diversity of scientific research, let alone the diversity of scientists themselves.

Scientific research methods

Scientific research is a robust and dynamic practice that employs multiple methods toward investigating phenomena, including experimentation, description, comparison, and modeling. Though these methods are described separately both here and in more detail in subsequent modules, many of these methods overlap or are used in combination. For example, when NASA scientists purposefully slammed a 370 kg spacecraft named Deep Impact into a passing comet in 2005, the study had some aspects of descriptive research and some aspects of experimental research (see our Experimentation in Scientific Research module). Many scientific investigations largely employ one method, but different methods may be combined in a single study, or a single study may have characteristics of more than one method. The choice of which research method to use is personal and depends on the experiences of the scientists conducting the research and the nature of the question they are seeking to address. Despite the overlap and interconnectedness of these research methods, it is useful to discuss them separately to understand the principal characteristics of each and the ways they can be used to investigate a question.

Experimentation: Experimental methods are used to investigate the relationship(s) between two or more variables when at least one of those variables can be intentionally controlled or manipulated. The resulting effect of that manipulation (often called a treatment) can then be measured on another variable or variables. The work of the French scientist Louis Pasteur is a classic example. Pasteur put soup broth in a series of flasks, some open to the atmosphere and others sealed. He then measured the effect that the flask type had on the appearance of microorganisms in the soup broth in an effort to study the source of those microorganisms (see our Experimentation in Science module). Description: Description is used to gather data regarding natural phenomena and natural relationships and includes observations and measurements of behaviors. A classic example of a descriptive study is Copernicus’s observations and sketches of the movement of planets in the sky in an effort to determine if Earth or the sun is the orbital center of those objects (see our Description in Scientific Research module). Comparison: Comparison is used to determine and quantify relationships between two or more variables by observing different groups that either by choice or circumstance are exposed to different treatments. Examples of comparative research are the studies that were initiated in the 1950s to investigate the relationship between cigarette smoking and lung cancer in which scientists compared individuals who had chosen to smoke of their own accord with nonsmokers and correlated the decision to smoke (the treatment) with various health problems including lung cancer (see our Comparison in Scientific Research module). Modeling: Both physical and computer-based models are built to mimic natural systems and then used to conduct experiments or make observations. Weather forecasts are an example of scientific modeling that we see every day, where data collected on temperature, wind speed, and direction are used in combination with known physics of atmospheric circulation to predict the path of storms and other weather patterns (see our Modeling in Scientific Research module).

These methods are interconnected and are often used in combination to fully understand complex phenomenon. Modeling and experimentation are ways of simplifying systems toward understanding causality and future events. However, both rely on assumptions and knowledge of existing systems that can be provided by descriptive studies or other experiments . Description and comparison are used to understand existing systems and are used to examine the application of experimental and modeling results in real-world systems. Results from descriptive and comparative studies are often used to confirm causal relationships identified by models and experiments. While some questions lend themselves to one or another strategy due to the scope or nature of the problem under investigation, most areas of scientific research employ all of these methods as a means of complementing one another toward clarifying a specific hypothesis , theory , or idea in science.

Scientific research methods, such as experimentation , description , comparison , and modeling ,

• are interconnected and are often used in combination.
• are more effective if used alone.

Research methods in practice: The investigation of stratospheric ozone depletion

Scientific theories are clarified and strengthened through the collection of data from more than one method that generate multiple lines of evidence . Take, for example, the various research methods used to investigate what came to be known as the “ozone hole.”

Early descriptive and comparative studies point to problem: In 1957, the British Antarctic Survey (BAS) began a descriptive study of stratospheric ozone levels in an effort to better understand the role that ozone plays in absorbing solar energy (MacDowall & Sutcliffe, 1960). For the next 20 years, the BAS recorded ozone levels and observed seasonal shifts in ozone levels, which they attributed to natural fluctuations. In the mid-1970s, however, the BAS began to note a dramatic drop in ozone levels that they correlated with the change of seasons in the Antarctic. Within a decade, they noted that a seasonal “ozone hole” (Figure 4) had begun to appear over the South Pole (Farman et al., 1985).

The development of new technology opens novel research paths: Concurrent with the early BAS studies, the British scientist James Lovelock was working on developing new technology for the detection of trace concentrations of gases and vapors in the atmosphere (Lovelock, 1960). One instrument that Lovelock invented was a sensitive electron capture detector that could quantify atmospheric levels of chlorofluorocarbons (CFCs). At the time, CFCs were widely used as refrigerants and as propellants in aerosol cans and they were thought to be stable in the atmosphere and thus harmless chemicals. In 1970, Lovelock began an observational study of atmospheric CFCs and found that the chemicals were indeed very stable and could be carried long distances from major urban air pollution sources by prevailing winds. Under the impression that CFCs were chemically inert, Lovelock proposed that the chemicals could be used as benign atmospheric tracers of large air mass movements (Lovelock, 1971).

Modeling and experimental research are used to draw causal connections: In 1972, F. Sherwood Rowland, a chemist at the University of California at Irvine, attended a lecture on Lovelock’s work. Rowland became interested in CFCs and began studying the subject with a colleague at Irvine, Mario Molina. Molina and Rowland were familiar with modeling research by Paul Crutzen, a researcher at the National Center for Atmospheric Research in Colorado, that had previously shown that nitrogen oxides are involved in chemical reactions in the stratosphere and can influence upper atmosphere ozone levels (Crutzen, 1970). They were also familiar with modeling research by Harold Johnston, an atmospheric chemist at the University of California at Berkeley, which suggested that nitrogen oxide emissions from supersonic jets could reduce stratospheric ozone levels (Johnston, 1971). With these studies in mind, they consulted experimental research published by Michael Clyne and Ronald Walker, two British chemists, regarding the reaction rates of several chlorine-containing compounds (Clyne & Walker, 1973). In 1974, Molina and Rowland published a landmark study in the journal Nature in which they modeled chemical kinetics to show that CFCs were not completely inert, and that they could be transported to high altitudes where they would break apart in strong sunlight and release chlorine radicals (Molina & Rowland, 1974). Molina and Rowland’s model predicted that the chlorine radicals, which are reactive, would cause the destruction of significant amounts of ozone in the stratosphere.

Descriptive and comparative research provide real-world confirmation : In 1976, a group of scientists led by Allan Lazrus at the National Center for Atmospheric Research in Boulder, Colorado, used balloons to carry instruments aloft that could sample air at high altitudes. In these samples, they were able to detect the presence of CFCs above the troposphere – confirming that CFCs did indeed reach the stratosphere and that once there, they could decompose in light (Lazrus et al., 1976). Further research conducted using balloons and high-atmosphere aircraft in the 1980s confirmed that chlorine and chlorine oxide radicals contribute to the loss of ozone over the Antarctic (McElroy et al., 1986). By the late 1980s, scientists began to examine the possible link between ozone loss and skin cancer because high levels of ultraviolet light, as would exist under an ozone hole, can cause skin cancer. In areas such as Southern Chile, where the Antarctic ozone hole overlaps with a populated land mass, a significant correlation was indeed found between the growing ozone hole and increasing rates of skin cancer (Abarca & Casiccia, 2002).

As a result of this collection of diverse yet complementary scientific evidence , the world community began to limit the use of CFCs and ratified the Montreal Protocol in 1988, which imposed strict international limits on CFC use. In 1995, Molina, Rowland, and Crutzen shared the Nobel Prize in chemistry for their research that contributed to our understanding of ozone chemistry.

The ozone story (further detailed in our Resources for this module; see The Ozone Depletion Phenomenon under Research) highlights an important point: Scientific research is multi-dimensional, non-linear, and often leads down unexpected pathways. James Lovelock had no intention of contributing to the ozone depletion story; his work was directed at quantifying atmospheric CFC levels. Although gaining an understanding of the ozone hole may appear as a linear progression of events when viewed in hindsight, this was not the case at the time. While each researcher or research team built on previous work, it is more accurate to portray the relationships between their studies as a web of networked events, not as a linear series. Lovelock’s work led Molina and Rowland to their ozone depletion models , but Lovelock’s work is also widely cited by researchers developing improved electron capture detectors. Molina and Rowland not only used Lovelock’s work, but they drew on the research of Crutzen, Johnston, Clyne, Walker, and many others. Any single research advance was subsequently pursued in a number of different directions that complemented and reinforced one another – a common phenomenon in science. The entire ozone story required modeling, experiments , comparative research, and descriptive studies to develop a coherent theory about the role of ozone in the atmosphere , how we as humans are affecting it, and how we are also affected by it.

The ozone research story shows that, in practice, scientific research is

• a linear step-by-step procedure.
• an interconnected web of related studies.

The real practice of science

Scientific research methods are part of the practice through which questions can be addressed scientifically. These methods all produce data that are subject to analysis and interpretation and lead to ideas in science such as hypotheses , theories , and laws . Scientific ideas are developed and disseminated through the literature, where individuals and groups may debate the interpretations and significance of the results. Eventually, as multiple lines of evidence add weight to an idea, it becomes an integral part of the body of knowledge that exists in science and feeds back into the research process . Figure 5 provides a graphical overview of the materials we have developed to explain the real practice of science, and the key elements are described below.

The Scientific Community: Scientists (see our Scientists and the Scientific Community module) draw on their background, experiences, and even prejudices in deciding on the types of questions they pursue and the research methods that they employ, and they are supported in their efforts by the scientific institutions and the community in which they work (see our Scientific Institutions and Societies module). Human nature makes it impossible for any scientist to be completely objective, but an important aspect of scientific research is that scientists are open to any potential result. Science emphasizes the use of multiple lines of evidence as a check on the objectivity of both individual scientists and the community at large. Research is repeated, multiple methods are used to investigate the same phenomenon, and scientists report these methods and their interpretations when publishing their work. Assuring the objectivity of data and interpretation is built into the culture of science. These common practices unite a community of science made up of individuals and institutions that are dedicated to advancing science. Rowland, Molina, Lovelock, and Crutzen each were guided by their personal interests and supported by their respective institutions. For example, in addition to his work with CFCs, James Lovelock is credited with proposing the Gaia hypothesis that all living and non-living things on the planet interact with one another much like a large, single organism. This perspective influenced his interest in looking at the movement of large air masses across the globe, work that was supported by funding from the National Aeronautics and Space Administration (NASA).

Data: Science is a way of understanding the world around us that is founded on the principal of gathering and analyzing data (see our Data Analysis and Interpretation module). In contrast, before the popularization of science, philosophical explanations of natural phenomena based on reasoning rather than data were common, and these led to a host of unsupported ideas, many of which have proven incorrect. For example, in addition to his ideas on vision, the Greek philosopher Empedocles also reasoned that because most animals are warm to the touch, they must contain fire inside of them (see our States of Matter module). In contrast, the initial conclusion of the presence of a hole in the stratospheric ozone layer was based on years of data collected by scientists at the British Antarctic Survey. The amount of uncertainty and error (see our Uncertainty, Error, and Confidence module) associated with these data was critical to record as well – a small error in Dobson units would have made the hole seemingly disappear. Using statistical methods (see our Statistics in Science module) and data visualization techniques (see our Using Graphs and Visual Data in Science module) to analyze data, the scientists at the BAS drew on their own experience and knowledge to interpret those data, demonstrating that the “hole” was more than a seasonal, natural shift in ozone levels.

Ideas in science: Scientific research contributes to the body of scientific knowledge, held in record in the scientific literature (see our Utilizing the Scientific Literature module) so that future scientists can learn from past work. The literature does not simply hold a record of all of the data that scientists have collected: It also includes scientists’ interpretations of those data. To express their ideas, scientists propose hypotheses to explain observations. For example, after observing, collecting, and interpreting data, Lovelock hypothesized that CFCs could be used by meteorologists as benign tracers of the movement of large air masses. While Lovelock was correct in his prediction that CFCs could be used to trace air movement, later research showed that they are not benign. This hypothesis was just one piece of evidence that Molina and Rowland used to form their theory of ozone depletion. Scientific theories (see our Theories, Hypotheses, and Laws module) are ideas that have held up under scrutiny and are supported by multiple lines of evidence. The ozone depletion theory is based on results from all of the studies described above, not just Lovelock’s work. Unlike hypotheses, which can be tenuous in nature, theories rely on multiple lines of evidence and so are durable. Still, theories may change and be refined as new evidence and analyses come to light. For example, in 2007, a group of NASA scientists reported experimental results showing that chlorine peroxide, a compound formed when CFCs are transported to the stratosphere and which participates in the destruction of ozone, has a slower reaction rate in the presence of ultraviolet light than previously thought (Pope et al., 2007). The work by Pope and his colleagues does not dispute the theory of ozone destruction; rather, it does suggest that some modifications may be necessary in terms of the reaction rates used in atmospheric chemistry models.

Despite the fact that different scientists use different methods , they can easily share results and communicate with one another because of the common language that has developed to present and interpret data and construct ideas. These shared characteristics allow studies as disparate as atmospheric chemistry, plant biology, and paleontology to be grouped together under the heading of “science.” Although a practicing scientist in any one of those disciplines will require very specialized factual knowledge to conduct their research , the broad similarities in methodology allow that knowledge to be shared across many disciplines.

Scientists use multiple methods to investigate the natural world and these interconnect and overlap, often with unexpected results. This module gives an overview of scientific research methods, data processing, and the practice of science. It discusses myths that many people believe about the scientific method and provides an introduction to our Research Methods series.

Key Concepts

• The practice of science involves many possible pathways. The classic description of the scientific method as a linear or circular process does not adequately capture the dynamic yet rigorous nature of the practice.
• Scientists use multiple research methods to gather data and develop hypotheses. These methods include experimentation, description, comparison, and modeling.
• Scientific research methods are complementary; when multiple lines of evidence independently support one another, hypotheses are strengthened and confidence in scientific conclusions improves.

Privacy Policy

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Research Methodology: An Introduction

Related Papers

Egesa Romans

According to Mugenda & Mugenda (2010), research is process of carrying out a diligent inquiry or a critical examination of a given phenomenonexhaustive study that follows some logical sequence. Mouly defines research as a process of arriving at effective solutions to problems through systematic collection, analysis and interpretation of data. Research also involves a critical analysis of existing conclusions or theories with regard to newly discovered facts Research is the continued search for knowledge and understanding of the world around us. Clifford Woody argues that research is the process of designing and redefining problems, formulating hypothesis or suggested solutions, collecting, organizing and evaluating data, making deductions and reaching conclusions and carefully testing the conclusions to determine whether they fit the formulated hypothesis.

Bakhtawer Zain

Research is any original and systematic investigation undertaken to increase knowledge and understanding and to establish facts and principles. It comprises the creation of ideas and generation of new knowledge that lead to new and improved insights and the development of new material, devices, products and processes. The word " research " perhaps originates from the old French word recerchier that meant to 'search again'. It implicitly assumes that the earlier search was not exhaustive and complete and hence a repeated search is called for.

Educational Journal

Ganizani Nkhambule

In the simplest of terms, the research definition is a process of seeking out knowledge. This knowledge can be new, or it can support an already known fact. The purpose of research is to inform and is based on collected and analyzed data. This exploration occurs systematically, where it is either tested or investigated to add to a body of knowledge. Research is a systematic and scientific approach to understanding the world around us. It is a process of inquiry that involves the collection and analysis of data to answer questions or solve problems.

Oscar Masaka

lulu farhan

Akshay Jain

Research may be very broadly defined as systematic gathering of data and information and its analysis for advancement of knowledge in any subject. research attempts to find answer intellectual and practical questions through application of systematic methods. Types of research can be classified in many different ways.

Melkammesrat Netsanet

RELATED PAPERS

STJ | Stellenbosch Theological Journal

Frits De Lange

Aquatic Sciences

Anton Schleiss

Mor Harchol-Balter

Procedia Economics and Finance

Linda Kestle

Edugama: Jurnal Kependidikan dan Sosial Keagamaan

Priyanggo Rahman

JAT Perspectives

Jennifer Igawa

Quaderni d'Italianistica

Elissa Tognozzi

Lorena Torres Bernardino

Updates in surgery

Stefano Bonardelli

The International Conference on ASEAN 2019

indra jaya wiranata

Lucas Kruger , Nuria Salmerón

Real Estate Management and Valuation

Joseph Ataguba

Quarterly Review of Film and Video

David Sterritt

Visitas al patio

Nayla Chehade

Proceedings of the National Academy of Sciences of the United States of America

Jerry Mellem

Farrukh Hussain

Montassar Zrida

Beiträge zur Geschichte der deutschen Sprache und Literatur

Hans-Olav Enger

tyghfg hjgfdfd

Tri Hesti Utaminingtyas

maryam kamaei

KSII Transactions on Internet and Information Systems

Alvin Sarraga Alon

Swathi Srinivasan

Brazilian Journal of Biology

Fabielle Marques

Journal of Radioanalytical and Nuclear Chemistry

hossein khoshhosn

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

Scientific Research and Methodology : An introduction to quantitative research and statistics

1 research: an introduction.

In this chapter, you will learn to:

• identify quantitative and qualitative research.
• identify the steps in the quantitative research process.

1.1 How do we know what we know?

If a soiled shirt is placed in the opening of a vessel containing grains of wheat, the reaction of the leaven in the shirt with fumes from the wheat will, after approximately twenty-one days, transform the wheat into mice.

This was called 'spontaneous generation' (or 'abiogenesis'). We now know this is incorrect... but how did the idea emerge? Through observation and experimentation .

Spontaneous generation was consistent with observations : the above recipe did produce mice in experiments. However, this hypothesis ('possible explanation') was rejected when later evidence, in better-designed experiments, contradicted the hypothesis. So, a new hypothesis was proposed to explain the appearance of the mice, which was tested against the evidence... and so on. Briefly, this is the evidence-based, scientific process .

More recently, the dangers of smoking were still being debated into the 1990s:

... a causal role for smoking [has] not been proved beyond reasonable doubt. --- Eysenck ( 1991 ) , p. 429

All scientific knowledge emerges in a similar way: observations lead to hypotheses, which are tested against evidence . If the evidence contradicts the hypothesis, the hypothesis is rejected. If the evidence is consistent with the hypothesis, the hypothesis is temporarily accepted (until any contradictory evidence emerges).

Hypotheses not contradicted by large amounts of evidence, over a long time, are sometimes called laws or theories (such as the 'law of conservation of energy'). Theories and laws can be disproven if contradictory evidence emerges. Knowledge in all scientific disciplines is accumulated using a similar evidence-based process.

1.2 Evidence-based research

Every discipline changes, develops, improves, and adapts---usually through research . Your discipline is not the same as it was \(10\) years ago; it will change in the next \(10\) years. Scientists, engineers and health practitioners need to know how to understand and adapt to this change.

To remain current in your discipline, understanding research is vital, even if you will not be conducting research yourself. You still need to know the language, tools, concepts and ideas of research: you need to know how to critique research. Research is the foundation of science.

Scientific research systematically answers questions using data ; that is, science seeks evidence-based answers . 'Evidence-based research' refers to research conclusions based on evidence , rather than hunches, feelings, intuition, hopes, or tradition. The evidence comes from analysing the collected data .

Definition 1.1 (Data) Data refers to information (observations or measurements) obtained from a study, such as numbers, labels, recordings, videos, text, etc.

A dataset refers to an organised and structured collection of data.

Research involves designing studies to collect data, and analysing that data; this book covers both.

1.3 Example: research in action

During 1988/1989, an unusually high number of cases of the Legionella longbeachae infection were observed in South Australia. The researchers wanted to identify the source of the infection to prevent further infections.

The researchers noticed that many of those infected were gardeners who had recently handled potting mix, so they hypothesised that the infection was associated with using potting mix. They designed a study to test this hypothesis, then collected data from \(100\) people ( \(25\) with the infection, and \(75\) people of similar age and sex without the infection).

The researchers classified and summarised the data, then analysed the data to reach an evidence-based conclusion: potting mix was partially, but not solely, responsible for the infections. The researchers communicated their recommendations to reduce the risks of people contracting the infection .

In this book, we learn about the six steps of research . Arrange these steps into the common order:

Step 1 : Design Classify, summarise Analyse Ask Collect Report the question.

Step 2 : Design Classify and summarise Analyse Ask Collect Report the study.

Step 3 : Design Classify and summarise Analyse Ask Collect Report the data.

Step 4 : Design Classify and summarise Analyse Ask Collect Report the data.

Step 5 : Design Classify and summarise Analyse Ask Collect Report the data.

Step 6 : Design Classify and summarise Analyse Ask Collect Report the results.

1.4 Types of research

Research can be broadly classified as qualitative or quantitative , which are different yet complementary approaches to answering research questions (Table  1.1 ). Both methods have advantages and disadvantages, and can be used together (called mixed methods research). Deciding to use qualitative, quantitative or mixed methods approaches depends on the purpose of the research.

Briefly, qualitative research leads to a deeper understanding, usually from a very narrowly-defined group. Meanings, motivations, opinions or themes often emerge from qualitative research. In contrast, quantitative research summarises and analyses data usually from large groups, using numerical methods, such as averages and percentages. In quantitative research, typically information about a large group of interest (a population ) is found from a subset of the population (a sample ).

Definition 1.2 (Quantitative research) Quantitative research summarises and analyses data using numerical methods, such as averages and percentages.

This book is about quantitative research.

Example 1.1 (Types of research) Suppose we wish to learn about the perceived benefits and barriers to people buying electric vehicles (EVs).

A qualitative research study might use two small focus groups : one group comprising people who purchased an EV, and another group comprising people who purchased a non-EV. The researchers ask each group about the reasons for their purchase.

A quantitative research study might survey a large number of buyers of EVs and buyers of non-EVs, and ask the buyers' age, sex, and multi-choice questions about the reasons for their purchase. The survey responses could be analysed by numerically summarising, and comparing, the responses for buyers and non-buyers of EVs.

A mixed methods study may combine both of the above.

1.5 The steps in research

The research process ideally follows the process in Fig.  1.1 , but this is not always possible or practical. The process is not always linear: researchers may jump from step to step as necessary, and research usually leads to new research questions. Nonetheless, each step is important:

FIGURE 1.1: The six basic steps in research

• Asking the research question (Chap.  2 ). Research begins with a question.
• Designing the study (Chaps.  3 to  9 ). Evidence-based research uses data to answer questions. A study must be designed to obtain that data, which includes determining who or what to study, how to find them, what information to obtain, and ensuring data are obtained ethically.
• Collecting the data (Chap.  10 ). The data collection process must be ethical, reproducible and clearly documented.
• Classifying and summarising the data (Chaps.  11 to  18 ). Before analysis, the data must be classified and summarised. (A computer is useful.)
• Analysing the data (Chaps.  19 to  39 ). Analysis refers to determining how the data answer the research question, and depends on the type of data and the research question. (A computer is useful.)
• Reporting the results (Chaps.  40 and  41 ). Communicating the results appropriately, accurately and ethically is important, including any limitations.

1.6 Using software in research

Statistical software (such as jamovi, R, SAS, SPSS, etc.) is useful for summarising and analysing data. Statistical software:

• is designed for working with large datasets.
• encourages reproducible research (Sect. 4.3 ).
• allows high-precision formatting and graphics.
• is powerful; with some programming skills, almost anything is possible.
• is specifically designed for analysing and working with data.

This book sometimes shows output from jamovi ( The jamovi Project 2022 ) , but using jamovi is not essential for understanding this book.

Using spreadsheets in research for storing and analysing data requires care; extremely expensive and dangerous errors have been made due to using spreadsheets ( AlTarawneh and Thorne 2017 ) :

• Spreadsheets may change data (e.g., reformatting entries as dates) when not appropriate ( Ziemann, Eren, and El-Osta 2016 ) .
• Spreadsheets may include formulas with errors that are difficult to locate and hence fix ( Panko 2016 ; London and Slagter 2021 ) .
• Spreadsheets often produce poor graphs ( Su 2008 ) .

Problems with spreadsheets, as with any software, are often due to human error, but spreadsheets make errors hard to find and fix . While spreadsheets can be useful for data collection and manipulation, they are not designed for scientific analysis. Be careful using spreadsheets for research and analysis.

1.7 Exercises

Answers to odd-numbered exercises are available in App.  E .

Exercise 1.1 Consider the research question: 'Which of three different junctional tourniquets are quickest, on average, to apply?' Is this RQ likely to be answered using a quantitative or qualitative research study? Explain.

Exercise 1.2 Consider the research question: 'Why do people dump rubbish in mangroves?' Is this RQ likely to be answered using a quantitative or qualitative research study? Explain.

Exercise 1.3 Consider the research question: 'What percentage of the population have side-effects from a medication?' Is this RQ likely to be answered using a quantitative or qualitative research study? Explain.

Exercise 1.4 Consider the research question: 'What is the average number of roof-top solar panels on domestic homes?' Is this RQ likely to be answered using a quantitative or qualitative research study? Explain.

An Introduction to Research Ethics and Scientific Integrity

• Living reference work entry
• First Online: 30 May 2019
• Cite this living reference work entry

• Ron Iphofen 2  

484 Accesses

1 Citations

This chapter outlines the aims for the handbook. A main aim is to be a first point of contact for contemporary information, issues, and challenges in the fields of research ethics and scientific integrity. It is aimed at researchers, reviewers, and policymakers to help them pursue the best ways forward in seeking ethics and integrity in all research across disciplines, methods, subjects, participants, and contexts. The authors form a global network of scholars, practitioners, and researchers with a range of experience and insights that scope a challenging field but one that is vital to the maintenance of research standards and public confidence in science. Fact-based policymaking remains under threat from political and ideological pressures. Scientists and researchers in all disciplines and professions hold a clear responsibility to protect their subjects, research participants, and society from pressures, interests, and prejudices that risk undermining the value of their work. This overview outlines how the handbook is constructed and how readers might gain from it.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Cadwalladr C (2018) Our Cambridge Analytica scoop shocked the world. But the whole truth remains elusive, The Guardian , 23rd December Accessed at: https://www.theguardian.com/uk-news/2018/dec/23/cambridge-analytica-facebook-scoop-carole-cadwalladr-shocked-world-truth-still-elusive

Cadwalladr C (2019) Cambridge Analytica a year on: ‘a lesson in institutional failure’, The Guardian, 17 March, accessed at: https://www.theguardian.com/uk-news/2019/mar/17/cambridge-analytica-year-on-lesson-in-institutional-failure-christopher-wylie

Cohen S, Young J (1973) The manufacture of news: a reader. Sage, London

Google Scholar  

Collingridge D (1980) The social control of technology. Frances Pinter, London

ESOMAR (2016) https://www.esomar.org/what-we-do/code-guidelines

Evans RJ (2002) Telling lies about Hitler: the holocaust, history and the David Irving trial. Verso, London

Fuhrman J (2013) Super immunity. HarperCollins, London

Goldacre B (2001) Bad pharma. HarperCollins, London

Gross J (2018) Poland’s death camp law is designed to falsify history, Financial Times, February 6, accessed at: https://www.ft.com/content/1c183f56-0a6a-11e8-bacb-2958fde95e5e

Hutton W, Adonis A (2018) Saving Britain: how we must change to prosper in Europe. Abacus, London

Ioris AAR (2015) Cracking the nut of agribusiness and global food insecurity: in search of a critical agenda of research. Geoforum 63:1–4. https://doi.org/10.1016/j.geoforum.2015.05.004

Article   Google Scholar  

Iphofen R (2017) Epistemological metaphors: orders of knowledge and control in the Encyclopedist myths of cyberspace. International Journal of Humanities and Cultural Studies. (ISSN 2356-5926) 4(2):122–141. https://www.ijhcs.com/index.php/ijhcs/article/view/3128

Levitt SD, Dubner SJ (2005) Freakonomics: a rogue economist explores the hidden side of everything. Allen Lane, London

Levitt SD, Dubner SJ (2009) SuperFreakonomics: global cooling, patriotic prostitutes, and why suicide bombers should buy life insurance. Harper Collins, New York

Portes R (2017) Who needs experts? The London Business Review, 9th May. Accessed at: https://www.london.edu/faculty-and-research/lbsr/who-needs-experts

Posner M (2006) Social sciences under attack in the UK (1981–1983), La revue pour l’histoire du CNRS [], 7|2002, 20 October 2006, 09 January 2017. http://histoire-cnrs.revues.org/547 ; https://doi.org/10.4000/histoire-cnrs.547

Schrag Z (2010) Ethical Imperialism: Institutional Review Boards and the Social Sciences, 1965–2009. The John Hopkins University Press, Baltimore

Download references

Author information

Authors and affiliations.

Independent Research Consultant, Chatelaillon Plage, France

Ron Iphofen

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Ron Iphofen .

Editor information

Editors and affiliations.

Chatelaillon Plage, France

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this entry

Cite this entry.

Iphofen, R. (2019). An Introduction to Research Ethics and Scientific Integrity. In: Iphofen, R. (eds) Handbook of Research Ethics and Scientific Integrity. Springer, Cham. https://doi.org/10.1007/978-3-319-76040-7_62-1

Download citation

DOI : https://doi.org/10.1007/978-3-319-76040-7_62-1

Received : 17 March 2019

Accepted : 01 May 2019

Published : 30 May 2019

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-76040-7

Online ISBN : 978-3-319-76040-7

eBook Packages : Springer Reference Religion and Philosophy Reference Module Humanities and Social Sciences Reference Module Humanities

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• No shipping costs from € 15, -
• Lists and tips from our own specialists
• Possibility of ordering without an account
• Customer support
• Educational: Humanities and social sciences, general

Research methodology

An introduction to scientific thinking and practice.

All bindings and editions

• Ordered on working days before 23h, dispatched the same day.
• Not in stock in our shop
• From €15,- no shipping costs.
• 30 days to change your mind and return physical products

Description

Je docent heeft dit gepersonaliseerde studieboek gecreëerd om je een optimale leerervaring te bieden. Het bevat Pearson content van onze gerenommeerde auteurs en bevat net die extra’s waarvan je docent weet dat je ze nodig hebt – niets meer en niets minder! Sommige docenten hebben er ook hun eigen, op maat gemaakte, leermaterialen aan toegevoegd, die specifiek gericht zijn op jouw opleiding.

Pearson werkt nauw samen met docenten en zet zich in voor het creëren van kwalitatief uitstekende leermiddelen, die op maat worden gemaakt. Geef je studievaardigheden een duwtje in de rug en begin nu met de voorbereiding van je toekomstige carrière. Meer informatie is te vinden op: www.pearson.com/nl.

Specifications

• Publisher Pearson Benelux B.V.
• Pub date Jul 2019
• Theme Educational: Humanities and social sciences, general
• Dimensions 297 x 210 x 18 mm
• Weight 812 gram
• EAN 9789043037884
• Paperback Paperback
• Language English

related products

Van klimaatcatastrofes naar een wij-Mensheid

De economie, de spelregels en het complot

Hoe word ik Meesteres?

Intervisie leren (4e editie)

Handboek ZZP Bouw 2024