empirical research methods in software engineering

Empirical research methods in software engineering

• Departments at LTH
• Department of Computer Science
• Software Engineering Research Group

Research output : Chapter in Book/Report/Conference proceeding › Book chapter › Research

Publication series

Subject classification (ukä).

• Computer Science

Access to Document

• 10.1007/978-3-540-45143-3_2

Fingerprint

• Software engineering Engineering & Materials Science 100%
• Engineering research Engineering & Materials Science 37%
• Project management Engineering & Materials Science 32%
• Experiments Engineering & Materials Science 11%

Research output

• 1 Book chapter

Research output per year

Empirical Research Methods in Software and Web Engineering

T1 - Empirical research methods in software engineering

AU - Wohlin, Claes

AU - Höst, Martin

AU - Henningsson, K

N2 - Software engineering is not only about technical solutions. It is to a large extent also concerned with organizational issues, project management and human behaviour. For a discipline like software engineering, empirical methods are crucial, since they allow for incorporating human behaviour into the research approach taken. Empirical methods are common practice in many other disciplines. This chapter provides a motivation for the use of empirical methods in software engineering research. The main motivation is that it is needed from an engineering perspective to allow for informed and well-grounded decision. The chapter continues with a brief introduction to four research methods: controlled experiments, case studies, surveys and postmortem analyses. These methods are then put into an improvement context. The four methods are presented with the objective to introduce the reader to the methods to a level that it is possible to select the most suitable method at a specific instance. The methods have in common that they all are concerned with quantitative data. However, several of them are also suitable for qualitative data. Finally, it is concluded that the methods are not competing. On the contrary, the different research methods can preferably be used together to obtain more sources of information that hopefully lead to more informed engineering decisions in software engineering.

AB - Software engineering is not only about technical solutions. It is to a large extent also concerned with organizational issues, project management and human behaviour. For a discipline like software engineering, empirical methods are crucial, since they allow for incorporating human behaviour into the research approach taken. Empirical methods are common practice in many other disciplines. This chapter provides a motivation for the use of empirical methods in software engineering research. The main motivation is that it is needed from an engineering perspective to allow for informed and well-grounded decision. The chapter continues with a brief introduction to four research methods: controlled experiments, case studies, surveys and postmortem analyses. These methods are then put into an improvement context. The four methods are presented with the objective to introduce the reader to the methods to a level that it is possible to select the most suitable method at a specific instance. The methods have in common that they all are concerned with quantitative data. However, several of them are also suitable for qualitative data. Finally, it is concluded that the methods are not competing. On the contrary, the different research methods can preferably be used together to obtain more sources of information that hopefully lead to more informed engineering decisions in software engineering.

U2 - 10.1007/978-3-540-45143-3_2

DO - 10.1007/978-3-540-45143-3_2

M3 - Book chapter

SN - 978-3-540-40672-3

BT - Lecture Notes in Computer Science (Empirical Methods and Studies in Software Engineering)

PB - Springer

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Empirical research methods for software engineering

2007, Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering - ASE '07

Related Papers

Eudis Teixeira , Liliane Fonseca

Context: Empirical studies are gaining recognition in the Software Engineering(SE) research community. In order to foster empirical research, it is essential understand the environments, guidelines, process, and other mechanisms available to support these studies in SE. Goal: Identifying the mechanisms used to support the empirical strategies adopted by the researches in the major Empirical Software Engineering (ESE) scientific venues. Method: We performed a systematic mapping study that included all full papers published at EASE, ESEM and ESEJ since their first editions. A total of 898 studies were selected. Results: We provide the full list of identified support mechanisms and the strategies that uses them. The most commonly mechanisms used to support the empirical strategies were two sets of guidelines, one to secondary studies and another to experiments. The most reported empirical strategies are experiments and case studies. Conclusions: The use of empirical methods in SE has increased over the years but many studies do not apply these methodsnor use mechanisms to guide their research. Therefore, the list of support mechanisms, where and how they were applied is a major asset to the SE community. Such asset can foster empirical studies aiding the choice regarding which strategies and mechanisms to use in a research. Also, we identified new perspectives and gaps that foster the development of resources to aid empirical studies.

Empirical Software Engineering

Norman Fenton

2021 Second International Conference on Information Systems and Software Technologies (ICI2ST)

Margarita Cruz

Marjo Kauppinen

Empirical Software …

Stefan Biffl

Felix Garcia

Alberto Sampaio

Empirical research is now common in the software engineering field. This paper aims to show the need for a reorientation of empirical research in software engineering through theory development and basic research. The importance of both for software engineering in the current development of empirical software engineering is explained and their relationship emphasized.

Lecture Notes in Computer Science

Walter Tichy

Maya Daneva

Abstract This article reports on an experimental evaluation of a unified checklist for case study and experimental research. The checklist aims at exhibiting the underlying, shared, structure of observational and experimental research, and is based on several published checklist in software engineering and other disciplines. We asked a small sample of subjects to apply it to two papers, one reporting on a case study and one reporting on an experiment.

RELATED PAPERS

World Technopolis Review

Cheol-Joo Cho

Mauricio nuñez

Tissue Engineering Part C: Methods

Angela Huang

Laval théologique et philosophique

René-Michel Roberge

Reyhan özer taniyan

Scientific Reports

Rushikesh Khadake

Muhammad Niqab

Pramesti Widya Kirana

Advanced Journal of Social Science

Elizabeth Abiola-Oke

Devir Educação

Zuleika Aparecida Claro Piassa

Plant cell reports

Dhinesh Kumar

Nilda Radman

Muhammad Sameer Khan

Thrombosis Research

Monica Castelli

Amninder Kaur

Journal of Molecular Biology

Gebhard F X Schertler

Annales Universitatis Paedagogicae Cracoviensis. Studia de Cultura

Magdalena Stoch

Rev Bras Zootecn

Marta Da Rocha

Shobirin Shopi

Nur Rakhmah Warni

Iranian Journal of Neonatology IJN

Reza Saeidi

Narodna umjetnost: Croatian Journal of Ethnology and Folklore Research

Ivana Krpan

Reports of Practical Oncology & Radiotherapy

莫纳什大学毕业证文凭办理成绩单修改 办理澳洲Monash文凭学位证书学历认证

International journal of nanomedicine

Nguyên Bảo Lê

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

Guide to Advanced Empirical Software Engineering pp 285–311 Cite as

Selecting Empirical Methods for Software Engineering Research

• Steve Easterbrook 4 ,
• Janice Singer 5 ,
• Margaret-Anne Storey 6 &
• Daniela Damian 6  

8784 Accesses

418 Citations

Selecting a research method for empirical software engineering research is problematic because the benefits and challenges to using each method are not yet well catalogued. Therefore, this chapter describes a number of empirical methods available. It examines the goals of each and analyzes the types of questions each best addresses. Theoretical stances behind the methods, practical considerations in the application of the methods and data collection are also briefly reviewed. Taken together, this information provides a suitable basis for both understanding and selecting from the variety of methods applicable to empirical software engineering.

This is a preview of subscription content, log in via an institution .

Buying options

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Unable to display preview.  Download preview PDF.

Bratthall, L. and Jørgensen, M. (2002) Can you trust a single data source exploratory software engineering case study? Journal of Empirical Software Engineering , 7(1), 9–26.

Article   MATH   Google Scholar  

Calhoun, C. (1995) Critical Social Theory: Culture, History, and the Challenge of Difference . Blackwell, Oxford, UK.

Google Scholar  

Chalmers, A. (1999) What Is This Thing Called Science? 3rd Edition, Hackett Publishing Co, Indianapolis.

Creswell, J.W. (2002) Research Design: Qualitative, Quantitative and Mixed Methods Approaches . 2nd Edition, Sage Publications, Thousand Oaks, CA.

Damian, D. and Chisan, J. (2006) An empirical study of the complex relationships between requirements engineering processes and other processes that lead to payoffs in productivity, quality and risk management, IEEE Transactions on Software Engineering , 32(8), 433–453.

Article   Google Scholar  

Damian, D.E., Eberlein, A., Shaw, M., and Gaines, B. (2000) Using different communication media in requirements negotiation, IEEE Software , 17(3), 28–36.

Davison, R.M., Martinsons, M.G., and Kock, N. (2004) Principles of canonical action research, Information Systems Journal , 14(1), 65–86.

Dittrich, Y. (2002) Doing Empirical Research on Software Development: Finding a Path Between Understanding, Intervention, and Method Development. In Social Thinking: Software Practice , Y. Dittrich, C. Floyd, and R. Klischewski, Eds. MIT Press.

Dittrich, Y., John, M., Singer, J., and Tessem, B. (2007) Editorial for the Special Issue on Qualitative Software Engineering Research, Information and Software Technology , 49(6), 531–539.

Flyvbjerg, B. (2006) Five misunderstandings about case study research, Qualitative Inquiry , 12(2), 219–245.

Glaser, B.G. and Strauss, A. (1967) Discovery of Grounded Theory: Strategies for Qualitative Research . Sociology Press, Mill Valley, CA.

Gregor, S. (2006) The Nature of Theories in Information Systems, MIS Quarterly , 30(3), 611–642.

Jørgensen, M. and Sjøberg, D.I.K. (2004) Generalization and Theory-Building in Software Engineering Research. IEE Proceedings, Workshop on Empirical Assessment in Software Engineering (EASE’04), at ICSE’04, pp. 29–36.

Kitchenham, B., Pickard, L., and Pfleeger, S.L. (1995) Case studies for method and tool evaluation, IEEE Software , 12(4), 52–62.

Klein, H.K. and Myers, M.D. (1999) A set of principles for conducting and evaluating interpretive field studies in information systems, MIS Quarterly , 23(1), 67–93.

Lau, F. (1999).Towards a framework for action research in information systems studies, Information Technology and People , 12(2), 148–175.

Lincoln, Y.S. and Guba, E.G. (1985) Naturalistic Inquiry . Sage, Beverly Hills, CA.

Littlejohn, S.W. and Foss, K.A. (2004) Theories of Human Communication . 8th Edition, Wadsworth Publishing, Belmont, CA.

McGrath, J.E. (1995) Methodology matters: doing research in the behavioral and social sciences. In Human–Computer Interaction: Toward the Year 2000 , R.M. Baecker, J. Grudin, W. Buxton, A., and Greenberg, S., Eds. Morgan Kaufmann Publishers, San Francisco, CA, pp. 152–169.

Meltzoff, J. (1998) Critical Thinking About Research: Psychology and Related Fields . American Psychological Association, Washington DC.

Menand, L. (1997) Pragmatism: A Reader . Vintage Press, New York.

Morse, J.M., Barrett, M., Mayan, M., Olson, K. and Spiers, J. (2002) Verification strategies for establishing reliability and validity in qualitative research, International Journal of Qualitative Methods , 1(2), 1–19.

Robinson, H., Segal, J. and Sharp, H. (2007) Ethnographically-informed empirical studies of software practice, Information and Software Technology , 49(6), 540–551.

Sandelowski, M. (1993) Rigor or rigor mortis: the problem of rigor in qualitative research revisited, Advances in Nursing Science , 16(2), 1–8.

Simon, H. (1996) The Sciences of the Artificial . 3rd Edition, MIT Press, Cambridge, MA.

Singer, J.A. and Vinson, N.G. (2002) Ethical issues in empirical studies of software engineering, IEEE Transactions on Software Engineering , 28(12), 1171–1180.

Varkevisser, C.M., Pathmanathan, I., and Brownlee, A. (2003) Designing and Conducting Health Systems Research Projects: Volume 1–Proposal Development and Fieldwork . Chapter 10: Data Collection Techniques. Available online at http://www.idrc.ca/en/ev-56605–201-1-DO_TOPIC.html .

Vinson, N.G. and Singer, J.A. (2004) Consent issues raised by observational research in organisations, NCEHR Communique , 12(2), 35–36.

Wieringa, R.J. and Heerkens, J.M.G. (2006) The methodological soundness of requirements engineering papers: a conceptual framework and two case studies, Requirements Engineering Journal , 11, 295–307.

Wohlin, C., Runesson, P., Höst, M., Ohlsson, M.C., Regnell, B., and Wesslén, A. (2000) Introduction to Experimentation in Software Engineering . Kluwer Academic Publishers, Boston, MA.

Yin, R.K. (2002) Case Study Research: Design and Methods . Sage, Thousand Oaks, CA.

Download references

Author information

Authors and affiliations.

Department of Computer Science, University of Toronto, M5S 2E4, Toronto, Ontario, Canada

Steve Easterbrook

Institute for Information Technology, National Research Council Canada, K1A 0R6, Ottawa, Ontario, Canada

Janice Singer

Dept. of Computer Science, University of Victoria, V8W 3P6, Victoria, BC, Canada

Margaret-Anne Storey & Daniela Damian

You can also search for this author in PubMed   Google Scholar

Editor information

Editors and affiliations.

Fraunhofer Center Maryland, 20742, College Park, MD, USA

Forrest Shull

National Research Council Canada, K1A 0R6, Ottawa, Ontario, Canada

Simula Research Laboratory, NO-1325, Lysaker, Norway

Dag I. K. Sjøberg

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag London Limited

About this chapter

Cite this chapter.

Easterbrook, S., Singer, J., Storey, MA., Damian, D. (2008). Selecting Empirical Methods for Software Engineering Research. In: Shull, F., Singer, J., Sjøberg, D.I.K. (eds) Guide to Advanced Empirical Software Engineering. Springer, London. https://doi.org/10.1007/978-1-84800-044-5_11

Download citation

DOI : https://doi.org/10.1007/978-1-84800-044-5_11

Publisher Name : Springer, London

Print ISBN : 978-1-84800-043-8

Online ISBN : 978-1-84800-044-5

eBook Packages : Computer Science Computer Science (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

The Future of Empirical Methods in Software Engineering Research

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

• Software Engineering Tutorial
• Software Development Life Cycle
• Waterfall Model
• Software Requirements
• Software Measurement and Metrics
• Software Design Process
• System configuration management
• Software Maintenance
• Software Development Tutorial
• Software Testing Tutorial
• Product Management Tutorial
• Project Management Tutorial
• Agile Methodology
• Selenium Basics

Empirical Research in Software Engineering

• AI in Software Engineering
• Re-engineering - Software Engineering
• Reverse Engineering - Software Engineering
• Defect Prevention in Software Engineering
• Software Processes in Software Engineering
• Software Re-Engineering
• Software Crisis - Software Engineering
• Requirements Elicitation - Software Engineering
• Product Metrics in Software Engineering
• Classification of Software - Software Engineering
• Software Evolution - Software Engineering
• Principles of Conventional Software Engineering
• Software Characteristics - Software Engineering
• Software Review - Software Engineering
• Comparison Testing in Software Engineering
• Changing Nature of Software - Software Engineering
• Inspection Metrics in Software Engineering
• ISO/IEC 9126 in Software Engineering
• Curve Fitting models - Software Engineering
• Types of Software Testing
• Spiral Model - Software Engineering
• Differences between Black Box Testing vs White Box Testing
• COCOMO Model - Software Engineering
• Coupling and Cohesion - Software Engineering
• Functional vs Non Functional Requirements
• Waterfall Model - Software Engineering
• Differences between Verification and Validation
• Software Development Life Cycle (SDLC)
• Software Requirement Specification (SRS) Format

Empirical Research is defined as any research or study where conclusions of the study is precisely drawn from concretely empirical evidence which is verifiable. Empirical Research aims to use statistical testing techniques to test the hypothesis and hence reduces the gap between theory and practice.

Importance of Empirical Research: It helps in improving, analyzing and assessing the procedures and processes of software development. It also provides guidelines in decision making. Empirical Research is useful to researchers, academicians and the software industry in different scenarios.

• Software Industry: Empirical Study can be used to answer the questions related to industrial practices and can improve the strategies and methods of software development. The predictive models built in Empirical Research can be implemented in similar industrial applications in near future. The empirical research allows software developers to apply the results of the experiment and ascertain that set of good procedures and processes are followed at some point of software development. Thus, the empirical research guides towards determining the best of the resultant software processes and products.
• Researchers: According to researchers, the results can be used to provide information about existing guidelines and trends regarding future research. Empirical study is useful in establishing the generalizability of results related to new subjects or data sets by researchers.
• Academicians: Empirical research helps academicians in finding answers of their question through interviewing different stakeholders, conducting a scientific experiment or conducting a survey. Academicians make predictions in the form of hypotheses. With the help of empirical research, these hypotheses can be tested, and their results can be shown as either being accepted or rejected. Thus, on the basis of result Academicians can make a conclusion about a particular theory or make some generalization.

Basic elements of Empirical Research:

• Purpose: The purpose states the objective of the research, specific motives in the form of research questions, relevance topics and reason of research.
• Process: Process gives a detailed sequence of steps need to be taken to conduct a research. It provides a method in which the research will be conducted. It provides details about the methodologies, techniques and procedures to be used in the research.
• Participants: Participants are those persons which are involved in the research as subjects. They are closely interviewed to obtain the research results. Ethical issues in Empirical research must be considered when dealing with participants so that they don’t get harmed in any way.
• Product: Outcome of research produces Product. The final outcome of Empirical Research provides the answers of the research questions.Any new technique or method can be considered as a product of the Empirical study or research. Few of the examples are conference article, journal paper, thesis, technical report.

Please Login to comment...

Similar reads.

• Software Engineering

Improve your Coding Skills with Practice

What kind of Experience do you want to share?