This textbook, which has become the leading source for empirical software engineering research through its earlier editions, provides an in-depth introduction to experimentation in software engineering. The focus is on the steps to go through when conducting experiments. It also introduces a structure for selecting a research design and an introduction to systematic literature reviews, surveys, and case study research. Experiments are valuable tools for all software engineers who evaluate and choose between different solutions, methods, techniques, languages, and tools.
The book has three main parts. Part I introduces empirical research through a structure for selecting a research design and explains several essential areas related to empirical research. Moreover, it provides introductions to systematic literature reviews, experiments, surveys, and case studies. Part II devotes one chapter to each of the five experiment steps: scoping, planning, execution, analysis, and result presentation. Part III completes the presentation with two examples. Assignments and statistical material are provided in appendices. The book provides indispensable information regarding empirical studies, particularly for experiments, but also for systematic literature studies, surveys, and case studies. This new edition comprises substantial new material. This includes a chapter on selecting a research design, which helps put experimentation into a broader context of other research approaches. Furthermore, a new chapter on survey research has been added. Novel contributions concerning A/B testing, replications, open science, validity threats, paired comparison design and tools for statistical analysis have been added. Moreover, systematic literature studies, experiments, and case study research descriptions have been consolidated. The book has also been updated based on the evolution of software engineering since the previous edition's publication.
The book is intended for students, teachers, researchers, and practitioners who would like to compare different options, for example, feature designs, or different development methods, techniques, languages, or tools. Teachers can use the book for undergraduate or graduate students; the book can be used as a course book on experimentation and empirical studies in general. Researchers will also benefit from the book by learning more about how to conduct empirical studies. Likewise, practitioners may use it as a "cookbook" when evaluating different solutions, alternatively for evaluating new methods or techniques before implementing them in their organisation.
The book has three main parts. Part I introduces empirical research through a structure for selecting a research design and explains several essential areas related to empirical research. Moreover, it provides introductions to systematic literature reviews, experiments, surveys, and case studies. Part II devotes one chapter to each of the five experiment steps: scoping, planning, execution, analysis, and result presentation. Part III completes the presentation with two examples. Assignments and statistical material are provided in appendices. The book provides indispensable information regarding empirical studies, particularly for experiments, but also for systematic literature studies, surveys, and case studies. This new edition comprises substantial new material. This includes a chapter on selecting a research design, which helps put experimentation into a broader context of other research approaches. Furthermore, a new chapter on survey research has been added. Novel contributions concerning A/B testing, replications, open science, validity threats, paired comparison design and tools for statistical analysis have been added. Moreover, systematic literature studies, experiments, and case study research descriptions have been consolidated. The book has also been updated based on the evolution of software engineering since the previous edition's publication.
The book is intended for students, teachers, researchers, and practitioners who would like to compare different options, for example, feature designs, or different development methods, techniques, languages, or tools. Teachers can use the book for undergraduate or graduate students; the book can be used as a course book on experimentation and empirical studies in general. Researchers will also benefit from the book by learning more about how to conduct empirical studies. Likewise, practitioners may use it as a "cookbook" when evaluating different solutions, alternatively for evaluating new methods or techniques before implementing them in their organisation.
From the reviews: "The revised edition includes new chapters and examples, which further cements its place as the premier book on the topics of designing, constructing, executing, and assessing experiments in software engineering. (...) The revisions will make this book even more valuable. (...) It ought to be required reading for all PhD students; every academic's bookcase should have a copy." (Michael Oudshoorn, Computing Reviews, October, 2012) "This book is a landmark in allowing us to train both the researcher and practitioner in software engineering experimentation." (Victor R. Basili, University of Maryland, MD, USA) "The additions and modifications in this revised version very nicely reflect the maturation of the field of empirical software engineering." (Anneliese A. Andrews, University of Denver , CO, USA) "The volume, a revised edition of a work by the same name ... published in 2000, presents a very useful review of methods in software engineering research. It is structured as a textbook, making it useful for an introductory graduate-level course or a fourth-year course at the undergraduate level. Practitioners and experts will also benefit from this book, as they can use it as a starting point for more in-depth approaches. ... Summing Up: Highly recommended. Upper-division undergraduates through professionals/practitioners." (L. Benedicenti, Choice, Vol. 50 (9), May, 2013)
`This is a well written and concise book, which provides the reader with the essentials necessary to design, conduct and analyse a software engineering experiment. Its value lies in that it is specifically written for the software engineering field, and has surveyed the major contributions by a number of leading researchers in this area. A `how to' book is always welcome, both as a useful starting point to the inexperienced and as a helpful reference and reminder of best practice to others.'
Software Testing Verification and Reliability, 11 (2001)
Software Testing Verification and Reliability, 11 (2001)