ERMSEI 2023 - Empirical Research Methods in Software Engineering and Informatics - September, 25-29, 2023

Traditionally, Software Engineering and Informatics have been strongly influenced by Mathematics. Empirical Research has always been around, too, but has gained traction only in the past 20 years. Today, empirical research represents a major approach that is highly visible in most important conferences and journals in the field. Many reviewers will demand empirical evidence when reviewing papers making statements about the real world that are falsifiable, i.e., may be tested empirically. Yet, in education, empirical research methods are still underrepresented.

Content & Outcome

Course Content

The course introduces you to empirical research methods in a very practical, hands-on way. There will be introductory lectures on techniques followed by group work on exercises, for instance:

analyze the study design in a given article with a view to finding shortcomings and threats to validity and how they impact the claims put forward;
design a study yourself for a given case study in a small group, and present the study design in a plenary discussion;
develop a research question and discuss alternative approaches to providing evidence for or against it.

This course has been running in different shapes and forms for ten years now.

Expected Outcome

This course is but a first step on a long journey. It aims to provide students with an overview and concrete starting point, possibly removing any inhibitions or hesitation that might be there. We aspire to equip participants with some skills and first practical exercises in conducting methodologically sound research. After attending this course, participants:

are capable of choosing an appropriate research paradigm for a given problem;
will be aware of the potential and limitations of empirical research methods; and
can assess the quality of the empirical research reported in an article, such as for a review.

We will cover controlled experiments, selected qualitative methods, and overall study design as far as the practical exercises are concerned. We explicitly encourage participants to bring along their own empirical research projects to work in the setting of this course. Time permitting, we will discuss these projects also in plenary sessions.

Intended Audience

The course is intended for people that have little or no background in empirical research methods. So, the ideal participant would be someone who has just started or is about to start their PhD in Computer Science / Software Engineering. More senior researchers are welcome, too, of course.

Program

This course will be hosted at the Software Engineering Research Lab (SERL) at Blekinge Institute of Technology (Valhallavägen 1, 371 79 Karlskrona, Sweden). The course will be from 2023-09-25 to 2023-09-29.

Monday

Tuesday

Wednesday

Thursday

Friday

Morning

9:00-12:00

Introduction

Controlled Experiments

Observations and Measurements

Physiological Measurements

Surveys

Field Work

Qualitative Methods:

Interviews

Secondary Research

Structured Literature Reviews

Mapping Studies

Research Design

Philosophical Perspective

Theory

Afternoon

13:00-16:00

Statistical Techniques

Threats to Validity

Practical exercise

(1 PM – 3 PM)

Observational Techniques

Grounded Theory

Ethnography

Tertiary Studies

Ethics

Practical exercise

Registration

Please register for participation via this form: https://forms.gle/9cwdPbvjuJRsttkB8

About the Instructors

Harald Störrle

Harald Störrle received a Dipl.-Inform. and a Dr.rer.nat. from the Universities of Hamburg (1997) and Munich (2000), respectively. From 2001 to 2009, he worked as a software architect and methodology consultant in industry, sidelining as an adjunct lecturer at the University of Munich. Starting in 2006, he held lecturer positions at the Universities of Innsbruck and Munich, and as an Associate Professor of Software Engineering at the Technical University of Denmark (DTU) in Lyngby near Copenhagen. Since 2017, he has worked as a principal consultant with QAware in Munich. His current role is that of Product Owner for an AI-powered digital speech assistant platform for Deutsche Telekom.

In his research activities, Harald applies empirical research methods to problems in modelling, requirements, and software processes. He advocates Evidence-Based Software Engineering, favouring methodological openness and diversity.

He is a Senior Member of the ACM, elected member of the ACM Europe Council, former vice chairperson of the German Chapter of the ACM, and was appointed to several ACM boards and committees, including the ACM practitioner board and the Digital Library Board.

Jefferson Seide Molléri

Jefferson Molléri holds a PhD in Software Engineering from the Institute of Technology of Blekinge (2019) – Sweden. Currently, he is an Associate Professor at the School of Economy, Innovation, and Technology at Kristiania University College. He teaches first- and second-cycle software engineering, information technology, and data science courses. He has over ten years of industrial experience in companies developing web-based and client-server applications.

As an academic researcher, Jefferson focuses on evidence-based software engineering and empirical research methods. He stands up for methodological quality, ethical and contributory research. Among his contributions to the field, he has published a report on empirical standards, a catalogue of empirical research guidelines, and checklists for assessing survey research.