An experiment on how Feature dependent variables affect program comprehensibility

Authors

Djan Santos, Cláudio Sant'Anna and Márcio Ribeiro

Abstract

Background: #ifdefs allow developers to define source code related to features that should or should not be compiled. A feature dependency occurs in a configurable system when source code snippets of different features share code elements, such as variables. Variables that produce feature dependency are called dependent variables. The dependency between two features may include just one dependent variable or more than one. It is reasonable to suspect that a high number of dependent variables and their use make the analysis of variability scenarios more complex. In fact, previous studies show that #ifdefs may affect comprehensibility, especially when their use implies feature dependency.

Aims: In this sense, the main goal of our experiment is to evaluate if the comprehensibility of configurable system source code varies according to the number of dependent variables. Testing this hypothesis is important so that we can recommend practitioners and researchers the extent to which writing #ifdef code with dependencies is harmful.

Method: To minimize this knowledge gap, we executed a controlled experiment with 12 participants who analyzed programs trying to specify their output. We quantified comprehensibility using metrics based on time and attempts to answer tasks correctly, participants' visual effort, and participants' heart rate.

Results: Our results indicate that the higher the number of dependent variables the more difficult it was to understand programs with feature dependency.

Conclusions: In practice, our results indicate that comprehensibility is more negatively affected in programs with higher number of dependent variables and when these variables are defined at a point far from the points where they are used.

Procedure

1. We cleaned all material and equipment with alcohol gel and asked the participant to fill out a consent form
2. The participant put on the smartwatch
3. Calibrate eye-tracking and smartwatch devices
4. We asked her or him to watch a two minute full-screen video of a fish swimming in an aquarium
5. Perform the task in order based in Latin square group

Source Codes of Our Experiment

Latin Square 2 x 2: Rows = Participants, Column = Domains, Four squares: programs with and without feature dependency.

Legend: 4DV (4 Dependent Variables), 2DV (2 Dependent Variables)

Tasks of Programs

Quantitative Collected Data

AOI of Programs

Programs Highlighting Variable Definitions and Variable Usages