How Does Feature Dependency Affect Configurable System Comprehensibility?
Authors
Djan Santos and Cláudio Sant'Anna
DOI: 10.1109/ICPC.2019.00016
Abstract
Background: Conditional compilation is often used to implement variability in configurable systems. This technique relies on #ifdefs to delimit feature code. Previous studies have shown that #ifdefs may hinder code comprehensibility. However, they did not explicitly took feature dependencies into account. Feature dependency occurs when different features refer to the same program element, such as a variable. Comprehensibility may be even more affected in the presence of feature dependency, as the developer must reason about different scenarios affecting the same variable.
Aim: Our goal is to understand how feature dependency affects the comprehensibility of configurable system source code.
Method: We carried out an experiment in which 30 developers debugged programs with different types of feature dependency. We record the time each of them spent to find a bug. Also, we used an eye-tracking device to record developers' gaze movements while they debugged programs.
Results: It took longer for developers to debug programs with global and interprocedural dependency. In addition, debugging programs with these same types of dependencies required higher visual effort.
Conclusion: Our study showed that #ifdefs affect comprehensibility in different degrees according with the type of feature dependencies. Therefore, when possible, developers should prefer to use intraprocedural dependency and take more care when dealing with code with global and interprocedural dependency.
Procedure
- Training on variability, features and configurations
- Warm-up task
- Presentation about the goal of the experiment and the risk of participation
- Sign the consent forms
- Calibrate eye-tracking device
- Perform the task in order based in Latin square group
Source Codes of Our Experiment
Latin Square 6 x 6: Row = Groups, Column = Variability Bugs, Characteristic = acronyms.
| Group | Variability Bugs | |||||
|---|---|---|---|---|---|---|
| Null pointer dereference | Assertion error | Logic error | Nested feature | Undefined variable | Uninitialized variable | |
| Group 1 | GI | GW | IAI | IAW | IEI | IEW |
| Group 2 | GW | IAI | IAW | IEI | IEW | GI |
| Group 3 | IAI | IAW | IEI | IEW | GI | GW |
| Group 4 | IAW | IEI | IEW | GI | GW | IAI |
| Group 5 | IEI | IEW | GI | GW | IAI | IAW |
| Group 6 | IEW | GI | GW | IAI | IAW | IEI |
Tasks: Find the bug in source codes with different characteristics
| With #IFDEF | Without #IFDEF |
|---|---|
| GI - Global dependency | GW - Global dependency equivalent |
| IAI - Intraprocedural dependency | IAW - Intraprocedural dependency equivalent |
| IEI - Interprocedural dependency | IEW - Interprocedural dependency equivalent |
Quantitative Collected Data
| Quantitative collected data | Script R | Attention map and gaze transitions diagram |
|---|---|---|
| Time to find bugs | Time.R | Attention map - Global Dependency Gaze transitions - Global Dependency |
| Correctly found bugs | Correctly.R | Attention map - Intraprocedural Dependency Gaze transitions - Intraprocedural Dependency |
| Number of fixations | Fixations.R | Attention map - Interprocedural Dependency |