Comparing Techniques for Derivation of Source Code Metric Thresholds: A Study with Web Developers
Source code metrics; Thresholds; Design problems, Software quality
Source code metrics quantify dierent software attributes and have the potential to support
the identication of design problems that may aect software comprehensibility and
maintainability. Identifying design problems can reveal parts of the source code that
need to be monitored more closely. However, one of the major challenges in using metrics
in source code quality monitoring activities is the denition of threshold values that are
capable of identifying design problems that are actually considered problems according
to developers' perceptions. Although there are a number of techniques for extracting
threshold values, the threshold values obtained by means of them generate many false
positives. That is, there are many of code elements (e.g., classes or methods) whose
metric values exceed threshold values that, when evaluated by developers, are not considered
to be problematic. Therefore, the participation of developers is important to
analyze threshold value eectiveness. Few studies evaluate the eectiveness of dierent
threshold extraction techniques based on the perception of developers. Therefore, the
goal of this work was to conduct an experimental study to evaluate the perception of
developers about design problems detected with threshold values obtained by means of
ve dierent techniques that extract threshold values from system benchmarks. In this
scenario, we conducted two studies, a preliminary one in which we analyzed two systems
and two developers' perception, and a second study in which we analyzed four systems
and eight developers. The results indicate that developers use design roles to assess
whether or not design problems exist, and techniques that take into account some contextual
information, such as reference systems or design papers to derive thresholds may
reduce the number of false-positives.