Most research in so
ftware e
ffort estimation has not considered chronology when selecting projects
for training and testing sets. A chronological split represents the use o
f a projects starting and completion dates, such that any model that estimates e
ffort
for a new project
p only uses as training data projects that were completed prior to
p’s start. Four recent studies in
vestigated the use o
f chronological splits, using mo
ving windows wherein only the most recent projects completed prior to a projects starting date were used as training data. The
first three studies (S1–S3)
found some e
vidence in
fa
vor o
f using windows; they all de
fined window sizes as being
fixed numbers o
f recent projects. In practice, we suggest that estimators think in terms o
f elapsed time rather than the size o
f the data set, when deciding which projects to include in a training set. In the
fourth study (S4) we showed that the use o
f windows based on duration can also impro
ve estimation accuracy.
Objective
This papers contribution is to extend S4 using an additional dataset, and to also investigate the effect on accuracy when using moving windows of various durations.
Method
Stepwise multivariate regression was used to build prediction models, using all available training data, and also using windows of various durations to select training data. Accuracy was compared based on absolute residuals and MREs; the Wilcoxon test was used to check statistical significances between results. Accuracy was also compared against estimates derived from windows containing fixed numbers of projects.
Results
Neither fixed size nor fixed duration windows provided superior estimation accuracy in the new data set.
Conclusions
Contrary to intuition, our results suggest that it is not always beneficial to exclude old data when estimating effort for new projects. When windows are helpful, windows based on duration are effective.