On MAPE-R as a Measure of Cross-Sectional Estimation and Forecast Accuracy

Abstract

Both demographers and economists evaluate the accuracy of their respective forecasts with measures
like mean square error, root mean square error, mean absolute percent error, and mean algebraic percent er-
ror. However, demographers tend to approach the issue of forecasting very differently than do economists.
Two of the distinctive features of the demographic tradition are the use of the cohort-component method
(instead of time-series models) and an emphasis on cross-sectional forecasts (instead of forecasts aggre-
gated over time). From the perspective of this demographic tradition, we examine “MAPE-R” (Mean
Absolute Percent Error-Rescaled), a recently developed measure of accuracy designed to overcome short-
comings noted in “MAPE” (Mean Absolute Percent Error), a measure commonly used to evaluate the
accuracy of population estimates and forecasts. We show that MAPE-R can be calculated simply, thus
overcoming the cumbersome calculation procedure used in its introduction and noted as a feature needing
correction. We find this closed form expression for MAPE-R to be a member of the family of power
mean-based accuracy measures. This enables it to be placed in relation to other members of this family,
which includes HMAPE (Harmonic Mean Absolute Percent Error), GMAPE (Geometric Mean Absolute
Percent Error), and MAPE. Given that MAPE-R was designed to be robust in the face of outliers, it is not
surprising to find that it is a valid estimator of the median of the distribution(s) generating the absolute
percent errors. Simulation studies suggest that MAPE-R is a far more efficient estimator of this median
than MEDAPE (Median Absolute Percent Error). Because the Box-Cox transformation on which MAPE-
R depends is known to be unstable, we suggest that this represents a line of further research into GMAPE,
which, like MAPE-R, is subject neither to the shortcomings observed for MAPE nor to the instability of
the Box-Cox transformation. While further lines of research are called for, nothing in our examination of
MAPE-R here rules out its use. It also meets the National Research Council’s major criteria as a summary
measure of accuracy. It is subject to some cautions, but these are no more restrictive than those affecting
other accuracy measures, many of which are widely used and have been for some years.

Copyright Notice

This paper was produced, in part, by a U.S. Government employee in the course of official duties. Therefore, it has no copyright in the U.S. Foreign copyrights may apply.

File

File nameFile typeSize
an12.sasASCII text1.19 kBInfo
ColemanSwansonJESM2007.pdfPDF document183.66 kBInfo
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License