On automatic determination of criteria to detect outliers for the absolute measurement of the geomagnetic field
Ito, N. & Fujii, I.
Abstract
We propose a procedure for automatic determination of criteria to distinguish spurious baseline values in the absolute measurement of the geomagnetic field based on a robust statistical method. The data used in this study are baseline values of the horizontal force (H), declination (D), and vertical component (Z) observed at Kanoya Magnetic Observatory from February 25, 2000 to December 31, 2000. The absolute measurement sessions were conducted 58, 58, and 72 times for H, Z, and D, respectively, in the period of interest. Eight baseline values were obtained in each session, giving a total of 464 baseline values each for H and Z, and 576 for D. The data set of each session has too few samples to be analyzed with a robust procedure, so the whole data set comprising 464 or 576 values was used. After the median was subtracted from the data of each session, a set of the residuals for all sessions was analyzed statistically. As a result, we found that the residuals of all sessions follow a Gaussian distribution, except for a small portion of the data set.
Quantile-Quantile (QQ) plots of the residuals normalized by the median absolute deviation (MAD) were made for each component and were used to estimate the outliers. We assumed that the outliers are normalized residuals larger than 7 for H and Z, and 5 for D, at which discontinuities were seen in the QQ plots for each component. As a result, 8, 12, and 13 outliers were detected for H, Z, and D, respectively. These criteria are slightly more robust than those estimated as three times the standard deviation that are computed from the data set without the outliers.
The results of the robust procedure used in this study indicate that the outliers can be automatically detected and that the procedure could detect outliers hidden in each session. For the baseline values in the year 2000, 5 and 7 times the MAD are good estimates of the criteria to detect the outliers. Further investigation on the robust criteria by analyzing data for other time periods will be required to improve the accuracy of the procedure.
Translated to English from paper in Japanese, originally published in: Technical Report of the Kakioka Magnetic Observatory, Vol. 1, No. 1, pp. 1-9, 2003 with permission of the authers.
