IRLS testing
Because the Iterative Reweighted Least Squares (IRLS) is the Best Linear Unbiased Estimator (BLUE) when residuals are not independent and identically distributed - it is a method of choice do deal with subjects outliers.
This function compares the quality of the fit of limo_irls.m for a regression, ANOVA and an ANCOVA versus Matlab robustfit function.
Both are using a bisquare function and the same tuning, which makes the comparison fair. Results show a similar profile for the data fit, which indicates that our function does what robustfit does. For each simulated data, we also compute the p-value and its equivalent under H0 using bootstrap (fitting WX to resampled Y). This shows that the bootstrap procedure estimates closely the null distribution when N>50. Given this result, IRLS is turned off automatically for small sample sizes.
Figure 1: Average distances between the data and the model (a.k.a. RMSE).
Additional simulations were performed, testing different bootstrap schemes (1) just an OLS (2) using WX and fitting resampled WY (same as WLS) or (3) using WX and fitting resampled Y. The latter is the one giving the closest p-values to the theoretical ones and thus the one implemented in LIMO.