Volume 9, Issue 4

Articles
In and Out
Trott's Corner
New Products
New Publications
Calendar
News Bulletins
New Resources
Classifieds

Editorial Policy
Staff and Contributors
Submissions
Subscriptions
Back Issues
Contact Information

Bootstrap Tutorial

# Regression

Now we will bootstrap a regression model. There are two ways to do this: we can assume that the matrix of explanatory variables is fixed or random. The fixed case is easier on both theoretical and computational grounds, but the random case is more realistic in most applications of interest. To keep the exposition simple, we will only examine the fixed regressors case. This case is applicable if, for example, the explanatory variables were created as part of a controlled experiment (see [1] for a more detailed discussion).

Since the and the variables are IID, the intercept term should be the mean of the variable (.5) and the coefficient on the variable should be about 0. In this case the coefficient on the variable is not significantly different from 0, as illustrated by the low value of the t-statistic.

To do the bootstrap, we form a set of the data pairs and then draw a random sample, with replacement, from that set. The same resampling function we used before works for this.

Now we simply run our regression on the resampled data and keep track of the regression coefficient. Here is a function that runs a regression and returns the slope coefficient.

Now we will do the bootstrap on the coefficient 1000 times.

The mean of the sampling distribution should be close to zero, and its standard deviation should be about what the regression computed for the standard error of the coefficient.

The bootstrap is a handy tool and particularly easy to implement in Mathematica. With the resampling functions described earlier, we can estimate the sampling distribution of virtually any statistic.