Appendix B — A Short R Primer

This appendix collects the R tools used in the computer tutorials. It assumes that you have used R or another statistical language before; it is a reference, not a course. Wickham, Çetinkaya-Rundel, and Grolemund (2023) is a free introduction to data handling in R, and the R documentation (?function in the console) is the reference for every function used below. Computer tutorial 0 at the end of the appendix practises the material.

B.1 Setting up

Software. Install R (https://cran.r-project.org) and RStudio (https://posit.co/download/rstudio-desktop/). RStudio is an editor around R; the computations are done by R.

Projects and scripts. Keep everything for this course in one folder with a subfolder data. In RStudio, create a project for this folder (File → New Project → Existing Directory). Opening the .Rproj file sets the working directory to the project folder, so data can be read with relative paths such as "data/mroz.csv", and the code runs unchanged on another computer. Avoid setwd("C:/Users/...") in scripts for the same reason. Write code in scripts (.R) or Quarto documents (.qmd), not in the console, so that every result can be reproduced.

Packages. A package is installed once and loaded in every session:

install.packages(c("sandwich", "lmtest", "ggplot2"))   # once
library(sandwich)                                       # every session

The tutorials use the following packages.

Package Used for Tutorials
sandwich, lmtest robust variance matrices, coefficient tests 1, 2, 5
MASS negative binomial data 1
nleqslv solving systems of moment equations 2
AER instrumental variables (ivreg) 2
boot bootstrap 2
statmod, randtoolbox (optional) Gauss–Hermite nodes, Halton sequences 3
ggplot2 graphics 4, 5
glmnet, randomForest lasso, ridge, elastic net, random forests 5

B.2 Objects

Everything in R is an object, created with the assignment operator <-.

x <- c(2, 4, 6, 8)            # numeric vector
length(x); x[2]; x[-1]; x[x > 3]
#> [1] 4
#> [1] 4
#> [1] 4 6 8
#> [1] 4 6 8
s <- seq(0, 1, by = 0.25)     # also 1:10, rep(0, 5)
b <- x > 3                    # logical vector
sum(b); mean(b)               # TRUE counts as 1
#> [1] 3
#> [1] 0.75

Indexing starts at 1. x[-1] drops the first element, and indexing with a logical vector keeps the elements where it is TRUE. Arithmetic is element by element: x * x squares each element, sum(x * x) is the inner product.

Matrices have two indices, [row, column]; leaving one index empty selects all rows or columns.

A <- matrix(1:6, nrow = 2)    # filled column by column
A; dim(A); A[2, ]; A[, 3]
#>      [,1] [,2] [,3]
#> [1,]    1    3    5
#> [2,]    2    4    6
#> [1] 2 3
#> [1] 2 4 6
#> [1] 5 6
cbind(1, x)                   # bind columns; the 1 is recycled
#>        x
#> [1,] 1 2
#> [2,] 1 4
#> [3,] 1 6
#> [4,] 1 8

Data frames are tables whose columns can have different types; columns are accessed with $ or by name. Lists collect objects of any type and are what most estimation functions return. Factors are categorical variables; in a regression a factor is expanded into dummies automatically.

df <- data.frame(id = 1:4, x = x, group = factor(c("a", "b", "a", "b")))
df$x; df[df$group == "a", ]; nrow(df)
#> [1] 2 4 6 8
#>   id x group
#> 1  1 2     a
#> 3  3 6     a
#> [1] 4
res <- list(estimate = 1.5, se = 0.2)
res$estimate
#> [1] 1.5

Missing values are coded NA. Most summary functions return NA if any value is missing unless called with na.rm = TRUE; complete.cases(df) marks rows without missing values.

B.3 Data import and description

All data sets of the course are comma-separated files (.csv), read with read.csv.

attend <- read.csv("../data/attend.csv")   # in your project: "data/attend.csv"
str(attend)                                # variables and types
#> 'data.frame':    680 obs. of  5 variables:
#>  $ termgpa: num  3.19 2.73 3 2.04 3.68 ...
#>  $ priGPA : num  2.64 3.52 2.46 2.61 3.32 ...
#>  $ ACT    : int  23 25 24 20 23 26 21 22 24 21 ...
#>  $ attend : num  84.4 68.8 93.8 96.9 100 ...
#>  $ soph   : int  1 0 0 1 1 1 0 1 0 1 ...
summary(attend)
#>     termgpa          priGPA           ACT            attend      
#>  Min.   :0.000   Min.   :0.857   Min.   :13.00   Min.   :  6.25  
#>  1st Qu.:2.138   1st Qu.:2.190   1st Qu.:20.00   1st Qu.: 75.00  
#>  Median :2.670   Median :2.560   Median :22.00   Median : 87.50  
#>  Mean   :2.601   Mean   :2.587   Mean   :22.51   Mean   : 81.71  
#>  3rd Qu.:3.120   3rd Qu.:2.942   3rd Qu.:25.00   3rd Qu.: 93.75  
#>  Max.   :4.000   Max.   :3.930   Max.   :32.00   Max.   :100.00  
#>       soph       
#>  Min.   :0.0000  
#>  1st Qu.:0.0000  
#>  Median :1.0000  
#>  Mean   :0.5765  
#>  3rd Qu.:1.0000  
#>  Max.   :1.0000

Useful functions for a first look: head, table (frequencies), aggregate or tapply (statistics by group), cor, subset, and write.csv to save a data frame.

tapply(attend$termgpa, attend$soph, mean)   # mean term GPA by year
#>        0        1 
#> 2.487417 2.684449
round(cor(attend[, c("termgpa", "attend", "priGPA")]), 3)
#>         termgpa attend priGPA
#> termgpa   1.000  0.560  0.653
#> attend    0.560  1.000  0.427
#> priGPA    0.653  0.427  1.000

B.4 Matrix algebra and OLS by hand

The operators and functions needed for estimators in matrix form:

R Mathematics
A %*% B matrix product \(\bA\bm B\)
t(A) transpose \(\bA'\)
crossprod(X), crossprod(X, y) \(\bX'\bX\), \(\bX'\by\)
solve(A), solve(A, b) \(\bA^{-1}\), solution of \(\bA\bm z=\bm b\)
diag(A), diag(k) diagonal of \(\bA\), identity matrix \(\bI_k\)
outer(x, y) matrix with elements \(x_iy_j\)

To compute the OLS estimator \(\hat\bbeta=(\bX'\bX)^{-1}\bX'\by\) and its heteroskedasticity-robust variance \((\bX'\bX)^{-1}\big(\sum_i\hat u_i^2\bx_i\bx_i'\big)(\bX'\bX)^{-1}\):

y <- attend$termgpa
X <- cbind(1, attend$attend, attend$priGPA, attend$ACT)
XtXi <- solve(crossprod(X))
bhat <- XtXi %*% crossprod(X, y)
u    <- drop(y - X %*% bhat)
meat <- crossprod(X * u)                 # sum of u_i^2 x_i x_i'
se_rob <- sqrt(diag(XtXi %*% meat %*% XtXi))
cbind(bhat = drop(bhat), se_rob)
#>             bhat      se_rob
#> [1,] -1.06032875 0.182373039
#> [2,]  0.01741182 0.001670940
#> [3,]  0.57473590 0.048226432
#> [4,]  0.03340267 0.006457915

X * u multiplies row \(i\) of \(\bX\) by \(\hat u_i\), so crossprod(X * u) equals \(\sum_i\hat u_i^2\bx_i\bx_i'\) without a loop. solve(crossprod(X), crossprod(X, y)) solves the normal equations directly, which is numerically more stable than inverting. The same results from the standard functions:

library(sandwich); library(lmtest)
fit <- lm(termgpa ~ attend + priGPA + ACT, data = attend)
coeftest(fit, vcov = vcovHC(fit, type = "HC0"))
#> 
#> t test of coefficients:
#> 
#>               Estimate Std. Error t value  Pr(>|t|)    
#> (Intercept) -1.0603288  0.1823730 -5.8141 9.402e-09 ***
#> attend       0.0174118  0.0016709 10.4204 < 2.2e-16 ***
#> priGPA       0.5747359  0.0482264 11.9174 < 2.2e-16 ***
#> ACT          0.0334027  0.0064579  5.1724 3.049e-07 ***
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

HC0 is the formula above; the default HC3 applies a small-sample correction.

B.5 Functions, loops and simulations

A function takes arguments and returns the value of its last expression (or of return()).

ols <- function(y, X) {
  b <- solve(crossprod(X), crossprod(X, y))
  u <- drop(y - X %*% b)
  list(coef = drop(b), sigma2 = sum(u^2) / (nrow(X) - ncol(X)))
}
ols(y, X)$coef
#> [1] -1.06032875  0.01741182  0.57473590  0.03340267

Repeating a computation is done with for loops or, more compactly, with sapply (apply a function to each element of a vector and simplify the result) and replicate (evaluate an expression many times). A Monte Carlo study always fixes the seed with set.seed so that the results can be reproduced. A template that is used throughout the tutorials:

set.seed(1)
one_draw <- function(n) {
  x <- rnorm(n); y <- 1 + 2 * x + rexp(n) - 1       # skewed, mean-zero errors
  coef(lm(y ~ x))[2]
}
est <- replicate(1000, one_draw(50))
c(mean = mean(est), sd = sd(est))
#>      mean        sd 
#> 1.9998029 0.1520735

Random numbers: rnorm, runif, rbinom, rexp, rpois, sample; the corresponding densities, cdfs and quantiles start with d, p and q (e.g. dnorm, pnorm, qnorm).

B.6 Optimisation

Maximum likelihood and GMM require numerical optimisation. optim minimises a function, so we pass the negative log-likelihood. With method = "BFGS" it uses the gradient (numerical if not supplied), and hessian = TRUE returns the Hessian at the optimum, whose inverse estimates the variance of the ML estimator (Chapter 1).

As an example, the normal linear regression model has log-likelihood \(\sum_i\{-\frac12\ln(2\pi\sigma^2)-\frac{(y_i-\bx_i'\bbeta)^2}{2\sigma^2}\}\). We estimate \(\ln\sigma\) instead of \(\sigma\) so that the parameter is unrestricted:

negll <- function(theta, y, X) {
  k <- ncol(X); b <- theta[1:k]; s <- exp(theta[k + 1])
  -sum(dnorm(y, mean = X %*% b, sd = s, log = TRUE))
}
start <- c(rep(0, ncol(X)), 0)
opt <- optim(start, negll, y = y, X = X, method = "BFGS", hessian = TRUE,
             control = list(maxit = 500, reltol = 1e-12))
opt$convergence                      # 0 = converged
#> [1] 0
round(cbind(ML = opt$par[1:4], OLS = coef(fit)), 4)
#>                  ML     OLS
#> (Intercept) -1.0603 -1.0603
#> attend       0.0174  0.0174
#> priGPA       0.5747  0.5747
#> ACT          0.0334  0.0334

Always check opt$convergence, try different starting values, and compare with a closed-form or built-in estimator where one exists. Scaling matters: if regressors differ in magnitude by several orders, rescale them or the optimiser may stop early. For systems of equations (just-identified moment conditions) nleqslv::nleqslv solves \(\bar{\bm g}(\btheta)=\bm 0\) directly.

B.7 Graphics

Base R graphics are quick for checks; ggplot2 gives better results for reports.

library(ggplot2)
ggplot(attend, aes(attend, termgpa)) +
  geom_point(alpha = 0.4) +
  geom_smooth(method = "lm", se = FALSE) +
  labs(x = "classes attended (%)", y = "term GPA") +
  theme_minimal()
Figure B.1: Term GPA against the percentage of classes attended, attend data.

Base equivalents: hist(x), plot(x, y), lines, abline(fit), par(mfrow = c(1, 3)) for several panels.

B.8 Good practice

  • Start each script with library() calls and, for simulations, set.seed().
  • Use relative paths within a project.
  • Write small functions instead of copying code, and check each one on a case where you know the answer.
  • Compare hand-coded estimators with the built-in function before using them in a simulation.
  • Comment what the code does and why, not how R works.
  • A Quarto document (.qmd) combines text, code and output in one file; the lecture notes and the tutorials are written this way.

B.9 Computer tutorial 0

Learning goals. Organise an R project; import, describe and subset data; compute OLS, its standard errors and a maximum likelihood estimator by hand and check them against built-in functions; write a small Monte Carlo study.

Packages. sandwich, lmtest. Data. attend.csv from the data sets of Wooldridge (2020): 680 students in an intermediate microeconomics course.

Variable Description
termgpa GPA for the term, from 0 (worst) to 4 (best); a student passes if termgpa \(\ge1\)
priGPA cumulative GPA prior to the term
ACT ACT score (university entrance test)
attend percentage of classes attended
soph = 1 if the student is a sophomore (second year), 0 if first year

B.9.1 Problem 1: Data handling

  1. Create a project folder with a subfolder data, save attend.csv there, and create an RStudio project and a script for this tutorial.

  2. Import the data as a data frame attend. Check the number of observations, the variable types and whether there are missing values.

  3. Extract termgpa as a numeric vector in two ways: from the data frame and from the data converted to a matrix. Create it also as an \(n\times1\) and a \(1\times n\) matrix, and check with identical() that converting back gives the original vector.

  4. Report descriptive statistics. Draw a histogram of termgpa with a title and an axis label, and a scatter plot of termgpa against attend. Compute the correlation matrix of termgpa, attend, priGPA and soph. What do you learn?

  5. Create a dummy passed (0/1, not TRUE/FALSE). How many students failed, by year?

  6. Compute the mean termgpa of students who attended more than 50% of the classes and of the others. Then compute mean attendance by year and pass status. Interpret, and say why these comparisons are not causal effects of attendance.

  7. Save the data of first-year students, without soph, as attend_fy.csv.

B.9.2 Problem 2: OLS by hand

Consider the model \[ \texttt{termgpa}_i=\beta_0+\beta_1\texttt{attend}_i+\beta_2\texttt{priGPA}_i+\beta_3\texttt{ACT}_i+u_i . \]

  1. Estimate the model with lm and interpret \(\hat\beta_1\).

  2. Write a function ols(y, X) that returns the coefficients, the conventional standard errors based on \(s^2(\bX'\bX)^{-1}\) with \(s^2=\hat\bu'\hat\bu/(n-k)\), and the heteroskedasticity-robust (HC0) standard errors. Check the results against lm and vcovHC(..., type = "HC0").

  3. Plot the fitted against the actual values, the residuals against the fitted values, and a histogram of the residuals. Is homoskedasticity plausible?

B.9.3 Problem 3: Maximum likelihood with optim

Assume additionally \(u_i\mid\bx_i\sim N(0,\sigma^2)\).

  1. Write the negative log-likelihood as a function of \(\btheta=(\bbeta',\ln\sigma)'\) and minimise it with optim(..., method = "BFGS", hessian = TRUE). Why is it convenient to estimate \(\ln\sigma\) rather than \(\sigma\)?

  2. Compare \(\hat\bbeta_{ML}\) with OLS and \(\hat\sigma^2_{ML}\) with \(s^2\). Explain the difference.

  3. Compute standard errors from the inverse Hessian. How do they relate to the conventional OLS standard errors?

B.9.4 Problem 4: A small Monte Carlo study

Simulate \(y_i=1+0.5x_i+u_i\) with \(x_i\sim N(0,1)\) and heteroskedastic errors \(u_i=|x_i|\,\varepsilon_i\), \(\varepsilon_i\sim N(0,1)\).

  1. For \(n=50\) and \(n=500\) and 2000 replications, record \(\hat\beta_1\) and whether the nominal 95% confidence interval contains \(0.5\), once with conventional and once with robust (HC0) standard errors. Use set.seed.

  2. Is \(\hat\beta_1\) unbiased? Compare the coverage rates of the two intervals and explain.