Appendix A — Large-Sample Asymptotics

This appendix collects the asymptotic tools used throughout the notes. They are assumed known from Research Metrics 1; the summary is for reference. Wooldridge (2010), Ch. 3, Hansen (2022a), Ch. 6 and Hansen (2022b), Ch. 7–8 give proofs.

A.1 Modes of convergence

Let \(\bm z_n\) be a sequence of random vectors.

Definition A.1 (Convergence in probability) \(\bm z_n\cvp\bm c\) if for every \(\varepsilon>0\), \(P(\|\bm z_n-\bm c\|>\varepsilon)\to0\) as \(n\to\infty\). We also write \(\operatorname{plim}\bm z_n=\bm c\). An estimator \(\hat\btheta_n\) is consistent if \(\hat\btheta_n\cvp\btheta_0\).

Definition A.2 (Convergence in distribution) \(\bm z_n\cvd\bm z\) if \(P(\bm z_n\le\bm t)\to P(\bm z\le\bm t)\) at every continuity point \(\bm t\) of the cdf of \(\bm z\). The distribution of \(\bm z\) is the limiting distribution.

Convergence in probability implies convergence in distribution; the converse holds only if the limit is a constant.

Order notation. \(\bm z_n=o_p(1)\) means \(\bm z_n\cvp\bm 0\); \(\bm z_n=O_p(1)\) means \(\bm z_n\) is bounded in probability (for every \(\varepsilon\) there is an \(M\) with \(P(\|\bm z_n\|>M)<\varepsilon\) for all \(n\)). Any \(\bm z_n\) that converges in distribution is \(O_p(1)\), and \(o_p(1)\cdot O_p(1)=o_p(1)\).

A.2 Laws of large numbers and central limit theorems

Theorem A.1 (Weak law of large numbers) If \(\bm y_1,\dots,\bm y_n\) are i.i.d. with \(\E\|\bm y_i\|<\infty\), then \(\bar{\bm y}_n=\frac1n\sum_i\bm y_i\cvp\E[\bm y_i]\).

Theorem A.2 (Central limit theorem (Lindeberg–Lévy)) If \(\bm y_1,\dots,\bm y_n\) are i.i.d. with mean \(\bm\mu\) and finite variance matrix \(\bm\Sigma\), then \[ \sqrt n\,(\bar{\bm y}_n-\bm\mu)=\frac1{\sqrt n}\sum_{i=1}^n(\bm y_i-\bm\mu)\cvd N(\bm 0,\bm\Sigma). \]

Both apply to functions of the data: in the notes \(\bm y_i\) is replaced by a score \(S_i(\btheta_0)\), a moment function \(\bpsi(\bw_i,\btheta_0)\) or a simulated quantity \(h(x_s)\).

A.3 Continuous mapping, Slutsky and the delta method

Theorem A.3 (Continuous mapping theorem) Let \(g\) be continuous.

  1. If \(\bm z_n\cvp\bm c\), then \(g(\bm z_n)\cvp g(\bm c)\).

  2. If \(\bm z_n\cvd\bm z\), then \(g(\bm z_n)\cvd g(\bm z)\).

For (i), continuity at \(\bm c\) suffices.

Theorem A.4 (Slutsky’s theorem) If \(z_n\cvd z\) and \(c_n\cvp c\) (a constant), then \(z_n+c_n\cvd z+c\), \(c_nz_n\cvd cz\), and \(z_n/c_n\cvd z/c\) if \(c\neq0\).

In matrix form: if \(\bm H_n\cvp\bm H\) and \(\bm b_n\cvd N(\bm\mu,\bm B)\), then \(\bm H_n\bm b_n\cvd N(\bm H\bm\mu,\bm H\bm B\bm H')\).

The matrix form is the workhorse of the notes. Every asymptotic normality proof writes \(\sqrt n(\hat\btheta-\btheta_0)=\bm H_n\bm b_n\), where \(\bm H_n\) is an (inverse) average of derivatives that converges in probability by the WLLN, and \(\bm b_n\) is \(\sqrt n\) times an average of mean-zero terms that is asymptotically normal by the CLT.

Theorem A.5 (Delta method) If \(\sqrt n(\hat\bbeta_n-\bbeta)\cvd N(\bm 0,\bV)\) and \(\bm g:\mathbb R^k\to\mathbb R^q\) is continuously differentiable at \(\bbeta\) with Jacobian \(\bm G=\partial\bm g(\bbeta)/\partial\bbeta'\), then \[ \sqrt n\big(\bm g(\hat\bbeta_n)-\bm g(\bbeta)\big)\cvd N\big(\bm 0,\ \bm G\bV\bm G'\big). \]

Why: by the mean value theorem, \(\bm g(\hat\bbeta_n)=\bm g(\bbeta)+\bm G(\bbeta^*)(\hat\bbeta_n-\bbeta)\) with \(\bbeta^*\) between \(\hat\bbeta_n\) and \(\bbeta\). Since \(\bbeta^*\cvp\bbeta\), \(\bm G(\bbeta^*)\cvp\bm G\), and Slutsky gives the result.

Example A.1 (Delta method) Let \(y_i\) be i.i.d. with mean \(\mu\) and variance \(\sigma^2\), and \(g(\mu)=e^\mu\). Then \(\sqrt n(e^{\bar y}-e^\mu)\cvd N(0,e^{2\mu}\sigma^2)\), and a standard error of \(e^{\bar y}\) is \(e^{\bar y}\hat\sigma/\sqrt n\).

A.4 Mean value expansions

Most proofs expand a first-order condition around the true value. For a differentiable vector function \(\bar\bg_n(\btheta)\), \[ \bar\bg_n(\hat\btheta)=\bar\bg_n(\btheta_0)+\pd{\bar\bg_n(\btheta^*)}{\btheta'}\,(\hat\btheta-\btheta_0), \] where \(\btheta^*=\btheta_0+\lambda(\hat\btheta-\btheta_0)\), \(0\le\lambda\le1\), lies on the segment between \(\hat\btheta\) and \(\btheta_0\) (strictly, a different \(\btheta^*\) for each row). If \(\hat\btheta\) is consistent, so is \(\btheta^*\). If \(\bar\bg_n(\hat\btheta)=\bm 0\) by definition of the estimator, solving for \(\hat\btheta-\btheta_0\) gives the representation used for the MLE (Theorem 1.5), MM (Theorem 2.1) and GMM (Theorem 2.2).

A.5 Consistency of extremum estimators

ML, GMM and MSL estimators maximize or minimize a sample objective \(Q_n(\btheta)\). The standard argument has three ingredients:

  1. Identification: the limit objective \(Q_0(\btheta)=\operatorname{plim}Q_n(\btheta)\) has a unique maximum at \(\btheta_0\) (for ML this follows from the KLIC inequality; for GMM from \(\E[\bpsi(\bw_i,\btheta)]=\bm 0\) only at \(\btheta_0\)).
  2. Uniform convergence: \(\sup_{\btheta\in\Theta}|Q_n(\btheta)-Q_0(\btheta)|\cvp0\), which holds under compactness of \(\Theta\), continuity and a dominance condition (a uniform law of large numbers).
  3. Continuity of \(Q_0\).

Then \(\hat\btheta_n\cvp\btheta_0\) (Wooldridge 2010, Thm. 12.2; Hansen 2022a, Ch. 22.4). Pointwise convergence of \(Q_n\) is not enough: the maximizer of \(Q_n\) could drift away from \(\btheta_0\).