Appendix A — Large-Sample Asymptotics
\[ \newcommand{\bm}[1]{\boldsymbol{#1}} \newcommand{\btheta}{\boldsymbol{\theta}} \newcommand{\bbeta}{\boldsymbol{\beta}} \newcommand{\bgamma}{\boldsymbol{\gamma}} \newcommand{\bx}{\boldsymbol{x}} \newcommand{\by}{\boldsymbol{y}} \newcommand{\bX}{\boldsymbol{X}} \newcommand{\bR}{\boldsymbol{R}} \newcommand{\br}{\boldsymbol{r}} \newcommand{\bh}{\boldsymbol{h}} \newcommand{\bz}{\boldsymbol{z}} \newcommand{\bZ}{\boldsymbol{Z}} \newcommand{\bW}{\boldsymbol{W}} \newcommand{\bA}{\boldsymbol{A}} \newcommand{\bV}{\boldsymbol{V}} \newcommand{\bu}{\boldsymbol{u}} \newcommand{\bw}{\boldsymbol{w}} \newcommand{\bpsi}{\boldsymbol{\psi}} \newcommand{\balpha}{\boldsymbol{\alpha}} \newcommand{\bdelta}{\boldsymbol{\delta}} \newcommand{\bmu}{\boldsymbol{\mu}} \newcommand{\bg}{\boldsymbol{g}} \newcommand{\bS}{\boldsymbol{S}} \newcommand{\bD}{\boldsymbol{D}} \newcommand{\bI}{\boldsymbol{I}} \newcommand{\E}{\mathbb{E}} \newcommand{\Var}{\operatorname{Var}} \newcommand{\AVar}{\operatorname{Avar}} \newcommand{\cvp}{\xrightarrow{\;p\;}} \newcommand{\cvd}{\xrightarrow{\;d\;}} \newcommand{\iid}{\overset{\text{iid}}{\sim}} \newcommand{\KLIC}{\operatorname{KLIC}} \newcommand{\pd}[2]{\frac{\partial #1}{\partial #2}} \]
This appendix collects the asymptotic tools used throughout the notes. They are assumed known from Research Metrics 1; the summary is for reference. Wooldridge (2010), Ch. 3, Hansen (2022a), Ch. 6 and Hansen (2022b), Ch. 7–8 give proofs.
A.1 Modes of convergence
Let \(\bm z_n\) be a sequence of random vectors.
Definition A.1 (Convergence in probability) \(\bm z_n\cvp\bm c\) if for every \(\varepsilon>0\), \(P(\|\bm z_n-\bm c\|>\varepsilon)\to0\) as \(n\to\infty\). We also write \(\operatorname{plim}\bm z_n=\bm c\). An estimator \(\hat\btheta_n\) is consistent if \(\hat\btheta_n\cvp\btheta_0\).
Definition A.2 (Convergence in distribution) \(\bm z_n\cvd\bm z\) if \(P(\bm z_n\le\bm t)\to P(\bm z\le\bm t)\) at every continuity point \(\bm t\) of the cdf of \(\bm z\). The distribution of \(\bm z\) is the limiting distribution.
Convergence in probability implies convergence in distribution; the converse holds only if the limit is a constant.
Order notation. \(\bm z_n=o_p(1)\) means \(\bm z_n\cvp\bm 0\); \(\bm z_n=O_p(1)\) means \(\bm z_n\) is bounded in probability (for every \(\varepsilon\) there is an \(M\) with \(P(\|\bm z_n\|>M)<\varepsilon\) for all \(n\)). Any \(\bm z_n\) that converges in distribution is \(O_p(1)\), and \(o_p(1)\cdot O_p(1)=o_p(1)\).
A.2 Laws of large numbers and central limit theorems
Theorem A.1 (Weak law of large numbers) If \(\bm y_1,\dots,\bm y_n\) are i.i.d. with \(\E\|\bm y_i\|<\infty\), then \(\bar{\bm y}_n=\frac1n\sum_i\bm y_i\cvp\E[\bm y_i]\).
Theorem A.2 (Central limit theorem (Lindeberg–Lévy)) If \(\bm y_1,\dots,\bm y_n\) are i.i.d. with mean \(\bm\mu\) and finite variance matrix \(\bm\Sigma\), then \[ \sqrt n\,(\bar{\bm y}_n-\bm\mu)=\frac1{\sqrt n}\sum_{i=1}^n(\bm y_i-\bm\mu)\cvd N(\bm 0,\bm\Sigma). \]
Both apply to functions of the data: in the notes \(\bm y_i\) is replaced by a score \(S_i(\btheta_0)\), a moment function \(\bpsi(\bw_i,\btheta_0)\) or a simulated quantity \(h(x_s)\).
A.3 Continuous mapping, Slutsky and the delta method
Theorem A.3 (Continuous mapping theorem) Let \(g\) be continuous.
If \(\bm z_n\cvp\bm c\), then \(g(\bm z_n)\cvp g(\bm c)\).
If \(\bm z_n\cvd\bm z\), then \(g(\bm z_n)\cvd g(\bm z)\).
For (i), continuity at \(\bm c\) suffices.
Theorem A.4 (Slutsky’s theorem) If \(z_n\cvd z\) and \(c_n\cvp c\) (a constant), then \(z_n+c_n\cvd z+c\), \(c_nz_n\cvd cz\), and \(z_n/c_n\cvd z/c\) if \(c\neq0\).
In matrix form: if \(\bm H_n\cvp\bm H\) and \(\bm b_n\cvd N(\bm\mu,\bm B)\), then \(\bm H_n\bm b_n\cvd N(\bm H\bm\mu,\bm H\bm B\bm H')\).
The matrix form is the workhorse of the notes. Every asymptotic normality proof writes \(\sqrt n(\hat\btheta-\btheta_0)=\bm H_n\bm b_n\), where \(\bm H_n\) is an (inverse) average of derivatives that converges in probability by the WLLN, and \(\bm b_n\) is \(\sqrt n\) times an average of mean-zero terms that is asymptotically normal by the CLT.
Theorem A.5 (Delta method) If \(\sqrt n(\hat\bbeta_n-\bbeta)\cvd N(\bm 0,\bV)\) and \(\bm g:\mathbb R^k\to\mathbb R^q\) is continuously differentiable at \(\bbeta\) with Jacobian \(\bm G=\partial\bm g(\bbeta)/\partial\bbeta'\), then \[ \sqrt n\big(\bm g(\hat\bbeta_n)-\bm g(\bbeta)\big)\cvd N\big(\bm 0,\ \bm G\bV\bm G'\big). \]
Why: by the mean value theorem, \(\bm g(\hat\bbeta_n)=\bm g(\bbeta)+\bm G(\bbeta^*)(\hat\bbeta_n-\bbeta)\) with \(\bbeta^*\) between \(\hat\bbeta_n\) and \(\bbeta\). Since \(\bbeta^*\cvp\bbeta\), \(\bm G(\bbeta^*)\cvp\bm G\), and Slutsky gives the result.
Example A.1 (Delta method) Let \(y_i\) be i.i.d. with mean \(\mu\) and variance \(\sigma^2\), and \(g(\mu)=e^\mu\). Then \(\sqrt n(e^{\bar y}-e^\mu)\cvd N(0,e^{2\mu}\sigma^2)\), and a standard error of \(e^{\bar y}\) is \(e^{\bar y}\hat\sigma/\sqrt n\).
A.4 Mean value expansions
Most proofs expand a first-order condition around the true value. For a differentiable vector function \(\bar\bg_n(\btheta)\), \[ \bar\bg_n(\hat\btheta)=\bar\bg_n(\btheta_0)+\pd{\bar\bg_n(\btheta^*)}{\btheta'}\,(\hat\btheta-\btheta_0), \] where \(\btheta^*=\btheta_0+\lambda(\hat\btheta-\btheta_0)\), \(0\le\lambda\le1\), lies on the segment between \(\hat\btheta\) and \(\btheta_0\) (strictly, a different \(\btheta^*\) for each row). If \(\hat\btheta\) is consistent, so is \(\btheta^*\). If \(\bar\bg_n(\hat\btheta)=\bm 0\) by definition of the estimator, solving for \(\hat\btheta-\btheta_0\) gives the representation used for the MLE (Theorem 1.5), MM (Theorem 2.1) and GMM (Theorem 2.2).
A.5 Consistency of extremum estimators
ML, GMM and MSL estimators maximize or minimize a sample objective \(Q_n(\btheta)\). The standard argument has three ingredients:
- Identification: the limit objective \(Q_0(\btheta)=\operatorname{plim}Q_n(\btheta)\) has a unique maximum at \(\btheta_0\) (for ML this follows from the KLIC inequality; for GMM from \(\E[\bpsi(\bw_i,\btheta)]=\bm 0\) only at \(\btheta_0\)).
- Uniform convergence: \(\sup_{\btheta\in\Theta}|Q_n(\btheta)-Q_0(\btheta)|\cvp0\), which holds under compactness of \(\Theta\), continuity and a dominance condition (a uniform law of large numbers).
- Continuity of \(Q_0\).
Then \(\hat\btheta_n\cvp\btheta_0\) (Wooldridge 2010, Thm. 12.2; Hansen 2022a, Ch. 22.4). Pointwise convergence of \(Q_n\) is not enough: the maximizer of \(Q_n\) could drift away from \(\btheta_0\).