Influence diagnostic analysis in the possibly heteroskedastic linear model with exact restrictions

Abstract

The local influence method has proven to be a useful and powerful tool for detecting influential observations on the estimation of model parameters. This method has been widely applied in different studies related to econometric and statistical modelling. We propose a methodology based on the Lagrange multiplier method with a linear penalty function to assess local influence in the possibly heteroskedastic linear regression model with exact restrictions. The restricted maximum likelihood estimators and information matrices are presented for the postulated model. Several perturbation schemes for the local influence method are investigated to identify potentially influential observations. Three real-world examples are included to illustrate and validate our methodology.

Appendices

Appendix 1: differentials for the Hessian matrix

We use matrix calculus as studied in Magnus and Neudecker (1999) to establish our results in both disturbance cases. In the spherical disturbance case, we present the differentials for the Hessian matrix in Appendix 1 and for the ${\varvec{\varDelta }}$ matrices in Appendix 2. In the non-spherical disturbance case, we obtain the differentials and matrices in a similar manner, so that they are and omitted here.

First, we take the differential of $\ell$ given in (5) with respect to ${\varvec{\beta }}$ and $\sigma ^2$ and obtain

$$\begin{aligned} \text {d}_{{\varvec{\beta }}} \ell= & {} \frac{1}{\sigma ^2}({\varvec{y}}-{\varvec{X}} {\varvec{\beta }})^\top {\varvec{X}} \text {d} {\varvec{\beta }} - \varvec{\uplambda }^\top {\varvec{R}} \text {d} {\varvec{\beta }}, \end{aligned}$$

(19)

$$\begin{aligned} \text {d}_{\sigma ^2} \ell= & {} - \frac{n}{2 \sigma ^2} \text {d} \sigma ^2 + {({\varvec{y}}-{\varvec{X}} {\varvec{\beta }})^\top ({\varvec{y}}-{\varvec{X}} {\varvec{\beta }}) \over 2 \sigma ^4}\text {d} \sigma ^2. \end{aligned}$$

(20)

Then, we take the differentials of the elements of the score vector given in (19) and (20) with respect to ${\varvec{\beta }}$ and $\sigma ^2$ as

$$\begin{aligned} \text {d}^2_{{\varvec{\beta }}} \ell= & {} - \frac{1}{\sigma ^2} \text {d} {\varvec{\beta }} {\varvec{X}}^\top {\varvec{X}} \text {d}{\varvec{\beta }}, \end{aligned}$$

(21)

$$\begin{aligned} \text {d}^2_{\sigma ^2} \ell= & {} \frac{n}{2 \sigma ^4} \text {d}\sigma ^2 \text {d} \sigma ^2 - \frac{1}{\sigma ^6}\text {d} \sigma ^2 ({\varvec{y}}-{\varvec{X}} {\varvec{\beta }})^\top ({\varvec{y}}-{\varvec{X}} {\varvec{\beta }})\text {d} \sigma ^2, \end{aligned}$$

(22)

$$\begin{aligned} \text {d}^2_{{\varvec{\beta }} \sigma ^2} \ell= & {} - \frac{1}{\sigma ^4} \text {d}{\varvec{\beta }}^\top {\varvec{X}}^\top ({\varvec{y}}-{\varvec{X}} {\varvec{\beta }})\text {d} \sigma ^2. \end{aligned}$$

(23)

We establish the Hessian matrix ${\varvec{H}}({\varvec{\theta }})$ from the differentials given in (21), (22) and (23).

Appendix 2: differentials for the perturbation schemes

We present the differentials for the perturbation schemes in the spherical disturbance case defined in Sect. 3.2 considering the log-likelihood functions $\ell _{{{\varvec{w}}}_1}$, $\ell _{{{\varvec{w}}}_2}$ and $\ell _{{{\varvec{w}}}_3}$ established in (16), (17) and (18), respectively. From the differentials, we get the ${\varvec{\varDelta }}$ matrices.

Model perturbation Taking the differential of $\ell _{{{\varvec{w}}}_1}$ with respect to ${\varvec{\beta }}$ and $\sigma ^2$, we obtain

$$\begin{aligned} \text {d}_\beta \ell _{{{\varvec{w}}}_1}= & {} \frac{1}{\sigma ^2} \text {d}{\varvec{\beta }}^\top {\varvec{X}}^\top {\varvec{W}} ({\varvec{y}}-{\varvec{X}}{\varvec{\beta }}) - \text {d}{\varvec{\beta }}^\top {\varvec{R}}^\top {\uplambda },\\ \text {d}_{\sigma ^2} \ell _{{{\varvec{w}}}_1}= & {} - \frac{n}{2 \sigma ^2}\text {d}\sigma ^2 + {({\varvec{y}}-{\varvec{X}}{\varvec{\beta }})^\top {\varvec{W}}({\varvec{y}}-{\varvec{X}} {\varvec{\beta }}) \over 2 \sigma ^4}\text {d} \sigma ^2. \end{aligned}$$

Taking the differential of $\text {d} \ell _{{{\varvec{w}}}_1}$ with respect to ${\varvec{w}}$, we obtain

$$\begin{aligned} \text {d}^2_{\beta w}\ell _{{{\varvec{w}}}_1}= & {} \frac{1}{\sigma ^2} \text {d}{\varvec{\beta }}^\top {\varvec{X}}^\top \text {d} {\varvec{W}}({\varvec{y}}-{\varvec{X}} {\varvec{\beta }}) \nonumber \\= & {} \frac{1}{\sigma ^2} \text {d}{\varvec{\beta }}^\top \left( \left( {\varvec{y}}-{\varvec{X}}{\varvec{\beta }}\right) ^\top \otimes {\varvec{X}}^\top \right) {\varvec{S}} \text {d}{\varvec{w}}, \\ \text {d}^2_{\sigma ^2 w}\ell _{{{\varvec{w}}}_1}= & {} \frac{1}{2 \sigma ^4}\text {d} \sigma ^2({\varvec{y}}-{\varvec{X}}{\varvec{\beta }})^\top \text {d}{\varvec{W}}(y-{\varvec{X}}{\varvec{\beta }}) \nonumber \\= & {} \frac{1}{2 \sigma ^4}\text {d} \sigma ^2\left( \left( {\varvec{y}}-{\varvec{X}}{\varvec{\beta }}\right) ^\top \otimes \left( {\varvec{y}}-{\varvec{X}}{\varvec{\beta }}\right) ^\top \right) {\varvec{S}} d{\varvec{w}}. \end{aligned}$$

Response perturbation Taking the differential of $\ell _{{{\varvec{w}}}_2}$ with respect to ${\varvec{\beta }}$ and $\sigma ^2$, we obtain

$$\begin{aligned} \text {d}_\beta \ell _{{{\varvec{w}}}_2}= & {} \frac{1}{\sigma ^2} \text {d}{\varvec{\beta }}^\top {\varvec{X}}^\top (y+w-{\varvec{X}}{\varvec{\beta }}) - \text {d}{\varvec{\beta }}^\top {\varvec{R}}^\top \varvec{\uplambda }, \\ \text {d}_{\sigma ^2} \ell _{{{\varvec{w}}}_2}= & {} - \frac{n}{2 \sigma ^2}\text {d} \sigma ^2 + {({\varvec{y}}+ {\varvec{w}}-{\varvec{X}}{\varvec{\beta }})^\top ({\varvec{y}}+{\varvec{w}}-{\varvec{X}}{\varvec{\beta }}) \over 2 \sigma ^4}\text {d} \sigma ^2. \end{aligned}$$

Taking the differential of $\text {d} \ell _{{{\varvec{w}}}_2}$ with respect to ${\varvec{w}}$ we obtain

$$\begin{aligned} \text {d}^2_{\beta w} \ell _{{{\varvec{w}}}_2}= & {} \frac{1}{\sigma ^2} \text {d}{\varvec{\beta }}^\top {\varvec{X}}^\top \text {d}{\varvec{w}}, \\\text {d}^2_{\sigma ^2 w} \ell _{{{\varvec{w}}}_2}= & {} \frac{1}{\sigma ^4}\text {d} \sigma ^2({\varvec{y}} + {\varvec{w}} -{\varvec{X}}{\varvec{\beta }})^\top \text {d}{\varvec{w}}. \end{aligned}$$

Covariable perturbation Taking the differential of $\text {d} \ell _{{{\varvec{w}}}_3}$ with respect to ${\varvec{\beta }}$ and $\sigma ^2$, we obtain

$$\begin{aligned} \text {d}_\beta \ell _{{{\varvec{w}}}_3}= & {} \frac{1}{\sigma ^2} \text {d}{\varvec{\beta }}^\top ({\varvec{X}} + {\varvec{W}} {\varvec{A}})^\top ({\varvec{y}}-({\varvec{X}}+ {\varvec{W}} {\varvec{A}}){\varvec{\beta }}) - \text {d}{\varvec{\beta }}^\top {\varvec{R}}^\top \varvec{\uplambda }, \\ \text {d}_{\sigma ^2} \ell _{{{\varvec{w}}}_3}= & {} - \frac{n}{2 \sigma ^2}\text {d} \sigma ^2 + \frac{1}{2 \sigma ^4}\text {d} \sigma ^2({\varvec{y}} -({\varvec{X}}+ {\varvec{W}} {\varvec{A}}){\varvec{\beta }})^\top ({\varvec{y}}-({\varvec{X}}+{\varvec{W}} {\varvec{A}}){\varvec{\beta }}). \end{aligned}$$

Taking the differential of $\text {d} \ell _{{{\varvec{w}}}_3}$ with respect to ${\varvec{w}}$, we obtain

$$\begin{aligned} \text {d}^2_{\beta w} \ell _{{{\varvec{w}}}_3}= & {} \frac{1}{\sigma ^2} \text {d}{\varvec{\beta }}^\top {\varvec{A}} \text {d}{\varvec{W}}^\top ({\varvec{y}}- ({\varvec{X}} +{\varvec{W}} {\varvec{A}}) {\varvec{\beta }}) - \frac{1}{\sigma ^2} \text {d}{\varvec{\beta }}^\top ({\varvec{X}} + {\varvec{W}} {\varvec{A}})^\top \text {d}{\varvec{W}} {\varvec{A}}{\varvec{\beta }}, \\ \text {d}^2_{\sigma ^2 w} \ell _{{{\varvec{w}}}_3}= & {} -\frac{1}{\sigma ^4}\text {d} \sigma ^2 ({\varvec{y}}-{\varvec{X}} {\varvec{\beta }} - {\varvec{W}} {\varvec{A}} {\varvec{\beta }})^\top \text {d} {\varvec{W}} {\varvec{A}} {\varvec{\beta }}. \end{aligned}$$