Robust Two Step Con…dence Sets, Statistic Isaiah Andrews

Robust Two Step Con…dence Sets,
and the Trouble with the First Stage F
Statistic
Isaiah Andrews
Discussion by Bruce Hansen
September 27, 2014
Discussion (Bruce Hansen)
Robust Con…dence Sets
Sept 27, 2014
1 / 10
Classic Weak IV
Y = X θ + e, θ scalar
X = Zπ + v
k instruments
Goal: 95% con…dence set (CS) for θ
Three contributions:
1
2
3
Numerically demonstrates that Stock-Yogo two-step method fails
miserably under heteroskedasticity
Proposes valid two-step CS with bounded size distortion
Introduces diagnostic for degree of non-robust size distortion
Discussion (Bruce Hansen)
Robust Con…dence Sets
Sept 27, 2014
2 / 10
Stock-Yogo Two-Step
CSNR = θ̂
2σθ̂
CSR = θ : S (θ )
0
F = T π̂ Σ̂π
1 π̂/k
χ2k ,1
α
where S (θ ) is AR-like statistic
, …rst-stage F
γ = maximum allowed size distortion
CSNR if F > c (γ)
CSSY =
CSR if F < c (γ)
where c (γ) is selected so that the maximum size distortion is
bounded below γ
Assumes homoskedasticity
Discussion (Bruce Hansen)
Robust Con…dence Sets
Sept 27, 2014
3 / 10
Stock-Yogo Fails Under Heteroskedasticity
Suppose we construct CSSY using hetero-robust variances, but use
Stock-Yogo critical values c (γ)
What is the worst-case coverage?
Numerical …nding: coverage is 0%
Conclusion: Stock-Yogo method fails miserably under
heteroskedasticity
Discussion (Bruce Hansen)
Robust Con…dence Sets
Sept 27, 2014
4 / 10
Comments
Consider 2SLS under weak IV and heteroskedastic. Suppose
1 0
ZZ
T
1
p Z 0e
σe T
1
p Z 0X
σv T
! Ik
! ζ2
! λ + ζ1
with (ζ 1 , ζ 2 ) jointly normal
Then as in Stock-Staiger, for 2SLS
β̂
β
v2
v1
Discussion (Bruce Hansen)
!
=
=
σu v2
σv v1
(λ + ζ 1 )0 ζ 2
(λ + ζ 1 )0 (λ + ζ 1 )
Robust Con…dence Sets
Sept 27, 2014
5 / 10
In homoskedastic case the asymptotic distribution v2 /v1 only depends
on k and the concentration parameter λ0 λ
This is because the covariance matrix of (ζ 1 , ζ 2 ) takes a simple form
Under heteroskedasticity, the distribution is much more complicated,
due to the covariance matrix
Still, “weak identi…cation” appears to be a problem due to small
values of the denominator v1 = (λ + ζ 1 )0 (λ + ζ 1 )
An estimator of E (v1 ) under homo- or heteroskedasticity is the
classic …rst-stage F statistic
I
Not the hetero-robust F statistic
Thus to detect weak instruments under heteroskedasticity, it may still
be appropriate to examine the classic F
However the Stock-Yogo critical values will not provide size control.
Recommendation: The Stock-Yogo approach may still work, but
developing appropriate cut-o¤s would be challenging.
Discussion (Bruce Hansen)
Robust Con…dence Sets
Sept 27, 2014
6 / 10
New Two-Step CS
Pick γ, the maximum size distortion (as in Stock-Yogo)
Set Kγ (θ ) = K (θ ) + a(γ)S (θ ) where K (θ ) is a Kleibergen-like
statistic and S (θ ) is the AR-like statistic
χ2k ,1
Preliminary robust: CSP (γ) = θ : Kγ (θ )
I
Has coverage exceeding 1
α
Robust: CSR (γ) = fθ : Kγ (θ ) Hk ,1
quantile of (1 + a(γ))χ21 + a(γ)χ2k 1
I
Has coverage exceeding 1
Two-Step: CS2 (γ) =
Property: CSp (γ)
I
α
γ
αg ,
where Hk ,1
α
is 1
α
CSNR
if CSp (γ)
CSR (γ) if CSp (γ)
CSNR
CSNR
CS2 (γ)
So CS2 (γ) has coverage exceeding CSp (γ) which exceeds 1
Under strong identi…cation, CSp (γ)
asymptotically
Discussion (Bruce Hansen)
α
α
γ
CSNR , so CS2 (γ) = CSNR
Robust Con…dence Sets
Sept 27, 2014
7 / 10
Diagnostic
De…ne γ̂ as the smallest γ such that CSp (γ̂)
CSNR
Recommends reporting CSNR , CSR (γ̂), and γ̂
If γ̂ is small (e.g. less than 10%), we focus on CSNR
If γ̂ is large (e.g. more than 10%) we focus on CSR (γ̂)
Discussion (Bruce Hansen)
Robust Con…dence Sets
Sept 27, 2014
8 / 10
Specification
28
2SLS
CUGMM
CUGMM, NW
HLIM
HFUL
First Stage F
Efficient K CS
2SLS K CS
S CS
Controls
Base Controls
Age, Age2
SOB
Instruments
QOB
QOB*YOB
QOB*SOB
# instruments
Observations
I
s.e.
—ˆ
0.0990
0.1000
0.1000
0.0999
0.0995
“ˆ
0.0207 2.5%
0.0208 1.9%
0.0213 1.3%
0.0213 1.3%
0.0210 1.8%
30.5822
[0.059,0.144]
[0.059,0.144]
[0.052,0.153]
—ˆ
II
s.e.
0.0806
0.0857
0.0857
0.0838
0.0836
“ˆ
0.0165 18%
0.0165 9.5%
0.0196 1.6%
0.0196 1.5%
0.0194 1.9%
4.6245
[0.0474,0.1261]
[0.0456,0.1239]
[-0.0023,0.1851]
III
s.e.
—ˆ
0.0600
0.0613
0.0613
0.0574
0.0576
IV
“ˆ
0.0292 57.9%
0.0292 55.5%
0.0504 7.3%
0.0515
8%
0.0496 6.4%
1.5788
[-0.5,0.5]
[-0.5,0.5]
[-0.4855,0.5]
Estimate
0.0811
0.1019
0.1019
0.0997
0.0995
s.e.
“ˆ
0.0111 89%
0.0110 40.5%
0.0219 1.9%
0.0211 3.7%
0.0210 3.7%
1.8232
[0.0581,0.1475]
[-0.5,0.146]
[-0.0175,0.2511]
Yes
No
No
Yes
No
No
Yes
Yes
No
Yes
Yes
Yes
Yes
No
No
3
Yes
Yes
No
30
Yes
Yes
No
28
Yes
Yes
Yes
178
329,509
329,509
329,509
329,509
Table 5: Results for Angrist and Krueger (1991) data. Specifications as in Staiger and Stock (1997): Y =log weekly
wages, X=years of schooling, instruments Z and exogenous controls as indicated. Base controls are Race, MSA, Married,
Region, and Year of Birth. QOB, YOB, and SOB are quarter, year, and state of birth dummies, respectively. CUGMM,
NW is CUGMM together with Newey and Windermeijer (2009) standard errors, while HLIM and HFUL are estimators
and standard errors proposed in Hausman et al. (2012). The maximal distortion cutoffs “ˆ are based on comparing nominal
95% Wald confidence sets to the appropriate robust confidence sets CSR,P as described in Section 3.5. Efficiently weighted
K statistic confidence sets with ˆ = ˆ ≠1
are used to calculate “ˆ for CUGMM with usual and NW standard errors, while
1g
2≠1
2SLS-weighted K statistic with ˆ = T1 Z Õ Z
is used for other estimators. For non-convex confidence sets we report
the convex hull.
Questions for Future Consideration
CS are inverse tests. Good CS are constructed from good (powerful)
tests
I
Can we think of CS2 as a good test statistic for θ?
Stock-Yogo motivated CS selection by testing the hypothesis of weak
instruments
I
Is the test “CSp
CSNR ” a good “test” of this hypothesis?
The paper examines CS coverage, e.g. test size
I
What about power? Does CS2 have good power?
The theory concerns CSR (γ) for …xed γ. Does CSR (γ̂) have similar
properties?
The diagnostic γ̂ is very clever. What are its properties? Is it an
estimate, or more like a p-value?
When we have multiple coe¢ cients, does it make sense to have γ̂
vary across coe¢ cients? (Would it make sense to pick the robust CS
for one coe¢ cient and the non-robust CS for another?)
Discussion (Bruce Hansen)
Robust Con…dence Sets
Sept 27, 2014
9 / 10
Uniformity Versus Bounded Distortion
When test statistics are (asymptotically) non-pivotal, there is not a
unique method to conduct tests or construct con…dence sets
One strong (Berkeley) tradition is to focus on uniform tests
I
I
Uniform tests are currently quite fashionable
They can have the pitfall of being quite conservative
This paper focuses on tests with bounded size distortion
I
I
I
Similar to Stock and Yogo (2005)
The advantage is that the tests have good size (and power) in the
leading case of strong identi…cation
There is bounded distortion under weak identi…cation
This seems to be a useful and practical compromise
We should not be overly concerned with uniformity (it does not
correspond to optimal decision-making)
Discussion (Bruce Hansen)
Robust Con…dence Sets
Sept 27, 2014
10 / 10