How extensive (long) should hindcasts be? Huug van den Dool Suranjana Saha

How extensive (long) should
hindcasts be?
Huug van den Dool
Climate Prediction Center, NCEP/NWS/NOAA
Suranjana Saha
Environmental Modeling Center, NCEP/NWS/NOAA
Explained Variance (%)
Feb 1981-2001; lead 3 (Nov starts); monthly T2m (US, CD data)
MODEL
CFS
EC
PLA
METF
UKM
INGV
LOD
CERF
MME8
(EW)
ALL
MODELS
MME3
(EW)
CFS+EC
+UKM
2.1
1.2
0.0
0.0
0.0
0.4
0.2
0.0
0.2
0.9
4.3
7.1
1.4
1.4
7.5
1.4
0.4
2.2
3.8
8.6
SEC21 11.2 8.0
(all 21 (0.33
years) cor)
0.4
0.4
8.6
0.6
0.1
0.5
2.0
17.0
SEC
SEC0
(NO
SE)
SEC8
(last 8
years)
Explained Variance=Square of Anom Correlation
SEC : Systematic Error Correction; EW : Equal Weights
CFS=CFS, USA; EC=ECMWF; PLA=Max Planck Inst, Germany;
METF=MeteoFrance, France; UKM=UKMetOffice; INGV=INGV, Italy,
LOD=LODYC, France; CERF=CERFACS, France
Anomaly Correlation (%)
Feb 1981-2001; lead 3 (Nov starts); monthly T2m (US, CD data)
WITH SEC21
WITH SEC8
SEC8-SEC21
SEC :
Systematic
Error
Correction
Need more
years to
determine the
SEC
where/when the
inter annual
standard
deviation is
large
CONCLUSIONS
• Without SEC (systematic error correction) there is no skill
by any method (for presumably the best month: Feb)
• With SEC (1st moment only), there is skill by only a few
models (5 out of 8 are still useless)
• MME not good when quality of models varies too much
• MME3 works well, when using just three good models
CONCLUSIONS (contd)
• CFS improves the most from extensive hindcasts (21
years noticeably better than 8) and has the most skill.
Other models have far less skill with all years included.
• Cross validation (CV) is problematic (leave 3 years out
when doing 8 year based SEC?)
• Need more years to determine the SEC where/when the
inter annual standard deviation is large
Skill in SST Anomaly Prediction for Nino-3.4
[DJF 97/98 to AMJ 04]
5-member CFS reforecasts
Skill in SST Anomaly Prediction for Nino-3.4
[DJF 97/98 to AMJ 04]
15-member CFS reforecasts
100
90
CFS
CMP
CCA
CA
MAR
CON
80
70
60
Anomaly Correlation [%]
Anomaly Correlation [%]
100
90
CFS
CMP
CCA
CA
MRK
CON
80
70
60
50
50
1
2
3
4
5
1
6
2
Forecast Lead [Month]
3
4
5
6
Forecast Lead [Month]
Skill in SST Anomaly Prediction for Nino-3.4
[DJF 81/82 to AMJ 04]
5-member CFS reforecasts
Skill in SST Anomaly Prediction for Nino-3.4
[DJF 81/82 to AMJ 04]
15-member CFS reforecasts
100
90
CFS
80
CMP
CCA
CA
70
MAR
60
Anomaly Correlation [%]
Anomaly Correlation [%]
100
90
CFS
CMP
CCA
CA
MRK
80
70
60
50
50
1
2
3
4
Forecast Lead [Month]
5
6
1
2
3
4
Forecast Lead [Month]
5
6