Summary Analysis of Reported Data Sets Lev Sirota, PhD

Summary Analysis of
Reported Data Sets
Lev Sirota, PhD
Center for Biologics Evaluation and Research
U.S. Food and Drug Administration
Selected Vaccines
2
General Philosophy
 Goals Hierarchy:
 Define Q-metric (%recovery close to 100%)
– Q=F(L,M,D,V,A,DM)
– Q=F(L,M,D,V,A)
– Q=F(L,M,D,V)
– Q=F(M,D,V)
– …..
– Q=F(V) ?
–Q ?
 Method optimization within each lab was possible approach
 Additional date were submitted by almost all participants
L-lab; M-method; D-dose; V-vaccine; A-adjuvant; DM-desorption method
3
Data
Study design: One lab - one method
with minimum vaccines vs. All
generated data.
•
•
•
•
•
Wider picture
Participants can better compare results
Complexity increases
Sparse data
Statistical analysis is more challenging
4
Reduction Strategies





Quality
Quality
Quality
Quality
Quality
per
per
per
per
per
Method by Lab, Vaccine, Dose
Method by Lab, Vaccine
Method by Lab,
Method
Vaccine?
5
% Recovery by Vaccine: All assays
Distributions
150
200
100
2000
3500
1000
3000
50
150
100
800
600
2500
0
0
SSI - DTaP-IPV 2
SP - Tetraxim 2
SP - Pediacel 2
GSK - Vac B 2
GSK - Vac A 2
400
2000
-50
-1000
-100
-2000
200
1500
1000
-150
50
-3000
-200
-250
0
-300
100.0% maximum
99.5%
97.5%
90.0%
75.0%
quartile
50.0%
median
25.0%
quartile
10.0%
2.5%
0.5%
0.0%
minimum
188.4
188.4
188.4
165.2
100.8
63.5
36.2
-7.4
-18.5
-18.5
-18.5
70.1054
54.839776
14.159569
100.47466
39.736144
15
Mean
Std Dev
Std Err Mean
upper 95% Mean
low er 95% Mean
N
0
-5000
-500
135.7
135.7
135.7
135.7
118.1
55.1
-71.0
-292.2
-292.2
-292.2
-292.2
100.0% maximum
99.5%
97.5%
90.0%
75.0%
quartile
50.0%
median
25.0%
quartile
10.0%
2.5%
0.5%
0.0%
minimum
10.748469
142.04544
47.34848
119.93426
-98.43732
9
Mean
Std Dev
Std Err Mean
upper 95% Mean
low er 95% Mean
N
-200
-400
Quantiles
Quantiles
1163
1163
1163
786
129
47
-56
-3008
-4020
-4020
-4020
100.0% maximum
3099
99.5%
3099
97.5%
3099
90.0%
3099
75.0%
quartile
224
50.0%
median
45
25.0%
quartile 7.87056
10.0%
-255
2.5%
-255
0.5%
-255
0.0%
minimum
-255
-276.0514
1258.806
325.02231
421.05209
-973.155
15
Mean
Std Dev
Std Err Mean
upper 95% Mean
low er 95% Mean
N
100.0% maximum
99.5%
97.5%
90.0%
75.0%
quartile
50.0%
median
25.0%
quartile
10.0%
2.5%
0.5%
0.0%
minimum
694.2
694.2
694.2
439.8
130.6
61.4
11.4
-146.0
-365.0
-365.0
-365.0
Moments
Moments
Moments
Moments
Moments
Mean
Std Dev
Std Err Mean
upper 95% Mean
low er 95% Mean
N
100.0% maximum
99.5%
97.5%
90.0%
75.0%
quartile
50.0%
median
25.0%
quartile
10.0%
2.5%
0.5%
0.0%
minimum
-4000
Quantiles
Quantiles
Quantiles
0
500
381.89129
1030.2384
343.41282
1173.8027
-410.0201
9
Mean
Std Dev
Std Err Mean
upper 95% Mean
low er 95% Mean
N
94.170238
216.43249
55.882628
214.02655
-25.68608
6 15
Standard Statistics for All Assays
% Recovery by Vaccine:
7
Binding Assays
Lab
Boryung Biopharma
Green Cross Co. Korea
GSK
Health Canada
Korean FDA
Korean Vaccine Co
NIBSC
Norwegian Medical Agency
Sanofi Pasteur
Method
Binding Assay - Monoclonal method Binding Assay - Polyclonal method
Desorption
Desorption
Citrate (2) EDTA or Citrate None
Citrate (1)
Citrate (2)
EDTA
0
0
0
1
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
1
0
0
0
0
1
0
0
0
0
0
0
0
0
0
0
0
1
0
1
1
1
0
1
1
1
0
0
(1) Pediacel desorbed with EDTA
(2) Pediacel not desorbed
8
% Recovery by Vaccine:
Binding Assays
Distributions
150
150
200
125
150
100
100
600
350
500
300
400
250
300
200
200
150
100
100
0
50
SSI - aP 2
SSI - DTaP-IPV 2
SP - Tetraxim 2
SP - Pediacel 2
GSK - Vac C 2
GSK - Vac B 2
GSK - Vac A 2
150
200
100
150
75
100
50
50
100.0% maximum
99.5%
97.5%
90.0%
75.0%
quartile
50.0%
median
25.0%
quartile
10.0%
2.5%
0.5%
0.0%
minimum
149.7
149.7
149.7
133.0
68.2
58.7
13.6
-12.9
-18.5
-18.5
-18.5
135.69
135.69
135.69
135.69
118.59
56.82
46.33
7.03
7.03
7.03
7.03
51.670409
44.910725
12.96461
80.205322
23.135496
12
Mean
Std Dev
Std Err Mean
upper 95% Mean
lower 95% Mean
N
100.0% maximum
99.5%
97.5%
90.0%
75.0%
quartile
50.0%
median
25.0%
quartile
10.0%
2.5%
0.5%
0.0%
minimum
213.59
213.59
213.59
213.59
107.25
41.41
24.53
4.32
4.32
4.32
4.32
73.184614
44.872936
15.864979
110.69933
35.669901
8
Mean
Std Dev
Std Err Mean
upper 95% Mean
lower 95% Mean
N
100.0% maximum
99.5%
97.5%
90.0%
75.0%
quartile
50.0%
median
25.0%
quartile
10.0%
2.5%
0.5%
0.0%
minimum
534.3
534.3
534.3
416.6
117.0
70.2
40.8
-32.8
-55.5
-55.5
-55.5
70.066102
65.11868
21.706227
120.12075
20.011453
9
Mean
Std Dev
Std Err Mean
upper 95% Mean
lower 95% Mean
N
100.0% maximum
99.5%
97.5%
90.0%
75.0%
quartile
50.0%
median
25.0%
quartile
10.0%
2.5%
0.5%
0.0%
minimum
342.47
342.47
342.47
342.47
106.88
44.88
17.66
1.75
1.75
1.75
1.75
101.0194
145.52102
42.008301
193.47905
8.5597521
12
Mean
Std Dev
Std Err Mean
upper 95% Mean
lower 95% Mean
N
100.0% maximum
99.5%
97.5%
90.0%
75.0%
quartile
50.0%
median
25.0%
quartile
10.0%
2.5%
0.5%
0.0%
minimum
130.64
130.64
130.64
127.99
90.97
59.01
11.74
2.21
0.00
0.00
0.00
80.225285
105.2618
35.087265
161.13666
-0.686095
9
Mean
Std Dev
Std Err Mean
upper 95% Mean
lower 95% Mean
N
100.0% maximum
99.5%
97.5%
90.0%
75.0%
quartile
50.0%
median
25.0%
quartile
10.0%
2.5%
0.5%
0.0%
minimum
207.47
207.47
207.47
207.47
142.85
82.30
41.30
3.69
3.69
3.69
3.69
Moments
Moments
Moments
Moments
Moments
Moments
Moments
Mean
Std Dev
Std Err Mean
upper 95% Mean
lower 95% Mean
N
100.0% maximum
99.5%
97.5%
90.0%
75.0%
quartile
50.0%
median
25.0%
quartile
10.0%
2.5%
0.5%
0.0%
minimum
0
Quantiles
Quantiles
Quantiles
Quantiles
Quantiles
0
0
-100
0
0
Quantiles
Quantiles
50
50
25
0
100
50
59.002229
44.500704
12.846247
87.276628
30.727831
12
Mean
Std Dev
Std Err Mean
upper 95% Mean
lower 95% Mean
N
9
88.750612
67.511245
25.516852
151.1881
26.313124
7
Standard Statistics: % Recovery
by Vaccine: Binding Assays
Statistic
GSK - Vac
A
GSK - Vac
B
GSK - Vac
C
SP Pediacel
SP Tetraxim
SSI - aP
SSI - DTaPIPV
Mean
52.8
76.6
75.5
122.1
85.3
92.4
62.1
Lower 95% Mean
35.6
49.9
36.6
27.8
39.1
56.9
41.3
Upper 95% Mean
70.0
103.3
114.3
216.3
131.5
128.0
82.8
10
Binding Assays: % Recovery of
PTx spike
Lab
Boryung Biopharma
Boryung Biopharma
Boryung Biopharma
GSK
GSK
GSK
Green Cross Co. Korea
Green Cross Co. Korea
Green Cross Co. Korea
Korean Vaccine Co
Korean Vaccine Co
Korean Vaccine Co
Korean FDA
Korean FDA
Korean FDA
Korean FDA
Korean FDA
Korean FDA
Norwegian Medical Agency
Norwegian Medical Agency
Norwegian Medical Agency
Sanofi Pasteur
Sanofi Pasteur
Sanofi Pasteur
Sanofi Pasteur
Sanofi Pasteur
Sanofi Pasteur
Health Canada
Health Canada
Health Canada
NIBSC
NIBSC
NIBSC
Method
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Polyclonal
Monoclonal
Monoclonal
Monoclonal
Monoclonal
Monoclonal
Monoclonal
Monoclonal
Monoclonal
Monoclonal
Polyclonal
Polyclonal
Polyclonal
Desorption
EDTA
EDTA
EDTA
EDTA
EDTA
EDTA
EDTA
EDTA
EDTA
EDTA
EDTA
EDTA
EDTA
EDTA
EDTA
Citrate (1)
Citrate (1)
Citrate (1)
Citrate (2)
Citrate (2)
Citrate (2)
EDTA or Citrate
EDTA or Citrate
EDTA or Citrate
None
None
None
Citrate (2)
Citrate (2)
Citrate (2)
EDTA
EDTA
EDTA
Spike (IU/ml)
2
10
20
2
10
20
2
10
20
2
10
20
2
10
20
2
10
20
2
10
20
2
10
20
2
10
20
2
10
20
2
10
20
GSK - Vac A
GSK - Vac B
49
49
64
57
75
60
32
11
31
45
45
80
86
247
50
56
54
57
119
102
176
157
154
148
82
106
103
123
97
116
3
6
7
6
8
9
0
97
88
0
0
0
-23
-17
-15
51
56
54
57
63
63
68
70
72
GSK - Vac C
32
48
44
7
29
31
466
102
72
83
138
140
81
102
100
4
4
5
37
40
43
67
70
66
SP - Pediacel SP - Tetraxim SSI - DTaP-IPV
-134
81
80
-22
45
61
-10
33
43
-25
-67
1
51
30
14
34
42
19
1480
259
254
474
447
132
279
266
89
277
188
46
82
74
166
66
57
153
130
25
42
11
66
96
27
43
89
129
15
10
3
13
17
8
15
11
152
81
119
87
116
88
46
0
34
0
37
0
94
35
74
-9
62
-4
63
33
11
65
30
81
64
30
78
151
173
109
48
86
88
43
63
81
11
Monoclonal Assays:
% Recovery of PTx spike
Lab
Sanofi Pasteur
Sanofi Pasteur
Sanofi Pasteur
Sanofi Pasteur
Sanofi Pasteur
Sanofi Pasteur
Health Canada
Health Canada
Health Canada
Desorption
EDTA or Citrate
EDTA or Citrate
EDTA or Citrate
None
None
None
Citrate (2)
Citrate (2)
Citrate (2)
Spike (IU/ml) GSK - Vac A GSK - Vac B
2
0
0
10
20
0
2
-23
10
-17
-15
20
2
51
56
57
10
54
20
63
63
GSK - Vac C
37
40
43
SP - Pediacel SP - Tetraxim SSI - DTaP-IPV
0
46
34
0
37
0
94
35
74
-9
62
-4
63
33
11
65
30
81
30
78
64
SSI – aP was not tested
Pediacel not desorbed (2)
12
Polyclonal Assays: % Recovery
of PTx spike
Lab
Boryung Biopharma
Boryung Biopharma
Boryung Biopharma
GSK
GSK
GSK
Green Cross Co. Korea
Green Cross Co. Korea
Green Cross Co. Korea
Korean Vaccine Co
Korean Vaccine Co
Korean Vaccine Co
Korean FDA
Korean FDA
Korean FDA
Korean FDA
Korean FDA
Korean FDA
Norwegian Medical Agency
Norwegian Medical Agency
Norwegian Medical Agency
NIBSC
NIBSC
NIBSC
Desorption Spike (IU/ml) GSK - Vac A GSK - Vac B GSK - Vac C SP - Pediacel SP - Tetraxim SSI - DTaP-IPV SSI - aP
94
-134
81
80
49
49
32
EDTA
2
49
-22
45
61
64
57
48
EDTA
10
33
43
41
44
-10
EDTA
20
75
60
-25
-67
1
49
32
11
7
EDTA
2
14
38
29
51
30
10
31
45
EDTA
37
34
42
19
45
80
31
EDTA
20
259
466
1480
259
254
2
86
247
EDTA
474
447
132
142
50
56
102
EDTA
10
266
89
105
57
72
279
EDTA
20
54
188
46
301
119
102
83
277
EDTA
2
166
180
157
138
82
74
EDTA
10
176
57
153
142
154
148
140
66
EDTA
20
106
81
130
25
42
72
EDTA
2
82
66
96
89
10
103
123
102
11
EDTA
27
43
89
86
97
116
100
EDTA
20
129
15
10
2
2
3
6
4
Citrate (1)
17
3
Citrate (1)
10
7
6
4
3
13
11
6
Citrate (1)
20
8
9
5
8
15
2
0
152
81
Citrate (2)
Citrate (2)
10
97
119
87
20
88
116
88
Citrate (2)
151
173
109
110
EDTA
2
68
67
EDTA
10
70
70
48
86
88
75
63
81
62
EDTA
20
72
66
43
(1) Pediacel desorbed with EDTA
(2) Pediacel not desorbed
13
Conclusions
 Collaborative studies are valuable tool for
methods assessment and decision making.
 To allow for method comparison, studies
should be carefully planned and executed
according to the protocol.
 Results should be reported using
standardized form.
 The variability of the recovery experiments
depends upon correlation between dose of
spiked analyte and the entity measured by
the method
14
Where Should We Go from Here ?
15
Acknowledgments







Juan Arciniega
Richard Isbrucker
Estelle Russek-Cohen
Dale Horne
Scott, John
Tsai-Lien Lin
Tammy Massie
16
THANK YOU
17
18
19
20
Methods




Binding Assay - Polyclonal method
Binding Assay - Monoclonal method
eHPLC
cAMP Assay
21