Development of an Open-Source Integrated Test Strategy for Skin Sensitization... J Pirone 1

Development of an Open-Source Integrated Test Strategy for Skin Sensitization Potency
J Pirone1, J Strickland2, M Smith1, N Kleinstreuer2, B Jones2, Y Dancik3, R Morris1, L Rinckel2, W Casey4, J Jaworska3
1SSS, Inc., Durham, NC, USA; 2ILS, RTP, NC, USA; 3P&G NV, Strombeek – Bever, Belgium; 4NICEATM/DNTP/NIEHS/NIH/HHS, RTP, NC, USA
Abstract
Methods (cont’d)
Regulatory authorities require testing to identify substances with the potential to cause allergic
contact dermatitis. Integrated testing strategies (ITS) that combine in silico and in vitro test
methods have been proposed to reduce or eliminate animal use for this testing. A published skin
sensitization ITS used a Bayesian network (BN ITS-2) to structure in silico and in vitro assay
results that map to the OECD Adverse Outcome Pathway for skin sensitization. This model was
developed using a commercial software package. To increase accessibility and algorithmic
transparency, we developed an open-source ITS (OS ITS-2) using tools in the R software
package to build and perform exact inference using a Bayesian network. R versions of widely
used algorithms for supervised discretization and latent class learning were substituted for
proprietary algorithms. The overall classification accuracies for the OS ITS-2 and the BN ITS-2
were the same, with three compounds misclassified by both methods. Two case studies of
representative substances, chlorobenzene and 2-mercaptobenzothiazole, were developed and
evaluated using the NICEATM skin sensitization database. Value of information was assessed for
the in vitro assays and in silico inputs. The OS ITS-2 increases availability and transparency of the
ITS and represents a major step in allowing the ITS to be reproduced and tested, properties that
are essential for implementation in a regulatory framework.
•
Refinements to the published BN ITS-2 for skin sensitization (Jaworska et al. 2013)
made in the OS ITS-2 include:
–
–
A change in the method for calculating the bioavailability parameters to improve
transparency (to assure public access to all of the calculations) and consistency
of predictions


The evaluation and promotion of alternative test methods for regulatory use in
assessing skin sensitization hazards are a priority of the Interagency Coordinating
Committee on the Validation of Alternative Methods (ICCVAM).
–
•
•
The murine local lymph node assay (LLNA), the first alternative test method
evaluated by ICCVAM, has been accepted internationally since 2002 for
assessing skin sensitization hazard (OECD 2010).
The BN ITS:
–
•
Compared with guinea pig methods, the LLNA reduces the use of animals and
eliminates the potential pain and distress associated with a positive response.
To further reduce and potentially eliminate animal use for skin sensitization testing,
potency results from the LLNA were used as the target endpoint to develop an
integrated testing strategy (ITS) using a Bayesian network (BN) (Jaworska et al.
2011, 2013).
–
–
Combines relevant in silico and in vitro data to make probabilistic predictions of
skin sensitization potency category (Table 1)
Is aligned with the adverse outcome pathway (AOP) for substances that initiate
the skin sensitization process by crossing the skin barrier and covalently binding
to skin proteins (OECD 2012)
Table 1. LLNA EC3 Correspondence to Skin
Sensitization Potency Categories
•
•
TIMES
Water solubility (Sw)
•
Vapor pressure (Pvp)
•
Density, pKa value(s), Log D, MW (i.e., from ACD/Labs v 12.0)
•
EpiSuite calculated melting point
Cfree
Potency Category
No EC3
Nonsensitizer
EC3 ≥ 10%
Weak
1% ≤ EC3 < 10%
EC3 < 1%
Moderate
Strong or extreme
The refined version of OS ITS-2 is referred to as OS ITS-2 lipid and is posted on
the NTP website at http://ntp.niehs.nih.gov/go/its.
The OS ITS-2 lipid model was trained to the target variable, LLNA potency category,
with 124 substances: 36 nonsensitizers, 28 weak sensitizers, 35 moderate
sensitizers, and 25 strong or extreme sensitizers.
•
The categorical LLNA potency predictions of the model were tested using
21 substances in an external text set: 6 nonsensitizers, 5 weak sensitizers, 5
moderate sensitizers, and 5 strong or extreme sensitizers.
•
Table 2. Libraries Utilized by OS ITS-2
Libraries
Description
For the training set, the accuracy of potency category predictions was greater for
the OS ITS-2 lipid model: 78% (97/124) vs. 76% (94/124) for the commercial
BN ITS-2 model.

Table 3. Variables for the Open-Source
ITS-2 Lipid Model
Variable
LLNA
Description
Measurement
Potency classification in four categories, 1 = nonsensitizer
based on the EC3 ranges in Table 1
2 = weak sensitizer

Abbreviation in
Figure 1
–
LLNA
Moderate
Sensitizer (5)
Strong
Sensitizer (5)
Nonsensitizer (7)
6
1
0
0
Weak Sensitizer (5) (4)
0
4
1
0
0
0
0
4
1
Strong Sensitizer (4) (5)
0
0
0
1
4
gRbase and gRain
Discretization
Contains implementations of several algorithms for supervised and
unsupervised discretization of variables
U937 Activation Test
In vitro test that uses the human myeloid EC150 (µM) for CD86 cell surface
cell line U937
marker expression
Direct Peptide Reactivity In chemico method that measures
Assay
peptide remaining after the test
substance binds to two model
heptapeptides
KeratinoSens Assay
In vitro test that detects electrophiles
using the Nrf2 electrophile-sensing
pathway in the HaCaT (immortalized
keratinocyte) cell line
CD86

1) Percent cysteine peptide remaining 1) DPRACys
2) Percent lysine peptide remaining
2) DPRALys
Used for learning the latent classes
The open-source model
the categories in Table 1 and as a Category 1B (other than strong) sensitizer by the
Globally Harmonized System (GHS). 2-Mercaptobenzothiazole is also a GHS Category
1B sensitizer based on human tests (geometric mean dose per unit area at the 5%
response = 1930
and a Category 1A (strong) guinea pig sensitizer (ICCVAM
2011).
Testing Strategy
–
Assume that the in silico information is available: log Kow, Cfree, AUC120, and
1
Nonsensitizer
Weak
Moderate
Strong/Extreme
0.07
0.13
0.43
0.37
1) Average concentration that
produces 1.5-fold enhanced activity
(µM)
2) Average concentration yielding
3-fold enhanced activity (µM)
Log Kow
logKow
Bioavailability
Concentration of chemical reaching the
mid-epidermal layer of skin calculated
using a transdermal transport model
(Kasting et al. 2008).
1) Free test substance concentration
in mid-epidermis multiplied by
thickness of viable epidermis
(0.01 cm) expressed as percent of
applied dose
1) Cfree
Case Studies
–

•
Nonsensitizer
Weak
Moderate
Strong/Extreme
0.011
0.069
0.61
0.31
The Cysteine latent variable has the highest mutual information for the LLNA,
–
After obtaining the KeratinoSens data, the probability for the moderate category
Chlorobenzene is a solvent and chemical intermediate. It is typically a nonsensitizer in
Potency Category Probabilities (KeratoSens Data)
the LLNA and in guinea pig skin sensitization tests (ICCVAM 2009). It is assumed to be
a nonsensitizer in humans due to a lack of evidence for skin sensitization (ICCVAM
Nonsensitizer
Weak
Moderate
Strong/Extreme
0.000045
0.036
0.67
0.29
2009).
•
•
Only DPRALys has any mutual information for the LLNA, 0.05 (Figure 3c).
–
•
After all information, including DPRA, is included, the probability for the moderate
–
In silico categorical prediction of skin
Three categories: nonsensitizer,
sensitization potency using TIMES
weak sensitizer, and
(Tissue Metabolism Simulator) software moderate/strong/extreme sensitizer
(V.2.25.7), an expert system that makes
predictions based on knowledge about
the parent compound and potential skin
metabolites (Dimitrov et al. 2005).
Because physicochemical properties may be obtained without wet laboratory work,
Potency Category Probabilities (All Variables)
assume that logKow, and other physicochemical properties for calculating the
bioavailability of chlorobenzene in skin are known and applied to the model. Cfree
Nonsensitizer
Weak
Moderate
Strong/Extreme
0.000096
0.053
0.71
0.24
and AUC120, measures of the bioavailability of chlorobenzene in the skin, are
Potency Category Probabilities
Using the OS ITS-2 lipid model, no substances were overclassified and 3
substances (14%) were underclassified.
Nonsensitizer
Weak
Moderate
Strong/Extreme
0.82
0.084
0.072
0.028

Figure 3. Testing Strategy for
2-Mercaptobenzothiazole
The latent variable Cysteine has the highest mutual information for the LLNA,
highest mutual information for Cysteine (0.27 and 0.39, respectively).
–
B.
A.
Table 4. Confusion Matrix for LLNA Potency Category
Predictions on the Training Set of
124 Substances
DPRACys
After obtaining the KeratinoSens data, including the IC50, the remaining variables
0.25
0.89
0.23
KEC1.5
31
29
2
1
1
2
1
Weak Sensitizer
(27) (26)
3
22
21
2
0
Moderate Sensitizer
(35)
1
3
3
4
26
24
5
4
Abbreviations: EC150 = effective concentration that produces a 1.5-fold increase in the CD86 cell surface
marker expression, the threshold for a positive response; EC3 = effective concentration that produces a
stimulation index of 3, the threshold for a positive response in the LLNA; LLNA = murine local lymph node
assay.
1
6
8
Moderate
Strong/Extreme
0.43
CD86
KEC3
KEC3
0.92
0.049
0.00097
0.031
0.07
0.09
KEC1.5
18
20
AUC120
Nonsensitizer
Weak
Moderate
Strong/Extreme
0.97
0.018
0.00020
0.0072
A.
B.
DPRACys
0.84
0.68
Cysteine
0.1
CD86
Cysteine
0.18
0.19
0.61
0.11
0.27
TIMES
AUC120
0.06
KEC1.5
LLNA
0.74
TIMES
0
0
logKow
BA
Cfree
DPRALLys
0.05
0
LLNA
0.74
Abbreviations: LLNA = murine local lymph node assay.
Cysteine
0.32
KEC1.5
CD86
0
IC50
DPRALLys
0.24
LLNA
0.16
0
CD86
0
The numbers in parentheses show the total number of chemicals predicted or observed in each category.
Categories are based on LLNA potency as indicated in Table 1. Numbers in bold red show the different
values yielded by the BN ITS-2 lipid developed using commercial software (Jaworska et al. 2013).
LLNA
LLNA
Acknowledgements
TIMES
0
0
logKow
logKow
BA
logKow
BA
logKow
BA
AUC120
Cfree
logKow
BA
AUC120
Cfree
AUC120
Cfree
AUC120
Cfree
The abbreviations for the variables are listed in Table 3, except for BA = bioavailability. Blue indicates
undefined variables, purple indicates the variables with the highest mutual information, and gray indicates
variables with known values. (A) When the TIMES, logKow, and bioavailability (Cfree and AUC120) are
known, the CD86 data have the highest mutual information for the LLNA. After the CD86 data are applied,
the highest mutual information for the LLNA is yielded by the latent variable Cysteine. (B) KeratinoSens data
have the highest mutual information for Cysteine. (C) After KeratinoSens data are added, the mutual
information for the remaining variable with value for the LLNA, DPRALys, is small.
DPRACys
KEC3
0.16
0.39
IC50
0.34
TIMES
0.24
KEC3
DPRALLys
0.88
C.
DPRACys
0.5
KEC3
KEC1.5
0.05
KEC1.5
When information on all the variables is applied, the probability for the
Figure 2. Testing Strategy for Chlorobenzene
0.1
0
0.07
0.42
0
nonsensitizer category increases by a small amount.
0.19
DPRALLys
Cysteine
TIMES
BA

CD86
IC50
DPRALLys
Cysteine
0.12
0.71
0
CD86
0.34
IC50
DPRALLys
0.28
TIMES
Potency Category Probabilities (All Variables)
Nonsensitizer
(36) (32)
1
2
Weak
DPRACys
DPRACys
0.28
KEC3
0.67
0.8
Nonsensitizer
1
Nonsensitizer (36)
R Development Core Team. 2008. R: A Language and Environment for Statistical Computing
(ISBN 3-900051-07-0). Vienna, Austria:R Foundation for Statistical Computing. Available: www.Rproject.org
C.
LLNA
Strong/Extreme
Sensitizer (25)
OECD. 2012. OECD Series on Testing and Assessment No. 168. The Adverse Outcome Pathway
for Skin Sensitisation Initiated by Covalent Binding to Proteins, Part 1: Scientific Assessment.
Paris:OECD Publishing. Available: http://www.oecd.org/env/ehs/testing/adverse-outcomepathways-molecular-screening-and-toxicogenomics.htm [accessed 2 Dec 2013]
0.32 (Figure 2b). The KeratinoSens variables, KEC1.5 and KEC3, have the
For the commercial BN ITS-2 lipid model, 1 substance (18%) was
overclassified and 2 substances (10%) were underclassified.
Strong/Extreme
Sensitizer (26) (31)
Kasting GB, Miller MA, Nitsche JM. 2008. Absorption and evaporation of volatile compounds
applied to skin. In: Dermatologic, Cosmeceutic and Cosmetic Development (Walters KA and
Roberts MS, eds). New York: Informa Healthcare USA, 385–400.
OECD. 2010. Test No. 429. Skin Sensitisation: Local Lymph Node Assay [adopted 22 July 2010].
In: OECD Guidelines for the Testing of Chemicals, Section 4: Health Effects. Paris:OECD
Publishing. Available: http://dx.doi.org/10.1787/9789264071100-en
Potency Category Probabilities (KeratinoSens Data)
TIMES
Jaworska J, Dancik Y, Kern P, Gerberick GF, Natsch A. 2013. Bayesian integrated testing
strategy to assess skin sensitization potency: from theory to practice. J Appl Toxicol 33: 1353–
1364.
category increases again slightly.
Cysteine
2) Area under the flux curve at 120 h
(percent of applied dose)
Jaworska J, Harol A, Kern PS, Gerberick GF. 2011. Integrating non-animal test information into an
adaptive testing strategy—skin sensitization proof of concept case. ALTEX 28: 211–225.
Testing Strategy
When the OS ITS-2 lipid model is trained to the training set of 124 substances, the
Dimitrov SD, Low LK, Patlewicz GY, et al. 2005. Skin sensitization: modeling based on skin
metabolism simulation and formation of protein conjugates. Int J
Toxicol 24: 189–204.
ICCVAM. 2011. ICCVAM Test Method Evaluation Report: Usefulness and Limitations of the
Murine Local Lymph Node Assay for Potency Categorization of Chemicals Causing Allergic
Contact Dermatitis in Humans. NIH Publication No. 11-7709. Research Triangle Park,
NC:National Institute of Environmental Health Sciences. Available at
http://iccvam.niehs.nih.gov/methods/immunotox/LLNA-pot/TMER.htm
increases:
applied to the model.
Moderate
Sensitizer (35)
Future work will
ICCVAM. 2009. Recommended Performance Standards: Murine Local Lymph Node Assay. NIH
Publication No. 09-7357. Research Triangle Park, NC:National Institute of Environmental Health
Sciences. Available at http://iccvam.niehs.nih.gov/methods/immunotox/llna_PerfStds.htm
mutual information for Cysteine (0.42 and 0.34, respectively) (Figure 3b).
included in the model. Assume that the TIMES result, an in silico prediction, is
Weak Sensitizer
(28)
•
References
0.09, and the KeratinoSens variables, KEC1.5 and KEC3, have the highest
1. Chlorobenzene
Using the commercial BN ITS-2 model, 21 substances (17%) were
overclassified and 9 substances (7%) were underclassified.
Observed Potency Category
OS ITS-2 lipid is available to the public for testing at http://ntp.niehs.nih.gov/go/its.
The variable CD86 has the highest mutual information for the LLNA, 0.28
Potency Category Probabilities (U937 Activation Test Data)
Chlorobenzene and 2-mercaptobenzothiazole are two case studies that illustrate how
the OS ITS-2 lipid model can use existing information to determine the in vitro or in silico
tests that would be most effective for determining the potency classification. Value of
information (VoI) from all possible sources determines which variable provides the most
information about the target. VoI was assessed by calculating the mutual information
between variables, which determines the uncertainty in one variable that is reduced by
knowing the results from another variable.
Using the OS ITS-2 lipid model, 15 substances (12%) were overclassified
(predicted category was more severe than observed in the LLNA) and
12 substances (10%) were underclassified (predicted category was less
severe than observed in the LLNA).
Predicted Potency
Category1
•
When probabilities are recalculated after obtaining the U937 activation test data:
IC50
2) AUC120
Represents a major step in allowing the ITS to be reproduced and tested,
properties that are essential for implementation in a regulatory framework
(Figure 3a).
2) KEC3
Octanol–water partition coefficient

 Add additional substances to the trained model as data are collected

1) KEC1.5
3) IC50
Increases the availability and transparency of the ITS
 Evaluate open source replacements for the TIMES-M in silico predictions and
open sources for physicochemical properties needed for the bioavailability
calculations
Potency Category Probabilities
The numbers in parentheses show the total number of chemicals predicted or observed in each category.
Categories are based on LLNA potency as indicated in Table 1. Numbers in bold red show the different
values yielded by the BN ITS-2 model developed using commercial software (Jaworska et al. 2013).

 Substitute the human cell line activation test for the U937 assay
TIMES (Figure 3a).
have small mutual information values. Thus, no further testing is needed (Figure 2c).
Physicochemical
Property
TIMES-M
µg/cm2)
0.52
1
poLCA
•
2-Mercaptobenzothiazole is used in manufacturing to accelerate the vulcanization of
Abbreviations: LLNA = murine local lymph node assay.
IC50
Supply tools for constructing, parameterizing, and performing
inference on graphical independence networks
The OS ITS-2 lipid model for skin sensitization potency adequately reproduces the
BN ITS-2 lipid model developed using commercial software.
•
Moderate Sensitizer (5)
For the test set, the accuracy of potency category predictions was identical for
the OS ITS-2 lipid model: 86% (18/21) vs. 86% (18/21) for the commercial BN
ITS-2 lipid model.

4 = strong or extreme sensitizer
The original and more recent versions of the BN ITS (Jaworska et al. 2011, 2013)
used commercial software.
We have developed an open-source (OS) version of the more recent BN ITS (ITS-2)
(Table 2) using the free and open-source statistical programming language R
(R v3.0.1, GNU Public License v3) (R Development Core Team 2008).
The LLNA potency category predictions of the OS ITS-2 lipid model using R for
discretization with the Class-attribute Interdependence Maximization (CAIM)
algorithm and latent class learning using the poLCA package are shown in Tables 4
and 5 for the training sets and test sets, respectively. The bold red numbers in the
tables show the results of the commercial software in cases where there is a
difference between the OS ITS-2 lipid model and the commercial BN ITS-2 lipid
model.
–
•
variable with the highest mutual information, 0.72, is TIMES (Figure 2a).
3) Concentration producing 50%
cytotoxicity (µM)
•
The arrows show the conditional
dependencies of the variables that
impact murine local lymph node
assay (LLNA) potency. LLNA
potency category is the target
variable. The remaining variables
are manifest variables.
Bioavailability and Cysteine are
latent variables for bioavailability
and cysteine binding, respectively.
The abbreviations for all variables
are listed in Table 3.
Results
The in vitro and in silico data variables relevant to skin sensitization used to train the
model are shown in Table 3. The structure of the OS ITS-2 lipid model is shown in
Figure 1.
Abbreviations: EC3 = effective concentration that produces a stimulation index of 3, the threshold for a
positive response in the LLNA; LLNA = murine local lymph node assay.
Methods
logKow
AUC120
3 = moderate sensitizer
EC3 Range
LLNA
Bioavailability
LogP (i.e., calculated via EpiSuite or ACD/Labs v 12.0 predicted value)
•
Weak
Sensitizer (5)
CD86
The prediction strategy for physicochemical properties was revised to
consider the following parameters:
2. 2-Mercaptobenzothiazole
rubber products. It is classified as a moderate sensitizer (ICCVAM 2011) according to
Nonsensitizer
(6)
DPRACys
DPRALys
Conclusions
Observed Potency Category 1
Predicted Potency
Category1
KEC3
Cysteine
The skin diffusion pathway for polar substances was eliminated from the
calculation as it remains under development and is not yet publicly available.
The bioavailability for the lipid diffusion pathway was calculated using a tool
available on the National Institute for Occupational Safety and Health website
(http://www.cdc.gov/niosh/topics/skin/finiteSkinPermCalc.html).
Case Studies (cont’d)
•
KEC1.5
Introduction
–
IC50
Correction of two errors in the experimental data
•
•
Table 5. Confusion Matrix for LLNA Potency Category
Predictions on the Test Set of 21 Substances
Figure 1. Structure of the OS ITS-2 Lipid
Cfree
The abbreviations for the variables are listed in Table 3, except for BA = bioavailability. Blue indicates
undefined variables, purple indicates the variables with the highest mutual information, and gray indicates
variables with known values. (A) With no information on chlorobenzene, the variable with the highest mutual
information is TIMES. (B) When the TIMES, logKow, and bioavailability (Cfree and AUC120) are known (b),
the KeratinoSens data have the highest mutual information for the latent variable Cysteine. (C) After
KeratinoSens data are applied, the mutual information for the remaining variables is small.
The Intramural Research Program of the National Institute of Environmental Health Sciences
(NIEHS) supported this poster. Technical support was provided by ILS, under NIEHS contracts
N01-ES 35504 and HHSN27320140003C, and SSS, Inc., under NIEHS contract GS-23F-9806H.
The views expressed above do not necessarily represent the official positions of any Federal
agency. Since the poster was written as part of the official duties of the authors, it can be freely
copied.