Indicatore - clinicalaudit.net

Clinical audit on quality indicators in radiotherapy
for Specific Tumor Site Treatment (STST)
Antonella Rosi
Health and Technology Department,
Istituto Superiore di Sanità
(ROMA)
Tampere 8-10 september 2008
QA in Radiotherapy: ISS role
The tasks of the Italian National Health Institute (Istituto Superiore di
Sanità ISS) include the production of technical and scientific advices to the
State and Regions on problems related to the risks of ionising and non
ionising radiations in the environment and/or in medical field
In this framework the ISS established a multidisciplinary Working Group for
Quality Assurance in Radiotherapy :
to develop guidelines on general radiotherapy
topics and on specific techniques
to organize and coordinate clinical
and dosimetric audits in radiotherapy
to organize training Courses on topics related
to Quality Assurance in Radiotherapy
Tampere 8-10 september 2008
ISS activity in RQA had been
addressed :
not only to:
Quality Controls of equipments
but especially to:
Patient related activities
Tampere 8-10 september 2008
The first step
Development of Guidelines for quality
assurance in radiotherapy
Evaluation
Tampere 8-10 september 2008
Quality Indicators
What is a Quality Indicator?
A measurable element for monitoring and
evaluating resources, processes or outcomes
of care
Where are they mainly applied?
Health Technology Assessment (HTA)
Continuous Quality Improvement
Programs
Measure what is measurable, what is not….
make it measurable
G.Galilei
Tampere 8-10 september 2008
Indicators to…..
Indicators are designed not only to identify
structures of excellence, but mainly to assess
operative conditions and draw up plans of
action to provide a continuous quality
improvement. A comprehensive indicator
system should:
encompass structural, process and outcome
dimensions
produce information useful for decision making
become both a sign and a source of motivation for
quality commitment
Tampere 8-10 september 2008
Two points as matter of concern
Interconnection of structure, process, and patient outcome

Top management and medical staff leadership
must be involved in CQI programs
Standard
threshold
The choice of thresholds and standards


Professional societies, governmental agencies and in general
health-care organizations
Sharing experiences among professionals
Tampere 8-10 september 2008
Indicators in Radiotherapy
General indicators to:

provide an overall evaluation of the Centre
Cionini L. et al Radiother Oncol. 2007 82(2):191-200
Specific Tumor Site Treatment (STST)
indicators to :

provide indications on the quality level in the
treatment of a specific tumor site (manuscript in
preparation)
Tampere 8-10 september 2008
The Grid
Items
Definitions
Topic
Rationale
Type of indicator
Numerator
Denominator
Stratification
Standard
Data collection
What you measure
Why you measure
Structure, process, outcome
Parameter value
Reference population
Recommended categories
Reference value (conformity)
Type (population, sample), time
period, frequency, responsible of
data collection, of data analysis, of
the interpretation
Tampere 8-10 september 2008
From the grid….
Rationale
Why to develope indicators on a particular
topic
What you plan to avoid what to promote
Which the advantages you expect to improve
the quality
How relevant is the indicator for the overall
quality of your product
From the grid…..
Standard
From literature data and/or from guidelines of
Scientific Associations
Empiric (on the basis of collected data)
Updated relating to technology improvment or to
additional resources
Consistent with Centre resources
Complying to minima criteria
From the grid…
Conformity
requires a reference value to be reached (standard)
 can be expressed as yes or no
requires a score attribution
allows a step by step evaluation
allows a graduated intercomparison among different Centres
Even indicators need to be evaluated
Are they ?
 proper (able to evaluate the phenomenon to
be monitored)?
 reproducible ?
 adoptable ?
 applicable? (in terms of time and costs)
 understandable ?
 able to demonstrate differences (among
Centres/among subsequent evaluations)?
And ……….. is it possible to exclude confounding
elements during data collection?
Tampere 8-10 september 2008
Indicators in Radiotherapy
Specific Tumor Site Treatment (STST)
indicators to :

provide indications on the quality level in the
treatment of a specific tumor site
Tampere 8-10 september 2008
Working Groups
Gynaecological tumors
Gianstefano GARDANI
Luigi BOVATI
Carlo CAPIRCI
Vincenzo CERCIELLO
Luca CIONINI
Claudio FIORINO
Alberto MAJORANA
Paolo MONTEMAGGI
Aldo SAINATO
Francesca TORTORETO
Breast
Maurizio AMICHETTI
Cynthia ARISTEI
Luisa BEGNOZZI
Antonella CIABATTONI
Franca FOPPIANO
Marina GUENZI
Cristina LEONARDI
Laura LOZZA
Secondo MAGRI
Sofia MEREGALLI
Angelo Filippo MONTI
Giovanni PENDUZZU
Emanuele PIGNOLI
Francesco SCIUMÈ
Bone Metastasis
Giovanni SILVANO
Luigi F. CAZZANIGA
Pietro D’ADDATO
Gianstefano GARDANI
Patrizia OLMI
Umberto RICARDI
Prostate
Riccardo VALDAGNI
Gianfranco BRUSADIN
Rita CONSORTI
Andrea CRESPI
Claudio FIORINO
Pietro GABRIELE
Giovanni MANDOLITI
Alessandra MIRRI
Alessio MORGANTI
Francesco SCIUMÈ
Antonella SORIANI
UO di Radioterapia, Università di Milano-Bicocca, A.O. S.Gerardo, Monza
UO di Radioterapia, AO S.Gerardo, Monza
Radioterapia Oncologica, Azienda USSL 18, Rovigo
Servizio di fisica sanitaria, Istituto Nazionale Cura Tumori Fond.Pascale, Napoli
Dipartimento di Oncologia, AO Pisana, Università degli Studi, Pisa
Servizio di Fisica Sanitaria IRCCS San Raffaele, Milano
Servizio di Fisica Sanitaria casa di Cura e Sollievo della Sofferenza San Giovanni Rotondo, Foggia
UO Radioterapia Presidio Oncologico M. Ascoli, Palermo
UO Radioterapia, AO Pisana, Pisa
UO Radioterapia Ospedale San Giovanni Calibita Fatebenefratelli, Roma
UO di Radioterapia oncologica Ospedale A.Businco, Cagliari e ATreP, Agenzia Provinciale per la Protonterapia, Trento
UO Radioterapia Università Di Perugia ed Ospedale Monteluce, Perugia
Servizio fisica sanitaria Ospedale San Giovanni Calibita Fatebenefratelli, Roma
UO Radioterapia San Filippo Neri, Roma
Servizio di Fisica Medica, Istituto per la ricerca sul Cancro, Genova
Radioterapia, Istituto per la ricerca sul Cancro, Genova
Divisione di Radioterapia, Istituto Europeo di Oncologia, Milano
Dipartimento di Radioterapia, Istituto Nazionale per lo Studio e la Cura dei Tumori, Milano
Servizio di Fisica Sanitaria, Azienda Istituti Ospitalieri, Cremona
UO di Radioterapia, AO S.Gerardo, Monza
Servizio di Fisica Sanitaria, Ospedale Sant’Anna, Como
Divisione di Radioterapia, Ospedale Mauriziano Torino e UO di Radioterapia, IRCC, Candiolo, Torino
Servizio di fisica sanitaria, Istituto Nazionale per lo Studio e la Cura dei Tumori, Milano
UO Radioterapia Presidio Oncologico M. Ascoli, Palermo
S.C. Radioterapia Oncologica - Azienda Sanitaria Locale TA/1, Taranto
Divisione di Radioterapia, Azienda Ospedaliera, S. Anna, Como
S.C. Radioterapia Oncologica - Azienda Sanitaria Locale TA/1, Taranto
UO di Radioterapia, Università di Milano-Bicocca, AO S. Gerardo, Monza
Dipartimento di Radioterapia, Istituto Nazionale per lo Studio e la Cura dei Tumori, Milano
Radioterapia Università di Torino, AO San Giovanni Battista di Torino
Direzione Scientifica, Istituto Nazionale per lo Studio e la Cura dei Tumori, Milano
UO Radioterapia Centro di riferimento oncologico, Aviano, Pordenone
Servizio di fisica sanitaria San Filippo Neri, Roma
Servizio di Fisica Sanitaria, Ospedale S. Gerardo, Monza
Servizio di Fisica Sanitaria IRCCS San Raffaele, Milano
Divisione di Radioterapia, Ospedale Mauriziano Torino e UO di Radioterapia, IRCC, Candiolo, Torino
Radioterapia Oncologica, Azienda USSL 18, Rovigo
UO Radioterapia IFO Istituto Regina Elena, Roma
UO Complessa di Radioterapia Univ Cattolica del S. Cuore - Centro di Ricerca Formazione ad Alta Tecnologia nelle Scienze Biomed, Campobasso
UO Radioterapia Presidio Oncologico M. Ascoli, Palermo
Laboratorio di Fisica Medica IFO Istituto Regina Elena, Roma
Working Groups
Lung
Ermanno EMILIANI
Giovanna BALASSO
Claudio FIORINO
Pietro GABRIELE
Gianstefano GARDANI
Giovanni MANDOLITI
Maria MORELLI
Nicola PERNA
Emanuele PIGNOLI
Enzo RAVO
Umberto RICARDI
Ruggero RUGGIERI
Francesco SCIUMÈ
Giovanni SILVANO
Antonella SORIANI
Rectum
Carlo CAPIRCI
Vincenzo CERCIELLO
Antonella CIABATTONI
Luca CIONINI
Brunello MORRICA
Luigi RAFFAELE
Vincenzo VALENTINI
Head and neck
Patrizia OLMI
Giovanna BALASSO
Filippo Grillo RUGGIERI
Stefania MAGGI
Giovanni PAVANATO
Mara SCISCIOLI
Carlo SOATTI UO
Pierluigi ZORAT
ISS
Manuela LUZI
Pierluigi MOROSINI
Paolo ROAZZI
Antonella ROSI
Vincenza VITI
Servizio di Radioterapia, Ospedale S. Maria delle Croci, Ravenna
UO Radioterapia Ospedale di Circolo, Varese
Servizio di Fisica Medica S.Raffaele Milano
Divisione di Radioterapia, Ospedale Mauriziano Torino e UO di Radioterapia, IRCC, Candiolo, Torino
UO di Radioterapia, Università di Milano-Bicocca, A.O. S.Gerardo, Monza
Radioterapia Oncologica, Azienda USSL 18, Rovigo
Servizio di Fisica Sanitaria, Ospedale S. Maria delle Croci, Ravenna
Servizio di Fisica Sanitaria - Azienda Sanitaria Locale TA/1, Taranto
Servizio di Fisica Sanitaria Istituto Nazionale Tumori, Milano
Unità di Radioterapia, Ist. Naz.Cura Tumori Fond.Pascale, Napoli
Radioterapia Università di Torino - Azienda Ospedaliera San Giovanni Battista di Torino
Servizio di fisica sanitaria, Azienda Ospedaliera "Bianchi - Melacrino - Morelli", Reggio Calabria
UO di Radioterapia Ospedale “G. Ascoli”, Palermo
S.C. Radioterapia Oncologica - Azienda Sanitaria Locale TA/1, Taranto
Laboratorio di Fisica Medica, Istituto Regina Elena, Roma.
Radioterapia Oncologica, Azienda USSL 18, Rovigo
Servizio di fisica sanitaria, Ist. Naz.Cura Tumori Fond.Pascale, Napoli
UO Radioterapia San Filippo Neri, Roma
Dipartimento di Oncologia, AO Pisana, Università degli Studi, Pisa
Unità di Radioterapia, Ist. Naz.Cura Tumori Fond. Pascale, Napoli
Azienda Policlinico Universitario, Catania
UO Radioterapia Università Cattolica Policlinico Gemelli, Roma
Dipartimento di Radioterapia, Istituto Nazionale per lo Studio e la Cura dei Tumori, Milano
UO Radioterapia Ospedale di Circolo, Varese
UO Radioterapia Ospedale Umberto I, Ancona
Servizio di Fisica Sanitaria, Ospedale Umberto I, Ancona
Radioterapia Oncologica, Azienda USSL 18, Rovigo
Divisione di Radioterapia, Ospedale Mauriziano Torino e UO di Radioterapia, IRCC, Candiolo, Torino
Radioterapia, Ospedale A. Manzoni, Lecco
UO Radioterapia ospedale Ca’ Foncello, Treviso
Servizio Informatico, Documentazione, Biblioteca ed Attività Editoriali
Centro Nazionale di Epidemiologia, Sorveglianza e Promozione della Salute
Servizio Informatico, Documentazione, Biblioteca ed Attività Editoriali
Dipartimento Tecnologie e Salute
Dipartimento Tecnologie e Salute
30 Centres have been enruled for clinical audit (18 Centres operating in
Hospitals, 8 in Universities and 4 in IRCCS (Research and Therapy
Institutes with Scientific Character)
7 Tumor sites
45 indicators
Bone Metastasis
6
Breast
6
Gynaecologic tumors
6
Head and neck
7
Lung
7
Prostate
7
Rectum
6
mainly process indicators
1 structure indicator at maximum
at least 1 outcome indicator
Tampere 8-10 september 2008
STST indicators
 an indicator has to provide a tool to evaluate and to
improve the procedures currently used in a Centre for the
treatment of a specific tumor site
 quality indicators for the treatment of a specific
tumor site are NOT a therapeutic protocol, but they
imply the existence of a protocol
 each Centre should define its own indicators
 indicators must be consistent with the Centre resources
 indicators have to comply with guidelines based on evidence
criteria
Tampere 8-10 september 2008
Clinical Indicators
Gynaeco.
Breast
Multidiscipl
approach
Acute
toxicity
Anaemia
monitoring
Dose-volume Overall
Brachy
histograms
treatment time
Process
Staging
Outcome
Dose to OARs
Process
Set-up
verification
Process
Multidiscipl
approach
Process
Process
Anatomical
Patient’s
Data (complet) satisfaction
Process
Technical
resources
Process
Staging
Process
Multidiscipl
Approach
Process
Volume
definition
Outcome
Set-up
verification
Follow-up
(radical RT)
Process
Set-up
errors
Process
Follow-up
(Execution)
Process
Follow-up
(complete)
Process
Rectum
toxicity
Process
Outcome
Process
Lung
Written
protocols
Process
Prostate
Process
Infrastructures
and
methodologies
Structure
Staging
Volume
definition
Process
Rectum
Structure
Multidiscipl
Approach
Process
Set-up
procedures
Set-up
verification
Process
Process
Dose-volume Acute
histograms
Toxicity
Head-neck
Process
Post-operative
waiting time
Process
Waiting time
(radical RT)
Process
Volume
definition
Process
Multidiscipl
approach
Outcome
Multidiscipl
protocols
Process
Staging
Structure/Process Structure/
Process
Waiting times
Extension of
illness
Process
Process
Process
Process
Fractionation
Multidiscipl
approach
Follow-up
(execution)
Follow-up
(complete)
Structure/Process Process
Process
Process
Process
Process
Bone
metastases
Quality of
life
Break (acute
toxicity)
Process
Bone Metastasis
Waiting Times
This indicator sholud be used in general clinical audits considering the influence of team
or machine numeric adequacy on waiting time for radiation treatments beginnings. It
could be a method for internal clinical audit only when human and technological
resources are appropriate
Diagnostic work up and dose fractionation
These indicators can be applied to evaluate the structure (or the organization) of the
department (availability of procedures concerning the diagnostic work up and the
dose fractionation prescription for patients with bone metastases) or for internal
clinical audit (have procedures been respected).
Conformity was reached by more than half of Centres to indicate a high quality standard
of treatment for palliation
Multidisciplinary approach
Conformity was not reached at any Centre. Such an indicator should be
analyzed prospectively after a training to sensibly physicians (radiation
oncologist and orthopedists) to approach together a patient with critical
bone metastasis
Follow up information
Appropriate information about radiation treatment results were available by
only one Centre to remark how difficult is to collect data about palliative
treatments
Breast
Regarding some critical issues of breast indicators, the following
evaluations can be made:
Anatomic data acquisition
This indicator should be used in general clinical audits due to the
possibility of different data acquisition systems among different Centres
Dose to OAR
This indicator revealed some different interpretations and
disagreement in the contouring of OAR among different Centres
Multidisciplinary approach
Very cross tool to use in radiotherapy as a common method in all
pathologies
Patients satisfaction
Could be useful to use a validated questionnaire from Scientific
Associations or Groups (such as EORTC…) to better compare
results from different Countries
Gynaecologic tumors
The small proportion of participants (27%) is probably due to the not wide diffusion of
brachytherapy facilities, inasmuch as the availability of this technique is often mandatory for the
pathology.
Multidisciplinary approach
The majority of the patients (91%) are planned for radiotherapy after discussion between two
specialists (gynecologist and radiotherapist) but only 50% after simultaneous physical
examination
Acute toxicity
The analysis of the questionnaires revealed that two conformity index must be considered, for
patients previously submitted to surgery and for those planned for radical treatment with radiochemotherapy
Monitoring of anemia
Very simple and easy to evaluate: the conformity index as high as 95% is clearly optimistic but
the authors consider it correct and stimulating for a necessary improvement
Dose Volume Histograms
This indicator should be used not as a method for internal audit but as audit among different
Centres; in fact the are only two possibilities: Centres always using DVH e Centres never using
them
Overall radiation treatment time
This must be considered one of the most important indicators, probably as common
audit for many other tumors. For gynaecological cancers, stratification between
postoperative radiotherapy and curative radio-chemotherapy is mandatory
Central boost with brachytherapy
As internal audit may be used only if the Centres follow specific guidelines. It may be useful for
audit among Centres when a stratification for stage of disease is considered
Head and neck
Waiting times for radical radiotherapy
Most Centres missed these indicators and the answering Centres get a low
conformity value. Patients selected for radical radiotherapy are usually affected by
low staging tumor. In this case a reduced waiting time is essential to avoid tumor
progression
Waiting times for post-operative radiotherapy
When radiotherapy follows surgery as exclusive treatment or combined with
antitumoral drugs is important to respect a treatment time from surgery to the end
of radiotherapy into a maximum of 100 days to obtain better results
Multidisciplinary approach and multidisciplinary protocols
The most Centres missed these indicators and even answering Centres get low
conformity values indicating the necessity to stress that in clinical audit a shared
approach in the clinical study of head and neck tumors is strongly suggested
Break for acute toxicity
The treatment interruption due to acute toxicity could reduce the local control
probability of tumor. Centres indicated different conformity value depending on
stratification. To avoid treatment interruption it is suggested to support patient
during treatment also with feeding tube or gastrostomy for a correct nutrition and
the maintenance of blood parameters
For more details on Head and Neck indicators see poster
Lung
A general result for lung tumor is that the treatment of this patology well comply
the standard as concern instrumental resources but it needs to be improved for
accepting and applying multidisciplinary guidelines for staging and sharing of
therapeutic decision.
Technical resources
This indicators depends on interconnection between top management and medical
staff leadership and can be suggested for an intercomparison among Centres
Staging and follow up
The low conformity values get by Centres testify the difficulty to share opinion
when patients are previously treated by other professional (many patients are
referred for diagnosis and staging to Oncologists or Pneumologists)
Volume definition
The low conformity values were get by most Centres with respect to ICRU
62. The use of score for each item of this indicator is an important tool to
monitor improvment in following audits
Set-up verification
The low conformity values get for this indicator are due to difficulties in
volume definition which influence the set-up verification. The different
conformity values obtained in the cases of technique change (different
items in stratification) suggest to separatly evaluate results of these two
different items
Prostate
The results obtained from data analysis indicated that already in 2004 prostate
cancer was treated at least with 3D CRT.
Indicators: infrastrucure and methodologies, volume definition, execution and
complete follow up resulted proper and well understood
Regarding some critical issues of two indicators, the following evaluations can be
made:
Staging
Among the suggested parameters (TNM, PSA, GPS and Comorbidity) the
TNM value was partially missed by Centres partially because most patients
underwent RT following ormonal therapy that can mask the real T value;
partially because the urologist rarely records the initial T value
Set-up errors
30% of audited Centres did not collect data for this indicator due to the
need of additional human resources to monitor set-up errors. A strict
reccomandation had be given by the working group to the involved
Centres to improve in this direction taking into account the high doses
used in prostate cancer treatment with 3D CRT
Rectum
Multidisciplinary approach
This indicator indicated different results for patients coming from the same Centre
(where systematic multidisciplinary approach was used), with respect to patients coming
from other different Centres
Set-up procedures
The analysis of this point indicates that all Centres obtained the minimum score, but large
differences in maximum score among different Centres were observed. So this indicator could be
use for a inter-Centres comparison
Set-up verification
The constraints for the compliance to this indicator were too selective and was not possible to
describe the different procedures used. It is necessary to modify it
Dose Volume Histogram
This indicator is not suggested for internal audit but as audit among different Centres, since some
Centres always used DVH, others Centres never used them
Quality of life evaluation
Only 3/14 Centres answered to this indicator. It is a strong signal for the necessity to
give more attention to this aspect of treatment implications
The AIRO association has realized a multicentric prospective study to monitor the Italian
therapeutic approach to gastro-intestinal cancers. For rectal cancer patients all the
indicators are included into the request information. In Italy it is the first time that
specific organ indicators are tested in a large national study.