Slides - Agenda

Stato Tracciatore
Operazione rivelatore:
run 2010,shutdown ‘10-’11
Run 2011
Consolidamento
Richieste 2012
Pixel upgrade (G.M. Bilei)
06 Sept 2011
Ettore Focardi
1
Brief Report on TK 2010 performance
•
The Strip Tracker has been taking collision very efficiently, all detector sub-system
have been running very stably.
–
–
–
–
–
•
HV-on during stable beam:
97.7%
–
–
•
•
•
•
Average 2010 Strip UpTime of during stable beam 98.83%
Improved along the year, starting with 98.56% and ending with 99.65%.
Down-time mainly due to DAQ crashes caused by Tracker and one rack failure due to a broken
turbine in August.
DCS and DQM never caused/detected problems to running
Cooling caused only once a problem due to a stop for leak rate increase
90% of dead time from beam dump handshake
2-3’ at start of Stable beam
Offline DQM data certification efficiency:
99.8%
Alive channels:
98.1%
Bad components (offline):
0.4%
For Good components: recHit efficiency
>99.9%
06 Sept 2011
Ettore Focardi
2
Relevant events of 2nd half 2010 and developments completed
• FEDs: CRC errors, Sync Lost draining
• VME power-supply failures (happened twice in 2010)
• Turbine problem in August
• Some questions on data certification for physics analysis.
• SS1 pump found almost dead: replaced
• Re-commissioning of two un-cooled lines in November
– focused commissioning effort in two days
• Increase of leak rate in SS2, dramatic in November
– Set up task force; access into UXC
• Important improvements completed:
–
–
–
–
–
Tracker DCS-protection in C-DCS :Versatile and flexible
Increase of CAEN mainframes from 4 to 8
FED data suppression when TRK in STDB: very efficient and useful
Setting of FED readout mode by central DAQ: very useful
General improvement in monitoring tools
06 Sept 2011
Ettore Focardi
3
Scheme of Operations for 2011
4
Tracker
Management
CMS Run
Coordination
CMS DPG
Coordination
Silicon Strip TK Run Coordinators
TK DPG
(continuous):
1 STRIP TK S.L. (
FOR OPERATION
1 TK Off-line Shift Leader
TK P5
TK Experts on-call:
TCS, DAQ DQM.
PS, TSS, FEDs, others
Short daily meeting
2 TK Off-Line shifters
(1sh/d): Tracker
CMS Central
CREW
06 Sept 2011
CMS-Centre
Ettore Focardi
Downtime
CMS inefficiency (in time) is 7%.
•
A snapshot of the downtime since the
beginning of 2011.
•
Pixel is responsible for 9% of the
experiment downtime.
Strips is responsible for 21%
•
06 Sept 2011
Ettore Focardi
5
Power Supply Status
• Power supply system is still very stable
• 1% failure rate (2PSU/month) in 2010
• < 1 PSU/month exchange in 2011
PSM exchanges in 2011
20
16
12
A4601H
A4602
A3486
8
4
0
30/01/11 14/03/11 25/04/11 06/06/11 18/07/11 29/08/11 10/10/11 21/11/11
09/01/11 20/02/11 04/04/11 16/05/11 27/06/11 08/08/11 19/09/11 31/10/11 12/12/11
Week
06 Sept 2011
Ettore Focardi
6
Cooling
• Silicon cooling system is very
stable !!!!
• Leaks rates are low
– SS1 plant: < 0.1 kg/day
– SS2 plant: ~0.5 kg/day (5 cooling lines closed)
• Effort made to reduce the pressure to the detector: 0.8 – 1 bar
reduction
– Module temperatures nearly unchanged
• Cooling plant pressure reduced from 9 bar to 8 bar
– Pressure switches applied to protect against pressure spikes during transition
• Pump running with Variable Frequency Drive (VFD)
– Extend lifetime
– More smooth operations
– No pressure glitches or vibrations
06 Sept 2011
Ettore Focardi
7
Cooling: SS2 Plant Leak History
C6F14 sniffer
2010
2011
new GC
06 Sept 2011
Ettore Focardi
8
DAQ
•
•
•
•
•
•
•
•
DAQ system is very stable. Downtime, however, can be reduced.
The Silicon Tracker has 60% (440) of all CMS Front-End Drivers!
TRK DAQ related downtime in 2011: 8.5 hours (19% CMS downtime)
53 downtime incidents during Stable Beams
Two isolated incidents required > 40’ for recovery
~75% incidents due to single FEDs stopping the DAQ (reconfigure of TRK: ~6 minutes)
In cases where a single FED stops the DAQ, a bus reset can help clean up the system
Most offending FED (87) was replaced and since has not caused any incidents
06 Sept 2011
Ettore Focardi
9
DCS & DQM
• DCS mostly stable
• One issue which caused cooling FSM do not properly restart
• PC re-installation exercise still to be completed
– Many lessons learned and bugs fixed during trail run
DQM:
•
•
•
Implemented “Audio alarm” to signal mismatch between APV modes
Started implementing checks for different CMS running conditions: Interfill,
Beam without collisions, Stable beams
– Different validation plots for different conditions
– Start with Tracker, then implement for all sub-systems
Detector performance to be checked prior to Stable beam declaration
– Checks for APV errors also when HV is OFF
06 Sept 2011
Ettore Focardi
10
Current Status of the Detector
•
Alive channels: 97.75%
– TIB/TID 94.3%
– TOB 98.1%
– TEC+ 98.2%
– TEC- 98.9%
•
Details:
– 2 CR shorted (~0.7%)
– 1 CR missing (0.35%)
– HV lines missing (0.1%)
– HV lines shorted (0.7%)
– Fibers/CCUs/… (0.4%)
•
Potential recovery of 2-3 Control Rings (0.7-0.9%) during LS1
06 Sept 2011
Ettore Focardi
11
History of Channels Excluded channels from DAQ
Strips
Able to quickly recover modules
associated with closed cooling loops
(before data taking).
Pixels
06 Sept 2011
Ettore Focardi
12
HV ON Time % during Stable Beams
~97 % for all partitions
06 Sept 2011
Ettore Focardi
13
Detector Leakage Currents
•
•
•
•
Closely monitoring evolution of leakage currents vs. time
Effects of radiation on silicon are already visible
Clear linear dependency with integrated luminosity: radiation induced
Safety limits adjusted and closely monitored (should be good up to some fb-1)
06 Sept 2011
Ettore Focardi
14
Pix HV Scan
• Barrel
: Forward
Cluster Charge
Cluster Size
06 Sept 2011
Ettore Focardi
15
Pix Efficiency
Barrel
Forward
06 Sept 2011
Ettore Focardi
16
Early radiation effects
•
•
•
•
Leakage current increasing with integrated luminosity
With present temperatures good to run up to O(10)/fb
Pixel can with stand 10E15/(cm2 .fb) radiations with some effect on resolution
Current radiation dosage:
–
–
–
•
Layer1: 2.4E12 (cm-2 .fb-1)
Layer2: 1.3E12 (cm-2 .fb-1)
Layer3: 0.82E12 (cm-2 .fb-1)
Still in good range
06 Sept 2011
Ettore Focardi
17
Alignment
Evolution of the statistical precision (E. Migliore)
Tilt Tk vs B
Drift of BPIX
06 Sept 2011
Ettore Focardi
18
Tracking
06 Sept 2011
Ettore Focardi
19
Consolidation work: TK Cooling
Operations of the cooling plants had to face the recovery from:
• SS2 HEX failure(11/2007) before TK installation,
decision to use C6F14 also in the Brine circuit -> less effective exchange
between Brine circuit and cooling plants, -> high T operation
• Chiller HEX (condenser) failure (5/2008)
• overpressure on cooling lines during leak search(8/2009)
module to cooling pipes thermal contact degraded
Continous work on C6F14 leaks in SS2 until April ’11 when leak rate <1kg/d
5 lines closed
• Humidity conditions marginal in the bulkhead region (THIP project)
cold operation difficult
06 Sept 2011
Ettore Focardi
20
Consolidation work
From these events a list of possible activities as cooling plant consolidation
projects in view of ETS (winter ‘11/’12) and LS1 (‘13/’14) can be grouped in :
Regular, M&O
ETS/LS1
Cold running
ETS/LS1
Monitoring
ETS
Leak repairs
LS1
Filters, etc.
HEXs tests
Vibrat monitor
Locate leaks
Pneum cab repl
Dinam perf
Vib/press meas
R&D leak repair
Plants sealing
Fup test/prep
C6F14 on Tk mt
Remote rep. met.
Impr inertion, purging
Cold run proced Sniff sys com
Impr doors cabin
DCS/WBM
Remote rep opp.
Rev leak se met.
Remote fill/drain
Motor ventil
Vibration analys
Perform plants(sim)
Smooth (auto)ON/OFF
proced
06
Sept 2011
Pump
fail
fol up
Ettore Focardi
21
Humidity improvement (THIP)
Inside the TK volume, dew point stable < 300C.
In the bulkhead and in the cable channels region, dew point is marginal  difficult to run cold
Humidity levels in the Pixel volume strongly dependent on magnet status.
During next ETS/LS1 we would like to improve the detector sealing (at CMS level) with:
Local boxes in the bulkhead frontier region or
Global sealing from the Tracker volume up to the VAC tank.
In both cases we have to increase the N2 flow.
uninsulated
cooling pipes
Bulkhead region
06 Sept 2011
Ettore Focardi
22
Power System
System is running smoothly.
Last major intervention: mainframe changes 4 SY1527 -> 8 SY2527, factor 2 gain in trans. time
Possible improvements:
1) Rack water cooling upgrade (manifolds, HEX)
2) Back board revision to make connection more robust (small quantity)
3)Possible Branch controller upgrade
06 Sept 2011
Ettore Focardi
23
Responsabilita’
No.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Sezione
Responsabilità non incluse nelle Attività
Nome
Tipo Responsabilità
m.u.
PI
Fi
Pg
Ct
Ba
MiB
Pd
To
A. Giassi
A.Messineo
F.Palla
A. Venturi
P.G. Verdini
G.Tonelli
Focardi
Meschini
Bilei
S. Costa
De Palma
Menasce
Dorigo
De Maria
L3
L3
L3
L3
L3
L3
L2
L3
L2
L2
L3
L3
L3
L2
TK DAQ coord
Coord. upgrade
MB Fase 2
TK Pog coord
TK TSS,environment
spokesperson emeritus
TK Tec Coord/INFN manag
Tk FB
TK Pix Ph1/Infn coord
TK RM
TK MB/Infn coord
TK Pix DB
Statistics coor
TK Operation coord
3
4
3
6
3
3
7
2
5
6
3
3
3
6
57
L1
L3
L2
L3
L3
L3
L1
L3
L3
L3
L2
L2
Deputy comp
Conv EWK tau/Tier2s
MB
B tag
H->zz
T2-Ba/distr.
coord Off CMS
coord. Simul./Off SLHC
Mat. Budg.
V+jets EWK coord
coord PVT
Simul prod man
6
4
3
3
3
3
7
4
3
3
6
2
47
FISICA+ SOFTWARE/COMP
1
2
3
4
5
6
7
8
9
10
11
12
06 Sept 2011
Pi
Ba
Fi
Pd
Boccali
G.Bagliesi
Tenchini
Rizzi
De Filippis
Spinoso
Silvestris
Tricomi
Sguazzoni
Ciulli
Azzi
Torassa
Ettore Focardi
24
TK ME
Respons [mu]
INFN
Tot
BaTk
CT
FI
MIBtk
PDTk
PG
PI
FTE
Attivita'
TK
RPC
Comp/Sw
11,5 Phys
TK
Comp/Sw
5,4 Phys
TK
Comp/Sw
13,9 Phys
EC
TK
Comp/Sw
6,3 Phys
DT
TK
Comp/Sw
7,3 Phys
TK
Comp/Sw
13,3 Phys
TK
Comp/Sw
27,6 Phys
DT
EC
TK
Comp/Sw
8,0 Phys
Totk
TOTAL
06 Sept 2011
E
93,3
Gran Totale
(mu)
Metab
FTE
[mu]
6,6
5,6
3,2
1,7
5,4
1,2
0,7
3,4
13,9
9,9
6,3
4,3
7,3
5,3
13,3
12,3
23,6
4,0
17,6
2,0
8,0
7,0
93,3
69,3
LI
LII
3,0
7,0
6,0
4,0
7,0
6,0
5,0
9,0
6,0
6,0
7,0
52,0
ESP
ESP
[mu]
simf
[mu]
Fir
Mo&A+ [mu]
[mu]
[mu] TOTAL
[mu] dat
LIII
PhD dovuti effettivi CERN
E
Sezione /FTE 11
8,0
24,0
24
20
28,6
0,0
0,0
3,0
6,0
18,0
20,0
12
23,2
3,0
0,0
3,7
55,5 4,8
6,0
18,0
18
12
21,4
0,0
4,0
0,0
0,0
25,4 4,7
5,0
12,0
36,0
36
29
50,9
0,0
0,0
3,0
0,0
3,0
53,9 3,9
0,0
0,0
3,0
6,0
18,0
18
14
21,3
0,0
0,0
3,0
0,0
3,0
24,3 3,9
0,0
8,0
24,0
24
18
23,3
3,0
0,0
3,0
0,0
6,0
32,3 4,4
8,0
24,0
24
22
39,3
0,0
0,0
0,0
39,3 3,0
6,0
22,0
24,0
63,0
63
51
90,6
9,0
9
3
14,0
0,0
6,0 110,6 4,0
0,0
0,0
0,0
0,0
8,0
24,0
24
20
33,0
0,0
0,0
6
0,0
0,0
33,0 4,1
45,0
Ettore Focardi
86,0
258,0
260,0
201,0
25
374,3
374,3
TK ESP ‘12 breakdown
Sezione
FTE
Attivita'
TK
ESP
ESP
TASK
[mu]
8
5
5
6
TK Oper&maint: DQM, Llreco, PS, Operat, Shifts
Shift leader, Det Performance & Oper: DQM,Llreco
Upgrade
BA tk
Software developm & maint:Hlreco,Llreco
6,6 Comp/Sw
TK
24
6
5
3
4
12
10
6
8
TK Oper&maint: PS, Shifts;
Det Performance & Oper: DQM,Llreco
CT
Upgrade
5,4
Software developm & maint: DQM,Llreco,simul
TK
TK Oper&maint: Operat, PS, Shifts;
Det Perf &Oper: DQM,Hlreco;
FI
Upgrade
13,9 Comp/Sw
TK
TK Soft dev& maint: Hlreco,Llreco,sim
6
6
3
3
TK Oper&maint: PS, Shifts;
Det Perf &Oper: Hlreco,DQM
MIBtk
Upgrade
6,3 Comp/Sw
TK
Software developm & maint: DQM,Hlreco
8
7
3
6
8
8
8
TK Oper&maint: Operat, cool, Shifts
Det Perf &Oper: Hlreco,Llreco
PDtk
Upgrade
7,3 Comp/Sw
TK
Software developm & maint: Llreco,simul
TK Oper&maint: Operat, Shifts
Det Perf &Oper: Hlreco, align;
PG
Upgrade
13,3 Comp/Sw
TK
TK Oper&maint: Operat, cool, TSS, DAQ, DQM, det, DCS,Shifts
Upgrade
27,6 Comp/Sw
TK
Dataflow,T2
TK Oper&maint: Operat,Shifts
Upgrade
Totk
06 Sept
TOTALE
Det Perf &Oper: align;Tk pog
8,0
2011 88,4
Soft dev & maint: align
36
18
24
24
CRAB,Data workflow
Soft dev & maint: HLT,Hlreco
PI
18
Ettore Focardi
40
10
12
10
11
2
9
2
240,0
63
24
9
26
consumi
Ba
6 +3+15+3
Ct
3+3
Fi
10+297+3
Mib
10+40
Pd
10
Pg
10+3
Pi
35+6+6
To
5
06 Sept 2011
test sensori + 3 setups fase1+ 15 Tier2+3 cam. Pul.
set-up fase1+3 cam. pul
test sensori, etc. +M&O B 297 (351 KCHF, cambio 1.18)+3 cam. Pul.
test sensori Diamante, carte DAQ etc..+ 40 sj acquisto sensori diamante
sj definizione progetto
set-up fase1
set-up fase1+3 cam. Pul.
5 test sensori, 15 manut. T2,15 C6F14 test moduli +6 setup fase1+6 cam
pul.
sviluppo chip
Ettore Focardi
27
TK M&O B 2012
Continua il programma di riduzione del contributo INFN, conteggio in PHD
afferenti al progetto TK.
Richiesta TK 2012: 1590 KCHF (1735 KCHF ‘11)
contributo globale TK INFN 297 K€ -> 351 KCHF (381 KCHF ‘11), 22%.
06 Sept 2011
Ettore Focardi
28
INV+APP
inventario
Ba
Apparati
207,5
Tier2
differential probe per test HDI+
CPU T3
Ct
5+10
Fi
10
Tier 3
10
Tier3
Mib
Pd
Pg
115
sj sensori Ph1
bump-bonding Ph1 (25)
+macchina per lavorazione
moduli (34) + Tier2
Pi
59+267,5
06 Sept 2011
Ettore Focardi
29
TK M&O B breakdown
Tracker
Mechanics small mechanical works, installation/repair tooling
Gas-system miscellanea: filters,etc.. Test sensor irrad., dew
point meter, recal, flowmeters.
Pix specific
2012
5
40
20
60
30
250
20
110
Cooling system
standard maintenance: filters, etc,
Brine, SS1,SS2, TS1,TS2 rev., leak rarte test setup, template setup
P5, fluid, fluid cleaning, plant 186
Standard electronics, PS (LV, HV)
modules
maintenance (repairs), fans replacements, caen interv on site, misc.
Standard electronics, RO Modules
pool rental,
VME rack PCs DAQ, network upg, Pixel FPGA, misc: crates, optical
modules…
Controls, (DCS, DSS) rad hard humidity sensors+ele
chain+cables, misc: CPUs, PLC batt., sensors, connectors, bulkhead
Areas working labs/strorage @186,904 and P5, Pix chiller,
electronics,PC, DCS,DSS,RP…
100
50
140
80
90
Communications
30
Store Items
50
Hired Manpower @ CERN (KCHF)
Contributo INFN al 22% 351 Kchf
06 Sept 2011
720
285
Ettore Focardi
1590
30
Richieste TK 2012
RICHIESTE TRACKER '12
Sezione
FTE '12
FTE '11
MI
met
ME
met mu
resp
ESP mu
mu
K
€
tot mu /
m
u
CONSUMO
K€
rich.
11mu
as11mu0
5/11
met
auto
ass.11K€
05/11
upg
PH1
INV
upg upgPh2+mis cam.
M&OB
Ph1 sj
c
Pul.
tot cons
APP
tot sj
TOT
ph1 sj
sj
10
Bari
6,6
Baoff
4,9
7,0
8,0
Catania
5,4
13,9
6,3
7,3
13,3
27,6
10
12
12
9,9
15
29
4,3
6
14
5,3
9
18
12,3
5
22
19,6
37
54
8
7,7
88,4
95,5
7
6
20
139,4
132,4
06 Sept 2011
69,3
104
201
35
108,9
25,4
4
,
102,9
0
5
23
3
, 204,8
8
57,5
3
,
7
26,1
53,9
24,3
32,3
39,3
110,6
33
89,9
17,0
10
4,6
3
7,4
4,0
8,1
4,2
20,9
7,6
9,5
4,5
11,0
4,9
10
20,0
7,3
10
41,4
13,0
6
12,0
5,2
6
3
26,6
152,6
69
26,9
30
12,0
Torino
3,4
13
16
41,0
Pisa
1,9
8
20,0
Perugia
28,6
6,1
11,0
Padova
20
13,5
9,5
Milano Bi
3
5,2
20,9
Firenze
5,6
9
4
,
115,8
0
5
10,2
15
3,5
207,
5
26
3
349,5
19
15
144,5
338,41
10
574,1
41,5
28,3
10
3
297
107,5
14,7
40
15
29,0
40,0
139,5
40
54,5
3
, 122,7
8
35
3
,
155,2
9
5
50
3
, 420,3
8
106,7
3
, 122,1
7
31,7
374,3
1442,7
347,4
1333,8
15,0
25,9
159
57
23,2
3
40,3
6
101,44
10
115
225,5
115
830.2
59
91,5
53,6
35
59
203,5
15,0
5
22,2
267.
5
156,2
55,5
625,77
365,0
177,0
680,0
602,26
Ettore Focardi
2730.8
40,0
35
0
174
2381,3
31
214