Summary notes of the
forty-fifth meeting of the LHC Commissioning Working Group
Tuesday May 6th, 14:00
CCC conference room 874/1-011
Minutes of the Previous Meeting and Matters
Arising
There were no comments on the
minutes of the 44th meeting. The 93rd
LHCC meeting was going to be held on 7-8 May 2008. A special meeting
between the DG and experiments was scheduled for this very evening. It was
interpreted to amount to the “2 months warning before beam”. Indeed, as
reported by Massimiliano shortly thereafter, the schedule
according to this meeting is as follows.
End of June 2008: full machine
cold at 2K; mid July 2008: close access
to entire machine, including experimental areas; and end of July: first attempt
to circulate beam.
Dry Runs (Mike)
Mike started his presentation by
reporting the present conditions of the LHC machine, which included the
cool-down of remaining sectors, extensive hardware commissioning, installation
or remaining components, roll out of various items, software development, etc,
while starting up the PS and SPS. The goal of the dry runs is to get ready as
much as possible, to debug, and to establish control and integration from the
CCC. This activity will lead into partial and later full machine checkout.
Multi-system tests are designed to check the integration and the readiness of
all systems. The targets comprise checks of all equipment control
functionality, driving all relevant systems in a synchronized way through the
standard operational sequence, and checking machine protection and interlock
systems. The objective also includes the deployment and testing of LSA. Mike
recalled the overall dry run schedule and the dry-run organization. He
commented that all dry-run activities would be an iterative process, and that
we must be prepared that not everything will be perfectly functioning at the
first time.
Mike next ran through various items
of the dry runs. Items indicated in red on Mike’s slides are those needing
attention. First he looked at the inject&transfer dry runs: Kickers are operational now. RF pre-pulses arrived the day before
the meeting. Abort gap keeper and advanced injection sequence are being
tested. Timing: This Thursday there would be an attempt to synchronize with
SPS and injectors, with the LHC as master, organized by Delphine.
Various modes would be tested: inject & dump, circulate & dump. Instrumentation was another item, and
so were certain applications:
Steering and threading, being tested and interfaced to LSA, injection quality
checks, multi-turn acquisition and analysis, and equipment control. Reyes prepared a sequencer for the various phases.
The injection, ramp & squeeze
dry runs comprise power converters/magnets, collimators, and the RF.
Oliver asked whether the test damper
system was part of the test. Mike replied no, and Gianluigi
added that the powering part had been tested locally. Some of the control is
available. Both pointed out that the low-level part of the hardware was not yet
there, however. At the beginning of beam commissioning, the damping
functionality may not be at hand, but it might already be possible to inject
noise via the damper. Oliver then inquired whether the timing signals for the
damper had been tested so far. Gianluigi answered,
no, only the power converters had yet been checked.
Concerning the PGC (powering of groups of circuits) as part of the circuit HWC,
Mike recalled some examples from February like ramping 138 power converters in
sector 4-5 to 5.3 TeV, and HWC shadowing. Now turning
to the collimators, their settings
generation (86 functions for all motors) had been taken care of by Stefano. Delphine, Eric
and the LSA team. The complex collimator parameter space is supported
within LSA. Collimators were driven in LSS3 simulating the changes needed
during a cycle. From their parking location they went into a coarse setting
first, then to their nominal injection positions and finally they further
closed during the ramp to 7 TeV. Data from 7
collimators (readings of 24 LVDTs) reveal an excellent tracking and position
control.
Oliver asked how the collimators
were driven, and whether the drive was linked to the magnet current. Mike
replied that the collimator drive was synchronized to the timing, and that
different groups of collimators would be driven separately. Replying to a
second question by Oliver whether an interlock is triggered if timing events go
out of synchronization, Mike explained that at the moment events were put into
a table and played via the timing system. He remarked that the events should
all got out, in principle, without any question.
BDS dry runs are driven by Verena.
Programmed dumps, inject & dump mode, and circulate & dump mode are
here tested together with all interfaces. A pertinent wiki
page. http://wikis/display/LHCOP/LBDS+Dry+Runs
gives
further information. Three dry runs were executed so far,
and the fourth one was scheduled for the present week (19). All the beam
instrumentation in the beam dump area was working for both beams, except for
interlocked BPMs.
Coming back to Mike’s
remark that the dry runs were an iterative process, Roger asked which fraction of the
components was already working for the beam dump Mike estimated about 80-90% at
the moment. He pointed out that reliability runs were presently ongoing, with
XPOC running in the background. Such tests required most of the features to be
working. Markus wrote the applications software for the dump supervision. The
beam energy could not yet be inferred from the main bends at the moment, but a
preprogrammed energy function was used. The beam energy tracking system (BETS)
was working however, and the IPOC (internal post operational check) was active
too. XPOC was checking BI data and LBDS data after each beam dump.
After touching local permit loops in
point 6 and logging, Mike commented that the sequencer was heavily used, and being subjected to reliability tests, arming
exercises etc.
Mike next quickly went through the
summary and lists of remaining open
issues. Some of the outstanding items do not have highest priority. An
important open item is machine
protection. Its control functionality needs to be tested before the machine
protection test. Oliver asked who was responsible for the machine protection
test and commissioning. Mike responded that Alick and
Jan were in charge, adding that they were drawing up a procedure. The system
must be tested before it gets serious. Replying to a further question by
Oliver, Mike explained that there would be an MTF to sign off for the machine
protection.
Lastly Mike reviewed the status of
other components: RBAC, run control, LSA, DIP mechanism, dispatches, and instrumentation. Controls infrastructure was being deployed as
needed. Oliver asked what the term “roll out” was referring to. Mike answered
it meant that as items became available his team would pick them up and
integrate them in the applications software and commissioning programme. More than 50% of the beam loss monitors were
already available.
Oliver asked about the status of the
matching monitors, and the proposed gas injection and pressure-bump test. Jean-Jacques first clarified that the gas
injection would be needed for the BGI,
and not for the matching monitor (OTR), before answering that the gas injection
would be an MD, and that the vacuum group would be ready to participate in
this. Miguel Jimenez elaborated that the hardware existed and a pumping station
was available on the surface. The proposed test would require its installation
in the tunnel. At the moment this experiment did not have the highest priority
for the vacuum group and it would be deferred to June or July. In parallel to
work in the arcs, the vacuum experts were busy with work on ATLAS and CMS.
Oliver asked whether the commissioning of the pressure bump could be done in
parallel. Miguel said the parallelism was not the issue per se, but simply the
limited resources. Jean-Jacques commented that the situation was similar for
BI, where other instruments like wire scanners or BSRT had higher priority than
the BGI. Gianluigi asked whether one could test the
pressure bump in June prior to beam injection.
Jean-Jacques remarked that the RF commissioning would start on the right
side of IP4, and that there would be no access possibility during the RF tests.
Miguel commented that the test could perhaps be done outside normal working
hours. Jean-Jacques cautiously replied that this might depend on the need for
patrol. It would certainly be easier to do the pressure-bump experiment before
the start of RF tests. The best time was
just after the cooldown to 80 K. Oliver
emphasized that one should try not to miss the window of opportunity.
Mike finally reviewed the dry-runs
objectives for instrumentation and hardware, listed the dry-run activities that
had been executed so far, and summarized the plan for week 19, before
acknowledging all the people who contributed to the success.
Roger asked why the PGC of sector
2-3 was included as a simulation in addition to the real PGC in 5-6. Mike
responded that this was done to increase the scope of the dry run. Replying to
another question of Roger, Mike confirmed that the provisional list of dry runs
was posted on the web. He pointed out that details can be found at: http://wikis/display/LHCOP/Home . In
a final remark, Mike pointed out that the tests were likely to become more
difficult once the SPS was up and running.
Demo of Vacuum Displays & PVSS Tools (Isabelle
Laugier)
Isabelle presented the control
system for the LHC vacuum system, which is the same system as the ones already
employed for LEAR, LINAC2&3 and SPS&TL. Her presentation covered the
following points: architecture, SPS and transfer lines, LHC, main
functionalities, interlocks to cryogenics, demonstration, and conclusion.
Isabelle first described the low-level and high-level architectures of
the vacuum control system. The sector valve control units are directly
connected to a PLC master. There are intermediate PLCs for the gauges and for
some of the pumps. Each PLC master is in turn connected to the PVSS server. The
user interfaces in the CCC are clients of that server.
The main vacuum-system components
are pumps and gauges. Three types of
pumps are ion pumps (VPI), mobile pumping groups (VPGM), and NEG coating after
activation. Among the gauges, we find Piezo gauges
(PGM, only for the cryo vacuum), Pirani
gauges (VGR) measuring down to 1e-4 mbar, Penning gauges (VGP) that are active
between 1e-4 and 1e-9 mbar, and the most sensitive ion gauges (VGI) which cam
measure between 1e-7 and 1e-12 mbar. The latter are installed only in “warm
sectors”.
The vacuum in the transfer lines is
controlled from the SPS side. On the LHC
vacuum display four regions can be clearly identified: An outer circle representing
the QRL and cryogenics vacuum, two inner circles blue and red, representing the
beam vacuums for beam 1 and beam 2, the dump lines and the 4 experimental
areas. A window with the action history shows the name of the person who
initiated a change. There is also the possibility to enter a comment. Most
people will only have reading rights. The first LHC display example reflected
the present situation. On the following display the normal LHC situation was
presented. Green color for the inside of beam-vacuum ring means that all sector
valves are open. If a valve is closed some sector(s) would be displayed in red.
Various vacuum display options are
provided: The pressure profile can be displayed for the entire machine or for
part of the machine. The type of equipment to be displayed can be selected. The
full machine profile extends over 2 pages. Vacuum readings here are shown for
the 2 beams, again distinguished by red or blue color. It will also be possible
to select one beam only for display.
Miguel Jimenez commented that at low pressure only the readings from the
ionization gauges would be credible, while the readings of the other gauges
and pumps would simply correspond to their measurement resolutions. Stephane asked whether there was one ionization gauge per
sector. Miguel replied, yes, approximately one every 20 m. Frank asked for a
clarification that these gauges would not be available for cold sectors. Miguel
confirmed that indeed there were no ion gauges and therefore no low-pressure
information for the cold sectors.
A synoptic display shows the layout for a part of the machine and the
state of vacuum equipment in this part. Detail windows can be opened for each
type of equipment. Since, for example, it is impossible to close a valve is the
beam is present, this type of situation will also be indicated in the detail
window of a valve. Pressure and state histories are available as well. At the
moment it is planned to log the VGP penning gauge readings every 3 minutes.
Another panel with global actions application allows us, for example, to open
all valves in one sector, or to stop or restart all ion pumps.
There are two levels of “interlock” to the cryogenics system if
the insulation vacuum exceeds a threshold pressure of 0.1 mbar or 500 mbar,
respectively. These “interlocks” occur with a delay of 30 s to avoid spikes.
There exists the possibility to send an alarm to CCC at the same time as the
“interlock”. Gianluigi commented that such
pre-warning could be of interest for LHC operation. At the moment of the
interlock in any case the vacuum piquet will receive an SMS. Miguel stressed that this SMS message was not
an “interlock” in the usual sense; it rather resembled an “alarm”, on which the
cryogenics experts could act as they wanted. There was no need to enter into
the machine to repair. In addition, the cryogenic system was likely to
encounter some perturbation already below the 500 mbar pressure level.
Isabelle next presented the various icons, explaining the different
symbols and their meanings. The color code indicates the status of a device.
She then addressed the interface to the control system: The vacuum control is
publishing data to LHC logging database. It will send alarms to LASER.
Interlocks and gauges pressure will be published to CMW and DIP. The sector
valve chain is part of the BIC system. Coming soon will be the usage of Diamon for the vacuum PLCs.
Replying to a question by Roger,
Isabelle explained that “Diamon” was a successor of
the former “Xcluck” (also see DIAMON
paper by P. Charrue et al. presented at ICALEPCS 2007 Knoxville).
An online help is available directly
from the application. A software user
manual exists as EDMS
document no. 904272.
Finally, Isabelle described the
vacuum sector valve interlocks. Each
sector valve has 3 or 4 gauges or pumps to interlock. To open a valve all
equipment must be OK. The valve will close if all but one of the (3-4) nearby
gauges or pumps show poor vacuum or are not functioning, and if, in addition,
the beam is OFF. Miguel Jimenez commented that the temperature information is
also part of the interlock. This is necessary to prevent a contamination of the
NEG coating.
Isabelle finished with some upbeat
conclusions.
Jean-Jacques commented on the logging interval of 3 minutes. Isabelle
replied that indeed certain events are logged on change instead. Gianluigi mentioned his proposal to log more
frequently. He and Frank would prepare a
list of pressure readings selected for faster logging. Logging every 2-3 seconds should be possible.
Non-Conformities and Their Consequences (Massimo)
Massimo reviewed the non-conformities (NCs) for sectors 5-6 and
4-5, and the follow up of powering
strategies during hardware commissioning.
Information about the
non-conformities is available from the LHC HWC web site: http://hcc.web.cern.ch/hcc/nonconform.html
. The non-conformities can be split into two classes: the second class is
related to powering and magnet performance, which is more relevant to us.
Massimo then pointed out that the MTF
system allows closing a non-conformity with the status “decision pending”,
which was unexpected and complicates the interpretation. Stephan Russenschuck will report about ELQA non-conformities at the
LTC in 4 weeks time. Therefore the ELQA aspects were not covered in detail by
Massimo’s presentation.
The statistics for sector 5-6 is: 16 NCs were found. Of
these 13 are closed, but 8 of these with decisions pending. 3 NCs are still
open. Only 1 NC is related to powering. 1 remaining NC refers to a test of the
main dipoles MBs to estimate the time needed for 7 TeV
commissioning. In the NC table shown by Massimo, green or no color means
“closed”. Red color indicates “open” NCs, while orange means that an “action is
underway”. The two main remaining issues for sector 5-6 are the orbit corrector MCBCV10.L6.B1
and the triplet. The MCBCV10.L6.B1
problem is related to the protection. It is not clear whether this magnet can
be used. Extra tests will be performed. Based on the results of these tests
either a repairing action is taken or its current will be limited. The second
problem concerns the polarity of triplet quadrupoles
and correctors. An analysis was performed by S. Russenschuck,
and presented at the MPP on 14/04/08 (“Polarity
Issues at the Inner Triplets and DFBXs”). No easy fix is possible. The only
possible approach is at the level of the patch panel for the power converters.
In particular two main leads were erroneously exchanged, giving rise to a
potential worry about the QX1 trim. A solution will be proposed.
Frank S. asked what exactly the term
“repair” meant. Massimo replied this could either be a change of polarity, or a
decision to fix it. Stephane remarked that sometimes
a wire was cut. He asked what the corresponding disposition statement would be.
Massimo said the statement about the cut wire was correct, e.g. .sometimes we
are missing a decapole or sextupole
corrector as result, but this type of problem concerned a different sector.
Jean-Jacques commented that the disposition could be “use as is” in such case. Stephane remarked that this would not be strictly correct
because an action had been taken.
Following this LHCCWG meeting, Massimo looked for non-conformities concerning failing spool
pieces in other parts of the ring. Only sector 1-2 is affected, where this
type of failure occurred in two instances:
·
Dipole
2412 in half cell 32L2: The MCD failed the ELQA and it was by-passed as well as
the MCO. In EDMS the status of the NCs is “closed
with warnings”. The NCs are correctly reported in the
HWC web site: it is displayed with a colour-code corresponding to “action
underway” and with the disposition “decision pending”.
·
Dipole
1277 in half cell 19R: The MCS failed the ELQA and it was by-passed. In EDMS the
status of the NC is “closed with warnings”. Again the NCs
are correctly reported in the HWC web site: it is displayed with a colour-code
corresponding to “action underway” and disposition “decision pending”.
This means that the action taken (by-passing the spool piece) is not recorded in the
official documents! Massimo announced that he would talk to Stephan and Davide to clarify the situation.
Frank asked whether it had not been
stated during the 89th
LTC meeting that the QX1 trim would not be connected for this year’s run.
Massimo replied this was not the case, and according to his understanding the
QX1 trims would be made operational. Stephane
confirmed that a repair of the triplet error does not look possible. Only the
cabling at the power converter side can be changed. Discussion with Stephan Russenschuck is underway.
Next Massimo summarized the
non-conformities of sector 4-5. Here
35 NCs were found, 34 of which are closed. 19 of the latter are closed with
“decision pending”. Massimo pointed out that the access to the NC information
was forbidden. So he could not look at the details. Nevertheless he could find
out at least that there are three main issues: (1) BPM in Q22.R4.B1; here a repair was decided; Massimo interpreted
this to mean that the BPM will be available; (2) triplet quadrupoles (and correctors) polarity
error - the problem is similar to sector 5-6; (3) closed orbit correctors RCBYH6.R4B1 and
RCBYV6.R4B2 are reported as problematic, but their situation is not clear.
Feedback from Andrzej is awaited. Another corrector, RCBYH5.R4B1, had been
reported as problematic at the LTC
of 13/02/08 in a presentation
by R. Wolf (“Potential
Orbit Corrector Short in IR4”); in view of the importance of this
corrector, a repairing action had been launched and was already completed.
Special tests will assess the final status and verify the success of the
repair.
In the last part of his
presentation, Massimo followed up on the powering strategy. He recalled that
powering levels for the 80 A, 120 A, 600 A elements
had been proposed and presented by him at the 89th LTC on 23/04/08
(“Settings
for 5 TeV Beam Operation”). A few refinements
were made after the LTC. EDMS documents for powering tests in sectors 5-6 and
7-8 had been prepared by Walter Venturini, and were
now ready for formal approval. The main remaining issue was the parameters used
for the HWC of the various circuits and their compatibility with operation, in
particular the current-change speed dI/dt and the acceleration dI^2/dt^2. Massimo showed some
tables presented by Walter at a special meeting that was held to address this
issue, listing ramp rate, current, and acceleration during the HWC powering
tests. For different sets of circuits
the acceleration values had been reduced by a factor 3-10,
and the ramp rate had also been decreased. These reductions had been imposed by
the quench protection system. Now we needed to make sure that these parameters were compatible with normal LHC
operation.
Stephane commented that these reduced rates were probably not
compatible with operation. Ralph S. explained that in particular there was a risk that the feedback would cause spurious
quenches. Rates up to 50 times higher had originally been assumed for the
feedback correctors, and they would be possible with regard to the power
converters. Bernhard Holzer and Oliver pointed out
that a scaling with the maximum current in each magnet would need to be made
before comparing to the ramp rate of the main circuits. Massimo commented that
the acceleration looked more problematic than the ramp rate. The latter was
probably OK for the standard LHC cycle. Also Massimo highlighted that the
feedback system requirements needed to be checked carefully. A next topical
meeting to clarify this issue was foreseen in less than 2 weeks time. In
parallel the quench protection system was being improved by adding new filters.
Replying to a question by Mike,
Ralph S. explained that for correcting 1 unit of chromaticity at 7 TeV in the order of 1 A correction was typically needed. This
number scales with energy (it is a factor 15.5 lower for 450 GeV). For the time being, the power converters would use
their maximum rate (dI/dt|_max
= 5 A/s -> d^2I/dt^2|_max = 250 A/s^2) for the feedback, unless an action
was taken. Several options exist to avoid this problem, e.g. implementing a
rate limiter for the power converters, or introducing a limit on dI/dt in the feedback algorithm. Bernhard
Holzer remarked that the beam would not survive a
chromaticity change of 1 unit within a millisecond, adding that such changes
would need to be slowed down in any case.
In summary, as far as NCs were
concerned, only a few orbit correctors and triplet polarity issues had been
worth mentioning.
Roger asked whether the NCs were
communicated to anybody. Oliver remarked that the situation reminded him of the
as-built database. It was striking that
we do not even have access to all the information. Gianluigi
highlighted that we should check all the NCs, including the closed ones.
Massimo clarified that the MPP
was the correct body which should discuss all these problems. Walter is a
member of the MPP and serves as our link. Ralph S. suggested that a current
acceleration limit could be enforced by e.g. limiting the ramp rate to less
than 0.1 A/s, which would translate into an effective acceleration limit as
well or through an additional d^2I/dt^2 specific rate limiter directly inside the
FGCs. The acceleration for which the system is being commissioned is two orders
of magnitude off from what the power converters could do, but it is only a
factor of about 2 off from what is needed for chromaticity control.
Gianluigi noticed that NCs related to instrumentation, e.g. BPM
offsets, should also appear in list of NCs. Massimo stated that the complete
list had been shown, including instrumentation troubles. There was only one
instrumentation NC, in sector 4-5.
Needs
or wishes for commissioning the TOTEM Roman pots (Ernst Radermacher)