Quench of LHC inner triplet magnet causes a small leak with major consequences

https://home.cern/news/news/accelerators/accelerator-report-quench-lhc-inner-triplet-magnet-causes-small-leak-major

by untilted • 3 years ago

239 166 3 years ago

sakex3• 3 years ago

Do we know if this could have an impact on the local population? Also is there any research on the effect of the LHC on people living in Geneva and around?

jjgreen• 3 years ago

I think they'll be OK: https://www.hasthelargehadroncolliderdestroyedtheworldyet.co...

fabian2k2• 3 years ago

During such a quench, the liquid helium in the magnet warms up and turns into a gas that is recovered by the cryogenic system to be re-liquified, ready to cool down the magnets again.

They're capturing the helium, so it isn't released usually. Not sure what they do with the nitrogen, I'd guess that is released as it is much cheaper. But in any case the amounts involved in such a quench are only a danger in the immediate vicinity, not outside. So if you're in the same room as a quenching magnet and the room isn't sufficiently sized or ventilated, there could be danger. But that's about the limit to it.

Ekaros2• 3 years ago

Took a tour in local university physics lab dealing with helium. My takeaway is that liquid nitrogen is essentially free. And used to protect or keep helium parts colder. So anyway in process you have so much of it that simply having it boil away or be discarded is not big deal.

meindnoch• 3 years ago

Severe mutations and deformities.

DoctorOetker2• 3 years ago

Brain damage for the schedule programmer.

SiempreViernes2• 3 years ago

Uh, I guess some of the locals will get second-hand bummed-outedness from their acquaintances at CERN?

Did you even click on the link dude? The picture showing frost on the outside of a metal tube in a sealed tunnel ~100 m under ground is the total extent of the exterior effects...

harperlee• 3 years ago

It is a small leak of helium that disables a refrigeration unit.

cominous• 3 years ago

Read the article. It was a >local< event at one of the magnets which caused the whole magnet to freeze over. The "consequences" are, that the whole accelerator needs to halt operations to repair the element. That takes quite a while and will postpone the schedule..

sakex3• 3 years ago

Sure but is it impossible that something would leak when injecting massive amounts of energy in a gigantic tunnel?

defrost• 3 years ago

You only love your collider

https://www.youtube.com/watch?v=1e1eLe1ihT0

imdsm• 3 years ago

Do we know if... the magnet heating up, and the helium leaking into the vacuum chamber... could have an impact on the local population? I think they'll be ok!

contravariant2• 3 years ago

I mean worst that could happen is that it's harmful to someone standing right next to it if it fails catastrophically (no clue if that's even a realistic concern in this case). But since this happens over 50 meters below ground I'm not too worried.

Not that it was put that far below ground for safety reasons, it was mostly to save on the amount of land required and to shield the detectors from pesky background radiation (not the other way around).

gostsamo2• 3 years ago

Noble gases are not chemically active, but some interdementional rifts might allow access of alternative life forms to our reality.

rob74• 3 years ago

I think you're mixing things up - the incident you are referring to happened at the Black Mesa Research Facility, not at the LHC...

Aardwolf2• 3 years ago

Unbelievable the LHC is already running for 15 years! When it was just finished it was still the heyday of funny flash animations on the internet, some of course joking about the LHC ending the world:

https://www.newgrounds.com/portal/view/469806

sakex3• 3 years ago

Yes I think it's great, but still it would be good to have research to understand the effect on local population.

It is intellectually dishonest to just say it is not possible for anything to leak and have adverse effects on people without actively researching the matter.

zamalek2• 3 years ago

You never know, it might have been one of those time-travelling tachyons coming back to prevent its own creation (an actual conspiracy theory during the initial problems at LHC).

For anyone genuinely concerned about LHC, we've detected cosmic rays hitting our atmosphere at orders of magnitude more energy than what LHC is doing. https://en.wikipedia.org/wiki/Oh-My-God_particle

pif• 3 years ago

What do you propose exactly? What should we monitor?

By the way, I worked at CERN for a few years up to the first months of operations of LHC. Furthermore, I've been living in the region since then; for several years, my home was exactly over the LHC tunnel.

I've never got a nightmare from that!

SoftTalker2• 3 years ago

Not sure, but the engineers all came running out of the room talking like Mickey Mouse.

aredox• 3 years ago

I was there when there was a (much bigger) quench during commissioning of the LHC [0][1]. It was also at the focusing quadrupole modules - maybe the same ones?

I could make a joke about how it's Fermilab's vengeance for not having their own accelerator, but that would be dishonest: these focusing modules are the most complex ones of the whole ring.

[0]http://stephatcern.blogspot.com/2008/12/photos-of-lhc-damage... [1]https://home.cern/news/press-release/cern/cern-releases-anal...

notanote• 3 years ago

The 2008 quench happened at the other side of the ring. (Sector 3-4 then, sector 7-8 now.) The physics run was going very smoothly up to now [0]. Last year part of the machine (RF) had to be brought to room temperature as well, after a cooling tower fault, and there was no beam for four weeks. I get the impression that this will take longer, but I hope not by much.

[0]https://atlas.web.cern.ch/Atlas/GROUPS/DATAPREPARATION/Publi...

SiempreViernes2• 3 years ago

Man, so relieved it's just cooling one section that is meant by "major consequences", title made me think of the explosion during commissioning. Looks like it's only sector S78 that needs attention: https://op-webtools.web.cern.ch/vistar/vistars.php?usr=LHC2

This report is impressively detailed for being put out just two days of the incident, but I guess LHC having world class technical diagnostics teams is not too surprising.

bayindirh• 3 years ago

When I was visiting CERN, the whole accelerator was in maintenance mode. We have seen a detector (IIRC it was ATLAS) which was disassembled in layers, so we were able to see almost all of it.

Our guide explained that whole schedule for maintenance and operation is set in stone and whole schedule is planned almost to hour resolution for the next six months or so. Also it's worth noting that, there is periodic maintenance that needs to be done, and that's also planned in advance.

Because of this stringent requirements, "several weeks" of shift is indeed a major problem, and by several I guess they are looking to ~8 weeks, if not more.

FireBeyond• 3 years ago

My uncle, who is not a physicist, but "one smart mf", was one of the heads of the beam instrumentation team at Fermilab, and spent a large part of his career designing, building and maintaining detectors. It utterly blows my mind.

vasco• 3 years ago

The "major consequences" seem to be for the schedule:

This incident will probably have a great impact on the LHC schedule, with machine operation unlikely to resume for at least several weeks.

I also read it instantly in case they had a black hole on their hands, even though I'd expect us all to be dead almost instantly if any real "major consequence" event truly happened in the accelerators.

3cats-in-a-coat2• 3 years ago

You can't create a black hole in LHC. Although sometimes when I read social media I wish they could.

dukwon• 3 years ago

"at least several weeks" is technically correct but I think a bit of an understatement. A sector takes about 3-4 weeks to warm up and 4-5 weeks to cool down. Then it needs to undergo powering tests and probably some "training" quenches.

Since the winter shutdown is scheduled for end of October (thanks to the energy crisis), there's a good chance we are finished with proton collisions for the year. If the leak can get fixed and the sector cooled down by mid/late September, we might have time for the heavy ion run. Last year's ion run was cancelled due to shortening the year, so 2 years in a row would not be great.

dark-star• 3 years ago

They would have written "Unforeseen Consequences" in that case ;-)

(https://developer.valvesoftware.com/wiki/Unforeseen_Conseque...)

krisoft• 3 years ago

I also read it instantly in case they had a black hole on their hands, even though I'd expect us all to be dead almost instantly if any real "major consequence" event truly happened in the accelerators.

Idk why people think this. Black holes are not magic. They have exactly as much gravity as the mass had which collapsed into them.

As you go about your life right now the gravity of Switzerland is pulling you towards it a tiny bit. If due to some crazy event the mass of the whole country would collapse into a black hole that black hole would still tug you the same way and the same amount.

bauruine• 3 years ago

I'd expect us all to be dead almost instantly

Not an expert at all but from what I've read you'd expect wrong. Even if they could create a black hole out of the Matterhorn it would be so small, less than a hydrogen atom, that it doesn't interact much with anything.

"Even though a microscopic black hole might contain the mass of a mountain, it would experience almost no friction as it passed through regular matter. It would fall through regular material as if it wasn’t there." [0]

So the worst thing would be the missing Matterhorn which pretty sure would upset the Swiss.

[0] https://www.universetoday.com/1930/are-microscopic-black-hol...

nonameiguess• 3 years ago

There was some hope at one point that something like the LHC might produce black holes, but not the type you're thinking of as massive gravity wells that form when stars collapse. It has been postulated that the early universe in the immediate aftermath of the Big Bang may have contained primordial black holes, and any that were large enough to still exist today might be a possible source of dark matter. Any formed in a particle collider would be particle mass, not star mass, and have the gravitational attraction of particles, and evaporate almost instantly due to Hawking radiation. They'd have been harmless, but if it had been possible, it would at least confirm they actually can exist and observing the decay would have been our only realistic hope of observing Hawking radiation likely at least in this century.

tux3• 3 years ago

Vistar is so cool :)

Also, here's a public grafana I just found with temperature charts in S78 and a bunch of other cryo data: https://dash.web.cern.ch/d/iDRuWWHGz/sector-78-trend

Not that I can do anything very interesting with that particular bit of information, but I'm always happy to see open data. We could really use more of that in public science, and CERN is really doing a great job

ggm• 3 years ago

The inner tube is hard vacuum? Does freeze over risk rupture and also incur re evacuation and time to get those last pesky digits of nothing?

fifteen1506• 3 years ago

For major click-bait for Half-Life 1 players, I propose the title to be prefixed with "UNFORESEEN CONSEQUENCES: ".

Gordonjcp• 3 years ago

I never thought I'd see a resonance cascade...

misterflibble• 3 years ago

Gordon! Get away from the beam!!!!

ComodoHacker• 3 years ago

This report really has disturbingly strong Half-Life 1 vibes for me.

FeepingCreature• 3 years ago

"It's not... it's not shutting down! Aah-- oh wait, it shut down. All good."

Aardwolf2• 3 years ago

The major consequence here is a few weeks of repair time, there wouldn't be much gameplay in Half-Life 1 if that were it :)

fancyfredbot2• 3 years ago

It is amazing that the beam was "dumped" just milliseconds before the magnet quenched! If that hadn't happened then the beam could have crashed into the magnet and caused a lot more damage.

Dumping the beam means diverting it out of the main LHC ring and crashing it into a specially designed buffer (I think it is a lump of steel or something). So cool to see this all happening automatically.

zelos• 3 years ago

It hadn't occurred to me that there would be significant energy contained in a beam of sub-atomic particles. Something like 100kg TNT equivalent (~500MJ)? Wow.

testtestabcdef• 3 years ago

That's crazy. I never would've thought that either. These small facts which you can barely read in "mainstream media" are what makes the LHC absolutely fascinating and impressive.

I was always like "Yeah that's just a small beam and the magnets are to navigate it in circles". In retrospect it does make sense now why people were concerned about the collision of such beams.

rcxdude• 3 years ago

Yeah, it has the energy of a semi truck going at a few hundred mph but in something that is about 1nanogram of mass (about that of the average human cell).

fancyfredbot2• 3 years ago

There's a great article on it here:

https://www.symmetrymagazine.org/article/parking-the-lhc-pro...

"Enough stored energy to melt a ton of copper"

"The beam dump is a solid graphite cylinder 8 meters long and under a meter in diameter. It’s wrapped in a stainless-steel case, filled with nitrogen gas, and surrounded by iron and concrete shielding."

tuardoui• 3 years ago

If like me, you want to see what it looks like : https://home.cern/news/news/accelerators/autopsy-lhc-beam-du...

jacquesm• 3 years ago

The amount of materials science knowledge so casually on display is amazing.

myself248• 3 years ago

So now I'm reading:

https://iopscience.iop.org/article/10.1088/1748-0221/16/11/P...

And I want microphones, I want to hear the thing ring with that 2.3kHz note. I want to feel the 27Hz wiggle and the 196Hz thump. I want to get the Slow-Mo guys in there to place their camera, and watch the thing jump when a beam hits it.

The amount of energy in that thing just defies intuitive understanding from reading a paper, I have to use other senses.

s0rce2• 3 years ago

According to the link below its an 8m long steel tube filled with various types of graphite.

unixhero• 3 years ago

Unforeseen consequences

Don't forget your crowbar

ur-whale• 3 years ago

"Major consequences", and here I was already think a beam of protons flying at near the speed of light was let loose towards the city of Geneva.

But no, just a section of the accelerator that needs fixing.

codeulike• 3 years ago

Quench

Nice verb

ThePhysicist3• 3 years ago

Helium leaks are a nightmare. During my PhD I worked with a self-built dilution cryostat that would often have leaks in the custom-built Indium seals. To find the leaks you'd have to pump the cryostat to high vacuum, hook up a portable mass spectrometer tuned to Helium to the pump circuit and then spray different parts of the cryostat with Helium from a regular gas canister. The He would then get sucked in through the leak and show up on the mass spectrometer, which was coupled to a loudspeaker so it would cause a sound whose pitch increased with the measured He density. Once you found the leak you had to vent the entire system, remove the faulty seal and replace it with a new one. All seals were handmade, i.e. you took a small filament of Indium, placed it between the flange and the housing (after carefully cleaning everything with Acetone) and carefully screwed it shut, turning each screw only a tiny bit at each turn and going around all the screws until you could see the Indium squeeze out of the edges.

Even worse, some leaks would only show up when the system got cooled down to liquid Helium temperature. When that happened you were out of luck as you can't cool the system down to 4K before spraying it with Helium, so you had to just guess where the leak might be and replace all seals in that area until you found the right one. Going even deeper in temperature would eventually turn the He4 suprafluid, which means that it loses all internal friction. In that state it would squeeze through even the tiniest molecular cracks, so again if that happened you just had to redo all the seals and hope they would hold.

simonjgreen• 3 years ago

That's sounds horrific. It makes me wonder what proportion of time you'd spend massaging and debugging your lab equipment vs. actually using it?

gaze2• 3 years ago

For an experimental physicist you basically debug until you get the data for your paper or until you find your sample is bad. So you’re debugging 99% of the time.

In many fields it’s rare to build new instrumentation. You fab with well known techniques with one subtle modification, so it’s not even clear what part of your time isn’t spent debugging.

causality0• 3 years ago

It's like a high-end fighter jet. An F-22 needs 43 man-hours of maintenance per flight hour.

gus_massa5• 3 years ago

And that does not include the problems with electric wires. We had an unofficial rule: after half an hour debugging the electric wires just change all of them. I still have nightmares about BNC connectors.

ThePhysicist3• 3 years ago

Basically 2-3 years of getting everything to work and testing each component, than 2-4 weeks of good measurements that comprise the main results of the PhD thesis.

NotYourLawyer2• 3 years ago

If you’ve got an apparatus with 20 parts (a low estimate) and they’re all independently 95% reliable (a high estimate), it’s only gonna be in a usable state 36% of the time.

user3939382• 3 years ago

This is a more extreme description of why I hate plumbing.

nazgulsenpai2• 3 years ago

As someone who is currently in a game of cat and mouse with an intermittent drip from underneath the kitchen sink, this was exactly the first thing that came to my mind also!

pjc50• 3 years ago

Nuclear reactors offer all the fun of leak tracing and invisible cracks with the added bonus that the fluids involved are extremely radioactive and contaminate everything they touch. This is why all the "just mass produce small reactors" and "just try thorium / molten salt" stuff hasn't taken off, and may never: all the beautiful theory disintegrates into man-years of laborious leak testing.

allenrb• 3 years ago

Seriously. It is annoying when some ancient pipe in my radiator plumbing decides to leak. This sounds orders of magnitude worse.

The things we humans put up with to try and understand the universe!

arbuge• 3 years ago

...and central air conditioning.

2 evaporator coil leaks in my 5 year old house already, another one currently suspected.

londons_explore2• 3 years ago

One would imagine there must be some kind of paint you can paint onto the outside of a vacuum flask, and have it be sucked into any gaps and solidify there.

It seems they already make a product for that - vacuum grease. It can get sucked into even a fairly large crack, and plug it up, as long as the depth of the crack is much longer than the width, then atmospheric pressure won't provide enough force to send it deeper into the crack.

And the grease itself is designed not to evaporate into the vacuum.

Whats wrong with using that?

remus• 3 years ago

Just speculating, but helium molecules are pretty small, so will be able to leak through tiny gaps where the paint/grease would not penetrate.

beowulfey• 3 years ago

Cryogenic temperatures maybe?

hwillis• 3 years ago

paint onto the outside of a vacuum flask, and have it be sucked into any gaps and solidify there.

Stuff like paint will tend to be very permeable- if it relies on a solvent drying out, then obviously its permeable enough to let the solvent through. Even if it does dry to a very solid state, there will be a very very long tail of evaporation of volatiles inside the substance.

It seems they already make a product for that - vacuum grease.

Vacuum grease isn't perfect- it's not recommended for the high end of high vacuum or for UHV. Or for single digits of kelvin.

Another big factor is that if you're doing science, you probably want to go out of your way to eliminate contamination like that. Guessing about the possible chemical interactions of a secret proprietary substance under extremely low temperature and pressure is not a fun way to get your doctorate.

ThePhysicist3• 3 years ago

Not sure if there's vacuum grease that can work at very low temperatures, the upper stage of the cryostat would go to 300 mK and the lower stage to 20 mK, at that temperature non-metallic compounds tend to get porous. There are materials that can work in there but to my knowledge nothing that was an easy solution, at least at the time.

ISL• 3 years ago

Those materials exist (see Apiezon vacuum grease and Lesker vacuum sealant). In less-demanding or desperate applications, they can work.

But like similar last-ditch repairs (think automobile muffler tape or radiator sealant), they only occasionally last. Usually the best thing to do is to go actually fix the problem.

The above advice assumes you're trying to maintain high vacuum (10^-5 torr or lower pressure). At low vacuum (10^-3 torr and above), that trickery can work wonders.

isoprophlex• 3 years ago

For a schlenk line in chemistry where you need 0.1 - 1 mbar, sure! But this is insane physics territory... Like other people said, grease isn't behaving like you think it would at millikelvin temperatures, or at pressures lower than those in the interstellar medium.

If you have a big aluminum chamber under ultra low pressure, guess what you're concentrating in there? Hydrogen and helium from the atmosphere that's seeping through your metal. Crazy stuff that's (sadly) not fixed by slapping PTFE paste or vacuum grease onto it.

s0rce2• 3 years ago

There are some products but in systems with metal seals you don't use grease/sealant as as they off-gas various contaminating small molecules

https://www.tedpella.com/vacuum_html/High_Vacuum_Leak_Sealan...

phkahler• 3 years ago

Maybe we should "just" build this in space. ;-)

wiml• 3 years ago

Space is far, far too hot at 3 Kelvin, and most of it is too high-pressure, too.

COGlory• 3 years ago

This sounds miserable!

javajosh• 3 years ago

I think I'll bookmark this comment and come back to it the next time I feel annoyed by tracking down a software bug.

YakBizzarro• 3 years ago

Oh yes, way too familiar situation... dry cryostats were really a game changer. Apart of the incredible advantage of not having to refill them frequently and on Sunday, they are much more reliable and without any cold seal

jerf• 3 years ago

AlphaPheonix on YouTube has an excellent and informative video of a very similar process of testing seals, though the vacuum is pulled for a different reason: https://www.youtube.com/watch?v=VD69crOFx10

throwaway290• 3 years ago

Based on you description... Couldn't you just go so cold He goes superfluid so you find all the leaks right up?

dreamcompiler• 3 years ago

Not my area, but I know He is very expensive (as is getting it cold enough to make He4) so using He4 as a leak detector is likely going to blow the entire experiment's budget.

willis936• 3 years ago

A good friend of mine made a leak checker from scrap parts while I was working at a university lab. It was just for a UHV system (no cryo). I wish I still worked with him.

The whizz sound leak checkers make is good fun.

NotYourLawyer2• 3 years ago

Ha, I also remember using a helium canister to look for leaks in a vacuum system in the lab. At least ours wasn’t at cryo temperatures though. Sounds like a real pain.

latchkey• 3 years ago

Could you hear the indium scream as it was being squeezed out?

https://youtu.be/mjiP5Q6g_aM?t=153

mensetmanusman• 3 years ago

Was it ever an option to cycle the system with temporal atomic layer deposition chemistry to slowly build up aluminum oxide barriers?

throwawaymobule2• 3 years ago

Helium leaks are especially annoying, because they'll brick most modern electronics for a day or so. iPhones especially.

If they have a PCB oscillator, the gas will seep its way in there and cause a clock failure.

formerly_proven• 3 years ago

This is a very nicely written, clear article. This is what good science communication looks like.

CrampusDestrus• 3 years ago

you mean "consequenches"

dr_faustus• 3 years ago

Just reading that gives you an inkling of an idea what kind of amazing feat of engineering the LHC is. I really hope governments keep funding basic research like that because it teaches us not only about nature but also how to built those incredible machines.

noobermin• 3 years ago

Bad time to be an LHC grad student.

enslavedrobot• 3 years ago

Quenching a magnet is the result of poor planning and poor maintenance. The only time I ever hear about magnets quenching is at CERN. I remember they quenched ~40 at once when they first turned it on, a multi-million dollar error. What the hell are they up to over there!?

trenchgun• 3 years ago

Tiny black holes that are going to swallow the earth in the coming hours?

lhoff• 3 years ago

Partially related recommendation for the german-speaking crowd here:

The Raumzeit podcast released the second episode of a longer series about the cern. In the frist one he discussed the history and success of the cern with the leader of the experimental physics department and in the second one he talks to the guy who is in charge of operations of the proton synchrotron about the actual accelerators.

https://raumzeit-podcast.de/2023/07/05/rz111-cern-geschichte...

https://raumzeit-podcast.de/2023/07/19/rz112-cern-die-beschl...

Animats• 3 years ago

Not too bad. The 2008 magnet failure took two years to repair, with over 50 magnets affected.

Quench of LHC inner triplet magnet causes a small leak with major consequences

Related Stories