Life at CERN Hack Chat

Big science, big rewards

Wednesday, March 1, 2023 12:00 pm PST Local time zone:
Hack Chat
Similar projects worth following

Daniel Valuch will host the Hack Chat on Wednesday, March 1 at noon Pacific.

Time zones got you down? Try our handy time zone converter.


You know the story -- work is just ... work. The daily grind, the old salt mine, the place where you trade your time and talent for the money you need to do other stuff in the few hours you're not at work. It's not the same for everyone, of course, but chances are good that just getting through the day is a familiar enough experience even for someone who's currently working his or her dream job.

We're going to go out on a limb here a bit, but it really seems like working at CERN, the European Organization for Nuclear Research, has got to be a dream gig for almost any engineer. CERN is the top place in the world for particle physics research and home to such ludicrously large machines as the famous Large Hadron Collider (LHC). The facilities and instruments at CERN attract tens of thousands of researchers from all over the world every year who produce multiple petabytes of data; perhaps not coincidentally, it's also the place where Tim Berners Lee invented the World Wide Web. Thanks, Sir Tim!

To say that being an electrical engineer and going to work at CERN every day might be a little like dropping a kid off at a combination candy store/bouncy house/petting zoo is probably not an understatement. When the biggest of Big Science is always on the menu, it must be hard to focus on this cool project or that new instrument. Then again, we're just guessing -- maybe it's all still "just work." Luckily, we found someone to ask: Daniel Valuch, currently an electrical engineer who is rapidly closing in on 25 years at the fabled institution. You'll recall Daniel from some of his side projects, like the most accurate pendulum clock in the world, or his super-clicky pseudorandom number generator. He's also teaching at the university level, and we've seen him give back to the community with his work for the "ZENIT in Electronics" contest, an annual STEM event that's currently in its 39th year of inspiring students. Daniel is going to stop by the Hack Chat so we can pick his brain about what it's like to work at CERN, what kind of projects he's worked on, and what a career in Big Science is all about.

[Featured image: CERN, CC BY 4.0, via Wikimedia Commons]

  • Hack Chat Transcript, Part 3

    Dan Maloney03/01/2023 at 22:11 0 comments

    dallas1:18 PM

    dallas1:19 PM
    -1 = 50meters or more underground

    daniel valuch1:19 PM
    160 meters where at point 4 of LHC

    Dan Maloney1:19 PM
    And here I was thinking I was being overly enthusiastic with my descriptions of working at CERN

    April Morone1:20 PM
    @Dan Maloney Lol.

    daniel valuch1:20 PM
    ok, youtube

    dallas1:20 PM
    my ears popped when going down there

    daniel valuch1:20 PM
    they installed a counter in the new elevators :-)

    daniel valuch1:21 PM
    SPS is different. "only" tens of meters, up to 60 I think. But 24 storeys

    Thomas Shaddack1:21 PM
    ever thought about making it faster so you could experience zero-g?

    April Morone1:22 PM

    dallas1:22 PM
    they are very very fast

    Dan Maloney1:22 PM
    At least there's a bathroom right off the elevator

    April Morone1:22 PM

    daniel valuch1:22 PM
    @Thomas Shaddack funny enough, exactly the opposite. The elevators are not used very often, when the machine runs, there is no access for months. In this particular one, I got stuck about 15 seconds down every time I was called

    daniel valuch1:23 PM
    first time panic. Second time not again... Third time. ok let's wait 20 seconds, it will unstuck itself

    dallas1:23 PM
    and have to be triple redundant in their operation because huffing up the stairs in case of an emergency isn't really an option

    Thomas Shaddack1:23 PM
    what are the emergencies? helium/nitrogen leak from the magnets?

    daniel valuch1:24 PM
    actually CERN is the only place in the world, where you must use the elevator in case of fire. And you must not use the stairs. 160 meters of staircase is good 50 storey skyscraper

    daniel valuch1:24 PM
    can you run it up in one go?

    dallas1:24 PM
    isnt there a dedicated fire crew for the complex?

    caladan1:24 PM
    What kind of radiation protection do you use?

    Thomas Shaddack1:25 PM
    I'd guess "couple dozen meters of solid rock shielding"...?

    daniel valuch1:25 PM
    @Thomas Shaddack yes, sometimes there is a helium release from the cryogenic system. It is usually collected via the recovery line, valves unfreeze and we carry on. But sometimes there is bigger loss of helium into atmosphere

    dallas1:25 PM
    there's a fob everyone I saw wearing around their neck that measures exposure

    daniel valuch1:26 PM
    @caladan yes, everything is 100 meters underground. You need to shield the machine from the cosmic rays and the human activity too

    caladan1:26 PM
    And the protection for people who have to access the machine?

    dallas1:26 PM


    Read this on Google

    daniel valuch1:26 PM
    yes, dosimetry is everywhere. But the total absorbed dose for all people working there in a year is a small franction of an annual limit for one person. It is very well organized

    dallas1:27 PM
    that is an image of the observation deck in the CMS cavern, the beam tube is within the concrete blocks below

    daniel valuch1:28 PM
    there are multiple layers of protection. Biometry, mechanical barriers etc. Even if a door would be forced, the LHC will stop and by the time the offender would have a chance to get to the machine, energy from the magnets will be extracted

    daniel valuch1:28 PM
    it is not an easy task if the stored energy in one arc is about a GJ. Hude resistors

    twlostow1:29 PM
    Yeah, for those who saw "angels and demons" movie - there are indeed retina/iris scanners at the LHC/SPS access doors. Everything else in this movie is BS, but the scanners are true.

    dallas1:29 PM
    the door interlocks are something... there's an rf fob, iris scanners, etc.

    daniel valuch1:29 PM
    and switches of size of a truck. When you stand next to that switch, and it operates when you don't expect it, you may need to change you brown emergency underwear

    dallas1:30 PM
    do you have a picture of one of the switches?

    dallas1:30 PM

    daniel valuch1:30 PM
    let me find one

    daniel valuch1:31 PM

    daniel valuch1:32 PM

    daniel valuch1:32 PM

    daniel valuch1:32 PM

    daniel valuch1:32 PM
    those are only the switch elements, it is built into an enclosure...

    Read more »

  • Hack Chat Transcript, Part 2

    Dan Maloney03/01/2023 at 22:10 0 comments

    caladan12:39 PM
    I think it's C++ on CentOS 7

    daniel valuch12:39 PM
    actually, guess what it was

    Rosy Schechter12:39 PM
    Here are the libraries for data analysis:

    daniel valuch12:40 PM
    what was the mean time between failures when LHC started in 2008? The first days?

    April Morone12:40 PM
    I love C++ and Python.

    daniel valuch12:40 PM
    any guesses?

    Thomas Shaddack12:40 PM
    What about something where even C won't do? VHDL, Verilog, what are you using for FPGAs?

    Dan Maloney12:40 PM
    Two weeks?

    simon.dancose12:40 PM
    1 second?

    Christopher12:40 PM
    Three months?

    daniel valuch12:40 PM
    now, 15 years later, when everything is tuned and works well, we can get even 2 weeks

    Thomas Shaddack12:41 PM
    Also, how are you coping with the molasses-like nature of light? That thing is s-l-o-w. How do you compensate for that in the signal speed?

    daniel valuch12:41 PM
    but what it was then?

    dallas12:41 PM
    4 hours?

    daniel valuch12:41 PM
    about 5 minutes. Impressive for such a complex system. You can have MTBF of millions of hours. Plug it in million times and here we go

    Christopher12:41 PM

    daniel valuch12:42 PM
    5 minutes. By the first week of running, it was about 1 hour. People working days and nights. That was the best times I have ever had. Hacking

    daniel valuch12:42 PM
    hacking such a machine. I still get goose bumps thinking about it even now

    dallas12:42 PM
    very high caliber hacking

    April Morone12:42 PM

    daniel valuch12:43 PM
    now the big lady is running reliably and quitely. We work on many smaller machines and projects

    caladan12:44 PM
    oh, you call it big lady?

    Dan Maloney12:44 PM
    Isn't there a Really Big Lady in the works now?

    daniel valuch12:44 PM
    sometimes so reliably, that I start to miss the nigh calls. LHC did not call for 3 months. What is going on :-)

    simon.dancose12:44 PM
    How does the LHC behave when there's an earthquake in Europe?

    April Morone12:44 PM

    twlostow12:45 PM
    it's not afraid of earthquakes. but it shivers in fear when a leap second arrives :-)

    daniel valuch12:45 PM
    we are restarting the complex as we speak. The Linacs are already running, it will take about 1 week for each subsequent machine. LHC is foreseen to have first beam the week before the easter

    daniel valuch12:45 PM
    @simon.dancose yes, LHC can see all Earth movements

    Thomas Shaddack12:46 PM
    How do you compensate for them?

    daniel valuch12:46 PM
    The Moon, water level in the Geneva lake, and all the Earth quakes.

    Thomas Shaddack12:46 PM
    Or are they just seen in the data?

    daniel valuch12:47 PM
    The moon and water is slow. Out of 9000 magnets in LHC, maybe 7000 are correctors. The moon phases are even programmed in the control system as a real time feed forward correction

    daniel valuch12:47 PM
    aperture where beam circulates in LHC at high energy is about 1x1mm. Any movement is visible

    simon.dancose12:47 PM
    cool, I figured so. Moon cycle needed pre-planned compensating.

    daniel valuch12:48 PM
    first the beam is scraped at collimators, for for a bigh earthquake, like the one from Turkey which was visible even by my pendulum clock, that would be an instant beam dump

    Dan Maloney12:49 PM
    In case you missed it:

    Dan Maloney12:49 PM


    An Atomic Pendulum Clock Accurate Enough For CERN

    That big grandfather clock in the library might be an impressive piece of mechanical ingenuity, and an even better example of fine cabinetry, but we'd expect that the accuracy of a pendulum timepiece would be limited to a sizable fraction of a minute per day.

    Read this on Hackaday

    daniel valuch12:49 PM
    and indeed, there are sensors and accelerometers all over the place. For example the huge earthquake in Indonesia at the beginning of 2000's was already recorded by the Atlas sensors

    daniel valuch12:49 PM

    daniel valuch12:49 PM

    At the...

    Read more »

  • Hack Chat Transcript, Part 1

    Dan Maloney03/01/2023 at 22:09 0 comments

    daniel valuch11:13 AM
    check, 1, 2, 3...

    Dr. Cockroach11:34 AM
    @daniel valuch Welcome aboard

    Dan Maloney11:42 AM
    Hello Daniel! Glad you're here, we'll get started in about 20 minutes but feel free to chat while we wait

    daniel valuch11:43 AM
    Hi Dan, thank you for the invitation. I came home earlier, cleaned my desk, prepared the computer, ready for the show. Up to you if we go the Swiss way, or we start already :-)

    Dan Maloney11:44 AM
    I like to kick things off right at noon -- maybe I'm Swiss at heart?

    daniel valuch11:44 AM
    how many people usually participate in the hackchats?

    SimonAllen11:44 AM
    It looks like I am in the right place.

    Nicolas Tremblay joined  the room.11:45 AM

    Emanuel joined  the room.11:45 AM

    Dan Maloney11:45 AM
    It varies, there are about 20 people total logged on right now, I expect that will probably double by the time we're really into it. Could be more though

    daniel valuch11:46 AM
    hehe, yes, that is the Swiss way. I remember being at a conference in Luzern. Everything was prepared, people waiting for registration in an ordered cue. On the other side of the desk were couple of ladies from the institute, everything prepared, ready to go. We were waiting for good 10 minutes, looking at each other, until it was 9:00:00.000 They started to a millisecond

    Dan Maloney11:50 AM
    You never know what could happen in that last millisecond, I suppose

    twlostow joined  the room.11:50 AM

    daniel valuch11:50 AM
    well, you can get into a trouble here in nanoseconds...

    twlostow11:50 AM
    Hey Daniel :-)

    Nicolas Tremblay11:50 AM
    The Fat lady could sing ... or not

    daniel valuch11:51 AM
    hi Tom, I'm glad to have a support here :-)

    Dr. Cockroach11:53 AM
    Looking like an awesome crowd forming :-)

    caladan joined  the room.11:57 AM

    daniel valuch11:59 AM
    I have no webcam looking at my pendulum clock in the office. Ho how do we know it is time to start?

    Synthron11:59 AM
    looks like I just got here in time :)

    Nicolas Tremblay12:00 PM
    Waiting on Dan's cue

    Dan Maloney12:00 PM
    OK folks, let's get started. I'm Dan, I'll be modding along with Dusan today as we welcome Daniel Valuch to the Hack Chat! Daniel works at CERN -- enough said!

    Welcome Daniel, thanks for dropping by. Can you tell us a little about your background?

    twlostow12:00 PM
    NTP should be good enough for our timing needs :-)

    SimonAllen12:00 PM
    It is time.

    Thomas Shaddack12:00 PM
    GPS PPS with Chrony FTW.

    April Morone12:00 PM
    Daniel: By looking at your e-mail that notifies you when that time is that it starts.

    Thomas Shaddack12:00 PM
    Stratum-1 NTP on LAN.

    Dan Maloney12:00 PM
    Sorry, I'm a slow typist ;-(

    Dusan Petrovic12:00 PM
    Hi Dan, welcome everyone!

    daniel valuch12:01 PM
    thank you again for the invitation

    daniel valuch12:02 PM
    I have prepared a bit about the background. I am an electronics engineer, doing the electronics as a passion and as a profession for pretty much 35 years now

    daniel valuch12:02 PM
    I grew up in Slovakia, where I have studied electronics and electrical engineering since the childhood. Actually, I have started with chemistry, thanks to the family background. It was very interesting, but the destiny finally brought me to electronics.

    The secondary school was a clear choice. I have contiued my master's studies at Slovak University of Technology in Bratislava.

    Since the secondary school, I was working as a sound engineer, first in a local radio, then the national radio. Not a coincidence, my friend's father, who initiated me into electronics was head of R&D in the state owned company, making studio equipment for the whole Eastern block. I started to teach too.

    I slowly started to move from audio to high frequencies.

    daniel valuch12:02 PM
    And in the last year of studies, I have applied for a studentship at CERN

    daniel valuch12:03 PM
    At the time, CERN still invited even students for an in person interview. A young guy, who never left the country before, hired...

    Read more »

View all 3 event logs

Enjoy this event?



Interested in attending?

Become a member to follow this event or host your own