How to use computing and Farm Altas Energias

Conectar a hep002

Para conectarse la primera vez a hep002 sigue estos pasos. Abrir 2 ventanas, una para trabajo local, otra para remota

conectarse a dec1:
  • ssh username@dec1.ifisica.uaslp.mx username es su nombre de usario para dec1
  • cd ~/.ssh (si no existe, crear el directorio)
  • ssh-keygen -t rsa (presionar "enter" cada vez que pregunta algo)

    • crear una clave de ssh en su computo de casa:
  • cd ~/.ssh (si no existe, crear el directorio)
  • ssh-keygen -t rsa (presionar "enter" cada vez que pregunta algo)
  • copiar el archivo id_rsa.pub a dec1: scp -p id_rsa.pub username@dec1.ifisica.uaslp.mx:

    • en la ventana de dec1
  • cat ../id_pub.rsh id_pub.rsh >> authorized_keys
  • Solo si tienes 2 username's diferente: scp -p authorized_keys id_rsa* username2@hep002: donde username2 es el de hep002
  • Si tienes 2 usernames, usa aqui el 2.. Si solo tienes uno, usa el unico: ssh username[2]@hep002

    • ya en hep002
  • cd ~/.ssh (si no existe, crear el directorio)
  • cp -p ../authorized_keys ../id_rsa* .
  • Incluir las siguentes lineas al archivo config (en este directorio, si no existe crearlo)
    Host *.ifisica.uaslp.mx
    ServerAliveInterval 60
    ServerAliveCountMax 9999999999

    • Ahora esta todo lista para conectarse directamente a hep002 cada vez sin que se piden passwd

    para conectarse cada vez desde la compu de su casa:
  • ssh -X -N -f -L 2223:hep002:22 username@dec1.ifisica.uaslp.mx; sleep 5; ssh -p 2223 -X username[2]@localhost

    • Una vez verifcada que todo funciona, que puede definir un comando nuevo (hep002) para conectarse mas facil. Se debe incluir la siguiente linea en su archivo .bashrc en la computadora de su case, y se activa cada vez que abre una ventana nueva
  • alias hep002='ssh -X -N -f -L 2223:hep002:22 username@dec1.ifisica.uaslp.mx; sleep 5; ssh -p 2223 -X username[2]@localhost'

    • Crear un directorio de trabajo de NA62 en hep002

    por cada version del software de analisis se debe crear un directorio de trabajo. Este solo es necesario a veces, Como ejemplo usamos la version v3.1.2

    en la ventana de hep002
  • cd /cvmfs/na62.cern.ch/offline/NA62FW/prod/v3.1.2/NA62Analysis
  • . ./scripts/env.sh
  • ./NA62AnalysisBuilder prepare ~/MyAnalysis-v3.1.2
  • cd ~/MyAnalysis-v3.1.2
  • . ./scripts/env.sh
  • copiar los .cc y .hh a Analyzers/src/ y include/
  • copiar los config-xxx para compilar su programa

    • Compilar y correr programas de Analysis de NA62

    en el directorio de trabaja una vez por cada sesion nueva
  • . ./scripts/env.sh

    • para complilar un programa
  • NA62AnalysisBuilder cofig-xxx

    • Correr un programa sobre un archivos de datos/mc
  • ./bin-cc7/programa -i inputfile -o outputfile.root

    • Correr un programa sobre muchas archivos, y que continua correr aun que se desconecta la ventana
  • ls inputfile*.root > listfile
  • ./bin-cc7/programa -l listfile -o outputfile.root >& logfile.log </dev/null &

    • Correr programas en paralelos, sobre muchas archivos, y que continua correr aun que se desconecta la ventana
  • ls inputfiles17*.root > listfile17
  • ls inputfiles18*.root > listfile18
  • ./bin-cc7/programa -l listfile17 -o outputfile17.root >& logfile17.log </dev/null &
  • ./bin-cc7/programa -l listfile18 -o outputfile18.root >& logfile18.log </dev/null &
  • Los 2 programas corren en paralelo. Ojo! No correr mas que 5 en paralelo!
  • Una vez terminada todo: hadd outputall.root outputfile17.root outputfile18.root

    • Copiar archivos entre su casa y hep002

    archive de hep002 a su casa:
  • en su casa: scp -p -P 2223 localhost:outfile.root .

    • archvo de casa a hep002
  • en su casa: scp -p -P 2223 program.cc localhost:

    • Batch System Farm Altas Energias

    Introduction

    The Farm available for the High Energy Group IF-UASLP consists of several types of computers. hep00 and hep002 are the main nodes, with disks and all software. 3 dual processor machines hep03 and hep04 have curently CC7 installed (for work on NA62 with hep002, with a total of 69 slots) and hep11, hep12 hep13, and hep14 are still on SL5 (together with hep00) for work on SELEX, with a total of 16 slots. All are located in the Computer center and can only be accessed via the batch system.

    The batch system to distribute jobs to the different machines is called Fermilab Batch System New Generation (fbsng). Below we will show some examples how to use it. After a setup fbsng on whatever machine, you can see which nodes are up with fbs nodes, and the queues existing with fbs queues. To submit jobs, use fbs exec. A full user manual can be accessed here, and there is a fbsng web site at Fermilab.

    In general you need a script which will do the work for every job. The batch system will create two files in you home directory (*.err and *.out) containing the stdout and stderr of you job. It will also (optional) send mail to you when a job is finished.

    In the following we will show three examples on how to use the farm in SLP.

    NA62 Example - Running over all files of 2018E

    As first example we will run Penta over the 58 files in 2018E, all in parallel. I wrote a script, run_2018E, to run Penta on one file. To facilitate the submission of 58 jobs, I wrote another script, do_2018E. Both scripts should be in your "MyAnalysis" directory. To submit all jobs, just type "./do_2018E". After some time, 58 .root files will appear in that directory. You have to use "hadd" to sum them together.

    SELEX Example - Running over the 1Million events

    As first example we will run soap over the 33 files, which makes up the one million event sample of run 10783. I wrote a script to run soap, which does all the work. Note that the input filename is variable, so you can use it for all files. Obviously you have to adapt the script to your need, and you have to create to own soap.cmd and recdf.ocs files, and maybe your own soap.exe. To submit 33 jobs in one go, you say (with a little bash trick)
    for i in `ls /usr/people/ehep1/e781/1M/*`; do fbs exec -q long -m jurgen `pwd`/soap_example.sh $i; done
    and you wait. A full run will take about 5 hours.

    CKM Example - Simulating the Pion RICH Resolution

    For this example I wrote 2 scripts. One, as before, will do the work itself, and can be seen here. The other is to submit the jobs. We make a loop over momenta between 13 and 22 GeV, for both pion and muon in the pion rich. Each job takes up to 30min to run.

    Last update at April 4, 2021 by Jurgen Engelfried