:Author: Patrick Huck (phuck@lbl.gov)
:Date: March 2013

Running STAR Jobs on CARVER
===========================

Transfer files via Globus Online (GO) to $SCRATCH
-------------------------------------------------

- After registering on http://www.globusonline.org add your local computer's
  ssh-key via "My Account -> Manage Identities -> Add SSH Public Key".
- PDSF is already included as an endpoint in GO as nersc#pdsf. See the
  endpoint drop-down menu in "Start Transfer".

CARVER needs to be added manually via:

 $ ssh username@cli.globusonline.org endpoint-add carver -p gsiftp://carvergrid.nersc.gov

After that choose the both endpoints in "Start Transfer" and choose the
directory you want to transfer. GO keeps you updated via email in case you
need to renew credentials, i.e. re-login on both endpoints in "Start Transfer"
or extend the deadline for a copy task for one day:

 $ ssh username@cli.globusonline.org modify --deadline 1d idnumber

Avoid GROUP_DIR problem
-----------------------

[quote, Iwona Sakrejda]
As a person managing STAR resources at NERSC Jeff Porter can give a user access
through NIM or the users themselves should write to consult@nersc.gov.
Actually, the second option might be the right approach when the user already is in
the rhstar group, but it is not showing up on carver.

Additional STAR Libraries
-------------------------

TODO: how to install additional STAR libraries

Get root4star running on Carver
-------------------------------

Find a nice presentation about the NERSC file system at
http://www.nersc.gov/assets/Uploads/FileSys.pdf.

- copy dbServer.xml & cshrc.star (rename to .cshrc.star) to your $HOME
- copy /lib and /usr/lib to solve an issue with the libXpm library:
  (include the setenv command in your .login.ext for the future)

  cp -dR /lib <destination on $SCRATCH>/
  cp -dR /usr/lib <destination on $SCRATCH>/
  setenv LD_LIBRARY_PATH  ${LD_LIBRARY_PATH}:/<destination on $SCRATCH>/lib:/<destination on $SCRATCH>/usr/lib

- A general starting point on how to submit jobs on Carver can be found at
  http://www.nersc.gov/users/computational-systems/carver/running-jobs/batch-jobs/
- run.pbs is a template used to setup the pbs batch scripts for
  each job. Change starver to your desired Star Library version.
- runAll.sh loops over a file list, sets up the mpirun command for
  each file and appends it to a copy of run.pbs. The pbs batch script is then
  submitted via qsub in runAll.sh. The individual run.pbs scripts are copied to
  the folder script/. See the respective files for details. Adjust runAll.sh to
  your liking.