Submitting Jobs to NERSC

Quick Description: Occasionally, for really large jobs, you may find yourself in need of more computing power than we have available to us on our home computers or even through our collaborations. You may have a better chance of running these jobs on government computers. In our lab, we traditionally use our account at NERSC, although there's no reason we couldn't use something else (and should probably look into XSEDE). Using NERSC works a bit differently than submitting a regular job, as described below, and more importantly, since these computers are shared, there's a typical 2 day wait before your queued job will actually run... And if you have multiple high memory jobs, you will also queue behind yourself. Explore the website to learn more.

The Point: Got a big job? Use a big computer.

Prerequisites: Same as for submitting a regular job. Obviously, you need access to the NERSC account (request by email for me to share the login credentials).

To manage the account and apply for more time, see NERSC Information Management: My Stuff > My ERCAP requests > Actions >ERCAP requests > Start New Request (refer to previous pdfs as guides for the new request)

Notes:

1. SSH into bartels@hopper.nersc.gov
2. "cd project" (and navigate to your folder...)
3. Submit using one of the following scripts:
• regular job "qsub ~/gscript"
• long job "qsub ~/gscript_long"
• medium memory job "qsub ~/gscript_med_mem"
• high memory job "qsub ~/gscript_high_mem"
• other possible scripts include: gqscript_double, qscript_quick, kscript, kscript_double, kscript_quick, kscript_long - I'm not really sure about the difference between these anymore...
4. Check your job using "qstat -u bartels"

Tips and Tricks:

Working with NERSC can be tricky, so it's useful to check out some of the known errors for our group. One common one that I'll mention here is that you may need to convert your files from DOS to UNIX format. To check a file format, run "file <filename>". To convert a file, run "dos2unix <filename>".

You can check your job in queue by navigating through the NERSC website: My NERSC > Queues and Scheduling > Queue Look

To copy to our computers: "scp <filename/folder name> bartels@pierce244-12.ucr.edu:/home/bartels/sharedData/<folder/fileName>"

For linux users (on your linux computer) to mount the nersc folder:
sshfs bartels@hopper.nersc.gov:/project/projectdirs/m1260 /home/<your nerscHome folder on your computer>