Difference between revisions of "Code:SGE"

From Predictive Chemistry
Jump to: navigation, search
(Created page with "Here's a basic template for queuing a job using MPI (here 4 cpus and 6h max run-time). <source lang="bash"> #$ -N a-test-job #$ -cwd #$ -o job-$JOB_ID.out #$ -e job-$JOB_ID.err …")
 
Line 1: Line 1:
  +
== User environment ==
  +
  +
Zeroth, set up your own system (for linux), by adding a host def for circe to your '''$HOME/.ssh/config''' file. Then you don't have to keep typing in circe's full path and your username when running ssh from linux.
  +
  +
Host circe
  +
User <username on circe>
  +
Hostname circe.rc.usf.edu
  +
ServerAliveInterval 30
  +
ServerAliveCountMax 120
  +
ForwardX11 yes
  +
  +
On circe, first, setup your user environment by creating/editing a '''$HOME/.bashrc'''
  +
  +
<source lang="bash">
  +
# .bashrc
  +
  +
# Source global definitions
  +
if [ -f /etc/bashrc ]; then
  +
. /etc/bashrc
  +
fi
  +
  +
# User specific aliases and functions
  +
alias mystat="qstat -u $USER -t"
  +
</source>
  +
  +
Then create $HOME/.bash_profile, which is run for interactive, login sessions on circe.
  +
  +
<source lang="bash">
  +
# .bash_profile
  +
  +
# Get the aliases and functions
  +
if [ -f ~/.bashrc ]; then
  +
. ~/.bashrc
  +
fi
  +
  +
# User specific environment and startup programs
  +
  +
PATH=$PATH:$HOME/bin
  +
#module load compilers/intel/11.1.064 mpi/openmpi-1.4.1/intel-11.1.064
  +
module purge
  +
module load compilers/intel/11.1.064 mpi/openmpi-1.4.1/intel-11.1.064 apps/cuda/5.0.35
  +
  +
# apps/nwchem/6.1.1
  +
#module load apps/namd/2.7 apps/iso2mesh/0.8.0 apps/netcdf/4.2.0 apps/gaussian/03e01_em64t apps/gromacs/4.5.5a
  +
  +
export PATH
  +
</source>
  +
  +
== Queue ==
  +
 
Here's a basic template for queuing a job using MPI (here 4 cpus and 6h max run-time).
  
Here's a basic template for queuing a job using MPI (here 4 cpus and 6h max run-time).
   
Line 25: Line 75:
 
</source>
  
</source>
   
−
For execution and status, use '''qstat'''
 +
For execution and status, use '''qstat''', or the '''mystat''' command, defined as an alias in '''.bashrc'''.
   
 
For more info, see [https://cwa.rc.usf.edu/projects/research-computing/wiki/Guide_to_GridEngine the CIRCE SGE guide.]
  
For more info, see [https://cwa.rc.usf.edu/projects/research-computing/wiki/Guide_to_GridEngine the CIRCE SGE guide.]

Revision as of 22:52, 25 November 2013

User environment

Zeroth, set up your own system (for linux), by adding a host def for circe to your $HOME/.ssh/config file. Then you don't have to keep typing in circe's full path and your username when running ssh from linux. 

Host circe
  User <username on circe>
  Hostname circe.rc.usf.edu
  ServerAliveInterval 30
  ServerAliveCountMax 120
  ForwardX11 yes

On circe, first, setup your user environment by creating/editing a $HOME/.bashrc

<source lang="bash">

  1. .bashrc
  1. Source global definitions

if [ -f /etc/bashrc ]; then 

       . /etc/bashrc

fi

  1. User specific aliases and functions

alias mystat="qstat -u $USER -t" </source>

Then create $HOME/.bash_profile, which is run for interactive, login sessions on circe.

<source lang="bash">

  1. .bash_profile
  1. Get the aliases and functions

if [ -f ~/.bashrc ]; then 

       . ~/.bashrc

fi

  1. User specific environment and startup programs

PATH=$PATH:$HOME/bin

  1. module load compilers/intel/11.1.064 mpi/openmpi-1.4.1/intel-11.1.064

module purge module load compilers/intel/11.1.064 mpi/openmpi-1.4.1/intel-11.1.064 apps/cuda/5.0.35

  1. apps/nwchem/6.1.1
  2. module load apps/namd/2.7 apps/iso2mesh/0.8.0 apps/netcdf/4.2.0 apps/gaussian/03e01_em64t apps/gromacs/4.5.5a

export PATH </source>

Queue

Here's a basic template for queuing a job using MPI (here 4 cpus and 6h max run-time).

<source lang="bash">

  1. $ -N a-test-job
  2. $ -cwd
  3. $ -o job-$JOB_ID.out
  4. $ -e job-$JOB_ID.err
  5. $ -l pcpus=4,h_rt=06:00:00
  1. Example minimum memory request:
  2. $ -l mpj=10G
  3. Max run-time sets queue:
  4. queue max run-time
  5. devel 1 hour
  6. short 6 hours
  7. medium 2 day
  8. long 1 week

mpirun parallel-executable </source>

Submit with <source lang="bash"> qsub job.sh </source>

For execution and status, use qstat, or the mystat command, defined as an alias in .bashrc.

For more info, see the CIRCE SGE guide.

Here's a run-down on the environment variables available during running (for scripting): <source lang="bash">

  1. Environment Menu:
  2. $JOB_ID: The job number assigned by the scheduler to your job
  3. 1844896
  4. $JOBDIR: The directory your job is currently running in
  5. <blank>
  6. $USER: The username of the person currently running the job
  7. anyuser
  8. $TMPDIR: The directory containing informational files about the job e.g. the machines allocated to run the job, etc.
  9. /tmp/1844896.1.default
  10. $SGE_O_WORKDIR: Similar to $JOBDIR
  11. /home/a/anyuser/test
  12. $HOME: User's home directory on execution machine
  13. /home/d/anyuser
  14. $JOB_NAME: The job name specified by -N option
  15. a-test-job
  16. $HOSTNAME: Name of execution host
  17. wh-520-7-5.rc.usf.edu
  18. $TASK_ID: You can submit array jobs. This would be the TASK number in the array job of the current task
  19. <blank>
  20. $SGE_CELL: SGE cell that the job is currently running in
  21. default
  22. $SGE_O_SHELL: Invoking shell of the current SGE job
  23. /bin/bash
  24. $QUEUE: Current running queue
  25. default

</source>

Retrieved from "https://predictivestatmech.org/w/index.php?title=Code:SGE&oldid=115"