Useful information


General EVE cluster job work

General info on EVE (EVE Wiki) is here

To submit a job use sbatch [script]

sbatch 01_download_data.sh


To check if a job is running use squeue -u [user]

squeue -u barratt


To keep an eye on your jobs constantly use watch squeue

watch squeue


To cancel a job use scancel [job_ID]

scancel 1234567890



Installing software on EVE

You can request software installation on EVE here However, you can also install your own software on /global/apps/ like so by running the commands to cuild conda environments (e.g. Stacks 2.61 shown below)

cd /global/apps/
module purge
module load Anaconda3
conda create --override-channels --channel iuc --channel conda-forge --channel bioconda --channel defaults --prefix stacks_2.61 stacks=2.61


You can then load the software you installed like below (e.g. as part of your job submission script):

module purge
module load Anaconda3
source activate /global/apps/stacks_2.61



Setup

To get us set up all with files and directory structure that you’ll need, let’s do the following together in the EVE cluster…


1. Login to the EVE cluster

First we will login to EVE, using either Terminal if you’re a mac user, or PuTTy if you’re a Windows user. You might need to make sure your VPN is connected to access.

# Open your Terminal (Mac) or PuTTy (Windows)
ssh USER@idiv-gateway.ufz.de
# [enter your password]
ssh frontend1
# [enter your password]



2. Copy the directory we provide in /public/ to your own workspace (will take about 17 mins)

Now we’ll copy the directory which is all ready to your own workspace. Ideally it’s best practice to have data in a /data/ directory but this will get quite complicated with all of us, so let’s keep it simple and store in your own /work/$USER/ directories

# This line will copy the ddRAD-seq_workshop directory and all files and subdirectories from /public/ddRAD-seq_workshop/ to your own work space
cp -r /public/ddRAD-seq_workshop /work/$USER


And we will also edit the submit scripts so that they read and write from your own ddRAD-seq_workshop directory. So copy and edit the following code in a text editor and then paste it into your Terminal/PuTTy. Thsi will replace all instance of $USER and %u with your username and all instances of with your email address:

cd /work/barratt/ddRAD-seq_workshop
sed -i 's/$USER/barratt/g' /work/barratt/ddRAD-seq_workshop/scripts/*.*
sed -i 's/%u/barratt/g' /work/barratt/ddRAD-seq_workshop/scripts/submit_scripts_stacks/*.*
sed -i 's/YOUREMAIL@gmail.com/christopher_david.barratt@idiv.de/g' /work/barratt/ddRAD-seq_workshop/scripts/*.*
sed -i 's/YOUREMAIL@gmail.com/christopher_david.barratt@idiv.de/g' /work/barratt/ddRAD-seq_workshop/scripts/submit_scripts_stacks/*.*



3. Move the subfolder ‘scripts’ from your /work/$USER/ddRAD-seq_workshop to /home/submit_scripts/ddRAD-seq_workshop

We do however want to make sure that our submit scripts are kept away in a separate folder. If you already have a folder for submitting scripts then you can move it there, otherwise I suggest you make one as follows:

# This code will make a ddRAD-seq_workshop folder inside a submit_scripts folder in your home directory (~/)
cd ~
mkdir /home/$USER/submit_scripts
mkdir /home/$USER/submit_scripts/ddRAD-seq_workshop


And then move the files as follows:

# This line will move scripts to your own submit_scripts directory
mv /work/$USER/ddRAD-seq_workshop/scripts /home/$USER/submit_scripts/ddRAD-seq_workshop/


OK, so now you should have a ddRAD-seq_workshop folder in your /work/$USER directory and another one in your /home/$USER/submit_scripts directory. You can check this by navigating around using cd /work/$USER or cd /home/$USER/submit_scripts and using ls to view the subdirectories and files. You can move one directory up simply using cd ../



4. Install packages needed in R

You can exit the EVE cluster now, and open RStudio in your computer. In the Console (bottom left panel usually), please copy and paste the following text, this will install all the packages you will need for the analyses (not so many for R). It might ask some questions on where to install from, you can say 1. All. If asked do you want to build this package from source you can say yes.

# This line will install the required packages
install.packages(c("adegenet", "LEA", "dplyr"))