General info on EVE (EVE Wiki) is here
To submit a job use sbatch [script]
sbatch 01_download_data.sh
To check if a job is running use squeue -u [user]
squeue -u barratt
To keep an eye on your jobs constantly use watch squeue
watch squeue
To cancel a job use scancel [job_ID]
scancel 1234567890
You can request software installation on EVE here However, you can also install your own software on /global/apps/ like so by running the commands to cuild conda environments (e.g. Stacks 2.61 shown below)
cd /global/apps/
module purge
module load Anaconda3
conda create --override-channels --channel iuc --channel conda-forge --channel bioconda --channel defaults --prefix stacks_2.61 stacks=2.61
You can then load the software you installed like below (e.g. as part of your job submission script):
module purge
module load Anaconda3
source activate /global/apps/stacks_2.61
To get us set up all with files and directory structure that you’ll need, let’s do the following together in the EVE cluster…
First we will login to EVE, using either Terminal if you’re a mac user, or PuTTy if you’re a Windows user. You might need to make sure your VPN is connected to access.
# Open your Terminal (Mac) or PuTTy (Windows)
ssh USER@idiv-gateway.ufz.de
# [enter your password]
ssh frontend1
# [enter your password]
/public/
to your own workspace (will take about 17 mins)Now we’ll copy the directory which is all ready to your own workspace. Ideally it’s best practice to have data in a /data/
directory but this will get quite complicated with all of us, so let’s keep it simple and store in your own /work/$USER/
directories
# This line will copy the ddRAD-seq_workshop directory and all files and subdirectories from /public/ddRAD-seq_workshop/ to your own work space
cp -r /public/ddRAD-seq_workshop /work/$USER
And we will also edit the submit scripts so that they read and write from your own ddRAD-seq_workshop
directory. So copy and edit the following code in a text editor and then paste it into your Terminal/PuTTy. Thsi will replace all instance of $USER and %u with your username and all instances of YOUREMAIL@gmail.com with your email address:
cd /work/barratt/ddRAD-seq_workshop
sed -i 's/$USER/barratt/g' /work/barratt/ddRAD-seq_workshop/scripts/*.*
sed -i 's/%u/barratt/g' /work/barratt/ddRAD-seq_workshop/scripts/submit_scripts_stacks/*.*
sed -i 's/YOUREMAIL@gmail.com/christopher_david.barratt@idiv.de/g' /work/barratt/ddRAD-seq_workshop/scripts/*.*
sed -i 's/YOUREMAIL@gmail.com/christopher_david.barratt@idiv.de/g' /work/barratt/ddRAD-seq_workshop/scripts/submit_scripts_stacks/*.*
/work/$USER/ddRAD-seq_workshop
to /home/submit_scripts/ddRAD-seq_workshop
We do however want to make sure that our submit scripts are kept away in a separate folder. If you already have a folder for submitting scripts then you can move it there, otherwise I suggest you make one as follows:
# This code will make a ddRAD-seq_workshop folder inside a submit_scripts folder in your home directory (~/)
cd ~
mkdir /home/$USER/submit_scripts
mkdir /home/$USER/submit_scripts/ddRAD-seq_workshop
And then move the files as follows:
# This line will move scripts to your own submit_scripts directory
mv /work/$USER/ddRAD-seq_workshop/scripts /home/$USER/submit_scripts/ddRAD-seq_workshop/
OK, so now you should have a ddRAD-seq_workshop folder in your /work/$USER
directory and another one in your /home/$USER/submit_scripts
directory. You can check this by navigating around using cd /work/$USER
or cd /home/$USER/submit_scripts
and using ls
to view the subdirectories and files. You can move one directory up simply using cd ../
You can exit the EVE cluster now, and open RStudio in your computer. In the Console (bottom left panel usually), please copy and paste the following text, this will install all the packages you will need for the analyses (not so many for R). It might ask some questions on where to install from, you can say 1. All. If asked do you want to build this package from source you can say yes.
# This line will install the required packages
install.packages(c("adegenet", "LEA", "dplyr"))