Hardware
There are currently three GPU nodes available on the Emmy system in Göttingen. These are similar to Skylake compute nodes, but equipped with four GPUs each:
- 2 x Intel Skylake Gold 6148 CPUs (40 cores per node)
- 768 GB memory
- ggpu01: 480 GB local storage on a solid state disk
- ggpu[02-03]: 2 TB local storage on a solid state disk
- 1 x Intel Omni-Path host fabric adapter
- 4 x NVIDIA Tesla V100 32GB
Getting Access
The node can be accessed using the gpu partition. Additionally you can specify, how many GPUs you need (by default you will get access to all) with the -G x
options, where x
is the number of GPUs you want to access. If you do not use MPI, please use the -c #cores
parameter to select the needed CPUs.
Example
the following command gives you access to 20 cores and two GPUs:
If you want to run multiple concurrent programs, each using one GPU, here is an example:
More explanation for the above example can be found here.
Software and Libraries
The CUDA and cuDNN modules are only available on the node. Please note that our current cuDNN version is only compatible with the CUDA 10.1 module, not with the older 9.2.
Additionally, the anaconda3/2019.03
python module has a TensorFlow environment tf-gpu
that is precompiled for GPU usage. You can use with
[nimboden@glogin1 ~]$ srun -p gpu -c 5 --pty -G 1 --interactive bash [nimboden@ggpu01 ~]$ module load anaconda3 Module for Anaconda3 2019.03 loaded. [nimboden@ggpu01 ~]$ source $CONDASH [nimboden@ggpu01 ~]$ conda activate tf-gpu (tf-gpu) [nimboden@ggpu01 ~]$ python tf_cnn_benchmarks.py [...]