a versatile package to perform molecular dynamics for systems with hundreds to millions of particles.
GROMACS is primarily designed for biochemical molecules like proteins, lipids and nucleic acids that have a lot of complicated bonded interactions, but since GROMACS is extremely fast at calculating the nonbonded interactions (that usually dominate simulations) many groups are also using it for research on non-biological systems, e.g. polymers and fluid dynamics.
Read more on the GROMACS home page.
GROMACS automatically use any available GPUs. To achieve the best performance GROMACS uses both GPUs and CPUs in a reasonable balance.
The following versions have been installed:
|Modules for running on CPUs|
Modules for running on GPUs
*Release notes can be find here.
These modules can be loaded by using a module load command. Note that Intel MPI module file should be loaded first:
module load impi/2019.5 gromacs/2019.6
This provides access to the binary gmx_mpi wich can be used to run simulations with sub-commands as gmx_mpi mdrun
In order to run simulations MPI runner should be used:
mpirun gmx_mpi mdrun MDRUNARGUMENTS
In order to load the GPU enabled version (avaiable only on the bgn nodes):
Modules can be loaded by using a module load command. Note that the following module files should be loaded first:
module load gcc/11.3.0 intel/2023.0.0 cuda/11.8 gromacs/2023.0_tmpi
A simple case of a GROMACS job using a total of 640 CPU cores for 12 hours. The requested amount of cores in the example does not include all available cores on the allocated nodes. The job will execute 92 ranks on 3 nodes + 91 ranks on 4 nodes. You can use this example if you know the exact amount of required ranks you want to use.
#!/bin/bash #SBATCH -t 12:00:00 #SBATCH -p standard96 #SBATCH -n 640 export SLURM_CPU_BIND=none module load impi/2019.5 module load gromacs/2019.6 mpirun gmx_mpi mdrun MDRUNARGUMENTS
In case you want to use all cores on the allocated nodes, there are another options of the batch system to request the amount of nodes and number of tasks. The example below will result in running 672 ranks.
#!/bin/bash #SBATCH -t 12:00:00 #SBATCH -p standard96 #SBATCH -N 7 #SBATCH --tasks-per-node 96 export SLURM_CPU_BIND=none module load impi/2019.5 module load gromacs/2019.6 mpirun gmx_mpi mdrun MDRUNARGUMENTS
Following script using four thread-MPI ranks. One is dedicated to the long-range PME calculation. Using the -gputasks 0001 keyword: the first 3 threads offload their short-range non-bonded calculations to the GPU with ID 0, the 4th (PME) thread offloads its calculations to the GPU with ID 1.
#!/bin/bash #SBATCH --time=12:00:00 #SBATCH --partition=gpu-a100 #SBATCH --ntasks=72 export SLURM_CPU_BIND=none module load gcc/11.3.0 intel/2023.0.0 cuda/11.8 module load gromacs/2023.0_tmpi export GMX_GPU_DD_COMMS=true export GMX_GPU_PME_PP_COMMS=true OMP_NUM_THREADS=9 gmx mdrun -ntomp 9 -ntmpi 4 -nb gpu -pme gpu -npme 1 -gputasks 0001 OTHER MDRUNARGUMENTS
To setup a whole node GPU job use the -gputasks keyword.
#!/bin/bash #SBATCH --time=12:00:00 #SBATCH --partition=gpu-a100 #SBATCH --ntasks=72 export SLURM_CPU_BIND=none module load gcc/11.3.0 intel/2023.0.0 cuda/11.8 module load gromacs/2023.0 export GMX_GPU_DD_COMMS=true export GMX_GPU_PME_PP_COMMS=true OMP_NUM_THREADS=9 gmx mdrun -ntomp 9 -ntmpi 16 -gputasks 0000111122223333 MDRUNARGUMENTS
Note: Settings of the Thread-MPI ranks and OpenMP threads is for achieve optimal performance. The number of ranks should be a multiple of the number of sockets, and the number of cores per node should be a multiple of the number of threads per rank.
PLUMED is an open-source, community-developed library that provides a wide range of different methods, such as enhanced-sampling algorithms, free-energy methods and tools to analyze the vast amounts of data produced by molecular dynamics (MD) simulations. PLUMED works together with some of the most popular MD engines.
Gromacs/20XX.X-plumed modules are versions have been patched with PLUMED's modifications, and these versions are able to run meta-dynamics simulations.
GROMACS contains many tools that for analysing your results such as read trajectories (XTC, TNG or TRR format) as well as a coordinate file (GRO, PDB, TPR) and write plots in the XVG format. A list of commands with a short description can be find organised by topic at the official website.
VMD is a molecular visualization program for displaying, animating, and analyzing large biomolecular systems using 3-D graphics and built-in scripting, it is free of charge, and includes source code..
Python packages, MDAnalysis and MDTraj, can read and write trajectory- and coordinate-files of GROMACS and both have a variety of used analysis functions. Both packages integrate well with Python's data-science packages like NumPy, SciPy and Pandas, and with plotting libraries such as Matplotlib.
Your tpr file (portable binary run input file) contains your initial structure, molecular topology and all of the simulation parameters. Tpr files are portable can be copied from one computer to another one, but you should always use the same version of mdrun and grompp. Mdrun is able to use tpr files that have been created with an older version of grompp, but this can cause unexpected results in your simulation.
Simulations often take longer time than the maximum walltime. By using mdrun with -maxh command will tell the program the requested walltime and GROMACS will finishes the simulation when reaching 99% of the walltime. At this time, mdrun creates a new checkpoint file and properly close all output files. Using this method, the simulation can be easily restarted from this checkpoint file.
mpirun gmx_mpi mdrun MDRUNARGUMENTS -maxh 24
In order to restart a simulation from checkpoint file you can use the same mdrun command as the original simulation and adding -cpi filename.cpt where the filename is the name of your most recent checkpoint file.
mpirun gmx_mpi mdrun MDRUNARGUMENTS -cpi filename.cpt
More detailed information can be find here.
GROMACS prints information about statistics and performance at the end of the md.log file which usually also contains helpful tips to further improve the performance. The performance of the simulation is usually given in ns/day (number if nanoseconds of MD-trajectories simulated within a day).
More information about performance of the simulations and "how to imporve perfomance" can be find here.