How to use GPUs

If you think your work would benefit from use of GPUs, there are several options available to you:

First, consider whether you need to use GPUs, or whether CPUs might be a better option. Please read the following guidelines for choosing between CPU and GPU (h/t Aaron Zhao).

GPUs contain a large number of processing units, and are efficient in handling massively parallel programs. In this case, if the task is too simple or too sequential, it is usually hard to achieve time-savings on GPU. For instance, tasks like image processing are likely to be embarrassingly parallel and are suitable workloads for GPUs.

Skewed or asynchronous computations are bad for GPUs. Since all threads in GPUs are synced, the computation time will be bounded by the longest running thread. In addition, if some threads have dependencies, these parallel cores are forced to wait for each other and compute serially so that they can resolve the dependencies.

If your code has many cache misses (does not fit into GPU’s shared memory ), it is likely to run less efficiently on GPUs. GPUs have slow global memory access compared to CPUs. However, if your data fits entirely on the shared memory or the shared memory miss rate is low, GPU is likely to run more efficiently. This effect, however, highly depends on how the memory access pattern is interleaved with your compute.

Also there's PCIe latency. GPUs nowadays are normally connected to CPUs through PCIe links. If your code needs to sync data between CPUs and GPUs frequently, running the code purely on CPUs might be faster since it avoids PCIe crosstalk.

Sign up for an HPC (High Performance Computing) account here.

All members of the University may apply for a free service level 3 (SL3) account.

Additionally the Department has made some funds available for use of the higher-priority service level 2 (SL2). More information ...

Or, check with your project supervisor whether there are any grant-specific funds available for paid use of the HPC.

Allow a week for your application to be approved (often it's faster). Once signed up, check out the documentation and note that use of the HPC is charged per hour per core, so use it with care! Try to use SL3 when starting out or with low priority tasks. Also be sure that you need to use GPU: it may be that the CPU services offered by the HPC will be sufficient for your needs.

As a few starter tips (for detail see the current documentation): you specify in your log-in host address whether you want to use CPU or GPU, you'll likely need to load software-specific modules into your environment to get started, there are example batch submission scripts for you to modify in /usr/local/Cluster-Docs/SLURM (see also Huiyuan Xie's HPC slurm example), and if you're on a paying tier you can check the balance on the account you're using with the “mybalance” command.

Ask sys-admin if you can have access to a virtual machine running on the Department's GPUs.

MPhil students have priority on this service but PhD students can have access too. More info at the sys-admin GPU page.

Check with your supervisor / others in your group.

Has your group bought its own GPUs? You might be granted access to them if so.

PIs might also be interested in HPC services available outside Cambridge.

for instance by application to the EPSRC or the Oxford-based JADE facility.

This wiki page was put together by Andrew Caines (apc38) thanks to help from Daniel Bates, Chris Hadley, Markus Kuhn, Graham Titmus, Aaron Zhao and Noa Zilberman. Any errors are my own: feel free to send me feedback or add your own hints and tips on GPU use.

If you have questions, there's also the cl-gpu-users mailing list for GPU users in the department. You can find the archives and manage your subscription here.