High Performance Computing Service: Difference between revisions

From Brain Mapping Unit
Jump to navigationJump to search
No edit summary
 
(One intermediate revision by the same user not shown)
Line 2: Line 2:
This page is a work in progress!
This page is a work in progress!


I'm just using it as a repository for helpful hints that I've found useful!
I wrote it in 2013 and it hasn't been updated since then. The HPC documentation has been though so you're probably best starting there and editing this page as you find out useful things!


Kirstie x
Kirstie x
Line 14: Line 14:
== Getting an account ==
== Getting an account ==


As with everything in Cambridge there's some paperwork to fill in. Download the application form from here: http://www.hpc.cam.ac.uk/applications.html, wait a week and then you're off.
As with everything in Cambridge there's some paperwork to fill in. Download the application form from here: http://www.hpc.cam.ac.uk/services/applying-for-resources, wait a week and then you're off.




Line 21: Line 21:
If you're ''inside the Cambridge firewall'' then you can connect via putty to:
If you're ''inside the Cambridge firewall'' then you can connect via putty to:


     '''[USERNAME]@login.hpc.cam.ac.uk''' eg: kw401@login.hpc.cam.ac.uk
     '''[USERNAME]@login.hpc.cam.ac.uk'''
    eg: kw401@login.hpc.cam.ac.uk


You can then create a vncserver session by following these instructions: http://www.hpc.cam.ac.uk/user/visualization.html.
You can then create a vncserver session by following these instructions: http://www.hpc.cam.ac.uk/user/visualization.html.
Line 27: Line 28:
If you're ''outside the Cambridge firewall'' (at home or in a coffee shop) then you should connect to your BCNI vnc session (eg via VNC Viewer or Chicken of the VNC) and then type:
If you're ''outside the Cambridge firewall'' (at home or in a coffee shop) then you should connect to your BCNI vnc session (eg via VNC Viewer or Chicken of the VNC) and then type:


     '''ssh -X [USERNAME]@login.hpc.cam.ac.uk''' eg: ssh -X kw401@login.hpc.cam.ac.uk
     '''ssh -X [USERNAME]@login.hpc.cam.ac.uk'''
    eg: ssh -X kw401@login.hpc.cam.ac.uk


This will transport you - in the same terminal window - to the HPC and allow you to use the graphics installed there.
This will transport you - in the same terminal window - to the HPC and allow you to use the graphics installed there.


Don't forget that if you open a ''new'' terminal window while logged into the BCNI that terminal will be connected to the BCNI! You'll have to ssh again :)
Don't forget that if you open a ''new terminal window'' while logged into the BCNI that terminal will be connected to the BCNI! You'll have to ssh again :)


== If you want to see all the jobs you've submitted ==
== If you want to see all the jobs you've submitted ==

Latest revision as of 08:42, 7 April 2016

Disclaimer

This page is a work in progress!

I wrote it in 2013 and it hasn't been updated since then. The HPC documentation has been though so you're probably best starting there and editing this page as you find out useful things!

Kirstie x

Introduction

If you have many jobs that are very similar and, importantly, independent then you can use the High Performance Computing Service [[1]].

The website is helpful, but below are a few commands that I think supplement their FAQs. Please do edit (or email me and I'll update the wiki: kw401@cam.ac.uk) if you have any further suggestions.


Getting an account

As with everything in Cambridge there's some paperwork to fill in. Download the application form from here: http://www.hpc.cam.ac.uk/services/applying-for-resources, wait a week and then you're off.


Logging in

If you're inside the Cambridge firewall then you can connect via putty to:

   [USERNAME]@login.hpc.cam.ac.uk
   eg: kw401@login.hpc.cam.ac.uk

You can then create a vncserver session by following these instructions: http://www.hpc.cam.ac.uk/user/visualization.html.

If you're outside the Cambridge firewall (at home or in a coffee shop) then you should connect to your BCNI vnc session (eg via VNC Viewer or Chicken of the VNC) and then type:

   ssh -X [USERNAME]@login.hpc.cam.ac.uk
   eg: ssh -X kw401@login.hpc.cam.ac.uk

This will transport you - in the same terminal window - to the HPC and allow you to use the graphics installed there.

Don't forget that if you open a new terminal window while logged into the BCNI that terminal will be connected to the BCNI! You'll have to ssh again :)

If you want to see all the jobs you've submitted

As you may guess, qstat gives you some statistics on the queues. To look for just your jobs type:

qstat -u $USER

This $USER variable contains your user ID.

If you want to delete all the jobs you've submitted

To just get the job IDs you need to use qselect and pipe these to qdel (which deletes the job) using xargs.

qselect -u $USER | xargs qdel

Type man xargs if you want to know more about this terribly useful command which pipes information from standard input into the next command.

More specifically if you want to delete only your running jobs:

qselect -u $USER -s R | xargs qdel

Or if you want to delete only your queued jobs:

qselect -u $USER -s Q | xargs qdel