Locker Setup
Our installs of locker currently run in a VM provided by the university computing service.
Locker Install Setup
Lockers are currently run from inside a Xen virtual machine, hosted by the University Computing Service. A second Xen machine handles the DNS records for the locker.cam.ac.uk domain.
The locker code is used within a minimal Debian install, together with NodeJS and MongoDB to form a container, one per user, which is isolated in it's own LXC container and runs as a virtual machine. This virtual machine stores all data that is uploaded from the iPhone app and controls access to it. It also allows migration to other hosts, should the user wish to move it.
Virtual Machine Setup
The VMs used for the current locker setup are hosted by the University Computing Service.
ns1.locker.cam.ac.uk ( 193.60.91.221 )
is the primary DNS server for the locker.cam.ac.uk domain.
db1.locker.cam.ac.uk ( 193.60.91.222 )
hosts all lockers, and the front end proxy server and web server.
DNS Setup
The Primary DNS for the lcoker.cam.ac.uk zone is handle by the 'ns1' VM. This machine is running Debian Linux (Squeeze), and uses the NSD name server to provide DNS functionality.
The file at /etc/nsd3/locker.cam.ac.uk.zone contains the name entries to manage the zone.
The DNS is setup so all locker addresses resolve to a singular IP, currently the same one as the 'DB1' VM has as it hosts all existing lockers. Thus somelocker.locker.cam.ac.uk points to the 'db1' VM IP, and Nginx running on that VM forwards this request based on the HOST field to the correct locker on an internally accessible only IP.
Nginx Setup
The 'DB1' VM runs an instance of Nginx, that behaves as a proxy for each users locker, as well as providing authentication and encryption.
Each locker has it's own config file for nginx in the /etc/nginx/sites-avalible/ which is symlinked to /etc/nginx/sites-enabled/ when active. This file contains the configuration for the proxy settings of the locker, namely which host name it should listen for, and where it should forward the request (In this case, which internal-only IP address is the user's locker running on), as well as the location of the htpasswd file used to authenticate users, which is generated using the htpasswd command.
LXC Container Setup
LXC needs to be installed on the target systems to create lockers. It is already configured on DB1.
"Linux Containers (LXC) are an operating system-level virtualization method for running multiple isolated server installs (containers) on a single control host. LXC does not provide a virtual machine, but rather provides a virtual environment that has its own process and network space. It is similar to a chroot, but offers much more isolation." - [1]
See http://lxc.sourceforge.net for more information specific to LXC.
Locker Management
Creating a Locker
Locker creation and setup involves a number of steps, and a helper script exist to simplify the process. Once you have logged on to the root locker server (currently db1.locker.cam.ac.uk), lockers reside in /lockers/containers/. This directory also includes the Scripts sub-directory which contains the helper scripts, list of internal IPs currently in-use by active lockers etc. Each sub-directory is named after each locker, and contains a self-titled directory which contains the actual container filesystem, a fstab and configuration file for LXC to mount the appropriate directories, and a log file for output of the container while running.
To create a locker, execute "python /lockers/containers/Scripts/SpinNewContainer.py username password" where username and password are the authentication credentials you wish to use for the new locker. This will create a new LXC container with a minimal install of Debian within it, alter the networking setup for the container and the locker instance within it, and also add a proxy routing entry for the nginx server to route username.locker.cam.ac.uk to the newly created container.
you will need to log in to the name server and add the username to the root server's IP in the DNS server configuration so that username.locker.cam.ac.uk will point to the right place. Currently this involves editing etc/nsd3/locker.cam.ac.uk.zone (adding an A record for the new username is usually all that's needed, ie:
username A 193.60.91.222
and then doing "nsdc rebuild" and then "nsdc reload". The Nginx proxy will then forward the request to the correct container based on the hostname.
Starting a Locker
You can start the locker using lxc-start -n xyz -d --logfile=/lockers/containers/xyz/xyz.log with the relevant username in place of xyz. The d argument starts the container in daemon mode - to start and log into a locker directly, remove this option.
After giving the container a few seconds to startup, you should be able to ssh in using the IP address assigned by the script in the first step. To then start the locker service, run cd /locker/ and then ./lockerBigStack > Logs/test.log 2>&1 &, which will start the node locker service, redirect it's output to the test.log file and create a mongodb store for it to use on first run.
You should then be able to access the locker by following instructions in Viewing Commutes in your Locker
Stopping a Locker
Stop the container with lxc-stop -n xyz.
Destroying a Locker
Destroy a container with lxc-destroy -n xyz. NB this removes the entry from with LXC which allows starting by name etc. It does not remove the containers file system.
Listing Existing Lockers
lxc-ls -l can be used to list existing lockers on the system.
Locker Directory Layout
From the root directory of the LXC Container (I.e. /lockers/containers/xyz/xyz/) the locker install can be found within locker/. Within this directory are a few notable items:
- Apps
Location of applications available to locker.
- Config
Location of various configuration JSON files, such as API keys used to sync to different services such as Flickr, Facebook, Foursquare etc, as well as locker's own config files, which applications it can use, ports to listen on etc.
- Connectors
Location of all connectors available to locker. Each connector is responsible for pulling in data from their parties and services such as Facebook and Twitter.
- Log
Logging files location.
- Me
This directory is created after first run, and contains the MongoDB database and raw JSON data downloaded or pushed to locker.
For more information creating locker applications, see Building Locker Apps
