previous episode

Using Singularity at Jefferson Lab

next episode

Using docker containers

Overview

Teaching: 10 min
Exercises: 10 min
Questions

  • Why should I know about docker containers if I am using singularity?

Objectives

  • Understand how container layering allows for portability and reusability.

  • Map the ecosystem of services that can take advantage of docker containers.

Up to this point we have exclusively talked about singularity as our interface to containers. However, you will likely be faced with blank stares if you talk about singularity containers with our friends at Amazon Web Services, GitHub, or literally any other large software organization that is investing in containerization and virtualization.

They will talk to you about docker.

Singularity, Docker: What’s all that about?

The most common container format currently in use is not singularity, but rather docker. Why do we not use docker on the interactive farm nodes at Jefferson Lab?

Unfortunately running docker in a scientific computing environment requires too many trade-offs on security to the point where it is not a workable solution. Effectively one would need to give superuser privileges to all docker users at Jefferson Lab…

This is where singularity comes in! At no point in this lesson did you need superuser privileges to run a guest operating system on the Jefferson Lab interactive farm node.

But wait, there’s more!

Singularity can natively load docker containers, directly from Docker Hub (where docker containers are typically housed). Instead of specifying shub we merely have to specify docker in the ‘protocol’ section of the URL.

Retrieving the Jefferson Lab Common Environment container directly from Docker Hub

We can now retrieve the Jefferson Lab Common Environment container:

singularity pull docker://jeffersonlab/jlabce:2.2

Instead of loading a full singularity container, docker uses a multitude of layers, one for each operation that was performed on the container. This allows docker to be more efficient with disk space (no need to store the same base layer more than once, after all).

By using singularity we are giving up some of the benefits of docker, but gaining the ability to run without superuser privileges. Let’s try this on the Jefferson Lab Common Environment container.

Retrieve the Jefferson Lab Common Environment container, tag 2.2, directly from Docker Hub

  1. Take a look at the Docker Hub page for the jlabce container
  2. Note the capitalization of ‘jeffersonlab’ (we’ve all made some bad choices in the past that we now regret)
  3. Pull the Docker Hub container using singularity pull
  4. Start a shell in the container from Docker Hub (which will have by default a different filename from the container from Singularity Hub)
  5. Unset OSRELEASE and load the Jefferson Lab Common Environment, version 2.2.

Solution

The downloading should have produced the following output:

You can see that 11 layers went into this container. If you were to now download the container with tag 2.1, which is based on the same underlying CentOS 7.3 layer, you would not need to download nearly as much. Nevertheless, the final singularity image (.simg) would again be about 2 GB large.

Retrieve Electron Ion Collider containers directly from Docker Hub

But, of course, now that we have gotten the hang of this, why stop here?

We can pull the environment for the Jefferson Lab EIC (JLEIC) simulation studies, tag 1.0.3.

Retrieve the Jefferson Lab Electron Ion Collider (JLEIC) container

The location on Docker Hub is electronioncollider/jleic:1.0.3

Solution

We can even pull the equivalent environment for the Brookhaven National Lab EIC simulation studies, tag r964.

Retrieve the Brookhaven National Lab Electron Ion Collider container

The location on Docker Hub is ayk1964/eicroot:r964

It would be unthinkable to run two disparate frameworks from different national labs based on different operating systems next to each other with minimal effort.

Retrieve MOLLER simulation and analysis containers directly from Docker Hub

We can pull the environment for the Jefferson Lab MOLLER simulation and analysis software, remoll and japan.

Retrieve the Jefferson Lab MOLLER simulation container

The location on Docker Hub is jeffersonlab/remoll:develop

Once you pulled the remoll:develop container, run the command remoll inside it, either using shell or using run.

Solution

Retrieve the Jefferson Lab MOLLER analyzer container

The location on Docker Hub is jeffersonlab/japan:develop

Solution

Submitting farm jobs with singularity

Of course, ultimately the goal is to submit farm jobs with singularity. Here is a basic xml job scripts which just prints the usage information for one of the japan components:

<Request>
  <Email email="username@jlab.org" request="false" job="false"/>
  <Project name="moller12gev"/>
  <Track name="simulation"/>
  <Name name="remoll-singularity"/>
  <Command><![CDATA[
   setenv SINGULARITY_CACHEDIR .
   module load singularity
   cd /volatile/halla/parity/$USER/singularity
   singularity run remoll-develop.simg build/remoll /volatile/halla/parity/$USER/singularity/runexample.mac
  ]]></Command>

  <Job>
  </Job>
</Request>

Since (currently) the command is run from a fixed directory inside the container, you have to both specify an absolute path to the macro on the command line and specify an absolute path to the output ROOT file inside the macro.

Key Points

  • Docker containers can be used on the widest array of platforms.

previous episode

next episode