Docker Wrapper Environment Variables¶
Features of the docker
application (a.k.a. the docker wrapper) are accessed
by setting environment variables before executing bsub
. All the examples
below show how to set the values inline with submitting the job, and multiples
of these settings are given by including all of them on the same command-line:
LSF_DOCKER_OPTTION_1=value LSF_DOCKER_OPTION_2="value with spaces" bsub ...
Or they can also be set within the shell by “exporting” them, essentially making them apply to every command that follows:
[user@compute1-client-1 ~]$ export LSF_DOCKER_OPTION_1=value
[user@compute1-client-1 ~]$ export LSF_DOCKER_OPTION_2="value with spaces"
[user@compute1-client-1 ~]$ bsub ...
LSF_DOCKER_ADD_HOST¶
Adds one more more host -> IPaddress mappings to the docker container. The
value of LSF_DOCKER_ADD_HOST
is a space-separated list of mappings in the
same format as the --add-host
argument of docker run
, ie.
hostname:IPaddress
LSF_DOCKER_ADD_HOST="host_a:192.168.1.2 host_b:172.16.10.2" bsub ...
LSF_DOCKER_ADD_HOST_FILE¶
Specify one more more space-separated files containing host -> IPaddress mappings. These files should have one mapping per line in the same format used by LSF_DOCKER_ADD_HOST:
host_a:192.168.1.2
host_b:172.16.10.2
These file names are then included in the LSF_DOCKER_ADD_HOST_FILE
variable:
LSF_DOCKER_ADD_HOST_FILE="file_a file_b" bsub ...
LSF_DOCKER_CGROUP¶
Set the value for the --cgroup-parent
argument to docker run
. There
is no mechanism for creating custom cgroups at this time, so this should not
be used unless specifically directed to by RIS.
LSF_DOCKER_ENTRYPOINT¶
Overrides the container’s default ENTRYPOINT
, and becomes the
--entrypoint
argument to docker run
LSF_DOCKER_ENTRYPOINT=/bin/bash bsub ...
LSF_DOCKER_ENV_FILE¶
Used to set envrionment variables within the running container. Expects a
space-separated list of file names containing environment variables. The
files should have one variable per line, and be formatted as
variable=value
:
SOME_VARIABLE=value
ANOTHER_VARIABLE=value with spaces
To make use of these files, set the LSF_DOCKER_ENV_FILE
and submit the
job:
LSF_DOCKER_ENV_FILE="env_file1 env_file2" bsub -q general -a 'docker(...)' ...
The file names are then used as --env-file
arguments to docker run
.
LSF_DOCKER_IPC¶
Control the --ipc
argument to docker run
. This will become useful when
building MPI applications.
export LSF_DOCKER_IPC=host
bsub -Is -G ${group_name} -q general-interactive -a 'docker(ubuntu)' /bin/bash
LSF_DOCKER_NETWORK¶
Make use of docker’s --network
argument. Most notablly for --network=host
.
See the Docker networking tutorial.
export LSF_DOCKER_NETWORK=host
bsub -Is -G ${group_name} -q general-interactive -a 'docker(ubuntu)' /bin/bash
LSF_DOCKER_PORTS¶
Specify network ports to expose from your docker container. Note that these must come from a range of “approved ports” (8000-8999) and should be “reserved” by the job scheduler. Here we map port 80 inside the container to port 8001 on the execution node, and we “reserve” port 8001 on that execution node through the scheduler.
export LSF_DOCKER_PORTS='8001:80'
bsub -q general -R 'select[port8001=1]' -a 'docker(httpd)' /usr/local/bin/httpd-foreground
The LSF_DOCKER_PORTS
variable works with most expressions accepted by
the -p
option of docker run
:
8001
: Expose a single port8001:80
: Expose port 8001 and forward it to port 80 within the container8001-8010
: Expose a range of ports8001-8010:2001-2010
: Expose a range of forwarded ports8001/tcp
: Expose only TCP port 80018001-8010/udp
: Expose a range of UDP ports
Additionally, more than one port may be exposed by separating each request with a space. For example:
LSF_DOCKER_PORTS='8000 8500 8600-8700`
See also Exposing Ports from Within Containers.
Note
Note: LSF_DOCKER_PORTS is affected by the LSF_DOCKER_NETWORK type, and does not work with “host” type networking.
LSF_DOCKER_PRESERVE_ENVIRONMENT¶
This is a boolean value instructing the job launcher to either preserve your shell environment or not.
Valid values are the strings true
and false
. The default value is true
.
When false, the container will have a minimal environment with only a few variables, including
HOSTNAME
set to the exec host the job runs on, and those variables defined in the container’s
Dockerfile with ENV
.
When true, the container will inherit most environment variables set when the job is submitted, except for a few:
HOSTNAME - becomes the exec host’s hostname
LSB_INTERACTIVE - removed within an interactive job so jobs started within that job are not interactive by default
values containing newlines - these variables are removed due to a bug in docker’s handling of environment variables
export LSF_DOCKER_PRESERVE_ENVIRONMENT=false
bsub -Is -G ${group_name} -q general-interactive -a 'docker(ubuntu)' /bin/bash
LSF_DOCKER_RUN_LOGLEVEL¶
Set the log level for the wrapper script:
export LSF_DOCKER_RUN_LOGLEVEL=DEBUG
bsub -Is -G ${group_name} -q general-interactive -a 'docker(ubuntu)' /bin/bash
Valid values are, in descending order of verbosity: CRITICAL, ERROR, WARNING, INFO, DEBUG
LSF_DOCKER_SHM_SIZE¶
Control the --shm-size
argument to docker run
. This will become useful
when building MPI applications.
export LSF_DOCKER_SHM_SIZE=4g
bsub -Is -G ${group_name} -q general-interactive -a 'docker(ubuntu)' /bin/bash
LSF_DOCKER_USE_LUCID_AUTH¶
Getting authentication through libnss-sss working for containers based on Ubuntu Lucid Lynx (10.04) requires extra files and devices from the host system to be mounted in the container. This option should not be used unless you are having trouble getting authentication working on a container based on Ubuntu Lucid.
LSF_DOCKER_USE_LUCID_AUTH=y bsub ...
LSF_DOCKER_WORKDIR¶
The default initial working directory within the container is the same as the
working directory when the job was submitted. This can be overridden with
LSF_DOCKER_WORKDIR
and becomes the -w
option to docker run
, and
must be a directory accessible within the container.
LSF_DOCKER_WORKDIR=/path/within/container bsub ...
LSF_DOCKER_VOLUMES¶
This is a space separated list of filesystem locations to pass into your docker
container by means of the --mount
flag. The format is src:dst
, where
src
means the filesystem location outside the container should be mapped to
dst
inside the container. (See also --mount
at Docker Volumes.)
export LSF_DOCKER_VOLUMES="/storage1/fs1/${STORAGE_ALLOCATION}:/storage1/fs1/${STORAGE_ALLOCATION} /scratch1/fs1/${COMPUTE_ALLOCATION}:/scratch1/fs1/${COMPUTE_ALLOCATION}"
bsub -Is -G ${group_name} -q general-interactive -a 'docker(ubuntu)' /bin/bash
Note
Elements in LSF_DOCKER_VOLUMES must be directories. Docker would allow you to pass in files, such that the following might be expected to work:
LSF_DOCKER_VOLUMES=”$HOME/etc/myfile:/etc/somefile”
But the RIS environment explicitly prevents this due to a security vulnerability.
A volume can be mounted read-only by appending :ro
to any of the mounts,
for example: src:dst:ro
.
LSF_DOCKER_RUN_RAW_ARGS¶
This variable tells bsub to use the command or entrypoint within the Docker image instead of requesting the command or entrypoint from the user as is the regular interaction without this variable.
This means that Docker images like the basic hello-world Docker image can be run on the Compute Platform.
The command also makes use of LSB_DOCKER_PLACE_HOLDER
which is, as the name suggests,
just a placeholder as bsub still needs an input to run. This is NOT passed as a command
to the Docker container.
Example Command:
LSF_DOCKER_RUN_RAW_ARGS=1 bsub -Is -G ${group_name} -q general-interactive -a 'docker(hello-world)' LSB_DOCKER_PLACE_HOLDER
Output:
Job <897631> is submitted to queue <qa-interactive>.
<<Waiting for dispatch ...>>
<<Starting on compute1-exec-83.ris.wustl.edu>>
Using default tag: latest
latest: Pulling from library/hello-world
Digest: sha256:ffb13da98453e0f04d33a6eee5bb8e46ee50d08ebe17735fc0779d0349e889e9
Status: Image is up to date for hello-world:latest
docker.io/library/hello-world:latest
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/get-started/