.. _`docker-wrapper-env-vars`: Docker Wrapper Environment Variables ==================================== .. contents:: :depth: 2 :local: Features of the ``docker`` application (a.k.a. the docker wrapper) are accessed by setting environment variables before executing ``bsub``. All the examples below show how to set the values inline with submitting the job, and multiples of these settings are given by including all of them on the same command-line: .. code:: LSF_DOCKER_OPTTION_1=value LSF_DOCKER_OPTION_2="value with spaces" bsub ... Or they can also be set within the shell by "exporting" them, essentially making them apply to every command that follows: .. code:: [user@compute1-client-1 ~]$ export LSF_DOCKER_OPTION_1=value [user@compute1-client-1 ~]$ export LSF_DOCKER_OPTION_2="value with spaces" [user@compute1-client-1 ~]$ bsub ... .. _`lsf-docker-add-host-detail`: LSF_DOCKER_ADD_HOST ------------------- Adds one more more host -> IPaddress mappings to the docker container. The value of ``LSF_DOCKER_ADD_HOST`` is a space-separated list of mappings in the same format as the ``--add-host`` argument of ``docker run``, ie. ``hostname:IPaddress`` .. code:: LSF_DOCKER_ADD_HOST="host_a:192.168.1.2 host_b:172.16.10.2" bsub ... .. _`lsf-docker-add-host-file-detail`: LSF_DOCKER_ADD_HOST_FILE ------------------------ Specify one more more space-separated files containing host -> IPaddress mappings. These files should have one mapping per line in the same format used by :ref:`LSF_DOCKER_ADD_HOST `: .. code:: host_a:192.168.1.2 host_b:172.16.10.2 These file names are then included in the ``LSF_DOCKER_ADD_HOST_FILE`` variable: .. code:: LSF_DOCKER_ADD_HOST_FILE="file_a file_b" bsub ... .. _`lsf-docker-cgroup-detail`: LSF_DOCKER_CGROUP ----------------- Set the value for the ``--cgroup-parent`` argument to ``docker run``. There is no mechanism for creating custom cgroups at this time, so this should not be used unless specifically directed to by RIS. .. _`lsf-docker-entrypoint`: LSF_DOCKER_ENTRYPOINT --------------------- Overrides the container's default ``ENTRYPOINT``, and becomes the ``--entrypoint`` argument to ``docker run`` .. code:: LSF_DOCKER_ENTRYPOINT=/bin/bash bsub ... .. _`lsf-docker-env-file-detail`: LSF_DOCKER_ENV_FILE ------------------- Used to set envrionment variables within the running container. Expects a space-separated list of file names containing environment variables. The files should have one variable per line, and be formatted as ``variable=value``: .. code:: SOME_VARIABLE=value ANOTHER_VARIABLE=value with spaces To make use of these files, set the ``LSF_DOCKER_ENV_FILE`` and submit the job: .. code:: LSF_DOCKER_ENV_FILE="env_file1 env_file2" bsub -q general -a 'docker(...)' ... The file names are then used as ``--env-file`` arguments to ``docker run``. .. _`lsf-docker-ipc-detail`: LSF_DOCKER_IPC -------------- Control the ``--ipc`` argument to ``docker run``. This will become useful when building *MPI* applications. .. code:: export LSF_DOCKER_IPC=host bsub -Is -G ${group_name} -q general-interactive -a 'docker(ubuntu)' /bin/bash .. _`lsf-docker-network-detail`: LSF_DOCKER_NETWORK ------------------ Make use of docker's ``--network`` argument. Most notablly for ``--network=host``. See the `Docker networking tutorial`_. .. _`Docker networking tutorial`: https://docs.docker.com/network/network-tutorial-host/ .. code:: export LSF_DOCKER_NETWORK=host bsub -Is -G ${group_name} -q general-interactive -a 'docker(ubuntu)' /bin/bash .. _`lsf-docker-ports-detail`: LSF_DOCKER_PORTS ---------------- Specify network ports to expose from your docker container. Note that these must come from a range of "approved ports" (8000-8999) and should be "reserved" by the job scheduler. Here we map port 80 inside the container to port 8001 on the execution node, and we "reserve" port 8001 on that execution node through the scheduler. .. code:: export LSF_DOCKER_PORTS='8001:80' bsub -q general -R 'select[port8001=1]' -a 'docker(httpd)' /usr/local/bin/httpd-foreground The ``LSF_DOCKER_PORTS`` variable works with most expressions accepted by the ``-p`` option of ``docker run``: * ``8001``: Expose a single port * ``8001:80``: Expose port 8001 and forward it to port 80 within the container * ``8001-8010``: Expose a range of ports * ``8001-8010:2001-2010``: Expose a range of forwarded ports * ``8001/tcp``: Expose only TCP port 8001 * ``8001-8010/udp``: Expose a range of UDP ports Additionally, more than one port may be exposed by separating each request with a space. For example: .. code:: LSF_DOCKER_PORTS='8000 8500 8600-8700` See also :ref:`Exposing Ports from Within Containers `. .. _`lsf-docker-preserve-environment-detail`: .. note:: Note: LSF_DOCKER_PORTS is affected by the LSF_DOCKER_NETWORK type, and does not work with "host" type networking. LSF_DOCKER_PRESERVE_ENVIRONMENT ------------------------------- This is a boolean value instructing the job launcher to either preserve your shell environment or not. Valid values are the strings ``true`` and ``false``. The default value is ``true``. When false, the container will have a minimal environment with only a few variables, including ``HOSTNAME`` set to the exec host the job runs on, and those variables defined in the container's Dockerfile with ``ENV``. When true, the container will inherit most environment variables set when the job is submitted, except for a few: * HOSTNAME - becomes the exec host's hostname * LSB_INTERACTIVE - removed within an interactive job so jobs started within that job are not interactive by default * values containing newlines - these variables are removed due to a bug in docker's handling of environment variables .. code:: export LSF_DOCKER_PRESERVE_ENVIRONMENT=false bsub -Is -G ${group_name} -q general-interactive -a 'docker(ubuntu)' /bin/bash .. _`lsf-docker-run-loglevel-detail`: LSF_DOCKER_RUN_LOGLEVEL ----------------------- Set the log level for the wrapper script: .. code:: export LSF_DOCKER_RUN_LOGLEVEL=DEBUG bsub -Is -G ${group_name} -q general-interactive -a 'docker(ubuntu)' /bin/bash Valid values are, in descending order of verbosity: CRITICAL, ERROR, WARNING, INFO, DEBUG .. _`lsf-docker-shm-size-detail`: LSF_DOCKER_SHM_SIZE ------------------- Control the ``--shm-size`` argument to ``docker run``. This will become useful when building *MPI* applications. .. code:: export LSF_DOCKER_SHM_SIZE=4g bsub -Is -G ${group_name} -q general-interactive -a 'docker(ubuntu)' /bin/bash .. _`lsf-docker-use-lucid-auth`: LSF_DOCKER_USE_LUCID_AUTH ------------------------- Getting authentication through libnss-sss working for containers based on Ubuntu Lucid Lynx (10.04) requires extra files and devices from the host system to be mounted in the container. This option should not be used unless you are having trouble getting authentication working on a container based on Ubuntu Lucid. .. code:: LSF_DOCKER_USE_LUCID_AUTH=y bsub ... .. _'lsf-docker-workdir-detail`: LSF_DOCKER_WORKDIR ------------------ The default initial working directory within the container is the same as the working directory when the job was submitted. This can be overridden with ``LSF_DOCKER_WORKDIR`` and becomes the ``-w`` option to ``docker run``, and must be a directory accessible within the container. .. code:: LSF_DOCKER_WORKDIR=/path/within/container bsub ... .. _`lsf-docker-volumes-detail`: LSF_DOCKER_VOLUMES ------------------ This is a space separated list of filesystem locations to pass into your docker container by means of the ``--mount`` flag. The format is ``src:dst``, where ``src`` means the filesystem location outside the container should be mapped to ``dst`` inside the container. (See also ``--mount`` at `Docker Volumes`_.) .. _`Docker Volumes`: https://docs.docker.com/storage/volumes/ .. code:: export LSF_DOCKER_VOLUMES="/storage1/fs1/${STORAGE_ALLOCATION}:/storage1/fs1/${STORAGE_ALLOCATION} /scratch1/fs1/${COMPUTE_ALLOCATION}:/scratch1/fs1/${COMPUTE_ALLOCATION}" bsub -Is -G ${group_name} -q general-interactive -a 'docker(ubuntu)' /bin/bash .. note:: Elements in LSF_DOCKER_VOLUMES must be *directories*. Docker would allow you to pass in files, such that the following might be expected to work: LSF_DOCKER_VOLUMES="$HOME/etc/myfile:/etc/somefile" But the RIS environment explicitly prevents this due to a security vulnerability. A volume can be mounted read-only by appending ``:ro`` to any of the mounts, for example: ``src:dst:ro``. .. _`lsf-docker-run-raw-args`: LSF_DOCKER_RUN_RAW_ARGS ----------------------- This variable tells bsub to use the command or entrypoint within the Docker image instead of requesting the command or entrypoint from the user as is the regular interaction without this variable. This means that Docker images like the basic hello-world Docker image can be run on the Compute Platform. The command also makes use of ``LSB_DOCKER_PLACE_HOLDER`` which is, as the name suggests, just a placeholder as bsub still needs an input to run. This is NOT passed as a command to the Docker container. Example Command: .. code:: LSF_DOCKER_RUN_RAW_ARGS=1 bsub -Is -G ${group_name} -q general-interactive -a 'docker(hello-world)' LSB_DOCKER_PLACE_HOLDER Output: .. code:: Job <897631> is submitted to queue . <> <> Using default tag: latest latest: Pulling from library/hello-world Digest: sha256:ffb13da98453e0f04d33a6eee5bb8e46ee50d08ebe17735fc0779d0349e889e9 Status: Image is up to date for hello-world:latest docker.io/library/hello-world:latest Hello from Docker! This message shows that your installation appears to be working correctly. To generate this message, Docker took the following steps: 1. The Docker client contacted the Docker daemon. 2. The Docker daemon pulled the "hello-world" image from the Docker Hub. (amd64) 3. The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading. 4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal. To try something more ambitious, you can run an Ubuntu container with: $ docker run -it ubuntu bash Share images, automate workflows, and more with a free Docker ID: https://hub.docker.com/ For more examples and ideas, visit: https://docs.docker.com/get-started/