.. _`ris-docker-basics`:

=================================================================
Docker Basics: Building, Tagging, & Pushing A Custom Docker Image
=================================================================

.. contents::
   :depth: 1
   :local:

.. Note::

    Connecting to get command line access: ``ssh wustlkey@compute1-client-1.ris.wustl.edu``

    Queue to use: ``workshop, workshop-interactive``

    Group to use: ``compute-workshop`` (if part of multiple groups)


In this tutorial we're going to create a custom docker container with Python3, R, and Git
-----------------------------------------------------------------------------------------

.. _`ris-docker-basics-prerequisites`:

Prerequisites
-------------

- A willingness to learn
- Docker Hub Account (`Sign Up Page <https://hub.docker.com/signup>`_)
- `Docker Desktop <https://www.docker.com/products/docker-desktop>`_ (OSX or Win 10 Pro) or `Docker installed on a Linux Distribution <https://docs.docker.com/install/linux/docker-ce/ubuntu/>`_
- An Editor or IDE of your choice (this demo uses `Visual Studio Code <https://code.visualstudio.com/>`_ with the `Docker Extension <https://code.visualstudio.com/docs/containers/overview#_docker-view>`_).

.. _`ris-docker-basics-video`:

Workshop Video
--------------

.. raw:: confluence

    <ac:structured-macro ac:name="html">
        <ac:plain-text-body><![CDATA[

.. raw:: confluence html

    <iframe width="560" height="315" src="https://www.youtube.com/embed/j1taS-7NfxI" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

.. raw:: confluence

    ]]></ac:plain-text-body>
        </ac:structured-macro>


.. _`ris-docker-basics-step-by-step`:

Step By Step Instructions
-------------------------

    CHOOSE A BASE IMAGE
        - `Docker Best Practices <https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#from>`__ suggests using an official image and using the smallest image that meets your needs. We're choosing Debian in this example because it's one of the smaller containers that already contains the libraries R and Python rely on.
        - `Docker hub <https://hub.docker.com/>`_ makes it easy to search for images. You'll see the Debian has an official image. Select this image.

        .. image:: images/dockerbasics/Debian.png

        - You can select the tags tab to see all available tags.

        .. image:: images/dockerbasics/tags.png

        - I chose the stretch-slim tag. Stretch was the name for the most recent version of Debian Linux when this was written. Slim is used to denote that it's a small image designed for containers.

        .. image:: images/dockerbasics/stretchslim.png

        - In your Dockerfile, write ``FROM debian:stretch-slim``. ``FROM`` is Docker's way of specifying that this is the base image layer. After that you're adding the image name ``debian`` with the version tag ``stretch-slim``.

        .. image:: images/dockerbasics/fromdebianstretchslim.gif

    INSTALLING SOFTWARE & DEPENDENCIES WITH ``apt-get``
        - Right now, we just have a containerized Linux distribution. We need to install R, Python, and Git. We can do this all using the apt package manager with the command apt-get.

        .. code::

            RUN apt-get update \
            && apt-get install -y --no-install-recommends r-base r-base-dev git python3 \
            && apt-get clean


        - ``RUN`` tells Docker to RUN the commands that follow.
        - ``apt-get update`` tells apt-get to get the most recent sources for packages you'll install. ``apt-get install`` is the command that proceeds the list of applications you wish to install.
        - A ``-y`` flag is required. Without it apt-get won't know that yes, you really want to install the software and dependencies specified and the build will fail.
        - The ``-- no-install-recommends`` flag is simply to keep the image smaller. You could omit it, but you'll find the resulting image will be larger.
        - ``r-base`` and ``r-base-dev`` are packages needed to install R. ``git`` is the package for downloading source code from a git repository. ``Python3`` is the package for installing Python 3. If you wanted a version of Python2, you'd get 2.7 if you left off the 3.
        - ``apt-get clean`` removes all packages from the cache to save space in the final image.

        .. Note::

            `Docker Best Practices <https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#run>`__ points out that having all these commands on the same line is is to prevent caching. If one command in this statement changes, the whole layer is rebuilt. One could have them in individual RUN statements although this is probably better suited for troubleshooting and initial development.


    BRING IN CODE
        - You've now got an image that contains Python, R, and git. But, it has no code to run. There are several ways to bring in code into your container. The first we'll cover is using git clone. Fully covering everything git can do is outside the scope of this tutorial, but `this page has great resources for learning more about git. <https://try.github.io/>`_
        - Git Clone
            - Bringing in code from git is as simple as running appending the url of the repo you want to "git clone'. Here I've got some code that will print ``hello`` in Python when we run our container.

            .. code::

                git clone https://github.com/DChandlerP/python-hello.git

        - COPY
            - You could use `Docker COPY <https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#add-or-copy>`_ for code that's on the same machine as where you're building your image. So if I put ``print("Hello1")`` in ``hello.py`` in the same directory as my Dockerfile I could add it to the root folder in the container just like this:

            .. code::

                COPY hello.py /


            - For this demonstration, let's stick with using git clone. My Dockerfile should look like this:

            .. image:: images/dockerbasics/pyhellodockerfilleimage.png

    DOCKER BUILD & TAG
        - `Docker build <https://docs.docker.com/engine/reference/commandline/build/>`_ has many options but it's often faster to just build and tag an image at the same time although this can be done separately if you prefer. The syntax to build and tag an image is is as follows:

        .. code::

            docker build -t your_dockerhub_username/image_name:tag .

        - The ``.`` indicates that you want to build in the current working directory. The ``-t`` flag is to indicate that you want to tag the image at the same time you do a build. The following is a concrete example using the author's Docker hub username run in the terminal:

        .. image:: images/dockerbasics/dockerbuildt.gif

        - Building was fast in that gif because all the layers were cached, if you haven't built a dockerfile before expect this to take a few minutes. It gave it the tag "latest" by default since I didn't provide a tag explicitly.

    DOCKER RUN
        - It's possible to skip this step, but it's a good idea to run an image locally before running it on LSF using the Docker wrapper to be sure it works in Docker. The syntax will generally look something like this:

        .. code::

            docker run -it your_dockerhub_username/image_name:optional_tag command /file_path/file

        - Or for a concrete example:

        .. image:: images/dockerbasics/hellopy.gif

    DOCKER PUSH
        - Currently this image is only on your local machine. You need to push it to Docker hub so that you pull it down and run it on any machine with Docker including our Compute1 cluster.

        .. code::

            docker push your_dockerhub_username/image_name:optional_tag

        .. image:: images/dockerbasics/dockerpushlatest.png

    RUNNING A CONTAINER ON COMPUTE1

        .. code::

            ssh wustlkey@compute1-client-1.ris.wustl.edu

        .. code::

            bsub -Is -q workshop-interactive -a 'docker(your_dockerhub_username/image_name:optional_tag)' command /file_path/file

        - You may be required to add ``-G group_name`` after ``bsub`` if you belong to more than one group.

        .. image:: images/dockerbasics/helloonlsf.png