Creating and Managing Docker VM Extensions in Azure

Docker technology is Linux-based. Although the concept of a container isn't new, the common toolset, packaging model and deployment mechanism of Docker has made the use of containers far simpler. As a quick introduction to the technology for Windows users, Krishna shows, step by step, how to set up, and configure a Docker Daemon in Azure within a VM, and install a MySQL image in a container from the Docker Repository.

In this article, I’ll first touch on why containers are important, and how they can be a better solution for some software development tasks than using virtual machines. I’ll then explain some of the jargon around Docker. With that out of the way, I’ll give you a step-by-step guide to setting up, and configuring a Docker Daemon in Azure within a VM, using Docker VM Extensions in Azure. Finally, I’ll show how to manage a functional MySQL Database inside a Docker container.

Why Containers are important for Software Development

Before the advent of Virtual machines and containers, developers generally preferred to develop client-server based applications on their own systems. Then they would test them on staging servers before deploying them on the production servers. This process has been rendered more complex by the increasing intricacies of the steps that are involved in building, configuring, and testing the application. Before software can be shipped, there are also such issues as the hardware cost, dependences and portability.

To avoid hardware provisioning becoming a problem during the software development cycle, increasing use is being made of virtualization. Virtualization allows several applications to be hosted concurrently, each of them using a small proportion of the physical resources that are available, on the same physical machine. Several guest machines, can be deployed on the host machines as Virtual Machines (VMs), whatever their operating system. VMs can also help to automate testing by allowing a range of test environments to be created by a script, used to run tests, and then removed. By using VMs on servers, you can make better use of existing hardware, scale them more easily, and make deployment simpler.

The hypervisor, it turns out, isn’t the most efficient way of using resources. Each virtual machine needs dedicated resources like such as CPU, Memory, Disk etc. but a majority of these resources are used by the hypervisor and the guest machine operating system itself, leaving only a minority available for use by applications.

To overcome some of the limitation of VMs, Containers have become increasingly popular. Like VMs, they can be used locally within an intranet, or in the cloud like Azure. Containers are quicker to start up, and avoid most of the the processing and storage overhead by instead using the process and file system isolation features of the Linux kernel to expose only kernel features to a container that is otherwise isolated from other processes. Containers allow developers to build applications directly on the host operation system without the need of the Hypervisor or VMs. To the application, the container seems to be a fully-isolated and independent OS. Although containerization has been a part of Linux for a long time, it is only fairly recently that, by creating a common toolset, packaging model and deployment mechanism, Docker has made it far simpler to containerize and distribute applications that can then run anywhere on any Linux host.

Docker

2296-figure1.png

Figure 1. Docker on the host machine

Docker uses Linux containers to publish the application on the host machine. The host machine can be any well-known Linux distribution such as Ubuntu. Docker can be installed on Windows or MAC host machine within a Linux virtual machine (or “VM”) installed on it. Multiple containers can be created on the host machine and will run Linux-based applications independently within each container. For example, we can bring up three local MySQL Servers on three different containers and run an application against them.

Docker is not a hypervisor and does not need dedicated physical resources. The Docker Engine provides application platforms called containers which isolate code, runtime, system resource, supporting/dependent binaries and libraries to run the applications. The container itself is the small basic version of Linux OS which runs on the host machine. Containers can be installed with any necessary supporting tool, libraries, code to run the application. These containers allows developers to build applications, image, and ship to any other host machine to run independently. Because it has all the supporting tools and libraries within the image, there will be no configuration required on the new host machine.

Docker Components

Before we take a more practical approach, there is some jargon that I’ll need to introduce:

  • Docker Engine or Daemon: The Docker Daemon is a Linux-based engine which runs on any a Linux host machine.
  • Docker Client: The Docker Client is the command line interface (CLI) to interact with the Docker Daemon. We will discuss the different Docker clients in later part of the article.
  • Docker Container: The Docker Container is the isolated Linux platform with all the necessary tools readily available like operation system, applications, supporting libraries etc. for developers to start building and running the application.
  • Docker Image: The Docker Image is the snapshot of the container that is created from containers. It is the complete read-only environment with base operation system, applications, supporting libraries etc. and can be readily shipped and deployed in any other host machine.
  • Docker Hub Repository: This is the repository to store and share images. Images can be stored in both public and private repositories.
  • Dockerfile: DockerFile helps to automate the image creation process from Docker container.

Docker Virtual Machine Extension Creation using Windows Azure

To get to the point of being able to install Docker containers on Windows Azure, you must have a Docker that can host containers for your applications on Azure. Before we can do that, we need to create an Azure Docker VM extension to the Azure Linux Agent. There are two ways to create Docker VM Extension using Windows Azure:

  1. Deploying Docker VM Extension using Azure Marketplace
  2. Deploying Docker VM Extension using Azure portal or Command-Line (CLI)

We’ll concentrate only on Deploying Docker VM Extension using Azure Marketplace, because this is quick and easy to setup and configure. Deploying Docker VM Extension using Azure portal or CLI is a more complex and lengthy process that is described here.

Deploying Docker VM Extension using Azure Marketplace

The Docker VM Extension can be easily created using the ‘Docker on Ubuntu Server’ Azure Marketplace image. This image contains Ubuntu LTSserver with Docker VM extension and latest Docker Engine pre-installed. This imaged is created and published on the Azure Marketplace by Canonical in collaboration with Microsoft. Creating Docker VM Extension using Azure market place image is quick, easy and it can be done in just few steps:

Installing the Docker Virtual Machine

  1. Login to the http://portal.azure.com
  2. Click on New > Create > Compute > Marketplace > everything

    2296-figure2.png

    Figure 2. Microsoft Azure Portal

  3. Search for ‘Docker on Ubuntu Server’ then click ‘Create’ button in the right corner

    2296-figure3.png

    Figure 3. Searching Docker on Ubuntu server image in Azure

  4. Provide the hostname, username, password and optionally other settings such as Server configuration and location could be specified. Finally click on ‘Create’ to create a new Ubuntu server with Docker VM Extension.

    2296-figure4.png

    Figure 4. Docker on Ubuntu Server Creating process

    The New virtual machine creation process will take some time but finally, the Docker VM running status should show up the icon pinned at the starboard of the Azure portal.

    2296-figure5.png

    Figure 5. Azure Portal with the Docker VM running status

Installing Docker on the client windows machine and connecting to Azure Docker VM Extension

Download the latest version of Docker from http://www.docker.com and start the installation using the Docker Toolbox Setup Wizard as seen in Figures 6 and 7.

2296-figure6.png

Figure 6. Docker Toolbox Setup Wizard

2296-figure7.png

Figure 7. Docker Toolbox components selection

Once installed, it creates three shortcut icons on the desktop:

  • Docker QuickStart Terminal
  • Kitematic (Alpha)
  • Oracle VM VirtualBox

2296-figure8.png

Figure 8. Docker post-installation application shortcuts

Oracle VM VirtualBox is a virtualization software which allows to install and manage multiple flavor of operation system. By default it is installed and configured with Docker on a Linux machine. This Docker machine can be connected and managed locally or remotely.

2296-figure9.png

Figure 9. Oracle VM VirtualBox with default Docker VM

Docker Quick Start Terminal is the client / management tool which allows users to connect to the default local Docker machine or to any remote Docker machine including Docker on Azure. To connect to local Docker machine, double click on Docker Quick Start Terminal. This will check the default Virtual Docker machine in the local Oracle VM VirtualBox, then start the machine and connect to it.

2296-figure10.png

Figure 10. Docker Quick Start Terminal Console

Connecting to the Azure Docker Machine and validating the installation

With the Docker Quick Start Terminal, we can  connect to the Azure Docker VM server using an SSH command in the format of: ssh username@dockervmname.cloudapp.net (for example: ssh krishna@dockervm-mcgwl02j.cloudapp.net). Once the command has been executed, it will connect to the Azure Docker machine and display the message “The authenticity of host <hostname> can’t be established”. When asked if you want to continue connecting, type yes and press enter; you will then be prompted to enter your password to authenticate. Once you have done this, it will connect to the remote machine and display the system and other necessary information. Finally, it will display the user prompt to manage the Docker machine.

2296-figure11.png

Figure 11. Connecting to the Azure Docker machine

Getting Information about the Docker Machine

Once you have conntected to the new Azure Docker VM server, you can validate the target server by executing some of the commands specified below:

sudo docker version displays both client and server version information.

2296-figure12.png

Figure 12. Azure Docker version information

sudo docker info displays information like Docker kernel version, operating system version, hardware specifications, etc.

2296-figure13.png

Figure 13. Azure Docker hardware, kernel and other information

Installing a Container

sudo docker run hello-world will cause Docker to try and find the hello-world image locally and when it cannot find anything, it will instead pull from the Docker hub. It then creates a new container from the image and finally runs the command. The Docker Daemon streams the output ‘Hello from Docker’ to the Docker Client which then sends it to the user terminal.

2296-figure14.png

Figure 14. Running hello-world application inside Docker

Communicating with the remote Docker Machine

Multiple users convenient tools can be used to connect to the remote Docker machine. By default, Azure Docker configures SSH on TCP port 22 to allow clients to connect and manage them remotely.

Below are the Windows clients used to connect to the Docker VM

  • Docker Quick Start Terminal installs as part of the Docker installation. It has been discussed above.
  • Command Prompt (cmd.exe) needs Dockers for Windows to be installed and also needs the environmental path pointing to the Dockers for Windows installation bin folder. Once the environmental path is set, then the SSH discussed above can be used to connect to the remote Docker machine.

  • PowerShell also needs Dockers for Windows to be installed and needs the environmental path to point to the Dockers for Windows installation bin folder. Below is the command to set environmental path on PowerShell console. Once the environmental path is set then the SSH discussed above can be used to connect to the remote Docker machine.

  • The Putty tool needs some configuration and private key to connect to Azure Docker VM. Dockers for Windows installation generates a private key; however, it is not compatible with Putty. Putty Key generator can be used to convert Dockers for Windows private key file to the format needed by putty.

To convert the same, start PuTTygen.exe tool and then click on Load button to select the Docker private key file at %USERPROFILE%\.ssh\id_boot2docker. Then click on Save private key button to save it to the local drive. This new private key can be used by PuTTY.exe to connect to the remote Docker machine.

2296-figure15.png

Figure 15. Docker Key conversion to Putty format using PuTTY Key Generator

Start putty.exe and provide the Azure DNS host name and select connection type as SSH.

2296-figure16.png

Figure 16. PuTTY Configuration by providing Destination Servername and connection type

Scroll down at the category hierarchy and click on Data. Then specify the Auto-login username.

2296-figure17.png

Figure 17. Configuring PuTTY with user name to send to the server when connected

Further scroll down the Category to expand SSH under connection and select Auth. Click on the Browse button under Private key file for authentication: and select the new Private Key file generated using Putty key gen. Click on the Open button to connect to the target Azure Docker VM.

2296-figure18.png

Figure 18. Configuring PuTTY with Private Key file for authentication

It prompts for PuTTY Security Alert, click on Yes to continue. Finally, putty connects to the Azure Docker VM and prompts for authentication. Key in the password and press Enter to connect to the remote machine.

2296-figure19.png

Figure 19. PuTTY connection established with the remote Azure Docker VM

Installing MySQL image into the Azure Docker VM

To take this to the next level, let us download and install the latest MySQL image from the Docker Repository on the Azure Docker VM and connect to it from the application. MySQL is the open source database which provides good performance, reliability and is easy to use. Docker Repository has multiple supported version of MySQL images which can be downloaded. In our scenario, let us download and install the latest version MySQL 5.7 image into the Azure Docker VM. Below are the steps to perform this.

  1. Connect to the Azure Docker VM Extension using Docker Client
  2. Run a $docker pull mysql:latest command to download the latest version of MySQL image into the Docker VM. Specifying the key word mysql:latest in the command will look for the latest image in the Docker repository and download the same.

    2296-figure20.png

    Figure 20. Downloading and installing latest MySQL image on the Docker machine.

  3. The downloaded MySQL Latest image can then be validated by using this command: $Docker Images

    2296-figure21.png

    Figure 21. Validating the downloaded MySQL images on Docker

  4. Now start the MySQL container using the command shown below. Provide the name of the MySQL container, MySQL root password and the key word mysql:latest to use the latest MySQL available image in the Docker VM.

    2296-figure22.png

    Figure 22. Creating the new container using MySQL images on Docker

  5. Then, we could also further validate the MySQL container using the command: $ docker ps – this command lists the running MySQL containers and provides important details like Container ID, standard port number (3306) to communicate, Container name, etc.

    2296-figure23.png

    Figure 23. Validating new container creating using MySQL images on Docker

Security Issues

Docker Containers offer some great features and flexibility. However, it is important to consider the security aspect to protect the application environment from possible threats. Dockers require root privileges on the host machine to run containers, hence it is necessary to make sure that only authorized persons are handling Dockers daemon. Use root permission under Dockers carefully and try to run services under non-root privileges whenever possible. Download Dockers images only from the trusted source whenever necessary.

Further Reading