In this page, you’ll learn everything you need to know about Docker architecture:
To master Docker you need to start with a clear understanding of its architecture, and how each component of the Docker system interacts with the others. Let’s look at Docker and its architecture and its various components in detail. Let us first compare containers to their closest cousin – Virtual Machines.
Containers vs. Virtual Machines
When compared to Virtual machines, the Docker platform moves up the abstraction of resources from the hardware level to the Operating System level. This allows for the realization of the various benefits of Containers e.g. application portability, infrastructure separation, and self-contained microservices. In other words, while Virtual Machines abstract the entire hardware server, Containers abstract the Operating System kernel. This is a whole different approach to virtualization and results in much faster and more lightweight instances .
Advantages
The main advantages of Docker are:
- Resource Efficiency: Process level isolation and usage of the container host’s kernel is more efficient when compared to virtualizing an entire hardware server.
- Portability: All the dependencies for an application are bundled in the container. This means they can be easily moved between development, test, and production environments.
- Continuous Deployment and Testing: The ability to have consistent environments and flexibility with patching has made Docker a great choice for teams that want to move from waterfall to the modern DevOps approach to software delivery.
The Docker Engine
First, let us look take a look at Docker Engine and its components so we have a basic idea of how the system works. Docker Engine allows you to develop, assemble, ship, and run applications using the following components:
- Docker Daemon: A persistent background process that manages Docker images, containers, networks, and storage volumes. The Docker daemon constantly listens for Docker API requests and processes them.
- Docker Engine REST API: An API used by applications to interact with the Docker daemon; it can be accessed by an HTTP client.
- Docker CLI: A command line interface client for interacting with the Docker daemon. It greatly simplifies how you manage container instances and is one of the key reasons why developers love using Docker.
Now that we see how the different components of the Docker Engine are used, let us dive a little deeper into the architecture.
Implementation
Docker is available for implementation across a wide range of platforms:
- Desktop: Mac OS, Windows 10.
- Server: Various Linux distributions and Windows Server 2016.
- Cloud: Amazon Web Services, Google Compute Platform, Microsoft Azure, IBM Cloud, and more.
Docker Architecture
The Docker architecture uses a client-server model and comprises of the Docker Client, Docker Host, Network and Storage components, and the Docker Registry / Hub. Let’s look at each of these in some detail.
Docker Client
The Docker client enables users to interact with Docker. The Docker client can reside on the same host as the daemon or connect to a daemon on a remote host. A docker client can communicate with more than one daemon. The Docker client provides a command line interface (CLI) that allows you to issue build, run, and stop application commands to a Docker daemon.
The main purpose of the Docker Client is to provide a means to direct the pull of images from a registry and to have it run on a Docker host. Common commands issued by a client are:
docker build
docker pull
docker run
DockerHost
The Docker host provides a complete environment to execute and run applications. It comprises of the Docker daemon, Images, Containers, Networks, and Storage. As previously mentioned, the daemon is responsible for all container-related actions and receives commands via the CLI or the REST API. It can also communicate with other daemons to manage its services. The Docker daemon pulls and builds container images as requested by the client. Once it pulls a requested image, it builds a working model for the container by utilizing a set of instructions known as a build file. The build file can also include instructions for the daemon to pre-load other components prior to running the container, or instructions to be sent to the local command line once the container is built.
Docker Objects
Various objects are used in the assembling of your application. The main requisite Docker objects are:
Images
Images are a read-only binary template used to build containers. Images also contain metadata that describe the container’s capabilities and needs. Images are used to store and ship applications. An image can be used on its own to build a container or customized to add additional elements to extend the current configuration. Container images can be shared across teams within an enterprise using a private container registry, or shared with the world using a public registry like Docker Hub. Images are a core part of the Docker experience as they enable collaboration between developers in a way that was not possible before.
Containers
Containers are encapsulated environments in which you run applications. The container is defined by the image and any additional configuration options provided on starting the container, including and not limited to the network connections and storage options. Containers only have access to resources that are defined in the image, unless additional access is defined when building the image into a container. You can also create a new image based on the current state of a container. Since containers are much smaller than VMs, they can be spun up in a matter of seconds, and result in much better server density.
Networking
Docker implements networking in an application-driven manner and provides various options while maintaining enough abstraction for application developers. There are basically two types of networks available – the default Docker network and user-defined networks. By default, you get three different networks on the installation of Docker – none, bridge, and host. The none and host networks are part of the network stack in Docker. The bridge network automatically creates a gateway and IP subnet and all containers that belong to this network can talk to each other via IP addressing. This network is not commonly used as it does not scale well and has constraints in terms of network usability and service discovery.
The other type of networks is user-defined networks. Administrators can configure multiple user-defined networks. There are three types:
- Bridge network: Similar to the default bridge network, a user-defined Bridge network differs in that there is no need for port forwarding for containers within the network to communicate with each other. The other difference is that it has full support for automatic network discovery.
- Overlay network: An Overlay network is used when you need containers on separate hosts to be able to communicate with each other, as in the case of a distributed network. However, a caveat is that swarm mode must be enabled for a cluster of Docker engines, known as a swarm, to be able to join the same group.
- Macvlan network: When using Bridge and Overlay networks a bridge resides between the container and the host. A Macvlan network removes this bridge, providing the benefit of exposing container resources to external networks without dealing with port forwarding. This is realized by using MAC addresses instead of IP addresses.
Storage
You can store data within the writable layer of a container but it requires a storage driver. Being non-persistent, it perishes whenever the container is not running. Moreover, it is not easy to transfer this data. In terms of persistent storage, Docker offers four options:
- Data Volumes: Data Volumes provide the ability to create persistent storage, with the ability to rename volumes, list volumes, and also list the container that is associated with the volume. Data Volumes sit on the host file system, outside the containers copy on write mechanism and are fairly efficient.
- Data Volume Container: A Data Volume Container is an alternative approach wherein a dedicated container hosts a volume and to mount that volume to other containers. In this case, the volume container is independent of the application container and therefore can be shared across more than one container.
- Directory Mounts: Another option is to mount a host’s local directory into a container. In the previously mentioned cases, the volumes would have to be within the Docker volumes folder, whereas when it comes to Directory Mounts any directory on the Host machine can be used as a source for the volume.
- Storage Plugins: Storage Plugins provide the ability to connect to external storage platforms. These plugins map storage from the host to an external source like a storage array or an appliance. A list of storage plugins can be found on Docker’s Plugin page.
Storage Plugins
There are storage plugins from various companies to automate the storage provisioning process. For example,
- HPE 3PAR
- EMC (ScaleIO, XtremIO, VMAX, Isilon)
- NetApp
There are also plugins that support public cloud providers like:
- Azure File Storage
- Google Compute Platform.
Docker Registries
Docker registries are services that provide locations from where you can store and download images. In other words, a Docker registry contains Docker repositories that host one or more Docker Images. Public Registries include Docker Hub and Docker Cloud and private Registries can also be used. Common commands when working with registries include:
docker push
docker pull
docker run
Service Discovery
Service Discovery allows containers to find out about the environment they are in and find other services offered by other containers.
It is an important factor when trying to build scalable and flexible applications.
Conclusion
Now that we have seen the various components of the Docker architecture and how they work together, we can begin to understand the rise in popularity of Docker containers, DevOps uptake and microservices. We can also see how Docker helps simplify infrastructure management by making underlying instances lighter, faster, and more resilient. Additionally, Docker separates the application layer from the infrastructure layer and brings much-needed portability, collaboration, and control over the software delivery chain. Docker is architected for modern DevOps teams, and understanding its architecture will help you get the most out of your containerized applications.
Public Registries
Docker Hub is the main public registry operated by Docker Inc. There are now many other registries like Quay.io, Artifactory, and Google Container Registry.