Docker: Difference between revisions

(9 intermediate revisions by the same user not shown)

Line 6:

It is important to realize that Docker is inherently a '''Linux''' technology.

'''Key aspects of containers'''

;Portability:

:Containers (or at least the image that created it) can be easily moved and run on different machines and environments without modification, ensuring consistent behavior.

;Isolation:

:Each container runs in its own isolated environment, preventing conflicts between different applications and improving security.

;Lightweight:

:Containers share the host operating system's kernel, making them smaller and faster to start than virtual machines.

;Scalability:

:Containers can be easily scaled up or down to meet the demands of an application.

;Automation:

:Containers are often used in conjunction with container orchestration platforms like [[Kubernetes]] to automate application deployment and management.

Line 20:

Line 38:

*https://docs.docker.com/develop/

*https://docs.docker.com/engine/

*https://docs.docker.com/engine/install/

Line 35:

Line 54:

Docker also supports containers storing files in-memory on the host machine. Such files are not persisted. If you’re running Docker on Linux, tmpfs mount is used to store files in the host’s system memory. If you’re running Docker on Windows, named pipe is used to store files in the host’s system memory.</blockquote><br />

That is all you need to know about Docker when it comes to sharing files between the host and the container.

That is all you need to know<ref>Not really. In practice you need to fully understand how the volumes and mounts work to avoid very common pitfalls like the [https://www.joyfulbikeshedding.com/blog/2021-03-15-docker-and-the-host-filesystem-owner-matching-problem.html '''host filesystem owner matching problem'''] In a nutshell, the best approach is to run your container with a UID/GID that matches the host's UID/GID. It can be hard to implement while addressing all caveats. Hongli Lai [https://www.joyfulbikeshedding.com/blog/2023-04-20-cure-docker-volume-permission-pains-with-matchhostfsowner.html wrote a tool to solve this] ([https://github.com/FooBarWidget/matchhostfsowner MatchHostFSOwner])</ref> about Docker when it comes to sharing files between the host and the container.

Volumes are the preferred way to [https://docs.docker.com/get-started/05_persisting_data/ persist data in Docker containers] and services.

Line 42:

Line 61:

Note: '''Docker Desktop''' is simply a fancy GUI '''client''' application that uses virtualization (a Linux Virtual Machine) to bundle the Docker Engine into your host OS.

Note: '''[[Docker Desktop]]''' is simply a fancy GUI '''client''' application that uses virtualization (a Linux Virtual Machine) to bundle the Docker Engine into your host OS.

*Volumes are the right choice when your application requires fully native file system behavior on Docker Desktop. For example, a database engine requires precise control over disk flushing to guarantee transaction durability. Volumes are stored in the Linux VM and can make these guarantees, whereas bind mounts are remoted to macOS or Windows OS, where the file systems behave slightly differently.

Line 56:

Line 75:

Each time you create a Docker container, and connect VSCode to that container, VSCode installs a server into the container so that it can communicate with it. So, there is a small (a couple minutes?) delay while this happens before you can edit or do anything. Best to stop a container and only re-build it when necessary to avoid the delay.

==Docker Images==

~~Bitnami~~ has ~~a [~~https://github.com/~~bitnami~~/~~bitnami~~-~~docker~~-~~mediawiki Docker Image for MediaWiki] Don't use Bitnami. You will thank me later~~.

The primary resource for Docker Images is '''Docker Hub''', but every major cloud provider has some sort of repository for storing and versioning your images. For example, if you're hosting your code on GitHub and using GitHub Actions for CI/CD, there is the GitHub Container Registry (GHCR)<ref>(GitHub now generalizes the discussion to 'packages' since you might be creating Docker OCI images, NPM Node.js packages, or Ruby Gems) https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry</ref>.

== Security ==

Bitnami had a Docker Image for MediaWiki. Don't use Bitnami. You will thank me later.<ref>I told you so. They weren't very good when they were available, and now they're not freely available at all. Bitnami has shifted to a paid offering.</ref>

==Security==

Docker apparently doesn't respect your host firewall by default - leading to the potential for a gaping security hole. This has been a [https://github.com/docker/for-linux/issues/690 reported bug since 2018]. One fix is to [https://www.smarthomebeginner.com/traefik-docker-security-best-practices/#10_Change_DOCKER_OPTS_to_Respect_IP_Table_Firewall set the DOCKER_OPTS] configuration parameter. Another is to add a jump rule to UFW. The bug report links to docs and multiple references.

==~~Future Reading~~==

== Docker Downsides ==

One major negative to the system architecture of Docker is that it relies on a server daemon. '''Unlike''' [[Podman]], Docker's Engine can use up 4GB of RAM just sitting idle. A similar thing happens with WSL2 on Windows <ref>https://news.ycombinator.com/item?id=26897095</ref>

~~#The compose application model https://docs.docker.com/compose/compose-file/02-model/~~

== Future Reading ==

~~#Understand how moby [https://github.com/moby/buildkit buildkit] is integrated with [https://github.com/docker/buildx buildx] (or docker) and use it.~~

~~#Interesting read about docker commit https://adamtheautomator.com/docker-commit/~~

# The compose application model https://docs.docker.com/compose/compose-file/02-model/

# Understand how moby [https://github.com/moby/buildkit buildkit] is integrated with [https://github.com/docker/buildx buildx] (or docker) and use it.

# Interesting read about docker commit https://adamtheautomator.com/docker-commit/

Inspect your running container based on it's container name: docker inspect $(docker container ls | awk '/app2/ {print $1}')

== Docker in Docker ==

Before you get 'fancy' with Docker, be sure to read and understand the Security best practices for Docker

https://cheatsheetseries.owasp.org/cheatsheets/Docker_Security_Cheat_Sheet.html

Containers (unlike virtual machines) share the kernel with the host, therefore kernel exploits executed inside a container will directly hit the host kernel. For example, a kernel privilege escalation exploit ([https://github.com/scumjr/dirtycow-vdso like Dirty COW]) executed inside a well-insulated container will result in root access in the host.

That said, https://devopscube.com/run-docker-in-docker/ presents several use cases and techniques for DinD.

=== Docker In Docker ===

The DinD method is '''Docker inside Docker'''. Docker provides a special Docker container with the <tt>dind</tt> tag which is pre-configured to run Docker inside the container image.

An example of Docker In Docker would be running Docker inside a Docker image on Windows Subsystem for Linux (WSL)

# Install Docker Desktop for Windows.

# Enable the WSL 2 backend in Docker Desktop settings.

# Set WSL 2 as your default version: <code>wsl --set-default-version 2</code> in PowerShell.

# Install a Linux distribution from the Microsoft Store (Ubuntu, for example).

# Open your Linux distribution (WSL 2 instance), and install Docker.

# Add your user to the Docker group to manage Docker as a non-root user: <code>sudo usermod -aG docker $USER</code>.

# Test Docker with: <code>docker run hello-world</code>.

Running Docker inside Docker can lead to some security and functionality issues. It's often better to use the Docker daemon of the host machine. You can do this by mounting the Docker socket from the host into the container. See the "Docker outside of Docker" section.

=== Docker outside of Docker ===

The DooD method, '''Docker outside of Docker,''' uses the docker socket of the host system from inside containers by mounting the host socket into the filesystem of the container.

Try curl to see how different processes can communicate through a socket on the same host: <syntaxhighlight lang="bash">

curl --unix-socket /var/run/docker.sock http://localhost/version

</syntaxhighlight>

While there are benefits to using the DooD method, you are giving your containers full control over the Docker daemon, which effectively is root on the host system. To mitigate these risks, again refer to [https://docs.docker.com/engine/security/ the Security model of Docker], the [https://cheatsheetseries.owasp.org/cheatsheets/Docker_Security_Cheat_Sheet.html Docker Security Cheat Sheet from OWASP] and be sure to run your Docker daemon as an unprivileged user.

=== For security, use Nestybox Sysbox runtime ===

See https://github.com/nestybox/sysbox<nowiki/>

Line 74:

Line 133:

[[Category:Virtualization]]

[[Category:DevOps]]

[[Category:Kubernetes]]

@@ Line 6: / Line 6: @@
 It is important to realize that Docker is inherently a '''Linux''' technology.
+'''Key aspects of containers'''
+;Portability:
+:Containers (or at least the image that created it) can be easily moved and run on different machines and environments without modification, ensuring consistent behavior.
+;Isolation:
+:Each container runs in its own isolated environment, preventing conflicts between different applications and improving security.
+;Lightweight:
+:Containers share the host operating system's kernel, making them smaller and faster to start than virtual machines.
+;Scalability:
+:Containers can be easily scaled up or down to meet the demands of an application.
+;Automation:
+:Containers are often used in conjunction with container orchestration platforms like [[Kubernetes]] to automate application deployment and management.
@@ Line 20: / Line 38: @@
 *https://docs.docker.com/develop/
+*https://docs.docker.com/engine/
 *https://docs.docker.com/engine/install/
@@ Line 35: / Line 54: @@
 Docker also supports containers storing files in-memory on the host machine. Such files are not persisted. If you’re running Docker on Linux, tmpfs mount is used to store files in the host’s system memory. If you’re running Docker on Windows, named pipe is used to store files in the host’s system memory.</blockquote><br />
-That is all you need to know about Docker when it comes to sharing files between the host and the container.
+That is all you need to know<ref>Not really. In practice you need to fully understand how the volumes and mounts work to avoid very common pitfalls like the [https://www.joyfulbikeshedding.com/blog/2021-03-15-docker-and-the-host-filesystem-owner-matching-problem.html '''host filesystem owner matching problem'''] In a nutshell, the best approach is to run your container with a UID/GID that matches the host's UID/GID. It can be hard to implement while addressing all caveats. Hongli Lai [https://www.joyfulbikeshedding.com/blog/2023-04-20-cure-docker-volume-permission-pains-with-matchhostfsowner.html wrote a tool to solve this] ([https://github.com/FooBarWidget/matchhostfsowner MatchHostFSOwner])</ref> about Docker when it comes to sharing files between the host and the container.
 Volumes are the preferred way to [https://docs.docker.com/get-started/05_persisting_data/ persist data in Docker containers] and services.
@@ Line 42: / Line 61: @@
-Note: '''Docker Desktop''' is simply a fancy GUI '''client''' application that uses virtualization (a Linux Virtual Machine) to bundle the Docker Engine into your host OS.
+Note: '''[[Docker Desktop]]''' is simply a fancy GUI '''client''' application that uses virtualization (a Linux Virtual Machine) to bundle the Docker Engine into your host OS.
 *Volumes are the right choice when your application requires fully native file system behavior on Docker Desktop. For example, a database engine requires precise control over disk flushing to guarantee transaction durability. Volumes are stored in the Linux VM and can make these guarantees, whereas bind mounts are remoted to macOS or Windows OS, where the file systems behave slightly differently.
@@ Line 56: / Line 75: @@
 Each time you create a Docker container, and connect VSCode to that container, VSCode installs a server into the container so that it can communicate with it. So, there is a small (a couple minutes?) delay while this happens before you can edit or do anything. Best to stop a container and only re-build it when necessary to avoid the delay.
 ==Docker Images==
-Bitnami has a [https://github.com/bitnami/bitnami-docker-mediawiki Docker Image for MediaWiki] Don't use Bitnami. You will thank me later.
+The primary resource for Docker Images is '''Docker Hub''', but every major cloud provider has some sort of repository for storing and versioning your images. For example, if you're hosting your code on GitHub and using GitHub Actions for CI/CD, there is the GitHub Container Registry (GHCR)<ref>(GitHub now generalizes the discussion to 'packages' since you might be creating Docker OCI images, NPM Node.js packages, or Ruby Gems) https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry</ref>.
-== Security ==
+Bitnami had a Docker Image for MediaWiki. Don't use Bitnami. You will thank me later.<ref>I told you so. They weren't very good when they were available, and now they're not freely available at all.  Bitnami has shifted to a paid offering.</ref>
+==Security==
 Docker apparently doesn't respect your host firewall by default - leading to the potential for a gaping security hole. This has been a [https://github.com/docker/for-linux/issues/690 reported bug since 2018]. One fix is to [https://www.smarthomebeginner.com/traefik-docker-security-best-practices/#10_Change_DOCKER_OPTS_to_Respect_IP_Table_Firewall set the DOCKER_OPTS] configuration parameter. Another is to add a jump rule to UFW. The bug report links to docs and multiple references.
-==Future Reading==
+== Docker Downsides ==
+One major negative to the system architecture of Docker is that it relies on a server daemon. '''Unlike''' [[Podman]], Docker's Engine can use up 4GB of RAM just sitting idle. A similar thing happens with WSL2 on Windows <ref>https://news.ycombinator.com/item?id=26897095</ref>
-#The compose application model https://docs.docker.com/compose/compose-file/02-model/
+== Future Reading ==
-#Understand how moby [https://github.com/moby/buildkit buildkit] is integrated with [https://github.com/docker/buildx buildx] (or docker) and use it.
-#Interesting read about docker commit https://adamtheautomator.com/docker-commit/
+# The compose application model https://docs.docker.com/compose/compose-file/02-model/
+# Understand how moby [https://github.com/moby/buildkit buildkit] is integrated with [https://github.com/docker/buildx buildx] (or docker) and use it.
+# Interesting read about docker commit https://adamtheautomator.com/docker-commit/
 Inspect your running container based on it's container name: docker inspect $(docker container ls | awk '/app2/ {print $1}')
+== Docker in Docker ==
+Before you get 'fancy' with Docker, be sure to read and understand the Security best practices for Docker
+https://cheatsheetseries.owasp.org/cheatsheets/Docker_Security_Cheat_Sheet.html
+Containers (unlike virtual machines) share the kernel with the host, therefore kernel exploits executed inside a container will directly hit the host kernel. For example, a kernel privilege escalation exploit ([https://github.com/scumjr/dirtycow-vdso like Dirty COW]) executed inside a well-insulated container will result in root access in the host.
+That said, https://devopscube.com/run-docker-in-docker/ presents several use cases and techniques for DinD.
+=== Docker In Docker ===
+The DinD method is '''Docker inside Docker'''. Docker provides a special Docker container with the <tt>dind</tt> tag which is pre-configured to run Docker inside the container image.
+An example of Docker In Docker would be running Docker inside a Docker image on Windows Subsystem for Linux (WSL)
+# Install Docker Desktop for Windows.
+# Enable the WSL 2 backend in Docker Desktop settings.
+# Set WSL 2 as your default version: <code>wsl --set-default-version 2</code> in PowerShell.
+# Install a Linux distribution from the Microsoft Store (Ubuntu, for example).
+# Open your Linux distribution (WSL 2 instance), and install Docker.
+# Add your user to the Docker group to manage Docker as a non-root user: <code>sudo usermod -aG docker $USER</code>.
+# Test Docker with: <code>docker run hello-world</code>.
+Running Docker inside Docker can lead to some security and functionality issues. It's often better to use the Docker daemon of the host machine. You can do this by mounting the Docker socket from the host into the container. See the "Docker outside of Docker" section.
+=== Docker outside of Docker ===
+The DooD method, '''Docker outside of Docker,''' uses the docker socket of the host system from inside containers by mounting the host socket into the filesystem of the container.
+Try curl to see how different processes can communicate through a socket on the same host: <syntaxhighlight lang="bash">
+curl --unix-socket /var/run/docker.sock http://localhost/version
+</syntaxhighlight>
+While there are benefits to using the DooD method, you are giving your containers full control over the Docker daemon, which effectively is root on the host system. To mitigate these risks, again refer to [https://docs.docker.com/engine/security/ the Security model of Docker], the [https://cheatsheetseries.owasp.org/cheatsheets/Docker_Security_Cheat_Sheet.html Docker Security Cheat Sheet from OWASP] and be sure to run your Docker daemon as an unprivileged user.
+=== For security, use Nestybox Sysbox runtime ===
+See https://github.com/nestybox/sysbox<nowiki/>
 {{References}}
@@ Line 74: / Line 133: @@
 [[Category:Virtualization]]
 [[Category:DevOps]]
+[[Category:Kubernetes]]