Blubber

Wikimedia Release Engineering's Blubber: a BuildKit frontend

Blubber (GitLab repo) (GitHub mirror repo) is a BuildKit frontend developed in Go^[1] by Wikimedia Release Engineering for building application container images. Its core functionality lies in defining container images from a minimal set of declarative constructs written in Yaml.

Key features[edit]

Composability, Determinism, Cache Efficiency, and Secure Defaults: Blubber prioritizes these characteristics in its image building process.
Integration with BuildKit: Blubber functions as a frontend for BuildKit, allowing it to work with buildctl and docker build commands.
Multi-platform Builds and Image Attestations: It supports building images for multiple platforms simultaneously and generating attestations such as Software Bill of Materials (SBOM) and provenance metadata.
YAML-based Configuration: Image definitions are written in YAML, promoting readability and ease of management.

Blubber work-alikes[edit]

While Blubber offers a specialized approach to container image building, several tools provide similar functionalities, particularly in the realm of building images from declarative specifications or within container environments:

Buildah: A command-line tool that can build images from Dockerfiles and create container images from scratch or by using an existing image as a starting point. It offers fine-grained control over the image building process and can be used to build images layer by layer.
Kaniko: A tool designed for building container images within a Kubernetes cluster. It doesn't require a Docker daemon and can build images from Dockerfiles and other sources.
Docker Buildx: An extension for Docker that leverages BuildKit's advanced features, including multi-platform builds and enhanced caching capabilities.
Ko: Focuses on building Go applications and packaging them into container images, simplifying the process of creating images for Go projects.
Pack: Separates the build container from the final run container, allowing for more secure and optimized images.
Apko: A build tool that strictly enforces the use of APK packages for image content, enabling the creation of minimal and secure base images, according to Chainguard^[2].

These tools, similar to Blubber, aim to provide more efficient, secure, and controlled ways of building container images compared to traditional Dockerfile-based approaches, catering to various use cases and preferences within the container ecosystem.

README[edit]

(the content below is embedded from the source repo: https://github.com/wikimedia/blubber/blob/main/README.md)

Blubber is a BuildKit frontend for building application container images from a minimal set of declarative constructs in YAML. Its focus is on composability, determinism, cache efficiency, and secure default behaviors.

Requirements

To use Blubber, you'll need:

buildkitd (included with Docker version 23.0 or greater)
A BuildKit client, any of the following:
- Docker Buildx (recommended, see building)
- Docker Compose (see compose usage)
- Docker Legacy Builder (docker build, Docker 23.0 or greater)

Usage

Configuration

A Blubber configuration starts with a syntax line and a version declaration.

# syntax = docker-registry.wikimedia.org/repos/releng/blubber/buildkit:v1.6.0
version: v4
variants:
  my-variant:
    base: docker-registry.wikimedia.org/bookworm

The syntax is a reference to a container image that BuildKit will use to process the configuration (known as a BuildKit frontend).

The version is the major version of the configuration schema accepted by Blubber. It rarely changes (only with breaking changes to the schema) and Blubber will let you know if it isn't right.

A configuration also includes one or more variants (akin to dockerfile stages).

Once you are ready to write your own configuration, see the examples for build patterns that are possible with Blubber and the configuration reference for documentation about all available fields.

Building a variant

You build a variant from a Blubber configuration using docker buildx build just as you would build a stage from a Dockerfile.

$ docker buildx build -f blubber.yaml --target my-variant --load .

Note the --load option will add the resulting image to Docker's image store so it can be used with docker run and other docker subcommands. See the docker buildx build CLI reference for all available output options including those used to publish images to remote registries.

Compatibility

Docker Bake

Bake is a feature of Docker Buildx for building multiple targets at once from any number of build contexts and configurations. It also lets you codify your outputs, tags, and other parameters otherwised passed in via individual build commands.

Blubber configuration can be referenced from Bake configuration just as you would reference a dockerfile.

# bake.hcl
group "default" {
  targets = ["my-variant"]
}

target "my-variant" {
  context = "."
  dockerfile = "blubber.yaml"
  outputs = ["type=registry"]
  tags = ["an.example/registry/my-project/my-variant:stable"]
}

You then build your target(s) with buildx bake.

$ docker buildx bake -f bake.hcl

Docker Compose

In the same way Blubber files can be referenced from Bake configuration, they can also be used with Docker Compose.

In the build section of your compose configuration, simply specify the path to your Blubber configuration file just as you would a dockerfile.

services:
  my-service:
    build:
      context: .
      dockerfile: blubber.yaml
      target: my-variant

Predefined build arguments

Blubber allows the same predefined build arguments that Docker allows for setting/determining information about the build environment such as OS and architecture, and available proxies.

Building for multiple platforms

Blubber supports building for multiple platforms at once and publishing a single manifest index for the given platforms (aka a "fat" manifest). See the OCI Image Index Specification for details.

See the documentation of buildx build for details on how to specify your target platforms.

Note that your build process must be aware of the environment variables set for multi-platform builds in order to perform any cross-compilation needed.

Image attestations

Blubber supports the creation and export of Software Bill of Materials (SBOM) and provenance metadata in the form of in-toto attestations.

Attestations are exported in the form of image manifests that live alongside your images within the same manifest list/index. See the upstream BuildKit documentation on image attestation storage for details.

See the documentation of buildx build for details on how to enable SBOM and provenance metadata creation during a build.

Docs[edit]

Roll up your sleeves[edit]

Have at it: https://github.com/freephile/meza/issues/88

See Also[edit]

Werf for Kubernetes is a higher-level general-purpose approach. Blubber might be useful in a "first phase" to gain excellence in defining a Dockerfile. Once that is successful, we could implement a "second phase" of application maturity where Werf is used to package the application and deploy it to multiple environments.

Blubber and werf are both tools for building container images, but they differ in their scope and ecosystem integration: Blubber is a specialized, Wikimedia-specific tool focused on declarative image building, while werf is a comprehensive, general-purpose CLI tool for implementing full CI/CD with Kubernetes.

Blubber[edit]

Primary Function: Blubber is a BuildKit frontend designed to build application container images using minimal, declarative YAML configurations.
Use Case: It is specifically used by Wikimedia CI to test code and create production-ready Docker images, offering a high degree of control, determinism, and cache efficiency within their specific infrastructure needs.
Scope: It is a domain-specific tool, tightly integrated into the Wikimedia ecosystem and designed for the specific needs of building images for their services.

werf[edit]

Primary Function: Werf is a command-line interface (CLI) tool for implementing a full GitOps-oriented CI/CD methodology for applications deployed to Kubernetes.
Use Case: It provides a complete solution for building, publishing, and deploying applications. It manages the entire lifecycle from source code to running application in Kubernetes, including capabilities like building images, deploying via a Helm alternative called Nelm, and managing cleanup policies.
Scope: It is a general-purpose, open-source project under the Cloud Native Computing Foundation (CNCF) umbrella, designed for a broad range of cloud-native development teams and the wider Kubernetes community.

Comparison Summary[edit]

Feature	Blubber (Wikimedia)	werf (CNCF Sandbox)
Primary Goal	Declarative, deterministic container image building	Full CI/CD implementation for Kubernetes (build, deploy, etc.)
Ecosystem	Integrated within Wikimedia infrastructure	General-purpose, CNCF project for Kubernetes
Configuration	YAML-based, minimal declarative constructs	Uses and integrates with and (via Nelm)
Deployment	Handles image building; deployment handled separately by Wikimedia's systems	Handles image building and deployment to Kubernetes as part of its core functionality

In essence, while both involve container image creation, Blubber is a specialized piece of the image building puzzle used internally by Wikimedia, whereas werf offers a holistic, end-to-end CI/CD solution for the general cloud-native market.== References ==