Automated deployment of MediaWiki

As a contributor to Meza, and developer of QualityBox, we've been closely following the efforts in the MediaWiki community around easier/automated deployments of MediaWiki. Actually, it's a goal of the MWStakeholders Group. And, it's a crucial aspect of the whole 2017 Wikimedia Technology Department Goals since at least 2018. Here is a list of some of the efforts and projects within and outside the WMF for automating deployments of MediaWiki:

• Our own efforts: MediaWiki on Kubernetes

WMF

Dev Images

WMF uses "Dev Images" in Release Engineering for their CI/CD pipelines https://gerrit.wikimedia.org/r/q/project:releng/dev-images+ There used to be a Gerrit repo for "dev-images", but it has now moved to GitLab https://gitlab.wikimedia.org/repos/releng/dev-images The repo is about Docker images for local MediaWiki development. Can be used with Docker Compose and Kubernetes. Published to the WikiMedia Docker Registry, e.g. https://dockerregistry.toolforge.org/wikimedia-buster/tags/

There was a 2019 issue / pull request to create a MySQL/MariaDB image instead of Bitnami image(s) for MySQL/MariaDB https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/38/diffs?commit_id=5a437dce4cd1921ce023b223541a962f1a520340

For Distributors

• Per RFC, distributors can override DefaultSettings.php with includes/PlatformSettings.php
• OPEN task: Allow DefaultSettings.php be overridden by environment variables - Important for Kubernetes

MediaWiki with Kubernetes

• https://gerrit.wikimedia.org/r/plugins/gitiles/releng/local-charts/ Gerrit repo for using Minikube and Helm charts to define MediaWiki local development environments. This was just an experiment?

See https://wikitech.wikimedia.org/wiki/MediaWiki_On_Kubernetes in Wikitech

The production code/repo for WMF's MediaWiki on Kubernetes is in the operations Deployment Charts repo. Mirror on GitHub https://github.com/wikimedia/operations-deployment-charts

One relevant project is the Prod Kubernetes Component and Prod Kubernetes workboard in Phabricator. These are organized under the SRE team. Wikimedia Site Reliability Engineering

MediaWiki on Docker

Docker Library Docs specifies how Docker official images are made (ie. all the official images on Docker Hub). There is specific information for MediaWiki at https://github.com/docker-library/docs/tree/master/mediawiki These images are what users get when using MediaWiki Docker (see section below for further details).

Compared to a "developer" process, you can see what is going into WMF production images at the Wikimedia base docker images for production repository by lavagetto (Giuseppe Lavagetto)

MediaWiki Docker

MediaWiki-Docker is distributed with MediaWiki core. Just follow the directions in DEVELOPERS.md to create your Docker-based development environment. The images, and more information can be found on the MediaWiki page in DockerHub. The actual upstream repository (where images are built) comes from a WMF repo mirror on GitHub Wikimedia Git repo of the Docker official image for MediaWiki Note that there are at least 3 variants you can choose from: Apache, FPM and FPM-Alpine.

There is a compose file https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+/HEAD/docker-compose.yml which can be overridden to specify variants, "recipes" and options. While the Docker Hub image just derives from PHP, the Compose file specifically derives from the Debian Linux PHP image.

MediaWiki Docker Dev

MediaWiki-Docker-Dev (Addshore's GitHub repo) for using Docker and MediaWiki (Developers) See also: MediaWiki API library It's a library for making it easier to work with the MediaWiki API. Good to know/use/contribute. One member of a group of libraries to interact with WikiData, etc. (docs) Addwiki repos are by Addshore, Jeroen DeDauw and Sam Wilson.

Node Services

• MediaWiki Node Services Run several MediaWiki nodejs services (RESTBase, Parsoid) in a single docker container. Useful for small / low-memory installs. By WMF's d00rman (Marko Obrovac - Croatia) and gwicke (Gabriel Wicke - Menlo Park)

Lex Sulzer

• Set up BlueSpice MediaWiki Enterprise Distribution by lexsulzer (Lex Sulzer at Dataspects - Zurich, Switzerland)
• Dataspects' Easy System Cloning

Configuration Management and Deployment at WMF

It should be noted that the WMF uses Puppet for configuration management (See the operations/mediawiki-config repo), and Scap3 (which is used in SWAT) (obsolete) for deployments. There is the Cloud VPS which is powered by wp:OpenStack for projects related to the Wikimedia movement. The environment includes access to a variety of data services. Cloud VPS is meant to make it easier for developers and system administrators to try out improvements to Wikimedia infrastructure (including MediaWiki), power research and analytics, and host projects that are not viable in the Toolforge environment. Virtual machines running on Cloud VPS are subject to the instance lifecycle.

Puppet

• https://wikitech.wikimedia.org/wiki/Puppet
• https://wikitech.wikimedia.org/wiki/Puppet_coding

Config Settings

With any automation tooling and deployment, you will inevitably need to refer to the long list of MediaWiki Configuration settings.