Reorganising my Docker images and setting up automated builds

Authored on
Changed
Tags

Yesterday I reorganised my Docker images. This article will summarise the changes and explain why such changes were made.

It was a comment from Ines Wallon about the fact that the hub Docker can be set to build Docker images automatically that triggered this reorganisation.

Why?

Until now, I used to rebuild my images from time to time when I made changes. This had the following disadvantages:

  • it takes time: manually running the build commands for the different images, one image after another, then uploading them is time-consuming. Especially uploading to the hub, having a fairly slow internet connection,
  • less good quality: with this reorganisation, I was able to realise that the PHP image (local) used as a base for my images was not up to date (Debian 8 instead of Debian 9).

With the automated build:

  • it's easier to have different versions of an image and all of them up to date,
  • built images have their parent updated,
  • no more uploading needed.

Docker automated build

When I first started using Docker, I hadn't looked at the automated build options because:

  • I was new to Docker and so had enough other points to explore
  • working manually at first allows you to see the different problems and better understand why there may be this or that organisation in the official or personal images of other people using Docker.
  • I'd been under the impression that this was only reserved for the Premium account.

For free accounts, there is a limit of one simultaneous build.

This isn't annoying for my use, I don't have 40 images to update 10 times daily.

Automated build doesn't necessarily mean automatically triggered build.

An automated build means creating a Docker image by filling in a Git repository (Github or Bitbucket) and specifying:

  • the Git branch to use,
  • the path to the Dockerfile,
  • the tag of the Docker image produced.

With these elements, we then have a button to trigger the build.

We can of course then have builds triggered automatically in several ways:

  • by linking Docker repositories : if images are built on another repository, this triggers the build on that repository.
  • by activating triggers (Webhooks) and then informing Github, for example, of the URLs and access keys to trigger these Webhooks.

Note: it's not possible to transform a Docker repository into a repository with automated build (unless I didn't see the option), you're forced to delete the manual repository then redo one in automated build mode if you want to keep the same image name.

Similarly once an automated build is plugged into a Git repository, there is no option to change Git repositories.

Reorganisation

Image tree, https://github.com/FlorentTorregrosa/docker-drupal-project/tree/8.x/docker-images:

Before:

  • php-apache-dev: docker-drupal-dev:php7, image for development, PHP 7.1 with Apache
    • the tag does not indicate PHP 7.1
    • docker in image name
  • php-apache: docker-drupal:php7, image to run Drupal, PHP 7.1 with Apache
    • the tag does not indicate PHP 7.1, nor Apache
    • docker in the image name
  • php-dev: drupal-php-dev, image to run Drupal with development tools, PHP 7.0 under Alpine
    • more up to date
    • the tag did not include any information
  • php: drupal-php, image to run Drupal, PHP 7.0 under Alpine
    • more up to date
    • the tag did not include any information
  • varnish: docker-varnish:latest, Varnish 4 under Alpine
    • no tag
    • docker in image name

After:

  • drupal-php:
    • 7.1
      • apache
    • 7.2
      • apache
  • drupal-php-dev:
    • 7.1
      • apache
    • 7.2
      • apache
  • varnish:
    • 4
      • alpine
    • 5
      • alpine

drupal-php:

  • image to run Drupal,
  • resume the tree structure of the official PHP image,
  • ready to have FPM versions if needed,
  • root folder corresponding to the image name.

drupal-php-dev:

  • image to run Drupal with development tools,
  • resuming the tree structure of the official PHP image,
  • ready to have FPM versions if needed,
  • root folder corresponding to the image name.

varnish:

  • image to have a Varnish,
  • several versions of Varnish now. No Varnish 6 because at the moment only present on the edge version of Alpine,
  • ready to have versions on other Linux distributions if needed,
  • root folder corresponding to the name of the image.

Deleted the PHP FPM versions, as they are no longer up to date and no longer used. I'll do them again when I need them.

I also took the opportunity to remove "docker" from the image names because:

  • if they're Docker images, we know it has something to do with Docker
  • it seems to me I've seen that the Docker company is careful about using the word Docker to reserve it for official uses.

The following Git and Docker repositories will be/have been removed:

Deleted because obsolete and I don't like leaving obsolete repositories lying around, it confuses readability. It's confusing when you go through public repositories to see how things are done and realise they're no longer in use.

In terms of setting up automated builds, there is now creation of images with tags in relation to the corresponding Dockerfile in the tree.

There's also build triggering for varnish and drupal-php images when pushed to the https://github.com/FlorentTorregrosa/docker-drupal-project repository.

The build of the drupal-php-dev image is triggered automatically when the drupal-php image is built. And since drupal-php is built with every push to the repository, even if it's not changed, this will update drupal-php-dev.

Side effect:

When pushing on https://github.com/FlorentTorregrosa/docker-drupal-project, it triggers the image builds, even if no files in the docker-images folder have been touched. And that's even when pushing, on a branch other than 8.x.

Thankfully, this isn't critical, and at least it will refresh images even if their Dockerfile hasn't been touched.

So now I understand better why I see the files needed for the Docker images in separate Git repositories from the project repositories and each image in its own repository.

Maybe I'll (re-)come to this organisation. But I had wanted to put the Docker images in the https://github.com/FlorentTorregrosa/docker-drupal-project repository so that when a configuration change was needed and there was both a change in an image and a change in the configuration files, this would be indicated via a single commit for greater readability.

To see, if I'll only release the docker-images folder in a separate repository or if I'll switch to one repository per image.

Problems encountered

With the switch to Debian 9, because the latest versions of the official php:7.1-apache and php:7.2-apache images are on Debian 9, I had the following problems:

  • the libpng12-dev package no longer exists, using libpng-dev,
  • the gnupg package is not already installed, I had to add it otherwise it would cause problems when installing NodeJs in the development image.

Building PHP 7.2 versions of the images helped to remember that the PHP mcrypt library marked as deprecated in PHP 7.1, is completely removed in PHP 7.2. Hence its removal from images.

Note: I took the opportunity to make a pass from Piwik to Matomo, now that there's an official image for Matomo.

Next steps

May be added Readme.md in the subfolders of each image to document how they work. Currently the operation is visible via use in the https://github.com/FlorentTorregrosa/docker-drupal-project repository, but if you go to the Docker hub page for the images this may not be obvious.

Documentation:

Comments

Add new comment

About text formats