GitHub Actions: add two stage MXE container build

Based on ideas from Anton - both the basic building of containers in the first
place as well as the workaround for the 6h build limit.

Because GitHub Actions are limited to 6 hours we split the creation of the MXE
container into two steps and push the intermediary container after stage 1 to
docker hub. Right now each of the steps takes about 3.5 hours, so hopefully
even with changes in the future this will continue to work.

This commit also introduces use of docker hub instead of GitHub's own registry
(since strangely right now GitHub actions cannot run containers from GitHub's
private registry).

In order for this to work, we need to have the docker credentials in secrets in
GitHub. As a result, only people who can create branches in our repository can
easily test changes to the container images. Others can modify the code to use
a different docker hub account and provide those secrets in their own GitHub
account. Not ideal, but of course we cannot allow every pull request to
potentially overwrite docker images in our "official" docker hub account.

Suggested-by: Anton Lundin <glance@acc.umu.se>
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>

6 files changed, 127 insertions, 10 deletions

diff --git a/.github/workflows/mxe-dockerimage-stage1.yml b/.github/workflows/mxe-dockerimage-stage1.yml
new file mode 100644
index 000000000..1452fab38
--- /dev/null
+++ b/.github/workflows/mxe-dockerimage-stage1.yml
@@ -0,0 +1,49 @@
+name: Docker Image CI
+
+on:
+  push:
+    paths:
+    - scripts/docker/mxe-build-container/*
+
+
+jobs:
+  mxe-build-container-stage1:
+    runs-on: ubuntu-latest
+    env:
+      # IMPORTANT: use the second digit to test new versions, each new official build should be n.0 with the first one being 1.0
+      #            as you test changes toward a new release, call those 1.1, 1.2, 1.3, etc
+      #            VERSION is just that version presented as a string constant
+      #
+      VERSION: ${{ '0.5' }}
+
+    steps:
+    - uses: actions/checkout@v1
+
+      # Because, reasons, we can't really do anything in YAML, so do this crazy shell callout thingy in order to assemble
+      # sane variables to use later in this job - if someone knows an easier way to do this, please let me know!
+      # This abomination below assembles the docker image NAME and appends ".stage1" to the VERSION
+    - name: set env
+      run: |
+        v=${{ env.VERSION }}
+        s=".stage1"
+        echo "::set-env name=NAME::subsurface/mxe-build-container:${v}${s}"
+
+    - name: Build and Publish stage 1 Docker image to Dockerhub
+      uses: elgohr/Publish-Docker-Github-Action@master
+      with:
+        name:       ${{ env.NAME }}
+        username:   ${{ secrets.DOCKER_USERNAME }}
+        password:   ${{ secrets.DOCKER_PASSWORD }}
+        dockerfile: 'Dockerfile-stage1'
+        workdir:    './scripts/docker/mxe-build-container/'
+
+    - name: Trigger stage 2 to run
+      # this triggers a dispatch event in our own repository - this allows us to work around the
+      # 6h runtime max (as on a two core system the total build time of this container is closer to 7h)
+      # we use the event_type to pass the version that we are building to the second stage
+      run: |
+        curl -XPOST -H 'authorization: token ${{ secrets.ACCESS_TOKEN }}' \
+                    -H "Accept: application/vnd.github.everest-preview+json" \
+                    -H "Content-Type: application/json" \
+                    https://api.github.com/repos/subsurface-divelog/subsurface/dispatches \
+                    --data '{"event_type": "${{ env.VERSION }}" }'
diff --git a/.github/workflows/mxe-dockerimage-stage2.yml b/.github/workflows/mxe-dockerimage-stage2.yml
new file mode 100644
index 000000000..531982a70
--- /dev/null
+++ b/.github/workflows/mxe-dockerimage-stage2.yml
@@ -0,0 +1,34 @@
+name: Docker Image CI stage 2
+
+# trigger this second stage via a repository dispaych event
+# the version that we are building is passed in via the event type that is available to us here
+# as github.event.action
+
+on: repository_dispatch
+
+jobs:
+  mxe-build-container-stage2:
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v1
+
+      # Grab the version from the event name that we were triggered by and add ".stage1" to find the docker image to start FROM
+      # And create the NAME of the final docker image
+    - name: set env
+      run: |
+        v=${{ github.event.action }}
+        s=".stage1"
+        echo "::set-env name=VERSION::${v}${s}"
+        echo "::set-env name=NAME::subsurface/mxe-build-container:${v}"
+
+    - name: Build and Publish stage 2 Docker image to Dockerhub
+      uses: elgohr/Publish-Docker-Github-Action@master
+      with:
+        name:       ${{ env.NAME }}
+        username:   ${{ secrets.DOCKER_USERNAME }}
+        password:   ${{ secrets.DOCKER_PASSWORD }}
+        dockerfile: 'Dockerfile-stage2'
+        workdir:    './scripts/docker/mxe-build-container/'
+        buildargs:  VERSION
+
diff --git a/scripts/docker/mxe-build-container/Dockerfile b/scripts/docker/mxe-build-container/Dockerfile-stage1
index b9def62bc..501bfe6ce 100644
--- a/scripts/docker/mxe-build-container/Dockerfile
+++ b/scripts/docker/mxe-build-container/Dockerfile-stage1
@@ -2,11 +2,12 @@
 # docker build -t boret/myimage:0.1 --build-arg=mxe_sha=123ABC456 .
 #
 
+# Start from Ubuntu
 From ubuntu:18.04
 ARG mxe_sha=master
 ENV _ver=${mxe_sha}
-RUN mkdir -p /win
-ADD settings.mk /win
+
+# update and set up the packages we need for this cross build
 RUN apt-get update  &&  apt-get upgrade -y
 RUN apt-get install -y \
     autoconf \
@@ -43,6 +44,10 @@ RUN apt-get install -y \
     xz-utils \
     lzip \
     scons
+
+# copy over the partial MXE settings (simply missing QtWebKit so we don't exceed the 6h limit)
+RUN mkdir -p /win
+ADD settings-stage1.mk /win/settings.mk
 RUN cd /win ; git clone git://github.com/mxe/mxe ; \
     cd mxe ; \
     git checkout ${_ver} ;
@@ -51,10 +56,3 @@ RUN cd /win/mxe ; \
     make -j 6 2>&1 | tee build.log ;
 RUN cd /win/mxe ; \
     make MXE_TARGETS=i686-w64-mingw32.static glib -j 6 2>&1 | tee -a build.log ;
-RUN cd /win/mxe ; \
-    mkdir -p neolit ; cd neolit ; git clone -b wip/win git://github.com/qt/qtconnectivity
-RUN cd /win/mxe/neolit/qtconnectivity ; \
-    PATH=/win/mxe/usr/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin /win/mxe/usr/i686-w64-mingw32.shared/qt5/bin/qmake qtconnectivity.pro ; \
-    PATH=/win/mxe/usr/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin make -j 6 ; \
-    PATH=/win/mxe/usr/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin make install ;
-RUN rm -rf /win/mxe/pkg
diff --git a/scripts/docker/mxe-build-container/Dockerfile-stage2 b/scripts/docker/mxe-build-container/Dockerfile-stage2
new file mode 100644
index 000000000..a9ae812a5
--- /dev/null
+++ b/scripts/docker/mxe-build-container/Dockerfile-stage2
@@ -0,0 +1,21 @@
+ARG VERSION
+
+# Start from the container that we created in stage 1
+From subsurface/mxe-build-container:$VERSION
+
+# this time the settings include QtWebKit - since everything else was built
+# during the first stage, QtWebKit will be the only thing that gets built
+ADD settings-stage2.mk /win/mxe/settings.mk
+RUN cd /win/mxe ; \
+    make -j 2 2>&1 | tee build.log ;
+RUN cd /win/mxe ; \
+    make MXE_TARGETS=i686-w64-mingw32.static glib -j 6 2>&1 | tee -a build.log ;
+
+# manually build the Win BLE version of QtConnectivity (we can drop this with Qt 5.14)
+RUN cd /win/mxe ; \
+    mkdir -p neolit ; cd neolit ; git clone -b wip/win git://github.com/qt/qtconnectivity
+RUN cd /win/mxe/neolit/qtconnectivity ; \
+    PATH=/win/mxe/usr/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin /win/mxe/usr/i686-w64-mingw32.shared/qt5/bin/qmake qtconnectivity.pro ; \
+    PATH=/win/mxe/usr/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin make -j 6 ; \
+    PATH=/win/mxe/usr/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin make install ;
+RUN rm -rf /win/mxe/pkg
diff --git a/scripts/docker/mxe-build-container/settings-stage1.mk b/scripts/docker/mxe-build-container/settings-stage1.mk
new file mode 100644
index 000000000..5563f259a
--- /dev/null
+++ b/scripts/docker/mxe-build-container/settings-stage1.mk
@@ -0,0 +1,15 @@
+# This is a template of configuration file for MXE. See
+# index.html for more extensive documentations.
+
+# This variable controls the number of compilation processes
+# within one package ("intra-package parallelism").
+JOBS := 2
+
+# This variable controls the targets that will build.
+MXE_TARGETS :=  i686-w64-mingw32.shared
+
+# The three lines below makes `make` build these "local packages" instead of all packages.
+LOCAL_PKG_LIST := qtbase qtconnectivity qtdeclarative qtimageformats qtlocation qtmultimedia qtquickcontrols qtquickcontrols2 qtscript qtsvg qttools qttranslations qtwebview libxml2 libxslt libusb1 libgit2 nsis curl libzip libftdi1
+.DEFAULT local-pkg-list:
+local-pkg-list: $(LOCAL_PKG_LIST)
+
diff --git a/scripts/docker/mxe-build-container/settings.mk b/scripts/docker/mxe-build-container/settings-stage2.mk
index aadfc4c6f..452961338 100644
--- a/scripts/docker/mxe-build-container/settings.mk
+++ b/scripts/docker/mxe-build-container/settings-stage2.mk
@@ -3,7 +3,7 @@
 
 # This variable controls the number of compilation processes
 # within one package ("intra-package parallelism").
-JOBS := 6
+JOBS := 2
 
 # This variable controls the targets that will build.
 MXE_TARGETS :=  i686-w64-mingw32.shared