Improving CI for your Magento module

Recently we hit a problem in our Force Login Magento 2 module: The module was not compatible anymore with Magento 2.1 due to a change in a constructor of a Magento base class we extended. This broke the DI configuration which in turn meant the module was not installable in a Magento 2.1 project any more. We were not able to spot this problem early on as our Travis build did only run against the latest Magento version 2.2. Inspired by this blog post of the heidelpay developers, I began to restructure our Travis build.

We make use of the Build Matrix feature of Travis CI. In case you are using GitLab CI you might be interested to read this blog post on how to configure matrix builds for GitLab CI.

Our build matrix is configured like this: We run the builds on PHP 7.0 and PHP 7.1 for both Magento 2.1 and Magento 2.2. Magento 2.1 is still compatible with PHP 5.6 but since we use the latest version of PHPUnit, we are not able to run those tests on PHP 5.6 - given that 5.6 is not supported any more, you should not use it anyways.

matrix:
  include:
  - php: 7.0
    env:
      - MAGE_21=true
      - MAGE_22=true
      - TEST_COVERAGE=false
  - php: 7.1
    env:
      - MAGE_21=false
      - MAGE_22=true
      - TEST_COVERAGE=true

To download the Magento Composer packages you need to be authenticated with repo.magento.com. To do that we make use of the auth.json configuration of Composer. Since we do not want to publically expose our credentials, the auth.json file in the Git repository is encrypted and gets decrypted by Travis CI on each build. This is done in the before_install hook as can be seen below:

before_install:
  - if [[ $encrypted_98dcc32c9b33_key != '' ]]; then 
        openssl aes-256-cbc -K $encrypted_98dcc32c9b33_key
        -iv $encrypted_98dcc32c9b33_iv -in auth.json.enc
        -out auth.json -d ; fi

For improved security Travis CI does not allow to decrypt the content when the build originates from a pull request as potentially someone could try to leak the credentials. Thus the variables are only set for local builds. As a downside this means the tests will only get executed after the merge the master branch which is a bit annoying but better than nothing.

In the script section we check which Magento version should be installed, based on the defined environment variable. Luckly we just have one direct dependency that needs to be installed in the "correct" version to make sure the tests run against the Magento 2.1 or Magento 2.2 codebase:

- if [[ $encrypted_98dcc32c9b33_key != '' && MAGE_21 == true ]]; then 
         composer remove magento/module-customer && 
         composer update magento/module-customer:~100.1 ; fi
- if [[ $encrypted_98dcc32c9b33_key != '' && MAGE_22 == true ]]; then 
     composer remove magento/module-customer && 
     composer update magento/module-customer:~101.0 ; fi

What is currently missing is an integration test to check if the module is actually installable in a real Magento environment. We'll be working on that, for sure.